Climate change is recognized globally as a threat to humanity. Almost all industries are working hard to innovate us out of the heat trap. Engineering changes are reducing our carbon footprint from driving vehicles to pouring concrete. There is one area, however, where innovation is leading to a steep increase in carbon emissions: data.
Image and Data courtesy of McKinsey Company
According to McKinsey US data center power consumption increased from six gigawatts in 2014 to 17 gigawatts in 2022. McKinsey projects that it will double again by 2030. Whereas data centers used 3% of power in the US in 2022 this is projected to grow to 8% by 2030. Consumers and businesses alike have an insatiable demand for digital services and there is very little we can do to reduce the rapid growth of demand. There are, however, ways to reduce the power consumption needed to produce many of these digital services.
When confronted with growing demand for computing power and data storage organizations have historically had a very simple solution: add more servers. As a small startup Moonshadow did not have the resources to endlessly add servers to keep up with our clients’ data growth so we were forced to look at a different strategy: do more with less. Instead of adding hardware we focused on making software more efficient. Moonshadow has developed MMZIP to reduce the footprint of tabular data by over 90% without losing any information. This means we store ten times as much data on the same hard disks and read times are 10 times faster. Moving data between servers is ten times faster and uses 90% less power because the data is so much smaller. Data storage and transmission costs are reduced to 10% of the original costs. Our technology is one of those rare instances were doing good for the planet saves money immediately while speeding up services at the same time. Our software runs on standard Linux servers and it does not require any specialized hardware. Moonshadow recently received its 20th patent on its technology.
Moonshadow’s software is optimized for data that can be stored in a database with rows and columns. This includes data that can be stored in SQL databases, spreadsheets or CSV’s. The compression ratios we achieve differ depending on the content of the data. Our compression ratios are especially good for data generated continuously by IoT devices, mobile apps or vehicles where data volumes quickly explode as the number of devices, the number of measurements per device and the reporting frequencies continue to increase rapidly. Interestingly, the higher the reporting frequency of a device, the more data is generated and the higher our compression ratios. We quantified the savings for connected vehicle data in this blog posting. Moonshadow offers several software products that reduce the carbon footprint of the data while speeding up data access. MMZIP is a stand-alone compression tool that is ideal for compressing, transporting, merging and selecting data. DB4IoT is developed for visualizing time-series data over maps. Moonshadow also offers engineering services to analyze existing data processing pipelines and develop custom solutions to reduce the data footprint while speeding up data access.
Please contact Eimar at moonshadow dot com to start reducing your data carbon footprint.
