Datalytyx launches Gallium (IoT Smart Data Compression Algorithm) as the first of a new era of Snowflake Data Services

Datalytyx has launched Gallium as the first of a new era of Snowflake Data Services.

Gallium is a pure software solution that uses machine learning to intelligently classify large volumes of time series data (aka. IoT data, telemetry data, etc.). Gallium dramatically reduces the amount of data points necessary to create analytic visualisations that are 99% accurate. In most cases only 1% of the source data is required, which significantly increases the performance of database queries used for exploring data within BI solutions.

After developing open source webhooks for Snowflake, Datalytyx can now deliver Gallium as a Snowflake Data Service. Organisations that have Terabytes or even Petabytes of data in Snowflake can get their analytics up to 700 times faster by using Gallium.

Founders of Snowflake, the fastest growing enterprise data warehouse in the cloud, Benoit Dageville and Thierry Cruanes expressed their enthusiasm for this solution.

“Thierry and I have been working closely with the Datalytyx team for the last year. Datalytyx are one of the most experienced teams around the deep insides of Snowflake and we have been enjoying working with them. Datalytyx work closely with Snowflake as part of their data platform, but are also specialists in IoT timeseries data. They have produced Gallium which is a unique time series compression algorithm with limited loss of fidelity. Whether its 5:1 or 500:1 compression, the business value to companies using Snowflake and Gallium for IoT timeseries data compression will be significant.

As part of the process to integrate Gallium directly into Snowflake, Datalytyx have developed an excellent opensource approach to launching external data services, like Gallium, from within Snowflake using only the Snowflake capabilities available today.

This is an example of great solutions coming from the Snowflake ecosystem to help Snowflake users. It’s great to see innovative solutions from leading partners like Datalytyx. This Gallium integration is the first of a new age of data services called natively from within Snowflake, and with the generous opensourcing of this code, we look forward to seeing many more.”

Using webhooks and Table Stream, we are able to schedule a task within Snowflake to drop the recently added data into files on a stage and then send references to these files to Gallium for classification. Once the results are completed, they are put back into a stage and loaded using SnowPipe.

Gallium on Snowflake gives Snowflake another advantage when compared to traditional time series databases which tend to be less flexible and scalable than Snowflake’s cloud based SQL data warehouse. With the addition of Gallium as a Snowflake Data Service, organisations can now take advantage of Snowflake’s ability to store and process time series data while also dramatically increasing the performance of their analytics.