StreamSets Deepens Snowflake Partnership with New Data Pipeline Engine for Snowpark
SAN FRANCISCO – (COMMERCIAL THREAD) – StreamSets, the industry’s leading DataOps platform provider, today announced the preview of its new data pipeline runtime engine using Snowflake’s new development experience, Snowpark, the Data Cloud company. The StreamSets engine for Snowpark combines the powerful DataOps capabilities of StreamSets with the flexibility of Snowpark to use languages ââsuch as Java and Scala. Snowpark and the StreamSets Engine for Snowpark provide a unique combination of power and ease of use for data engineers to create, operate and monitor intelligent data pipelines for Snowflake Data Cloud.
Many Snowflake customers are already using the StreamSets DataOps platform to build and operate intelligent data pipelines that ingest streaming, batch, or data capture modification (CDC) data into Snowflake Data Cloud. Now, with Snowpark and the StreamSets Engine for Snowpark, data engineers can go beyond SQL to express powerful data pipeline logic with the StreamSets DataOps platform using their preferred language, such as Scala and Java, via an intuitive graphical interface. These pipelines can then run directly to Snowflake’s data cloud, giving even more users the ability to gain powerful insight into the data without having to manage additional processing systems.
Specifically, Snowflake customers can use the StreamSets DataOps platform and the StreamSets engine for Snowpark to get the most out of their Data Cloud investments with the following key benefits:
-
A high-performance data engineering platform leveraging the power and flexibility of Scala while running entirely in Snowflake’s data cloud, eliminating the need for additional Spark-based systems that add cost, complexity and latency
-
A powerful design environment that delivers 10x the productivity of data engineers who create and orchestrate enterprise-grade data pipelines, combining the ease of drag-and-drop design with custom code extensibility and a comprehensive SDK
-
A single experience to build, operate, and monitor all pipelines for the Snowflake Data Cloud, including streaming, CDC, and batch data pipelines ingesting data into the Data Cloud, as well as ELT pipelines running workloads of work directly in the data cloud via Snowpark
-
Intelligent data pipelines designed to continuously deliver data to support a DataOps practice, with built-in resiliency to changes in data structures, schema, and semantics
-
Data engineering delivered as a cloud service with no infrastructure to build or manage
âSnowpark is transforming data engineers and developers who prefer languages ââother than SQL for development,â said Isaac Kunen, senior product manager, Snowflake. âWith the new StreamSets engine for Snowpark, data engineers will be able to create and operate Scala-based data pipelines designed for DataOps, bringing the power and performance of Snowflake’s data cloud to data engineers. most knowledgeable companies. ”
âOur customers have made incredible progress using DataOps practices and democratizing their data with Snowflake,â said Judy Ko, Chief Product Officer, StreamSets. âWe see Snowpark as the next big step, making sophisticated data engineering accessible to everyone, so that all types of data, from all sources, can be delivered continuously through the Data Cloud. ”
The StreamSets Snowpark Engine is available in preview today. You can register by contacting [email protected].
About flow sets
At StreamSets, our mission is to make data engineering teams successful. Only StreamSets offers a platform dedicated to creating the intelligent data pipelines needed to power DataOps on hybrid and multi-cloud architectures. That’s why the world’s largest companies trust StreamSets to power millions of data pipelines for modern business intelligence, AI / ML, and smart applications. With StreamSets, data engineers spend less time fixing and more time doing. To learn more, visit www.streamsets.com and follow us on LinkedIn.