Data streaming platform adds support for ML models written in popular languages
by Max Smolaks 10 March 2020
American software developer Hazelcast has updated its open source stream processing engine to natively support Python- and Java-based machine learning models.
The 4.0 release of Hazelcast Jet adds new “inference runners” for the two of the most popular languages used in ML programming, simplifying deployment of real-time machine learning inference.
“This approach completely eliminates the impedance mismatch between the data scientist and data engineer since Hazelcast Jet can handle the data ingestion, transformation, scoring and post-processing,” said Greg Luck, CTO of Hazelcast.
Hazelcast’s flagship product is an eponymous open source platform based on In-Memory Data Grid (IMDG) technology: it involves a set of networked computers pooling together their RAM to share data amongst applications running in the cluster. With this approach, data is processed in a highly parallel fashion, which boosts performance; using memory instead of storage improves performance even further.
Jet is a stream processing engine based on the IMDG core, designed for fast handling of big data sets, including those used in low latency machine learning.
The latest version enables users to run ML inference locally to the processing job, without having to convert models into other languages, and feed real-time streaming data directly into the model.
“Last year we simplified streaming by delivering the industry’s only all-in-one processing system, eliminating the need for complex IT designs built from many independent components. Now we’re moving the industry forward again by simplifying how enterprises can deploy ultra-low latency machine learning models within that efficient system,” said Kelly Herrell, CEO of Hazelcast.
“There are millions of dollars to be won when microseconds count and Hazelcast Jet makes that a reality faster than any alternative, especially for applications leveraging artificial intelligence and machine learning.”
The new Jet release also includes expanded database support and other updates focused on data integrity.
You can download Hazelcast Jet 4.0 here.