Download
SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. SynapseML builds on Apache Spark and SparkML to enable new kinds of machine learning, analytics, and model deployment workflows. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with the Open Neural Network Exchange (ONNX), LightGBM, The Cognitive Services, Vowpal Wabbit, and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of data sources. SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. For production-grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.
Features
- SynapseML requires Scala 2.12, Spark 3.2+
- Requires Python 3.6+
- See the API documentation for Scala and for PySpark
- Leverage the Microsoft Cognitive Services at Unprecedented Scales in your existing SparkML pipelines
- Train Gradient Boosted Machines with LightGBM
- Serve any Spark Computation as a Web Service with Sub-Millisecond Latency
Project Samples
