Search Results for "python libraries"

Deequ is a library built atop Apache Spark that enables defining “unit tests for data” — that is, formal constraints or checks on datasets to ensure data quality along dimensions such as completeness, uniqueness, value ranges, correlations, etc. It can scale to large datasets (billions of rows) by translating those data checks into Spark jobs. Deequ supports advanced features like a metrics repository for storing computed statistics over time, anomaly detection of data quality metrics, and...

SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. SynapseML builds on Apache Spark and SparkML to enable new kinds of machine learning, analytics, and model deployment workflows. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with the Open Neural Network Exchange (ONNX), LightGBM, The Cognitive Services, Vowpal Wabbit,...

Scala and polyglot Spark library (Scala, PySpark, SparkSQL, ... ) focused on reading OpenStreetMap Pbf files.

Assorted projects. General-purpose libraries for Python, C++, Scala, bash, and others. Meta-programming tools. System utilities. UI components. Web APIs. Configuration files. Benchmarks. Programming competition entries. And much more.

The node2vec project provides an implementation of the node2vec algorithm, a scalable feature learning method for networks. The algorithm is designed to learn continuous vector representations of nodes in a graph by simulating biased random walks and applying skip-gram models from natural language processing. These embeddings capture community structure as well as structural equivalence, enabling machine learning on graphs for tasks such as classification, clustering, and link prediction....

Java-based scientific graphics with support for Java, Groovy, MATLAB, Python, the R statistical environment, Scala and SciLab.