Search Results for "big data"
Sort By:
Showing 37 open source projects for "big data"
View related business solutions-
Component Content Management System for Software DocumentationPaligo is an end-to-end Component Content Management System (CCMS) solution for technical documentation, policies and procedures, knowledge management, and more.
-
Easy management of simple and complex projectsPlan more projects with Worksection. Use Gantt chart and Kanban boards to organize your projects, get your team onboard and assign tasks and due dates.
-
1
Big-AGI
AI suite powered by state-of-the-art models and providing advanced AI
...The workspace includes advanced features like Beam, which enables multi-model consensus and comparative responses to improve reliability and reduce hallucination, and robust persona management to tailor responses to specific roles or workflows. Big-AGI can be self-hosted or deployed in cloud environments, giving users full control over data and model access limits and avoiding vendor lock-in. -
2
DATA SCIENCE ROADMAP
Data Science Roadmap from A to Z
DATA SCIENCE ROADMAP is an educational repository designed to guide learners through the process of becoming proficient in data science and machine learning. The project presents a structured roadmap that outlines the knowledge and skills required for different stages of a data science career. Topics typically include programming with Python, statistics, mathematics, machine learning algorithms, data visualization, and big data technologies. ... -
3
FinMind
Open Data, more than 50 financial data
In the era of big data, data is the foundation of everything. We collect more than 50 kinds of Taiwan stock related information and provide download, online analysis, and backtesting. Regardless of the program, you can download data through the api provided by FinMind, or you can download data directly from the website. After data is available, statistical analysis, regression analysis, time series analysis, machine learning, and deep learning can be performed. ... -
4
Vespa
The open big data serving engine
Make AI-driven decisions using your data, in real-time. At any scale, with unbeatable performance. Vespa is a full-featured text search engine and supports both regular text search and fast approximate vector search (ANN). This makes it easy to create high-performing search applications at any scale, whether you want to use traditional techniques or a modern vector-based approach. You can even combine both approaches efficiently in the same query, something no other engine can do.... -
The leading LMS solution for mission critical learning needsStreamline and integrate your complex learning, compliance, content monetization, and external training capabilities while keeping your people safe and delivering profits with Seertech’s LMS solution.
-
5
marimo
A reactive notebook for Python
marimo is an open-source reactive notebook for Python, reproducible, git-friendly, executable as a script, and shareable as an app. marimo notebooks are reproducible, extremely interactive, designed for collaboration (git-friendly!), deployable as scripts or apps, and fit for modern Pythonista. Run one cell and marimo reacts by automatically running affected cells, eliminating the error-prone chore of managing the notebook state. marimo's reactive UI elements, like data frame GUIs and plots,... -
6
.NET for Apache Spark
A free, open-source, and cross-platform big data analytics framework
.NET for Apache Spark provides high-performance APIs for using Apache Spark from C# and F#. With these .NET APIs, you can access the most popular Dataframe and SparkSQL aspects of Apache Spark, for working with structured data, and Spark Structured Streaming, for working with streaming data. .NET for Apache Spark is compliant with .NET Standard - a formal specification of .NET APIs that are common across .NET implementations. This means you can use .NET for Apache Spark anywhere you write... -
7
LlamaIndex
Central interface to connect your LLM's with external data
LlamaIndex (GPT Index) is a project that provides a central interface to connect your LLM's with external data. LlamaIndex is a simple, flexible interface between your external data and LLMs. It provides the following tools in an easy-to-use fashion. Provides indices over your unstructured and structured data for use with LLM's. These indices help to abstract away common boilerplate and pain points for in-context learning. Dealing with prompt limitations (e.g. 4096 tokens for Davinci) when the context is too big. ... -
8
ROOT
Analyzing, storing and visualizing big data, scientifically
ROOT is a unified software package for the storage, processing, and analysis of scientific data: from its acquisition to the final visualization in the form of highly customizable, publication-ready plots. It is reliable, performant and well supported, easy to use and obtain, and strives to maximize the quantity and impact of scientific results obtained per unit cost, both of human effort and computing resources. ROOT provides a very efficient storage system for data models, that... -
9
airda
airda(Air Data Agent
airda(Air Data Agent) is a multi-smart body for data analysis, capable of understanding data development and data analysis needs, understanding data, generating data-oriented queries, data visualization, machine learning and other tasks of SQL and Python codes. -
Unimus makes Network Automation and Configuration Management easy.We aim to make automation, disaster recovery, change management and configuration auditing painless and affordable for a network of any size.
-
10
ChatGLM2-6B
ChatGLM2-6B: An Open Bilingual Chat LLM
ChatGLM2-6B is the second-gen Chinese-English conversational LLM from ZhipuAI/Tsinghua. It upgrades the base model with GLM’s hybrid pretraining objective, 1.4 TB bilingual data, and preference alignment—delivering big gains on MMLU, CEval, GSM8K, and BBH. The context window extends up to 32K (FlashAttention), and Multi-Query Attention improves speed and memory use. The repo includes Python APIs, CLI & web demos, OpenAI-style/FASTAPI servers, and quantized checkpoints for lightweight local deployment on GPUs or CPU/MPS. -
11
Umbrel
A beautiful personal server OS for Raspberry Pi or any Linux distro
Run your personal server with a Bitcoin and Lightning node in your home, self-host open source apps like Nextcloud and Matrix to break away from big tech, and take full control of your data. For free. All our interactions on the internet today are mediated by a few companies who offer “free” services in exchange for storing our data on their servers to spy on us. Running a personal server fundamentally changes that. You and your family’s photos, videos, files, notes, passwords, everything, have nothing to do with someone else’s computer. ... -
12
HASH
The best way to use and work with blocks
...You can read more about our big-picture vision at hash.dev -
13
All-in-RAG
Big Model Application Development Practice 1
All-in-RAG is an open-source educational project designed to teach developers how to build applications using retrieval-augmented generation techniques. The repository provides a structured learning path that covers both theoretical foundations and practical implementation steps for RAG systems. It explains the full development pipeline required to create knowledge-aware AI assistants, including data preparation, document indexing, vector embedding generation, and retrieval strategies. The... -
14
NeuroMatch Academy (NMA)
NMA Computational Neuroscience course
NMA Computational Neuroscience course. We have curated a curriculum that spans most areas of computational neuroscience (a hard task in an increasingly big field!). We will expose you to both theoretical modeling and more data-driven analyses. The Neuro Video Series is a series of 12 videos that covers basic neuroscience concepts and neuroscience methods. These videos are completely optional and do not need to be watched in a fixed order so you can pick and choose which videos will help you brush up on your knowledge. ... -
15
Sail
A drop-in Apache Spark replacement written in Rust
Sail is an open-source distributed computation framework designed to unify batch processing, stream processing, and AI workloads into a single, high-performance engine. It is built entirely in Rust, eliminating JVM overhead and enabling predictable performance, fast startup times, and improved memory safety compared to traditional big data frameworks. Sail is compatible with the Spark Connect protocol, which means existing Spark SQL and DataFrame workloads can run without code changes, making adoption seamless for teams already using Spark-based pipelines. The framework is designed to operate across a variety of environments, including local machines, Kubernetes clusters, and cloud deployments, allowing flexible scaling based on workload requirements. ... -
16
AlphaTree
DNN && GAN && NLP && BIG DATA
AlphaTree is an educational repository that provides a visual roadmap of deep learning models and related artificial intelligence technologies. The project focuses on explaining the historical development and relationships between major neural network architectures used in modern machine learning. It presents diagrams and documentation describing the evolution of models such as LeNet, AlexNet, VGG, ResNet, DenseNet, and Inception networks. The repository organizes these architectures into a... -
17
Angel
A Flexible and Powerful Parameter Server for large-scale ML
Angel is a high-performance distributed machine learning and graph computing platform based on the philosophy of Parameter Server. It is tuned for performance with big data from Tencent and has a wide range of applicability and stability, demonstrating an increasing advantage in handling higher-dimension models. Angel is jointly developed by Tencent and Peking University, taking account of both high availability in industry and innovation in academia. With a model-centered core design concept, Angel partitions the parameters of complex models into multiple parameter-server nodes and implements a variety of machine learning algorithms and graph algorithms using efficient model-updating interfaces and functions, as well as a flexible consistency model for synchronization. ... -
18
HPCC Systems
End-to-end big data in a massively scalable supercomputing platform.
Important: As of April 20, 2026, this project can now be found at https://github.com/hpcc-systems/HPCC-Platform/releases. HPCC Systems® (www.hpccsystems.com) from LexisNexis® Risk Solutions is a proven, open source solution for Big Data insights that can be implemented by businesses of all sizes. With HPCC Systems, developers can design applications with Big Data at their core, enabling businesses to better analyze and understand data at scale, improving business time to results and decisions. HPCC Systems offers a consistent data-centric programming language, two processing platforms and a single, complete end-to-end architecture for efficient processing. -
19
MOA - Massive Online Analysis
Big Data Stream Analytics Framework.
A framework for learning from a continuous supply of examples, a data stream. Includes classification, regression, clustering, outlier detection and recommender systems. Related to the WEKA project, also written in Java, while scaling to adaptive large scale machine learning. -
20
Parkiet
Parquet format file GUI editor
Parquet file viewer and editor written in Java and SWT. It uses Apache Avro library for reading and writing edited parquet files. Only Parquet files with simple data type columns are supported. -
21
Faum
Fast Autonomous Unsupervised Multidimiensional Classification
This is the proof-of-concept implementation of the FAUM Clustering method. This implementation was used to perform the published results and is now released in the hope that it will be useful. -
22
Alink
Alink is the Machine Learning algorithm platform based on Flink
Alink is Alibaba’s scalable machine learning algorithm platform built on Apache Flink, designed for batch and stream data processing. It provides a wide variety of ready-to-use ML algorithms for tasks like classification, regression, clustering, recommendation, and more. Written in Java and Scala, Alink is suitable for enterprise-grade big data applications where performance and scalability are crucial. It supports model training, evaluation, and deployment in real-time environments and integrates seamlessly into Alibaba’s cloud ecosystem. -
23
SentimentAnalysis-Rick&Morty
Rick & Morty Sentiment Analysis - End-of-Degree Project - UNIR
The remarkable progress in the field of Big Data has driven the development of new technologies in natural language processing and data analysis. Text mining is a fascinating application of data analysis that extracts relevant information from related writings in different linguistic contexts. And therefore, in natural language processing, sentiment analysis and classification stands out as a key application supported by text mining. -
24
AppShark
Static taint analysis platform to scan vulnerabilities
AppShark is an open-source static taint-analysis platform developed by ByteDance designed to scan Android application packages (APKs) for potential security or privacy vulnerabilities. It performs code analysis without executing the app — inspecting APK contents, tracking data flows (taints), and detecting risky patterns such as insecure file access, unsafe API usage, resource-leak possibilities, or misconfigurations. Users can customize scanning via rule sets (written in JSON), defining... -
25
SparrowRecSys
A Deep Learning Recommender System
SparrowRecSys is an open-source deep learning recommendation system framework designed to demonstrate the architecture and implementation of modern industrial-scale recommender systems. The project integrates multiple machine learning models and data processing pipelines to simulate how real-world recommendation platforms operate. It includes components for offline data processing, feature engineering, model training, real-time data updates, and online recommendation services. SparrowRecSys...
