Search Results for "processing"
Sort By:
Showing 213 open source projects for "processing"
View related business solutions-
Cycloid: Hybrid Cloud DevOps collaboration platformEnable your developers to do their best work and increase time-to-market speed with a leading DevOps and Hybrid Cloud platform.
-
We help you deliver Virtual and Hybrid Events using our Award Winning end-to-end Event Management PlatformEventsAIR have been anticipating and responding to the ever-changing event industry needs for over 30 years, providing innovative solutions that empower event organizers to create successful events around the globe.
-
1
Kapacitor
Open source framework for processing, monitoring, and alerting
Open source framework for processing, monitoring, and alerting on time series data. Kapacitor is a real-time data processing engine for monitoring and alerting, specifically designed to work with time-series data from InfluxDB. -
2
Benthos
Fancy stream processing made operationally mundane
Benthos is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads. It comes with a powerful mapping language, is easy to deploy and monitor, and ready to drop into your pipeline either as a static binary, docker image, or serverless function, making it cloud native as heck. Delivery guarantees can be a dodgy subject. Benthos processes and... -
3
Apache Beam
Unified programming model for Batch and Streaming
Apache Beam is an open source, unified programming model to define both batch and streaming data-parallel processing pipelines, as well as certain language-specific SDKs for constructing pipelines and Runners. These pipelines are executed on one of Beam’s supported distributed processing back-ends, which include Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow. Beam is especially useful for Embarrassingly Parallel data processing tasks, and caters to the different needs and backgrounds of end users, SDK writers and runner writers. -
4
SageMaker Spark Container
Docker image used to run data processing workloads
Apache Spark™ is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. -
Planfix: Manage Projects, Team's Tasks and Business ProcessesPlanfix is like a souped-up business process management system for folks who really know their stuff. It's built to help you dive deeper and gives you more options than your run-of-the-mill project and task management systems. Best part? Even small businesses and non-profits can get in on the action.
-
5
Spring Batch
Spring Batch is a framework for writing batch applications using Java
A lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that will enable extremely high-volume and high performance batch jobs through optimization and partitioning techniques. ... -
6
Apache Spark
A unified analytics engine for large-scale data processing
Apache Spark is a unified engine for large-scale data processing, offering APIs for batch jobs, streaming, machine learning, and graph computation. It builds on resilient distributed datasets (RDDs) and the newer DataFrame/Dataset abstractions to provide fault-tolerant, in-memory computation across clusters. Spark’s execution engine handles scheduling, shuffles, caching, and data locality so users can focus on transformations rather than infrastructure plumbing. -
7
Membrane Core
The core of Membrane Framework, multimedia processing framework
membrane_core is the foundation of the Membrane multimedia framework for Elixir, providing the abstractions and runtime needed to build real-time audio and video pipelines. It models media processing as a graph of lightweight, supervised OTP processes—elements connected by links—so work is isolated, fault-tolerant, and easy to scale or reconfigure at runtime. The core defines a clear lifecycle and callback API for elements, plus concepts like buffers, events, and capabilities/format negotiation to keep components interoperable and type-safe. ... -
8
Apache Flink
Stream processing framework with powerful stream
Apache Flink is a distributed engine for stateful computations over data streams and batches, designed for low-latency processing at scale. Its core runtime executes dataflow graphs with fine-grained backpressure and checkpointing, allowing applications to recover consistently from failures. Flink’s event-time model and watermarks enable accurate out-of-order processing, windowing, and complex time semantics that typical real-time systems struggle with. Developers program against high-level APIs—DataStream and Table/SQL—to express transformations, joins, and stateful patterns, while specialized libraries support CEP, machine learning workflows, and connectors. ... -
9
Broadway
Concurrent and multi-stage data ingestion and data processing
...Developers can enrich pipelines with batching, concurrency control, and metrics reporting, simplifying the management of complex data ingestion and processing systems. Broadway is often used for event processing, stream handling, and background jobs, offering both performance and clarity in Elixir’s functional style. -
Captain Compliance - Data Privacy and Compliance SoftwareCaptain Compliance handles your data privacy requirements so you can be privacy compliant. No more compliance stress, stop stressing over regulatory risks – just privacy protection managed by experts. Our user-friendly platform backed by privacy professionals simplifies the process of navigating regulations, giving your customers transparent choices, and building essential trust for your organization.
-
10
Pachyderm
Data-Centric Pipelines and Data Versioning
...Automatically produces an immutable record for all activities and assets. Pachyderm is used across a variety of industries and use cases. Pachyderm provides a powerful solution to optimize data processing, MLOps, and ML Lifecycles. -
11
Watermill
Building event-driven applications the easy way in Go
Go library for building event-driven applications. Our goal was to create a tool that is easy to understand, even by junior developers. It doesn't matter if you want to do Event-driven architecture, CQRS, Event Sourcing or just stream MySQL Binlog to Kafka. Watermill was designed to process hundreds of thousands of messages per second. Every component is built in a way that allows you to configure it for your needs. You can also implement your own middleware for the router. Watermill is... -
12
Hexagon
Hexagon is a microservices toolkit written in Kotlin
Hexagon is a microservices toolkit written in Kotlin. Its purpose is to ease the building of server applications (Web applications or APIs) that run inside a cloud platform. The Hexagon Toolkit provides several libraries to build server applications. These libraries provide single standalone features and are referred to as "Ports". -
13
NNCF
Neural Network Compression Framework for enhanced OpenVINO
NNCF (Neural Network Compression Framework) is an optimization toolkit for deep learning models, designed to apply quantization, pruning, and other techniques to improve inference efficiency. -
14
Asynq
Simple, reliable, and efficient distributed task queue in Go
Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be scalable yet easy to get started. Client puts tasks on a queue. Server pulls tasks off queues and starts a worker goroutine for each task. Tasks are processed concurrently by multiple workers. Task queues are used as a mechanism to distribute work across multiple machines. -
15
Celery
Distributed task queue (development branch)
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling. Celery has a large and diverse community of users and contributors, you should come join us on IRC or our mailing-list. Celery is Open Source and licensed under the BSD License. A task queue’s input is a unit of work called a task. Dedicated worker processes constantly monitor task queues for new work to perform. ... -
16
gse
Go efficient multilingual NLP and text segmentation
Go efficient multilingual NLP and text segmentation; support English, Chinese, Japanese and others. Gse is implements jieba by golang, and try add NLP support and more feature. Support common, search engine, full mode, precise mode and HMM mode multiple word segmentation modes. Support user and embed dictionary, Part-of-speech/POS tagging, analyze segment info, stop and trim words. Support multilingual: English, Chinese, Japanese and others. Support Traditional Chinese. Support HMM cut text... -
17
Botonic
Build chatbots and conversational experiences using React
Botonic is a full-stack Javascript framework to create chatbots and modern conversational apps that work on multiple platforms, web, mobile and messaging apps (Messenger, Whatsapp, Telegram, etc). Building modern applications on top of messaging apps like Whatsapp or Messenger is much more than creating simple text-based chatbots. Botonic is a full-stack serverless framework that combines the power of React and Tensorflow.js to create amazing experiences at the intersection of text and... -
18
BotSharp
AI Multi-Agent Framework in .NET
Conversation as a platform (CaaP) is the future, so it's perfect that we're already offering the whole toolkits to our .NET developers using the BotSharp AI BOT Platform Builder to build a CaaP. It opens up as much learning power as possible for your own robots and precisely control every step of the AI processing pipeline. BotSharp is an open source machine learning framework for AI Bot platform builder. This project involves natural language understanding, computer vision and audio processing technologies, and aims to promote the development and application of intelligent robot assistants in information systems. Out-of-the-box machine learning algorithms allow ordinary programmers to develop artificial intelligence applications faster and easier. ... -
19
Piscina
A fast, efficient Node.js Worker Thread Pool implementation
Piscina is a fast, efficient Node.js worker thread pool implementation, designed to perform heavy CPU-bound tasks in parallel, thereby improving application performance. -
20
protoactor-go
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
...Grain abstraction, which provides a straightforward approach to building distributed interactive applications, without the need to learn complex programming patterns for handling concurrency, fault tolerance, and resource management. This allows Proto.Actor to leverage in-process performance for realtime stream processing. -
21
Apache Sedona
Cluster computing framework for processing large-scale geospatial data
...According to our benchmark and third-party research papers, Sedona has 50% less peak memory consumption than other Spark-based geospatial data systems for large-scale in-memory query processing. Sedona offers Scala, Java, Spatial SQL, Python, and R APIs and integrates them into underlying system kernels with care. You can simply create spatial analytics and data mining applications and run them in any cloud environments. -
22
GoAWK
A POSIX-compliant AWK interpreter written in Go, with CSV support
GoAWK now uses a bytecode compiler and includes native support for CSV files. AWK is a fascinating text processing language, and The AWK Programming Language is a wonderfully concise book describing it. The A, W, and K in AWK stand for the surnames of the three original creators: Alfred Aho, Peter Weinberger, and Brian Kernighan. Kernighan is also an author of The C Programming Language (“K&R”), and the two books have that same each-page-packs-a-punch feel. -
23
Argo Workflows
Workflow engine for Kubernetes
...Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG). Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes. Run CI/CD pipelines natively on Kubernetes without configuring complex software development products. Argo Workflows is the most popular workflow execution engine for Kubernetes. It can run 1000s of workflows a day, each with 1000s of concurrent tasks. Our users say it is lighter-weight, faster, more powerful, and easier to use. ... -
24
KotlinPoet
A Kotlin API for generating .kt source files
KotlinPoet is a Kotlin and Java API for generating .kt source files. Source file generation can be useful when doing things such as annotation processing or interacting with metadata files (e.g., database schemas, protocol formats). By generating code, you eliminate the need to write boilerplate while also keeping a single source of truth for the metadata. -
25
Colly
Elegant Scraper and Crawler Framework for Golang
Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. Distributed scraping. Caching, automatic encoding of non-unicode responses. Robots.txt support. Google App Engine support.
