Open Source Linux Big Data Tools - Page 3
Sort By:
Big Data Tools for Linux
View 45 business solutions-
DataHub is the leading open-source data catalog helping teams discover, understand, and govern their data assets.Built on an open source foundation with a thriving community of 13,000+ members, DataHub gives you unmatched flexibility to customize and extend without vendor lock-in. DataHub Cloud is a modern metadata platform with REST and GraphQL APIs that optimize performance for complex queries, essential for AI-ready data management and ML lifecycle support.
-
Empowering Companies To Excel In Safety Data Sheet ComplianceEffortless setup and maintenance: Simplified management and seamless online access to safety data sheets for your team
-
1
Neuro
The Neuro crypto currency
The Neuro NRO cryptocurrency is designed to support solutions of machine learning tasks, big data and neural networks. Neuro is a scientific-technical project uniting scientists, engineers and programmers inspired by the idea to build something big, kind and bright. From the first stages of work, we will be engaged in the development of new architectures and algorithms of neural networks. Someday we will undoubtedly enter the annual ImageNet Challenge contest to compete with such giants as GoogLeNet Inception and Microsoft ResNet. At further stages of the work, we adapt the neural networks to calculate molecular interactions in protein environments. Our system will help to look for new types of drugs for cancer, Alzheimer's and other serious problems of modern medicine. We plan to make a serious contribution to the increase of human life expectancy. -
2
OCW Test - Out of Commerce Works
Program for out of commerce works detection
The OCW Test program has been designed to provide assistance in the detection of works outside trade, taking as reference a list of works from a specific bibliographic catalog. In this first version, the program operates on the identifiers of the books of the library of the Complutense University of Madrid. However, the program can be reedited, to work on any bibliographic catalog. -
3
ODD Platform
First open-source data discovery and observability platform
Unlock the power of big data with OpenDataDiscovery Platform. Experience seamless end-to-end insights, powered by unprecedented observability and trust - from ingestion to production - while building your ideal tech stack! Democratize data and accelerate insights. Find data that fits your use case and discover hints left by your peers to leverage existing knowledge. Explore tags, ownership details, links to other sources and other information to shorten and simplify data discovery phase. Forget unnerved stakeholders and wasting too much time on digging the root cause of data issues when it fails. With ODD’s automatic company-wide ingestion-to-product lineage you’ll have answers in just seconds and stakeholders won’t need to wait. Sleep well, knowing all your data is in check. Forget manual testing, days of debugging, and weeks of worrying. Know the impact of each code change with automatic testing. Enjoy lineage and alerts powered with data quality information. -
4
Oasis Development Tool
OASIS Development Tool
The OASIS Development Tool is an innovative IDE for Code Generation-, Code Debugging- and Visual Coding- using the OASIS Programming Language. The OASIS Programming Language is a 4GL Concurrency- and Database Language running round a distributed OASIS Runtime Machine Environment (RME) as interpreted OASIS Scripts sequenced into OASIS Polyglot Runtime Components (PRC) with just in time patterns. The IDE is designed specifically for the OASIS Programming Language. The IDE is focused around the concept of Visual-, Online-, Data-Centric-, Concurrent-, and Runtime- Code, whilst remaining an IDE to handle OASIS Programming. The IDE has a number of visual code drag and drop features. The Tool is by no means a representative of the Cyclical UML Model- and Code concept, but rather a replacement. The IDE Tool is focused around (Team Based) System Engineering, Meta Programming, Visual Coding, Concurrent Processing and, Databases and Big Data. -
PageDNA: Web-to-Print eCommerce SoftwarePageDNA enables successful eCommerce strategies for commercial print sales organizations, internal print shops, and brand owners. PageDNA’s online ordering platform increases print volume while decreasing touch costs for all stakeholders: clientele, print operations, and the organizations they support.
-
5
Oblivious Bloom Intersection
Oblivious Bloom Intersection
This page is about the PSI implementation described in the paper: When Private Set Intersection Meets Big Data: An Efficient and Scalable Protocol (CCS 2013). -
6
Occursions
Fast customizable time series web database for big data like log files
Our goal is to create the world's fastest extendable, non-transactional time series database for big data (you know, for kids)! Log file indexing is our initial focus. For example append only ASCII files produced by libraries like Log4J, or containing FIX messages or JSON objects. Occursions was built by a small team sick of creating hacks to remotely copy and/or grep through tons of large log files. We use it to index around a terabyte of new log data per day. You can use it too. Who doesn't have `just too many' log files? Occursions asynchronously tails log files and indexes the individual lines in each log file as each line is written to disk so you don't even have to wait for a second after an event happens to search for it. Occursions uses custom disk backed data structures to create and search its indexes so it is very efficient at using CPU, memory and disk. You can extend Occursions with shared libraries to support your own file formats, even binary file formats! -
7
Open Source Data Quality and Profiling
World's first open source data quality & data preparation project
This project is dedicated to open source data quality and data preparation solutions. Data Quality includes profiling, filtering, governance, similarity check, data enrichment alteration, real time alerting, basket analysis, bubble chart Warehouse validation, single customer view etc. defined by Strategy. This tool is developing high performance integrated data management platform which will seamlessly do Data Integration, Data Profiling, Data Quality, Data Preparation, Dummy Data Creation, Meta Data Discovery, Anomaly Discovery, Data Cleansing, Reporting and Analytic. It also had Hadoop ( Big data ) support to move files to/from Hadoop Grid, Create, Load and Profile Hive Tables. This project is also known as "Aggregate Profiler" Resful API for this project is getting built as (Beta Version) https://sourceforge.net/projects/restful-api-for-osdq/ apache spark based data quality is getting built at https://sourceforge.net/projects/apache-spark-osdq/ -
8
PROPER is a package for visual evaluation of ranking classifiers for biological big data mining studies in the mathematical language MATLAB. It is an efficient tool for optimization and comparison of the state-of-the-art ranking classifiers by generating over 20 different high quality two- and three-dimensional performance curves.
-
9
PROPER is a package for visual evaluation of ranking classifiers for biological big data mining studies in the mathematical language MATLAB. It is an efficient tool for optimization and comparison of the state-of-the-art ranking classifiers by generating over 20 different high quality two- and three-dimensional performance curves.
-
Feroot AI automates website security with 24/7 monitoringFeroot unifies JavaScript behavior analysis, web compliance scanning, third-party script monitoring, consent enforcement, and data privacy posture management to stop Magecart, formjacking, and unauthorized tracking.
-
10
PanoramaServer
Open Source Panorama Server for free virtual tour of 360 degrees views
Ideal for creating virtual tours of panoramic views for all sorts including property exhibition for brokers at real estate agencies/property agents, tour guide for indoor/outdoor venues, information to public/private facilities for curators, travel journal for tourist as log book, backdrop setting for storytelling, treasure hunt like games, big data mining for pattern through computer vision in artificial intelligence, etc. It is like creating your own Google Map Street View. All is required by the user is to have photos of equirectangular format (panorama) taken from 3D cameras common for on-site premises. These images can be referenced by the PanoramaServer to create virtual travels with 360 degrees view where viewers can navigate to different locations, view information, etc. If made available online to general public over the internet, can even share the link of your virtual trips. PanoramaServer is free as it is open source licensed. -
11
Querybook
Big Data Querying UI, combining collocated table metadata
Querybook is Pinterest’s open-source big data IDE via a notebook interface. Querybook’s core focus is to make composing queries, creating analyses, and collaborating with others as simple as possible. Organize rich text, queries, and charts into a notebook to easily document your analyses. Work collaboratively with others in a DataDoc and get real-time updates. The Query Editor is aware of your tables and their columns, as such it provides autocompletion, syntax highlighting, and the ability to hover or click on a table to view its information. No need to leave Querybook to create charts to quickly visualize your results. With a familiar interface easily create line, bar, stacked area, pie, horizontal bar, donut, scatter, and table charts. Add them then to your DataDoc to complete your data narrative. -
12
R Hadoop for Big Data
Download Free Associated R open source script files for big data analy
Download Free Associated R open source script files for big data analysis with Hadoop and R These are R script source file from Ram Venkat from a past Meetup we did at http://www.meetup.com/R-Matlab-Users/events/85160532/ Also, there is a long video and Powerpoint presentation slide PDF with R files at: http://quantlabs.net/blog/2012/11/how-to-use-hadoop-and-r-for-big-data-parallel-processing-free-download-pdf/ Download source files from http://quantlabs.net/blog/2012/11/download-free-associated-r-open-source-script-files-for-big-data-analysis-with-hadoop-and-r-rstats-hadoop/ -
13
Random Bits Forest
RBF: a Strong Classifier/Regressor for Big Data
We present a classification and regression algorithm called Random Bits Forest (RBF). RBF integrates neural network (for depth), boosting (for wideness) and random forest (for accuracy). It first generates and selects ~10,000 small three-layer threshold random neural networks as basis by gradient boosting scheme. These binary basis are then feed into a modified random forest algorithm to obtain predictions. In conclusion, RBF is a novel framework that performs strongly especially on data with large size. -
14
Random Bits Regression
Random Bits Regression is a strong general predictor.
We proposed an accurate, robust and fast general predictor (RBR) for regression and classification in big data era. The application of this method is very broad, from science to industry, finance and health. The accuracy and robustness improvement of our method over existing method could bring huge benefits in some critical applications. For example, natural disaster prediction, stock price prediction, personal/population disease prediction. The fast-speed nature of our method not only allows big data analysis but also enables real-time recognition and predictions. The RBR framework also hints the mechanism of brain function and leads to a "wide learning" hypothesis. We believe that this method will make a great impact and enable many downstream applications. -
15
Redis Desktop Manager
:wrench: Cross-platform GUI management tool for Redis
Redis Desktop Manager is a fast, open source Redis database management application based on Qt 5. It's available for Windows, Linux and MacOS and offers an easy-to-use GUI to access your Redis DB. With Redis Desktop Manager you can perform some basic operations such as view keys as a tree, CRUD keys and execute commands via shell. It also supports SSL/TLS encryption, SSH tunnels and cloud Redis instances, such as: Amazon ElastiCache, Microsoft Azure Redis Cache and Redis Labs. -
16
Relation Tags
Source code for be able to use Relation Tags.
Source code for be able to use Relation Tags. It is part of project VocabularyMem but can be used separately. Relation Tags are tags which can be relationed together . For example tag "Paris" and tag "France" can be relationed with a relation "is part of". This code is created from 0 and is able to define which type of relation we use, using most elemental mathematic properties. It is strongly recommended to read "Relation Tags guide for programmers". Inside source zip, also contains dialogs for set properties of this extended tags. All this dialogs files finish either with "...dlg.cpp" or ",,,dlg.h". Please read "readme" file. It is recommended to use a binary matrix class like BinMatrix in order to have enough speed for calculations of implicit relations in a system of bogus tags with big data. Need to be compiled with C++11 and Qt libraries -
17
SZT-bigdata
SZT‑bigdata is an open source project
SZT‑bigdata is an open-source project analyzing real Shenzhen metro (subway) card usage data using big‑data frameworks like Spark, Hadoop, Hive, Kafka, Flink, ClickHouse, HBase, and Elasticsearch. Aimed at exploring transit passenger flow patterns and system optimization using a variety of Scala-based technologies. -
18
Sample Level Musical Timeline
Sample Level Modulation of Musical Timeline
Sample Level Modulation of Musical Timeline Mingfeng Zhang Dept. of Electrical and Computer Engineering, University of Rochester In this toolbox we provide signal processing tools to allocate music events (samples of musical notes) to specified time locations with sample level accuracy. In this implementation, we use computational tools to add in micro-timing variations in J.S. Bach four-part chorales as a "visualizer" for big data. By extracting data patterns from multiple time scales, we implement a tool that musicians can perform the big data at different resolutions. This toolbox will need the following supporting toolboxes: MIDI TOOLBOX https://www.jyu.fi/hum/laitokset/musiikki/en/research/coe/materials/miditoolbox MIR TOOLBOX https://www.jyu.fi/hum/laitokset/musiikki/en/research/coe/materials/mirtoolbox Please add the path in MATLAB for these two toolbox. Please also read the project document file (readme.doc/pdf) for more details -
19
SentimentAnalysis-Rick&Morty
Rick & Morty Sentiment Analysis - End-of-Degree Project - UNIR
The remarkable progress in the field of Big Data has driven the development of new technologies in natural language processing and data analysis. Text mining is a fascinating application of data analysis that extracts relevant information from related writings in different linguistic contexts. And therefore, in natural language processing, sentiment analysis and classification stands out as a key application supported by text mining. Through the extraction of information from textual data, it becomes possible to identify and comprehend the sentiments and emotions conveyed. In this end-of-degree work, we analyze and classify the dialogue of characters in an English-language television series as "Rick and Morty" using Python. The objective is to identify and categorize the feelings and emotions expressed in the text, comparing the human perception of the characters' personalities with the results obtained using natural language processing techniques. -
20
TensorBase
TensorBase is a new big data warehousing with modern efforts
TensorBase hopes the open source not become a copy game. TensorBase has a clear-cut opposition to fork communities, repeat wheels, or hack traffic for so-called reputations (like Github stars). After thoughts, we decided to temporarily leave the general data warehousing field. For people who want to learn how a database system can be built up, or how to apply modern Rust to the high-performance field, or embed a lightweight data analysis system into your own big one. You can still try, ask or contribute to TensorBase. The committers are still around the community. We will help you in all kinds of interesting things pursued in the project by us and maybe you. We still maintain the project to look forward to meeting more database geniuses in this world, although no new feature will be added in the near future. -
21
Universal Java Matrix Package
sparse and dense matrix, linear algebra, visualization, big data
The Universal Java Matrix Package (UJMP) is an open source Java library which provides sparse and dense matrix classes, as well as a large number of calculations for linear algebra such as matrix multiplication or matrix inverse. Operations such as mean, correlation, standard deviation, replacement of missing values or the calculation of mutual information are supported, too. The Universal Java Matrix Package provides various visualization methods, import and export filters for a large number of file formats, and even the possibility to link to JDBC databases. Multi-dimensional matrices as well as generic matrices with a specified object type are supported and very large matrices can be handled even when they do not fit into memory. -
22
Vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python
Data science solutions, insights, dashboards, machine learning, deployment. We start at 100GB. Vaex is a high-performance Python library for lazy Out-of-Core data frames (similar to Pandas), to visualize and explore big tabular datasets. It calculates statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid for more than a billion (10^9) samples/rows per second. Visualization is done using histograms, density plots and 3d volume rendering, allowing interactive exploration of big data. Vaex uses memory mapping, zero memory copy policy and lazy computations for best performance (no memory wasted). Cut development cut development time by 80%. Your prototype is your solution. Create automatic pipelines for any model. -
23
ankus
Data Mining and Machine Learning Algorithms based on MapReduce
[The feature of ankus] * ankus is a 'web-based big data mining project and tool'. - MapReduce-based data mining/machine learning algorithms library - Hadoop-based distributed bigdata system - offering a web-based GUI for easy use [The ankus project & License] * The ankus project consists of three as an open source. * ankus has Dual licensed under the community and commercial licenses. * community license is following GPLv3 - Some algorithms in Core Project do not under the OSS License [Demonstration Site] http://www.openankus.org:18080 [Official website & E-mail] www.openankus.org ankus@openankus.org [ankus video list] http://bit.ly/ankus_video [community] http://www.facebook.com/groups/openankus (Korean Groups) http://www.facebook.com/openankus (English Groups) http://bit.ly/ankus_forum (Google groups user forum) -
24
apache spark data pipeline osDQ
osDQ dedicated to create apache spark based data pipeline using JSON
This is an offshoot project of open source data quality (osDQ) project https://sourceforge.net/projects/dataquality/ This sub project will create apache spark based data pipeline where JSON based metadata (file) will be used to run data processing , data pipeline , data quality and data preparation and data modeling features for big data. This uses java API of apache spark. It can run in local mode also. Get json example at https://github.com/arrahtech/osdq-spark How to run Unzip the zip file Windows : java -cp .\lib\*;osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c .\example\samplerun.json Mac UNIX java -cp ./lib/*:./osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c ./example/samplerun.json For those on windows, you need to have hadoop distribtion unzipped on local drive and HADOOP_HOME set. Also copy winutils.exe from here into HADOOP_HOME\bin -
25
deshang
Software to support deshang research
Deshang research project mainly focus on collecting students' behaviors and using big data technologies to analyze the factors which might make effects on behavior changing and to build strategies set of parents and teacher guiding. This SF project aims to provide interface and backend analysis functionalities for project Deshang. The softwares used are WAMP (Window Apache + MySQL + PHP) with phpMyAdmin (web base MySQL admin console) included, WordPress (3.8.1 chinese version), Sphinx as search engine and libMMSeg chinese directionary for Sphinx.
