Open Source Linux Big Data Tools - Page 4

Build a standard data schema, and then implement various connectors to import systems you are familiar with for analysis. fooltrader is a quantitative analysis trading system designed using big data technology, including data capture, cleaning, structuring, calculation, display, backtesting and trading. Its goal is to provide a unified framework for the whole market (stock, futures, bonds, foreign exchange, digital currency, macroeconomics, etc.) for research, backtesting, forecasting, and trading. Its applicable objects include quantitative traders, teachers, and students majoring in finance, people interested in economic data, programmers, and people who like freedom and the spirit of exploration. You could write the Strategy using an event-driven or time walkway and view and analyze the performance in a uniform way.

The Esri Geometry API for Java can be used to enable spatial data processing in 3rd-party data-processing solutions. Developers of custom MapReduce-based applications for Hadoop can use this API for spatial processing of data in the Hadoop system. The API is also used by the Hive UDF’s and could be used by developers building geometry functions for 3rd-party applications such as Cassandra, HBase, Storm and many other Java-based “big data” applications.

The giServer is an easy-to-use integration server for process automation and event-driven or scheduled execution of batch jobs. Instead of using complex XML configuration files an elaborate GUI for batch job management is included. Some possible usage scenarios are: - Automatic processing of incoming data files - Big Data applications - Process automation - Data Mining/Aggregation applications - Automatic Reporting - Processing and analysis of database records

Apache Gravitino is a high-performance, geo-distributed, and federated metadata lake. It manages metadata directly in different sources, types, and regions, providing users with unified metadata access for data and AI assets.

OLAP technology is very useful for decision makers and data mining tools with BIG data. In this direction, we implement iCubing project with several multidimensional data cube approaches for cube indexing, querying, updating and mining. There are also several cube types, i.e. alphanumeric cubes, text cubes with unstructured data and geo cube with geo types, dimensions, measures and hierarchies, so the OLAP area continues a hard challenge after more than 20 years of the seminal paper of Jim Gray et al. in 1997. Our team has more than 15 years of experience in developing OLAP kernels.

How to extract meaningful information from big data has been a popular open problem. Decision tree, which has a high degree of knowledge interpretation, has been favored in many real world applications. However noisy values commonly exist in high-speed data streams, e.g. real-time online data feeds that are prone to interference. When processing big data, it is hard to implement pre-processing and sampling in full batches. To solve this trade-off, we propose a new decision tree so called incrementally optimized very fast decision tree (iOVFDT). Inheriting the use of Hoeffding bound in VFDT algorithm for node-splitting check, it contains four optional strategies of functional tree leaf, which improve the classifying accuracy. In addition, a multi-objective incremental optimization mechanism investigates a balance among accuracy, mode size and learning speed...

inMap is a big data visualization library based on Baidu Map. It focuses on the display of scatter, heat map, grid, and aggregation in the direction of big data. It is committed to making big data visualization easy to use.

This is a SAP addon to handle JSON data within SAP ABAP Programs. It comes in the customer exchange namespace /CEX/ and has to be installed as an SAP transport request. The addon supports object oriented JSON methods to process deep structured JSON data. Building JSON data from SAP data objects and parsing JSON data back to SAP data objects are supported. See the WIKI for some examples. Thanks to the SAP community and especially to Rüdiger Plantiko for the basic work (http://ruediger-plantiko.blogspot.de/2010/12/ein-json-parser-in-abap.html). Enjoy! last Changes: - JSON HTTP Client - HTTP Auth for Basic, SAP Basic+SSO, WSSE - Bugfixes: Big Integer, negative Integer - Array with has_next/next - Object with robust set_text method - OpenWeatherMap.org Example (see files/example)

Paralline executes a python function (or lambda function) or a script over each line of huge text files, in parallel processes and aggregates the result to a list.

This is a BIg Data project

wZD is a server written in Go language that uses a modified version of the BoltDB database as a backend for saving and distributing any number of small and large files, NoSQL keys/values, in a compact form inside micro Bolt databases (archives), with distribution of files and values in BoltDB databases depending on the number of directories or subdirectories and the general structure of the directories. Using wZD can permanently solve the problem of a large number of files on any POSIX compatible file system, including a clustered one. Outwardly it works like a regular WebDAV server.