DataEngineering · Batch Processing

Batch Processing

• dropbox/PyHive on GitHub, opens in a new tab

  1.7Kstars

  Python interface to Hive and Presto.

  Python·Other·steady·4 months ago

• brexhq/substation on GitHub, opens in a new tab

  403stars

  A cloud native data pipeline and transformation toolkit written in Go.

  Go·MIT License·steady·7 months ago

• datamechanics/delight on GitHub, opens in a new tab

  345stars

  A free & cross platform monitoring tool (Spark UI / Spark History Server alternative).

  Scala·Other·slowing·2 years agoarchived

• apache/incubator-hivemall on GitHub, opens in a new tab

  313stars

  Scalable machine learning library for Hive/Hadoop.

  Java·Apache License 2.0·dormant·4 years agoarchived

• Stratio/deep-spark on GitHub, opens in a new tab

  197stars

  Connecting Apache Spark with different data stores. Deprecated.

  Java·Apache License 2.0·dormant·10 years agoarchived

• shmlkv/dna-claude-analysis on GitHub, opens in a new tab

  54stars

  Personal genome analysis toolkit with Python scripts analyzing raw DNA data across 17 categories (health risks, ancestry, pharmacogenomics, nutrition, psychology, etc.) and generating a terminal-style single-page HTML visualization.

  Python·MIT License·steady·6 months ago

• rbmuller/datatrax on GitHub, opens in a new tab

  10stars

  Pure-Go classic machine learning toolkit and data engineering utilities. Eight algorithms with zero external dependencies.

  Go·MIT License·steady·4 months ago

• asavinov/bistro on GitHub, opens in a new tab

  8stars

  A light-weight engine for general-purpose data processing including both batch and stream analytics. It is based on a novel unique data model, which represents data via functions and processes data via columns operations as opposed to having only set operations in conventional approaches like MapReduce or SQL.

  Java·MIT License·dormant·8 years ago