large-data

C++ DataFrame for statistical, financial, and ML analysis in modern C++

A Kafka Serde that reads and writes records from and to Blob storage (S3, Azure, Google) transparently.

Tabular Data Viewer 🀄 VSCode extension for viewing very large local and remote CSV and TSV data files with Tabulator Table, Perspective Datagrid and D3FC Chart Views 📊📈

The official repo for [ACM CSUR'24] "Empowering Agrifood System with Artificial Intelligence: A Survey of the Progress, Challenges and Opportunities"

Columnar query engine for larger-than-RAM data in R

Fast multiprocessing.Queue alternative for large Python objects and NumPy arrays: payloads move through shared memory instead of pipes. Typed, no dependencies, Linux/macOS/Windows.

Leaflet WebGL plugin for rendering one million+ (1M) point markers with GPU projection and FBO picking

Windows VSS (Volume Shadow Copy) path detector for 7-Zip and backup tools

This repository contains introduction to pandas which is a software library written for the Python programming language for data manipulation and analysis.

Wrapping single instance learning algorithms for fitting them to data for multiple instance learning

Evidence-backed Codex Skill for finding production-scale risks in data-heavy features before they become outages.

Read large JSON files in PHP one item at a time — arrays or objects, without loading the file into memory. Faster than JsonMachine; conformance checked against JSONTestSuite.

File Search-Sorting Algorithms for DBMS

💯 A Ruby on Rails app to generate Fizzbuzz numbers up to 100,000,000,000

Home Credit Risk Classification

Analyzes Grid Engine log files converting them into a format suitable for time series analysis using Grafana.

Using Python to analyze large datasets

External sorting implementation in Java for processing CSV datasets larger than memory using run generation and multi-way merge.

A Java library for working with `byte[]` and `ByteBuffer` through unified, segmented byte-array views with compile-time read-only, write-only, and read-write access control, plus support for 64-bit indexing, slicing, and joining across multiple buffers.

This project demonstrates various real-time data streaming approaches in an Express.js server