ml-evaluation

An End-to-End Evaluation Framework for Entity Resolution Systems

Benchmark your MCP server.

Early-warning research benchmark for synthetic-data-induced model degradation using cross-dataset drift metrics, warning lead time, and statistical validation.

Fix the pass bar before the test runs. PRML is an open spec and CLI that hashes and timestamps evaluation criteria (SHA-256, RFC 3161) so anyone can check them later. Python, JS, Go, Rust.

TorchMetrics-based evaluation metrics for robotics policies and robot learning

Live index of LLM evaluation tools and benchmarks, refreshed every 15 minutes from GitHub

✍️ Collaborate on writing technical content for the Giskard Community

An open-source Streamlit web app to generate beautiful confusion matrices for multi-class machine learning models. Supports numeric and string labels, CSV upload, manual label entry, custom color maps, and displays evaluation metrics like Accuracy, Precision, Recall, and F1-score. Users can download the confusion matrix as an image.

Measurement-disciplined optimizer for Claude Agent Skills — three-gate system (stability, effect size, function preservation), anchored rubric, train/holdout split. Inspired by Karpathy's autoresearch and alchaincyf's darwin-skill.

GitHub Action for SWE-bench Pro evaluation powered by mcpbr

A hands-on distributed streaming system built with Python and Apache Kafka to process network logs and events in real time, while documenting the learning process step by step.

Enterprise-grade machine learning observability platform that detects data drift, concept drift, and performance degradation in production models. Features statistical drift detection (KS test, PSI), real-time alerting, Redis caching, and FastAPI backend.

Data Science Challenge from Coursera Project : Loan Default Prediction

Safety-first legal NLP system with chunk-based long-document processing, deterministic inference, clause extraction, and rule-based risk engine — built for traceability and deployment constraints.

Field manual for the PRML v0.1 specification. 13 patterns, 4 anti-patterns, 4 working examples. CC0.

Open-source, local-first research engineering system for running, evaluating, comparing, and reproducing computer-science experiments.

This project contains codes and paperwork based on the course CSI5155 at University of Ottawa (delivered by Professor Dr. Herna Viktor).

A decision-oriented benchmark framework for evaluating action-conditioned world models beyond static AI benchmarks.

Browser-based ML evaluation dashboard. Upload a predictions CSV, get confusion matrix, ROC curve, precision-recall analysis, and per-class metrics — all computed client-side.

Reproducible Search Ads research benchmark: 149.6M public KDD rows, GPU CTR modeling, 17M-session controlled experimentation, causal inference, and safe policy evaluation.