ml-evaluation

An End-to-End Evaluation Framework for Entity Resolution Systems

Benchmark your MCP server.

Early-warning research benchmark for synthetic-data-induced model degradation using cross-dataset drift metrics, warning lead time, and statistical validation.

TorchMetrics-based evaluation metrics for robotics policies and robot learning

Fix the pass bar before the test runs. PRML is an open spec and CLI that hashes and timestamps evaluation criteria (SHA-256, RFC 3161) so anyone can check them later. Python, JS, Go, Rust.

Live index of LLM evaluation tools and benchmarks, refreshed every 15 minutes from GitHub

✍️ Collaborate on writing technical content for the Giskard Community

An open-source Streamlit web app to generate beautiful confusion matrices for multi-class machine learning models. Supports numeric and string labels, CSV upload, manual label entry, custom color maps, and displays evaluation metrics like Accuracy, Precision, Recall, and F1-score. Users can download the confusion matrix as an image.

Measurement-disciplined optimizer for Claude Agent Skills — three-gate system (stability, effect size, function preservation), anchored rubric, train/holdout split. Inspired by Karpathy's autoresearch and alchaincyf's darwin-skill.

Enterprise-grade machine learning observability platform that detects data drift, concept drift, and performance degradation in production models. Features statistical drift detection (KS test, PSI), real-time alerting, Redis caching, and FastAPI backend.

Safety-first legal NLP system with chunk-based long-document processing, deterministic inference, clause extraction, and rule-based risk engine — built for traceability and deployment constraints.

GitHub Action for SWE-bench Pro evaluation powered by mcpbr

A hands-on distributed streaming system built with Python and Apache Kafka to process network logs and events in real time, while documenting the learning process step by step.

Field manual for the PRML v0.1 specification. 13 patterns, 4 anti-patterns, 4 working examples. CC0.

Data Science Challenge from Coursera Project : Loan Default Prediction

This project contains codes and paperwork based on the course CSI5155 at University of Ottawa (delivered by Professor Dr. Herna Viktor).

Open-source, local-first research engineering system for running, evaluating, comparing, and reproducing computer-science experiments.

Browser-based ML evaluation dashboard. Upload a predictions CSV, get confusion matrix, ROC curve, precision-recall analysis, and per-class metrics — all computed client-side.

A decision-oriented benchmark framework for evaluating action-conditioned world models beyond static AI benchmarks.

Reproducible ML research & evaluation harness for financial time series: time-aware validation, PyTorch models, cost-aware metrics, experiment contracts and audit gates.