bigcodebench

Here are 4 public repositories matching this topic...

abhaymundhara / llm-benchmark-suite

Benchmark suite for evaluating LLMs and SLMs on coding and SE tasks. Features HumanEval, MBPP, SWE-bench, and BigCodeBench with an interactive Streamlit UI. Supports cloud APIs (OpenAI, Anthropic, Google) and local models via Ollama. Tracks pass rates, latency, token usage, and costs.

python benchmark evaluation gemini openai code-generation claude streamlit humaneval llm ollama swe-bench mbpp bigcodebench

Updated Apr 23, 2026
Python

edward-lcl / factor-ut-untrusted-decomposer

Star

Factor(UT): Controlling Untrusted AI Decomposers — AAAI 2026 workshop paper on monitoring untrusted decomposition in code generation workflows.

ai-safety deception-detection aaai-2026 inspect-ai bigcodebench

Updated Aug 11, 2026
Jupyter Notebook

luka-krixvon / resample-or-reroute-experiment

Star

Resample or reroute after a weak-verifier stop? Pre-registered measurements of recoverable stopping debt on MBPP+, a two-sided action-support gate on BigCodeBench that fails closed, a LiveCodeBench observability ladder, and the exchangeable-actions reference showing realized-maximum gaps carry no selector signal. Artifacts for arXiv:2607.08665v3.

identification reproducibility verifier pre-registration negative-results llm-routing test-time-compute livecodebench mbpp bigcodebench

Updated Sep 4, 2026
Python

edward-lcl / bcb-deep-equal

Star

Semantic deep-equality for BigCodeBench floating-point outputs. Published on PyPI.

testing pypi floating-point python-package bigcodebench

Updated Apr 1, 2026
Python

Add this topic to your repo

To associate your repository with the bigcodebench topic, visit your repo's landing page and select "manage topics."

Learn more