Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.
-
Updated
Oct 26, 2026 - Python
#
ai-benchmark
Here are 203 public repositories matching this topic...
An agent benchmark with tasks in a simulated software company.
-
Updated
Nov 17, 2025 - Python
Frontier Models playing the board game Diplomacy.
-
Updated
Jun 1, 2026 - Python
JevBench by Benchmark Heaven: benchmark and leaderboard for AI decision models, measuring typed decision accuracy, calibration, latency and cost.
machine-learning leaderboard calibration model-evaluation systemone ai-benchmark decision-models jev jevbench jev-compatible decisionmodels benchmark-heaven
-
Updated
Oct 15, 2026 - Python
Track 33,707+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.
benchmarking benchmark evaluations evaluation benchmarks datasets benchmark-framework eval benchmark-suite evaluation-framework benchmark-datasets model-cards llm-evaluation ai-benchmark agent-benchmark agentic-benchmarking agent-benchmarking agent-bench agent-benchmarks agentic-benchmark
-
Updated
Oct 28, 2026 - Python
Ranking LLMs on agentic tasks
-
Updated
May 21, 2026 - Jupyter Notebook
The definitive benchmark for AI agents on OpenClaw. 45 tasks across 4 tiers. Powered by MyClaw.ai
-
Updated
Jul 20, 2026 - Python
开源劳动者AI研究与评测项目：建设劳动知识、公开评测基准与共同治理机制｜Open worker-centered AI research and evaluation project.
labor-rights workers-rights ai-ethics working-class responsible-ai ai-evaluation ai-governance open-source-ai ai-benchmark worker-ai algorithmic-management worker-centered-ai working-class-ai labor-studies
-
Updated
Oct 19, 2026 - Python
AI Stupid Level — independent, continuous LLM benchmarks (coding, reasoning, tool use), drift detection and a benchmark-driven Smart Router. Web app source; methodology public, test tasks private.
typescript nextjs drift-detection model-drift llm-evaluation ai-gateway llm-router llm-leaderboard ai-benchmark openai-compatible llm-benchmark
-
Updated
Oct 26, 2026 - TypeScript
🤖 A curated list of resources for testing AI agents - frameworks, methodologies, benchmarks, tools, and best practices for ensuring reliable, safe, and effective autonomous AI systems
testing qa benchmark machine-learning evaluation chaos artificial-intelligence chaos-monkey testing-tools awesome-list quality-assurance ai-safety ai-agents chaos-engineering llm llm-evaluation agentic-ai ai-benchmark agent-evaluation
-
Updated
May 28, 2025
llmBench is a high-depth benchmarking tool designed to measure the raw performance of local LLM runtimes (Ollama, llama.cpp) while providing deep hardware intelligence.
-
Updated
Mar 15, 2026 - Python
A visual benchmark testing whether leading coding agents repeat the same design patterns across 100 neutral website briefs.
-
Updated
Sep 19, 2026 - HTML
📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON with historical tracking.
benchmark machine-learning ai leaderboard text-to-video llm lmsys ai-benchmark chatbot-arena arena-ai
-
Updated
Oct 29, 2026 - Python
Benchmarking and evaluation resources for trust, robustness, and reliability analysis in graph-based AI systems.
-
Updated
Oct 24, 2026 - Python
An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark and a playthrough app.
python lua modding mcp game-ai claude script-extender linux-gaming baldurs-gate-3 llm-agents ai-benchmark model-context-protocol agent-evaluation agent-harness long-horizon-agents
-
Updated
Oct 29, 2026 - Python
Benchmark Heaven: AI model benchmarks with sources and modeled costs, plus the JevBench and ImageJevBench decision model leaderboards.
image-benchmark leaderboard model-evaluation model-comparison ai-benchmark decision-models jevbench jev-compatible benchmark-heaven
-
Updated
Oct 29, 2026 - HTML
MindTrial: Evaluate and compare AI language models (LLMs) on text-based tasks with optional file/image attachments and tool use. Supports multiple providers (OpenAI, Google, Anthropic, DeepSeek, Mistral AI, xAI, Alibaba, Moonshot AI, OpenRouter), custom tasks in YAML, and HTML/CSV/JSON reports.
opensource openai xai artificial-intelligence-projects anthropic ai-tool openrouter qwen deepseek mistral-ai llm-evaluation-framework google-gemini-ai llm-benchmarking moonshot-ai language-models-ai llm-comparison ai-benchmark ai-evaluation-tools grok-ai ai-model-comparison
-
Updated
Oct 27, 2026 - Go
Precision-aware retrieval benchmark for LLM memory systems.
benchmark information-retrieval ai memory evaluation memory-benchmark ai-agents long-term-memory rag llm ai-memory ai-benchmark memory-evaluation
-
Updated
Sep 17, 2026 - TypeScript
Benchmark of AI coding agents building web apps and deploying them to Cloudish, DigitalOcean, Render and Vercel, measuring success, time, tokens, cost, human steps and security.
digitalocean devops benchmark deployment render paas cloud-deployment ai-agents claude vercel cloudish anthropic evals llm-evaluation coding-agents ai-benchmark claude-code
-
Updated
Oct 27, 2026 - Python
GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks
svg design benchmark layout evaluation typography graphic-design lottie vlm multimodal vision-language-model ai-benchmark
-
Updated
May 5, 2026 - Python
Add this topic to your repo
To associate your repository with the ai-benchmark topic, visit your repo's landing page and select "manage topics."