Snapshot
A read-only, content-addressed copy of your repository. Your files and your git index are never touched; secrets, symlinks and caches are left out.
RepoBoss injects real bugs into your repository, hides the tests, lets an AI agent try to fix them, and judges every change against your protected test suite. No leaderboards built on somebody else's code: measured results on yours.
Every number RepoBoss reports comes from running your real test suite. Nothing is estimated.
A read-only, content-addressed copy of your repository. Your files and your git index are never touched; secrets, symlinks and caches are left out.
Located on the syntax tree (Python ast, tree-sitter for everything else) with exact spans and hashes, never search-and-replace.
Compile errors, timeouts and missing dependencies are rejected.
Tests are hidden, build configuration is read-only, and the attempt runs isolated with no network by default.
Only allowed changes go into a clean tree with the protected tests. Outcome, patch, tokens, model version and time are stored.
A constant drifts: the kind of bug that hides in plain sight.
src/shopcalc/settings.py -TAX_RATE = 0.22 +TAX_RATE = 0.44
A data file reference breaks. Only strings that point to real files qualify.
src/catalog.rs -.join("catalog.csv") +.join("catalogs.csv")
An off-by-one boundary inside a function. Generics are never touched.
src/pricing.js - if (quantity <= 0) { + if (quantity < 0) {
RepoBoss ships a tool-using agent loop. The model lists, reads and searches files, edits them through
checked tools, then calls finish. Point it at a local model through Ollama, or at any
OpenAI-compatible API.
qwen3:14b@bdbd181c33f2).# a local model, no account needed [agents.qwen] type = "llm" provider = "ollama" model = "qwen3:14b" # any OpenAI-compatible API [agents.gpt] type = "llm" provider = "openai" base_url = "https://api.openai.com/v1" model = "gpt-4.1-mini" api_key_env = "OPENAI_API_KEY"
$ repoboss run ch-60427357 --agent qwen --attempts 5 $ repoboss compare --challenge ch-60427357
31 attempts on 15 challenges · Ollama · local runner · numbers copied from the RepoBoss database.
Broken file names are visible in the code, and the model fixed them almost every time. A changed constant is not: with the tests hidden, the right value cannot be inferred. That is the kind of insight RepoBoss surfaces about your setup. A few attempts are descriptive, never a general ranking.
repoboss detect recognises npm/pnpm/yarn, Python, Cargo, Go, .NET, Maven, Gradle, Bundler, Composer, CMake, Mix and Dart projects.
End to end = verified with the real toolchain on the bundled examples. The others are covered by unit tests of the parsers and of bug injection.
Every interface drives the same engine, so results are identical everywhere.
repoboss gui: a dark desktop app in pure Python (Tkinter) for Windows, macOS and Linux.
repoboss dashboard: a local, read-only web dashboard with demo labels, isolation badges and patches.
repoboss interactive: guided menus with no flags to remember.A native macOS app with the same sections is in macos/. It has not been built on a Mac yet, so it is
published as source only. On macOS today, use repoboss gui, the dashboard or the terminal.
The Docker runner installs dependencies into an image, removes every source file from it, and runs each attempt in a fresh container: no network, CPU/memory/PID limits, all capabilities dropped, nothing from your machine mounted. Links coming back are rejected. A local runner exists for trusted code, and it says so on every run.
Honest status: the Docker runner is implemented and unit-tested, but its end-to-end run has not been executed yet on a machine with Docker. Docker reduces risk; it is not a perfect boundary.
pip install git+https://github.com/diegobruvhrpiugvjrigjtibjte/repoboss
repoboss init --runner local
repoboss challenge create --repo path/to/your/project
repoboss run ch-XXXXXXXX --agent qwen --attempts 5
repoboss gui
No. RepoBoss measures agents on your repositories, in your environment. With a handful of attempts it shows descriptive numbers and says so; it never ranks models in general.
The tests are not in its copy, configuration files are read-only, and the evaluation rebuilds a clean tree from the original snapshot. Touching a protected path rejects the attempt outright.
A compiler points straight at the bug. A good challenge must build and fail a test, so the agent has to understand the code.
There is no native integration yet, because none has been verified against the real tool. Any agent that runs non-interactively can be plugged in as a command agent today.
Yes. Use a local model through Ollama, or try the pipeline with the built-in demo adapters, which are clearly labelled and never presented as a model's result.
Measure it on your code.