io.github.KevinRabun/judges

List all available judges on the Agent Tribunal panel, including their areas of expertise and what they evaluate.

Run policy-aware tribunal evaluation with named policy profiles (startup, regulated, healthcare, fintech, public-sector), evidence calibration from runtime metrics, specialty-per-judge feedback, confidence scoring, and uncertainty reporting. Use this when code must meet specific compliance or vertical requirements.

List files and directories in the workspace. Useful for exploring project structure before evaluating code.

Read the contents of a file in the workspace. Returns the file text, or an error if the file is too large or missing.

Evaluate code with progressive judge-by-judge reporting. Returns intermediate counts as each judge completes, useful for large files where full tribunal takes time.

Generate the boilerplate files to add a new judge to the Judges Panel. Creates the judge definition (with self-registration), evaluator skeleton, and tells you the one line to add to index.ts. Validates that the judge ID and rule prefix are unique.

Generate a starter plugin template for the Judges Panel. Creates a self-contained plugin file with custom rules, optional custom judges, and lifecycle hooks.

Clone a public repository URL, run the full judges panel across source files, and generate a consolidated markdown report.

Run a 3-step app-builder workflow: tribunal review, plain-language risk translation, and prioritized remediation tasks with AI-fixable P0/P1 items.

Analyze a PACKAGE MANAGER manifest file (NOT infrastructure code) for supply-chain risks, version pinning issues, typosquatting indicators, and dependency hygiene. ONLY accepts: package.json, requirements.txt, Cargo.toml, go.mod, pom.xml, .csproj. Do NOT use this for Bicep, Terraform, ARM templates, CloudFormation, Dockerfiles, or any other infrastructure/deployment configuration — use evaluate_code or evaluate_code_single_judge for those.

Run the benchmark suite and check results against quality thresholds. Returns pass/fail with metric details including F1, precision, recall, and detection rate. Use in CI pipelines to prevent quality regressions.

Run the full benchmark suite and return a detailed dashboard with per-judge, per-category, and per-difficulty breakdowns. Includes precision, recall, F1, false positive rates, and individual case results. Use this to understand overall system quality and identify weak spots.

Evaluate multiple code files in a single call. Returns per-file verdicts with scores and findings, plus aggregate statistics.

Evaluate code and automatically generate fix patches for all findings that have auto-fix support. Returns the evaluation verdict alongside ready-to-apply patches. Use this for a single-step

Run a focused evaluation using only the specified judges. Use this after an initial full evaluation to re-check specific areas — for example, re-run only

Get the current evaluation session status — how many evaluations have been run, detected frameworks, verdict history per file, and stability indicators. Useful for understanding what the tribunal has already reviewed.

Record user feedback on a finding — mark it as a true positive (tp), false positive (fp), or won

Evaluate code changes from a git diff. Parses the unified diff from a git repository, identifies changed files and lines, and runs the full tribunal on each changed file — filtering findings to only those on changed lines. Supports both live git repos (provide repoPath + base ref) and pre-computed diffs (provide diffText).

Explain a Judges Panel finding in plain language. Provides OWASP/CWE references, risk context, and remediation guidance based on the rule ID and finding details.

Set the triage status of a tracked finding. Records the decision (accepted-risk, deferred, wont-fix, false-positive) with attribution and reason. Feeds back into calibration to reduce future false positives.

Get finding lifecycle statistics for the project: open, fixed, recurring, and triaged finding counts plus trends. Shows whether code quality is improving or declining over evaluation runs.

Analyze finding suppression patterns: FP rates by rule, suppression rates by judge, auto-suppress candidates, and actionable recommendations for tuning. Use this to understand which rules need calibration.

List findings that have been triaged (accepted-risk, deferred, wont-fix, false-positive). Optionally filter by triage status. Shows the triage decision, reason, and who made it.

Re-evaluate code with developer-provided context from a multi-turn conversation. Accepts disputed findings, accepted findings, and additional context to adjust the evaluation. This is the agentic feedback loop — the developer explains their intent and the tribunal re-evaluates with that context, applying auto-tune and confidence filtering.

Evaluate code with the Judges Panel and automatically apply all available auto-fix patches. Returns the fixed code along with a summary of applied and remaining findings. Use this to fix security, performance, and quality issues in a single step.