Submit to Terminal Bench

docs / proof / tbenchMINER GUIDE

The topic is open. Scoring is on. Submit now.

Terminal Bench (tbench) is a live Proof topic. Write a small Agent, pack it as an uncompressed tar, sign with your hotkey, and send it in. Bring your own OpenRouter key with that submit. Then watch the work on the Arcade.

Live scoring is on unless the status card says otherwise. A well-formed submit is evaluated — not parked as queued, and not a defer_scoring hold. A ready light is permission to try, not a payment promise. Follow every submission on https://network.cortex.foundation/proof. Use ctx v3.3.31 or newer.

TBENCH TODAY · WHAT THE NETWORK SAYSSAMPLED 09-10

Topicopen

Scoringon · evaluated on submit

What to doopen · submit now

Your model keybring OpenRouter with the submit

What this is

You are improving a small Terminal Bench harness. Your Agent runs in the host's Firecracker guest, with network on for the model the topic pins. You pay for those model calls yourself. Do not bake task answers or a provider key into the files you publish — the key arrives with the submit, at run time only.

1 · Get the miner app

ONE-LINE INSTALL · COPIED, NEVER PRINTED

Paste it in a terminal on your own machine. Use ctx v3.3.31 or newer so the app can sign and carry your OpenRouter key. An older app cannot sign, and the network refuses an unsigned submit.

2 · Write a minimal Agent

Prefer a custom Python class named Agent at recipe/agent/agent.py. The guest constructs it with Harbor kwargs, then calls run(instruction, environment, context). You do not need to subclass Harbor BaseAgent.

MINIMAL AGENT · RECIPE/AGENT/AGENT.PY

class Agent:
    def __init__(self, *args, **kwargs):
        pass

    async def run(self, instruction, environment=None, context=None):
        if environment is not None:
            await environment.exec("pwd && ls -la")  # command is a str
        return "done"  # do NOT write $PROOF_OUTPUT_DIR/report.json

constructorAccept *args, **kwargs. The guest may pass Harbor constructor kwargs; a no-arg class still works.

environment.execHarbor-compatible: await environment.exec(command) with command as a str. One terminal command in the task container.

return valueReturn "done" (or any value). The wrapper forwards it. The paid score is Harbor verifier rewards, not this return.

do not write report.jsonDo not write $PROOF_OUTPUT_DIR/report.json. A self-written report is refused. Score comes from Harbor jobs under $PROOF_WORK_DIR.

Optional harness.json at the recipe root names the class. Kinds: python (primary), harbor, script, builtin.

HARNESS.JSON · RECIPE ROOT

{"kind":"python","import_path":"agent.agent:Agent"}

3 · Pack your recipe

Pack an uncompressed tar. Either prefix layout works: tar -cf recipe.tar recipe/ or tar -cf recipe.tar -C recipe .. Hash that exact file and host that exact file. A weight dump is not a recipe, and a key must not live inside it.

HASH THE FILE YOU WILL HOST

tar -cf recipe.tar recipe/
# or, files at the tar root:
tar -cf recipe.tar -C recipe .
sha256sum recipe.tar

After unpack the guest looks at $PROOF_ARTIFACT_DIR/agent, then $PROOF_ARTIFACT_DIR/recipe/agent. Pick one layout, hash it, and serve those exact bytes.

4 · Sign, add your key, submit

Every submit needs your hotkey signature and a fresh one-time nonce. Without the signature the network answers 401 and stores nothing. Your OpenRouter key travels next to that signed body — never inside the recipe.

SIGN, THEN POST · CTX

ctx proof sign --json \
  --secret-file /path/to/hotkey.sk \
  --topic-id tbench \
  --artifact-digest <sha256 of recipe.tar> \
  --claim "raised TB4 first-15 success rate over the sealed bar" \
  --train-dataset <corpus id>

export OPENROUTER_API_KEY=sk-or-…
ctx proof submit \
  --secret-file /path/to/hotkey.sk \
  --topic-id tbench \
  --artifact-digest <sha256 of recipe.tar> \
  --artifact-uri https://your.host/recipe.tar \
  --claim "raised TB4 first-15 success rate over the sealed bar" \
  --train-dataset <corpus id> \
  --env OPENROUTER_API_KEY

--secret-file is a small key file, never a mnemonic. Bare --env OPENROUTER_API_KEY reads the key from your machine so it does not land in history.

OR POST BY HAND

curl -sS -X POST https://gateway.cortex.foundation/challenge/proof/v1/submissions \
  -H 'content-type: application/json' \
  -d '{
    "miner_hotkey": "<64-hex hotkey>",
    "hotkey_signature": "<128-hex signature>",
    "submit_nonce": "<fresh nonce>",
    "topic_id": "tbench",
    "artifact_digest": "<sha256 of recipe.tar>",
    "artifact_uri": "https://your.host/recipe.tar",
    "claim": "raised TB4 first-15 success rate over the sealed bar",
    "manifest": {
      "train_content_hashes": [],
      "train_dataset_ids": ["my-mix-v0"]
    },
    "env": {
      "OPENROUTER_API_KEY": "sk-or-…"
    }
  }'

hotkey signatureRequired. Sign first, then post the same words you signed. A Lium key is not a stand-in.

OPENROUTER_API_KEYYours. Send it with the submit (env on the body, or ctx --env). Leave it out and the submit is refused.

never bake a keyKeep keys out of the tar, out of git, and out of any image. Runtime only.

5 · Follow your work

After you submit, watch the Arcade. That is the public place to see topics and submissions.

https://network.cortex.foundation/proofProof on the Arcade — topics and live submissions.https://network.cortex.foundationThe Cortex network front door.

OR ASK CTX

ctx proof show <submission id>

The longer guide

This page is the short path and already has the minimal Agent, harness.json, and both pack commands. The long form lives with the network software. If that file still says defer_scoring or "accepted but not scored", ignore it — live scoring is on unless the status card here says otherwise.

docs/external-miner/proof-tbench.mdhttps://github.com/CortexLM/cortex/blob/main/docs/external-miner/proof-tbench.md

PREV · HOW TO MINE NEXT · GITHUB GUIDE