Agentic Proxy Quickstart (Phase 24)

yaarbal proxy is a loopback OpenAI-compatible server for agentic coding tools (Cline, or any client that speaks openai-python's wire shape). It holds your own Ed25519 spend key, signs its own settles, parses tool calls exactly as your own node emitted them, and routes tool-carrying traffic only to the node you own.

1. What you get

A local server at http://127.0.0.1:7100/v1 that:

• Streams and non-streams POST /v1/chat/completions, lists GET /v1/models — reflecting the peers actually reachable right now, not the static model catalog.
• Signs and submits its own /credits/settle, exactly once per completion, from your own spend identity.
• Parses tool calls from your own node's raw text output (Qwen3.5's <function=…> convention) and returns them in the OpenAI tool_calls[] wire shape.
• Refuses tool-carrying requests to any node it does not identify as your own — plain chat keeps the whole fleet; only tool calling is gated.

What it is not:

• Not an autocomplete/FIM endpoint. Measured unfit (spike-028/REQUIREMENTS.md "Out of Scope"): a ~300ms per-request protocol floor blows a ~300ms tab-completion budget, and the single inference slot would be monopolized by every keystroke.
• Not built on llama-server. Tool calls are parsed proxy-side from the plain-GGUF text output (spike 047); no second inference backend is introduced.
• Not a tracker-side gateway. Custodying your private key on the tracker to sign settles on your behalf would repeat the "seeder submitting its own settlement" anti-pattern this network specifically forbids — your key never leaves this machine.

2. Install

The proxy is part of the yaarbal wheel. Install it the way the CLI Quickstart §1 describes — the wheel (yaarbal-<version>-py3-none-any.whl) plus the installer that comes with it, no source checkout needed:

YAARBAL_WHEEL=./yaarbal-0.1.0-py3-none-any.whl bash install.sh
source ~/.rentmygpu-venv/bin/activate

On Windows: .\install.ps1 -Wheel .\yaarbal-0.1.0-py3-none-any.whl. Do not pass --no-seeder / -NoSeeder: the proxy needs the seeder extra. (Team members with a source checkout: pip install -e ".[seeder]" with the llama-cpp-python index for your platform, or scripts/install.sh.)

That extra pulls in flask, structlog, and llama-cpp-python — the proxy reads the model's tokenizer.chat_template straight out of the GGUF's own metadata, so the model file must be present locally for any tool-carrying request:

yaarbal models pull qwen3.5-9b-q4_k_m

(Plain chat does not need the file locally — see §9.)

3. Run

yaarbal proxy

Defaults: binds 127.0.0.1:7100 (never any other interface — loopback-only is a security property, there is deliberately no --host flag), auto-discovers a model per request if none is specified, and ships with thinking off by default (§5).

Flags:

• --port <n> — override the loopback port (default 7100; deliberately not Ollama's 11434, so the two can run side by side).
• --model <catalog-id> — default model when a request omits model. Optional; otherwise the proxy auto-discovers a reachable peer per request.
• --tracker-url <url> — same fallback chain as yaarbal spend: RENTMYGPU_TRACKER_URL, then [tracker] url in ~/.rentmygpu/config.toml.
• --identity <path> — the spend identity that signs settle envelopes. Defaults to ~/.rentmygpu/spend-identity.key. This must never be the seeder's own identity.key (the same rule docs/QUICKSTART-CLI.md §5 states for yaarbal spend) — the tracker rejects a settle where leecher_id == seeder_id (credits_routes.py), and the routing gate identifies your own node by the earn identity at ~/.rentmygpu/identity.key instead, so the two stay distinct by construction on own-node traffic.
• --thinking — default enable_thinking=True for every request (§5's opt-out).
• --allow-trusted-seeders — widen tool-carrying eligibility to ledger-trusted seeders. OFF by default; see "Widening beyond your own node" (§6).

The API key: generated on first run, saved 0600 at ~/.rentmygpu/proxy-key, and printed exactly once. Reprint it any time with:

yaarbal proxy --show-key

It never leaves the machine and is not a network credential — it is only what your local client uses to authenticate to your own loopback server.

4. Point a client at it

OpenAI SDK

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:7100/v1", api_key="rmgp-...")

client.models.list()

resp = client.chat.completions.create(
    model="qwen3.5-9b-q4_k_m",
    messages=[{"role": "user", "content": "Say hi"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")

resp = client.chat.completions.create(
    model="qwen3.5-9b-q4_k_m",
    messages=[{"role": "user", "content": "List files in the current directory."}],
    tools=[{
        "type": "function",
        "function": {
            "name": "list_files",
            "description": "List files in a directory",
            "parameters": {"type": "object", "properties": {"path": {"type": "string"}}},
        },
    }],
)
print(resp.choices[0].message.tool_calls[0].id)  # 32-char id

Cline

In Cline's settings: provider "OpenAI Compatible", base URL http://127.0.0.1:7100/v1, API key from yaarbal proxy --show-key, model id qwen3.5-9b-q4_k_m.

Keep Cline's own tool-approval gate switched on. It is the real safety boundary here, not this proxy. Spike 048's open finding stands: the parser cannot distinguish an injected tool call from one the model genuinely decided to make — this proxy only narrows who can produce the call (your own node), not whether any given call is safe to execute. Never configure Cline to auto-approve tool execution against this proxy. See "Widening beyond your own node" (§6) before allowlisting any peer or enabling the trusted opt-in — manual approval is required for both.

5. Thinking is off by default (and how to turn it on)

enable_thinking defaults to false. Measured reason (spike 047d, 32 trials each way, single-turn): disabling the model's <think>...</think> reasoning preamble cuts paid output tokens 68.8% at 0.0 percentage points of accuracy change, and turns a 100k-token welcome grant into roughly ~521 agentic steps instead of ~162 — about 3.2x more completions from the same credits. Thinking is a real-money cost lever, not a quality one, on this workload.

Re-measured over long multi-turn agentic loops (Phase 27 VALD-03, decided in docs/VALIDATION-REPORT.md section 9 against section 5's numbers): spike 047's single-turn result holds. A paired A/B over 18 real agentic-loop runs (9 per arm, three tasks, enable_thinking alternating inside every task/rep pair) measured pass_rate 9/9 vs 9/9 (0.0 pt delta) and a 70.6% paid-token reduction (4,536 vs 15,430 billed completion tokens over the same 78/91 turns). The decision was to keep the default off — the two qualifications the report itself insists on being read alongside the win: (1) both arms scored the maximum on the three measured tasks, so this A/B confirms enable_thinking=false costs no accuracy on these tasks, not that the two settings are equally accurate on a harder or longer task set; (2) protocol fidelity favoured thinking off too — 0 degraded turns in 78 vs 13 in 91 with thinking on (11 of those missing: the model reasoned and then didn't call, recovered every time by the harness's client-side nudge). Cost also grows with loop length under thinking-on (169.56 tokens/turn here vs 123 on spike 047's single turns) — the longer the loop, the more the on-arm reasons per turn.

Three opt-outs, in precedence order (highest wins):

1. Per-request, top-level: {"enable_thinking": true, ...}, or the vLLM-style nested form some clients send: {"chat_template_kwargs": {"enable_thinking": true}}.
2. Per-process, CLI flag: yaarbal proxy --thinking.
3. Per-process, environment: RENTMYGPU_PROXY_THINKING=true.

When thinking is on, reasoning never appears inside content. It streams and completes separately as reasoning_content (both as a delta field on streaming chunks and as a top-level message field on non-streaming responses) — treat it as scratch work, never as the answer.

6. Routing and refusals

The routing gate runs at peer selection, before any prompt byte leaves the machine. It applies only to requests that carry tools — plain chat is ungated and reaches the whole fleet, including nodes you do not own.

A tool-carrying request is refused outright, never silently downgraded to a stranger's GPU and never silently answered as plain chat with tools stripped. Every error is an OpenAI-shaped error object (type: invalid_request_error or server_error) naming the code:

Busy behaviour under contention (Phase 27 SUPL-01)

The seeder serves one generation at a time (BoundedSemaphore(1), §"Routing and refusals" above). A second request that arrives while the slot is held queues silently — no bytes at all — until either the slot frees up (served normally, 200) or the seeder's own queue timeout expires (seeder_busy, 503, with a Retry-After header).

Measured (docs/VALIDATION-REPORT.md sections 2 and 6): a contender queued behind a short (~6.8s) occupier waited 5,042 ms, then was served normally. A contender queued behind a long (~40.8s) occupier received zero bytes for 30,045 ms before the busy answer — the full prior fixed 30s queue timeout. Separately, a sweep of 1–6 concurrent short agentic sessions against one seeder measured zero busy turns at every point, with the worst single queued wait 5,472 ms (p95 4,863 ms) at n=6 — well under either timeout.

Decided (section 9): the queue timeout is lowered to a 15.0s default, about 2.7x the measured 5.5s worst-case queued wait above, while halving the prior silent-wait ceiling. It is now env-configurable per seeder node via RENTMYGPU_SEEDER_QUEUE_TIMEOUT (float seconds; invalid, non-positive, non-finite, or above-threading.TIMEOUT_MAX values fall back to 15.0 with a warning) — an operator can restore 30s, or choose any other value, without a code change. The busy response shape itself (503 {"error": "busy", "retry_after": 5}, Retry-After: 5 header) is unchanged; only the wait before it moves. One consequence of a lower default, stated plainly: a session that would have been served after 15–30s of queued wait under the old default now gets a 503 instead — the ceiling above the measured n=6 sweep point remains unmeasured under the new default.

History trim (Phase 26 CTX-03). Before context_length_exceeded fires, the proxy first tries to make the conversation fit: it drops the OLDEST whole turn first (a tool_calls assistant turn and its tool result travel together, never split) against a budget derived from the serving seeder's advertised n_ctx, and re-renders. The system message and your current (live) message are never dropped. context_length_exceeded now means the history no longer fits even after every droppable turn has been dropped — a genuinely unrecoverable request, not just a long one. A trim is logged (proxy.history_trimmed, counts only) and is never written into the response — it never appears in the completion text or the SSE stream.

Three generic Flask-level errors can also occur outside the routing/completion path: not_found (404, unknown path), method_not_allowed (405, wrong HTTP verb on a known path), and payload_too_large (413, request body over the server's size cap).

Widening beyond your own node

Manual approval is required for ANY widened configuration. Your own node is the only configuration with no untrusted producer. Widening WHICH seeder may receive tool-carrying traffic — via the allowlist or the trusted opt-in below — never widens WHO approves what a tool call does. The proxy never configures, signals, or documents an auto-approval of side-effecting tools against a node you do not own; that decision stays entirely in your agentic client, every time, for every non-owned node (see "Keep Cline's own tool-approval gate switched on" above — this section reinforces that passage, it does not soften it).

Allowlisting a peer. The allowlist is a small JSON file at ~/.rentmygpu/agentic-allowlist.json (override with RENTMYGPU_PROXY_ALLOWLIST_PATH), managed with three CLI subflags:

yaarbal proxy --allow-peer <peer_id>    # add a peer_id, print the new count, exit
yaarbal proxy --deny-peer <peer_id>     # remove a peer_id, exit
yaarbal proxy --list-allowed            # print the allowlist, one peer_id per line, exit

Changes take effect on the next tool-carrying request without restarting the proxy — the file is re-read per request, not snapshotted at process start. Removing an entry refuses the next request again with not_own_node_and_not_allowlisted, immediately, no restart. A malformed allowlist file is treated as empty — never widened — and logs a proxy.allowlist_load_failed warning once per broken file version. Allowlisted peers are still not your own node: every completion they serve carries X-RentMyGPU-Require-Approval: true (below), and manual approval in your client is still required.

The trusted opt-in. --allow-trusted-seeders (or RENTMYGPU_PROXY_ALLOW_TRUSTED) is off by default and makes ledger-trusted seeders eligible for tool-carrying traffic. Enabling it is a deliberate acceptance of what trusted actually measures, not a safety feature: spike 049 measured that a seeder self-grants trusted with about 20 one-token settles in about 2 minutes of wall time at the settle rate limit, and the attack is net-free — crossing the reputation threshold auto-releases the 100 escrowed credits straight back to the seeder (0.1% of one 100k welcome grant, refunded). trusted is escrow-release probation (D-08), not an integrity attestation, and it cannot authorize code execution on your machine. This is exactly why a trusted seeder is refused trusted_but_not_sufficient (§6 table) while the opt-in is off — the refusal is not a judgment that the seeder "looks shady", it is a statement that the flag alone was never designed to carry this weight. If you enable the opt-in anyway, you are accepting that cost with open eyes, not being told it is safe.

What the proxy tells your client. Every completion routed to a non-owned node carries X-RentMyGPU-Require-Approval: true on the response — present only when the serving node is NOT your own (false only when your own node served it, meaning no untrusted producer produced that completion; the header is absent entirely on plain chat). false is a statement about who produced the tokens, never a recommendation to auto-approve. On the trusted opt-in path only, the response additionally carries X-RentMyGPU-Routing-Warning with the cost warning above. Two structured log events fire on every such completion, never once per process or once per peer: proxy.non_own_node_route (the allowlist path) and proxy.trusted_route (the trusted opt-in path). The warning text is never written into the completion content or into an SSE delta — it would land where an agentic client parses tool calls from, corrupting the exact output you are billed for.

Selection order. Among eligible candidates, selection is tiered and deterministic, never random across tiers: your own node first, then allowlisted peers in allowlist order, then trusted-opt-in peers last. The most user-controlled identity always wins.

Production note. The deployed tracker at https://rmg-tracker-oracle.yaarbal.app is behind this revision. Until it is redeployed from a build that includes this phase's commits, its /peers, /model/<name>/peers, and GET /credits/<peer_id> responses carry no trusted key at all — not false, simply absent, which every consumer treats identically to false (fail closed). That means --allow-trusted-seeders makes no production peer eligible today, and trusted_but_not_sufficient cannot be observed against the production tracker until the redeploy lands. The allowlist path is unaffected and works against any tracker, deployed or not. RENTMYGPU_PROXY_OWN_IDENTITY_PATH exists for operators whose earn identity lives at a non-default path — pointing it at an absent file makes own_peer_id resolve to None, useful for running the proxy as a deliberate non-owner during testing.

Malformed tool calls are not errors

A turn that contains any tool call the parser refuses — most often a small model hallucinating a plausible name the client never declared (goose 1.49 declares no file-read tool at all, so Qwen3.5-9B reaching for read is expected) — still fails whole: no tool_calls delta and no tool_calls message field is emitted, and the well-formed subset is withheld too (a partial call set in an agentic loop is the silent difference spike 048 forbids). A call to a function you did not declare is never handed to your client, under any input.

What changed (debug session goose-unknown-function-fail-whole): the turn no longer ends as an in-stream server_error frame / HTTP 502. goose, and the OpenAI SDKs, treat any in-stream error object as a fatal provider error regardless of its type — goose retried the identical request three times and then aborted the session, discarding the prose the model had already produced. Now the turn ends as a normal completion:

• the prose the model produced is delivered (streamed as usual / message.content);
• finish_reason is "content_filter" — OpenAI's vocabulary for "content was omitted due to a flag from a filter", which is exactly what happened — or "length" when the call was cut off because the seeder hit max_tokens;
• a top-level rentmygpu.tool_call_rejection object on the terminal chunk (streaming) or the completion (non-streaming) names both the offending call and the tools you declared, so the client — or its model on the next turn — can recover:

"choices": [{"index": 0, "delta": {}, "finish_reason": "content_filter"}],
"usage": {"prompt_tokens": 0, "completion_tokens": 24, "total_tokens": 24},
"rentmygpu": {
  "tool_call_rejection": {
    "code": "malformed_tool_call",
    "message": "the model called `read`, which is not a declared tool. No tool call was emitted for this turn (fail-whole); the prose is intact. Declared tools: analyze, shell, write.",
    "malformed": [{"reason": "unknown_function", "name": "read"}],
    "declared_tools": ["analyze", "shell", "write"]
  }
}

finish_reason is deliberately never "stop" here: a client that ignores the rentmygpu extension must still be able to tell a refused call from a model that simply emitted none. Extra top-level keys are ignored by every OpenAI-compatible client we tested; the choices shape is untouched.

The proxy does not re-prompt the model on your behalf — it is stateless per request, a silent second generation would double your bill, and your conversation history would no longer match what the model actually saw. An agent that wants automatic self-correction can append a user message built from declared_tools ("read is not a tool; use shell") and re-request. The tokens of the refused turn are billed as usual (they were generated); the event is logged as proxy.tool_call_rejected.

7. Billing

Every completion settles exactly once, through the same /credits/settle path yaarbal spend already uses, signed by your spend identity — there is no second settle implementation anywhere in this codebase.

Own-node use is not free. Even when the proxy routes a completion to a GPU you yourself own, your spend wallet is debited tokens × weight (a tier table by declared parameter size — e.g. a 9B model is 5 credits/token) plus the flat per-settle network tax, and your earn wallet is credited the tokens×weight amount minus that network tax and any escrow hold. This is exactly how the network already prices every other spender; the proxy gets no exemption for talking to your own hardware.

Billed tokens are always engine tokens — the count the seeder itself reports — never SSE chunk counts (which can under-count by up to 50x on a chunked transport, spike 030). Billing is clamped to the frames the proxy actually decrypted, so a cancelled or dropped stream bills only what was delivered; zero frames delivered bills nothing.

Insufficient credits are refused with a 402 insufficient_credits before any prompt byte is rendered or sent — never a stream that starts and gets truncated. When a request IS affordable, max_tokens is silently clamped downward to what your balance can afford; the clamped value is reported in the X-RentMyGPU-Max-Tokens response header, and a completion that hits that clamp reports finish_reason: "length" rather than "stop". usage.prompt_tokens is always 0 in every response — prompt tokens bill zero credits on this network.

The unsettled record. ~/.rentmygpu/proxy-unsettled.jsonl is a durable, append-only JSONL log the proxy writes to before it retries a failed settle, so a witnessed debit is never lost even if the process crashes mid-retry. Each line has a status of pending (about to retry), settled (confirmed), or failed (retries exhausted, the tracker never confirmed this debit). On startup, yaarbal proxy scans this file and logs a proxy.unsettled_records warning if any request_id's latest status is pending or failed — a failed line means the proxy believes it owes a debit the tracker never acknowledged; there is no automatic reconciliation, the operator inspects the line by hand.

State the residual plainly: every settle retry re-mints a fresh signed envelope (new nonce, new timestamp) over the same request_id, relying on the tracker's in-memory settle dedup (credits_routes.py's _settle_dedup) for idempotency — there is no UNIQUE constraint on settlements.request_id at the database layer. The whole retry sequence is bounded by RENTMYGPU_PROXY_SETTLE_MAX_SPAN (default 30s), deliberately kept below the tracker's 35s --graceful-timeout (deploy/Dockerfile), so that a tracker restart landing inside an in-flight retry window is unlikely — but not impossible. If it happens, the same completion could be settled twice: real money, a small window, disclosed here rather than hidden. A database-level UNIQUE constraint on settlements.request_id would close this structurally; it is a separate future item, out of this phase's scope.

8. Environment knobs

Every RENTMYGPU_PROXY_* name declared in rentmygpu/proxy/config.py:

9. Limitations (this phase)

• Tool-carrying requests need the model's GGUF present locally. The tool-call convention is read from that specific model's own tokenizer.chat_template, never assumed — without the file pulled (yaarbal models pull <id>), a tools request is refused chat_template_unavailable. Plain chat to a model you have not pulled still works: the proxy falls back to the seeder-advertised chat_dialect renderer (the same bytes yaarbal spend --chat sends), on which the thinking prefill from §5 does not apply.
• Sampling parameters — temperature, top_p, stop, n, parallel_tool_calls — are accepted on the wire and silently ignored, because the seeder's /infer takes none of them.
• The effective context is now bounded by the serving seeder's advertised n_ctx, not a fixed constant: the proxy derives both its history-trim budget and its context_length_exceeded refusal cap from whatever window that peer announces (Phase 26 CTX-02/CTX-03). A legacy seeder that advertises nothing is treated as 4096 for trimming purposes and gets today's RENTMYGPU_PROXY_MAX_PROMPT_CHARS (32,000 chars) refusal floor. See QUICKSTART-CLI.md's "Context window and prompt cap" subsection for raising n_ctx on a seeder you operate yourself.
• The settle/submit rate tier for an agentic loop is 60/minute per verified identity (RENTMYGPU_RATE_AUTH_SPEND_PER_MIN, applied to POST /credits/settle and POST /jobs/submit only), with a 20x per-IP backstop (1,200/min per IP). The unedited RATE_DEFAULT_PER_MIN (10/min) still governs /credits/faucet, /credits/benchmark-claim, and /jobs/<id>/cancel. See ABUSE-POSTURE.md §5 (internal reference) for the full two-counter design.
• Multi-turn agent-matrix validation (driving Cline/aider through a real multi-step task end to end) is Phase 27 — this phase's automated coverage stops at the single-turn and two-turn wire-shape contract.
• Only the Qwen3.5 tool-call convention is parsed. Other model families' tool-call syntax (PROX-08) is deferred beyond v1.4.

10. Disclosures

• Prompts sent to a seeder are visible to that seeder's owner.
• The network tax applies to your own seeder's earnings too — even chatting against your own GPU is not exempt.
• Rate limits apply to all requests.
• Credits and access may reset during the beta.
• Terms of Service: https://rmg-web.yaarbal.app/tos