ratemy.sh

ratemy.sh · evidence-backed release review

Writing it was the easy part.

AI builds your product in a weekend. Nobody tells you whether you'd dare put real users, real data, or real money behind it. These five skills do — each auditing one layer of what you just built, all sharing one evidence contract and one willingness to say NOT READY.

Audit · R-014 target: real-money ref: sha256:9f3c…e2a1

Issue list

BLOCKER · F-002Cancellation does not stop tool calls — the agent keeps spending money after the user presses Stop.

HIGH · F-005Retry re-executes a non-idempotent charge — one approval can bill twice.

To verify

UNKNOWN · U-003Tenant isolation has not been exercised — one user's data may reach another.

Evidence lanes

deterministic-checksFAIL
critical-journey-e2ePASS
probabilistic-evalUNVERIFIED
continuous-evidenceN/A — no deployment yet
Maximum safe target: internal-demo
Blocking gates: duplicate-real-charge, runaway-execution
NOT READY

Why they say no when others say yes

The review is a contract, not a vibe.

All five share these clauses. They are the reason a verdict here is worth reading.

§ 1

No score averages away a veto.

Hard gates — authority bypass, cross-boundary leakage, irreversible data loss, duplicate real charges, fabricated completion — block the release regardless of the weighted average. Accepting the risk doesn't turn a gate into a pass.

§ 2

Evidence has states.

Every claim is labeled E3 reproduced, E2 instrumented, E1 static, or E0 unverified. A reachable code defect is real, but its runtime consequence stays labeled as inference until it's exercised.

§ 3

A plausible diff is not a fix.

Findings advance to verified-fixed only on fresh passing evidence from an independent retest context — which also audits the fix code itself. The pass that wrote the fix never grades its own work.

§ 4

Green CI proves structure, not behavior.

Repository checks and schema validation cannot masquerade as a critical-journey run. Public or privileged targets require fresh runtime evidence on the exact build under review — a mutable latest is not an identity.

Which one

Pick by what you're about to ship.

"I vibe-coded an app and I want to charge for it."

rate-my-code at the real-stakes level. It verifies payment invariants, retries, authorization, recovery, and monitoring before you take a cent.

"The API is mine and it's about to face real traffic."

rate-my-server. Tenant isolation, transactions, idempotency, backpressure, migrations, and whether a restore actually restores.

"I built an agent that can touch real accounts."

rate-my-agent for what it decides, rate-my-harness for whether the runtime can stop it. Both, if the answer matters.

"I wrote a Skill and I want to publish it."

rate-my-skill. Trigger precision, with-skill versus without-skill uplift, injection resistance, and package integrity.