Evaluator + optimizer · interactive example
Could your agent work better?
Test the change. Check the cost.
Compare what the agent does today with a proposed change. See mistakes, changed tool paths and potential cost differences before you decide what to release.
Synthetic demonstration. No customer connection, no external model call, no automatic changes. Use anonymized or synthetic inputs only—never credentials, customer records or proprietary source code.
1 · Your agent / test harnessCapture inputs, expected answers, actual outputs, tool steps and usage.
2 · Forge evaluator + optimizerCompare labeled results and action paths. Re-price usage to identify options to test.
3 · Your release + approval processYour team reviews the evidence. Execution, policy and storage stay under your authority.
Start with a worked example
A payment agent missed a review requirement. Try a fix, a cheaper-but-riskier change, or a hidden extra action. These are supplied synthetic outputs, not calls to a real payment system.
Describe your workflow, or paste measured results
A description produces a measurement plan, not a quality score. For a computed comparison, supply outputs from the same test cases and the labels you expect.
Editable sample / paste workflow JSON
Each case contains expected.action, baseline and candidate output.action, and ordered trajectory steps. Add token usage for the cost illustration. Maximum 30 cases.
Computed result · nothing applied
What to improve or investigate
Where this plugs in
Inspect evaluator, optimizer and price evidence
Different jobs. One improvement loop.
Evaluator: checks outputs against your labeled answers and schema, compares baseline with candidate, and flags changed action paths. It cannot make a weak test suite representative.
Optimizer: calculates advisory model-swap and cache options from token usage. Here the unsigned synthetic rates keep swaps NOT_CLEARED, even when a candidate passes. Real savings require qualified prices, measured candidate usage and accepted outcomes.
Decision accountability: supports the consequential action in your qualified integration. This lab does not replace the signed decision/replay demonstration or claim to prove a whole deployment.