You built an agent. How do you know it actually works before it ships? This track answers that question in five parts — batch testing, utterance design, response scoring, regression in CI and red-teaming — each anchored in a real project scenario.
The series
Each part is a standalone build, but they stack: the test set from Part 1 becomes the regression baseline in Part 4 and the attack catalog in Part 5. Read them in sequence the first time.
Build a test set from 200 real customer questions, run it as a batch and read the results — how a retail bank turned "the demo works" into a go-live gate.
Read tutorialParaphrases, edge cases, adversarial phrasing and multilingual inputs — a telco tests "why is my bill high" fifteen ways and finds three real bugs.
Read tutorialAccuracy, completeness, tone and hallucination gates — a healthcare provider builds a calibrated LLM judge in Apex with the Models API.
Read tutorialOne instruction tweak silently broke refund routing. Baseline your test set and rerun it on every change with sf agent test in GitHub Actions.
Read tutorialPrompt injection, off-topic traps and data leakage — an insurer attacks its own agent and finds the bug that instructions alone would never stop.
Read tutorialHow it fits together
Every layer exists because a demo can't catch a specific class of failure. Here's the map.
| Layer | The failure it catches | Scenario in the series |
|---|---|---|
| Batch testing | Real customer questions landing on the wrong topic or action. | A retail bank finds 29 of 200 real questions misroute — before go-live, not after. |
| Utterance design | The agent handling one phrasing of an intent and failing fourteen others. | A telco's "you've overcharged me" routes to the wrong topic while the polite version passes. |
| Response evaluation | Right topic, wrong answer: invented facts, missing caveats, off tone. | A healthcare agent quotes a fasting window its own Knowledge article contradicts. |
| Regression & CI | A one-sentence instruction edit silently changing behavior everywhere. | An e-commerce refund flow breaks for two weeks with zero errors logged. |
| Red-teaming | Hostile input extracting data or actions the user isn't entitled to. | An insurer's agent returns another customer's policy excess — until the action enforces access in code. |
Keep going
Testing proves behavior; these posts are where the behavior gets built — and hardened.
New to the platform itself? Start here before the testing track — builder, topics, actions and where a developer fits in.
Read tutorialTyped inputs, validation and confirmation in @InvocableMethod actions — the enforcement layer your red-team tests will probe.
Read tutorialDeterministic transitions for the flows where improvisation is unacceptable — policy as code, conversation as LLM.
Read tutorial