Recorded conversations, mock business tools and inspectable outcomes. A small exploratory comparison of configured voice systems.
Open-source framework · Written report · Data · File hashes
A blocked attempt can be followed by a correct saved order. A strict state failure can also come from the simulated caller or a grading rule. These flags are not an overall model verdict.
S027: saved after a blocked attempt · S033: lookup was available but skipped · Read the audit · Captured tool declarations and prompts
State checks include the saved result, required action sequence and message count. Policy checks record blocked prohibited actions. Neither proves spoken consent, truthful communication or a valid simulated caller. Personal listening scores and verified response latency are separate.
| Language | Target | Audio | Original state pass | With erratum | Original state fail | Attempt checks pass | Attempt checks fail |
|---|
USD components below are estimates and posted carrier charges, not full invoices. Each audio statistic gives calls equal weight. Energy gaps are exploratory measurements; acknowledgments may count as responses. Active dBFS is not LUFS or speaking speed. Download all trial rows as CSV.
| Language / Target | Target model USD | Simulator USD | Carrier USD | ASR USD | Energy gap ms | Active dBFS | Duration s |
|---|
An asterisk marks the declared correction for an empty optional stock-request order reference. Original grades remain available in each sample; no other checks are relaxed.
Model names are visible here. Use the separate local listening study to save a personal score before revealing labels.
| Sample | Language | Target / Caller | Case | State checks | Attempt flags | Caller review |
|---|
Download original WAV · Original level and speed. Channel 0: caller; channel 1: target.
ASR can omit or invent speech. Compare the audio and native transcripts before attributing a mistake.
Second machine transcript for every sample. Original ASR remains above. Neither transcript replaces listening.
Energy-based gaps are proxies, not validated response latency. Active RMS is not LUFS or speaking speed.
These are the SDK-built requested declarations. The common-contract check does not prove identical provider enforcement or optimal prompts. The raw forms and documented differences are preserved.