← All briefings
Enterprise AISpeech AIPilot evaluation

China Speech AI: Follow the Final Transcript into the Draft

The final caption may be correct while the saved draft still uses provisional text. Test transcript revisions, missing final events and the record that receives them.

China AI Tour Research5 min read
Conceptual black-and-orange illustration of a headset surrounding audio waveform cards, including highlighted and faded recordings.
Conceptual black-and-orange illustration of a headset surrounding audio waveform cards, including highlighted and faded recordings.

A speech demo ends with the correct transcript on screen. The draft record created earlier in the session still contains the first, provisional wording. To evaluate that workflow, follow the transcript revision into the receiving application instead of checking only the last sentence displayed.

During a China speech-AI evaluation, use an approved synthetic recording and a test destination. Agree which transcript state may create or update a draft. A live caption, a final transcript segment and a person-confirmed business field are different checkpoints; the demonstration should identify which one the receiving workflow uses.

Define a revision before testing recognition

Start with a controlled event fixture. The intended utterance is “Prepare a draft request for fifty units.” Give it a session identifier and a segment identifier. In the fixture, the first event proposes “fifteen units” as provisional text; the second replaces it with “fifty units” and marks the segment final.

These events are deliberately supplied to test the receiving application. They are not a claim about how often a speech model makes this error. Test recognition quality separately using the actual recording and an agreed reference transcript.

For this exercise, choose a simple destination rule: provisional events may update a preview, while a final segment may populate one draft field. Repeating an event must not create another draft, and an older provisional revision must not overwrite the finalized value. The workflow owner should map this rule to the service's documented event semantics before running the fixture.

Delivered eventExpected previewExpected draft quantity
Segment S1, revision 1, provisional: fifteenShows provisional fifteenUnset
Segment S1, revision 2, final: fiftyShows final fifty50
Exact repeat of revision 2Still final fiftyStill 50; no second draft
Delayed repeat of revision 1Must not replace the accepted final stateStill 50

Record the destination's stored values after each step. A screen that briefly shows the right number is insufficient if the export or saved draft retains the earlier one.

Keep segment identity and revision identity separate

Ask the host to show how the receiver connects an update to the segment it replaces. A session can contain multiple segments, so the session identifier alone does not establish which text changed. Likewise, the segment identifier alone does not distinguish an old revision from its replacement.

The fixture uses explicit revision numbers for clarity. An actual service may express replacement through another documented identifier or event relationship. The evaluator needs a traceable ordering and replacement rule, not a particular field name invented for the demo.

Retain the original events, the receiver's accepted state and the reason it ignored or held an event. A duplicate may remain in an audit log without becoming another business record. If the destination result is uncertain after a timeout, use the agent-timeout guide to distinguish checking the existing draft from blindly creating another one.

Close the stream without completing the segment

Run another fixture in which the provisional event arrives but the final event does not. Then close the test connection. Under the agreed rule, the draft quantity remains unset and the segment remains unresolved. The receiver should expose that state for the designated recovery or review step.

A connection ending is not, by itself, the final-segment event required by this fixture. Ask what the real integration treats as evidence of completion and which service guarantees support that interpretation. Record whether a reconnect recovers the missing segment, returns a repeated event or leaves the result unknown.

Also include a second segment with its own identifier and a different quantity. Confirm that a late revision of S1 does not alter the second segment's field. This tests the replacement boundary as well as the handling of old events.

Inspect the record that people will use

After the final event, export the draft or read back the receiving test record. Compare its quantity, segment reference and accepted transcript version with the event trace. If a person corrects the field, preserve that correction separately so an automated replay cannot silently replace the approved value.

For a pilot report, separate final-transcript correctness from downstream-field correctness. Count segments that remained provisional, segments whose final event was received, final values that reached the destination correctly, and duplicate draft records. Record the time at which the usable draft became available, rather than treating the first visible caption as the finished result.

The contact-centre review guide covers which recordings enter an evaluation and how unscorable cases remain visible. This exercise adds a different check: which version of the transcript actually reached the workflow.

NIST AI RMF 1.0 (2023), MEASURE 2.1 calls for documenting test sets, metrics and evaluation tools. Keep the event fixtures and destination readbacks together so the team can repeat the same paths after an integration change.

Leave the demonstration with the final transcript, the saved draft and the revision trace linking them. That evidence shows whether a correct last caption became the correct operational input.