China AI Vision: Does Each Result Stay with the Right Part?
Reverse two inspection responses and trace the destination records. Test whether delayed, repeated or unmatched results can change the wrong part disposition.
A vision model can classify two inspection images correctly while the receiving application attaches each result to the wrong part. During a China manufacturing AI evaluation, ask the team to demonstrate how a result retains its part identity when responses arrive in a different order from the requests.
Use an offline test harness with synthetic part records and saved images. The useful evidence is a trace from the part record to the image, inference request, returned result and proposed disposition. A green indicator on a screen leaves most of that chain untested.
Give two parts different expected outcomes
Create synthetic parts A and B. Under the agreed inspection policy, A should be held and B should pass. Establish those reference outcomes before the exercise. Record the image associated with each part and give each inference request a distinct identifier. If images are copied or renamed, retain a stable image identifier or byte hash so the evaluator can check which image was actually submitted.
First run the ordinary sequence. Submit A and B, deliver their results in that order, and confirm that the receiving application proposes hold for A and pass for B. Preserve the input and output records. This establishes that the fixture can produce and receive both outcomes.
For the next run, keep the candidate results unchanged. Delay A's response and deliver B's first. The exercise tests the association between results and parts, so it can use recorded model responses with fixed outcomes. That arrangement does not measure classifier accuracy.
Reverse delivery order and inspect the destination records
| Delivery step in the reversed run | Required observation in the test records |
|---|---|
| B's pass result arrives first | B receives the pass result. A remains pending |
| A's hold result arrives later | A receives the hold result. B retains its pass result |
| Both responses have arrived | Each result is traceable to its original request, image and part |
Check the persisted destination records, not just the order of messages on the demonstration screen. An implementation that assigns the first returned result to the first waiting part can display both valid model outputs and still make the wrong disposition proposal for each part.
Ask the operator to explain which identifier the receiver uses for the match. A batch number shared by both parts cannot distinguish them. A timestamp may help reconstruct events, but it does not replace a documented association when several requests are in flight. Keep the receiving application's own record of the match, including the request identifier it accepted.
Test an unmatchable result and a repeated result
In the same offline fixture, deliver a result with a request identifier that does not correspond to any pending inspection. Agree beforehand that the receiver must leave it unmatched for investigation and must not assign it to whichever part is waiting. Verify that neither A nor B acquires a new disposition from that message.
Then deliver an exact duplicate of B's earlier response. Under the proposed rule for this exercise, a repeated response must not create a second disposition event or replace A's result. The receiver may retain the duplicate in its audit log. Separate that log entry from a new business action.
Finally, create a new inspection attempt for A and delay its response. Deliver the old attempt's response again. The team should show how the receiver distinguishes the old attempt from the current one, even though the part ID is the same. Record whether the old result was rejected, quarantined or retained as history under the agreed workflow. It must not silently satisfy the pending new inspection.
Take the evidence to the physical handoff
This fixture can establish that the software preserves the intended associations for the paths tested. For a production pilot, the factory team also needs evidence that the tracked part remains the same physical item through capture, movement and disposition. Ask for an authorized recorded trace or a host-controlled test-cell walkthrough covering that handoff. A correct database match alone cannot establish where the item actually went.
Keep two acceptance questions separate. The vision error-rate guide examines whether the classifier makes the right decision. This exercise examines whether that decision reaches the right part and inspection attempt. Both matter before a model score can support a release process.
NIST AI RMF 1.0 (2023), MEASURE 2.3 calls for documented performance evaluation under conditions similar to deployment. Ask the supplier to include concurrent requests, delayed delivery and repeated messages when those conditions occur in the intended workflow. Leave with the exact paths demonstrated, the destination records, and the unresolved physical-tracking checks.