China AI Vision Demos: Test a Production Changeover
A vision-inspection demonstration can look strong on one product. A controlled changeover reveals whether thresholds, labels and operators travel with the model.
Direct answer. Ask the host to compare two defined product batches and explain every configuration change between them. The important question is not whether the system detects an obvious defect; it is what must change before a new product is released safely.
Define the change before discussing accuracy
On a China AI factory visit, distinguish a new color from a new material, package shape or defect definition. These changes create different evaluation problems. A cosmetic mark accepted on one finish may be unacceptable on another. Without the factory's labeling policy, an accuracy percentage has no stable business meaning.
Request representative, permitted samples from the original and changed batches. Include accepted variation, confirmed defects and borderline cases. Keep the sample origins visible to the evaluator but avoid supplying sensitive customer drawings. A host-selected demonstration remains a demonstration, not an independent validation.
Separate four kinds of change
| Layer | Ask the host to show | Why it matters |
|---|---|---|
| Imaging | Lighting, position and exposure settings | The input may change before the model does |
| Labels | The written acceptance boundary | Inspectors may disagree about the target |
| Model | Version and threshold used for each batch | A silent configuration swap hides transfer limits |
| Release | Who approves production use | A correct prediction is not a release authorization |
Ask how false rejects affect reinspection workload and how missed defects are detected later. These are separate consequences. A single overall success rate can conceal an unacceptable trade-off between them. If the sample is too small to estimate either rate, record the demonstration outcome without inventing a percentage.
Our changeover worksheet applies the context and generalization questions in NIST AI RMF Core. The framework supports documenting testing conditions and limits; it does not establish a factory's acceptable defect rate.
Make the follow-up request actionable
Ask for a redacted changeover record: initial settings, sample counts, disputed labels, rejected configurations and final approval. A supplier unable to share production data may still explain the procedure and demonstrate it with synthetic samples. Lack of access is a limit on your conclusion, not evidence that the system failed.
For your own pilot, use two explicitly scoped product families, independent label adjudication and a pre-agreed release owner. Connect these questions to the smart-factory evidence guide before planning the visit.