China AI Pilots: Size the Human Review Queue
Evaluate an enterprise AI pilot with review-arrival counts, usable reviewer time and two worked examples that expose both total-capacity gaps and burst-time delays.
An AI demonstration can finish quickly because a reviewer is waiting beside the presenter. An operating workflow has to share people's time with other work. When several cases arrive together, the review queue can determine how long the customer or employee waits.
For a team evaluating enterprise AI during a China visit, ask the host to show the interval between a case entering human review and an authorized person completing it. Then compare incoming review work with the time that qualified reviewers actually have available.
Define the review job before counting capacity
Separate the kinds of decisions sent to people. Checking a low-confidence transcription, deciding a policy exception and approving an external commitment may require different authority and skills. Capacity in one category cannot automatically cover another.
GOVERN 3.2 in NIST's AI Risk Management Framework 1.0 (2023) addresses defined roles and responsibilities for human-AI configurations and oversight. Use that allocation of responsibility as a starting point. The capacity worksheet here is a proposed operating exercise with assumed figures.
For each review category, name the decision, the role permitted to make it and what counts as a completed review. A case sent back for missing information is still unresolved. Preserve its identity when it re-enters the queue so repeated attempts do not inflate the completion count.
Measure arrivals, available time and waiting separately
| Record | What the pilot team should capture | Why it matters |
|---|---|---|
| Review arrivals | Cases entering each review category in a defined interval | A daily total can hide a short surge |
| Active review effort | Time spent reading, checking, deciding and recording the outcome | A quick approval click can omit most of the work |
| Available review time | Qualified reviewer time allocated to that queue, excluding other duties | A name on the roster does not establish usable capacity |
| Waiting and completion | Entry, first-work and resolution timestamps, plus age of open cases | Fast handling after pickup can coexist with a long wait |
| Returns and reassignment | Original case ID, reason and next authorized owner | Repeated routing can consume time without resolving the case |
Use a controlled simulation or aggregate records that the host is authorized to share. During the pilot, retain a count and the age of unresolved cases at the end of each observation window. Looking only at completed cases loses the work still waiting.
Worked example: the four-hour total does not fit
Illustrative planning exercise. All quantities and durations below are assumptions.
Suppose a workflow receives 120 items over four hours and sends 20% to human review. That creates 24 review cases. Assume every case takes exactly five active minutes for a qualified reviewer to finish, including its decision record.
The review demand is 24 × 5 = 120 reviewer-minutes. If the team allocates only 90 usable reviewer-minutes during that period, the idealized capacity is 90 / 5 = 18 complete reviews. Even with all that time used productively, six cases remain, representing 30 minutes of unfinished review work.
Check the allocation against the actual rota. Time assigned to another approval queue, a meeting or a different service cannot also be counted as available for this one. In a real pilot, record the range of review durations and the reasons for long cases; a single average can conceal work that needs a specialist.
Test a burst even when the total appears sufficient
Now suppose the team increases its allocation to 120 reviewer-minutes, enough for the assumed four-hour workload in total. That still leaves a timing question.
In a separate burst test, place 12 of the five-minute cases in the queue at the same instant. Give one authorized reviewer uninterrupted time, with no parallel processing or handoff overhead. The last case cannot finish before 12 × 5 = 60 minutes. A proposed 15-minute completion target would therefore fail for some of the cases even under those favorable assumptions.
Agree whether the target concerns first response, the start of review or final resolution. Record waiting time and active handling time separately. Increasing a four-hour allocation does not establish that the right reviewer will be present during the interval when the cases arrive.
Agree what happens when the queue grows
Before expanding the pilot, name the owner of the oldest unresolved cases and the trigger for additional authorized coverage or reduced intake. Keep the required approval in place when capacity is exhausted. An overload response that silently skips review changes the control being evaluated.
Compare observed arrivals, usable reviewer time and case age for both ordinary and busy periods. Leave the discussion with a specific unresolved capacity question: which review category lacks coverage, during which interval, and what record would demonstrate that the gap has closed?
Use the AI governance field guide for the wider ownership and change-control discussion. Bring the queue worksheet to an industry-focused China AI program so the demonstration can be scoped around the workload and review roles your team needs to evaluate.