Service-quality analysis: A sudden change in an outsourced support team's quality score deserves an explanation before it becomes a story about better or worse work. The scoring process may have changed alongside the conversations being scored.
In its release notes through October 23, edited October 26, Zendesk says it plans to update the AI model powering AutoQA on November 11, 2026. The company warns that models can interpret prompt instructions differently, potentially affecting category accuracy scores and overall ratings. Organizations using custom prompts may need to adjust them. November 11 is a future scheduled change, not a completed rollout as of this article.
A score needs a measurement history
For a business buying outsourced customer service, that announcement raises a practical reporting question. If the score moves across a model update, how much of the movement reflects the service and how much might reflect the measurement process?
A provider's report should identify the conversations reviewed, the period covered and the scoring method. Ask whether the score came from automated assessment, manually submitted reviews or a combination. The label “quality” is too broad to answer those questions by itself.
Keep a simple change record beside the regular report. Include relevant scorecard edits, prompt revisions and announced model changes, with their effective dates. This is an operating recommendation for interpreting results, not a claim that every system automatically provides a complete historical record.
Compare the same work where possible
A useful review begins with examples that both the business and provider can inspect. Choose a manageable set of conversations and record why they were included. Include ordinary work as well as cases that prompted disagreement. Avoid presenting a handful of examples as a representative estimate of the whole operation.
If a score changes, examine the underlying judgment. Did the conversation omit information? Did a reviewer apply a different definition? Did the updated prompt interpret an instruction differently? Those explanations lead to different actions, so they should remain separate in the discussion.
Do not promise a clean before-and-after experiment unless the necessary system access actually exists. A team may be unable to rerun the earlier model. In that case, preserve the earlier results and explain the comparison's limits instead of reconstructing a result that was never measured.
Use calibration for human agreement
Zendesk's calibration guidance describes reviewers rating the same conversations and comparing their scores and comments. It also says calibration reviews do not change the workspace's existing Internal Quality Score and do not trigger reviewer notifications. A manager therefore still needs to organize the session and explain its purpose.
For an outsourced team, invite the people who own the service standard on both sides. Discuss the reason behind a rating, not just the number. Agreement about an example is more useful when it clarifies what the next reviewer should look for in another conversation.
That process can reveal unclear instructions as well as differences in judgment. If the standard itself needs revision, record the decision and its start date. Otherwise, later reports may compare results produced under definitions that quietly changed.
Keep automated and manual evidence distinct
The Zendesk QA Reviews dashboard guide describes manually submitted reviews and their quality indicators. Its scope is a reminder to check what a dashboard contains before treating one chart as a complete account of automated and human assessment.
Ask for enough context to follow a result back to the work it describes. A report can state that a model change occurred, identify the affected period and separate confirmed observations from possible explanations. It does not need to pretend that every movement has already been explained.
The vendor notice does not predict the size or direction of a particular provider's score change. The useful response is preparation: preserve the measurement history, keep review responsibilities clear and investigate surprising results before drawing conclusions about service quality.
AI-generated conceptual editorial image; the people depicted are not Zendesk employees or an actual service-review team.