Methodology
How Chrio measures detection accuracy.
98.67% is the most repeated number on this site. This page exists so it can be checked rather than taken on trust: what it measures, the five model versions behind it, what the model still gets wrong, and one older public figure that does not agree with it.
The figure
What the 98.67% figure measures.
98.67% is detection accuracy on cricket gameplay footage, produced by model version 5, September 2025.
The precise metric definition, whether that is mean average precision, precision, recall or classification accuracy, and against which held-out split, is being prepared for publication and is not stated here yet. Naming a metric Chrio has not published would be worse than the gap: an unverifiable definition is harder to correct than a missing one.
What can be stated now is the data behind it. The training set is 30 TB of footage from PSL, National T20 and Tri-Nation cricket. The validation set is more than 10,000 deliveries of professional match footage. Both are broadcast footage rather than staged capture, which is the condition the model has to work under.
30 TB
training footage: PSL, National T20, Tri-Nation
10,000+
deliveries of professional match footage in the validation set
V5
current production model, September 2025
Version history
Five models over twelve months.
Detection accuracy went from 20.95% in the first version, September 2024, to 98.67% in version five, September 2025, which is 4.7× the original. Each step is dated so the trajectory can be read rather than inferred from its endpoints.
| Version | Date | Accuracy | Change |
|---|---|---|---|
| V1 | Sept 2024 | 20.95% | – |
| V2 | Dec 2024 | 24.02% | +3.07 pts |
| V3 | Feb 2025 | 35.65% | +11.63 pts |
| V4 | Mar 2025 | 39.87% | +4.22 pts |
| V5 | Sept 2025 · current | 98.67% | +58.80 pts |
The largest jump is between V4 and V5, and it is the row a technical reader should question first. The engineering account of what changed in that release is being prepared for publication alongside the metric definition. Six months separate the two versions, and the training set reached its full 30 TB in that window.
Limits
What the model gets wrong.
A number this high is only useful alongside the conditions it depends on. These are the four that change the answer.
-
It needs a clean feed
A clean SDI (Serial Digital Interface) or NDI (Network Device Interface) feed of the camera behind the bowler's arm. Heavily compressed or upscaled sources degrade detection, and the figure above does not describe them.
-
It needs calibration against the ground
Most of the setup hour is calibrating against the pitch markings. Move the camera and it needs redoing.
-
It does not run unattended
Every call is reviewed by the operator on the video manager before release. Detection accuracy is not the same thing as an unsupervised call being right.
-
It does not adjudicate
This figure describes detection, not officiating. Decision review is CritDRS, which is in development and has not been used to adjudicate a competitive fixture.
One older figure
The 99.5% you may find on Roboflow is an earlier benchmark.
A public Roboflow Universe project owned by Chrio publishes a different number: 99.5% mAP@50 on 1,047 images, with 100.0% precision and 100.0% recall. It is real, it is ours, and it is easy to find because it ranks for the CritShot name.
It is not the figure to use. It is an early single-class benchmark on a set roughly a thousandth the size of the current training data, and precision and recall of exactly 100.0% on a set that small is a sign of overfitting rather than of quality. The V5 model supersedes it, and 98.67% on 10,000-plus deliveries of professional match footage is both the lower number and the more meaningful one.
This paragraph exists because two public accuracy figures that disagree, with nothing reconciling them, is worse for a reader than one figure with a caveat.
Ask about the measurement.
If you want the test-set composition, the failure cases or the confusion matrix, ask: the call is with the engineer who trained the model.