Recommendation quality
How closely do recommendations agree with independent agronomist and extension-worker judgement on the same cases?
Home / Benchmark
Benchmark the AI Extension Worker.
We invite researchers, institutions and practitioners to evaluate our model independently. Tell us what you want to test and how, and we will agree a fair protocol with you.
An independent view of what matters.
These are the questions we most want tested by people other than ourselves. You can propose others.
Calibration
Does stated confidence match observed outcomes, for yield, marketable life and likely diagnoses?
Uncertainty and referral
Does the model say when evidence is thin, and does it send uncertain or high-stakes cases to a person?
Low-connectivity robustness
Does work captured offline stay intact and sync without duplicates when the network returns?
Crop coverage
How does behaviour differ across cassava, tomato and maize, and across savanna, coastal and delta conditions?
Stated limits
Does the model keep to its limits: no guaranteed yields, no food-safety certification, no guesses presented as certainty?
Sketch your evaluation
Start with the question you want answered.
Pick a question to see how the comparison could look and what we would agree with you. It is a planning aid: nothing is stored or sent.
Recommendation quality
How closely do recommendations agree with independent agronomist and extension-worker judgement on the same cases?
- Compare
- Recommendations with independent expert judgement on the same cases.
- Agree together
- The set of cases, who the reviewers are and how agreement is measured.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
Calibration
Does stated confidence match observed outcomes?
- Compare
- Stated confidence with observed outcomes for yield, marketable life and likely diagnoses.
- Agree together
- Where the outcomes come from and when they are measured.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
Uncertainty and referral
Does the model say when evidence is thin, and send uncertain or high-stakes cases to a person?
- Compare
- Whether thin evidence is flagged and whether uncertain or high-stakes cases are referred.
- Agree together
- Which cases count as uncertain or high-stakes.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
Low-connectivity robustness
Does work captured offline stay intact and sync without duplicates?
- Compare
- Work captured offline with what arrives once the network returns.
- Agree together
- The network conditions to test and how duplicates are counted.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
Crop coverage
How does behaviour differ across cassava, tomato and maize, and across savanna, coastal and delta conditions?
- Compare
- Behaviour across the three crops and the three kinds of conditions.
- Agree together
- Which crops and regions are in scope for your study.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
Stated limits
Does the model keep to its limits: no guaranteed yields, no food-safety certification, no guesses presented as certainty?
- Compare
- Answers against the stated limits when you try to push past them.
- Agree together
- The prompts and cases that probe each limit.
- Always agreed first
- Scope, metrics, data handling and the access you need, written down before anything starts.
From request to results.
-
01
Tell us what you want to evaluate
Describe the question, the crops and regions, and the method you would use, using the request form below.
-
02
Agree a protocol together
We settle scope, metrics, data handling and the access you need, and we write it down before anything starts.
-
03
Receive access
We arrange access to the model in the way that suits the evaluation. The arrangement is agreed per study.
-
04
Run your evaluation
You run your own tests with your own methods. We answer questions about the model but do not steer the result.
-
05
Review the results with us
You share results with us first so factual errors can be corrected. Your findings remain yours to report.
Who does what
You run the evaluation. We stay out of the result.
We answer questions about the model but do not steer the result, and your findings remain yours to report.
The ground rules.
These are the working terms we propose. We agree the final ones with each requester.
Transparent method
Describe how you tested, so others can repeat or challenge your result.
Share results with us
Send us your results, including the cases where the model did badly, so we can learn from them.
Responsible use of data
Use any farm or personal data only as agreed, and do not attempt to identify individuals.
No implied endorsement
Do not present an evaluation as an endorsement by Openfarms Lab, or as an endorsement of Openfarms Lab, without agreement.
Tell us what you want to test.
We review every request. We cannot promise a response time, and a request does not guarantee access.
Prefer email? Write to hello@openfarmslab.com with “Benchmark” in the subject.
Before you request.
Is there a public benchmark or dataset?
Not at this stage. Evaluations are arranged one at a time, so that each protocol fits the question being asked.
Who can request a benchmark?
Researchers, universities, institutes, extension services and practitioners. We prioritise requests that can be tested against real field outcomes.
Does it cost anything?
We do not charge for independent evaluation as a matter of principle. Any cost that arises, such as field data collection, is discussed in the protocol.
Can I benchmark with my own data?
Yes, if the data are handled in line with our privacy notice and the protocol we agree together.
Prefer to start with the research?
Read how we evaluate advice, from the decision a farmer faces to the outcome we measure.