Assurance review survey at risk of fraud and skewing data

Failure to meet best practice market research guidelines has left a vital survey into the future of farm assurance at risk of fraud and collecting skewed data.

The survey allows individuals to submit multiple responses, does not protect against non-human responses, and does not verify whether respondents are real farmers.

See more: Independence of Red Tractor review under threat

It is being carried out by Promar with input from Supply Chain InSites (SCI) as part of the wide-ranging review into assurance commissioned by the NFU and AHDB.

The quality of data collection has also been questioned, with respondents from mixed farms unable to differentiate which elements of assurance they may or may not be happy with.

For example, feedback submitted by an assured cereals farmer with non-assured sheep will be applied to both sectors, skewing the data.

Staffordshire arable farmer and founder-owner of The Farming Forum, Clive Bailye, who has years of IT experience, said: “I don’t believe it’s possible for the survey to generate quality feedback.

“The ability to submit multiple responses is crazy. Monitoring an IP address is no defence for that, as it’s very easily got around.

“At the very least, an email address should have been requested to ensure responses are genuine. That’s a bare minimum.”

Other issues, such as a failure to randomise response options to prevent bias on lists, statements and requests for positive and negative feedback appear to have been adjusted since Farmers Weekly contacted Promar.

But concerns around the use of subjective metrics (descriptive words) instead of objective ones (a scale of 1-5) remain unresolved.

Grounded Research director Clare Otridge said the agricultural sector “lags behind others” in safeguarding against data and response fraud.

To maintain the trust of farmers, she called on the industry to implement enhanced security measures and avoid assumptive questioning in data collection.

“Only by adopting these best practices can we ensure that the voice of the farming community is accurately represented, particularly on such crucial issues that will affect the life and livelihood of so many,” she said.

In a statement, Promar said it was confident the survey would “elicit valuable insight for the industry” and added it had already addressed some of the points raised by Farmers Weekly.

An FAQ section on the website provides answers to common questions and explains the security measures already in place.

It also sets out the reasoning behind a decision to allow anonymous responses – in order to encourage people to fill in the survey.