The power of data analytics in improving decision making about future land use is being highlighted by a project at AHDB’s Strategic Farm East to use data to identify marginal land and land most suitable for environmental schemes.

The project, led by David Clarke, soils and farming systems technician at Niab, has used 10 years of yield data together with operation and input costs to create margin maps across 36 fields on E J Barker and Sons’ Lodge Farm near Stowmarket in Suffolk.

See also: Advice on clamping sugar beet for optimal harvest storage

“That equates to more than 1.4m data points, which is bigger than an Excel spreadsheet can take. The challenge is to make sense and implement management decisions from it.”

To do so, he created a five-step process as part of his PhD project.

1. Clean yield data

There will always be inconsistencies in yield data, whether it is from the header not cutting at full width, combine changing speed, or turning on headlands, Mr Clarke says.

“If those points are not removed either by the combine yield monitoring or whatever analytical software you use, it can cause quite drastic inconsistency when you analyse it afterwards, particularly on the headlands where there is a lot of turning.”

To combat this, he has used and developed methods to remove data generated from:

• Very slow or high combine speeds
• Where the combine has changed direction quickly and points before and after
• Where data is very different from neighbouring data (helps remove reduced header width cuts)

Yields can also be calibrated against on-farm measured weighbridge or weigh cell data and corrected accordingly, so when margin data is added there isn’t an overestimation or underestimation from the field, he says.

The cleaned yield maps typically have quite a lot of missed data points on headlands, he notes. “But it should leave enough accurate data from the headlands, as in this case, the Barkers go round the headlands two or three times while opening up the field.”

2. Apply an economic cost to each data point

While cleaned yield data can start to identify areas of high or poor performance it is hard to attribute that directly to economic performance. “Our next step was to give each point an actual margin.

“Brian Barker has very accurate records for each of his fields, so we can calculate either a gross or net margin for each yield map point by multiplying the yield by the grain price minus the total costs to give the net margin.”

In Mr Barker’s case, this was relatively straightforward, as inputs were standard across fields. But it would be more complicated with variable inputs.

© Tim Scrivener

3. Identify areas of the field that perform the same each year

Once net margins have been calculated, a statistical method called clustering can be used to identify areas of fields that perform the same each year, he explains.

It compares each point in the field over the years to identify ones that fluctuate in a similar pattern over time – for example, those that are always high yielding or low yielding.

These can then be used to create a cluster map of the field, with similarly performing areas each year highlighted.

“You can then start to look at what this means, for example comparing wheat yields and margins for each different area across the years.”

In the Shrubbery Field example (see graphic), five areas of different performance were found by this method with the green and blue area (cluster 3 and 4) in the centre of the field consistently the highest margin part of the field, with the orange area (cluster 1), which is next to an area of woodland, usually the poorest part of the field, he says.

“If you look at the average margin across the rotation, the blue and green areas are earning on average £780-£840/ha. However the orange cluster only averaged £240/ha between 2011 and 2015.”

Most of this area was taken out of production in 2019 and put into 0.9ha of pollen and nectar mix paying £451/ha – based on this, the analysis has provided the economic justification for taking this area out of production.

While, in this case, the data was used to justify a decision already made, the principle applies to future decision-making, he says.

“We can identify other poor performing clusters across the farm then compare with the cost of implementing a scheme including establishment and management costs and the total payment received by the scheme, to make decisions to be based on the economic returns from arable production compared with environmental options,” he explains.

 Result of cluster analysis of Shrubbery Field by David Clarke

© Niab

4. Use other data sources to further help decision-making

Other data sources can also be useful for when making decisions, particularly about future land use and entry into environmental stewardship. “For example, erosion risk mapping using topographic wetness and Lidar-elevation data can help identify areas potentially at risk from soil or water movement.

“This allows you to look for areas that are at high environmental risk, as well as underperforming and where they co-exist.”

5. Compare across fields as well as in-field

The analysis also allows fields across the farm to be compared, particularly where fields have been in the same cropping rotation, says Mr Clarke.

Single field analysis allows the poorest performing areas to be identified, but it’s unlikely that parts of every field will be removed from production. Across field analysis provides the opportunity to highlight the poorest performing parts of the farm as well as giving greater insight into where measures should be targeted, he explains.

David Clarke was speaking during two AHDB webinars on how to identify marginal land