General Rules

The general process described here applies to all aggregations, regardless of the termType.

Finding Cycles

Aggregations are triggered for a particular product, region, and period. The region can be a country, or World. The period is 20 years in length and is fixed at specific dates (e.g., 1990-2009, 2010-2026).

For crop aggregations, Cycles can only be included if the relevant product is the primaryProduct of the Cycle. This restriction does not apply to processedFood aggregations.

Excluded Cycles

Cycles can be excluded from aggregations for two reasons. One is that they do not represent commercial practices, which means they have commercialPracticeTreatment set to FALSE. The other is that they have a blank value for the primaryProduct. This is to exclude sources that are focused on collecting measurement data and are unlikely to reflect real farms. Cycles with a 0 value for the primaryProduct are not excluded, as these are important in representing crop failures and immature periods of plantations.

Calculation of aggregated values

Number values

An aggregated value for a data item is generally a simple mean of the values in the underlying Cycles. For example, if three Cycles have the input Urea (kg N) with values 20, 30, and 90, the aggregated value is (20 + 30 + 90) / 3 = 46.7.

There are two key exceptions. One is for the primaryProduct in crop aggregations, values are adjusted to the same Dry matter before the mean is calculated. The other is if a weighting structure is used, a weighted mean is calculated. Examples of weighting structures include: organic/irrigation production, country production, and plantation phase. Further detail on these weighting structures can be found in the Crop page.

Where no weighting structure applies, every contributor counts for the same share of the result and the mean is a simple one:

$$\bar{v} = \sum_i \frac{v_i}{n_T}$$

Where:

• $\bar{v}$ = the aggregated value, rounded to 4 significant figures
• $v_i$ = the value contributed by the underlying Cycle or sub-aggregation $i$, stored for a sub-aggregation but not for a Cycle
• $n_T$ = the number of contributors combined

Where a weighting structure does apply, each contributor counts for the share its weight gives it:

$$\bar{v} = \sum_i v_i \times \frac{w_i}{W_T}$$

Where:

• $\bar{v}$ = the aggregated value, rounded to 4 significant figures
• $v_i$ = the value contributed by the underlying Cycle or sub-aggregation $i$, stored for a sub-aggregation but not for a Cycle
• $w_i$ = the weight of $i$
• $W_T$ = the total of every $w_i$

The two are the same calculation with every weight set to 1, but only one of them ran for any given value, so only that one is shown against it.

Weights are not normalised when they are calculated, so each is divided by the total $W_T$ here — which is what the percentages shown in the Cycle and Term descriptions represent. A total weight of 0 is treated as 1, so weights that all collapse to zero do not cause a division by zero.

The $v_i$ of the Cycles behind a sub-aggregation are not stored: a sub-aggregation can cover tens of thousands of Cycles, and copying their values into the aggregation would multiply the dataset for no information gain — only the weight each had is recorded. The handful of sub-aggregations a country aggregation combines are bounded, so their values are stored. The Cycles that went in are listed under aggregatedCycles, and their values can be read from the Cycles themselves. The number of Cycles behind a value is published as observations, and the spread as min, max, sd and the percentiles of the distribution.

Boolean values

For terms with boolean units, the aggregated value is the majority of the underlying values: it is TRUE when at least half of the underlying Cycles have the value TRUE, and FALSE otherwise. Weights are not applied to boolean values.

Completeness

Aggregations operate at the level of completeness areas. For example, all fertiliser data from Cycles where completeness.fertiliser = TRUE are aggregated together. If a Cycle has completeness.fertiliser = FALSE, the fertiliser data from that Cycle are excluded, but data from other completeness areas which are TRUE are still aggregated. This approach allows the pooling of data from many incomplete Cycles to create an overall complete dataset.

Blank values occur for two reasons: the user has uploaded a data item with a blank value, or a data item exists in one underlying Cycle but not another. For example, Cycle A has the input Urea (kg N) with value = 70, but Cycle B does not have the input Urea (kg N) despite having completeness.fertiliser = TRUE. Cycle B effectively has a blank value for Urea (kg N).

For blank values with an associated completeness area (e.g., fertiliser): if the area is complete, blank values within this area are treated as value = 0; if the area is incomplete, blank values are skipped.

This zero-filling is what allows a term reported by only part of the population to be averaged over the whole population rather than only where it occurs. It leaves no trace in the aggregated data — the zero-filled Cycles report nothing for the term — so it is the one step that cannot be reproduced from the published values alone. Where no weighting structure applies, the zero-filled contributors are counted alongside the reporting ones:

$$\bar{v} = \sum_i \frac{v_i}{n_R + n_Z}$$

Where:

• $\bar{v}$ = the aggregated value
• $v_i$ = the value reported for the term, stored for a sub-aggregation but not for a Cycle
• $n_R$ = the number of Cycles or sub-aggregations that report the term
• $n_Z$ = the number of them that are complete for the area but report no value

Where a weighting structure applies, the same split is made over the weights rather than the counts:

$$\bar{v} = \sum_i v_i \times \frac{w_i}{W_R + W_Z}$$

Where:

• $\bar{v}$ = the aggregated value
• $v_i$ = the value reported for the term, stored for a sub-aggregation but not for a Cycle
• $w_i$ = the weight of a Cycle or sub-aggregation that reports the term
• $W_R$ = the total weight of those that report the term
• $W_Z$ = the total weight of those that are complete for the area but report no value

Only the reporting values contribute to the sum; the zero-filled ones enter through the denominator alone, which is what pulls the result down.

An area is complete on the aggregated Cycle when at least one of the Cycles or sub-aggregations it combines is complete for it, as the data of that contributor already covers the area. animalPopulation is the exception: it is never complete on an aggregation of a liveAnimal or liveAquaticSpecies product, as the animals are not aggregated.

For blank values without an associated completeness area (e.g., measurement), the general rule is that blank values are skipped. There are exceptions to this for percentage values, that are detailed below.

Site

The Sites to aggregate are found by looking at the cycle.site.id of each Cycle in an aggregation. This means that if two Cycles occur on the same Site, the Site effectively appears twice in the aggregation calculations.

The infrastructure node is not included in aggregations.

Management

To aggregate Site management data, all the data items with dates that fall within the aggregation period, e.g., 2010-2026, are first averaged. If the sum of these average values is over 100 for items within the same sumIs100Group, e.g., land cover or tillage, they are rescaled so the sum is 100.

The average values are then given new start and end dates based on the aggregation period. For 2010-2026, these would be 2010-01-01 and 2026-12-31.

The same method is used to aggregate management data that falls into the previous aggregation period.

Measurement

Aggregated measurement values are calculated for each depth interval present in the underlying Cycles.

Cycle

The transport and transformation nodes are not included in the aggregated Cycle.

Inputs

Inputs are aggregated by first finding the Cycles which have the relevant completeness area set to TRUE, adding 0 values to any blank values, and then averaging the values.

Primary Product

Even if the product completeness area is marked as incomplete, the value of the primaryProduct is still aggregated, to reduce data loss and improve the yield estimate.

The economicValueShare is aggregated for all Cycles where the value of the primaryProduct is greater than 0 and completeness.product = TRUE. Cycles with 0 yield values are excluded as usually nothing is produced in these Cycles so they have no economic value. An average is taken of all available economicValueShare for each product. If the sum of these averages is over 100, they are rescaled so that the total economicValueShare of the aggregated Cycle is 100.

The example below shows the aggregation of the primary product value and economicValueShare.

Full example

• Aggregated value = mean of all Cycles with a value (including 0): (1000 + 5000 + 4000 + 2000 + 6000 + 0) / 6 = 3000
• Aggregated economicValueShare = mean of the economicValueShare of Cycles with completeness.products = TRUE and a value greater than 0, i.e. Cycles 1 and 3: (20 + 100) / 2 = 60

Cycle 2 is excluded because it has no economicValueShare, Cycle 7 because its yield is 0, and Cycles 4, 5 and 6 because their product data are incomplete.

Because economicValueShare is a share of the Cycle's total revenue rather than a physical quantity, it also has to account for how prevalent the product is across the population. A product reported by only a low-weight part of the population holds a smaller share of the aggregation's revenue than its own sub-aggregation suggests, and is rescaled down accordingly:

$$EVS = \overline{evs} \times \frac{W_P}{W_T}$$

Where:

• $EVS$ = the aggregated economic value share, rounded to 2 decimal places
• $\overline{evs}$ = the mean of the reported economic value shares
• $W_P$ = the total weight of the sub-aggregations that report this product
• $W_T$ = the total weight of all sub-aggregations

This rescale applies to economicValueShare only. A product's value (and its min, max and sd) is deliberately left unscaled: it is the average where the product is produced, and multiplying it by the reporting fraction would turn it into a total-area intensity — collapsing, for example, a crop residue reported only by a small organic sub-aggregation from 4400 to 17.

Secondary products

For secondary products (i.e., products which are not the primaryProduct), the value and economicValueShare are only aggregated from Cycles where completeness.product = TRUE. This is because secondary products are more likely to be omitted from Cycles which do not have complete product data.

Practices

How practices are aggregated depends on the glossary they are in, their units, and whether they are part of a sumIs100Group.

Those in the Operation glossary have an associated completeness area and so are aggregated like inputs.

For practices without percentage units, e.g., Current plantation age, blank values in the underlying Cycles are skipped. This is because for a lot of these practices, it does not make sense to assume a value is 0 just because it is missing.

For practices that do have percentage units, if they are in a sumIs100Group, only Cycles where the sum across the sumIs100Group is 100 are included in the aggregation. This is to ensure the aggregated value will also sum to 100.

If a Cycle has value for at least one percentage term in a group, any other percentage terms in that group which are treated has having 0 values. This ensures that the sum of percentage terms within a group adds to 100% in the aggregation. For example, if Cycle A has tillageConventional = 100 and no value for tillageReduced or tillageNoTill, the blank values for tillageReduced and tillageNoTill are treated as 0.

For practices with percentage units that are not in a sumIs100Group or the System or Standards & Labels glossaries, e.g., Green manure incorporated, blank values in the underlying cycles are skipped. This is the same method used for practices without percentage units.

For practices with percentage units, that are in the System or Standards & Labels glossaries, blank values are treated as 0. This means that when the term is missing from a Cycle, e.g., Organic, we are assuming that its value is 0.

Only crop aggregations are split into organic and conventional sub-aggregations, where Organic labels the sub-aggregation as 100. Every other product type has no such split, so Organic is published as an ordinary aggregated practice: the mean over every underlying Cycle, with the Cycles that do not report it contributing 0 as above. An aggregation built from five Cycles of which one is organic therefore reports Organic = 20.

Emissions

Tier 1 and Tier 2

Tier 1 and 2 emissions are aggregated from the underlying Cycles, rather than being recalculated from the aggregated Cycle. This is because emissions models consider more information than is aggregated, e.g., specific dates or practices with Boolean values.

Aggregated emissions values are calculated as an average of underlying Cycles, with blank values being skipped. If the underlying Cycles have different method tiers, e.g., tier 1 and tier 2, these emissions are aggregated together and the lowest tier is used as the method tier for the aggregated emission.

Background

For crop and processedFood products, background emissions are recalculated using the aggregate Cycle data as it reduces the processing time compared to aggregating the background emissions from the underlying Cycles. These are the only term types for which the models are re-run.

Every other product type, including animalProduct, has its background emissions averaged from the underlying Cycles in the same way as Tier 1 and Tier 2 emissions. Each aggregated emission keeps the methodModel of the Cycles it came from, which is linkedImpactAssessment where the impacts of the input come from another Cycle on the platform.

Where the models are re-run, one class of background emission is still aggregated rather than recalculated: those associated with a Seed input that has a linked Impact Assessment. The values are then aggregated from the underlying Cycles where completeness.seed=TRUE. This is because for Saplings, background emissions come from linked nursery Cycles, whose impacts differ from those of the Cycle using them.

Where the Seed input has no linked Impact Assessment, the background emissions are skipped in the aggregation and recalculated from the aggregated Cycle. This gives a more complete result than aggregating the underlying Cycles, where the model estimating these emissions has often not run.

Impact Assessment

The aggregated Impact Assessments are recalculated using the aggregated Cycle data, and the aggregated economicValueShare values.

Data quality score

Every aggregation is scored against a set of pass/fail checks, and the result is published as aggregatedQualityScore out of aggregatedQualityScoreMax. The pair appears on the aggregated Cycle and on the aggregated Impact Assessment, in the JSON-LD and in the Data API, as two columns of the tabular mirror published on the AWS Open Data registry, and on the platform as a Data Quality tag reading LOW, MEDIUM or HIGH — the tag shows HIGH when every applicable check passed, MEDIUM when one or two failed, and LOW below that — with a tooltip listing each check and whether it was met.

The score is a count of the checks that passed. It is not a percentage, not a confidence interval, and not a statement about any individual underlying Cycle: a Cycle contributing to an aggregation with a high score is not thereby of higher quality, and an upload carries no quality score of its own. There are five checks.

The completeness areas that count towards the score are a subset of the completeness fields, and the subset differs by termType: a crop aggregation is scored on cropResidue, electricityFuel, fertiliser, otherChemical, pesticideVeterinaryDrug, product, seed and water, while a processedFood aggregation is scored on electricityFuel, ingredient, otherChemical, product and water alone.

aggregatedQualityScoreMax varies between aggregations, because a check that does not apply is counted in neither the score nor the maximum. The maximum is therefore 3 for a product that is not a crop, 4 for a crop aggregated at country level, and 5 for a crop aggregated over a region or the World. A score of 3 means three checks out of a possible three, four or five, so the two fields have to be read together.

A low score does not mean an aggregation was withheld. Aggregations are published whatever they score, and one scoring 0 is available in the same way as one scoring its maximum. The score is published so that a data user can decide how much weight to give an aggregation, and so that the reason a value looks unlike the Cycles behind it can be traced to a specific check.