Forest loss drivers: fix the Curtis table, add GFW/Sims et al. (2025) as default source

[mscrawford]

f35_forest_lost_share sets the forest disturbance in MAgPIE module 35; at the default s35_forest_damage = 2 only its shifting column is read. Curtis et al. (2018) define shifting agriculture as clearing followed by regrowth, which is what module 35 models. The problems were in the transcription and the country mapping.

Curtis et al. (2018) source (ForestLossDrivers)

• forest_loss.csv gave Africa 3 Mha of tree cover loss; Table 1 gives 39. The rows summed to 279 Mha against the printed global 314 (315 with 39). Africa is 92% shifting agriculture, so the SSA rate has been 13x too low in every input revision since December 2020.
• mapping.csv put the 19 Middle East and North Africa countries in Latin America. The article lists no countries per region, so they are reassigned by geography: North Africa to Africa, the Middle East to Russia/China/South Asia, Reunion to Africa. Europe's 1% moves from shifting agriculture to wildfire, as printed.
• The table stays a madrat source: the corrected table is added to the shared source folder as forest_loss_fixed.csv, which readForestLossDrivers() reads. forest_loss.csv and mapping.csv there stay unchanged for older mrland versions. The country mapping ships in inst/extdata/regional/regionmappingCurtis2018.csv. The rows are normalised to sum to 100: the printed rows sum to 99-102 (rounding, cells printed "<1%"), and each driver is total_loss * percent/100 / 15, so a row over 100 would attribute more loss than was observed.
• calcForestLossByDriver() distributes the regions to countries by FRA 2020 naturally regenerating forest area in 2010, the area and year calcForestLossShare() divides by, so every country in a source region carries one rate.

New default source GFWLossByDriver

• Tree cover loss by country, year and dominant driver from the GFW data-api: gadm__tcl__iso_change v20260424, i.e. UMD loss crossed with the Sims et al. (2025) 1 km driver map (by release date its annual update v1.13, 2001-2025), 30% canopy threshold, no API key.
• calcForestFireLoss() is renamed calcForestLossByDriver(); it and calcForestLossShare() gain src ("GFW" default, "Curtis") and period (GFW only, default 2015:2024).
• The share is loss divided by FRA 2020 naturally regenerating forest area, the natural forest MAgPIE applies it to: 2020 for GFW, 2010 for Curtis.
• Only shifting_cultivation goes to MAgPIE, from either source; the unused FRA fire column overall is dropped (magpie#934 drops it from driver_source). Permanent agriculture, hard commodities and settlements remove the forest, logging is MAgPIE's own timber harvest, and wildfire and other natural disturbances are not land use but would be reported as land-use-change CO2. All eight classes go to validation instead (calcValidTreeCoverLoss: observed tree cover loss by driver (GFW, Sims et al. 2025) mrvalidation#45).
• calcForestDisturbances() removed: module 35 loads f35_forest_disturbance_share but reads it nowhere.
• Shares above 1 are still clamped to 1.

Effect on the shifting column

share per year	develop	src = "Curtis"	src = "GFW"
SSA	0.00028	0.00359	0.00354
IND	0	0	0.00159
OAS	0.00082	0.00082	0.00257
LAM	0.00166	0.00170	0.00027
MEA	0.00103	0.00247	0.0000001
JPN	0.00044	0.00044	0.0000001
CAZ, CHA, EUR, NEU, REF, USA	< 0.0002	< 0.0002	< 0.00004

Curtis and GFW agree in SSA because that is the transcription fix. They part company where Curtis's seven regions straddle the twelve model regions: India and China sit in one source region whose shifting share rounds to zero, so no table edit can give them a rate distinct from Russia's.

Latin America carries the largest change and the weakest classification: Sims et al. report ~50% producer's accuracy for shifting cultivation there against ~80% globally.

[mscrawford]

To @codeZeilen and @pascal-sauer: This has a lot of safety guards on this older dataset. I kept them in, but at the same time am skeptical about this being worth the extra code. Should I remove the checks?

[mscrawford]

One thing that came up in the process of making this bugfix is that Michael Windisch's old code on forest disturbances was removed from MAgPIE but not removed from the input data that goes into MAgPIE.

[pascal-sauer]

Seems like you're talking to claude and not reviewers, so I removed reviewers again for now, so they don't get notified each time.

[flohump]

Thanks @mscrawford, both the Curtis fix and the switch to GFW make sense.

Review produced with the help of Claude.

Suggestions:

1. Merge together with magpie#934 and the new input data revision, since the rename breaks develop.
2. Keep Curtis as a sensitivity source, but make it usable: it is a hand-transcribed table shipped in inst/extdata, so read it from there directly and drop downloadForestLossDrivers() (the HPC source folder still holds the uncorrected 2020 forest_loss.csv and mapping.csv and is never re-staged); and in the Curtis branch rename shifting_agriculture to shifting_cultivation and keep only that column, as the GFW branch does, so magpie#934 can read it.
3. Denominator: MAgPIE applies the rate to natural forest, but it is divided by tree cover extent (incl. plantations). By region this barely matters for SSA/LAM, but OAS gets 0.00175 instead of 0.00243 and IND 0.00287 instead of 0.00165 (FRA naturally regenerating forest as denominator). I'd divide by FRA naturally regenerating forest, as the Curtis branch does.
4. Decomposition run in magpie#934 with fixed Curtis, to separate the bug fix (SSA) from the source switch to GFW.

[mscrawford]

Thanks @mscrawford, both the Curtis fix and the switch to GFW make sense.

Review produced with the help of Claude.

Suggestions:

1. **Merge together** with magpie#934 and the new input data revision, since the rename breaks develop.

2. **Keep Curtis as a sensitivity source, but make it usable:** it is a hand-transcribed table shipped in `inst/extdata`, so read it from there directly and drop `downloadForestLossDrivers()` (the HPC source folder still holds the uncorrected 2020 `forest_loss.csv` and `mapping.csv` and is never re-staged); and in the Curtis branch rename `shifting_agriculture` to `shifting_cultivation` and keep only that column, as the GFW branch does, so magpie#934 can read it.

3. **Denominator:** MAgPIE applies the rate to natural forest, but it is divided by tree cover extent (incl. plantations). By region this barely matters for SSA/LAM, but OAS gets 0.00175 instead of 0.00243 and IND 0.00287 instead of 0.00165 (FRA naturally regenerating forest as denominator). I'd divide by FRA naturally regenerating forest, as the Curtis branch does.

4. **Decomposition run** in magpie#934 with fixed Curtis, to separate the bug fix (SSA) from the source switch to GFW.

Hi Florian, thanks for the review. A few very quick thoughts with follow-up later.

1. Absolutely, the order will be mrland + magpiesets -> mrvalidation -> magpiemodel.
2. The download function is actually just reading from extdata, where the table is stored. I like that because it's keeping the metadata in a nice way, and would defend this judgement call. Happy to discuss.
3. That's a good idea -- thanks.
4. I already have these runs complete. I will update the magpiemodel PR hopefully soon.

[pascal-sauer]

Suggested change

	staged <- file.copy(list.files(extdata, full.names = TRUE), ".", overwrite = TRUE)
	staged <- file.copy(list.files(extdata, full.names = TRUE), ".", overwrite = TRUE, copy.mode = FALSE)

Packages and all their files (including extdata) in the renv cache are write protected. copy.mode = FALSE ensures write protection is dropped when copying (makes cleanup easier, we only want to protect renv cache).

[pascal-sauer]

Suggested change

	if (extdata == "") {
	stop("ForestLossDrivers package data not found. Is mrland installed correctly?")
	}

Handling this case is probably not worth it I'd say. In general LLMs try to cover every single angle that something can go wrong, but that is often unnecessary in my opinion.

[pascal-sauer]

@tscheypidi what's your take on this pattern, is this idiomatic madrat? download function pulling from extdata?

[pascal-sauer]

One problem we definitely have with this: the download function is only called once. If the data in extdata is updated this update will not be applied until then source folder is manually removed or until the download function is run manually.

[mscrawford]

Ah, crap. I hadn't considered that... What do you think the best way forward is here?

[mscrawford]

On second thought though: This is the Curtis data, taken from a table of a particular paper. Therefore, the data in extdata will never change. Granted, it was bugged once, and this doesn't necessarily help in the generalized case. So I'm happy to move the metadata into a readSource function, e.g.

[mscrawford]

Solved differently: Curtis is now a plain readSource() from the source folder, with no download function and no copy in the package, so there is nothing to go stale. I'll replace the file in the shared source folder with the merge.

[mscrawford]

Correction: no file in the shared source folder is replaced. The corrected table is a new file there, forest_loss_fixed.csv, read since ce477f3; forest_loss.csv and mapping.csv stay untouched for older mrland versions.

[pascal-sauer]

duplicated author list, but that's ok

[pascal-sauer]

So we have a data product combined from 3 sources is my understanding. Does it make sense to provide a little more detail how they are combined?

[pascal-sauer]

Suggested change

#' @importFrom madrat calcOutput readSource

[pascal-sauer]

Suggested change

	calcForestLossShare <- function(source = "GFW", period = 2015:2024) {
	calcForestLossShare <- function(src = "GFW", period = 2015:2024) {

Renaming gets around the linter issues

[pascal-sauer]

Suggested change

	#' @importFrom madrat toolCountryFill
	#' @importFrom magclass getItems

[pascal-sauer]

Suggested change

	"of",
	"Sims et al. (2025) plus 'Unknown'. NOTE: the driver map underlying",
	"of Sims et al. (2025) plus 'Unknown'. NOTE: the driver map underlying",

[pascal-sauer]

Suggested change

#' @importFrom magclass as.magpie magpiesort

[mscrawford]

Thanks, all four are in 6f8b408.

1. Order as agreed: mrland and magpiesets, then mrvalidation, then magpie#934 with the new input revision.
2. Only shifting_cultivation now, from both sources. I kept Curtis a normal madrat source rather than reading inst/extdata.
3. Done: the share is divided by FRA 2020 naturally regenerating forest (2010 for Curtis), so the tree cover extent download is removed.
4. I'll update magpie#934 with the decomposition once the new inputs are built.

[mscrawford]

Thanks, all your suggestions are in 6f8b408. Beyond them, the tree cover extent download is removed, since Florian's point 3 made it unused.