From f506032a3fefeab814e4bbb9d7fcc479c286afcd Mon Sep 17 00:00:00 2001 From: Louisa Emmons <33166333+lkemmons@users.noreply.github.com> Date: Thu, 3 Apr 2025 10:08:15 -0600 Subject: [PATCH 1/6] Update README_preprocessor.md Add link to "All_countries" file --- README_preprocessor.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README_preprocessor.md b/README_preprocessor.md index b7e1ef6..7f8b4af 100644 --- a/README_preprocessor.md +++ b/README_preprocessor.md @@ -87,6 +87,9 @@ Alternatively `Download ZIP` button is available at https://github.com/NCAR/finn Next, copy your fire detection shapefile(s) into the directory ../finn/preprocessor/data/. These files need to be UNZIPPED. +The "All_countries" shape file (used for assigning global region) can be retrieved from: +https://ftp.acom.ucar.edu/user/emmons/FINN_region_shapefiles/ + ### 4. Building the Docker image The next step is to build the Docker image. To build the Docker image, execute the following command from the terminal, in the directory where `Dockerfile` exists (this project directory): @@ -262,7 +265,7 @@ Once you have edited the first cell, you can go ahead and run the code, as you d NOTE: If running a recent year, the year-specific MODIS LCT and VCF files may not be yet available. This will lead to an error statement in Section 5. If the year-specific data are unavailable, we recommend choosing the most recent year available for your processing. You will have to go back and edit the first cell and restart the kernel. -NOTE: [Minimum in struction to use QGIS](https://github.com/yosukefk/finn_preproc/wiki/Minimum-Instruction-for-using-QGIS-with-FINN-preprocessor) available to visualize burned area and raster dataset stored in PostGIS. +NOTE: [Minimum instructions to use QGIS](https://github.com/yosukefk/finn_preproc/wiki/Minimum-Instruction-for-using-QGIS-with-FINN-preprocessor) available to visualize burned area and raster dataset stored in PostGIS. ### 7 What you'd do day-to-day From 084557203d7e4152dfdc29dfa3608b65bd4f84ea Mon Sep 17 00:00:00 2001 From: Louisa Emmons <33166333+lkemmons@users.noreply.github.com> Date: Thu, 28 May 2026 11:45:50 -0600 Subject: [PATCH 2/6] Add description of FINN preprocessor code Created by Claude Opus 4.7 --- FINN_preprocessor_description.md | 315 +++++++++++++++++++++++++++++++ 1 file changed, 315 insertions(+) create mode 100644 FINN_preprocessor_description.md diff --git a/FINN_preprocessor_description.md b/FINN_preprocessor_description.md new file mode 100644 index 0000000..da13c9d --- /dev/null +++ b/FINN_preprocessor_description.md @@ -0,0 +1,315 @@ +# FINN Preprocessor — Detailed Description of `work_nrt.py` and the `code_anaconda` Pipeline + +## 1. What the preprocessor does + +The Fire INventory from NCAR (FINN) preprocessor turns satellite active-fire (AF) point detections from MODIS and VIIRS into a daily, georeferenced layer of burned-area polygons, each tagged with land-cover, vegetation-cover-fraction (VCF), and region attributes. Those polygons are then handed downstream to the FINN emissions code, which multiplies area by land-cover- and region-specific emission factors to produce trace-gas and aerosol emissions for atmospheric chemistry models. + +`preprocessor/code_bashinterface/work_nrt.py` is the top-level driver script. It is the bash-line equivalent of the legacy Jupyter notebook workflow, intended for daily near-real-time (NRT) operation. It does not do any geometry itself; it imports a handful of Python modules from `preprocessor/code_anaconda/`, and each of those modules in turn launches `psql` against a PostgreSQL+PostGIS database where almost all of the heavy lifting happens. The Python layer is essentially a thin orchestrator that: + +1. parameterizes SQL files with the user's tag and date range, +2. shells out to `psql -f