Skip to content

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Maine Plat-Map Digitizer

Turns municipal tax-map PDFs for towns in Maine into georeferenced parcel geometries (newline-delimited GeoJSON), anchored to the U.S. Census county subdivision ("cousub") town boundary, and validated against the information printed on the plats.

Each town is a self-contained digitizer under src/me_<town>/. They share a common goal, output format, and a library of reusable techniques, but differ in the kind of PDF they start from (scanned raster vs. born-digital vector) and in how sheets are positioned without ground control.

Accuracy caveat (applies to every town). These maps carry no embedded georeferencing — no GCPs, no coordinate grid ticks, no /GEO. Georeferencing is best-effort from the town boundary + map scale, so the output is a digitized approximation for visualization and relative analysis, not survey-grade. Per-sheet placement error ranges from ~17 m (vector maps that share parcels) to ~130 m (maps placed only by an index sheet). Results are reported honestly per town.


The towns

All towns are in Maine. Boundary reference is the Census 2024 TIGER cousub (tl_2024_23_cousub).

Town Dir County (cousubfp) Source maps PDF kind Parcels Containment Coverage
Windsor me_windsor Kennebec (86165) O'Donnell 1976, AutoCAD plot Raster (1-bit CCITT) 2,969 100% ~67%
Newfield me_newfield York (48750) O'Donnell 1977 Raster (RGB scan) 2,094 99.99% ~95%
Parsonsfield me_parsonsfield York (56870) O'Donnell 1976 Raster (faint scan) — 100% ~35% / ~99%¹
Cornish me_cornish York (14485) O'Donnell 2024 Raster (mixed 1-bit/RGB) 2,484 100% ~81%
Leeds me_leeds Androscoggin (38565) James W. Sewall, 1"=500' Raster (grayscale scan) 2,183 100% ~93%²
Otisfield me_otisfield Oxford (55960) O'Donnell 1969 Raster (mixed 1-bit/RGB) 1,263 99.96% ~95%²
Pittston me_pittston Kennebec (59110) O'Donnell 1975 / updated 2025 Vector 1,391 100% ~90%
Somerville me_somerville Lincoln (69645) Northern Geomantics 2024 Vector 407 100% ~70%
Farmingdale me_farmingdale Kennebec (24670) 2024 (CivicPlus) Vector 784 100% ~34%
Temple me_temple Franklin (75980) Acme/Mitchell 1976→2005 Raster (1-bit CCITT) 705 100% ~99.9%³
Livermore me_livermore Androscoggin (40665) O'Donnell 2025 Raster (mixed 1-bit/RGB) 3,511 100% 95.5% / ~100% land²

² Coverage is of land (cousub − water): Leeds is 7.7% water (Androscoggin Lake) and Otisfield 9.75% (Thompson Lake + ponds), excluded via TIGER AREAWATER and not parcelled.

¹ Parsonsfield coverage depends on mode — see "the roads question" below. All overlap counts are ~0; see each town README for exact validation numbers.

³ Temple's nine rural sheets vary in scale (500′/600′/800′) and overlap rather than tile, and the town (and its match-line grid) is tilted ~21° from UTM north, so sheets are placed by a true-scale 3×3 grid in the town's oriented frame and clipped at the seams. Coverage is of land. The four urban (U) sheets are village blow-ups of R9 and are vectorized but not merged (no boundary control). See its README.


Two families of map

The single biggest factor in how a town's pipeline is shaped is whether the PDF contains real vector geometry or just a picture.

Raster maps (Windsor, Newfield, Parsonsfield, Cornish, Leeds)

The parcel linework is a baked-in image (a 1-bit CCITT or flattened RGB raster) — page.get_drawings() returns nothing useful. These are recovered by image processing:

  • Vectorize by watershed. Binarize the ink → seal boundary gaps → seed parcel cores by distance transform → cv2.watershed → trace basins to polygons (RETR_CCOMP so nested parcels become holes, not overlaps). This recovers large rural lots that a naive "drop the largest background region = road" approach loses to leakage.
  • Cornish adds road-sealing + neck-cutting. Every parcel fronts a road, and a single boundary gap leaks a parcel into the open road corridor (this was Windsor's ~67% coverage ceiling). Cornish seals road corridors so parcels disconnect, floods the exterior from the frame, then recovers swallowed interior parcels by cutting at distance-transform "necks" — lifting capture ~64% → ~82%.
  • Labels come from OCR (the text is part of the image), sometimes fused with the few surviving vector glyphs.

Vector maps (Pittston, Somerville, Farmingdale)

Geometry is read directly from page.get_drawings() — no image processing. Two sub-styles:

  • Closed-stroke fills (Pittston): each parcel is a tan-filled, dark-gray closed boundary stroke → extract the closed strokes, drop "container" strokes that enclose many parcels.
  • Shared-edge line network (Somerville, Farmingdale): parcels are the faces of a line network, not individual closed shapes → collect boundary segments + fill rings, snap gaps with shapely.set_precision, polygonize, and keep the faces that contain a lot label.

How sheets get positioned (the hard part)

With no GCPs, the town boundary is the only control. The global anchor is always a fit of the drawn town silhouette / town line to the cousub by a trimmed rigid/similarity ICP with the foot→meter scale fixed (search rotation

  • translation only — free-scale ICP collapses). How individual sheets are placed within that anchor depends on the data:
  • Shared-parcel stitch (Pittston): GIS-tiled sheets share identical parcels across seams → solve a pure-translation stitch by RANSAC over shared parcels. Best accuracy (~17 m).
  • Chamfer match (Cornish): sheets share no parcels, but their road networks do → place each sheet by chamfer-matching its roads to the georeferenced Overall sheet, seeded by its index-label position.
  • Grid / index placement (Windsor, Newfield, Parsonsfield, Somerville, Farmingdale): reconstruct the sheet layout from an index sheet or matchline labels and drop each sheet into its cell. Coarsest placement (~130 m); OCR of index cell numbers is unreliable on these scans, so the layout is a reviewed lookup, verified with each town's index.py/sheet_layout.py overlay.

Scale is calibrated from the map, never assumed from the legend. Windsor's legend says 1"=400' but the embedded /Measure dict proves 1"=500'; Farmingdale's title says 1"=200' but the plotted scale bar measures 1"=440'. Leeds has no /Measure dict and no scale bar, so its scale is calibrated from the maps' own printed frontage dimensions (drawn edge length vs printed feet) — which confirm the title's 1"=500'. Always trust the /Measure dict, the scale bar, or the printed dimensions over the title text.

Leeds also decouples two scales (a first for this repo): a parcel scale (from the dimensions, for correct areas) and a smaller layout scale (the inter-sheet spacing, so the reconstructed 6×3 sheet grid fits the cousub). Adjacent Sewall sheets overlap in the match-line strip, so the spacing is tighter than a full face; each sheet is clipped to its match-line cell and the halves of a straddling lot abut exactly.


The roads question

A standing user requirement across all towns: do not detect roads, water, or alphanumeric characters as parcels. This is genuinely hard on these maps and each town carries filters for it:

  • Roads/ROW: drop long, narrow faces (high oriented-bbox aspect ratio + small short-side width) with no lot label.

  • Text/lettering: big road and place-name labels are stroked at the same line width as parcels, so letter counters (the holes in O/A/D/B) polygonize into fake 1–2 ac parcels → drop faces whose median boundary edge is too short (letter strokes are ~8–12 ft; real lot edges ≥ ~30 ft) or whose compactness is low at small area (keep large irregular lots — river-following woodlots are real).

  • Zoning + water (Leeds): Leeds maps overprint zoning-district boundary curves (big rounded blobs around the LR/RR/PA letters) that close into giant fake parcels — dropped by a container filter (a basin enclosing ≥4 other basins is not a lot). Water (Androscoggin Lake / rivers, 7.7%) is excluded with the authoritative TIGER AREAWATER mask (cousub − AREAWATER == ALAND), so it is never parcelled.

This trades coverage for correctness. On the faint raster towns (Parsonsfield), excluding roads also drops the many lots whose gapped boundaries leak into the road network — which is why Parsonsfield reports both a "lots-to-centreline" (~99%) and an "exclude-roads" (~35%) coverage.


Repository layout

src/me_<town>/        self-contained digitizer per town (run_all.py orchestrates)
  download.py           fetch tax-map PDFs from the town site
  render.py             rasterize pages (raster towns) / extract (vector towns)
  calibrate.py          scale + neatline from /Measure dict or scale bar
  vectorize.py|extract  parcels from ink (raster) or drawings (vector)
  labels.py / ocr.py    lot id, acreage, owner — OCR and/or vector text
  georeference.py       cousub-ICP anchor + per-sheet placement
  clip.py               clip parcels to the cousub line (containment 100%)
  validate.py           overlaps, containment, area/boundary reconciliation
  build_app.py          per-town Leaflet viewer.html
  geo_utils.py          shared helpers (copied per town; see below)
  README.md             town-specific data facts, decisions, and results

data/<town>/          source PDFs + regenerable intermediates (gitignored)
  reference/            shared TIGER cousub shapefile
output/<town>/        parcels.geojsonl, per-sheet files, viewer.html, area CSV
                        (Windsor's outputs live at output/ root — it was first)
qa/<town>/            overlay/QA renders
requirements.txt      Python deps

Each town's modules import their siblings flatly (import config as C); run a stage standalone from inside the package dir, or run the whole pipeline with python3 src/me_<town>/run_all.py.

geo_utils.py is duplicated into each town package rather than shared from a common module — the reusable core is the technique, not a single import. Key shared helpers: write_geojsonl/read_geojsonl (the output format) and geodesic_area_m2/geodesic_acres.


Output format

Parcels are written as newline-delimited GeoJSON (.geojsonl — one Feature per line, coordinates rounded to 7 dp), not a FeatureCollection. It is readable, diffable, and GDAL reads it as GeoJSONSeq. The per-town viewer.html parses it client-side. Town boundary files stay FeatureCollection.

Gotcha: rounding coords to 7 dp on write can re-introduce self-intersections that make_valid fixed in projected space. Re-snap with shapely.set_precision(geo, 1e-7) and re-make_valid after converting to WGS84, so the written geometry is valid.

Area is always geodesic

All real-world areas are measured on the ellipsoid with pyproj.Geod(ellps="GRS80"), never in a projected CRS (UTM / State Plane) or as planar lon/lat. Projected CRSs distort area; geodesic area matches authoritative figures (Census ALAND, e.g. Windsor's reference 92,003,888.06 m² matches to 0.00007%). Geometry operations (intersection/union) may run in a projected CRS for robustness, but the resulting area is measured geodesically.


Setup

Python 3.11+ plus system tools: GDAL CLI (ogr2ogr), poppler (pdfinfo/pdftoppm), tesseract (OCR).

pip install -r requirements.txt
playwright install chromium   # only for me_parsonsfield (Cloudflare challenge)

Then per town:

python3 src/me_<town>/download.py    # fetch the PDFs
python3 src/me_<town>/run_all.py      # full pipeline → output/<town>/

See each town's README for source URLs, data quirks, the decisions behind its pipeline, and exact validation numbers.

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages