Skip to content

Enable and fix the TGO CaSSIS driver#720

Open
oleg-alexandrov wants to merge 12 commits into
DOI-USGS:mainfrom
oleg-alexandrov:cassis_support
Open

Enable and fix the TGO CaSSIS driver#720
oleg-alexandrov wants to merge 12 commits into
DOI-USGS:mainfrom
oleg-alexandrov:cassis_support

Conversation

@oleg-alexandrov

@oleg-alexandrov oleg-alexandrov commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

The TGO CaSSIS driver was disabled and did not work. This re-enables it and fixes the issues so the generated CSM ISD matches the ISIS camera to a median of about 7e-4 pixel and a maximum of about 0.013 pixel, across 130 framelets of two real stereo pairs (details below).

The fixes:

  • ephemeris_start_time now unpacks pyspiceql.utcToEt()[0], which returns a (time, kernels) tuple, as the other drivers already do.
  • CaSSIS SummingMode is an enum (0 = 1x1, 1 = 2x2, 2 = 4x4), not a summing factor. The driver now emits the ISIS-style factor (1 for mode 0) rather than the raw 0; a 0 makes the USGSCSM frame model non-self-consistent, with groundToImage returning infinity.
  • The detector center is converted from the ISIS 0.5-based CCD convention to the CSM 0-based convention by subtracting 0.5, as the LRO, MRO, Dawn, MESSENGER, MEX, Kaguya, and KPLO drivers already do. Without it the CSM look is off from ISIS by sqrt(0.5^2 + 0.5^2), about 0.707 pixel.
  • The driver now emits the CaSSIS rational distortion (a CassisDistortion mix-in carrying the 36 IK distortion coefficients). The matching USGSCSM CASSIS distortion type is in a companion PR.

Validated against ISIS with ASP cam_test on two real stereo pairs at different Mars locations, every framelet: MY34_002014_180_1 (equatorial, 60 framelets: 30 left, 30 right) and MY34_005684_218_1 (southern, 70 framelets: 35 left, 35 right). Across all 130 framelets the ISIS-vs-CSM reprojection error has a median of at most about 7e-4 pixel and a maximum of about 0.013 pixel; the left and right looks are essentially identical (per-look maxima 0.01294, 0.01294 for the first pair and 0.01295, 0.01300 for the second). The maximum is the worst single corner pixel, the round-trip residual of the closed-form distortion, the same behavior as ISIS.

To reproduce one framelet from scratch (the PSA image is public, no login):

B=https://archives.esac.esa.int/psa/ftp/ExoMars2016/em16_tgo_cas/data_calibrated/Science_Phase/Orbit_Range_2000_2099/Orbit_2014/Science/272560849/PAN
f=cas_cal_sc_20180506T223500-20180506T223504-2014-16-PAN-272560849-0-0__4_0
wget -q "$B/$f.dat" "$B/$f.xml"
tgocassis2isis from=$f.xml to=$f.cub
# inject the spacecraft clock from the XML exposuretimestamp (hex-ascii):
ts=$(grep -ioE '<em16_tgo_cas:exposuretimestamp>[0-9a-f]+' $f.xml | sed 's/.*>//')
editlab from=$f.cub options=addkey grpname=Instrument keyword=SpacecraftClockStartCount value="$(echo $ts | xxd -r -p)"
spiceinit from=$f.cub spkpredicted=true
isd_generate -k $f.cub $f.cub
cam_test --image $f.cub --cam1 $f.cub --cam2 $f.json --session1 isis --session2 csm --sample-rate 100

which prints, for the ISIS camera versus the CSM camera:

Number of samples: 60

cam1 to cam2 camera direction diff norm
Min:    1.03477e-10
Median: 1.04035e-10
Max:    1.0453e-10

cam1 to cam2 camera center diff (meters)
Min:    0.000411701
Median: 0.000411701
Max:    0.000411701

cam1 to cam2 pixel diff
Min:    0.00018285
Median: 0.000642043
Max:    0.0129958

cam2 to cam1 pixel diff
Min:    5.72071e-05
Median: 0.000580168
Max:    0.0128892

The editlab step is a quirk of the public PSA calibrated product, not part of this change: its exported label does not carry SpacecraftClockStartCount (which spiceinit needs), so it is injected from the XML exposuretimestamp. That is an ISIS ingest detail on the cube side; ALE is unaffected.

Unit tests are in test_cassis_drivers.py. The existing integration test (test_cassis_load) is enabled (it was xfail), and its golden ISD (tests/pytests/data/isds/cassis_isd.json) is updated to the fixed driver's output (summing 1, the cassis distortion, and the half-pixel detector center). The existing test_ephemeris_start_time now mocks utcToEt returning a tuple. New tests: test_sample_summing and test_line_summing for the SummingMode enum, and test_detector_center_sample and test_detector_center_line for the ISIS 0.5-based to CSM 0-based half-pixel conversion.

One minor item: CaSSIS is currently the only instrument that requests the surface light-time correction (it is the only one setting LIGHTTIME_CORRECTION LT+S together with LT_SURFACE_CORRECT), where ISIS samples the target body at the surface-light-time-adjusted time. To match that without changing the shared sensor_position for all sensors, the driver overrides sensor_position for CaSSIS only.

Requires the companion USGSCSM CASSIS distortion PR DOI-USGS/usgscsm#512. Developed with assistance from Claude (Anthropic).

Licensing

This project is mostly composed of free and unencumbered software released into the public domain, and we are unlikely to accept contributions that are not also released into the public domain. Somewhere near the top of each file should have these words:

This work is free and unencumbered software released into the public domain. In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain.

  • I dedicate any and all copyright interest in this software to the public domain. I make this dedication for the benefit of the public at large and to the detriment of my heirs and successors. I intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

oleg-alexandrov and others added 9 commits June 11, 2026 13:29
Re-enable the TGO CaSSIS driver (was disabled) and fix two bugs that
broke it: ephemeris_start_time must unpack utcToEt()[0], and CaSSIS
SummingMode is an enum (0=1x1, 1=2x2, 2=4x4) not a factor, so emit the
ISIS-style summing (sumMode*2, or 1) rather than the raw 0 (a 0 makes the
USGSCSM frame model non-self-consistent).

Add the CaSSIS rational ratio-of-quadratics distortion (Tulyakov/Ivanov,
EPFL; the model ISIS uses in TgoCassisDistortionMap): a CassisDistortion
mix-in packing the 36 INS<ikid>_OD_A{1,2,3}_{CORR,DIST} coefficients, the
driver using it, the CASSIS enum value, and cassis parsing in Util.cpp.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In NaifSpice.sensor_position, the LT+S + correct_lt_to_surface branch
computes adjusted_time (= ephem - obs_tar_lts + radius_lt) but then queried
the target body position (ssb_tars) at the raw ephemeris time. The body
moves along its orbit during the surface light time, leaving a constant
camera-center bias (tens of meters) versus ISIS. Sample ssb_tars at
adjusted_time. General fix, affects any sensor with LT_SURFACE_CORRECT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ISIS uses 0.5-based CCD coordinates (pixel centers at half integers) while
CSM is 0-based. The CaSSIS driver took the IK boresight directly, leaving the
CSM look offset from ISIS by half a pixel in each of sample and line, i.e.
sqrt(0.5^2 + 0.5^2) ~ 0.707 px. Subtract 0.5 from detector_center_sample and
detector_center_line, as the LRO, MRO, Dawn, MESSENGER, MEX, Kaguya and KPLO
drivers already do. cam_test ISIS-vs-CSM then drops from 0.707 px to ~7e-4 px.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the surface light-time fix out of the shared NaifSpice.sensor_position
and into a TGOCassisIsisLabelNaifSpiceDriver.sensor_position override, so it
applies to CaSSIS only and leaves the shared path unchanged for every other
sensor. The override is the shared LT+S surface-correction branch with the one
change that the target body is sampled at the surface-light-time adjusted time;
CaSSIS is a single-record framer so this is exact. Behavior and results are
unchanged for CaSSIS (cam_test ISIS-vs-CSM ~7e-4 px).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Un-xfail test_cassis_load now that the driver works, and regenerate the golden
cassis_isd.json (summing 1, the CaSSIS distortion, the half-pixel detector
center). Fix test_ephemeris_start_time to mock utcToEt returning a tuple (it
returns (et, kernels) now). Add honest unit tests for sample/line_summing (the
SummingMode enum) and detector_center_sample/line (the ISIS 0.5-based to CSM
0-based conversion).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generate the test ISD with attach_kernels False (as the MEX test does) so the
machine-specific absolute kernel paths are not written into the committed golden
cassis_isd.json. Regenerate the golden without them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@oleg-alexandrov oleg-alexandrov requested a review from Kelvinrr June 11, 2026 22:50
The real-kernel *_load tests across many missions began failing once CI
picked up spiceql 1.4.0 (environment.yml had spiceql>=1.3.0 unpinned and
the CI env is rebuilt fresh each run). With searchKernels=False and
useWeb=False the tests rely entirely on the committed per-image kernels,
which 1.4.0 no longer resolves into the planetary ephemeris and CK frames
the way 1.3.x did. Pin to <1.4 to restore the green test environment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@oleg-alexandrov

oleg-alexandrov commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

The CI failures are not from this PR. A recent spiceql release (1.4.0) breaks the real-kernel *_load tests repo-wide.

environment.yml had spiceql unpinned, so CI started pulling 1.4.0 (released after main last passed). Under 1.4.0 the committed per-image kernels no longer resolve (SPKINSUFFDATA, "ck rotation ... can not be found") across many missions (apollo, voyager, mex, lro, and others); the test furnishing uses searchKernels=False and useWeb=False, so there is no fallback. The mocked tests are unaffected. This hits main and every open PR, not just this one.

Commit bf72317 here pins spiceql to >=1.3.0,<1.4 makes the PR pass.

oleg-alexandrov added a commit to oleg-alexandrov/usgscsm that referenced this pull request Jun 11, 2026
The CASSIS distortion type added here references ale::DistortionType::CASSIS,
which lives in the companion ALE PR (DOI-USGS/ale#720) and is not yet on ale
main, so the build fails with "CASSIS is not a member of ale::DistortionType".
Point the ale submodule at oleg-alexandrov/ale cassis_support so CI can build
and test. This must be reverted to ale main once ale#720 is merged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread environment.yml Outdated
- nlohmann_json
- numpy
- spiceql>=1.3.0
- spiceql>=1.3.0,<1.4

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.4.1 fixed the bugs, you can unpin.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unpinned. Tests pass.

Done with Claude/AI assistance.

spiceql 1.4.1 restores the per-image kernel furnishing that 1.4.0 broke,
so the <1.4 cap added earlier is no longer needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@oleg-alexandrov

Copy link
Copy Markdown
Collaborator Author

Tests pass with fixed SpiceQL. Pin to prev version of this removed.

@Kelvinrr

Copy link
Copy Markdown
Collaborator

The editlab step is a quirk of the public PSA calibrated product, not part of this change: its exported label does not carry SpacecraftClockStartCount (which spiceinit needs), so it is injected from the XML exposuretimestamp. That is an ISIS ingest detail on the cube side; ALE is unaffected.

Im unfamiliar with TGO Cassis processing, but is this a required step using ISIS-style (not CSM) cameras? Or is it more about how ESA is managing things? Should we document this on the asc-public-docs? Maybe something I can bring up with my meetings with the TGO folk.

@oleg-alexandrov

oleg-alexandrov commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

It's not a CSM-vs-ISIS thing. It is about which CaSSIS product you start from, and it affects the ISIS-native camera just as much as CSM. The tgocassis2isis ingester has two label translations. For the instrument-team products that carry the full CaSSIS_Header and FSW_HEADER, TgoCassisInstrument.trn maps the FSW exposure timestamp into SpacecraftClockStartCount automatically, so there is nothing to do. For the calibrated products exported to the ESA PSA, the em16_tgo_cas namespace files you download from the archive, the fallback translation TgoCassisExportedInstrument_PSA.trn fills only StartTime and has no SpacecraftClockStartCount group, so the cube comes out without it.

That keyword is required on the cube regardless of camera model. The ISIS TgoCassisCamera constructor reads SpacecraftClockStartCount at TgoCassisCamera.cpp lines 63-65, and spiceinit builds its cache through the camera, so both fail if the keyword is missing. ALE and CSM are downstream of all this and are unaffected. The editlab step is therefore a cube-side ISIS ingest matter, not anything specific to CSM.

It is worth noting that the value itself is not used for the geometry. The camera reads the keyword but computes the time from StartTime in UTC, per the long-standing TODO in TgoCassisCamera.cpp. So the keyword has to be present, but its value is effectively unused for timing. A placeholder would be enough to let the camera construct, though injecting the real timestamp is cleaner and stays correct if CaSSIS is ever switched to SCLK as that TODO intends.

It is suggested to fix this in tgocassis2isis itself rather than relying on the manual editlab step. The ingester already opens and parses the same XML to choose the PSA translation, at tgocassis2isis.cpp lines 71-79, and the em16_tgo_cas exposuretimestamp field is sitting in that same document. A few lines in the PSA-exported path could read it, hex-decode it the way the manual step does, and set SpacecraftClockStartCount on the Instrument group, next to the existing StartTime cleanup. The translation file alone cannot do this because it cannot hex-decode, which is likely why it was left out. That would make the PSA products work for both the ISIS camera and CSM with no manual step, and the editlab line in the recipe above would disappear. I am happy to prepare that ISIS pull request; this thread already has all the information needed.

Documenting it on asc-public-docs is worthwhile in the meantime, for anyone ingesting the PSA or ODE exported calibrated framelets. Once the ingester is fixed the note can collapse to a single line pointing at the ISIS version that handles it.

Prepared by Claude (Anthropic).

Comment thread src/Util.cpp Outdated
} catch (...) {
throw std::runtime_error(
"Could not parse the cassis distortion model coefficients.");
coefficients = std::vector<double>(36, 0.0);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do? It's right after the throw.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. That line is unreachable dead code: the throw above it unwinds out of the catch, so the zero-fill never runs, and even if it did the local coefficients vector is destroyed during stack unwinding and never returned to any caller, so it would have no effect. The vector is also already default-constructed to a valid empty state at its declaration, so there is no uninitialized-value concern either. I removed it.

The same dead line exists in the older LUNARORBITER, RADTAN, and KPLOSHADOWCAM cases (this case was copied from them). I did not touch those to keep this PR focused, but I am happy to clean them up here or in a separate PR if you prefer.

Done with Claude/AI assistance.

The zero-fill assignment came right after the throw and was
unreachable. Drop it.

Done with Claude/AI assistance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants