Fairmat 2024: use NXcoordinate_system together with NXtransformations

[lukaspie]

This PR is concerned with coordinate systems and the transformations that are intricately linked to them. It was started for multiple reasons:

• There is the frequent need to explicitly describe which coordinate system is used in an experiment.
• NeXus by default uses a coordinate system that is the same as that used by McStas. However, this can be limiting:
  • there are now several techniques coming in where there is no explicit beam defined: temperature-dependant IV measurements, scanning probe microscopy (STM, AFM, Atom Probe, or multi-beam EM), etc.
  • in some communities, there already exists an agreement on which coordinate system shall be used, which is different than what NeXus uses. As an example, there is an existing ISO standard for data transfer in surface chemical analysis that uses a coordinate system that is left-handed and defined based on the sample stage. It would be good if in an application definition (e.g. NXxps) one could explicitly state which coordinate system is to be used. There has already been some discussion on this, see NXmpes #1464 (comment)
    • Specifically on this, for the XPS community, there are certain angles (e.g. analyser-to-beam) that are needed to perform rigorous analysis. It would be helpful if one could write these in the existing coordinate system (as in the ISO standard), without needing to transform back from the NeXus coordinate system.
• NeXus (with its concept of NXtransformations) does already allow to define a specific coordinate system (see e.g. discussion in NXtransformations: clarify that these are active transformations + example #1278). But, this has several limitations:
  • these transformations always depend on a tool that can read them out and perform the transformations itself. We would also like to allow that one can give an coordinate system explicitly, both in a machine and human readable way
  • left-handed CSs are currently not supported
  • fractional coordinates are not supported

Therefore, we introduce in this PR two new base classes: NXcoordinate_system and NXrotation_conventions.

Explanation of using NXcoordinate_system and NXtransformation:

• An application defintion contains one or more of NXcoordinate_system right below NXentry. See how to use different number of instances NXcoordinate_system from its lengthy docstring.
• in NXcoordinate_system, there is a depends_on, and (NXtransformation) group
• in NXtransformation AXISNAME depends_on, one can place either ".", or the path to an NX_coordinate_system
• if a depends_on attribute or field links to a NXcoordinate_system, it should pick the respective depends_on field in that class, and apply the specified TRANSFORMATIONS

Example of how this is implemented in a (proposed) application definition: NXXps. Note that this one still uses the logical grouping NXcoordinate_system_set, but we removed this because it is not really needed. We would simply have xps_coordinate_system(NXcoordinate_system) directly under NXxps/NXentry.

[lukaspie]

Copying here discussion that was buried in #1464 for clarity:

LP: we shall align to and extend the existing ISO standard for XPS. The concept of coordinate systems allow connecting multiple conventions.
PM: Which use cases needs a different coordinate system?
LP: ISO standard requires a different coordinate system
SB: Another example is when NeXus coordinate system being linked to incident beam does not exists because no beam exists in a specific experiment type, like temperature dependent IV measurement.
PM: shall we make this a standard, so all sw shall implement the ability of changing coordinate systems?
HB: NeXus shall allow multiple coordinate systems, as MX is also using it and other fields also require that. It is preferred to have a clearly described way how to do that.
PC: NeXus uses McStas as default, but other coordinate systems can also be used. Important is that the use of NXtransformations shall be consistent throughout the datasets.
SB: This is exactly what the proposal is about: provide a simple solution either on how to use NXtransformations when a base change is required to connect two coordinate systems, or on describing a coordinate system which is used in the definition and data file.
PM: Cannot we use a label for this purpose?
SB: Indeed that is what hapenning: Currently, the NXtransformations chain is followed along @depends_on until it ends with '.'. The proposal is that instead of ending with a '.', it could end with a label pointing to a Coordinate System which either describes a coordinate system on its own, or connetcs it to another coordinate system using NXtransformations.

[mkuehbach]

"_set" is a restricted keyword thus NXcoordinate_system_set class needs a name change.

NXcoordinate_system_set is meant to provide a specialized NXobject with several commonly used NXcoordinate_system conventions with their own specialized NXcoordinate_system (CS) such that this collection can directly be used in appdefs instead of having each app demanding that if they wish to define one or multiple of the CS defined in NXcoordinate_system need to type the definitions again.

Possibilities:

1. Rename NXcoordinate_system_collection (lousy but straightforward)
2. Move top-level docstring of CS_set class to CS (+ would not loose context but - would loose predefined CS (processing_reference_frame, sample_reference_frame, etc.)
3. Similar to 2. delegate the definition of several CS to each appdef with maxOccurs not constraints, in practice people will then likely write CS1, CS2, CS3 etc.

[mkuehbach]

Some comments from @PeterC-DLS made in #1421 will be taken over here

[woutdenolf]

Would that work?

Yep that sounds good to me. Thanks!

I think this was already the case, that if you wanted to use mcstas, you needed to end the chain of ".", right?

No idea. I hope so because not every local coordinate system is relatable to mcstas.

https://manual.nexusformat.org/classes/base_classes/NXtransformations.html

This class will usually contain all axes of a sample stage or goniometer or a detector. The NeXus default McSTAS coordinate frame is assumed, but additional useful coordinate axes may be defined by using axes for which no transformation type has been specified.

https://manual.nexusformat.org/design.html#nexus-geometry

NeXus supports description of the shape, position and orientation of objects in The NeXus Coordinate System. Position and orientation can be defined as Coordinate Transformations using the NXtransformations class.

It somewhat suggests that McSTAS is "the default coordinate frame" but never says what this means and it never says if you omit depends_on it automatically applies or not. So lets assume "not" and "the current coordinate system is not defined with respect to anything else".

[woutdenolf]

dbefd09 solves the last issue for me. Ready for a vote imo! @phyy-nx

[phyy-nx]

Hi all, as discussed in the Telco, we can now move this PR to an online vote. NIAC committee members please vote on this PR using emojis on this comment. 👍 for yes, 👎 for no, anything else (for example 👀) to abstain. We need 14 votes to hit quorum so please review and vote!

@mkuehbach @lukaspie, there are a couple of unresolved comments, can you review and resolve them? I don't think they affect the vote. Thanks.

[phyy-nx]

Vote has passed! Once the reviews have finished, comments have been resolved, an approval posted, this can be merged.

[lukaspie]

@PeterC-DLS could you please resolve your one remaining comment?

@phyy-nx @PeterC-DLS @woutdenolf if you have nothing else you would still like to change, I could appreciate an approval afterwards.

[PeterC-DLS]

LGTM