From 7157d25ccad2be20fdb110a2701fcbcb5edbf7d6 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 13 Mar 2023 15:17:04 -0700 Subject: [PATCH 01/19] Add new datatype, the union of NX_CHAR and NX_NUMBER --- nxdlTypes.xsd | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/nxdlTypes.xsd b/nxdlTypes.xsd index 8811142b90..06ec4b8333 100644 --- a/nxdlTypes.xsd +++ b/nxdlTypes.xsd @@ -619,4 +619,14 @@ + + + Any valid character string or NeXus number representation + + + + From db047de1b74c5c8052adda685405fec282a6540c Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 13 Mar 2023 15:17:33 -0700 Subject: [PATCH 02/19] Change AXISNAME field of NXdata to allow character values --- base_classes/NXdata.nxdl.xml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 774c653253..68d4b918cc 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -273,13 +273,17 @@ but not to describe how the data is to be plotted. (https://www.nexusformat.org/NIAC2018Minutes.html#nxdata-plottype--attribute) - + Dimension scale defining an axis of the data. Client is responsible for defining the dimensions of the data. The name of this field may be changed to fit the circumstances. Standard NeXus client tools will use the attributes to determine how to use this field. + + Most dimensions scales will be sequences of numbers but if an + axis is better represented using names, such as channel names, + an array of NX_CHAR can be provided. From 3dc02baa3beeb1dabd63c987f076b7404188387b Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 13 Mar 2023 15:44:45 -0700 Subject: [PATCH 03/19] Whitespace --- base_classes/NXdata.nxdl.xml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 68d4b918cc..2ee17d382b 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -275,16 +275,16 @@ - Dimension scale defining an axis of the data. - Client is responsible for defining the dimensions of the data. - The name of this field may be changed to fit the circumstances. - Standard NeXus client tools will use the attributes to determine - how to use this field. + Dimension scale defining an axis of the data. + Client is responsible for defining the dimensions of the data. + The name of this field may be changed to fit the circumstances. + Standard NeXus client tools will use the attributes to determine + how to use this field. Most dimensions scales will be sequences of numbers but if an axis is better represented using names, such as channel names, an array of NX_CHAR can be provided. - + A *dimension scale* must have a rank of 1 and has length ``n``. From 6d2370324cc8aacd963be57796a8282388089098 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Fri, 17 Mar 2023 11:01:15 -0700 Subject: [PATCH 04/19] Add NXdata attribute default_slice Which slice of data to show in a plot by default. This is useful especially for datasets with more than 2 dimensions. --- base_classes/NXdata.nxdl.xml | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 2ee17d382b..8cd34d8537 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -115,6 +115,38 @@ the axes attribute can be omitted. + + + Which slice of data to show in a plot by default. This is useful especially for + datasets with more than 2 dimensions. + + Should be an array of strings of length equal to the number of dimesions + in the data, with the following possible values: + * ".": All the data in this dimension should be included + * Integer: Only this slice should be used. + * String: Only this slice should be used. Use if AXISNAME_indices is a string + array. + + Example:: + + data: NXdata + @signal = "data" + @axes = ["image_id", "channel", ".", "."] + @image_id_indices = 0 + @channel_indices = 1 + @default_slice = [".", "difference", ".", "."] + image_id = [1, ..., nP] + channel = ["threshold_1", "threshold_2", "difference"] + data = uint[nP, nC, i, j] + + Here, a data array with four dimensions, including the number of images + (nP) and number of channels (nC), specifies more dimensions than can be + visualized with a 2D image viewer for a given image. Therefore the + default_slice attribute specifies that the "difference" channel should be + shown by default. + + + + + + + + These symbols will be used below to illustrate the coordination of the rank and sizes of datasets and the + preferred ordering of the dimensions. Each of these are optional (so the rank of the datasets + will vary according to the situation) and the general ordering principle is slowest to fastest. + The type of each dimension should follow the order of scan points, detector output (e.g. pixels), + then time-of-flight (i.e. spectroscopy, spectrometry). Note that the output of a detector is not limited + to single values (0D), lists (1D) and images (2), but three or higher dimensional arrays can be produced + by a detector at each trigger. + + + Rank of the ``data`` field associated with this detector + number of scan points + number of detector pixels in the slowest direction + number of detector pixels in the second slowest direction + number of detector pixels in the third slowest direction + + + + Description and metadata for a single channel from a multi-channel detector. + + Given an NXdata group linked as part of an NXdetector group that has an axis with named channels (see the + example in NXdata), the NXdetector will have a series of NXdetector_channel groups, one for each + channel, named CHANNELNAME_channel. + + Example, given these axes in the NXdata group: + @axes = ["image_id", "channel", ".", "."] + + And this list of channels in the NXdata group: + channel = ["threshold_1", "threshold_2", "difference"] + + The NXdetector group would have three NXdetector_channel groups: + detector: NXdetector + ... + threshold_1_channel: NXdetector_channel + threshold_energy = float + flatfield = float[i, j] + pixel_mask = uint[i, j] + threshold_2_channel: NXdetector_channel + threshold_energy = float + flatfield = float[i, j] + pixel_mask = uint[i, j] + difference_channel: NXdetector_channel + threshold_energy = float[2] + + + + Energy at which a photon will be recorded + + + + Response of each pixel given a constant input + + + + + + + + + + Custom pixel mask for this channel. May include nP as the first dimension for + masks that vary for each scan point. + + + + + + + + From 3382235c2c52c8d28b2270d69cc1f8d5c1c75380 Mon Sep 17 00:00:00 2001 From: "Aaron S. Brewster" Date: Tue, 18 Apr 2023 15:02:00 -0700 Subject: [PATCH 08/19] Apply suggestions from code review Co-authored-by: soph-dec <73243774+soph-dec@users.noreply.github.com> --- base_classes/NXdetector_channel.nxdl.xml | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/base_classes/NXdetector_channel.nxdl.xml b/base_classes/NXdetector_channel.nxdl.xml index cac2d7ab0a..32b4f1ea5e 100644 --- a/base_classes/NXdetector_channel.nxdl.xml +++ b/base_classes/NXdetector_channel.nxdl.xml @@ -51,28 +51,31 @@ Description and metadata for a single channel from a multi-channel detector. - Given an NXdata group linked as part of an NXdetector group that has an axis with named channels (see the - example in NXdata), the NXdetector will have a series of NXdetector_channel groups, one for each + Given an :ref:`NXdata` group linked as part of an NXdetector group that has an axis with named channels (see the + example in :ref:`NXdata </NXdata@default_slice-attribute>`), the NXdetector will have a series of NXdetector_channel groups, one for each channel, named CHANNELNAME_channel. - Example, given these axes in the NXdata group: + Example, given these axes in the NXdata group:: + @axes = ["image_id", "channel", ".", "."] - And this list of channels in the NXdata group: + And this list of channels in the NXdata group:: + channel = ["threshold_1", "threshold_2", "difference"] - The NXdetector group would have three NXdetector_channel groups: - detector: NXdetector + The NXdetector group would have three NXdetector_channel groups:: + + detector:NXdetector ... - threshold_1_channel: NXdetector_channel + threshold_1_channel:NXdetector_channel threshold_energy = float flatfield = float[i, j] pixel_mask = uint[i, j] - threshold_2_channel: NXdetector_channel + threshold_2_channel:NXdetector_channel threshold_energy = float flatfield = float[i, j] pixel_mask = uint[i, j] - difference_channel: NXdetector_channel + difference_channel:NXdetector_channel threshold_energy = float[2] From ab45175d848945e2716c43167fd2c07c3a7df007 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Tue, 18 Apr 2023 15:25:40 -0700 Subject: [PATCH 09/19] Add NXdetector_channel to NXdetector and NXmx Co-authored-by: Sophie Hotz --- applications/NXmx.nxdl.xml | 13 +++++++++++++ base_classes/NXdetector.nxdl.xml | 12 ++++++++++++ 2 files changed, 25 insertions(+) diff --git a/applications/NXmx.nxdl.xml b/applications/NXmx.nxdl.xml index 0c71dab11f..35ff93d79b 100644 --- a/applications/NXmx.nxdl.xml +++ b/applications/NXmx.nxdl.xml @@ -764,6 +764,19 @@ plate, CMOS, ... + + + + Group containing the description and metadata for a single channel from a multi-channel + detector. + + Given an :ref:`NXdata` group linked as part of an NXdetector group that has an axis with + named channels (see the example in :ref:`NXdata </NXdata@default_slice-attribute>`), + the NXdetector will have a series of NXdetector_channel groups, one for each channel, + named CHANNELNAME_channel. + + + diff --git a/base_classes/NXdetector.nxdl.xml b/base_classes/NXdetector.nxdl.xml index a6bb0b8c68..aee91bd13b 100644 --- a/base_classes/NXdetector.nxdl.xml +++ b/base_classes/NXdetector.nxdl.xml @@ -421,6 +421,18 @@ + + + Group containing the description and metadata for a single channel from a multi-channel + detector. + + Given an :ref:`NXdata` group linked as part of an NXdetector group that has an axis with + named channels (see the example in :ref:`NXdata </NXdata@default_slice-attribute>`), + the NXdetector will have a series of NXdetector_channel groups, one for each channel, + named CHANNELNAME_channel. + + + Spectral efficiency of detector with respect to e.g. wavelength From 4ba96454071a9ac9921417fc4609642b6ce169ce Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 19 Jun 2023 09:56:43 -0700 Subject: [PATCH 10/19] Fix formatting --- base_classes/NXdata.nxdl.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 504eff8a50..f86bdffae8 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -122,10 +122,11 @@ Should be an array of strings of length equal to the number of dimesions in the data, with the following possible values: + * ".": All the data in this dimension should be included * Integer: Only this slice should be used. * String: Only this slice should be used. Use if AXISNAME_indices is a string - array. + array. Example:: From 6f2a05678f1d6c4b6db1d5305b322c0350d7e382 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 19 Jun 2023 09:56:58 -0700 Subject: [PATCH 11/19] Add example of default_sice using an integer as a string --- base_classes/NXdata.nxdl.xml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index f86bdffae8..18f8cb11dd 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -146,6 +146,10 @@ default_slice attribute specifies that the "difference" channel should be shown by default. + Alternate version using an integer would look like this (note 2 is a string):: + + @default_slice = [".", "2", ".", "."] + From 146a96ae4192150a2a7e9cad11ef1e3a5bde8799 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Mon, 19 Jun 2023 10:11:08 -0700 Subject: [PATCH 12/19] Bring in additional fields from NXdetector into NXchannel --- base_classes/NXdetector_channel.nxdl.xml | 52 ++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/base_classes/NXdetector_channel.nxdl.xml b/base_classes/NXdetector_channel.nxdl.xml index 32b4f1ea5e..a27ff7e85d 100644 --- a/base_classes/NXdetector_channel.nxdl.xml +++ b/base_classes/NXdetector_channel.nxdl.xml @@ -83,6 +83,13 @@ Energy at which a photon will be recorded + + + True when the flat field correction has been applied in the + electronics, false otherwise. + + + Response of each pixel given a constant input @@ -92,6 +99,24 @@ + + + Errors of the flat field correction data. + The form flatfield_error is deprecated. + + + + + + + + + + True when the pixel mask correction has been applied in the + electronics, false otherwise. + + + Custom pixel mask for this channel. May include nP as the first dimension for @@ -103,4 +128,31 @@ + + + + The value at which the detector goes into saturation. + Especially common to CCD detectors, the data + is known to be invalid above this value. + + For example, given a saturation_value and an underload_value, the valid + pixels are those less than or equal to the saturation_value and greater + than or equal to the underload_value. + + The precise type should match the type of the data. + + + + + + The lowest value at which pixels for this detector would be reasonably + measured. The data is known to be invalid below this value. + + For example, given a saturation_value and an underload_value, the valid + pixels are those less than or equal to the saturation_value and greater + than or equal to the underload_value. + + The precise type should match the type of the data. + + From 355b4081e0cd95f27a28cd8383a05c71f92f4a50 Mon Sep 17 00:00:00 2001 From: "Aaron S. Brewster" Date: Tue, 20 Jun 2023 06:04:15 -0700 Subject: [PATCH 13/19] Update base_classes/NXdetector_channel.nxdl.xml Co-authored-by: soph-dec <73243774+soph-dec@users.noreply.github.com> --- base_classes/NXdetector_channel.nxdl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/base_classes/NXdetector_channel.nxdl.xml b/base_classes/NXdetector_channel.nxdl.xml index a27ff7e85d..0d6fd50c8b 100644 --- a/base_classes/NXdetector_channel.nxdl.xml +++ b/base_classes/NXdetector_channel.nxdl.xml @@ -37,7 +37,7 @@ will vary according to the situation) and the general ordering principle is slowest to fastest. The type of each dimension should follow the order of scan points, detector output (e.g. pixels), then time-of-flight (i.e. spectroscopy, spectrometry). Note that the output of a detector is not limited - to single values (0D), lists (1D) and images (2), but three or higher dimensional arrays can be produced + to single values (0D), lists (1D) and images (2D), but three or higher dimensional arrays can be produced by a detector at each trigger. From 48a82e49a1fd0c33f9042545ffde9aaca06f24a3 Mon Sep 17 00:00:00 2001 From: "Aaron S. Brewster" Date: Tue, 19 Sep 2023 20:23:24 -0700 Subject: [PATCH 14/19] Bugfix in description Co-authored-by: Sophie Hotz Co-authored-by: soph-dec <73243774+soph-dec@users.noreply.github.com> --- base_classes/NXdata.nxdl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 18f8cb11dd..7452096908 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -125,7 +125,7 @@ * ".": All the data in this dimension should be included * Integer: Only this slice should be used. - * String: Only this slice should be used. Use if AXISNAME_indices is a string + * String: Only this slice should be used. Use if ``AXISNAME`` is a string array. Example:: From ac8c5d09f5f1fbca5c6a502bc1cfc66d974d16d9 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Tue, 19 Sep 2023 20:30:52 -0700 Subject: [PATCH 15/19] Full example --- base_classes/NXdata.nxdl.xml | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 7452096908..1032efb44b 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -148,7 +148,15 @@ Alternate version using an integer would look like this (note 2 is a string):: - @default_slice = [".", "2", ".", "."] + data:NXdata + @signal = "data" + @axes = ["image_id", "channel", ".", "."] + @image_id_indices = 0 + @channel_indices = 1 + @default_slice = [".", "2", ".", "."] + image_id = [1, ..., nP] + channel = ["threshold_1", "threshold_2", "difference"] + data = uint[nP, nC, i, j] From 8ff58ed43a4cb4d3ffd610a9db5f017cfc156e00 Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Tue, 19 Sep 2023 20:37:56 -0700 Subject: [PATCH 16/19] Better example --- base_classes/NXdetector_channel.nxdl.xml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/base_classes/NXdetector_channel.nxdl.xml b/base_classes/NXdetector_channel.nxdl.xml index 0d6fd50c8b..8cfc2ca87a 100644 --- a/base_classes/NXdetector_channel.nxdl.xml +++ b/base_classes/NXdetector_channel.nxdl.xml @@ -71,10 +71,14 @@ threshold_energy = float flatfield = float[i, j] pixel_mask = uint[i, j] + flatfield_applied = bool + pixel_mask_applied = bool threshold_2_channel:NXdetector_channel threshold_energy = float flatfield = float[i, j] pixel_mask = uint[i, j] + flatfield_applied = bool + pixel_mask_applied = bool difference_channel:NXdetector_channel threshold_energy = float[2] From bb6906e29d439011faedcae9dcb995e93c1c6b8b Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Thu, 19 Oct 2023 10:33:18 -0400 Subject: [PATCH 17/19] Typo --- base_classes/NXdata.nxdl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 1032efb44b..78b82e5562 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -120,7 +120,7 @@ Which slice of data to show in a plot by default. This is useful especially for datasets with more than 2 dimensions. - Should be an array of strings of length equal to the number of dimesions + Should be an array of strings of length equal to the number of dimensions in the data, with the following possible values: * ".": All the data in this dimension should be included From d7ae84cd016d97842d8e01d302032b8436d010de Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Thu, 19 Oct 2023 10:34:10 -0400 Subject: [PATCH 18/19] Added NX_CHAR as type for default_slice --- base_classes/NXdata.nxdl.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 78b82e5562..f2431e3711 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -115,7 +115,7 @@ the axes attribute can be omitted. - + Which slice of data to show in a plot by default. This is useful especially for datasets with more than 2 dimensions. From 78566e1aba63779db736001edf6e816f24667f7a Mon Sep 17 00:00:00 2001 From: Aaron Brewster Date: Thu, 19 Oct 2023 10:37:18 -0400 Subject: [PATCH 19/19] Actually use NXCHAR_OR_NUMBER --- base_classes/NXdata.nxdl.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index f2431e3711..a232ac3b43 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -115,12 +115,12 @@ the axes attribute can be omitted. - + Which slice of data to show in a plot by default. This is useful especially for datasets with more than 2 dimensions. - Should be an array of strings of length equal to the number of dimensions + Should be an array of length equal to the number of dimensions in the data, with the following possible values: * ".": All the data in this dimension should be included