ESRF id31 new detector default and Q plot in all SIXC reference frames

CHANGELOG.md

@@ -66,6 +66,79 @@ A ***critical bug*** was fixed that affects bulk CTR calculations:
   released versions, up to and including v1.5.0. See the CTR structure-factor
   documentation for details.
 
+GUI changes:
+
+- The position readout of the image plot now shows ``HKL`` as a single
+  bracketed triplet instead of three separate ``H``, ``K`` and ``L`` fields,
+  and reports the momentum transfer of the selected reciprocal-space frame as
+  ``Q[alpha]``, ``Q[lab]``, ``Q[omega]``, ``Q[chi]``, ``Q[phi]`` or
+  ``Q[cryst]``. The field is relabelled when the frame selection changes.
+  Both triplets are shown with five decimals, and the two fields are sized so
+  that the full triplet is visible instead of being elided. Pixel coordinates
+  are shown with two decimals, which is the space this needs.
+
+GUI fixes:
+
+- Cancelling or resetting the machine parameter or the crystal parameter
+  dialog now restores the configuration that was active when the dialog was
+  opened. Both dialogs apply every edit immediately, so restoring the widgets
+  alone left the edited values active, and the discarded configuration stayed
+  in use until it was overwritten or a config file was loaded.
+
+ESRF ID31 beamline support and reciprocal-space display:
+
+- Detector images can now be displayed in reciprocal space. The new ``Q-plot``
+  toolbar action rebins the currently shown image (single image, maximum, or
+  sum) onto a regular grid of in-plane and out-of-plane momentum transfer
+  using pyFAI's ``FiberIntegrator``, which requires pyFAI >= 2025.1. With an
+  older pyFAI the action reports an error and stays disabled. A drop-down next
+  to the action selects the reciprocal-space frame: ``Q_alpha`` (the surface
+  frame returned by ``QAlpha``, the default), ``Q_lab``, ``Q_omega``,
+  ``Q_chi``, ``Q_phi`` and ``Q_cryst``. The frames that undo the ``omega``
+  rotation are only defined for a single image and are refused for maximum and
+  sum images. **This feature is experimental and its conventions may still
+  change.**
+- The reciprocal-space conversion lives in the new module
+  ``orgui.app.qconversion`` and is exact for arbitrary azimuthal references.
+  It is deliberately part of the application layer rather than of
+  ``orgui.datautils.xrayutils``, so that it is not mistaken for production
+  reciprocal-space code. pyFAI's ``sample_orientation`` flag can only express
+  quarter turns and its rotations are composed about fixed axes, neither of
+  which matches orGUI's continuous azimuth convention, so orGUI supplies
+  ``FiberIntegrator`` with its own unit definitions. The result agrees with the
+  per-pixel ``QAlpha`` calculation to numerical precision; see the geometry
+  documentation for when to use which.
+- Added the compiled extension ``_qconversion_cpp``, which converts a full
+  detector image in a single pass. A 6.2 megapixel Pilatus 6M image is
+  converted in roughly 50 ms; a numpy implementation is used when the extension
+  has not been built. The pyFAI integrator is reused while the conversion is
+  unchanged, so stepping through images no longer resets it for every image.
+- HDF5 files that are still being written can now be refreshed from the GUI.
+  The tree view is rebuilt and the current scan reloaded without restarting
+  the application.
+- Added a ``BlissScan_EBS_p4`` backend for the ID31 Pilatus4 detector, an
+  example backend under ``examples/backend/ID31_EBS_p4_backend.py``, and the
+  ``ch8153`` beamtime.
+- The default backend is now ``id31_default_p4`` and the fallback geometry
+  describes a Pilatus4 4M CdTe detector.
+- Maximum and sum images are now untoggled when the image number changes, and
+  the maximum/sum toolbar icons no longer get out of sync with the displayed
+  image.
+- The Q-plot now stays switched on and follows the display instead of being
+  turned off. It is rebuilt when the image number changes, when the maximum or
+  sum image is toggled, when the frame is changed, and when the machine angles,
+  the azimuthal reference, the energy or the orientation matrix are edited.
+- **BREAKING CHANGE:** HDF5 file locking is now disabled by default
+  (``HDF5_USE_FILE_LOCKING=False``), so that files still open for writing by
+  the acquisition system can be read. A manually set environment variable
+  still wins, and ``--hdflocking`` / ``-l`` restores the previous behavior.
+- **BREAKING CHANGE:** ID31-style scan objects are now recognized from the
+  configured backend class instead of a hardcoded list of beamtime ids. This
+  fixes the ``id31_default_p4`` backend, which the old list did not cover, and
+  makes new ID31 beamtimes work without code changes. Custom backends whose
+  class name does not contain ``BlissScan`` are no longer treated as
+  ID31-style, even if their beamtime id was previously listed.
+
 
 ## [1.5.0] (2026-06-07)
 

doc/source/geometry.rst

@@ -66,6 +66,252 @@ where :math:`\vec{H}_0` and :math:`\vec{H}_1` are vectors in reciprocal lattice
 units. orGUI calculates the intersections of this line with the Ewald sphere and
 converts the corresponding detector angles back to detector pixel coordinates.
 
+Momentum Transfer and the Q-Plot
+--------------------------------
+
+The central plot of orGUI normally shows the detector image in **pixel
+coordinates**; the reciprocal-space position of a pixel is reported in
+reciprocal lattice units as the cursor is moved. The ``Q-plot`` toolbar action
+switches the display to **momentum transfer**, the representation commonly used
+for grazing-incidence work. This is a change of the displayed coordinates only,
+it does not alter the loaded data.
+
+Momentum transfer is split into a component parallel to the surface and one
+along the surface normal,
+
+* :math:`q_\parallel` -- in-plane momentum transfer, labelled ``qip`` and
+  plotted as ``q_par``,
+* :math:`q_\perp` -- out-of-plane momentum transfer, labelled ``qoop`` and
+  plotted as ``q_perp``.
+
+orGUI can calculate these two quantities in two different ways. Both describe
+the same reciprocal space, but they differ in what they return and in when they
+are used.
+
+Per pixel with ``QAlpha``
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is the route used everywhere else in orGUI, for example for reflection
+positions and for the rod integration. The detector calibration converts pixel
+coordinates into the surface angles ``gamma`` and ``delta``, and the Vlieg
+diffraction equation converts those angles into a Cartesian momentum transfer
+vector in the ``alpha`` frame,
+
+.. math::
+
+   \begin{aligned}
+   Q_x &= K \sin(\delta)\cos(\gamma) \\
+   Q_y &= K \left[\cos(\delta)\cos(\gamma) - \cos(\alpha)\right] \\
+   Q_z &= K \left[\sin(\gamma) + \sin(\alpha)\right]
+   \end{aligned}
+
+with :math:`K = 2\pi/\lambda`. The out-of-plane component is :math:`Q_z`, and
+the in-plane component is the radial component in the surface plane:
+
+.. code-block:: python
+
+   import numpy as np
+
+   detectorCal = ubcalc.detectorCal   # DetectorCalibration.Detector2D_SXRD
+   angles = ubcalc.angles             # HKLVlieg.VliegAngles
+   alpha_i = ubcalc.mu                # incidence angle in rad
+
+   gamma, delta = detectorCal.surfaceAngles(alpha_i, shape=image.shape)
+   Qxyz = angles.QAlpha(alpha_i, delta, gamma)   # inverse Angstrom
+
+   q_perp = Qxyz[..., 2]                          # out-of-plane
+   q_par = np.hypot(Qxyz[..., 0], Qxyz[..., 1])   # in-plane, unsigned
+
+The result has the same shape as the detector image: every pixel keeps its own
+momentum transfer. The sampling is therefore irregular and curved in
+:math:`(q_\parallel, q_\perp)`, which is exactly what is needed to look up
+where a reflection falls, but it cannot be handed to an image widget directly.
+
+On a regular grid with ``FiberIntegrator``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. warning::
+
+   The ``Q-plot`` and the :mod:`orgui.app.qconversion` module are
+   **experimental**. They live in the application layer, not in
+   ``orgui.datautils.xrayutils``, precisely so that they are not mistaken for
+   the production reciprocal-space code. The conventions described here may
+   still change.
+
+This is the route used by the ``Q-plot`` action. pyFAI's ``FiberIntegrator``
+computes the same per-pixel momentum transfer and additionally **rebins** the
+intensities onto a regular grid, which is what makes the result displayable as
+an image. orGUI drives it through :mod:`orgui.app.qconversion`, which supplies
+pyFAI with orGUI's own angle conventions:
+
+.. code-block:: python
+
+   from orgui.app import qconversion
+
+   res = qconversion.integrateImage(
+       detectorCal,
+       image,
+       alpha_i,                 # incidence angle in rad
+       frame="Q_alpha",         # see "Reciprocal-space frames" below
+   )
+
+   res.intensity     # rebinned image, shape (npt_oop, npt_ip)
+   res.inplane       # q_par axis
+   res.outofplane    # q_perp axis
+
+``FiberIntegrator`` was introduced in pyFAI 2025.1. With an older pyFAI the
+``Q-plot`` action is unavailable and reports an error instead.
+
+Which one to use
+~~~~~~~~~~~~~~~~
+
+.. list-table::
+   :header-rows: 1
+   :widths: 22 39 39
+
+   * -
+     - ``QAlpha``
+     - ``FiberIntegrator``
+   * - Returns
+     - one :math:`(q_\parallel, q_\perp)` per pixel
+     - intensities rebinned onto a regular grid
+   * - Sampling
+     - irregular, curved
+     - regular, uniform
+   * - In-plane sign
+     - unsigned, :math:`q_\parallel \ge 0`
+     - signed
+   * - Requires
+     - nothing beyond orGUI
+     - pyFAI >= 2025.1
+   * - Use for
+     - reflection positions, integration, any calculation
+     - displaying a whole image in reciprocal space
+
+Use ``QAlpha`` whenever the momentum transfer of a specific pixel or reflection
+is needed. Use ``FiberIntegrator`` when a complete image has to be shown or
+exported in reciprocal-space coordinates, and accept that the rebinning
+quantises positions to the width of one grid cell.
+
+Reciprocal-space frames
+~~~~~~~~~~~~~~~~~~~~~~~
+
+``QAlpha`` returns the momentum transfer in the **alpha frame**, the frame of
+the sample surface. The drop-down next to the ``Q-plot`` action selects which
+frame the image is displayed in; the remaining frames are reached by undoing
+the sample rotations, following
+:math:`\vec{H} = UB^{-1}\,\Phi^{-1}\,X^{-1}\,\Omega^{-1}\,\vec{Q}_\alpha`.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 20 46 34
+
+   * - Frame
+     - Reached from the alpha frame by
+     - Defined for
+   * - ``Q_alpha``
+     - nothing, this is the default
+     - any image
+   * - ``Q_lab``
+     - the ``alpha`` rotation
+     - any image
+   * - ``Q_omega``
+     - undoing ``omega``
+     - a single image only
+   * - ``Q_chi``
+     - undoing ``omega`` and ``chi``
+     - a single image only
+   * - ``Q_phi``
+     - undoing ``omega``, ``chi`` and ``phi``
+     - a single image only
+   * - ``Q_cryst``
+     - additionally undoing the orientation matrix ``U``
+     - a single image only
+
+For every frame the out-of-plane component is the ``z`` axis of that frame and
+the in-plane component is the radial component in its ``xy`` plane. Maximum and
+sum images combine many ``omega`` angles, so the frames that undo ``omega`` are
+refused for them. ``Q_cryst`` additionally needs the orientation matrix.
+
+``Q_cryst`` is the Cartesian reciprocal-space frame of the crystal, so it holds
+:math:`B\vec{H}`. Multiplying ``Q_phi`` by :math:`UB^{-1}`, or equivalently
+``Q_cryst`` by :math:`B^{-1}`, gives the reciprocal lattice coordinates
+:math:`\vec{H} = (h, k, l)^T`, the same result as
+:meth:`~orgui.datautils.xrayutils.HKLVlieg.VliegAngles.anglesToHkl`.
+
+Why orGUI supplies its own pyFAI units
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+pyFAI's built-in fiber units cannot express orGUI's geometry, for two
+independent reasons.
+
+**The azimuth is continuous, ``sample_orientation`` is not.** pyFAI's
+``sample_orientation`` is the EXIF dihedral group acting on the image array, so
+it only spans quarter turns. It describes how the image array is laid out, not
+where the sample sits. orGUI's azimuthal reference rotates the ``alpha``
+rotation axis about the incident beam and takes arbitrary values, so it is
+passed through ``tilt_angle`` instead, while ``sample_orientation`` stays at the
+identity:
+
+.. math::
+
+   \mathrm{tilt\_angle} = -\left(\mathrm{azimuth} + \frac{\pi}{2}\right)
+
+The quarter turn is the offset between orGUI's azimuthal reference and pyFAI's
+fiber convention. Encoding the azimuth in ``sample_orientation`` instead would
+only be correct for azimuths that are exact multiples of 90 degrees.
+
+**The rotations are composed in the opposite order.** pyFAI combines the
+incidence and tilt rotations extrinsically, about fixed axes, as
+:math:`R_x(-\mathrm{tilt})\,R_y(\mathrm{incidence})`. In orGUI the azimuth
+rotates the ``alpha`` axis itself, so the incidence rotation has to be applied
+about the *already rotated* axis,
+:math:`R_y(\mathrm{incidence})\,R_x(-\mathrm{tilt})`. The two agree only when
+one of the two angles vanishes.
+
+:mod:`~orgui.app.qconversion` therefore builds its own
+``pyFAI.units.UnitFiber`` objects that implement orGUI's convention, and hands
+them to ``FiberIntegrator``. pyFAI still performs the rebinning; only the
+coordinate definition is replaced.
+
+With that in place the two routes agree to numerical precision. This is
+verified in ``orgui/app/test/test_q_conversion.py``, which compares them per
+pixel for flat and tilted detectors, at several incidence angles and at
+azimuths that are deliberately not multiples of 90 degrees. The same file
+checks that ``Q_phi`` reproduces ``anglesToHkl``, that ``Q_cryst`` reproduces
+:math:`B\vec{H}`, that every frame preserves :math:`|\vec{Q}|`, and that a
+single bright pixel is placed by the rebinning within one grid cell of the
+position predicted by ``QAlpha``.
+
+Performance
+~~~~~~~~~~~
+
+Detector images are large, so the conversion is collapsed into a single affine
+relation before any pixel data is touched. With
+:math:`n = |(x, y, z)|` every component reduces to
+
+.. math::
+
+   q_j = k \left( \frac{G_{j0} x + G_{j1} y + G_{j2} z}{n} - c_j \right)
+
+where the matrix :math:`G` and the offset :math:`\vec{c}` absorb the sample
+orientation map, the beam and incidence rotations, the axis relabelling and the
+frame rotation. A compiled kernel,
+``orgui/app/cpp/qconversion_cpp.cpp``, evaluates both displayed quantities in a
+single pass; a plain numpy implementation is used when the extension has not
+been built. The result is cached, because pyFAI evaluates the in-plane and the
+out-of-plane unit separately but passes the same pixel positions to both, so an
+image is converted once rather than twice.
+
+On a 6.2 megapixel Pilatus 6M the compiled kernel converts a full image in
+roughly 50 ms, and the second unit evaluation is served from the cache.
+
+.. note::
+
+   ``numexpr`` is deliberately not used here. Version 2.11.0 returns wrong
+   results for a small fraction of multi-threaded evaluations of expressions of
+   this kind, which is not acceptable for a coordinate transform.
+
 Further Reading
 ---------------