Open the 2dseq memory map read-only - #215
Merged
Merged
Conversation
read_array maps a filesystem path with np.memmap and no mode. numpy's default
is "r+", so every dataset load asks the operating system for write access to the
raw data file. Nothing writes it: the array is copied out of the map on the same
line.
The consequence is that a dataset on a read-only file or mount cannot be loaded
at all -- which is how a shared scanner archive is normally exposed:
PermissionError: [Errno 13] Permission denied: '.../2/pdata/1/2dseq'
The parameter files are unaffected, since those are opened in text mode, so the
failure looks selective: acqp, method, visu_pars and reco all read, and only the
binary fails, once per scan.
Reproduced by copying a PV6 study, chmod -R a-w, and loading it; on the same
file, mode="r+" raises and mode="r" reads all 1048576 values.
The archive branch below already reads through path.open("rb") and was never
affected.
test_dataset_reads_from_a_read_only_source covers it, in test_paths.py because
that module's study is synthetic and needs no corpus. It fails with
PermissionError without this change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A dataset on a read-only file or mount cannot be loaded at all:
Cause
read_arrayinbrukerapi/paths.pymaps a filesystem path without giving a mode:np.memmapdefaults tomode="r+", so every dataset load asks the operating system for write access to the raw data file. Nothing writes it — the array is copied out of the map on the same line.This matters because a shared scanner archive is normally exposed read-only. The parameter files are unaffected, since those are opened in text mode, so the failure looks oddly selective:
acqp,method,visu_parsandrecoall read, and only the binary fails, once per scan.Reproduction
Copy any PV6 study,
chmod -R a-w, and load it. On the same file:The change
mode="r"on that one call. The archive branch just below already reads throughpath.open("rb")and was never affected.test_dataset_reads_from_a_read_only_sourcecovers it. It lives intest_paths.pybecause that module's study is synthetic and needs no corpus; it fails withPermissionErrorwithout the change.Test suite
Run against a local PV5.1/PV6.0.1/PV7.0.0 corpus:
master): 4 failed, 245 passed, 51 skippedThe same four failures occur either way — they are corpus-dependent and unrelated to this change.
ruff checkclean at the pinned 0.16.0.Found while diagnosing a user report against a downstream converter, where the raw data lives on a read-only archive mount.