Registration of H&E of continuous sections

[Microbiods]

Hi VALIS team,

Thanks for the amazing work! I am currently trying to apply VALIS to a 3D reconstruction task involving approximately 100 consecutive H&E sections, but I have encountered some challenges. With the default settings, the registration results are not very satisfactory. When I attempt to use a larger input image size, like max_processed_image_dim_px=1024 or enable micro-alignment, I consistently encounter CUDA out-of-memory (OOM) errors.

Each of my original images is quite large (approximately 30,000 × 30,000 pixels), and since the slice thickness is 10 µm, there are noticeable distortions and folding artifacts between adjacent sections. I suspect these factors may be contributing to the bad performance and memory issues. Can you give me some suggestions on recommended parameter choices or settings that might be better suited for this type of large-scale, high-resolution 3D reconstruction task?

Thanks.

Here are the overlapping results for original, rigid transformation, and non-rigid

transformation using default settings.

[cdgatenbee]

HI @Microbiods,

Sorry the results aren't satisfactory. Based on the preview image you also shared over e-mail (below), it does look like there is a great deal of stretching/compression/folding in each slice. Additionally, 10um between slides is fairly large, and think the images used for benchmarking 3D images were closer to 5um (described here, with VALIS' results here ). Unfortunately, VALIS can only do so much to fix these issues, so while it may be possible to improve the alignment, it will likely not be perfect.

Given the code you shared over e-mail, I did notice something else that might be causing an issue. If the filenames match those in the preview image, you may be aligning the images in the incorrect order. This is because sorted uses lexical sorting, and so doing something like:

f_list = ["1.tif", "2.tif", "10.tif", "20.tif"]
print(sorted(f_list))

will return f_list = ['1.tif', '10.tif', '2.tif', '20.tif'], and so images 1-9 will be in the wrong place. Instead, you can use valtils.sort_nicely to sort the files numerically:

from valis import valtils
valtils.sort_nicely(f_list) # sorting done in place

Finally, there should be a solution to running out of memory on the GPU. It's possible to use the CPU for feature matching, which should give you a lot more memory. Additionally, you can reduce the number of features that get detected and stored, which should also reduce memory usage. This is all set when you create the feature detector and matcher, which gets passed to the registrar. In this example, I'm both reducing the number of features detected and forcing feature detection and matching to be on the CPU, but you may not need to do both:

from valis import registration, feature_matcher, feature_detectors

device = "cpu"
matcher = feature_matcher.LightGlueMatcher(match_filter_method=feature_matcher.DEFAULT_RANSAC_NAME, feature_detector=feature_detectors.DiskFD(rgb=True, device=device, num_features=2000), device=device) # Run on CPU and reduce number of features detected
registrar = registration.Valis(src_dir, dst_dir, matcher=matcher)
rigid_registrar, non_rigid_registrar, error_df = registrar.register()

Please try that out and let me know if it helps improve the alignments.

Best,
-Chandler