Installation problems

[bisnow33]

Hi ,
i'm trying to use your script count_probes.py on Ubuntu but there is something wrong. The most errors (around 30 lines) i got concern the numpy module with this kind of message :

/home/user/env/local/lib/python2.7/site-packages/scipy/linalg/basic.py:17: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._solve_toeplitz import levinson

With one of my pictures i got one result in the output directory named projection.png , so i was wondering if it could be a problem with my pictures ? If it's the case can you point me a link with a picture working with count_probes.py ?

Best regards,

Tristan

[tjelvar-olsson]

Hi Tristan,

What types of images have you collected? This tool assumes that you are working with a confocal 3D image stack.

The first step of the process is to convert your microscopy image into a bunch of tiff files. This is achieved using the script scripts/protoimg/unpack.py.

Did you manage to use this script to convert your microscopy image into a set of tiff files?

I'm not sure the error messages you see are to do with the software. They seem to depend on how the numpy/scipy/scikit-image stack was installed on your system. For more details see:

https://stackoverflow.com/a/40846742

and

numpy/numpy#11628

Best,
Tjelvar

[tjelvar-olsson]

Hi Tristan,

I've been in touch with Susan who did a lot of the experimental work. She has uploaded a load of microscopy images to DataDryad.

https://datadryad.org/resource/doi:10.5061/dryad.9pq6k

You should be able to test the tool using one of the PP2A stacks from there.

Best,
Tjelvar

[bisnow33]

Hi Tjelvar,
Thank you for your help. I updated the numpy with the 1.8.0 and i don't have the problem anymore.
I downloaded some data you linked ( thanks again ) to test the installation but now i got other errors when i use scripts/protoimg/unpack.py :

File "scripts/protoimg/unpack.py", line 76, in
main()
File "scripts/protoimg/unpack.py", line 73, in main
unpack_many(args.image_dir, args.data_root)
File "scripts/protoimg/unpack.py", line 36, in unpack_many
unpack(image_path, output_path)
File "scripts/protoimg/unpack.py", line 61, in unpack
subprocess.call(unpack_cmd)
File "/usr/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 394, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

I'm sorry for that "stupid" but i don't see what i did wrong ....

Best
Tristan

[bisnow33]

Hi again Tjelvar,
well i solved the problems concerning unpack.py, i still have the this message (with the images you linked me) but i don't think it's important for the next step ;
Unknown DetectorType value 'Camera' will be stored as "Other"
Unknown DetectorType value 'Camera' will be stored as "Other"
Unknown DetectorType value 'Camera' will be stored as "Other"

So now during the next step whe i run python ./scripts/count_probes.py i have this error messages :

Traceback (most recent call last):
File "./scripts/count_probes.py", line 112, in
main()
File "./scripts/count_probes.py", line 109, in main
imsave=imsave_with_outdir, pchannel=pchannel)
File "./scripts/count_probes.py", line 80, in segment_count_annotate
segmentation = load_stack_and_segment(stack_path, imsave)
File "/home/tridubos/Téléchargements/FISHcount-master/scripts/segmentation_from_stack.py", line 130, in load_stack_and_segment
return segmentation_from_stacks(nuclear_stack, autof_stack, imsave)
File "/home/tridubos/Téléchargements/FISHcount-master/scripts/segmentation_from_stack.py", line 116, in segmentation_from_stacks
seeds = generate_segmentation_seeds(nuclear_stack)
File "/home/tridubos/Téléchargements/FISHcount-master/scripts/segmentation_from_stack.py", line 48, in generate_segmentation_seeds
eq_proj = equalize_adaptive(max_nuclear_proj, n_tiles=16, name='equalized_nuclear_proj')
File "/home/tridubos/Téléchargements/FISHcount-master/scripts/protoimg/transform.py", line 105, in equalize_adaptive
clip_limit=clip_limit)
File "/home/tridubos/env/local/lib/python2.7/site-packages/skimage/exposure/_adapthist.py", line 82, in equalize_adapthist
out = _clahe(*args)
File "/home/tridubos/env/local/lib/python2.7/site-packages/skimage/exposure/_adapthist.py", line 152, in _clahe
hist = clip_histogram(hist, clip_limit)
File "/home/tridubos/env/local/lib/python2.7/site-packages/skimage/exposure/_adapthist.py", line 250, in clip_histogram
hist[under] += 1
IndexError: boolean index did not match indexed array along dimension 0; dimension is 256 but corresponding boolean dimension is 26

Best ,

Tristan

[tjelvar-olsson]

Hi Tristan,

Which version of OME tools are you using?

I think I remember seeing an error message like this at some point and I think it had to do with the way that some version of the OME tools started interpreting the false colour map in the microscopy image.

I think the code in the branch "ome-tools-v5" fixes this problem.

https://github.com/JIC-CSB/FISHcount/tree/ome-tools-v5

Best,
Tjelvar

[bisnow33]

Hi Tjelvar,

I tried to download the other folder but i got same errors. I checked the scripts with md5sum and it seems that only 2 scripts are changed between the 2 versions : find_probe_locs.pyc and segmentation_from_stack.pyc.

Do you have other idea ?

Best

Tristan

[tjelvar-olsson]

Hi Tristan,

This is the difference in the scripts that you should see:

1ef8dca

Note line 58.

Best,
Tjelvar

[bisnow33]

Hi ,
I check the difference this morning it's ok but i have still errors :

Traceback (most recent call last):
File "./scripts/count_probes.py", line 112, in
main()
File "./scripts/count_probes.py", line 109, in main
imsave=imsave_with_outdir, pchannel=pchannel)
File "./scripts/count_probes.py", line 80, in segment_count_annotate
segmentation = load_stack_and_segment(stack_path, imsave)
File "/home/tridubos/Téléchargements/FISHcount-ome-tools-v5/scripts/segmentation_from_stack.py", line 127, in load_stack_and_segment
autof_stack = Stack.from_path(path, channel=2)
File "/home/tridubos/Téléchargements/FISHcount-ome-tools-v5/scripts/protoimg/stack.py", line 41, in from_path
stack_array = read_stack_array(path, channel)
File "/home/tridubos/Téléchargements/FISHcount-ome-tools-v5/scripts/protoimg/imgutil.py", line 58, in read_stack_array
stack_array = np.dstack(all_image_arrays)
File "/home/tridubos/env/local/lib/python2.7/site-packages/numpy/lib/shape_base.py", line 421, in dstack
return _nx.concatenate([atleast_3d(_m) for _m in tup], 2)
ValueError: need at least one array to concatenate

Best ,

Tristan

[tjelvar-olsson]

Hi Tristan,

Did you use the new version of the script to convert your microscopy file to a bunch of tiff files. I.e. did you rerun the step below:

python scripts/protoimg/unpack.py lif_input_dir tiff_output_dir

It might be worth deleting the previously generated .tiff files first to make sure that they get regenerated with the new script.

Best,
Tjelvar

[bisnow33]

Hi Tjelvar,
Yes i used the new unpack.py version.
I think i'm not using good version of module but i don't understand which one.
When i install this :

pip install pillow
pip install scipy
pip install six
pip install "scikit-image==0.10.1"
pip install freetype-py
pip install libtiff
pip install numpy==1.7.1

i have that for the unpack :

 python scripts/protoimg/unpack.py /home/tridubos/Bureau/aa/ /home/tridubos/Bureau/aa2
/home/tridubos/Bureau/aa/DL21-08-2018.czi
ZeissCZIReader initializing /home/tridubos/Bureau/aa/DL21-08-2018.czi
Unknown DetectorType value 'Camera' will be stored as "Other"
Unknown DetectorType value 'Camera' will be stored as "Other"
Unknown DetectorType value 'Camera' will be stored as "Other"
[Zeiss CZI] -> /home/tridubos/Bureau/aa2/DL21-08-2018/DL21-08-2018_S%s_C%c_Z%z.tif [Tagged Image File Format]
	Converted 41/228 planes (17%)
	Converted 88/228 planes (38%)
	Converted 135/228 planes (59%)
	Converted 204/228 planes (89%)
	Converted 228/228 planes (100%)
[done]
4.981s elapsed (1.495614+17.59649ms per plane, 559ms overhead) 

But then when i run :
python ./scripts/count_probes.py /home/tridubos/Bureau/aa2/ /home/tridubos/Bureau/ImageTEST/aa_OUT

/home/tridubos/.local/lib/python2.7/site-packages/scipy/__init__.py:115: UserWarning: Numpy 1.8.2 or above is recommended for this version of scipy (detected version 1.7.1)
  UserWarning)
RuntimeError: module compiled against API version 9 but this version of numpy is 7
Traceback (most recent call last):
  File "./scripts/count_probes.py", line 8, in <module>
    import scipy.misc
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/misc/__init__.py", line 68, in <module>
    from scipy.interpolate._pade import pade as _pade
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/interpolate/__init__.py", line 175, in <module>
    from .interpolate import *
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 20, in <module>
    import scipy.linalg
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/linalg/__init__.py", line 190, in <module>
    from .misc import *
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/linalg/misc.py", line 5, in <module>
    from .blas import get_blas_funcs
  File "/home/tridubos/.local/lib/python2.7/site-packages/scipy/linalg/blas.py", line 214, in <module>
    from scipy.linalg import _fblas
ImportError: numpy.core.multiarray failed to import

I saw that when i install scipy it's updating numpy ... Can you give me the version of module you have ?

Best ,

Tristan

[ppouchin]

Hi,

To make it work, I had to use scikit-image 0.12.3 (replacing calls to skimage.filter by calls to skimage.filters in transform.py).
I also had to edit line 89 of the file segmentation_from_stack.py because the name (4th argument) is passed as a positional argument, which does not work without arguments 2 and 3...

[tjelvar-olsson]

@bisnow33 It looks like your scipy/numpy versions are incompatible.

I should have created a proper requirements.txt file when I was working on this project (hindsight...). Can you try using the versions listed below?

libtiff==0.4.0
numpy==1.10.4
Pillow==3.1.1
scikit-image==0.11.3
scipy==0.17.0

@ppouchin great to hear that you got it to work! :)

[bisnow33]

Hi Tjelvar,

So i found the solution. So with the modifications of @ppouchin you have to install in order :

pip install pillow  
pip install six  
pip install freetype-py  
pip install "scikit-image==0.12.3"  
pip install --no-binary numpy numpy==1.13.3  
pip install --no-binary scipy scipy==1.1.0  
pip install --no-binary libtiff libtiff==0.4.2  

I think there is dependency between modules specially with libtiff scipy ans numpy , is it why the order is important ?

There is an other modification to do , you have to replace skimage.filters.gaussian_filter by skimage.filters.gaussian in transform.py.

Best

[domenico-simone]

Hello,

I'm trying to get the tool to work but I get the same errors I see in this thread. Is there any way to get the edited scripts (as they are supposed to work) and the exact versions of the needed modules, eg to create a conda environment?

Thanks

Regards,

Domenico

[ppouchin]

Have you tried @tjelvar-olsson requirements (with no code modifications)?
#1 (comment)

If I remember correctly, I followed the README, but had to modify the code because without pinned versions, I got an incompatible version of scikit-image (too recent). So I went back to 0.12.3, which still was a bit different from 0.11.3. So I had to edit transform.py and:

• replace skimage.filter by skimage.filters
• replace skimage.filters.gaussian_filter by skimage.filters.gaussian_filter .

Although I still think you would need to edit line 89 of the file segmentation_from_stack.py anyway: replace 'equal_autof' by name='equal_autof'.