Installing on M1 macs is nearly impossible

[evan-wehi]

PyQt5 is nearly impossible to install on M1 macs and it doesn't seem like anyone cares to fix it. This means Cell-ACDC is nearly impossible to install. For example, I gave up. See this discussion here: https://stackoverflow.com/questions/65901162/how-can-i-run-pyqt5-on-my-mac-with-m1chip-ppc64el-architecture. That thread is over a year old and most of the work arounds don't actually work any more. Others that might work are convoluted and clumsy. How hard would it be to port Cell-ACDC to PyQt6?

[ElpadoCan]

Hi Evan, I'm sorry to hear you are having these issues, but we should be able to fix it.

I have a mac M1 myself and I have Cell-ACDC installed. Did you try with conda from conda-forge? You can use the command conda install -c conda-forge pyqt. Can you try that in a new test env with python 3.9? If it works everything else should work, but you need to force cellacdc to be installed without dependencies and then install them manually. Otherwise pip will not realise that PyQt5 is already installed and will try to install it. From pip it never worked for me.

To install cellacdc without dependencies you can run the command pip install --no-deps cellacdc.

Then you can install the dependencies manually. The good thing at this point is that you can mix and match conda and pip by installing the package from either conda or pip depending on what works. For conda I recommend always installing from the conda-forge channel (with conda install -c conda-forge).

These are the packages needed:

numpy
opencv-python-headless
natsort
h5py
pyqtgraph>=0.12.4
scikit-image>=0.18
tqdm
seaborn
scikit-learn
psutil
boto3
requests
setuptools-scm

Additionally, depending on the segmentation model you want to use, you also will need tensorflow and/or pytorch.

I know it is not ideal but it's the only way I found that works at the moment. Let me know how it goes!

For PyQt6, it is a fantastic idea, is the installation easier on mac M1? Because then I will start working on that immediately, I never thought about it. It should not be too hard it's a matter of few attributes I think.

I will create an issue about that where you can track the progress.

Cheers,
Francesco

[ElpadoCan]

Hi Evan, I want to thank you for opening this discussion because I discovered this amazing library that provides support for all the major Qt bindings library (including PyQt6).

The change should be very straightforward, I will definitely do it next week. Basically, it will give users the freedom to install any of the supported GUI libraries. You can keep track of the progress on issue #321. Feel free to write there should you have any questions or comments. Have a good day/evening/night depending on your time-zone :D.

Best,
Francesco

[evan-wehi]

Hi Francesco, Thanks for the reply and also thanks for fixing it. I think the problem depends on the history of your python and conda environments. It took a while for the conda community to provide a native silicon conda so if your conda installation is still an intel version you will get the intel python and intel pyqt5 which works. I upgraded my conda to native silicon because I needed the performance for some things I was doing. Building PyQt5 for silicon fails. Most of the work arounds in the SO thread involve reverting to an intel version of python. It’s possible, but a lot of effort. BTW, this came about because I presented your paper at our journal club and I wanted to try it out but couldn’t. We are an immunology lab so can I put in a vote for better support for symmetric division 😊 Evan.

[ElpadoCan]

Hi Evan, I agree, the whole Apple Silicon experience is still pretty rough in some cases. However, on my mac M1 I have native arm conda and pyt5 works fine from conda-forge.

I used to have the Intel version of python, conda etc, but since I upgraded to the latest macOS Ventura 13.3 I wanted to try the arm version and the only pyqt5 that worked is from conda-forge. I think the latest macOS is also a requirement in this case. Personally, I manage python installations with pyenv (installed from homebrew) and it works very nicely.

Anyway, it's definitely not the best experience since you have to install packages manually (I wonder if they will ever add the functionality to pip to make it aware of packages already installed with conda) so I'm happy I found the qtpy library that supports all the major Qt bindings.

It's great to hear that my paper gets presented in Journal clubs 😁. My greatest fear was that Cell-ACDC would have been a useless effort but it seems like there is an interest which gives me more energy to continue to do this.

In any case, I totally agree, we need better support for symmetric cell division and I already have some interesting ideas on how to do that. We wrote a grant for exactly that reason so fingers crossed. We actually collaborate with an immunology lab that works with live cell imaging of T-cells and they are quite happy with acdc. They already adopted it for the entire pipeline, including annotating cell division.

Anyway, I will keep you updated on the progress and feel free to reach out whenever you want.

Best,
Francesco

[evan-wehi]

Hi Francesco, Following these instructions I was able to get it running – looks very nice and very professional. I didn’t try it beyond getting the UI to launch because I don’t have any data at hand. It’s always a gamble with these sorts of open source projects. The person who drives the project has to leave it because there is no money and project fades away. Our lab has done filming in the past, before the recent revolution in computer vision, and it was a lot of work. This is why I thought they might be interested in your paper. The lab also has a lot of manually annotated data which can potentially be used for transfer learning. I might see if I can get a student to try out cell-acdc with our existing data. Yes, please keep me updated on your work! Evan.

…

-- Evan Thomas Mathematical Biologist, Hodgkin lab The Walter and Eliza Hall Institute of Medical Research Mobile: +61423000246 | Email: ***@***.******@***.***> From: Francesco Padovani ***@***.***> Date: Friday, 5 May 2023 at 16:35 To: SchmollerLab/Cell_ACDC ***@***.***> Cc: Evan Thomas ***@***.***>, Author ***@***.***> Subject: Re: [SchmollerLab/Cell_ACDC] Installing on M1 macs is nearly impossible (Discussion #320) Hi Evan, I'm sorry to hear you are having these issues, but we should be able to fix it. I have a mac M1 myself and I have Cell-ACDC installed. Did you try with conda from conda-forge? You can use the command conda install -c conda-forge pyqt. Can you try that in a new test env with python 3.9? If it works everything else should work, but you need to force cellacdc to be installed without dependencies and then install them manually. Otherwise pip will not realise that PyQt5 is already installed and will try to install it. From pip it never worked for me. To install cellacdc without dependencies you can run the command pip install --no-deps cellacdc. Then you can install the dependencies manually. The good thing at this point is that you can mix and match conda and pip by installing the package from either conda or pip depending on what works. For conda I recommend always installing from the conda-forge channel (with conda install -c conda-forge). These are the packages needed: numpy opencv-python-headless natsort h5py pyqtgraph>=0.12.4 scikit-image>=0.18 tqdm seaborn scikit-learn psutil boto3 requests setuptools-scm Additionally, depending on the segmentation model you want to use, you also will need tensorflow and/or pytorch. I know it is not ideal but it's the only way I found that works at the moment. Let me know how it goes! For PyQt6, it is a fantastic idea, is the installation easier on mac M1? Because then I will start working on that immediately, I never thought about it. It should not be too hard it's a matter of few attributes I think. I will create an issue about that where you can track the progress. Cheers, Francesco — Reply to this email directly, view it on GitHub<#320 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFIAGDIHKQOGS7APCGIYV7DXESNSXANCNFSM6AAAAAAXWSSG5M>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ElpadoCan]

Hi Evan,

That's great, I'm very happy to hear you could install it and that you like the UI! Let me know how it goes when you try with some data.

Note that unfortunately, I still haven't found a solution for the first module that uses Bio-Formats (written only in Java) to convert raw microscopy files into the data structure required by Cell-ACDC. Unfortunately, the library javabridge (used to run Java commands into Python) used by the package python-bioformats is not supported on Mac M1. I haven't tried in the last couple of months though, I will check if there is some progress on that front.

In the meantime, you will have to create the data structure with ImageJ/Fiji. The data structure is recommended only if you plan to use Cell-ACDC extensively. For testing out, you can drag and drop a TIFF file.

I agree about the open-source projects. We are now trying hard to get Cell-ACDC to survive beyond me, let's see how it goes. Have a good day!

Best,
Francesco

[ElpadoCan]

Hi Evan,

I just wanted to let you know that I tried module 0. of Cell-ACDC (where you can use BioFormats to convert a microscopy file and create the required data structure) and it works! And it works out of the box! Just click on "Use BioFormats" and Cell-ACDC will automatically install the required libraries. I guess that when I tried some months ago I was still on the Intel version of Python who knows...

Cheers,
Francesco

[evan-wehi]

It’s always nice when something just works! 😊

…

-- Evan Thomas Mathematical Biologist, Hodgkin lab The Walter and Eliza Hall Institute of Medical Research Mobile: +61423000246 | Email: ***@***.******@***.***> From: Francesco Padovani ***@***.***> Date: Friday, 12 May 2023 at 18:15 To: SchmollerLab/Cell_ACDC ***@***.***> Cc: Evan Thomas ***@***.***>, Author ***@***.***> Subject: Re: [SchmollerLab/Cell_ACDC] Installing on M1 macs is nearly impossible (Discussion #320) Hi Evan, I just wanted to let you know that I tried module 0. of Cell-ACDC (where you can use BioFormats to convert a microscopy file and create the required data structure) and it works! And it works out of the box! Just click on "Use BioFormats" and Cell-ACDC will automatically install the required libraries. I guess that when I tried some months ago I was still on the Intel version of Python who knows... Cheers, Francesco — Reply to this email directly, view it on GitHub<#320 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFIAGDNYF437OEH452XA2CLXFXWQXANCNFSM6AAAAAAXWSSG5M>. You are receiving this because you authored the thread.Message ID: ***@***.***>