Docker running, output error

As far as I can tell, Docker is running correctly...

sad@sad:~$ sudo docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.

When trying to run prok-cheetah, I get this error:

sad@sad:~/GALES/bin$ ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od /tmp/demo -v cheetah -rd /dbs
Starting pipeline. If this is your first time running this pipeline there will be some latency as the necessary Docker images are downloaded. This is a one-time cost.
Running command: cwl-runner --outdir /tmp/demo /tmp/demo/prok-annotation-cheetah.cwl /tmp/demo/prok-annotation-cheetah.json
/usr/local/lib/python2.7/dist-packages/cwltool/init.py:17: CWLToolDeprecationWarning:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as the Python 2.7 version of cwltool won't be
maintained after that date.

""", category=CWLToolDeprecationWarning)
INFO /usr/local/bin/cwl-runner 1.0.20190815141648
INFO Resolved '/tmp/demo/prok-annotation-cheetah.cwl' to 'file:///tmp/demo/prok-annotation-cheetah.cwl'
INFO [workflow ] start
INFO [workflow ] starting step barrnap
INFO [step barrnap] start
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.38/images/json?all=1: dial unix /var/run/docker.sock: connect: permission denied
ERROR Workflow error, try again with --debug for more information:
Docker is not available for this tool, try --no-container to disable Docker, or install a user space Docker replacement like uDocker with --user-space-docker-cmd.: Command '['docker', 'images', '--no-trunc', '--all']' returned non-zero exit status 1.
ERROR: Return code 1 when running the following command: cwl-runner --outdir /tmp/demo /tmp/demo/prok-annotation-cheetah.cwl /tmp/demo/prok-annotation-cheetah.json

I looked at a similar issue, but have unable to resolve it. I am running Linux Mint, rebooted as well. If it's not Docker, it would appear to be permissions. What would I change where if that is the case?

[geraldmc]

Try running the cwl command directly:

cwl-runner --outdir /tmp/demo /tmp/demo/prok-annotation-cheetah.cwl /tmp/demo/prok-annotation-cheetah.json (or whatever the error message says the correct paths are).

[jorvis]

That's interesting, if you were able to do the test. Make sure your system as a 'docker' group and that your user is added to it, then restart your machine.

I ran this line: sad@sad:~/GALES/bin$ cwl-runner --outdir /tmp/demo /tmp/demo/prok-annotation-cheetah.cwl /tmp/demp/prok-annotation-cheetah.json Output error /usr/local/lib/python2.7/dist-packages/cwltool/__init__.py:17: """, category=CWLToolDeprecationWarning) INFO /usr/local/bin/cwl-runner 1.0.20190815141648 Traceback (most recent call last): File "/usr/local/bin/cwl-runner", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line 644, in main fetcher_constructor=loadingContext.fetcher_constructor) File "/usr/local/lib/python2.7/dist-packages/cwltool/load_tool.py", line 88, in resolve_tool_uri raise ValidationException("Not found: '%s'" % argsworkflow) schema_salad.validate.ValidationException: *Not found: '/tmp/demo/prok-annotation-cheetah.cwl'* I do get a deprecation warning as I used CWL with Python 2.7. Would reinstalling CWL with Python 3 help any? As for file not found, I'm entirely unclear as how to remedy this :( Stephanie

…

On Sun, Aug 18, 2019 at 6:50 PM Joshua Orvis ***@***.***> wrote: That's interesting, if you were able to do the test. Make sure your system as a 'docker' group and that your user is added to it, then restart your machine. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSUYL5PFPM2FXCY563LQFH37FA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4RO5GI#issuecomment-522383001>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSW27APHOCSKLJNRISLQFH37FANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

That error is reporting a file not missing. Does this file appear?

ls -l /tmp/demo/prok-annotation-cheetah.cwl

It is located in ~/GALES/cwl/workflows Not in a tmp/demo folder (no tmp folder in GALES that I am aware of)

…

On Mon, Aug 19, 2019 at 6:54 PM Joshua Orvis ***@***.***> wrote: That error is reporting a file not missing. Does this file appear? ls -l /tmp/demo/prok-annotation-cheetah.cwl — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSRK4WVBMGPUDOWAAD3QFNFEPA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4UZTPI#issuecomment-522820029>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSSUOZZKGAVETGTAMLDQFNFEPANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

No, that's where the source copy exists. It is populated with files for YOUR run and then placed in the directory where you specify with the -od option. In your command, you said "-od /tmp/demo". So does that directory not exist on your system?

So, no. I didn't create that directory. In that event, I will just change the -od flagged destination to see if that works. sad@sad:~/GALES/bin$ ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od */tmp/demo* -v cheetah -rd /dbs

…

On Mon, Aug 19, 2019 at 7:24 PM Joshua Orvis ***@***.***> wrote: No, that's where the source copy exists. It is populated with files for YOUR run and then placed in the directory where you specify with the -od option. In your command, you said "-od /tmp/demo". So does that directory not exist on your system? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSVESMBAOPR7KZWPOLLQFNIVRA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4U3AAI#issuecomment-522825729>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSV2E6WGM24WHYKMHNLQFNIVRANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

I created a folder GALES/bin/output and ran this command ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od output -v cheetah -rd /dbs Seems I have remote permission issues, though I'm unsure. *Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.38/images/json?all=1: dial unix /var/run/docker.sock: connect: permission denied* ERROR Workflow error, try again with --debug for more information: Docker is not available for this tool, try --no-container to disable Docker, or install a user space Docker replacement like uDocker with --user-space-docker-cmd.: Command '['docker', 'images', '--no-trunc', '--all']' returned non-zero exit status 1. ERROR: Return code 1 when running the following command: cwl-runner --outdir /home/sad/GALES/bin/output /home/sad/GALES/bin/output/prok-annotation-cheetah.cwl /home/sad/GALES/bin/output/prok-annotation-cheetah.json sad@sad:~/GALES/bin$

…

On Mon, Aug 19, 2019 at 7:45 PM Stephanie Dean ***@***.***> wrote: So, no. I didn't create that directory. In that event, I will just change the -od flagged destination to see if that works. ***@***.***:~/GALES/bin$ ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od */tmp/demo* -v cheetah -rd /dbs On Mon, Aug 19, 2019 at 7:24 PM Joshua Orvis ***@***.***> wrote: > No, that's where the source copy exists. It is populated with files for > YOUR run and then placed in the directory where you specify with the -od > option. In your command, you said "-od /tmp/demo". So does that directory > not exist on your system? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#21?email_source=notifications&email_token=ACTZPSVESMBAOPR7KZWPOLLQFNIVRA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4U3AAI#issuecomment-522825729>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ACTZPSV2E6WGM24WHYKMHNLQFNIVRANCNFSM4IMQRUKQ> > . > -- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

That looks like the sort of permissions issues you get when you don't have docker completely set up. Check the output of this command and you should see 'docker' among them:

$ groups

If you don't, you need to add the unix group 'docker' and then add your user to that group, then restart.

Ran the below and rebooted. I did receive earlier the hello world from docker without an apparent issues. Again, I went into GALES/bin and ran cwl-runner --outdir /output/prok-annotation-cheetah.cwl /output/prok-annotation-cheetah.json And it seems I have the correct files in the correct location for the above command. [image: image.png] sad@sad:~$ cd GALES/bin sad@sad:~/GALES/bin$ groups sad adm cdrom sudo dip plugdev lpadmin sambashare sad@sad:~/GALES/bin$ sudo groupadd docker [sudo] password for sad: groupadd: group 'docker' already exists sad@sad:~/GALES/bin$ sudo usermod -aG docker sad sad@sad:~/GALES/bin$ newgrp docker sad@sad:~/GALES/bin$ docker run hello-world Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/ sad@sad:~/GALES/bin$ sudo systemctl enable docker Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable docker

…

On Mon, Aug 19, 2019 at 9:03 PM Joshua Orvis ***@***.***> wrote: That looks like the sort of permissions issues you get when you don't have docker completely set up. Check the output of this command and you should see 'docker' among them: $ groups If you don't, you need to add the unix group 'docker' and then add your user to that group, then restart. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSQXRDLG65D7KLNBSN3QFNUH5A5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4U7NQI#issuecomment-522843841>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSTLK5F35YVYV6KBQULQFNUH5ANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

OK, you did all that, restarted, and still get the same output? You didn't post anything after these modifications.

I updated python -dev, made sure everything prior (cwl, docker, biocode) was installed correctly. I uninstalled, reinstalled and rebooted several times. As far as I can tell it is correct. So, ran this again and got the same permissions error. * ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od ../output -v cheetah -rd /dbs* Starting pipeline. If this is your first time running this pipeline there will be some latency as the necessary Docker images are downloaded. This is a one-time cost. Running command: cwl-runner --outdir /home/sad/GALES/output /home/sad/GALES/output/prok-annotation-cheetah.cwl /home/sad/GALES/output/prok-annotation-cheetah.json /usr/local/lib/python2.7/dist-packages/cwltool/__init__.py:17: CWLToolDeprecationWarning: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as the Python 2.7 version of cwltool won't be maintained after that date. """, category=CWLToolDeprecationWarning) INFO /usr/local/bin/cwl-runner 1.0.20190815141648 INFO Resolved '/home/sad/GALES/output/prok-annotation-cheetah.cwl' to 'file:///home/sad/GALES/output/prok-annotation-cheetah.cwl' INFO [workflow ] start INFO [workflow ] starting step aragorn INFO [step aragorn] start *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?* ERROR Workflow error, try again with --debug for more information: Docker is not available for this tool, try --no-container to disable Docker, or install a user space Docker replacement like uDocker with --user-space-docker-cmd.: Command '['docker', 'images', '--no-trunc', '--all']' returned non-zero exit status 1. *ERROR: Return code 1 when running the following command: cwl-runner --outdir /home/sad/GALES/output /home/sad/GALES/output/prok-annotation-cheetah.cwl /home/sad/GALES/output/prok-annotation-cheetah.json*

…

On Tue, Aug 20, 2019 at 12:51 PM Joshua Orvis ***@***.***> wrote: OK, you did all that, restarted, and still get the same output? You didn't post anything after these modifications. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSQ6WDADON4FR5ZSZGLQFRDMRA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4XOXNQ#issuecomment-523168694>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSQNWEZJUXZSFXRF6H3QFRDMRANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

I'm not sure what thing is being missed but that is a classic message for when you don't have rights to run docker.

It's not the best practice, but in a pinch you can run the same command but prefix it with 'sudo' to run it with elevated privs.

I tried it with sudo. I am at a loss and officially frustrated. sad@sad:~/GALES/bin$ sudo ./run_prok_pipeline -i ../test_data/genomes/GCA_002790715.1_ASM279071v1_genomic.fna -od ../output -v cheetah -rd /dbs Starting pipeline. If this is your first time running this pipeline there will be some latency as the necessary Docker images are downloaded. This is a one-time cost. Running command: cwl-runner --outdir /home/sad/GALES/output /home/sad/GALES/output/prok-annotation-cheetah.cwl /home/sad/GALES/output/prok-annotation-cheetah.json /usr/local/lib/python2.7/dist-packages/cwltool/__init__.py:17: CWLToolDeprecationWarning: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as the Python 2.7 version of cwltool won't be maintained after that date. """, category=CWLToolDeprecationWarning) INFO /usr/local/bin/cwl-runner 1.0.20190815141648 INFO Resolved '/home/sad/GALES/output/prok-annotation-cheetah.cwl' to 'file:///home/sad/GALES/output/prok-annotation-cheetah.cwl' INFO [workflow ] start INFO [workflow ] starting step aragorn INFO [step aragorn] start Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? ERROR Workflow error, try again with --debug for more information: Docker is not available for this tool, try --no-container to disable Docker, or install a user space Docker replacement like uDocker with --user-space-docker-cmd.: Command '['docker', 'images', '--no-trunc', '--all']' returned non-zero exit status 1. ERROR: Return code 1 when running the following command: cwl-runner --outdir /home/sad/GALES/output /home/sad/GALES/output/prok-annotation-cheetah.cwl /home/sad/GALES/output/prok-annotation-cheetah.json

…

On Tue, Aug 20, 2019 at 6:44 PM Joshua Orvis ***@***.***> wrote: I'm not sure what thing is being missed but that is a classic message for when you don't have rights to run docker. It's not the best practice, but in a pinch you can run the same command but prefix it with 'sudo' to run it with elevated privs. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21?email_source=notifications&email_token=ACTZPSW4Q3BWHVR2DIPTKMLQFSMYFA5CNFSM4IMQRUK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4YFFVA#issuecomment-523260628>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACTZPSSP5L7E3QD5RNHCMG3QFSMYFANCNFSM4IMQRUKQ> .

-- A towel, it says, is about the most massively useful thing an interstellar hitchhiker can have. Partly it has great practical value. You can wrap it around you for warmth as you bound across the cold moons of Jaglan Beta; you can lie on it on the brilliant marble-sanded beaches of Santraginus V, inhaling the heady sea vapours; you can sleep under it beneath the stars which shine so redly on the desert world of Kakrafoon; use it to sail a miniraft down the slow heavy River Moth; wet it for use in hand-to-hand-combat; wrap it round your head to ward off noxious fumes or avoid the gaze of the Ravenous Bugblatter Beast of Traal (such a mind-bogglingly stupid animal, it assumes that if you can't see it, it can't see you — daft as a brush, but very very ravenous); you can wave your towel in emergencies as a distress signal, and of course dry yourself off with it if it still seems to be clean enough. — Douglas Adams, *The Hitchhiker's Guide to the Galaxy*

[jorvis]

Let's step back a bit. What happens on the command line if you do only this:

$ docker images