Extend the frontend to support SnakeMake export introduced in RestAPE PR 95

[eladrion]

Overview

This PR extends the results view with the possibility to retrieve the SnakeMake files generated by APE via RestAPE.
Also, the ZIP file now contains both CWL and SnakeMake files.

Changes

Extended the GenerationResults to show download links SMK for SnakeMake files next to the CWL download links.
Added snakemake_name to WorkflowSolution type.

Testing

Tested on the institutional private instance by generating results and downloading them

Checklist

• Code follows project standards.
• Self-reviewed and commented where needed.
• Documentation updated if needed.
• No new warnings or errors.

[vedran-kasalica]

HI @eladrion, it's a great idea to integrate Snakemake as an option to be downloaded. I would be a bit careful in the way that it is presented to the user. Only a small number of domains currently generate executable CWL files, and for those that don't (e.g., proteomics non-executable), there is a (really basic) disclaimer
stating the limitation.

Providing for each domain, both CWL and Snakemake, might be confusing (especially if one format is better annotated/executable than the other) and probably not needed. The same goes for the zip file with both formats. An idea could be to introduce some metadata on the domain level that specifies which formats are desirable or supported based on the annotation, and then just provide those to the user. What do you think?

In addition, after thinking about it a bit more, I have another question :) The main reason we focused on the CWL was to allow automated execution/benchmarking of workflows, is that still the idea? If it is, exeution of multiple Snakemake files should be supported, and I assume that the workflomics-benchmarker should be extended to support execution of Snakemake workflows as well. Disclaimer, that library is not in the best shape. And if it's not, it would be good to make a clear distinction in the main explore menu, while choosing the domain, which workflow language is supported by the domain and whether they are "executable" and maybe even what it entails.

[eladrion]

Hi @vedran-kasalica, you're right. Users may expect to get executable workflows. Something we may not be able to accommodate due to missing tool annotations. I think it is already helpful to give users an Idea of what could work but disclaiming that workflows may not be executable. If they really require executable workflows, we need to accommodate this. In APE 2.6, I (luckily) introduced a flag to restrict the generated files (CWL/SMK/sh) to those where the implementation (CWL) or code (SMK, sh) is provided. We could add a configuration checkbox "Restrict to executable workflows" in the results config which would be domain agnostic. We could extend RestAPE and Workflomics-frontend to give the users the choice. What do you think about the idea? I would then also introduce this as part of RestAPE 1.1 and Workflomics-frontend 1.2.

[vedran-kasalica]

That is a nice idea. Maybe the button can be disabled in case the domain does not support any for example, but that might be a bit tricky. I was thinking, do we want users to choose between CWL and Snakemake, or it's too many choices? This might be a question for users :)

[eladrion]

Hi @vedran-kasalica. I am not sure about the choice. Actually, we currently support CWL, Snakemake and shell scripts but only CWL and Snakemake is available in the frontend. The only viable option (usability-wise) I see is giving in the configuration section the choice by listing checkboxes for each supported output. Concerning deactivating the button, this is rather complex. We would have to store somewhere (probably in the db) whether the domain has tools with this annotation. So we would have to do an initialisation phase where we collect the current information about the domain (during startup). But then, changes in the tool annotations may invalidate our information until the next startup phase. I do not see an easy solution here and actually, the tool annotation is freshest at time of request.

[vedran-kasalica]

That's a good point @eladrion, a dynamic "disable" button would be quite hard to implement. Maybe a simple solution is to focus on CWL and Snakemake, always generate both, and instead of having now a generic executable field in the db, we can have executable_cwl and executable_snakemake. We are currently using that field to show that proteomics non-executable disclaimer, so instead, we can have a disclaimer per format that we show only when the user tries to download that format (this could improve the current disclaimer design), what do you think?

I know you are also short on time, so I can help with the front end, but I don't have the database running locally, so it might take me some time to set it up for local development.

The reason we abandoned the .sh outputs was that it was not a standardised language, which made writing documentation and instructions quite hard + the scripts were not easy to share or read. We only had 2 use cases for it (ImageMagick and GMT), both annotated and used only by our group.