December 2020 Meetup of the NL-RSE Community.
Location
The call will be conducted with the ZOOM conferencing platform. Follow Joining the Meeting instructions.
Zoom link will be published soon.
Registration
The registration is free
Please register via this Eventbrite page.
Agenda
| Time | Responsible | Type | Subject |
|---|---|---|---|
| 09:00 - 09:15 | hello & coffee | ||
| 09:15 - 09:20 | Mateusz Kuzak | Welcome and purpose | |
| 09:20 - 09:25 | Mateusz Kuzak | Agenda and ground rules | |
| 09:25 - 09:30 | everyone | Introductions and announcements | |
| 09:30 - 12:30 | Mattia Mancini | tutorial | From a bash script to CWL |
| 12:30 - 12:35 | discussion | Parking Lot | |
| 12:35 - 12:40 | discussion | Evaluation |
Abstracts
From a bash script to CWL
In modern experimental sciences, the increasing amount of data to process and the development of standardized procedures to analyze it, is driving the utilization of automatic processing pipelines. Such pipelines combine different software components to carry out the computation. A wide range of solutions, ranging from bash scripts to python, can be used to perform such orchestration.
In ASTRON for the European Open Science Cloud project, we chose to use the Common Workflow Language (CWL) to describe one of our standard processing pipelines.
In this demo and tutorial, we will show our experience in converting our pipeline into the CWL (v1.0) standard and our experience with the available tools to run it and visualize it. We will follow by showing you how to create a CWL step and a CWL pipeline to describe an example workflow. Finally, we will conclude the meeting with our take on why we choose CWL and why this proved to be fit to solve our problems, and we would be welcome your opinions about that.
Our use case comes from astronomy, but this tutorial is focused on transferable skills which can be applied in other domains.
For the tutorial, we will be using google colab please make sure that you can execute it in your browser.
If you want to run the software on your computer, you will require the following tools installed on your laptop:
- Python 3.6 (or higher)
- cwltool (to execute the workflow) - we recommend to use it in a virtual environment (here is a guide on how to install it)
- singularity/docker (in case of the software we use in the demo is not available for your system,
here are install instructions for singularity and docker) -- if you are not running linux/mac we strongly suggest using google colab!
As there is not much time during the tutorial, we suggest that, if you want to run the software on your computer, you make sure that you can run this example and docker or singularity is available.
Call for Contributions
We are always looking for speakers and exciting topics, please contact the host Mateusz Kuzak if you would like to give a talk or are interested in learning about some specific topics.
Contact
If you have any questions, please contact the host Mateusz Kuzak
NL-RSE. The community of Research Software Engineers from Dutch universities, knowledge institutes, companies and other relevant organizations for sharing knowledge, organizing meetings and raising awareness for the scientific recognition of research software.