BioHackathon-MENA MD workflow on Sapporo
Aim: Run MD workflow in nextflow, then check the reproducibility by tonkaz
Things to do
Compose the MD workflow with nextflow
Launch Sapporo
Run MD workflow on Sapporo (x2)
get ro-crate for each run and compare
Compose the MD workflow
done by Batool! https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/blob/main/test_data/moleculardymics_test.nf
Launch Sapporo
can run on M1 mac but the x86_64 container performance is too bad
sapporo container -> nextflow container -> conda env
successfully created the nextflow container in ARM64 but conda packages are not available
Sapporo running on a remote server (AMD64)
use curl to send a request
passing the main nf workflow file as a attachment, nextflow does not accept the nf file in a random location but only in a github repo with the name main.nf
require conda environment to install dependencies
the nextflow container image (nextflow/nextflow:22.04.4) does not contain conda environment
needed to create the nextflow-with-conda image to run the wf
https://github.com/inutano/docker-nextflow-with-conda/
https://github.com/inutano/docker-nextflow-with-conda/pkgs/container/nextflow-with-conda
done: multi-platform build in github actions
ref: https://docs.docker.com/build/ci/github-actions/examples/#multi-platform-images
code:bash
$ ls
tags.json workflow_attachment.json workflow_engine_parameters.json workflow_params.json
$ jq . tags.json
{}
$ jq . workflow_attachment.json [
{
"file_name": "moleculardymics_test.nf",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/moleculardymics_test.nf"
},
{
"file_name": "input.pdb",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/input.pdb"
},
{
"file_name": "ions.mdp",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/ions.mdp"
},
{
"file_name": "md.mdp",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/md.mdp"
},
{
"file_name": "minim.mdp",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/minim.mdp"
},
{
"file_name": "npt.mdp",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/npt.mdp"
},
{
"file_name": "nvt.mdp",
"file_url": "https://github.com/Open-Science-Community-Saudi-Arabia/moleculardymics/raw/main/test_data/nvt.mdp"
}
]
$ jq . workflow_engine_parameters.json
{
"-dsl1": "",
"-with-conda": "true"
}
$ jq . workflow_params.json
{}
$ curl -X POST \
-F "workflow_type=NFL" \
-F "workflow_type_version=1.0" \
-F "workflow_url=moleculardymics_test.nf" \
-F "workflow_params=<workflow_params.json" \
-F "workflow_engine_name=nextflow" \
-F "workflow_engine_parameters=<workflow_engine_parameters.json" \
-F "tags=<tags.json" \
-F "workflow_attachment=<workflow_attachment.json" \
localhost:1122/runs