Invalid S3 URL when running ControlFREEC

[szymonwieloch]

When running ControlFREEC I got the following error message:

[null] NOTE: Can't stage file s3:///ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/Control-FREEC/out100m2_hg38.gem -- file does not exist -- Execution is retried (1)
-[nf-core/sarek] Pipeline completed with errors-
[null] NOTE: Can't stage file s3:///ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/Control-FREEC/out100m2_hg38.gem -- file does not exist -- Execution is retried (1)
Unable to re-submit task `ControlFREEC (a35760da-3f13-40a3-8537-e7e841baa6a1_sample_vs_a35760da-3f13-40a3-8537-e7e841baa6a1_control)`
Unable to re-submit task `ControlFREEC (e8f63b36-a0b9-406e-905e-69b7621f33ba_sample_vs_e8f63b36-a0b9-406e-905e-69b7621f33ba_control)`

Indeed the mentioned URL seems to refer to a non-existing file:

aws s3 ls s3://ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/
                           PRE ASCAT/
                           PRE GATKBundle/
                           PRE GermlineResource/
                           PRE intervals/

This URL seems to be defined in the conf/igenomes.config file:

mappability             = "${params.igenomes_base}/Homo_sapiens/GATK/GRCh38/Annotation/Control-FREEC/out100m2_hg38.gem"

My guess is that it used to work but then somebody rearranged files on the S3 bucket and now it does not. You probably need to update the URLs.

[maxulysse]

Hi, I found what was the issue with that, I have them in an s3 bucket, but the sync to iGenomes must have had an issue, I'll ask @ewels about that

[maxulysse]

@ewels Is there any update on the sync of s3://rnafusion-sarek to AWS iGenomes?

[maxulysse]

@XLuyu @szymonwieloch files should be there now.
Can you check if it works for you?

[ewels]

Sync done ♻️ - sorry it took me so long!

[maxulysse]

Sync done recycle - sorry it took me so long!

No worries, I should have make sure it was there to begin with.

[szymonwieloch]

@maxulysse @ewels Hi, it seems that it works correctly. At least the problem with missing files is gone, now I am struggling with some memory issues and I haven't run the pipeline yet, but clearly the previous problem is gone. Thank you for fixing this!

[ewels]

Great stuff, thanks for letting us know!