Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process directive not always applying to run #3154

Closed
anoronh4 opened this issue Aug 26, 2022 · 2 comments
Closed

process directive not always applying to run #3154

anoronh4 opened this issue Aug 26, 2022 · 2 comments

Comments

@anoronh4
Copy link

anoronh4 commented Aug 26, 2022

Bug report

Under certain circumstances, with the newest version of nextflow (22.04.5), my process directives for errorStrategy and maxRetries are being ignored.

process.errorStrategy = { task.attempt <= 3 ? 'retry' : 'ignore' }
process.maxRetries = 3

is set from a config profile (using includeConfig statements), and the log says it has loaded the profile.

Expected behavior and actual behavior

Based on the directives we expect a failing process to retry or ignore. However the first process to encounter an error terminates without retrying, and kills the nextflow run, so it doesn't ignore either. Strangely, pipeline works as expected with older versions such 21.10.6.

Also a bit strange, if i move the process.* directives above into a process closure as such:

process {
...
errorStrategy = { task.attempt <= 3 ? 'retry' : 'ignore' }
maxRetries = 3
} 
// process.errorStrategy = { task.attempt <= 3 ? 'retry' : 'ignore' }
// process.maxRetries = 3

then it will work. Also, if i reorder my includeConfig commands the main nextflow config, i can also get the error to go away. So I can work around the problem but it still seems to me that there is an issue.

Steps to reproduce the problem

I found this while working on a bigger project that is not publicly available but i have reproduced it here:
https://github.com/anoronh4/nextflow_config_bug

Program output

N E X T F L O W  ~  version 22.04.5
Launching `test.nf` [prickly_panini] DSL2 - revision: efce412888
executor >  local (5)
executor >  local (5)
[a5/0ea02b] process > built2Fail (3) [100%] 5 of 5, failed: 1 ✘
WARN: There's no process matching config selector: SampleRunMultiQC
WARN: There's no process matching config selector: AggregateRunMultiQC
Error executing process > 'built2Fail (1)'

Caused by:
  Process `built2Fail (1)` terminated with an error exit status (1)

Command executed:

  echo "TERMINATE"
  if [ "DUP" == "DUP" ] ; then exit 1 ;fi

Command exit status:
  1

Command output:
  TERMINATE

Command wrapper:
  nxf-scratch-dir myserver:/scratch/anoronh4/nxf.U7VknhKC3E
  TERMINATE

Work dir:
  /path/to/mydir/bug_report/work/8e/62341efe0eb72c7b513bc36041167f

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Environment

  • Nextflow version: 22.04.5
  • Java version: 11
  • Operating system: Linux
  • Bash version: GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
@bentsherman
Copy link
Member

I think it's the same as #2966 , sometimes config options with the dot notation can be ignored if the block notation is also used. For now put everything in a block (closure) to be safe. We intend to address this problem by overhauling the config syntax (#2723).

@stale
Copy link

stale bot commented Mar 18, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants