forked from ga4gh/workflow-execution-service-schemas
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathworkflow_execution_service.swagger.yaml
657 lines (600 loc) · 24.4 KB
/
workflow_execution_service.swagger.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
basePath: '/ga4gh/wes/v1'
swagger: '2.0'
info:
title: Workflow Execution Service
version: 1.0.0
x-logo:
url: 'https://www.ga4gh.org/wp-content/themes/ga4gh-theme/gfx/GA-logo-horizontal-tag-RGB.svg'
description: >
## Executive Summary
The Workflow Execution Service (WES) API provides a standard way for users to submit workflow requests to workflow execution systems, and to monitor their execution. This API lets users run a single workflow (currently [**CWL**](https://www.commonwl.org/) or [**WDL**](http://www.openwdl.org/) formatted workflows, other types may be supported in the future) on multiple different platforms, clouds, and environments.
Key features of the API:
- can request that a workflow be run
- can pass parameters to that workflow (e.g. input files, cmdline arguments)
- can get information about running workflows (e.g. status, errors, output file locations)
- can cancel a running workflow
## Introduction
This document describes the WES API and provides details on the specific endpoints, request formats, and response. It is intended to provide key information for developers of WES-compatible services as well as clients that will call these WES services.
Use cases include:
- "Bring your code to the data": a researcher who has built their own custom analysis can submit it to run on a dataset owned by an external organization, instead of having to make a copy of the data
- Best-practices pipelines: a researcher who maintains their own controlled data environment can find useful workflows in a shared directory (e.g. Dockstore.org), and run them over their data
## Standards
The WES API specification is written in OpenAPI and embodies a RESTful service philosophy. It uses JSON in requests and responses and standard HTTP/HTTPS for information transport.
## Authorization and Authentication
Users must supply credentials that establish their identity and authorization in order to use a WES endpoint. We recommend that WES implementations use an OAuth2 [**bearer token**](https://oauth.net/2/bearer-tokens/), although they can choose other mechanisms if appropriate. WES callers can use the `auth_instructions_url` from the [**`service-info` endpoint**](https://ga4gh.github.io/workflow-execution-service-schemas/#/WorkflowExecutionService/GetServiceInfo) to learn how to obtain and use a bearer token for a particular implementation.
<br><br>
The WES implementation is responsible for checking that a user is authorized to submit workflow run requests. The particular authorization policy is up to the WES implementer.
<br><br>
Systems like WES need to also address the ability to pass credentials with jobs for input and output access. In the current version of WES, the passing of credentials to authenticate and authorize access to inputs and outputs, as well as mandates about necessary file transfer protocols to support, are out of scope. However, parallel work on the Data Object Service is addressing ways to pass around access credentials with data object references, opening up the possibility that a future version of WES will provide concrete mechanisms for workflow runs to access data using credentials different than those used for WES. This is a work in progress and support of DOS in WES will be added in a future release of WES.
schemes:
- https
consumes:
- application/json
produces:
- application/json
paths:
/service-info:
get:
summary: Get information about Workflow Execution Service.
description: |-
May include information related (but not limited to) the workflow descriptor formats, versions supported, the WES API versions supported, and information about general service availability.
x-swagger-router-controller: ga4gh.wes.server
operationId: GetServiceInfo
responses:
'200':
description: ''
schema:
$ref: '#/definitions/ServiceInfo'
'400':
description: The request is malformed.
schema:
$ref: '#/definitions/ErrorResponse'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
tags:
- WorkflowExecutionService
/runs:
get:
summary: List the workflow runs.
description: >-
This list should be provided in a stable ordering.
(The actual ordering is implementation dependent.)
When paging through the list, the client should
not make assumptions about live updates, but should assume the
contents of the list reflect the workflow list at the moment
that the first page is requested. To monitor a specific
workflow run, use GetRunStatus or GetRunLog.
x-swagger-router-controller: ga4gh.wes.server
operationId: ListRuns
responses:
'200':
description: ''
schema:
$ref: '#/definitions/RunListResponse'
'400':
description: The request is malformed.
schema:
$ref: '#/definitions/ErrorResponse'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
parameters:
- name: page_size
description: >-
OPTIONAL
The preferred number of workflow runs to return in a page.
If not provided, the implementation should use a default page size.
The implementation must not return more items
than `page_size`, but it may return fewer. Clients should
not assume that if fewer than `page_size` items are
returned that all items have been returned. The
availability of additional pages is indicated by the value
of `next_page_token` in the response.
in: query
required: false
type: integer
format: int64
- name: page_token
description: >-
OPTIONAL
Token to use to indicate where to start getting results. If unspecified, return the first
page of results.
in: query
required: false
type: string
tags:
- WorkflowExecutionService
post:
summary: Run a workflow.
description: >-
This endpoint creates a new workflow run and
returns a `RunId` to monitor its progress.
The `workflow_attachment` array may be used to upload files
that are required to execute the workflow, including the primary
workflow, tools imported by the workflow, other files
referenced by the workflow, or files which are part of the
input. The implementation should stage these files to a
temporary directory and execute the workflow from there.
These parts must have a Content-Disposition header with a
"filename" provided for each part. Filenames may include
subdirectories, but must not include references to parent
directories with '..' -- implementations should guard against
maliciously constructed filenames.
The `workflow_url` is either an absolute URL to a workflow
file that is accessible by the WES endpoint, or a relative URL
corresponding to one of the files attached using
`workflow_attachment`.
The `workflow_params` JSON object specifies input parameters,
such as input files. The exact format of the JSON object
depends on the conventions of the workflow language being
used. Input files should either be absolute URLs, or relative
URLs corresponding to files uploaded using
`workflow_attachment`. The WES endpoint must understand and
be able to access URLs supplied in the input. This is
implementation specific.
The `workflow_type` is the type of workflow language and
must be "CWL" or "WDL" currently (or another alternative
supported by this WES instance).
The `workflow_type_version` is the version of the workflow language
submitted and must be one supported by this WES instance.
See the `RunRequest` documentation for details about other fields.
x-swagger-router-controller: ga4gh.wes.server
operationId: RunWorkflow
responses:
'200':
description: ''
schema:
$ref: '#/definitions/RunId'
'400':
description: The request is malformed.
schema:
$ref: '#/definitions/ErrorResponse'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
consumes:
- multipart/form-data
parameters:
- in: formData
name: workflow_params
type: string
format: application/json
- in: formData
name: workflow_type
type: string
- in: formData
name: workflow_type_version
type: string
- in: formData
name: tags
type: string
format: application/json
- in: formData
name: workflow_engine_parameters
type: string
format: application/json
- in: formData
name: workflow_url
type: string
- in: formData
name: workflow_attachment
type: array
items:
type: string
format: binary
tags:
- WorkflowExecutionService
/runs/{run_id}:
get:
summary: Get detailed info about a workflow run.
description: >-
This endpoint provides detailed information about a given workflow run.
The returned result has information about the outputs produced by this workflow
(if available), a log object which allows the stderr and stdout to be retrieved,
a log array so stderr/stdout for individual tasks can be retrieved,
and the overall state of the workflow run (e.g. RUNNING, see the State section).
x-swagger-router-controller: ga4gh.wes.server
operationId: GetRunLog
responses:
'200':
description: ''
schema:
$ref: '#/definitions/RunLog'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'404':
description: The requested workflow run not found.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
parameters:
- name: run_id
in: path
required: true
type: string
tags:
- WorkflowExecutionService
/runs/{run_id}/cancel:
post:
summary: Cancel a running workflow.
x-swagger-router-controller: ga4gh.wes.server
operationId: CancelRun
responses:
'200':
description: ''
schema:
$ref: '#/definitions/RunId'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'404':
description: The requested workflow run wasn't found.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
parameters:
- name: run_id
in: path
required: true
type: string
tags:
- WorkflowExecutionService
/runs/{run_id}/status:
get:
summary: Get quick status info about a workflow run.
description: >-
This provides an abbreviated (and likely fast depending on implementation)
status of the running workflow, returning a simple result with the
overall state of the workflow run (e.g. RUNNING, see the State section).
x-swagger-router-controller: ga4gh.wes.server
operationId: GetRunStatus
responses:
'200':
description: ''
schema:
$ref: '#/definitions/RunStatus'
'401':
description: The request is unauthorized.
schema:
$ref: '#/definitions/ErrorResponse'
'404':
description: The requested workflow run wasn't found.
schema:
$ref: '#/definitions/ErrorResponse'
'403':
description: The requester is not authorized to perform this action.
schema:
$ref: '#/definitions/ErrorResponse'
'500':
description: An unexpected error occurred.
schema:
$ref: '#/definitions/ErrorResponse'
parameters:
- name: run_id
in: path
required: true
type: string
tags:
- WorkflowExecutionService
definitions:
DefaultWorkflowEngineParameter:
type: object
properties:
name:
type: string
description: The name of the parameter
type:
type: string
description: Describes the type of the parameter, e.g. float.
default_value:
type: string
description: The stringified version of the default parameter. e.g. "2.45".
description: >-
A message that allows one to describe default parameters for a workflow
engine.
Log:
type: object
properties:
name:
type: string
description: The task or workflow name
cmd:
type: array
items:
type: string
description: The command line that was executed
start_time:
type: string
description: When the command started executing, in ISO 8601 format "%Y-%m-%dT%H:%M:%SZ"
end_time:
type: string
description: When the command stopped executing (completed, failed, or cancelled), in ISO 8601 format "%Y-%m-%dT%H:%M:%SZ"
stdout:
type: string
description: >-
A URL to retrieve standard output logs of the workflow run or
task. This URL may change between status requests, or may
not be available until the task or workflow has finished
execution. Should be available using the same credentials
used to access the WES endpoint.
stderr:
type: string
description: >-
A URL to retrieve standard error logs of the workflow run or
task. This URL may change between status requests, or may
not be available until the task or workflow has finished
execution. Should be available using the same credentials
used to access the WES endpoint.
exit_code:
type: integer
format: int32
description: Exit code of the program
description: Log and other info
ServiceInfo:
type: object
properties:
workflow_type_versions:
type: object
additionalProperties:
$ref: '#/definitions/WorkflowTypeVersion'
description: >-
A map with keys as the workflow format type name (currently only CWL and WDL are used
although a service may support others) and value is a workflow_type_version object which
simply contains an array of one or more version strings
supported_wes_versions:
type: array
items:
type: string
description: The version(s) of the WES schema supported by this service
supported_filesystem_protocols:
type: array
items:
type: string
description: >-
The filesystem protocols supported by this service, currently these may include common
protocols using the terms 'http', 'https', 'sftp', 's3', 'gs', 'file', or 'synapse', but others
are possible and the terms beyond these core protocols are currently not fixed.
This section reports those protocols (either common or not) supported by this WES service.
workflow_engine_versions:
type: object
additionalProperties:
type: string
description: >-
The engine(s) used by this WES service, key is engine name (e.g. Cromwell) and value is version
default_workflow_engine_parameters:
type: array
items:
$ref: '#/definitions/DefaultWorkflowEngineParameter'
description: >-
Each workflow engine can present additional parameters that can be sent to the
workflow engine. This message will list the default values, and their types for each
workflow engine.
system_state_counts:
type: object
additionalProperties:
type: integer
format: int64
description: >-
The system statistics, key is the statistic, value is the count of runs in that state.
See the State enum for the possible keys.
auth_instructions_url:
type: string
description: >-
A web page URL with human-readable instructions on how to get an
authorization token for use with a specific WES endpoint.
contact_info_url:
type: string
description: >-
An email address URL (mailto:) or web page URL with contact information
for the operator of a specific WES endpoint. Users of the
endpoint should use this to report problems or security
vulnerabilities.
tags:
type: object
additionalProperties:
type: string
description: >-
A key-value map of arbitrary, extended metadata outside the scope of the above but useful
to report back
description: >-
A message containing useful information about the running service, including supported versions and
default settings.
State:
type: string
enum:
- UNKNOWN
- QUEUED
- INITIALIZING
- RUNNING
- PAUSED
- COMPLETE
- EXECUTOR_ERROR
- SYSTEM_ERROR
- CANCELED
- CANCELING
default: UNKNOWN
description: >-
- UNKNOWN: The state of the task is unknown.
This provides a safe default for messages where this field is missing,
for example, so that a missing field does not accidentally imply that
the state is QUEUED.
- QUEUED: The task is queued.
- INITIALIZING: The task has been assigned to a worker and is currently preparing to run.
For example, the worker may be turning on, downloading input files, etc.
- RUNNING: The task is running. Input files are downloaded and the first Executor
has been started.
- PAUSED: The task is paused.
An implementation may have the ability to pause a task, but this is not required.
- COMPLETE: The task has completed running. Executors have exited without error
and output files have been successfully uploaded.
- EXECUTOR_ERROR: The task encountered an error in one of the Executor processes. Generally,
this means that an Executor exited with a non-zero exit code.
- SYSTEM_ERROR: The task was stopped due to a system error, but not from an Executor,
for example an upload failed due to network issues, the worker's ran out
of disk space, etc.
- CANCELED: The task was canceled by the user.
- CANCELING: The task was canceled by the user, and is in the process of stopping.
RunListResponse:
type: object
properties:
runs:
type: array
items:
$ref: '#/definitions/RunStatus'
description: >-
A list of workflow runs that the service has executed or is executing.
The list is filtered to only include runs that the caller has permission to see.
next_page_token:
type: string
description: >-
A token which may be supplied as `page_token` in workflow run list request to get the next page
of results. An empty string indicates there are no more items to return.
description: The service will return a RunListResponse when receiving a successful RunListRequest.
RunLog:
type: object
properties:
run_id:
type: string
description: workflow run ID
request:
$ref: '#/definitions/RunRequest'
description: The original request message used to initiate this execution.
state:
$ref: '#/definitions/State'
description: The state of the run e.g. RUNNING (see State)
run_log:
$ref: '#/definitions/Log'
description: The logs, and other key info like timing and exit code, for the overall run of this workflow.
task_logs:
type: array
items:
$ref: '#/definitions/Log'
description: The logs, and other key info like timing and exit code, for each step in the workflow run.
outputs:
type: object
description: The outputs from the workflow run.
RunRequest:
type: object
properties:
workflow_params:
type: object
description: |-
REQUIRED
The workflow run parameterizations (JSON encoded), including input and output file locations
workflow_type:
type: string
description: |-
REQUIRED
The workflow descriptor type, must be "CWL" or "WDL" currently (or another alternative supported by this WES instance)
workflow_type_version:
type: string
description: |-
REQUIRED
The workflow descriptor type version, must be one supported by this WES instance
tags:
type: object
additionalProperties:
type: string
description: |-
OPTIONAL
A key-value map of arbitrary metadata outside the scope of `workflow_params` but useful to track with this run request
workflow_engine_parameters:
type: object
additionalProperties:
type: string
description: >-
OPTIONAL
Additional parameters can be sent to the workflow engine using this field. Default values
for these parameters can be obtained using the ServiceInfo endpoint.
workflow_url:
type: string
description: >-
REQUIRED
The workflow CWL or WDL document.
When `workflow_attachments` is used to attach files, the `workflow_url` may be a relative path
to one of the attachments.
description: |-
To execute a workflow, send a run request including all the details needed to begin downloading
and executing a given workflow.
RunId:
type: object
properties:
run_id:
type: string
description: workflow run ID
RunStatus:
type: object
required:
- run_id
properties:
run_id:
type: string
state:
$ref: '#/definitions/State'
description: Small description of a workflow run, returned by server during listing
WorkflowTypeVersion:
type: object
properties:
workflow_type_version:
type: array
items:
type: string
description: |-
an array of one or more acceptable types for the `workflow_type`
description: Available workflow types supported by a given instance of the service.
ErrorResponse:
description: >-
An object that can optionally include information about the error.
type: object
properties:
msg:
type: string
description: A detailed error message.
status_code:
type: integer
description: The integer representing the HTTP status code (e.g. 200, 404).