Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#627 expired SAS tokens] Runner downloads task JSON from URL #628

Closed
BMurri opened this issue Feb 29, 2024 · 0 comments · Fixed by #649
Closed

[#627 expired SAS tokens] Runner downloads task JSON from URL #628

BMurri opened this issue Feb 29, 2024 · 0 comments · Fixed by #649
Assignees
Labels
enhancement New feature or request TES Priority: P2 Groomed to a Priority 2 issue
Milestone

Comments

@BMurri
Copy link
Collaborator

BMurri commented Feb 29, 2024

Problem:
The runner today can both upload AND download files from blob storage by obtaining the needed SAS at the point of need, but it cannot obtain the task JSON that way, which blocks the way to no longer use any SAS tokens in batch task configurations, due to the fact that it requires the task JSON in the local filesystem.

Solution:

  1. Enable obtaining the authentication metadata (for both TERRA and CoA/standalone deployments) via environment variables.
  2. Allow the task JSON to be provided via the command-line as a URL.
  3. When the task JSON is a URI, use the current mechanism for downloading files to download the URL and use the content as the task JSON.
  4. Consider whether the task JSON should be placed in/near the working directory (in the local filesystem) or if it should be streamed directly to memory and held there until final task completion.

AC

  • Unit tests to verify correct behaviors
  • Consider integration tests to demonstrate correct behavior.
@BMurri BMurri added the enhancement New feature or request label Feb 29, 2024
@BMurri BMurri added the TES Priority: P2 Groomed to a Priority 2 issue label Mar 1, 2024
@BMurri BMurri added this to the next milestone Mar 1, 2024
@BMurri BMurri self-assigned this Mar 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request TES Priority: P2 Groomed to a Priority 2 issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant