Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditionally quote env vars #7071

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

saurabhkoshatwar
Copy link

@saurabhkoshatwar saurabhkoshatwar commented Feb 22, 2025

Resolves #6997

This PR conditionally quotes environment variable values—only wrapping those containing special characters (like parentheses) that could trigger bash errors. Safe values remain unquoted.

@saurabhkoshatwar saurabhkoshatwar changed the title conditionally quote env vars Conditionally quote env vars Feb 22, 2025
@saurabhkoshatwar saurabhkoshatwar marked this pull request as draft February 22, 2025 20:29
@saurabhkoshatwar saurabhkoshatwar marked this pull request as ready for review February 22, 2025 21:05
@loadams loadams self-assigned this Feb 24, 2025
@qiuosier
Copy link

qiuosier commented Feb 25, 2025

This fix avoided some errors. However, there is still error with variable containing double quotes.
For example, when the var is a valid JSON string,

var = "{\"a\": \"b\"}"

after quoting, it will become

"{"a": "b"}"

then, in bash

export var="{"a": "b"}"

will export the var as

{a: b}

which is no longer a valid JSON string.

Shall we use shlex.quote() instead?

@saurabhkoshatwar
Copy link
Author

@loadams cpu-torch-latest ci stuck, please rerun

@tjruwase
Copy link
Contributor

tjruwase commented Mar 1, 2025

@saurabhkoshatwar, thanks for the PR. Is it possible to add a unit test here?
https://github.com/deepspeedai/DeepSpeed/blob/master/tests/unit/launcher/test_user_args.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] mpi based training error
4 participants