Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support module.classname config specification for plugins #1108

Merged

Conversation

jmartin-tech
Copy link
Collaborator

Original config support was intended to maintain this support in a deprecated format. Planned feature functionality suggests this should officially be maintained and is helpful to have aligned with how the cli accepts plugin values.

Verification

List the steps needed to make sure this thing works

  • Supporting configuration such as generator configuration file hf_RigoChat_gpu.yaml
plugins:
  generators:
    huggingface.Model:
      name: IIC/RigoChat-7b-v2
      hf_args:
        device: cuda
        trust_remote_code: true
        torch_dtype: float16
  • garak -m huggingface.Model -p test.Test --config hf_RigoChat_gpu.yaml
  • Verify via log review that the model configuration applies on instantiation.

Specific Hardware:

  • requires cuda capable GPU for the example config however mps could be substituted on arm based macOS.

Original config support was intended to maintain this support in a deprecated format.
Planned feature functionality suggests this should officially be maintained and is
helpful to have aligned with how the cli accepts plugin values.

Signed-off-by: Jeffrey Martin <[email protected]>
Copy link
Collaborator

@erickgalinkin erickgalinkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me!

@jmartin-tech jmartin-tech merged commit d78d975 into NVIDIA:main Feb 24, 2025
9 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Feb 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants