configurable
should validate dict
format expectations
#926
Labels
architecture
Architectural upgrades
configurable
should validate dict
format expectations
#926
Summary
garak._plugins.load_plugin()
andConfigurable
expect to be passed either_config
or adict
with a required format. Currently_load_config()
does not validate adict
object meets the format requirements and error messages are unclear.Basic example
Here is an example invalid config using
plugins
as the top level key instead ofgenerators
:Motivation
This supports simpler maintenance and loading of generator instantiation, by validating the input in one location.
We also have some difficulty loading generators from places other than
garak.cli
, depending on the generator; improving this makes probes and detectors more powerful in the range of what generators they can use. For example, LLMaaJ #419 works well with GPT-4, but writing this in a detector means some logic duplication.The text was updated successfully, but these errors were encountered: