allow attention processors to have different signatures #6915
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| logger.warning( | ||
| f"cross_attention_kwargs {unused_kwargs} are not expected by {self.processor.__class__.__name__} and will be ignored." | ||
| ) | ||
| cross_attention_kwargs = {k: w for k, w in cross_attention_kwargs.items() if k in attn_parameters} |
There was a problem hiding this comment.
Could I get an explanation on how this ensures the scale parameter from cross_attention_kwargs gets passed appropriately?
There was a problem hiding this comment.
It won't get passed if the attention processor does not accept a scale parameter - it will throw a warning instead.
For example, IPAdapterAttnProcessor has a scale parameter in its signature, but it does not use it. We only added it so that we can use it together with the default attention processors
With this, we don't need to do that anymore. - Attention Processors will have fixed signatures that only include parameters that they actually use
There was a problem hiding this comment.
another example is in this PR https://github.com/huggingface/diffusers/pull/6847/files#diff-96aeb208ac8e69bdb368e0a481a0cd50173b3f4bf19e5da500c251614820c8cdR1201
it introduced a new parameter for IPAdapterAttnProcessor ip_adapter_mask - and they had to add same parameter to AttenProcessor.call in order for these two attention processors to work together. We don't need to do that too
There was a problem hiding this comment.
Thanks for explaining. The PR is now a go.
|
thank you very much for your work, I really needed this. This also should close #6601 |
|
|
…6915) add Co-authored-by: yiyixuxu <yixu310@gmail,com>
fix #6700
also solve this issue discussed here: #6814
allowing attention processor with different signatures to be mixed and mathed together