SWAPING OUT CLIP VITL-14 WITH VIT-TINY 14

[nighting0le01]

Hello Please tell me how to :

1. swap out the vision encoder with VIT-TINY from hugging face hub or timm.
2. which checkpoint or pretrained weights should I use for it
3. what files do i need to make changes to run the pretraining and finetuning stages with it??
4. suggestions on comparing its performace with VIT-L qualitatively and quantatively