chore: draft release note v0.8.0

[numb3r3]

Release Note

This release contains 3 new features, 1 performance improvement, and 2 documentation improvements.

🆕 Features

Support large onnx model files (#828)

Before this release, the onnx model file is limited to 2GB. Now we support
large onnx models which are archived into zip files, in which several small
onnx files are stored for subgraphs. As a result, we are now able to serve
all of the CLIP models via onnxruntime.

Support ViT-B-32, ViT-L-14, ViT-H-14 and ViT-g-14 trained on laion-2b (#825)

Users can now serve four new CLIP models from
OpenCLIP trained on the Laion-2B
dataset: ViT-B-32, ViT-L-14, ViT-H-14 and ViT-g-14. The ViT-H-14 model achieves
78.0% zero-shot top-1 accuracy on ImageNet and 73.4% on zero-shot image retrieval
at Recall@5 on MS COCO. This is the best-performing open source CLIP model.
To use the new models, simply specify the model name, e.g.,
ViT-H-14::laion2b-s32b-b79k in the FLOW yaml. For example:

jtype: Flow
version: '1'
with:
  port: 51000
executors:
  - name: clip_t
    uses:
      jtype: CLIPEncoder
      with:
        name: ViT-H-14::laion2b-s32b-b79k
      metas:
        py_modules:
          - clip_server.executors.clip_torch

Please refer to model support to see the full list of supported models.

In-place result in clip_client; preserve output order by uuid (#815)

The clip_client module now supports in-place embedding. This means the result of a call
to the CLIP server to get embeddings is stored in the input DocumentArray, instead of
creating a new DocumentArray. Consequently, the DocumentArray returned by a call to
Client.encode now has the same order as the input DocumentArray.

This could cause a breaking change if code depends on Client.encode to return a new
DocumentArray instance.

If you run the following code, you can verify that the input DocumentArray now contains
the embeddings and that the order is unchanged.

from docarray import DocumentArray, Document
from clip_client import Client

c = Client('grpc://0.0.0.0:51000')

da = [
    Document(text='she smiled, with pain'),
    Document(uri='apple.png'),
    Document(uri='apple.png').load_uri_to_image_tensor(),
    Document(blob=open('apple.png', 'rb').read()),
    Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),
    Document(
        uri='data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'
    ),
]

c.encode(da)
print(da.embeddings)

🚀 Performance

Drop image content to boost latency (#824)

Calls to Client.encode no longer return the input image with the embedding. Since embeddings
are now inserted into the original DocumentArray instance, this is unnecessary network
traffic. As a result, the system is now faster and more responsive. Performance improvement
is dependent on the size of the image and network bandwidth.

📗 Documentation Improvements

Clip benchmark on zero-shot classification and retrieval tasks (#832)

We now provide benchmark information for CLIP models on zero-shot classification and retrieval tasks.
This information should help users to choose the best CLIP model for their specific use-cases.
For more details, please read the Benchmark page in the CLIP-as-Service User Guide.

🤟 Contributors

We would like to thank all contributors to this release:

• felix-wang(@github_user)
• Ziniu Yu(@github_user)
• Jie Fu(@github_user)

[ZiniuYu]

Support ViT-L/14, ViT-H/14 and ViT-g/14 trained on laion-2b

we also have the new ViT-B-32::laion2B-s34B-b79K model

[scott-martens]

Done. See first comment.