Rust implementation of the SSIMULACRA2 metric

[LastBreeze]

Would it be possible to one day have ssim2 score as an optional metric for greater psychovisual accuracy over VMAF? Just one example is that VMAF performs poorly in areas of low luma, claiming everything is fine when the video is far from it.
Rust implementation: https://github.com/rust-av/ssimulacra2

[alexheretic]

Having alternative quality metrics would indeed be possible.

The first step is figuring out how use it to compare two video streams, then integrating it in ab-av1 should be possible. Probably easiest to find an ffmpeg invokation, since that is how vmaf is done.

[LastBreeze]

ffmpeg having ssim2 would be nice. But although the image comparison has been around for a while, the ssim2 app for measuring video is quite new, so we probably won't see it in ffmpeg for years. Could it be added to ab-av1 as an external app and invoked like ffmpeg is for vmaf, or could the rust code be integrated into ab-av1?

[alexheretic]

Perhaps it would be possible to pipe ffmpeg into a ab-av1 ssimulacra2 process. So ffmpeg handles converting into a standard stream of frames or whatever ssimulacra2 needs to work. This needs investigation.

[MCOfficer]

Is there a way to pipe 2 videos (distorted and reference) to stdin? Maybe there's some ffmpeg magic that alternates frames from both sources.

Alternatively, perhaps you can use ffmpeg-sidecar to avoid linking to a decoder of some sort.