Intel/Nvidia super resolution support?

[emoose]

Hi, thanks for the work on MPC-BE and the video renderer.

Recently Chromium added support for NV's "RTX Super Resolution" feature for web videos, and seems it also supports an Intel GPU version too.

It's talked about on reddit here: https://www.reddit.com/r/nvidia/comments/10rdpz3/google_chrome_110_released_with_rtx_video_super/

I'm wondering if there are any plans to also support it here too?

Looking at Chromium code it seems to be handled inside ToggleNvidiaVpSuperResolution & ToggleIntelVpSuperResolution functions, which make use of ID3D11VideoContext::VideoProcessorSetOutputExtension, seen here: https://github.com/chromium/chromium/blob/ef06e5ca0f64c5b939797f0d54b4b8720387f7b9/ui/gl/swap_chain_presenter.cc#L211

Looks like this project also uses ID3D11VideoContext, but no idea how much support could be added for those...

It might be a little early to ask seeing as NV driver that supports it still isn't released too, just thought I'd mention the chromium support in case you hadn't seen it, hope you can consider adding it.

[Aleksoid1978]

1 - What gives this "Super Resolution" ?
2 - Dev's don't have RTX.

[emoose]

Doesn't seem to be much info out for it yet, I guess NV will post more info once the driver for it releases, this article was the most I found so far: https://www.pcgamer.com/rtx-video-super-resolution-ai-upscaling-feature/

The new tool uses AI to improve the quality of video streaming for recent Nvidia graphics card owners. It works by removing blocky compression artefacts and then upscaling video resolution—so cleaning the input then blowing it up to the big screen.

It'll upscale video with a native resolution anywhere between 360p and 1440p, and it'll work with video up to 144Hz frame rate.

I couldn't really find anything about the Intel version, guessing it's probably based on the XeSS game upscaler.

Dev's don't have RTX.

Ah right, well hope you can look into it in future, or maybe an RTX owner experienced with D3D11Video code can try looking into it some time.

[Aleksoid1978]

I will close the issue until the developers have the necessary equipment.

[emoose]

The drivers were released today, had a try adding the superres code when I made this issue, but didn't have a way to test till now though of course.

Didn't seem to have any effect at first, but tweaking VideoRenderer settings seemed to get it working (just needed Use for resizing option enabled), here's a quick comparison with 480p video: https://imgsli.com/MTU4MzM1

If you're interested I can post code for it up in a little bit, atm it just always tries enabling SuperRes if it can (chrome/edge do the same too, there's a SuperRes option in NVCP which has to be enabled for it to actually take effect, guess chrome is relying on that to decide the users choice)

---

E: uploaded code to emoose@a684e0a, feel free to use it or redo it a completely different way as you like (happy to test anything if needed too)

Seems the default settings after running Reset_Settings.cmd work fine actually, can toggle the SuperRes in NV control panel and it makes a noticeable difference.

[clsid2]

Some feedback from people who tried it:
https://old.reddit.com/r/nvidia/comments/11e7e2h/rtx_video_super_resolution_faq/

[Aleksoid1978]

I don't have planned use this features until don't have supported hardware.

[Hackjjang]

https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.0

[clsid2]

@emoose
I have read that it doesn't activate for high resolution video (and HDR). Maybe don't call the activation code when that is the case?

Currently InitializeD3D11VP will return a failure if SetSuperRes fails. But isn't superres a non-fatal failure?

[mctrafik]

I think the support here would be really nice for others fortunate enough to have an RTX card. I fully understand the devs thought. Before I had an RTX card, I didn't care 1 bit because it's not something that benefits a lot of people. For me though, it's really nice to be able to see old family videos in MPC in better quality. I don't want to forget how I got the upscaling to work next time I reinstall.

[Aleksoid1978]

Again, I won't add this functionality until I have the right hardware.
Stop talk about this.

[Hackjjang]

https://www.microsoftedgeinsider.com/en-us/whats-new

Microsoft Edge Browser
VSR - Nvidia RTX 20/30/40 & AMD RX 5700-7800 series support.

[v0lt]

MpcVideoRenderer 0.6.6.2005+ has the Super Resolition option and is slowly being tested by me on Intel UHD 750.

[emoose]

@v0lt did you see my comment at emoose@a684e0a#commitcomment-103082201? AFAIK github doesn't always notify for those...

If you always enable the super res extension during init (not sure exactly what stage it needs to be done before), after that you can enable/disable it during the playback fine, so would be possible to let the option apply without needing to restart it.

One way to do it could be something like

InitializeD3D11VP:
  // d3d11 init stuff here
  EnableSuperRes(true);
  this->m_curSuperResState = -1; // initially not true or false, RenderFrame will handle setting to users setting value

// unsure what best function to add the following is
// I used UpdateRenderRect so I could enable/disable based on frame dimensions
RenderFrame: 
  // if user changed setting, or we haven't set superres to users choice yet
  if (this->m_curSuperResState != userOptions.enableSuperRes) then
    EnableSuperRes(userOptions.enableSuperRes)
    this->m_curSuperResState = userOptions.enableSuperRes

Just an idea of how to let it work, maybe doing this is too messy to bother with though.
(would still advise adding something for the P010 format memory leak mentioned in the comment)

Still, glad to see it's being added now 😸

[v0lt]

Switching option during playback is done.
The use of Super Resolution depending on the size is controlled by the driver, as far as I know.
I see no reason to hide the P010 bug (if there is one), let the vendor (Nvidia?) fix it.

[clsid2]

Does Intel work in x86 player? There were some reports that Nvidia VSR only work in x64.