nvenc not available

[sandikata]

Hello,

Is nvenc a part of opensource driver stack?

Regards,
Rosen Aleksandrov

[mtijanic]

Hello and thank you for your interest!

The opensource kernel driver has the same support for nvenc and tools that use it as the proprietary variant. The firmware/microcode that runs on the nvdec microcontrollers is not published, however.

I hope this answers your question, but please elaborate if this is not what you meant. Thanks.

[sandikata]

Well, but however on my RTX 3050 for example obs-studio fails with nvenc, it says no supported hardware found.

[mtijanic]

I do believe that is a bug and we will investigate. Thank you for the report!

[somewhatfrog]

Same on 3060ti
OpenEncodeSessionEx failed: out of memory (10): (no details)

Problem doesn't exits on regular 515.

[mobin-2008]

Hi. I have the same problem with the GTX 1650 (TU117).

[h264_nvenc @ 0x5594ec491d40] OpenEncodeSessionEx failed: out of memory (10): (no details)
[h264_nvenc @ 0x5594ec491d40] No capable devices found
warning: [NVENC encoder: 'recording_h264'] Failed to open NVENC codec: Generic error in an external library

I have not had such a problem with the version 510 driver in the past.

OBS Log:
OBS.log

dmesg:
dmesg_nvidia.log

[mobin-2008]

Hello and thank you for your interest!

The opensource kernel driver has the same support for nvenc and tools that use it as the proprietary variant. The firmware/microcode that runs on the nvdec microcontrollers is not published, however.

I hope this answers your question, but please elaborate if this is not what you meant. Thanks.

Does this mean nvenc is currently unavailable?

[GamingLiamStudios]

I found that applying this patch fixes the error. The drivers may be thinking that the maximum amount of nvenc streams are constantly being used, hence nvenc being unavailable.

Tested on Arch 5.17.9
Driver version: 515.43.04-7
GPU chip: GA106

[PAR2020]

Internal bug 3661377 for tracking

[sandikata]

It's a little off-topic, but…

Is someone suffering a choppy record with nvenc? Even on 120fps ?
It is always choppy (not smooth as expected, but choppy like 20fps).

[somewhatfrog]

still not fixed in 515.48

[PAR2020]

Thanks for the confirmation. On Jun 6, 2022, at 1:35 AM, kindagoose ***@***.***> wrote:  still not fixed in 515.48 — Reply to this email directly, view it on GitHub<#104 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALYJBVEBMOHOK4PRVBILVN3VNWZ5RANCNFSM5VYJD7FQ>. You are receiving this because you commented.Message ID: ***@***.***>

[PAR2020]

@sandikata, or others still seeing this issue in the latest driver drops - could you please capture a log file (sudo nvidia-bug-report.sh) and include the output with NVreg_RmMsg=":"?
This issue should have been resolved in 515.48.07 driver version. Thanks.

[rustatian]

@PAR2020 Hello. The issue still exists in the latest nvidia-open drivers from the archlinux repo (Driver Version: 515.57).
Obs-studio:

[hevc_nvenc @ 0x55ae6c8aa600] OpenEncodeSessionEx failed: out of memory (10): (no details)
[hevc_nvenc @ 0x55ae6c8aa600] No capable devices found

nvidia-bug-report.log.gz

[PAR2020]

Thanks @rustatian, that log has been very helpful. Problem identified, dev team working on a solution.

[1995parham]

I have tested the 515.65.01-4 on arch and seems the issue is gone, but maybe I am wrong.

[balenamiaa]

Hello. I face the same issue on a 3080 TI.

driver version:

> pacman -Q | grep nvidia
nvidia-open 515.65.01-7

obs log:

15:27:55.216: [NVENC encoder: 'simple_h264_recording'] Failed to open NVENC codec: Generic error in an external library
15:27:56.702: ---------------------------------
15:27:56.702: [NVENC encoder: 'simple_h264_recording'] settings:
15:27:56.702: 	rate_control: CQP
15:27:56.702: 	bitrate:      0
15:27:56.702: 	cqp:          21
15:27:56.702: 	keyint:       250
15:27:56.702: 	preset:       hq
15:27:56.702: 	profile:      high
15:27:56.702: 	width:        1276
15:27:56.702: 	height:       720
15:27:56.702: 	2-pass:       false
15:27:56.702: 	b-frames:     2
15:27:56.702: 	psycho-aq:    1
15:27:56.702: 	GPU:          0

output of sudo NVreg_RmMsg=":" nvidia-bug-report.sh is attached
nvidia-bug-report.log.gz

If desired, I'll start the X server with startx -- -logverbose 6 for more details. Just let me know

[troykelly]

I found that applying this patch fixes the error. The drivers may be thinking that the maximum amount of nvenc streams are constantly being used, hence nvenc being unavailable.

Tested on Arch 5.17.9 Driver version: 515.43.04-7 & 520.61.05 GPU chip: GA106

Note

520.61.05 can not yet be patched. Do not upgrade until there is a patch. 520 does not resolve the issue.

I've lost a little over a week to this. Thank you @GamingLiamStudios

This was the only way I could get this working.

• esxi 7
• Ubuntu 22.04.1 LTS

# Ensure secure boot is disabled - ie mokutil --disable-validation
sudo apt -y install build-essential curl dkms
echo blacklist nouveau | sudo tee /etc/modprobe.d/blacklist-nvidia-nouveau.conf
echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/blacklist-nvidia-nouveau.conf
echo options nvidia NVreg_OpenRmEnableUnsupportedGpus=1 | sudo tee /etc/modprobe.d/nvidia-unsupported-gpu.conf
sudo update-initramfs -u
sudo /sbin/reboot
git clone https://github.com/keylase/nvidia-patch.git
curl -LO https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda_11.7.1_515.65.01_linux.run
chmod +x ./cuda_11.7.1_515.65.01_linux.run
sudo ./cuda_11.7.1_515.65.01_linux.run -m=kernel-open
sudo /sbin/reboot
# Confirm working
nvidia-smi
cd nvidia-patch
sudo bash ./patch.sh
sudo /sbin/reboot
# Check can encode now
curl -LO http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
ffmpeg -i BigBuckBunny.mp4 -c:v h264_nvenc  output.mp4

[sandikata]

I could say I've fixed all this issues by replacing Nvidia with AMD (I was happy with AMD last 10 years).

[atirut-w]

I could say I've fixed all this issues by replacing Nvidia with AMD (I was happy with AMD last 10 years).

I can't get an AMD card unless I use getting into uni or this new year as an excuse :(

My next card would definitely be AMD, though, considering how perfectly AMD seems to work with Linux.

[sandikata]

Yeah, you may laugh to my comment, but since a long time Nvidia getting the worst on Linux compared to AMD.

Out of box hardware acceleration works with MESA GL (Not Nvidia ones). Already saw how bad is nvenc on Linux as it doesn't cover my expectations, and lack of hardware support for HEVC on Nvidia (Where AMD has won again as they have Hardware Support for HEVC).

Long story in short. AMD HEVC for me works better than Nvidia Nvenc (Maybe in prior to quality on same bit rate nvenc is better, but overall it's opposite).

My next card would definitely be AMD, though, considering how perfectly AMD seems to work with Linux.
AMD on Linux it is a way better last years (since they opened the source, and migrated to MESA GL Stack)

I am sorry, but RTX 3050 will be replaced with RX 6750 XT :)

I am so tired of struggling with Nvidia GL stack which limit my experience in Linux.

[ich777]

I can also confirm that NVENC is not working in combination with the Nvidia Container Runtime.
Seems like the Nvidia Open Source driver is not allowed to use NVENC in conjunction with the libraries from the proprietary driver.

[sandikata]

Closing this, as i am not interested anymore, sold Nvidia RTX 3050 in order to replace it with AMD RX6750XT.
Nvidia+Linux = Pain in the Ass!

[ich777]

@sandikata wouldn't it be better to let it open and solve this issue, you can at least unsubscribe if you are not interested in a real solution.
Buying a new card is no option for me.

[vans163]

AMD has no nvenc tho lol their video sdk is a mess and quality is worse than ffmpeg ultrafast. Its like you needed a shovel but bought a ticket to the Caribbean instead and said f*** digging.

[atirut-w]

I thought this got fixed or smth. This is pretty rude considering the reason imo.

[ich777]

@vans163 & @atirut-w please visit or subscribe to #378

I will let my issue open until it's solved since this issue over here was closed too soon without a real solution.

[troykelly]

@sandikata wouldn't it be better to let it open and solve this issue, you can at least unsubscribe if you are not interested in a real solution. Buying a new card is no option for me.

I'd argue it's best to keep this open as well in the interest of getting it fixed - but - as indicated by the lack of engagement from NVIDIA as NVIDIA don't know what is causing the issue and can't provide a timeframe - I think their preference is clearly for us to use the hack to get around this issue.

Does the application of the hack not resolve the issue for you?

EDIT: Yes, bit of a snarky tone. The known-bad driver should be withdrawn, or at least very very clearly noted that it doesn't actually work. Spending a little over a week to find this issue with zero engagement from NV is not what I expect from them.

[mtijanic]

Hi, as mentioned in #378 we do know the root cause of this issue, but we can't give an ETA just yet, sorry. It's being actively looked it.

[sandikata]

@sandikata wouldn't it be better to let it open and solve this issue, you can at least unsubscribe if you are not interested in a real solution. Buying a new card is no option for me.

I'd argue it's best to keep this open as well in the interest of getting it fixed - but - as indicated by the lack of engagement from NVIDIA as NVIDIA don't know what is causing the issue and can't provide a timeframe - I think their preference is clearly for us to use the hack to get around this issue.

Does the application of the hack not resolve the issue for you?

EDIT: Yes, bit of a snarky tone. The known-bad driver should be withdrawn, or at least very very clearly noted that it doesn't actually work. Spending a little over a week to find this issue with zero engagement from NV is not what I expect from them.

Well, keep waiting, I think you're naive if you think they will fix something for free. This is Nvidia, if they're going to do something it will be for an amount of money. If driver is the same (it is really) they for sure know the root of this issue, but not interested on fixing it for free.

And even more, Nvidia on Linux do not have any acceleration as it doesn't support proper va-api. So i'ts not only one point.

PS: even unsubscribe doesn't help and still receiving email's for this closed and unsubscribed ticket.

[Dungeonseeker]

Well, keep waiting, I think you're naive if you think they will fix something for free. This is Nvidia, if they're going to do something it will be for an amount of money. If driver is the same (it is really) they for sure know the root of this issue, but not interested on fixing it for free.

And even more, Nvidia on Linux do not have any acceleration as it doesn't support proper va-api. So i'ts not only one point.

PS: even unsubscribe doesn't help and still receiving email's for this closed and unsubscribed ticket.

1. This isn't NVIDIA, this is NVIDIA open, as in open source. Yes NV is a shitty company in many ways but the Linux and FOSS community is able to fix issues here.

2. Again incorrect, nvidia-vaapi-driver is a thing and with a bit of reading its more than possible to get NVDEC support working in userspace. My laptop right now is using NVDEC to decode videos in Firefox on Wayland.

[ich777]

@Dungeonseeker I think you've missed that it's already fixed... 😃