Grype failed to load vulnerability database: database metadata not found

[Atharex]

What happened:
Running Grype on an Azure DevOps Pipeline task occasionally fails with the following error:

[info] checking github for the current release tag 
[info] fetching release script for tag='v0.77.4' 
[info] checking github for the current release tag 
[info] using release tag='v0.77.4' version='0.77.4' os='linux' arch='amd64' 
[info] installed /usr/local/bin/grype 
failed to load vulnerability db: vulnerability database is invalid (run db update to correct): database metadata not found: /home/vsts/.cache/grype/db/5

##[error]Bash exited with code '1'.

Rerunning the pipeline several times eventually brings it to completion, yet this transient error prevents us from relying on grype scanning automation.

What you expected to happen:
The pipeline to run consistently successfully

How to reproduce it (as minimally and precisely as possible):
Run an Azure DevOps pipeline task where Grype is used (my current version is v0.77.4)

Environment:

• Output of grype version: v.0.77.4
• OS (e.g: cat /etc/os-release or similar): Linux, amd64

[tgerla]

Hi @Atharex, thanks for the report. I think we will need more information to help you troubleshoot this. This doesn't look like a network error reaching the Grype DB's servers. It looks like an error on the filesystem. Can you share some more details about the Azure setup? What Linux distribution, etc? It would probably be worthwhile to add "-vv" to your Grype command line for more verbose logging. You might also want to try adding a "grype db update" command to the top of your pipeline and see if the failures continue. Just a hunch, but do you by chance have multiple Grype processes running simultaneously on the same underlying volume?

[djclarkson]

Same issue with grype installed via brew on mac sonoma.

grype ./
 ✔ Vulnerability DB                [138 MB / 166 MB]  
 ✔ Indexed file system                                                                                                                                                                                         .
 ✔ Cataloged contents                                                                                                                           cdb4ee2aea69cc6a83331bbe96dc2caa9a299d21329efb0336fc02a82e1839a8
   ├── ✔ Packages                        [1,239 packages]  
   └── ✔ Executables                     [0 executables]  
[0000]  WARN no explicit name and version provided for directory source, deriving artifact ID from the given path (which is not ideal)
failed to load vulnerability db: unable to update vulnerability database: unable to download db: context deadline exceeded (Client.Timeout or context cancellation while reading body)

[spiffcs]

Thanks @djclarkson - if you try and run it again are you still seeing the timeout? In the meantime we're looking at logs for our CDN to see where/when the requests are being dropped for certain users

[walton-io]

Bumping as this is still an intermittent issue. Last week, Friday this was working as expected.

This morning, Monday, any Grype scan we do we get this error:

failed to load vulnerability db: unable to update vulnerability database: unable to download db: context deadline exceeded (Client.Timeout or context cancellation while reading body)

[spiffcs]

@walton-io thanks for the bump - we're working with our hosting provider to try and identify the situations that cause these drops

Context deadline exceeded is a client timeout and doesn't really tell you a lot about the servers response.
The grype config has the ability to up these timeouts if you're still seeing intermittent issues:

I've included a link to the config and fields in the readme that you can tweak to give the client more time

grype/README.md

Lines 755 to 761 in 1ab36b4

	# Timeout for downloading GRYPE_DB_UPDATE_URL to see if the database needs to be downloaded
	# This file is ~156KiB as of 2024-04-17 so the download should be quick; adjust as needed
	update-available-timeout: "30s"
	# Timeout for downloading actual vulnerability DB
	# The DB is ~156MB as of 2024-04-17 so slower connections may exceed the default timeout; adjust as needed
	update-download-timeout: "120s"