You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
And we can still improve this, using some of these approaches:
Send compressed scan results: I was able to compress the ~350MB scan result to under 25MB using zstd or gzip ... xz is way too slow otherwise. For a start using the built in Gzip compression of HTTP clients and servers should help a lot.
Adapt timeout to payload: The 60-second DEFAULT_TIMEOUT is not sufficient for sending scan results, we should have a variable timeout depending on the size of the scan results.
Consider paginated upload in multiple chunks or streamed upload using JSON lines.
The text was updated successfully, but these errors were encountered:
pombredanne
changed the title
send_scan_project_results is failing for larger scans
PurlDB-requested package index scans i failing for larger packages with large scans
Apr 3, 2024
pombredanne
changed the title
PurlDB-requested package index scans i failing for larger packages with large scans
PurlDB-requested package index scans is failing for larger packages with large scans
Apr 3, 2024
pombredanne
changed the title
PurlDB-requested package index scans is failing for larger packages with large scans
PurlDB-requested package index scans can fail for larger packages with large scans and many scans at once
May 8, 2024
send_scan_project_results
inpurldb-scan-queue-worker
isn't able to properly send the large scan results.This issue fixed the critical problems:
And we can still improve this, using some of these approaches:
DEFAULT_TIMEOUT
is not sufficient for sending scan results, we should have a variable timeout depending on the size of the scan results.The text was updated successfully, but these errors were encountered: