Allow Disabling --link in collectstatic or Make collectstatic Optional

[john-h-watson]

Description

I am encountering an issue with the Cloud Native Buildpacks for Python when deploying a Django project. The build process automatically runs collectstatic, which was also the case with the classic Heroku Buildpack for Python (see #108). However, in the Cloud Native Buildpack, the --link option is enforced by default, and there is no way to disable collectstatic entirely.

Issue Details

• My static files are stored in an S3 bucket, and I use collectfasta to keep them up to date.
• When collectstatic runs with --link, it results in the following error and the build fails:

CommandError: Can't symlink to a remote destination.

• Unlike in the classic Heroku Buildpack, there is no way to disable collectstatic, making it impossible to build my Django project with the Cloud Native Buildpack.

Related Issues & References

• Issue Support disabling automatic Django static asset generation #109 discusses scenarios where disabling collectstatic might be necessary. I’d like to contribute my use case as another reason why this should be configurable.
• Issue Django collectstatic uses --link which causes uWSGI security errors #186 describes another --link problem where a workaround exists, but in my case, there is no viable workaround.

Proposed Solution

I would suggest either:

1. Adding an option to disable --link in collectstatic.
2. Allowing collectstatic to be disabled entirely, similar to the classic Heroku Buildpack.

This change would improve compatibility for users who store static files externally and do not require collectstatic during the build process.

Thank you for your time!

[edmorley]

Hi! Thank you for the great write-up 😄

I didn't know that Django collectstatic explicitly rejected the --link flag when using non-local storage backends. (The Django docs sadly don't mention that caveat.)

Checking now, I see that the error is raised here:
https://github.com/django/django/blob/41239fe34d64e801212dccaa4585e4802d0fac68/django/contrib/staticfiles/management/commands/collectstatic.py#L113-L114

It seems there isn't any way to override that error - it's a shame Django doesn't allow backends to determine how to handle that flag instead, or else treat it as a warning. Though I can see how rejecting CLI args that aren't suitable for the current backend makes sense from the perspective of a single project where the author controls the collectstatic command being run.

I'd previously found that due to a bug/limitation of ManifestStaticFilesStorage, the hashed files it creates are copies and not symlinks even when using --link:
heroku/heroku-buildpack-python#1060 (comment)

This means that for apps using WhiteNoise and it's WHITENOISE_KEEP_ONLY_HASHED_FILES there is minimal benefit of using --link, since the symlinked files aren't the ones that are kept anyway.

As such --link only really provides the following benefits (for apps using local storage):

• image size reduction for apps that aren't using WHITENOISE_KEEP_ONLY_HASHED_FILES
• reduced I/O during collectstatic since the intermediate files (that are deleted by WHITENOISE_KEEP_ONLY_HASHED_FILES) are symlinks rather than copies - which presumably speeds up collectstatic slightly (would need to benchmark to actually confirm impact - though I'm guessing the local I/O cost is probably dwarfed by the compression time, so probably doesn't improve performance by much)

Given there's no easy way for us to detect whether a project is using remote storage or not (without introspecting the Django config and then allow-listing or deny-listing certain storage backend names, which seems fragile) it makes me wonder whether we should just stop using --link at all, rather than add a buildpack config option for enabling/disabling link?

I'll mull on it a bit :-)