Ignore files to watch but keep them in docker context

[jephthia]

How can I specify to a docker_build command to not rebuild the image when certain files are changed? We have a Cargo workspace which requires all workspace members to be present in the filesystem in order to build any member. Meaning that we need to include all apps/services within the docker context of each app being built.

Given an example workspace such as:

apps
  app1
  app2
  ...
  appN

In order to build app1, we specify the root directory . as the context, same for app2, app3, etc.

Next, we need to instruct Tilt on how to build app1:

docker_build(
  "app1",
  context=".",
  dockerfile="config/docker/App1.Dockerfile",
  target="dev",
  live_update=[sync('apps/app1', '/app/apps/app1')]
)
...

We now need a way to tell this docker_build to not rebuild the image when app2, app3, etc, are changed because changes to their source files do not affect the build of this app.
Tried the only option (only=["apps/app1"]) but it seems to delete all other files from my docker context (why?)
This effectively causes all apps to be rebuilt whenever any app is changed, nullifying the benefits of live_update

I'm new to Tilt so I might have missed it but what is the best way to accomplish this?

[nicks]

You'd usually use tiltignore for this, see - https://docs.tilt.dev/file_changes

[jephthia]

Could we re-open the issue? The options described in the link shared don't seem to apply to the problem I've described above:

• The ignore option in the docker_build command will indeed stop Tilt from rebuilding when changes are detected which is what we'd need, but for some reason Tilt also goes ahead and deletes those files from the docker build context which breaks the build, those files are needed
• The only option in the docker_build command will similarly allow me to specify to Tilt which files should cause a rebuild but this option will also delete all other files from the docker build context, breaking the build
• .tiltignore this option is applied globally for all images so cannot be used in my use case, as we need to specify ignores for specific apps/services
• watch_settings seems equivalent to .tiltignore, it is applied globally

Essentially, I want to tell Tilt to ignore certain files from being watched when building a specific service, I can't specify this globally, since each service will need to ignore its own set of files.
In order words, Tilt should not trigger a rebuild when those files are changed, but that doesn't mean that Tilt should go ahead and delete all those files from the build as it'll break the build.

How do I accomplish that, what am I missing?

[nicks]

ya, by design we don't support the use-case of "this file is critical to the build, but i don't want changes to it to trigger rebuilds" -- this would introduce drift.

[jephthia]

by design we don't support the use-case [...] this would introduce drift

I'm not sure I understand that rationale, maybe I'm missing something?

1. The behavior in question for this option, Tilt already allows you to do exactly this, but on a global setting (.tiltignore, watch_settings), meaning that if I only had one single image to build, I could use those provided options and get the exact result I'm seeking, i.e, Tilt's design already allows you to get this drift so it sounds like a contradiction to say that Tilt doesn't support this use-case (though again, I might be missing something here?)
2. This is a tool for local development, there's millions of different hacks and rituals that developers need to put together to make things work locally, most of the time, it isn't because we enjoy doing that but out of necessity due to the limitations and restrictions of tools, languages, frameworks, etc. i.e., if we could just get it to work in a normal and simple way we would but often times it isn't that simple unfortunately :(
3. When it comes to production, I do agree with that rationale though, do spend some extra time to ensure your setup (Terraform, ArgoCD, FluxCD, etc) is valid and won't introduce drift but production doesn't need to deal with many things that you'll need locally. The only reason why this drift is wanted here is to enable live updates to get a fast development cycle which doesn't apply to production
4. Upon searching more on this I found this issue docker_build only param is counterintuitive #5897, and I do agree with the sentiment, if I've taken the time to setup a .dockerignore file in order to dictate which files and folders must exist in the build, it doesn't make much sense to me that if I want to prevent Tilt from needlessly rebuilding an image when I know that the change does not at all require a rebuild, that Tilt would then also go ahead and delete files off of the build context due to wanting to prevent a drift that I know isn't an issue at all for this image

With that said, after spending several days trying to find a solution, I think I've finally found the right incantations to get this behavior so I'll drop it here for other poor souls who might end up dealing with the same issue:

def build_image(ref, context, dockerfile, target=None, extra_watch=[], extra_ignore=[], live_update=[]):
  command = ['docker', 'build']

  command += ['-f', dockerfile]
  command += ['-t', '$EXPECTED_REF']

  if target:
    command += ['--target', target]

  command += [shlex.quote(context)]

  command = ' '.join(command)

  # Unlike with `docker_build()` the Dockerfiles aren't watched by default and don't
  # seem to trigger a rebuild when updated, this fixes it
  watch_patterns = [dockerfile, dockerfile + '.dockerignore'] + extra_watch

  negate_ignores = []

  for watch in watch_patterns:
    negate_ignores.append('!' + watch)

  ignore = dockerignore_patterns(dockerfile) + negate_ignores + extra_ignore

  custom_build(
    ref,
    command=command,
    deps=[context],
    ignore=ignore,
    live_update=live_update,
  )

def dockerignore_patterns(dockerfile):
  """
  Given the path to a Dockerfile, a list of the ignore
  glob patterns (i.e., non-empty, non-comment lines) found in the
  corresponding .dockerignore file will be returned.

  ### For example:
    Given: `dockerfile=config/docker/App1.Dockerfile` all
    glob patterns found in: `config/docker/App1.Dockerfile.dockerignore`
    will be returned.

  Args:
    dockerfile: A string path to the Dockerfile.

  Returns:
    A list of strings representing the ignore patterns.
  """
  content = str(read_file(dockerfile + '.dockerignore'))

  patterns = []
  for line in content.split('\n'):
    line = line.strip()

    # Skip comments (#...) and empty lines.
    if not line or line.startswith('#'):
      continue

    patterns.append(line)

  return patterns


build_image(
  "app1",
  context=".",
  dockerfile="config/docker/App1.Dockerfile",
  target="dev",
  extra_ignore=["apps", "!apps/app1"],
  live_update=[
    sync('apps/app1', '/app/apps/app1'),
    sync('libs', '/app/libs'),
  ]
)

build_image(
  "app2",
  context=".",
  dockerfile="config/docker/App2.Dockerfile",
  target="dev",
  extra_ignore=["apps", "!apps/app2"],
  live_update=[
    sync('apps/app2', '/app/apps/app2'),
    sync('libs', '/app/libs'),
  ]
)

... other apps

This feels fragile and really hacky for what I'd consider should be a default available behavior in docker_build but so far it seems to work!

A bit of explanation on what this does:

After reading the documentation more times than I can count now and trying different things, I confirmed that I really couldn't use neither the only nor ignore option of docker_build so I then thought to maybe reach for custom_build in hopes that it'd be different but reading the documentation, the ignore option of custom_build said the same thing as docker_build:

custom_build: ignore [...] - set of file patterns that will be ignored. Ignored files will not trigger builds and will not be included in images.

So I initially assumed I couldn't use it, but after being desperate enough I tried it anyways and it turns out the documentation seems incorrect!
The ignore option of custom_build does indeed retain the ignored files in the image which is the behavior I would expect in the first place. It did require some additional hacks though because it seems that as opposed to docker_build, custom_build doesn't auto recognize the *.Dockerfile.dockerignore file so I had to manually grab its content in dockerignore_patterns(), (note that my pattern is to have an identical file to the Dockerfile with an appended .dockerignore if you use a different pattern you'll have to update that function).

Lastly, the weird negate_ignores is due to how I setup my dockerignore files (this is a monorepo):

# config/docker/App1.Dockerfile.dockerignore

# Ignore everything
*

# Allow files and directories of the app being built
!apps/app1
!libs
!Cargo.toml
!Cargo.lock

# Allow Cargo manifests for all workspace members
# This is needed because Cargo cannot build without modifiying
# the lockfile if any member is missing (even if the missing
# member is unrelated to this app). This is the root cause
# of needing all these hacks but there's nothing I can do about
# that, that's how Rust workspaces work.
!apps/**/Cargo.toml
!apps/**/src/main.rs

# Ignore inside allowed directories
**/target
**/secrets
**/envs

So when passing: extra_ignore=["apps", "!apps/app1"] to build_image() it will ignore everything in apps, but then re-allow/watch everything in apps/app1

All in all, either this is the intended behavior by Tilt for custom_build and so the documentation would need some fixing or this is a bug in behavior, in which case, please don't patch it 😄

Would Tilt re-consider its stance on not wanting to provide this option for docker_build due to drift (as this is already perfectly valid behavior by Tilt)? I could provide additional details on our particular setup and why this is wanted, if need be.

An option in docker_build such as watch_ignore or skip_watch or exclude_from_watch would solve this and allow us to get rid of this fragile hack and I assume that this would be an option that would be useful to many other setups as the concept itself is fairly generic and not specific to any particular setup.

Many thanks for working on this tool, Tilt is awesome and it definitely was worth taking the time to switch from Skaffold!