Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent duplicate app dir paths #223

Merged
merged 7 commits into from
Jan 21, 2020
Merged

Prevent duplicate app dir paths #223

merged 7 commits into from
Jan 21, 2020

Conversation

jromero
Copy link
Member

@jromero jromero commented Jan 3, 2020

Changes

  • Make app dir absolute
    • In regards to discussions with @ekcasey: it's unexpected to think that -app ./some/path/ and -app /my-app/some/path/ would place the application source in different directories within the final image
  • Clean src path to tar to prevent duplicate entries eg. some/dir/ vs some/dir vs some/dir/.
  • Update tool dependency golangci - I did this in an attempt to solve issues with scopelint.
  • Update Makefile to support incoming GOFLAGS - IntelliJ/Goland's built-in terminal provides this flag that then collides with -mod=vendor and we get an error stating the it can only be specified once.

Resolves #222

Sidenote

A concern that come up with the current implementation is that the app dir isn't configurable and is based directly off of the -app argument. For example, in theory -app=/home/jromero/my-app would yield with that directory location in the final image as /home/jromero/my-app/. My initial thought would be that there would be a standard app path location. Seems like this might have existed in PR #35 but has since been removed.

  • After discussing this with @ekcasey, it appears that this is intended for now due to the fact that it's intended to run in a container in which case paths could be modified to suit the need. Some concerns over specific CI systems still exist by chose to defer until such case arises.

Related to:

@jromero jromero force-pushed the fix/222-robust-paths branch from 06cc2f3 to 45cfe68 Compare January 3, 2020 19:39
Resolves #222

Signed-off-by: Javier Romero <[email protected]>
@jromero jromero force-pushed the fix/222-robust-paths branch from 45cfe68 to 5a1ef31 Compare January 3, 2020 19:54
@zmackie zmackie requested a review from nebhale January 6, 2020 16:15
ekcasey
ekcasey previously requested changes Jan 6, 2020
Copy link
Member

@ekcasey ekcasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaning the app dir path seems like a good thing to do. However, I feel that we should address the issue in the archive package (in our layer creation logic) that results in duplicate entries in these cases. If the path is valid that archive package should handle it correctly. I also think we should test edge cases (.e.g a path with .) in tar_test.go.

@sclevine
Copy link
Member

sclevine commented Jan 6, 2020

Also may be a good idea to clean the layer directory, like we do for builder:

lifecycle/builder.go

Lines 58 to 69 in 5a1ef31

platformDir, err := filepath.Abs(b.PlatformDir)
if err != nil {
return nil, err
}
layersDir, err := filepath.Abs(b.LayersDir)
if err != nil {
return nil, err
}
appDir, err := filepath.Abs(b.AppDir)
if err != nil {
return nil, err
}

@jromero
Copy link
Member Author

jromero commented Jan 7, 2020

RE: @ekcasey
Although I agree with the sentiment as a whole I find it cumbersome to think about maintaining two robust implementations of archiving functionality. In practice, we've resolved these issues already in pack and have a different solution for how this occurs. I wonder if a better approach wouldn't be to instead move (or centralize) archiving functionality so that pack and lifecycle share the same solution.

@ekcasey
Copy link
Member

ekcasey commented Jan 8, 2020

@jromero One could make a case for moving some of this logic to imgutil I suppose, given that the package in question provides utilities for creating layers. I'm fairly ambivalent on that question. With less duplication we won't have to fix bugs in two places. However, the logic is not completely identical due to the differing contexts (e.g lifecycle adds parent dirs) and moving it to imgutil adds a bit more friction to the process of making changes.

Whether we extract this package to a shared location or not, I do believe we should fix the core issue in the archive package rather than, or in addition to, working around it.

@zmackie zmackie added this to the lifecycle-0.6.0 milestone Jan 13, 2020
@jromero jromero requested a review from a team as a code owner January 17, 2020 19:36
@jromero jromero force-pushed the fix/222-robust-paths branch from 11d1c49 to 1b5fb99 Compare January 17, 2020 19:46
@jromero jromero changed the title Clean appDir path Prevent duplicate app dir paths Jan 17, 2020
@jromero jromero requested a review from ekcasey January 17, 2020 20:03
@ekcasey ekcasey merged commit 411a329 into master Jan 21, 2020
@ekcasey ekcasey deleted the fix/222-robust-paths branch January 21, 2020 14:59
@natalieparellano
Copy link
Member

Accepting - on 5b9f575 (without the fix) invoking the exporter with -app /workspace/. produces the following error:

[exporter] ERROR: failed to export: failed to write image to the following tags: [index.docker.io/library/fake-app-new:latest: save image 'index.docker.io/library/fake-app-new:latest': Error: No such image: 471f8ab89748b6e40135612e8ef4c89715df0e661b7c9eed8f93cfefa4293248]

On master, we were able to successfully build the app image with the same invocation.

Note, our acceptance steps used pack as the platform. We did the following:

  • make build the lifecycle binaries
  • cp out/lifecycle/exporter /tmp/fake-app/real-exporter
  • wrote a shell script saved to out/lifecycle/exporter to run real-exporter from /workspace in the build container
  • make package && pack create-builder test-builder --builder-config ./builder.toml
  • pack build test-app --path /tmp/fake-app --builder test-builder --buildpack /tmp/fake-buildpack <- observe the build fail or succeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handling "." in path when image is built with local implementations.
6 participants