Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1694859 - image source extraction w/ symlink fails
Summary: image source extraction w/ symlink fails
Status: POST
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Build
Version: 4.1
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.1.0
Assignee: Adam Kaplan
QA Contact: wewang
Depends On:
TreeView+ depends on / blocked
Reported: 2019-04-01 21:18 UTC by Ben Parees
Modified: 2019-04-16 13:50 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed:
Target Upstream Version:

Attachments (Terms of Use)
relative symlink bc with source (deleted)
2019-04-09 19:32 UTC, Adam Kaplan
no flags Details
relative symlink bc no source (deleted)
2019-04-09 19:32 UTC, Adam Kaplan
no flags Details
abs symlink bc source (deleted)
2019-04-09 19:34 UTC, Adam Kaplan
no flags Details
abs symlink bc link only (deleted)
2019-04-09 19:35 UTC, Adam Kaplan
no flags Details
symlink bc test case (deleted)
2019-04-09 20:00 UTC, Adam Kaplan
no flags Details
source Dockerfile (deleted)
2019-04-10 18:15 UTC, Adam Kaplan
no flags Details

Description Ben Parees 2019-04-01 21:18:28 UTC
Description of problem:
Performing a build where the image source input path specified is a symlink.  The extraction process appears to copy the symlink, not the content referenced by the symlink, meaning the build ultimately fails when it can't find the expected content.

This appears to be a regression from the behavior we had in 3.x when extracting image content to use for build inputs.

How reproducible:

Steps to Reproduce:
1. Create an input image with a symlink like "/symlink" that points to "/some/file"
2. Create a build that uses the input image:
    - as: someimage
        kind: DockerImage
      - destinationDir: .
        sourcePath: /symlink

3. Make the build try to use the referenced input content as "./some/file"

Actual results:
build fails, "./some" is not a valid path in the input content

Expected results:
build succeeds, ./some/file is a valid file because the symlink was followed during content extraction/copying.

Additional Info:
Should first confirm the behavior on 3.11 but we are seeing this as a difference in behavior when moving CI from 3.x to 4.x.

Comment 1 Ben Parees 2019-04-03 18:37:19 UTC
fyi we hit this because i was using an old ci-operator binary which referenced content via the symlink location.  The new ci-operator binary uses the paths to the real files and thus works on 4.x clusters.

But this is still a valid bug/regression in behavior (well actually maybe we need to confirm what the 3.x behavior was...I had assumed ci-operator was always doing this and it worked on 3.x but perhaps that assumption is not valid).  

Regardless it does not appear to block from moving to 4.x. (unless there are specific build configurations that do use symlinks in a way that breaks here)

Comment 2 Ben Parees 2019-04-03 18:39:13 UTC
specifically this change in ci-operator moved from symlink to real file:

Comment 3 Adam Kaplan 2019-04-04 20:51:00 UTC
Testing this on a local 3.11 cluster - looks like COPYing symlinks only works if:
a) The symlink and target are extracted.
b) The relative paths between the symlink and target are preserved.

I suspect this is because the source is TARed and sent as input to the docker daemon.

In 4.0 this doesn't work - buildah mounts in the local filesystem as the context for its build. So if our local file system contains a symlink with an absolute path, it will likely break.

Comment 4 Adam Kaplan 2019-04-09 19:32:11 UTC
Created attachment 1553922 [details]
relative symlink bc with source

BuildConfig with symlink image source that works on 3.11. Uses relative paths.

Comment 5 Adam Kaplan 2019-04-09 19:32:58 UTC
Created attachment 1553923 [details]
relative symlink bc no source

Build config with symlink, but only copies the link. Fails on 3.11.

Comment 6 Adam Kaplan 2019-04-09 19:34:25 UTC
Created attachment 1553925 [details]
abs symlink bc source

BuildConfig with source using absolute symlinks. Fails on 3.11, though there may be some edge cases where this succeeds.

Comment 7 Adam Kaplan 2019-04-09 19:35:14 UTC
Created attachment 1553927 [details]
abs symlink bc link only

BuildConfig with absolute symlink, no source copied. Always fails on 3.11.

Comment 8 Adam Kaplan 2019-04-09 20:00:07 UTC
Created attachment 1553931 [details]
symlink bc test case

Attached bc succeeds on 3.11, breaks on 4.1.

This is a blocker IMO - developers will extract a parent directory and not care about the content within.

Comment 9 Adam Kaplan 2019-04-10 18:15:30 UTC
Created attachment 1554339 [details]
source Dockerfile

Dockerfile for source, symlink uses a relative path

Comment 10 wewang 2019-04-15 07:37:28 UTC
Add a test case:OCP-23174 for the bug

Comment 11 Adam Kaplan 2019-04-16 13:50:25 UTC
PR to ensure we're extracting symlinks in images correctly:

Note You need to log in before you can comment on or make changes to this bug.