Use workspace modules with BP_GO_WORK_USE

[mamachanko]

Summary

When a submodule uses replace directives referring to modules within the same repository using a relative path, e.g. ../../, then the submodule cannot be built with paketo-buildpacks/go-build.

With Go's workspaces it's possible to include a submodule in the set of main modules so such a submodule can be built.

By allowing users to set BP_GO_WORK_USE the buildpack will first run go work init followed by go work use <modules...>, before building with go build <...>.

If this PR would be considered and ever get merged, I'd be happy to follow-up with a companion docs PR for https://paketo.io/docs/howto/go/.

Use Cases

I want to build cert-manager’s images w/ buildpacks. Unfortunately, but to the best of my knowledge, as of >=1.12 vanilla checkouts (e.g. [email protected]) cannot be built with paketo-buildpacks/[email protected] for their use of relative replace directives. What I've tried:

• path=./cmd/controller doesn’t work b/c then it can’t see ../../
• BP_GO_TARGET=./cmd/controller doesn’t work because it’s not contained by the main module
• BP_GO_BUILD_FLAGS="-C ./cmd/controller" doesn’t work b/c of a limitation (or by design) in paketo-buildpacks/go-build. (it doesn’t put -C first, which go build expects. fixing it probably doesn’t help, b/c the buildpack makes assumptions about the working dir. it’s complicated)

(There’s a chance my research wasn’t exhaustive. Can you think of a way to build images off of vanilla cert-manager?)

However, with Go workspaces a vanilla check-out of cert-manager can be built. This is even documented by cert-manager.io/building/#go-workspaces. When the workspace uses ./cmd/controller it’s part of the main module and the buildpack succeeds. However, go.work is not being checked in by upstream. Afaik it's not common to check in go.work* either. As a workaround I could augment every version of upstream I want to build with go.work, but that’s not ideal. I'd rather use build a vanilla checkout with buildpacks.

After building this branch and building paketo-buildpacks/go with it, cert-manager's images built.

pack build \
  cert-manager_controller:0.0.0-mamachanko \
  --builder paketobuildpacks/builder-jammy-tiny \
  --buildpack /Users/*****/workspace/go/build/buildpackage.cnb \
  --env BP_GO_WORK_USE="./cmd/controller" \
  --env BP_GO_TARGETS="./cmd/controller"

Checklist

• I have viewed, signed, and submitted the Contributor License Agreement.
• I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
• I have added an integration test, if necessary.
• I have reviewed the styleguide for guidance on my code quality.
• I'm happy with the commit history on this PR (I have rebased/squashed as needed).

[mamachanko]

@robdimsdale @ForestEckhardt Unfortunately, I couldn't figure out how to run integration tests locally. Reverse engineering the .github/workflows/test-pull-request.yml wasn't successful either since I couldn't figure out which --builder to pass to ./scripts/integration.sh. I was hoping this PR would do that for me, but a semver label is missing.

[ForestEckhardt]

Hey there! This PR look great. My only call out is that I would like to see some unit test for the new functionality that you added. I would be more than happy to consult with you on adding those or back filling them and letting you take a look. Just let me know!

[mamachanko]

My only call out is that I would like to see some unit test for the new functionality that you added.

I shall happily add unit tests!

Until then, would you be able to instruct me on how to run the integration locally?

[mamachanko]

🤔 It's consistently expecting 0s while other tests want 1s. I can't quite tell why.

[ForestEckhardt]

It looks like all of the integration tests are failing at the moment. Could you take a look in that?

[mamachanko]

It looks like all of the integration tests are failing at the moment. Could you take a look in that?

Done. Finally I figured out how to run the integration tests for a builder locally with:

./scripts/integration.sh --builder paketobuildpacks/builder-jammy-buildpackless-tiny --token "$GIT_TOKEN"

I understand that it runs across a matrix of --builder on CI, but I hope that tiny was a good-enough approximation locally.

Also, thanks to ForestEckhardt/freezer#10 I was able to figure out how to rescue myself from "failed to open cacheManager: EOF" with rm -rf $HOME/.freezer-cache.

All that being said,

./scripts/unit.sh
./scripts/integration.sh --builder paketobuildpacks/builder-jammy-buildpackless-tiny --token "$GIT_TOKEN"

passes for me.

[ForestEckhardt]

I think that this look great!