Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve upgrade test coverage #2855

Merged
merged 15 commits into from
Nov 14, 2023
Merged

Improve upgrade test coverage #2855

merged 15 commits into from
Nov 14, 2023

Conversation

t0yv0
Copy link
Member

@t0yv0 t0yv0 commented Oct 5, 2023 •
edited
Loading

Increase upgrade test coverage. Fixes https://github.com/pulumi/home/issues/3130

@github-actions
Copy link

github-actions bot commented Oct 5, 2023

Does the PR have any schema changes?

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

Maintainer note: consult the runbook for dealing with any breaking changes.

@t0yv0 t0yv0 mentioned this pull request Oct 5, 2023
Closed
@t0yv0
Copy link
Member Author

t0yv0 commented Oct 5, 2023

Looks like there's a problem here that tests under provider/* are not being run.

@t0yv0 t0yv0 mentioned this pull request Oct 6, 2023
Merged
t0yv0 added a commit to pulumi/ci-mgmt that referenced this pull request Oct 7, 2023
@t0yv0
Add unit testing to the provider (#662)
8765da6 
Fixes #661

The reason I'm working on this now is that I'd like to really move
upgrade tests out of examples and under provider/ in AWS (and soon GCP).
pulumi/pulumi-aws#2855
@t0yv0 t0yv0 force-pushed the t0yv0/simplify-upgrade-tests branch from 54da24d to 32ba400 Compare November 10, 2023 21:41
@codecov Codecov
Copy link

codecov bot commented Nov 10, 2023 •
edited
Loading

Codecov Report

Merging #2855 (45a5983) into master (4d4bf52) will not change coverage.
The diff coverage is n/a.

❗ Current head 45a5983 differs from pull request most recent head db28df6. Consider uploading reports for the commit db28df6 to get more accurate results

@@      Coverage Diff       @@
##   master   #2855   +/-   ##
==============================
==============================

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

@t0yv0 t0yv0 changed the title Move upgrade test functionality away into a framework Improve upgrade test coverage Nov 10, 2023
@t0yv0
Copy link
Member Author

t0yv0 commented Nov 10, 2023

I'm confused why https://github.com/pulumi/pulumi-aws/actions/runs/6830265431/job/18578614635#step:24:209 invocation is not running the new tests seemingly.

The command is:

        cd provider && go test -v -json -count=1 -cover -timeout 2h -tags=${{ matrix.language }} -parallel 4 . 2>&1 | tee /tmp/gotest.log | gotestfmt

Locally I see this running the new tests.

@t0yv0 t0yv0 requested a review from a team November 10, 2023 23:55
VenelinMartinov
VenelinMartinov reviewed Nov 13, 2023
View reviewed changes
provider/provider_test.go Outdated Show resolved Hide resolved
provider/provider_nodejs_test.go Outdated Show resolved Hide resolved
provider/provider_yaml_test.go Outdated Show resolved Hide resolved
@t0yv0 t0yv0 marked this pull request as ready for review November 13, 2023 21:57
@t0yv0 t0yv0 requested review from VenelinMartinov and a team November 13, 2023 21:57
@t0yv0
Copy link
Member Author

t0yv0 commented Nov 13, 2023

Looks like we got past spurious test failures by force-cleaning the test AWS account.

mjeffryes
mjeffryes approved these changes Nov 13, 2023
View reviewed changes
provider/provider_yaml_test.go
Comment on lines +15 to +18
func TestBucket(t *testing.T) {
test(t, filepath.Join("test-programs", "bucket"))
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why you switched to making a separate test function for each program rather than just iterating the test directory and generating one per program.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe I'm taking style guidance from @danielrbradley on this; both are idiomatic Go as far as I can tell. This style is even easier to point-and-click running a test in VS Code.

@t0yv0 t0yv0 merged commit d8c0a45 into master Nov 14, 2023
16 checks passed
@t0yv0 t0yv0 deleted the t0yv0/simplify-upgrade-tests branch November 14, 2023 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mjeffryes mjeffryes mjeffryes approved these changes

@VenelinMartinov VenelinMartinov Awaiting requested review from VenelinMartinov

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@t0yv0 @mjeffryes @VenelinMartinov