Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CodeQL for Github Action not failing on test case failures #109

Open
arpit-srivastava-ibm opened this issue Feb 3, 2023 · 1 comment
Open

Comments

@arpit-srivastava-ibm
Copy link
Member

Describe the bug
We noted that Build for Github Action did not fail even after the test cases failed.

To Reproduce

  • Open any PR. PR for reference - feat: bot management #107
  • Go to checks for the PR
  • Open Analyze (go) on the left side tab
  • Expand the Autobuild step and check test cases

Expected behavior
It is expected for Github Action to fail, as the test cases have failed.

Screenshots
image

Must gather (please complete the following information):

  • SDK Version [e.g. 1.2.1] - v0.36.0
  • Go Version [e.g. 1.12.15] - go1.18.1 darwin/amd64
  • Name of service that you're trying to use (if applicable)
  • Name of operation that you're trying to invoke (if applicable)

Additional context
It is expected for Github Action to fail, if the test cases are failing. This breaks the basic idea of building and testing the PR before merging it to the master.
Point to mention that we have Travis build for merge commits only.

@arpit-srivastava-ibm
Copy link
Member Author

arpit-srivastava-ibm commented Mar 1, 2023

@moisesatwork I believe we will need to write custom scripts for CodeQL to run the build in the first place. Right now, the build is auto generated, hence most of the test cases are skipped.

Also, 1 more important point to note here is that the integration test cases need env files to run, which we will have to encrypt in order to use during the build. (Similar to how we do for Travis builds)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant