-
Notifications
You must be signed in to change notification settings - Fork 40k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Finish saving test results on failure #76039
Conversation
/sig testing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/approve
/lgtm
Please follow up with cherry-pick PR for 1.14 branch
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: johnSchnake, timothysc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/test pull-kubernetes-integration |
Manually confirmed that the patch does fix the error case mentioned in the ticket; a failing ginkgo run still ends up saving the results and getting marked as completed via Sonobuoy. |
/test pull-kubernetes-e2e-gce |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/hold
@@ -13,7 13,6 @@ | |||
# See the License for the specific language governing permissions and | |||
# limitations under the License. | |||
|
|||
set -o errexit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it'd be better to change the wait
line to be something like
wait "$(pgrep ginkgo)" && ret=0 || ret=$?
and then the last line of this script would be
exit ${ret}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify to ensure I understand correctly,
is the reason you want to do that so that it isn't just customized for Sonobuoy (waiting for the done file) but also a consumer could wait for the non-zero exit code for indication of pass/fail?
I guess those two things are at odds:
- if we set errexit and the script fails anywhere results wont be "published" and Sonobuoy would hang
- if we dont set errexit and the script fails anywhere the exit code wont be properly reported and it might report success
If my understanding above is accurate, I agree that the lesser evil is to do as you suggest. A hang would at least be indicative of a problem whereas a false-positive is a larger issue that could go undiscovered.
Also 1/TIL to that particular bash logic; I dont think I'd seen that exact trick used before but it seems pretty nifty.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we generally like keeping errexit
on scripts so that we fail fast on syntax errors or other issues - you're sorta required to handle all error cases, or everything will just fail.
another alternative if we want to make sure logs are always uploaded is to add something like
trap saveResults EXIT
and then you wouldn't even need to explicitly call saveResults
- it'd always run on script exit.
The conformance image should be saving its results regardless of the results of the tests. However, with errexit set, when ginkgo gets test failures it exits 1 which prevents saving the results for Sonobuoy to pick up. Fixes: kubernetes#76036
8669cd6
to
4fec7c7
Compare
/lgtm |
/priority important-soon |
…76039-upstream-release-1.14 Automated cherry pick of #76039: Finish saving test results on failure
The conformance image should be saving its results
regardless of the results of the tests. However,
with errexit set, when ginkgo gets test failures
it exits 1 which prevents saving the results
for Sonobuoy to pick up.
Fixes: #76036
/kind bug
Special notes for your reviewer:
Currently working on building this image and testing it works as expected.
Does this PR introduce a user-facing change?: