Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection recovery test is flaky #1140

Closed
lukebakken opened this issue Feb 7, 2022 · 1 comment · Fixed by #1148
Closed

Connection recovery test is flaky #1140

lukebakken opened this issue Feb 7, 2022 · 1 comment · Fixed by #1148
Assignees
Milestone

Comments

@lukebakken
Copy link
Contributor

lukebakken commented Feb 7, 2022

I've spent quite a bit of time trying to figure out the cause of flaky test runs such as this:

https://ci.appveyor.com/project/rabbitmq/rabbitmq-dotnet-client/builds/42483470

  Failed TestBasicAckAfterChannelRecovery [15 s]
  Error Message:
     waiting on a latch timed out
  Expected: True
  But was:  False

This is the test that is usually flaky:

https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/6.x/projects/Unit/TestConnectionRecovery.cs#L78-L84

TL;DR of what the test does:

  • Opens an auto-recovering consuming connection and model
  • Opens a publishing connection and model
  • Sets up event handlers and reset events for connection shutdown and recovery on the consuming connection/model
  • Publishes messages, and mid-publishing uses rabbitmqctl to forcibly close the connection
  • Waits for connection recovery
  • Ensures the rest of the messages are consumed

Update - this is due to a bug introduced here:

9e4b5ac#diff-8ac157517768ceec4089eb71059693ba4529f49c165989d85327d5c16e317007R146

@lukebakken
Copy link
Contributor Author

lukebakken commented Feb 21, 2022

The bug that broke this test was introduced here:

9e4b5ac#diff-8ac157517768ceec4089eb71059693ba4529f49c165989d85327d5c16e317007R146

cc @bollhals . The above change delays the assignment of the new channel to _innerChannel to the point where deliveries could be using the defunct channel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant