Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stack traces for failures now include caller's test code #786

Merged
merged 1 commit into from
Jun 25, 2018
Merged

Stack traces for failures now include caller's test code #786

merged 1 commit into from
Jun 25, 2018

Conversation

mikelovesrobots
Copy link
Contributor

Thanks for making Detox. We use it quite a bit. Test failures can sometimes be hard to parse though, so this PR adds stack traces that include the caller's test code.

For comparison's sake, here's what a failure looks like on wix/detox (master):

    Error: Cannot find UI element.
    Exception with Action: {
      "Action Name" : "Tap",
      "Element Matcher" : "(((respondsToSelector(accessibilityIdentifier) && accessibilityID('LeftHeaderButtonxxx')) && !(kindOfClass('RCTScrollView'))) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((respondsToSelector(accessibilityIdentifier) && accessibilityID('LeftHeaderButtonxxx')) && kindOfClass('RCTScrollView'))))))",
      "Recovery Suggestion" : "Check if the element exists in the UI hierarchy printed below. If it exists, adjust the matcher so that it accurately matches element."
    }
    
    Error Trace: [
      {
        "Description" : "Interaction cannot continue because the desired element was not found.",
        "Error Domain" : "com.google.earlgrey.ElementInteractionErrorDomain",
        "Error Code" : "0",
        "File Name" : "GREYElementInteraction.m",
        "Function Name" : "-[GREYElementInteraction matchedElementsWithTimeout:error:]",
        "Line" : "124"
      }
    ]
      
      at Client.execute (node_modules/detox/src/client/Client.js:74:13)

And with this branch:

    Error: Cannot find UI element.
    Exception with Action: {
      "Action Name" : "Tap",
      "Element Matcher" : "(((respondsToSelector(accessibilityIdentifier) && accessibilityID('LeftHeaderButtonxxx')) && !(kindOfClass('RCTScrollView'))) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((respondsToSelector(accessibilityIdentifier) && accessibilityID('LeftHeaderButtonxxx')) && kindOfClass('RCTScrollView'))))))",
      "Recovery Suggestion" : "Check if the element exists in the UI hierarchy printed below. If it exists, adjust the matcher so that it accurately matches element."
    }
    
    Error Trace: [
      {
        "Description" : "Interaction cannot continue because the desired element was not found.",
        "Error Domain" : "com.google.earlgrey.ElementInteractionErrorDomain",
        "Error Code" : "0",
        "File Name" : "GREYElementInteraction.m",
        "Function Name" : "-[GREYElementInteraction matchedElementsWithTimeout:error:]",
        "Line" : "124"
      }
    ]

       6 | export const signUpFlow = async (mailbox: TempMailbox) => {
       7 |   // user can cancel the sign up process
       8 |   await elementByPath('Landing/createAccount').tap()
     > 9 |   await elementByPath('LeftHeaderButtonxxx').tap()
      10 | 
      11 |   await expect(elementByPath('Landing/createAccount')).toBeVisible()
      
      at Client.execute (node_modules/detox/src/client/Client.js:73:28)
      at InvocationManager.execute (node_modules/detox/src/invoke.js:11:33)
      at ActionInteraction.execute (node_modules/detox/src/ios/expect.js:178:29)
      at Element.tap (node_modules/detox/src/ios/expect.js:270:63)
      at _callee$ (__e2e__/signUpFlow.ts:9:49)
      at tryCatch (node_modules/regenerator-runtime/runtime.js:62:40)
      at Generator.invoke [as _invoke] (node_modules/regenerator-runtime/runtime.js:296:22)
      at Generator.prototype.(anonymous function) [as next] (node_modules/regenerator-runtime/runtime.js:114:21)
      at fulfilled (node_modules/tslib/tslib.js:104:62)

rotemmiz
rotemmiz approved these changes Jun 13, 2018
View reviewed changes
Copy link
Member

@rotemmiz rotemmiz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a great change!
I wanted to see it in action, bit it seems like it still doesn't cover everything.
https://travis-ci.org/wix/detox/jobs/391519786#L5707

Could you please take a look at this error output and suggest what to do (not the fact that the test fails of course, the stack trace still doesn't seem right)

@mikelovesrobots
Copy link
Contributor Author

Hmm... that's interesting. I think the stack trace is more or less as I'd expect but the highlighted source listing isn't. I don't know how Jest decides what's the client code and what's the test framework code. Maybe it just scans for _callee$ and since the stack trace in travis doesn't have that it shows the most recent frame?

I'm gonna take a look into it and get back to you.

@mikelovesrobots
Copy link
Contributor Author

mikelovesrobots commented Jun 13, 2018 •
edited
Loading

This was a little hard to track down, what's going on is in jest-message-util, there's a function called getTopFrame inside formatStackTrace that decides which frame in the stack trace is worth highlighting, and it does that by skipping all the stack frames that have common things like "node_modules" or "jest" or "<anonymous>." in the string (among others).

When a user of detox is running their tests, detox will be inside node_modules so it'll skip that and show the user's own code, but when you're running the detox package's tests, it's going to show you the top frame.

@rotemmiz
Copy link
Member

In the test app's case, detox is inside node modules as well, but as a symlink. It only makes sense if jest ignores symlinks (WAT?).

Anyway, merging, I would very much appreciate if you can take a look at detox test proiand help fix it.

Thanks!

@rotemmiz rotemmiz merged commit 8428c58 into wix:master Jun 25, 2018
@wix wix locked and limited conversation to collaborators Jul 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@rotemmiz rotemmiz rotemmiz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mikelovesrobots @rotemmiz