Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Server crashed when client tty was closed #1993

Closed
Tracked by #1568
raphCode opened this issue Dec 5, 2022 · 6 comments
Closed
Tracked by #1568

Server crashed when client tty was closed #1993

raphCode opened this issue Dec 5, 2022 · 6 comments

Comments

@raphCode
Copy link
Contributor

raphCode commented Dec 5, 2022

  • I had a zellij session running on my server
  • I connected via mosh and zelllij a
  • I noticed another mosh session + zellij client was already running, restricting the zellij window size
  • killall mosh-server
  • Upon re-login I discovered that the zellij session died

This workflow worked in the past, so I am not sure what broke here.
zellij 0.33.0 on Arch

ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.662 [stdin_handler] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: stdin_handler
             location: At zellij-client/src/os_input_output.rs:141:39
             message: called `Result::unwrap()` on an `Err` value: Os { code: 5, kind: Uncategorized, message: "Input/output error" }
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.662 [stdin_handler] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: stdin_handler
             location: At zellij-client/src/os_input_output.rs:141:39
             message: called `Result::unwrap()` on an `Err` value: Os { code: 5, kind: Uncategorized, message: "Input/output error" }
WARN   |zellij_utils::ipc        | 2022-12-05 21:22:07.664 [server_router] [zellij-utils/src/ipc.rs:204]: Error in IpcReceiver.recv(): InvalidMarkerRead(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" })
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.664 [server_router] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: server_router
             location: At zellij-server/src/lib.rs:279:42
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to handle instruction for client 2
    1: received empty message from client
INFO   |zellij_server::wasm_vm   | 2022-12-05 21:22:07.668 [wasm      ] [zellij-server/src/wasm_vm.rs:328]: wasm main thread exits
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.674 [async-std/runti] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: async-std/runtime
             location: At zellij-server/src/pty.rs:776:30
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to spawn terminals for layout
    1: failed to listen for bytes from PTY
    2: failed to async-send to screen
    3: failed to send message to screen
    4: Originating Thread(s)

    5: failed to send message to channel
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.674 [async-std/runti] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: async-std/runtime
             location: At zellij-server/src/pty.rs:776:30
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to spawn terminals for layout
    1: failed to listen for bytes from PTY
    2: failed to async-send to screen
    3: failed to send message to screen
    4: Originating Thread(s)

    5: failed to send message to channel
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.675 [async-std/runti] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: async-std/runtime
             location: At zellij-server/src/pty.rs:538:22
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to run async task for terminal 13
    1: failed to listen for bytes from PTY
    2: failed to async-send to screen
    3: failed to send message to screen
    4: Originating Thread(s)

    5: failed to send message to channel
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.683 [async-std/runti] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: async-std/runtime
             location: At zellij-server/src/pty.rs:776:30
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to spawn terminals for layout
    1: failed to listen for bytes from PTY
    2: failed to async-send to screen
    3: failed to send message to screen
    4: Originating Thread(s)

    5: failed to send message to channel
WARN   |zellij_utils::ipc        | 2022-12-05 21:22:07.705 [server_router] [zellij-utils/src/ipc.rs:204]: Error in IpcReceiver.recv(): InvalidMarkerRead(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" })
ERROR  |zellij_utils::errors::not| 2022-12-05 21:22:07.705 [server_router] [zellij-utils/src/errors.rs:452]: Panic occured:
             thread: server_router
             location: At zellij-server/src/lib.rs:279:42
             message: Program terminates: a fatal error occured

Caused by:
    0: failed to handle instruction for client 1
    1: received empty message from client
@raphCode
Copy link
Contributor Author

raphCode commented Dec 5, 2022

@har7an
I like the new error reporting!
Is it possible to add the file and line number in the "Caused by:" traces? In my case the context strings are unique so a simple grep on the codebase brings up the correct locations, but I am not sure if that is the case globally.

@imsnif
Probably relevant for you since crashes are severe.

@raphCode raphCode mentioned this issue Dec 5, 2022
Open
16 tasks
@imsnif
Copy link
Member

imsnif commented Dec 5, 2022

I"m pretty sure this is: #1955

@raphCode
Copy link
Contributor Author

raphCode commented Dec 5, 2022

You are right, this is a dup, sorry.
I already reported this in #1949 and fixed by #1965

Looking through the git blames, seems that this was introduced by: #1881
where one instance of logging was replaced with an Err which was not marked as non-fatal.
I looked at the PR to spot any additional crash locations, but found nothing too obvious.

@raphCode raphCode closed this as completed Dec 5, 2022
@har7an
Copy link
Contributor

har7an commented Dec 6, 2022

I like the new error reporting!

Glad to hear it!

Is it possible to add the file and line number in the "Caused by:" traces?

Well... Due to the way LoggableError is currently implemented, this depends on rust-lang/rust#87417, which hasn"t come very far yet, I"m afraid. I can try and explore an alternative implementation of the LoggableError, of course, and see whether that will maintain error locations.

In my case the context strings are unique so a simple grep on the codebase brings up the correct locations, but I am not sure if that is the case globally.

I assume they mostly are. But even if we had duplicates, the "trace" of the error cause will allow us to nail it down to one location in the end.

Looking through the git blames, seems that this was introduced by: #1881

Oh well, in that case:

  • Sorry for having crashed your zellij session!
  • Thanks for double-checking. :)

@har7an
Copy link
Contributor

har7an commented Dec 6, 2022

Well... Due to the way LoggableError is currently implemented, this depends on rust-lang/rust#87417,

Forget that, I misunderstood the question.

I think it should be possible to add more information to the error locations mentioned in Caused by:. This would probably require a partial reimplementation of with_context, or our own anyhow::Context trait for that matter. I"m just not sure what impact this would have on application performance/size, because I don"t know how std::panic::Location::caller() works under the hood. I"ll open a new issue for it so we can discuss it there.

@raphCode
Copy link
Contributor Author

raphCode commented Dec 6, 2022

I assume they mostly are. But even if we had duplicates, the "trace" of the error cause will allow us to nail it down to one location in the end.

You are right. So probably file locations aren"t really important, just work on this if you feel like it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@imsnif @raphCode @har7an