Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Timeout error after 5 minutes of generation #3067

Closed
4 tasks done
TheTerrasque opened this issue Jun 12, 2024 · 0 comments
Closed
4 tasks done

Timeout error after 5 minutes of generation #3067

TheTerrasque opened this issue Jun 12, 2024 · 0 comments

Comments

@TheTerrasque
Copy link

TheTerrasque commented Jun 12, 2024 •
edited
Loading

Bug Report

Description

Bug Summary:
When ollama reply generation goes over 5 minutes, the server generates a timeout error.

Steps to Reproduce:
Load a slow model, have it generate a long reply.

Expected Behavior:
The full reply gets displayed in browser

Actual Behavior:
After 5 minutes, the connection is closed, giving an error.

Environment

  • Open WebUI Version: v0.3.3

  • Ollama (if applicable): 0.1.43

  • Operating System: Windows 10

  • Browser (if applicable): Firefox / Edge

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
image

Docker Container Logs:

ERROR:    Exception in ASGI application

Traceback (most recent call last):

  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi

    result = await app(  # type: ignore[func-returns-value]

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__

    return await self.app(scope, receive, send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__

    await super().__call__(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__

    await self.middleware_stack(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__

    await self.app(scope, receive, _send)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__

    with collapse_excgroups():

  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__

    self.gen.throw(typ, value, traceback)

  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap

    await func()

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response

    return await super().stream_response(send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in stream_response

    async for chunk in self.body_iterator:

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream

    raise app_exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro

    await self.app(scope, receive_or_disconnect, send_no_error)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__

    with collapse_excgroups():

  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__

    self.gen.throw(typ, value, traceback)

  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap

    await func()

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response

    return await super().stream_response(send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in stream_response

    async for chunk in self.body_iterator:

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream

    raise app_exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro

    await self.app(scope, receive_or_disconnect, send_no_error)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__

    await self.simple_response(scope, receive, send, request_headers=headers)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response

    await self.app(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__

    with collapse_excgroups():

  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__

    self.gen.throw(typ, value, traceback)

  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap

    await func()

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response

    return await super().stream_response(send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in stream_response

    async for chunk in self.body_iterator:

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream

    raise app_exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro

    await self.app(scope, receive_or_disconnect, send_no_error)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__

    with collapse_excgroups():

  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__

    self.gen.throw(typ, value, traceback)

  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap

    await func()

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 217, in stream_response

    return await super().stream_response(send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in stream_response

    async for chunk in self.body_iterator:

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 181, in body_stream

    raise app_exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro

    await self.app(scope, receive_or_disconnect, send_no_error)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__

    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app

    await app(scope, receive, sender)

  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__

    await self.middleware_stack(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app

    await route.handle(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 485, in handle

    await self.app(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__

    await super().__call__(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__

    await self.middleware_stack(scope, receive, send)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__

    await self.app(scope, receive, _send)

  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__

    with collapse_excgroups():

  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__

    self.gen.throw(typ, value, traceback)

  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups

    raise exc

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap

    await func()

  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 250, in stream_response

    async for chunk in self.body_iterator:

  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 50, in __anext__

    rv = await self.read_func()

         ^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 317, in readline

    return await self.readuntil()

           ^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 351, in readuntil

    await self._wait("readuntil")

  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 311, in _wait

    with self._timer:

  File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in __exit__

    raise asyncio.TimeoutError from None

TimeoutError

Screenshots (if applicable):
[Attach any relevant screenshots to help illustrate the issue]

Installation Method

Open-Webui: Kubernetes deployment of docker image, service access via load balancer IP.
Ollama: Direct deployment on bare metal, using official linux executable.

Additional Information

Another user have experienced the same issue: #2208 (comment)

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
@TheTerrasque TheTerrasque changed the title Timeout error after 5 minutes on chat Timeout error after 5 minutes of generation Jun 12, 2024
@open-webui open-webui locked and limited conversation to collaborators Jun 12, 2024
@tjbck tjbck converted this issue into discussion #3086 Jun 12, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@TheTerrasque