getting this error : `KeyError: 'model'` #3634

hemangjoshi37a · 2024-07-04T10:33:16Z

hemangjoshi37a
Jul 4, 2024

getting this error on running latest docker image :

    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 807, in update_embedding_function
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    |     with collapse_excgroups():
    |   File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/main.py", line 793, in check_url
    |     await get_all_models()
    |   File "/app/backend/main.py", line 839, in get_all_models
    |     ollama_models = await get_ollama_models()
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/ollama/main.py", line 231, in get_all_models
    |     app.state.MODELS = {model["model"]: model for model in models["models"]}
    |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/app/backend/apps/ollama/main.py", line 231, in <dictcomp>
    |     app.state.MODELS = {model["model"]: model for model in models["models"]}
    |                         ~~~~~^^^^^^^^^
    | KeyError: 'model'
     ------------------------------------
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/main.py", line 807, in update_embedding_function
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/main.py", line 793, in check_url
    await get_all_models()
  File "/app/backend/main.py", line 839, in get_all_models
    ollama_models = await get_ollama_models()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/ollama/main.py", line 231, in get_all_models
    app.state.MODELS = {model["model"]: model for model in models["models"]}
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/ollama/main.py", line 231, in <dictcomp>
    app.state.MODELS = {model["model"]: model for model in models["models"]}
                        ~~~~~^^^^^^^^^
KeyError: 'model'
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1]

exinknighthigh · 2024-07-07T16:21:38Z

exinknighthigh
Jul 7, 2024

Not sure why this is closed. This almost looks like a problem with config.py when setting environmental values and so models cant be pulled. In my case, I have ollama running as a separate container on my localhost to which I have a open-webui container connect:

docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

When reaching out, open-webui should be using host.docker.internal, not localhost or 127.0.0.1. And yet that is exactly what seems to be happening below with "health" calls to some random port:

app.state.MODELS = {model["model"]: model for model in models["models"]}
~~~~~^^^^^^^^^
KeyError: 'model'
INFO:apps.openai.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO: 127.0.0.1:59842 - "GET /health HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application

Exception Group Traceback (most recent call last):
| File "/usr/local/lib/python3.11/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
| yield
| File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 190, in call
| async with anyio.create_task_group() as task_group:
| File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
| raise BaseExceptionGroup(
| ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)

6 replies

hemangjoshi37a Jul 12, 2024
Author

ok but how to make these changes in the docker container inside python file ? how to correct in side a docker image . please help

hemangjoshi37a Jul 12, 2024
Author

also why other people are able to run it without error and only some people like me are getting this error ?

justinh-rahb Jul 12, 2024
Collaborator

Make sure you're on the latest main or dev release always. Manual docker pull if you must, or use Watchtower.

exinknighthigh Jul 13, 2024

Here's what I've established. Using the open-webui:ollama docker run command with ollama embedded works fine:

docker run -d -p 3000:8080 -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Using non embedded ollama, open-webui:cuda, does not.

docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

Assume that I have ollama running as a separate container running on the same physical host and easily accessible from that host via http://localhost:11343/api/[whatever].

Terminal into the running open-webui:cuda container and curl http://host.docker.internal:11434/api/tags works fine with model json appearing fine.

open-webui (uvicorn), on the other hand, exhibits the behavior hemangjoshi37a indicates as there seems to be some problem parsing the model json returned from ollama running on the other container.

tristaninf Jul 26, 2024

I had the same issue and just solved it. I was trying to use open-webui/pipelines with open-webui and had http://host.docker.internal:9099 in settings -> connections -> Ollama API section which made it trip.

Removing it stopped the KeyError: 'model' error

alx · 2024-07-26T05:36:03Z

alx
Jul 26, 2024

Same issue here, found that ollama 0.3.0 doesn't have model["model"] key inside its http://localhost:11434/api/tags response.

I've fixed it filling model["model"] value with model["name"] value during the post-processing of the response from ollama api call.

alx@172bc4e

3 replies

justinh-rahb Jul 26, 2024
Collaborator

If there really was an issue with this, how come our issue queue is not filling up? I've updated to 0.3.0 and my Ollama models are still showing up, no errors.

thearyadev Jul 26, 2024

i just updated to 0.3.0.

alx Jul 27, 2024

Thanks for your feedback, I was running ollama:latest docker image, I've switched to ollama:0.3.0 and got the correct /api/tags response that includes models["model"] key.

/api/tags on ollama:0.3.0 docker image:

{
  "models": [
    {
      "name": "llama3.1:latest",
      "model": "llama3.1:latest",
      "modified_at": "2024-07-26T04:30:57.118694745Z",
      "size": 4661226402,
      "digest": "62757c860e01d552d4e46b09c6b8d5396ef9015210105427e05a8b27d7727ed2",
      "details": {
        "parent_model": "",
        "format": "gguf",
        "family": "llama",
        "families": [
          "llama"
        ],
        "parameter_size": "8.0B",
        "quantization_level": "Q4_0"
      }
    }
  ]
}

/api/tags on ollama:latest docker image:

{
  "models": [
    {
      "name": "llama3.1:latest",
      "modified_at": "2024-07-26T04:30:57.118694745Z",
      "size": 4661226402,
      "digest": "62757c860e01d552d4e46b09c6b8d5396ef9015210105427e05a8b27d7727ed2"
    }
  ]
}

eedev · 2024-07-27T09:50:20Z

eedev
Jul 27, 2024

I have been looking for quite some time now, yelling at my screen, and finally just found out what was wrong for me.

I had setup ollama using https://github.com/ollama/ollama/blob/main/docs/linux.md in WSL2 and had forgotten about it.

Along the way I also setup ollama using the windows executable. So lately what I had been updating was the windows executable, not the linux binary.

I was stuck at version 0.1.x, was able to confirm that using http://192.168.1.xxx:11434/api/version

I decided to uninstall both linux version and windows version and run a docker container using a compose file.

Now it works.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

getting this error : `KeyError: 'model'` #3634

{{title}}

Replies: 3 comments 9 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

getting this error : KeyError: 'model' #3634

Replies: 3 comments · 9 replies

hemangjoshi37a Jul 12, 2024 Author

hemangjoshi37a Jul 12, 2024 Author

justinh-rahb Jul 12, 2024 Collaborator

justinh-rahb Jul 26, 2024 Collaborator

getting this error : `KeyError: 'model'` #3634

Replies: 3 comments 9 replies

hemangjoshi37a Jul 12, 2024
Author

hemangjoshi37a Jul 12, 2024
Author

justinh-rahb Jul 12, 2024
Collaborator

justinh-rahb Jul 26, 2024
Collaborator