ONNX opset error #11334
ONNX opset error
#11334
-
|
Just did a fresh install of Immich using the docker-compose method. [07/24/24 17:13:18] INFO Starting gunicorn 22.0.0
[07/24/24 17:13:18] INFO Listening at: http://[::]:3003 (9)
[07/24/24 17:13:18] INFO Using worker: app.config.CustomUvicornWorker
[07/24/24 17:13:18] INFO Booting worker with pid: 10
[07/24/24 17:13:21] INFO Started server process [10]
[07/24/24 17:13:21] INFO Waiting for application startup.
[07/24/24 17:13:21] INFO Created in-memory cache with unloading after 300s
of inactivity.
[07/24/24 17:13:21] INFO Initialized request thread pool with 16 threads.
[07/24/24 17:13:21] INFO Application startup complete.
[07/24/24 17:14:04] INFO Downloading visual model
'XLM-Roberta-Large-Vit-B-16Plus'. This may take a
while.
/opt/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1194: UserWarning: `local_dir_use_symlinks` parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder as`local_dir`.
For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.
warnings.warn(
Fetching 35 files: 100%|██████████| 35/35 [00:15<00:00, 2.31it/s]
[07/24/24 17:14:19] INFO Loading visual model
'XLM-Roberta-Large-Vit-B-16Plus' to memory
[07/24/24 17:14:19] INFO Setting execution providers to
['OpenVINOExecutionProvider',
'CPUExecutionProvider'], in descending order of
preference
[07/24/24 17:14:19] INFO Attempt #2 to load visual model
'XLM-Roberta-Large-Vit-B-16Plus' to memory
[07/24/24 17:14:19] ERROR Exception in ASGI application
╭─────── Traceback (most recent call last) ───────╮
│ /usr/src/app/main.py:152 in predict │
│ │
│ 149 │ │ inputs = text │
│ 150 │ else: │
│ 151 │ │ raise HTTPException(400, "Either │
│ ❱ 152 │ response = await run_inference(inputs │
│ 153 │ return ORJSONResponse(response) │
│ 154 │
│ 155 │
│ │
│ /usr/src/app/main.py:175 in run_inference │
│ │
│ 172 │ │ response[entry["task"]] = output │
│ 173 │ │
│ 174 │ without_deps, with_deps = entries │
│ ❱ 175 │ await asyncio.gather(*[_run_inference │
│ 176 │ if with_deps: │
│ 177 │ │ await asyncio.gather(*[_run_infer │
│ 178 │ if isinstance(payload, Image): │
│ │
│ /usr/src/app/main.py:169 in _run_inference │
│ │
│ 166 │ │ │ except KeyError: │
│ 167 │ │ │ │ message = f"Task {entry[' │
│ output of {dep}" │
│ 168 │ │ │ │ raise HTTPException(400, │
│ ❱ 169 │ │ model = await load(model) │
│ 170 │ │ output = await run(model.predict, │
│ 171 │ │ outputs[model.identity] = output │
│ 172 │ │ response[entry["task"]] = output │
│ │
│ /usr/src/app/main.py:213 in load │
│ │
│ 210 │ │ return model │
│ 211 │ │
│ 212 │ try: │
│ ❱ 213 │ │ return await run(_load, model) │
│ 214 │ except (OSError, InvalidProtobuf, Bad │
│ 215 │ │ log.warning(f"Failed to load {mod │
│ '{model.model_name}'. Clearing cache.") │
│ 216 │ │ model.clear_cache() │
│ │
│ /usr/src/app/main.py:188 in run │
│ │
│ 185 │ if thread_pool is None: │
│ 186 │ │ return func(*args, **kwargs) │
│ 187 │ partial_func = partial(func, *args, * │
│ ❱ 188 │ return await asyncio.get_running_loop │
│ 189 │
│ 190 │
│ 191 async def load(model: InferenceModel) -> │
│ │
│ /usr/lib/python3.10/concurrent/futures/thread.p │
│ y:58 in run │
│ │
│ /usr/src/app/main.py:200 in _load │
│ │
│ 197 │ │ │ raise HTTPException(500, f"Fa │
│ 198 │ │ with lock: │
│ 199 │ │ │ try: │
│ ❱ 200 │ │ │ │ model.load() │
│ 201 │ │ │ except FileNotFoundError as e │
│ 202 │ │ │ │ if model.model_format == │
│ 203 │ │ │ │ │ raise e │
│ │
│ /usr/src/app/models/base.py:53 in load │
│ │
│ 50 │ │ self.download() │
│ 51 │ │ attempt = f"Attempt #{self.load_a │
│ else "Loading" │
│ 52 │ │ log.info(f"{attempt} {self.model_ │
│ '{self.model_name}' to memory") │
│ ❱ 53 │ │ self.session = self._load() │
│ 54 │ │ self.loaded = True │
│ 55 │ │
│ 56 │ def predict(self, *inputs: Any, **mod │
│ │
│ /usr/src/app/models/clip/visual.py:62 in _load │
│ │
│ 59 │ │ self.mean = np.array(self.preproce │
│ 60 │ │ self.std = np.array(self.preproces │
│ 61 │ │ │
│ ❱ 62 │ │ return super()._load() │
│ 63 │ │
│ 64 │ def transform(self, image: Image.Image │
│ 65 │ │ image = resize_pil(image, self.siz │
│ │
│ /usr/src/app/models/base.py:79 in _load │
│ │
│ 76 │ │ ) │
│ 77 │ │
│ 78 │ def _load(self) -> ModelSession: │
│ ❱ 79 │ │ return self._make_session(self.mo │
│ 80 │ │
│ 81 │ def clear_cache(self) -> None: │
│ 82 │ │ if not self.cache_dir.exists(): │
│ │
│ /usr/src/app/models/base.py:111 in │
│ _make_session │
│ │
│ 108 │ │ │ case ".armnn": │
│ 109 │ │ │ │ session: ModelSession = A │
│ 110 │ │ │ case ".onnx": │
│ ❱ 111 │ │ │ │ session = OrtSession(mode │
│ 112 │ │ │ case _: │
│ 113 │ │ │ │ raise ValueError(f"Unsupp │
│ 114 │ │ return session │
│ │
│ /usr/src/app/sessions/ort.py:28 in __init__ │
│ │
│ 25 │ │ self.providers = providers if pro │
│ 26 │ │ self.provider_options = provider_ │
│ self._provider_options_default │
│ 27 │ │ self.sess_options = sess_options │
│ self._sess_options_default │
│ ❱ 28 │ │ self.session = ort.InferenceSessi │
│ 29 │ │ │ self.model_path.as_posix(), │
│ 30 │ │ │ providers=self.providers, │
│ 31 │ │ │ provider_options=self.provide │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:41 │
│ 9 in __init__ │
│ │
│ 416 │ │ disabled_optimizers = kwargs["di │
│ kwargs else None │
│ 417 │ │ │
│ 418 │ │ try: │
│ ❱ 419 │ │ │ self._create_inference_sessi │
│ disabled_optimizers) │
│ 420 │ │ except (ValueError, RuntimeError │
│ 421 │ │ │ if self._enable_fallback: │
│ 422 │ │ │ │ try: │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:47 │
│ 2 in _create_inference_session │
│ │
│ 469 │ │ self._register_ep_custom_ops(ses │
│ available_providers) │
│ 470 │ │ │
│ 471 │ │ if self._model_path: │
│ ❱ 472 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 473 │ │ else: │
│ 474 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 475 │
╰─────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Load model
from
/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/visual/m
odel.onnx
failed:/home/onnxruntimedev/onnxruntime/onnxruntime
/core/graph/model_load_utils.h:46 void
onnxruntime::model_load_utils::ValidateOpsetForDoma
in(const
std::unordered_map<std::basic_string<char>, int>&,
const onnxruntime::logging::Logger&, bool, const
string&, int) ONNX Runtime only *guarantees*
support for models stamped with official released
onnx opset versions. Opset 5 is under development
and support for this is limited. The operator
schemas and or other functionality may change
before next ONNX release and in this case ONNX
Runtime will not guarantee backward compatibility.
Current official support for domain ai.onnx.ml is
till opset 4.
[07/24/24 17:14:19] INFO Setting execution providers to
['OpenVINOExecutionProvider',
'CPUExecutionProvider'], in descending order of
preference
[07/24/24 17:14:20] INFO Attempt #3 to load visual model
'XLM-Roberta-Large-Vit-B-16Plus' to memory
[07/24/24 17:14:20] ERROR Exception in ASGI application
╭─────── Traceback (most recent call last) ───────╮
│ /usr/src/app/main.py:152 in predict │
│ │
│ 149 │ │ inputs = text │
│ 150 │ else: │
│ 151 │ │ raise HTTPException(400, "Either │
│ ❱ 152 │ response = await run_inference(inputs │
│ 153 │ return ORJSONResponse(response) │
│ 154 │
│ 155 │
│ │
│ /usr/src/app/main.py:175 in run_inference │
│ │
│ 172 │ │ response[entry["task"]] = output │
│ 173 │ │
│ 174 │ without_deps, with_deps = entries │
│ ❱ 175 │ await asyncio.gather(*[_run_inference │
│ 176 │ if with_deps: │
│ 177 │ │ await asyncio.gather(*[_run_infer │
│ 178 │ if isinstance(payload, Image): │
│ │
│ /usr/src/app/main.py:169 in _run_inference │
│ │
│ 166 │ │ │ except KeyError: │
│ 167 │ │ │ │ message = f"Task {entry[' │
│ output of {dep}" │
│ 168 │ │ │ │ raise HTTPException(400, │
│ ❱ 169 │ │ model = await load(model) │
│ 170 │ │ output = await run(model.predict, │
│ 171 │ │ outputs[model.identity] = output │
│ 172 │ │ response[entry["task"]] = output │
│ │
│ /usr/src/app/main.py:213 in load │
│ │
│ 210 │ │ return model │
│ 211 │ │
│ 212 │ try: │
│ ❱ 213 │ │ return await run(_load, model) │
│ 214 │ except (OSError, InvalidProtobuf, Bad │
│ 215 │ │ log.warning(f"Failed to load {mod │
│ '{model.model_name}'. Clearing cache.") │
│ 216 │ │ model.clear_cache() │
│ │
│ /usr/src/app/main.py:188 in run │
│ │
│ 185 │ if thread_pool is None: │
│ 186 │ │ return func(*args, **kwargs) │
│ 187 │ partial_func = partial(func, *args, * │
│ ❱ 188 │ return await asyncio.get_running_loop │
│ 189 │
│ 190 │
│ 191 async def load(model: InferenceModel) -> │
│ │
│ /usr/lib/python3.10/concurrent/futures/thread.p │
│ y:58 in run │
│ │
│ /usr/src/app/main.py:200 in _load │
│ │
│ 197 │ │ │ raise HTTPException(500, f"Fa │
│ 198 │ │ with lock: │
│ 199 │ │ │ try: │
│ ❱ 200 │ │ │ │ model.load() │
│ 201 │ │ │ except FileNotFoundError as e │
│ 202 │ │ │ │ if model.model_format == │
│ 203 │ │ │ │ │ raise e │
│ │
│ /usr/src/app/models/base.py:53 in load │
│ │
│ 50 │ │ self.download() │
│ 51 │ │ attempt = f"Attempt #{self.load_a │
│ else "Loading" │
│ 52 │ │ log.info(f"{attempt} {self.model_ │
│ '{self.model_name}' to memory") │
│ ❱ 53 │ │ self.session = self._load() │
│ 54 │ │ self.loaded = True │
│ 55 │ │
│ 56 │ def predict(self, *inputs: Any, **mod │
│ │
│ /usr/src/app/models/clip/visual.py:62 in _load │
│ │
│ 59 │ │ self.mean = np.array(self.preproce │
│ 60 │ │ self.std = np.array(self.preproces │
│ 61 │ │ │
│ ❱ 62 │ │ return super()._load() │
│ 63 │ │
│ 64 │ def transform(self, image: Image.Image │
│ 65 │ │ image = resize_pil(image, self.siz │
│ │
│ /usr/src/app/models/base.py:79 in _load │
│ │
│ 76 │ │ ) │
│ 77 │ │
│ 78 │ def _load(self) -> ModelSession: │
│ ❱ 79 │ │ return self._make_session(self.mo │
│ 80 │ │
│ 81 │ def clear_cache(self) -> None: │
│ 82 │ │ if not self.cache_dir.exists(): │
│ │
│ /usr/src/app/models/base.py:111 in │
│ _make_session │
│ │
│ 108 │ │ │ case ".armnn": │
│ 109 │ │ │ │ session: ModelSession = A │
│ 110 │ │ │ case ".onnx": │
│ ❱ 111 │ │ │ │ session = OrtSession(mode │
│ 112 │ │ │ case _: │
│ 113 │ │ │ │ raise ValueError(f"Unsupp │
│ 114 │ │ return session │
│ │
│ /usr/src/app/sessions/ort.py:28 in __init__ │
│ │
│ 25 │ │ self.providers = providers if pro │
│ 26 │ │ self.provider_options = provider_ │
│ self._provider_options_default │
│ 27 │ │ self.sess_options = sess_options │
│ self._sess_options_default │
│ ❱ 28 │ │ self.session = ort.InferenceSessi │
│ 29 │ │ │ self.model_path.as_posix(), │
│ 30 │ │ │ providers=self.providers, │
│ 31 │ │ │ provider_options=self.provide │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:41 │
│ 9 in __init__ │
│ │
│ 416 │ │ disabled_optimizers = kwargs["di │
│ kwargs else None │
│ 417 │ │ │
│ 418 │ │ try: │
│ ❱ 419 │ │ │ self._create_inference_sessi │
│ disabled_optimizers) │
│ 420 │ │ except (ValueError, RuntimeError │
│ 421 │ │ │ if self._enable_fallback: │
│ 422 │ │ │ │ try: │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:47 │
│ 2 in _create_inference_session │
│ │
│ 469 │ │ self._register_ep_custom_ops(ses │
│ available_providers) │
│ 470 │ │ │
│ 471 │ │ if self._model_path: │
│ ❱ 472 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 473 │ │ else: │
│ 474 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 475 │
╰─────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Load model
from
/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/visual/m
odel.onnx
failed:/home/onnxruntimedev/onnxruntime/onnxruntime
/core/graph/model_load_utils.h:46 void
onnxruntime::model_load_utils::ValidateOpsetForDoma
in(const
std::unordered_map<std::basic_string<char>, int>&,
const onnxruntime::logging::Logger&, bool, const
string&, int) ONNX Runtime only *guarantees*
support for models stamped with official released
onnx opset versions. Opset 5 is under development
and support for this is limited. The operator
schemas and or other functionality may change
before next ONNX release and in this case ONNX
Runtime will not guarantee backward compatibility.
Current official support for domain ai.onnx.ml is
till opset 4.
[07/24/24 17:14:20] INFO Setting execution providers to
['OpenVINOExecutionProvider',
'CPUExecutionProvider'], in descending order of
preference
[07/24/24 17:14:20] INFO Attempt #4 to load visual model
'XLM-Roberta-Large-Vit-B-16Plus' to memory
[07/24/24 17:14:20] INFO Setting execution providers to
['OpenVINOExecutionProvider',
'CPUExecutionProvider'], in descending order of
preference
[07/24/24 17:14:21] INFO Attempt #5 to load visual model
'XLM-Roberta-Large-Vit-B-16Plus' to memory
[07/24/24 17:14:21] ERROR Exception in ASGI application
╭─────── Traceback (most recent call last) ───────╮
│ /usr/src/app/main.py:152 in predict │
│ │
│ 149 │ │ inputs = text │
│ 150 │ else: │
│ 151 │ │ raise HTTPException(400, "Either │
│ ❱ 152 │ response = await run_inference(inputs │
│ 153 │ return ORJSONResponse(response) │
│ 154 │
│ 155 │
│ │
│ /usr/src/app/main.py:175 in run_inference │
│ │
│ 172 │ │ response[entry["task"]] = output │
│ 173 │ │
│ 174 │ without_deps, with_deps = entries │
│ ❱ 175 │ await asyncio.gather(*[_run_inference │
│ 176 │ if with_deps: │
│ 177 │ │ await asyncio.gather(*[_run_infer │
│ 178 │ if isinstance(payload, Image): │
│ │
│ /usr/src/app/main.py:169 in _run_inference │
│ │
│ 166 │ │ │ except KeyError: │
│ 167 │ │ │ │ message = f"Task {entry[' │
│ output of {dep}" │
│ 168 │ │ │ │ raise HTTPException(400, │
│ ❱ 169 │ │ model = await load(model) │
│ 170 │ │ output = await run(model.predict, │
│ 171 │ │ outputs[model.identity] = output │
│ 172 │ │ response[entry["task"]] = output │
│ │
│ /usr/src/app/main.py:213 in load │
│ │
│ 210 │ │ return model │
│ 211 │ │
│ 212 │ try: │
│ ❱ 213 │ │ return await run(_load, model) │
│ 214 │ except (OSError, InvalidProtobuf, Bad │
│ 215 │ │ log.warning(f"Failed to load {mod │
│ '{model.model_name}'. Clearing cache.") │
│ 216 │ │ model.clear_cache() │
│ │
│ /usr/src/app/main.py:188 in run │
│ │
│ 185 │ if thread_pool is None: │
│ 186 │ │ return func(*args, **kwargs) │
│ 187 │ partial_func = partial(func, *args, * │
│ ❱ 188 │ return await asyncio.get_running_loop │
│ 189 │
│ 190 │
│ 191 async def load(model: InferenceModel) -> │
│ │
│ /usr/lib/python3.10/concurrent/futures/thread.p │
│ y:58 in run │
│ │
│ /usr/src/app/main.py:200 in _load │
│ │
│ 197 │ │ │ raise HTTPException(500, f"Fa │
│ 198 │ │ with lock: │
│ 199 │ │ │ try: │
│ ❱ 200 │ │ │ │ model.load() │
│ 201 │ │ │ except FileNotFoundError as e │
│ 202 │ │ │ │ if model.model_format == │
│ 203 │ │ │ │ │ raise e │
│ │
│ /usr/src/app/models/base.py:53 in load │
│ │
│ 50 │ │ self.download() │
│ 51 │ │ attempt = f"Attempt #{self.load_a │
│ else "Loading" │
│ 52 │ │ log.info(f"{attempt} {self.model_ │
│ '{self.model_name}' to memory") │
│ ❱ 53 │ │ self.session = self._load() │
│ 54 │ │ self.loaded = True │
│ 55 │ │
│ 56 │ def predict(self, *inputs: Any, **mod │
│ │
│ /usr/src/app/models/clip/visual.py:62 in _load │
│ │
│ 59 │ │ self.mean = np.array(self.preproce │
│ 60 │ │ self.std = np.array(self.preproces │
│ 61 │ │ │
│ ❱ 62 │ │ return super()._load() │
│ 63 │ │
│ 64 │ def transform(self, image: Image.Image │
│ 65 │ │ image = resize_pil(image, self.siz │
│ │
│ /usr/src/app/models/base.py:79 in _load │
│ │
│ 76 │ │ ) │
│ 77 │ │
│ 78 │ def _load(self) -> ModelSession: │
│ ❱ 79 │ │ return self._make_session(self.mo │
│ 80 │ │
│ 81 │ def clear_cache(self) -> None: │
│ 82 │ │ if not self.cache_dir.exists(): │
│ │
│ /usr/src/app/models/base.py:111 in │
│ _make_session │
│ │
│ 108 │ │ │ case ".armnn": │
│ 109 │ │ │ │ session: ModelSession = A │
│ 110 │ │ │ case ".onnx": │
│ ❱ 111 │ │ │ │ session = OrtSession(mode │
│ 112 │ │ │ case _: │
│ 113 │ │ │ │ raise ValueError(f"Unsupp │
│ 114 │ │ return session │
│ │
│ /usr/src/app/sessions/ort.py:28 in __init__ │
│ │
│ 25 │ │ self.providers = providers if pro │
│ 26 │ │ self.provider_options = provider_ │
│ self._provider_options_default │
│ 27 │ │ self.sess_options = sess_options │
│ self._sess_options_default │
│ ❱ 28 │ │ self.session = ort.InferenceSessi │
│ 29 │ │ │ self.model_path.as_posix(), │
│ 30 │ │ │ providers=self.providers, │
│ 31 │ │ │ provider_options=self.provide │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:41 │
│ 9 in __init__ │
│ │
│ 416 │ │ disabled_optimizers = kwargs["di │
│ kwargs else None │
│ 417 │ │ │
│ 418 │ │ try: │
│ ❱ 419 │ │ │ self._create_inference_sessi │
│ disabled_optimizers) │
│ 420 │ │ except (ValueError, RuntimeError │
│ 421 │ │ │ if self._enable_fallback: │
│ 422 │ │ │ │ try: │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:47 │
│ 2 in _create_inference_session │
│ │
│ 469 │ │ self._register_ep_custom_ops(ses │
│ available_providers) │
│ 470 │ │ │
│ 471 │ │ if self._model_path: │
│ ❱ 472 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 473 │ │ else: │
│ 474 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 475 │
╰─────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Load model
from
/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/visual/m
odel.onnx
failed:/home/onnxruntimedev/onnxruntime/onnxruntime
/core/graph/model_load_utils.h:46 void
onnxruntime::model_load_utils::ValidateOpsetForDoma
in(const
std::unordered_map<std::basic_string<char>, int>&,
const onnxruntime::logging::Logger&, bool, const
string&, int) ONNX Runtime only *guarantees*
support for models stamped with official released
onnx opset versions. Opset 5 is under development
and support for this is limited. The operator
schemas and or other functionality may change
before next ONNX release and in this case ONNX
Runtime will not guarantee backward compatibility.
Current official support for domain ai.onnx.ml is
till opset 4.
[07/24/24 17:14:21] INFO Setting execution providers to
['OpenVINOExecutionProvider',
'CPUExecutionProvider'], in descending order of
preference
[07/24/24 17:14:21] ERROR Exception in ASGI application
╭─────── Traceback (most recent call last) ───────╮
│ /usr/src/app/main.py:152 in predict │
│ │
│ 149 │ │ inputs = text │
│ 150 │ else: │
│ 151 │ │ raise HTTPException(400, "Either │
│ ❱ 152 │ response = await run_inference(inputs │
│ 153 │ return ORJSONResponse(response) │
│ 154 │
│ 155 │
│ │
│ /usr/src/app/main.py:175 in run_inference │
│ │
│ 172 │ │ response[entry["task"]] = output │
│ 173 │ │
│ 174 │ without_deps, with_deps = entries │
│ ❱ 175 │ await asyncio.gather(*[_run_inference │
│ 176 │ if with_deps: │
│ 177 │ │ await asyncio.gather(*[_run_infer │
│ 178 │ if isinstance(payload, Image): │
│ │
│ /usr/src/app/main.py:169 in _run_inference │
│ │
│ 166 │ │ │ except KeyError: │
│ 167 │ │ │ │ message = f"Task {entry[' │
│ output of {dep}" │
│ 168 │ │ │ │ raise HTTPException(400, │
│ ❱ 169 │ │ model = await load(model) │
│ 170 │ │ output = await run(model.predict, │
│ 171 │ │ outputs[model.identity] = output │
│ 172 │ │ response[entry["task"]] = output │
│ │
│ /usr/src/app/main.py:213 in load │
│ │
│ 210 │ │ return model │
│ 211 │ │
│ 212 │ try: │
│ ❱ 213 │ │ return await run(_load, model) │
│ 214 │ except (OSError, InvalidProtobuf, Bad │
│ 215 │ │ log.warning(f"Failed to load {mod │
│ '{model.model_name}'. Clearing cache.") │
│ 216 │ │ model.clear_cache() │
│ │
│ /usr/src/app/main.py:188 in run │
│ │
│ 185 │ if thread_pool is None: │
│ 186 │ │ return func(*args, **kwargs) │
│ 187 │ partial_func = partial(func, *args, * │
│ ❱ 188 │ return await asyncio.get_running_loop │
│ 189 │
│ 190 │
│ 191 async def load(model: InferenceModel) -> │
│ │
│ /usr/lib/python3.10/concurrent/futures/thread.p │
│ y:58 in run │
│ │
│ /usr/src/app/main.py:200 in _load │
│ │
│ 197 │ │ │ raise HTTPException(500, f"Fa │
│ 198 │ │ with lock: │
│ 199 │ │ │ try: │
│ ❱ 200 │ │ │ │ model.load() │
│ 201 │ │ │ except FileNotFoundError as e │
│ 202 │ │ │ │ if model.model_format == │
│ 203 │ │ │ │ │ raise e │
│ │
│ /usr/src/app/models/base.py:53 in load │
│ │
│ 50 │ │ self.download() │
│ 51 │ │ attempt = f"Attempt #{self.load_a │
│ else "Loading" │
│ 52 │ │ log.info(f"{attempt} {self.model_ │
│ '{self.model_name}' to memory") │
│ ❱ 53 │ │ self.session = self._load() │
│ 54 │ │ self.loaded = True │
│ 55 │ │
│ 56 │ def predict(self, *inputs: Any, **mod │
│ │
│ /usr/src/app/models/clip/visual.py:62 in _load │
│ │
│ 59 │ │ self.mean = np.array(self.preproce │
│ 60 │ │ self.std = np.array(self.preproces │
│ 61 │ │ │
│ ❱ 62 │ │ return super()._load() │
│ 63 │ │
│ 64 │ def transform(self, image: Image.Image │
│ 65 │ │ image = resize_pil(image, self.siz │
│ │
│ /usr/src/app/models/base.py:79 in _load │
│ │
│ 76 │ │ ) │
│ 77 │ │
│ 78 │ def _load(self) -> ModelSession: │
│ ❱ 79 │ │ return self._make_session(self.mo │
│ 80 │ │
│ 81 │ def clear_cache(self) -> None: │
│ 82 │ │ if not self.cache_dir.exists(): │
│ │
│ /usr/src/app/models/base.py:111 in │
│ _make_session │
│ │
│ 108 │ │ │ case ".armnn": │
│ 109 │ │ │ │ session: ModelSession = A │
│ 110 │ │ │ case ".onnx": │
│ ❱ 111 │ │ │ │ session = OrtSession(mode │
│ 112 │ │ │ case _: │
│ 113 │ │ │ │ raise ValueError(f"Unsupp │
│ 114 │ │ return session │
│ │
│ /usr/src/app/sessions/ort.py:28 in __init__ │
│ │
│ 25 │ │ self.providers = providers if pro │
│ 26 │ │ self.provider_options = provider_ │
│ self._provider_options_default │
│ 27 │ │ self.sess_options = sess_options │
│ self._sess_options_default │
│ ❱ 28 │ │ self.session = ort.InferenceSessi │
│ 29 │ │ │ self.model_path.as_posix(), │
│ 30 │ │ │ providers=self.providers, │
│ 31 │ │ │ provider_options=self.provide │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:41 │
│ 9 in __init__ │
│ │
│ 416 │ │ disabled_optimizers = kwargs["di │
│ kwargs else None │
│ 417 │ │ │
│ 418 │ │ try: │
│ ❱ 419 │ │ │ self._create_inference_sessi │
│ disabled_optimizers) │
│ 420 │ │ except (ValueError, RuntimeError │
│ 421 │ │ │ if self._enable_fallback: │
│ 422 │ │ │ │ try: │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:47 │
│ 2 in _create_inference_session │
│ │
│ 469 │ │ self._register_ep_custom_ops(ses │
│ available_providers) │
│ 470 │ │ │
│ 471 │ │ if self._model_path: │
│ ❱ 472 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 473 │ │ else: │
│ 474 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 475 │
╰─────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Load model
from
/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/visual/m
odel.onnx
failed:/home/onnxruntimedev/onnxruntime/onnxruntime
/core/graph/model_load_utils.h:46 void
onnxruntime::model_load_utils::ValidateOpsetForDoma
in(const
std::unordered_map<std::basic_string<char>, int>&,
const onnxruntime::logging::Logger&, bool, const
string&, int) ONNX Runtime only *guarantees*
support for models stamped with official released
onnx opset versions. Opset 5 is under development
and support for this is limited. The operator
schemas and or other functionality may change
before next ONNX release and in this case ONNX
Runtime will not guarantee backward compatibility.
Current official support for domain ai.onnx.ml is
till opset 4.
[07/24/24 17:14:21] ERROR Exception in ASGI application
╭─────── Traceback (most recent call last) ───────╮
│ /usr/src/app/main.py:152 in predict │
│ │
│ 149 │ │ inputs = text │
│ 150 │ else: │
│ 151 │ │ raise HTTPException(400, "Either │
│ ❱ 152 │ response = await run_inference(inputs │
│ 153 │ return ORJSONResponse(response) │
│ 154 │
│ 155 │
│ │
│ /usr/src/app/main.py:175 in run_inference │
│ │
│ 172 │ │ response[entry["task"]] = output │
│ 173 │ │
│ 174 │ without_deps, with_deps = entries │
│ ❱ 175 │ await asyncio.gather(*[_run_inference │
│ 176 │ if with_deps: │
│ 177 │ │ await asyncio.gather(*[_run_infer │
│ 178 │ if isinstance(payload, Image): │
│ │
│ /usr/src/app/main.py:169 in _run_inference │
│ │
│ 166 │ │ │ except KeyError: │
│ 167 │ │ │ │ message = f"Task {entry[' │
│ output of {dep}" │
│ 168 │ │ │ │ raise HTTPException(400, │
│ ❱ 169 │ │ model = await load(model) │
│ 170 │ │ output = await run(model.predict, │
│ 171 │ │ outputs[model.identity] = output │
│ 172 │ │ response[entry["task"]] = output │
│ │
│ /usr/src/app/main.py:213 in load │
│ │
│ 210 │ │ return model │
│ 211 │ │
│ 212 │ try: │
│ ❱ 213 │ │ return await run(_load, model) │
│ 214 │ except (OSError, InvalidProtobuf, Bad │
│ 215 │ │ log.warning(f"Failed to load {mod │
│ '{model.model_name}'. Clearing cache.") │
│ 216 │ │ model.clear_cache() │
│ │
│ /usr/src/app/main.py:188 in run │
│ │
│ 185 │ if thread_pool is None: │
│ 186 │ │ return func(*args, **kwargs) │
│ 187 │ partial_func = partial(func, *args, * │
│ ❱ 188 │ return await asyncio.get_running_loop │
│ 189 │
│ 190 │
│ 191 async def load(model: InferenceModel) -> │
│ │
│ /usr/lib/python3.10/concurrent/futures/thread.p │
│ y:58 in run │
│ │
│ /usr/src/app/main.py:200 in _load │
│ │
│ 197 │ │ │ raise HTTPException(500, f"Fa │
│ 198 │ │ with lock: │
│ 199 │ │ │ try: │
│ ❱ 200 │ │ │ │ model.load() │
│ 201 │ │ │ except FileNotFoundError as e │
│ 202 │ │ │ │ if model.model_format == │
│ 203 │ │ │ │ │ raise e │
│ │
│ /usr/src/app/models/base.py:53 in load │
│ │
│ 50 │ │ self.download() │
│ 51 │ │ attempt = f"Attempt #{self.load_a │
│ else "Loading" │
│ 52 │ │ log.info(f"{attempt} {self.model_ │
│ '{self.model_name}' to memory") │
│ ❱ 53 │ │ self.session = self._load() │
│ 54 │ │ self.loaded = True │
│ 55 │ │
│ 56 │ def predict(self, *inputs: Any, **mod │
│ │
│ /usr/src/app/models/clip/visual.py:62 in _load │
│ │
│ 59 │ │ self.mean = np.array(self.preproce │
│ 60 │ │ self.std = np.array(self.preproces │
│ 61 │ │ │
│ ❱ 62 │ │ return super()._load() │
│ 63 │ │
│ 64 │ def transform(self, image: Image.Image │
│ 65 │ │ image = resize_pil(image, self.siz │
│ │
│ /usr/src/app/models/base.py:79 in _load │
│ │
│ 76 │ │ ) │
│ 77 │ │
│ 78 │ def _load(self) -> ModelSession: │
│ ❱ 79 │ │ return self._make_session(self.mo │
│ 80 │ │
│ 81 │ def clear_cache(self) -> None: │
│ 82 │ │ if not self.cache_dir.exists(): │
│ │
│ /usr/src/app/models/base.py:111 in │
│ _make_session │
│ │
│ 108 │ │ │ case ".armnn": │
│ 109 │ │ │ │ session: ModelSession = A │
│ 110 │ │ │ case ".onnx": │
│ ❱ 111 │ │ │ │ session = OrtSession(mode │
│ 112 │ │ │ case _: │
│ 113 │ │ │ │ raise ValueError(f"Unsupp │
│ 114 │ │ return session │
│ │
│ /usr/src/app/sessions/ort.py:28 in __init__ │
│ │
│ 25 │ │ self.providers = providers if pro │
│ 26 │ │ self.provider_options = provider_ │
│ self._provider_options_default │
│ 27 │ │ self.sess_options = sess_options │
│ self._sess_options_default │
│ ❱ 28 │ │ self.session = ort.InferenceSessi │
│ 29 │ │ │ self.model_path.as_posix(), │
│ 30 │ │ │ providers=self.providers, │
│ 31 │ │ │ provider_options=self.provide │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:41 │
│ 9 in __init__ │
│ │
│ 416 │ │ disabled_optimizers = kwargs["di │
│ kwargs else None │
│ 417 │ │ │
│ 418 │ │ try: │
│ ❱ 419 │ │ │ self._create_inference_sessi │
│ disabled_optimizers) │
│ 420 │ │ except (ValueError, RuntimeError │
│ 421 │ │ │ if self._enable_fallback: │
│ 422 │ │ │ │ try: │
│ │
│ /opt/venv/lib/python3.10/site-packages/onnxrunt │
│ ime/capi/onnxruntime_inference_collection.py:47 │
│ 2 in _create_inference_session │
│ │
│ 469 │ │ self._register_ep_custom_ops(ses │
│ available_providers) │
│ 470 │ │ │
│ 471 │ │ if self._model_path: │
│ ❱ 472 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 473 │ │ else: │
│ 474 │ │ │ sess = C.InferenceSession(se │
│ self._read_config_from_model) │
│ 475 │
╰─────────────────────────────────────────────────╯
Fail: [ONNXRuntimeError] : 1 : FAIL : Load model
from
/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/visual/m
odel.onnx
failed:/home/onnxruntimedev/onnxruntime/onnxruntime
/core/graph/model_load_utils.h:46 void
onnxruntime::model_load_utils::ValidateOpsetForDoma
in(const
std::unordered_map<std::basic_string<char>, int>&,
const onnxruntime::logging::Logger&, bool, const
string&, int) ONNX Runtime only *guarantees*
support for models stamped with official released
onnx opset versions. Opset 5 is under development
and support for this is limited. The operator
schemas and or other functionality may change
before next ONNX release and in this case ONNX
Runtime will not guarantee backward compatibility.
Current official support for domain ai.onnx.ml is
till opset 4.
[07/24/24 17:19:51] INFO Shutting down due to inactivity.
[07/24/24 17:19:51] INFO Shutting down
[07/24/24 17:19:51] INFO Waiting for application shutdown.
[07/24/24 17:19:51] INFO Application shutdown complete.
[07/24/24 17:19:51] INFO Finished server process [10]
[07/24/24 17:19:52] ERROR Worker (pid:10) was sent SIGINT!
[07/24/24 17:19:52] INFO Booting worker with pid: 184
[07/24/24 17:19:54] INFO Started server process [184]
[07/24/24 17:19:54] INFO Waiting for application startup.
[07/24/24 17:19:54] INFO Created in-memory cache with unloading after 300s
of inactivity.
[07/24/24 17:19:54] INFO Initialized request thread pool with 16 threads.
[07/24/24 17:19:54] INFO Application startup complete. |
Beta Was this translation helpful? Give feedback.
Answered by
bramvanneerven
Jul 27, 2024
Replies: 3 comments 3 replies
-
|
Are you using any hardware acceleration settings? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Yes, I commented out the extends key in |
Beta Was this translation helpful? Give feedback.
-
|
I think I had a similar problem a while back, just curious what containers are you using? Linux or Windows? If you're using windows containers try switching to linux might work sometimes 🤷♂️. Also you sure that you meet all the prerequisites here with all the correct configurations? |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
I am running Linux already 🙂. Trying to get it to work with an Intel Arc A380. I followed the guide you linked, so I am not quite sure where the problem comes from. |
Beta Was this translation helpful? Give feedback.
-
|
Just switch from Windows to linux containers multiple times 😭. But seriously, it could be either a compatibility problem on your side or an image problem on Immich. I would just use a past immich version for both the Immich and the machine learning containers and see if it works. If it doesn't work then we could be 90% sure its a problem on your machine with that specific GPU. So yeah, I would recommend you use a past version of Immich for the installation or use a different model. And also, is it your first time installing immich or are you upgrading it? |
Beta Was this translation helpful? Give feedback.
-
|
Upgrading to |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
bramvanneerven
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Upgrading to
v1.110.0solved the issue for me.