8 Releases published by 1 person

• v1.59.1

  published Jan 21, 2025

• v1.59.2

  published Jan 22, 2025

• v1.59.3

  published Jan 22, 2025

• v1.59.3.dev1

  published Jan 23, 2025

• v1.59.5

  published Jan 23, 2025

• v1.59.6

  published Jan 24, 2025

• v1.59.7

  published Jan 25, 2025

• v1.59.8

  published Jan 27, 2025

59 Pull requests merged by 9 people

• (UI enhancement) - allow onboarding wildcard models on UI

  #8034 merged Jan 27, 2025

• (UI) - Adding new models enhancement - show provider logo

  #8033 merged Jan 27, 2025

• Litellm dev 01 25 2025 p4

  #8006 merged Jan 26, 2025

• Fix custom pricing - separate provider info from model info

  #7990 merged Jan 26, 2025

• (Fix) langfuse - setting LANGFUSE_FLUSH_INTERVAL

  #8007 merged Jan 26, 2025

• (QA / testing) - Add e2e tests for key model access auth checks

  #8000 merged Jan 26, 2025

• Bug fix - [Bug]: If you create a key tied to a user that does not belong to a team, and then edit the key to add it to a team (the user is still not a part of a team), using that key results in an unexpected error

  #8008 merged Jan 26, 2025

• Supported nested json schema on anthropic calls via proxy + fix langfuse sync sdk issues

  #8003 merged Jan 26, 2025

• (Feat) set guardrails per team

  #7993 merged Jan 25, 2025

• (Prometheus) - emit key budget metrics on startup

  #8002 merged Jan 25, 2025

• (QA / testing) - Add unit testing for key model access checks

  #7999 merged Jan 25, 2025

• add type annotation for litellm.api_base (#7980)

  #7994 merged Jan 25, 2025

• add type annotation for litellm.api_base

  #7980 merged Jan 25, 2025

• refactor: cleanup dead codeblock

  #7936 merged Jan 25, 2025

• (UI) Allow admin to expose teams for joining

  #7992 merged Jan 25, 2025

• Ensure base_model cost tracking works across all endpoints

  #7989 merged Jan 25, 2025

• fix(spend_tracking_utils.py): revert api key pass through fix

  #7977 merged Jan 25, 2025

• (Feat) - Add GCS Pub/Sub Logging integration for sending DB SpendLogs to BigQuery

  #7976 merged Jan 25, 2025

• (UI) - Usage page show days when spend is 0 and round spend figures on charts to 2 sig figs

  #7991 merged Jan 25, 2025

• (Testing) e2e testing for team budget enforcement checks

  #7988 merged Jan 25, 2025

• (Feat) - allow setting default_on guardrails

  #7973 merged Jan 24, 2025

• Ollama ssl verify = False + Spend Logs reliability fixes

  #7931 merged Jan 24, 2025

• Retry for replicate completion response of status=processing (#7901)

  #7965 merged Jan 24, 2025

• Retry for replicate completion response of status=processing

  #7901 merged Jan 24, 2025

• Add datadog health check support + fix bedrock converse cost tracking w/ region name specified

  #7958 merged Jan 24, 2025

• Litellm dev 01 23 2025 p2

  #7962 merged Jan 24, 2025

• (GCS fix) - don"t truncate payload

  #7964 merged Jan 24, 2025

• (UI) Set guardrails on Team Create and Edit page

  #7963 merged Jan 24, 2025

• (Feat) allow setting guardrails on a team on the API

  #7959 merged Jan 24, 2025

• (Feat) - emit litellm_team_budget_reset_at_metric and litellm_api_key_budget_remaining_hours_metric on prometheus

  #7946 merged Jan 24, 2025

• (UI) - Set/edit guardrails on a virtual key

  #7954 merged Jan 24, 2025

• (Feat) - Allow Admin UI users to view spend logs even when not storing messages / responses

  #7952 merged Jan 24, 2025

• docs: fix typo

  #7953 merged Jan 24, 2025

• (Testing + Refactor) - Unit testing for team and virtual key budget checks

  #7945 merged Jan 24, 2025

• Refactor prometheus e2e test

  #7919 merged Jan 23, 2025

• Add attempted-retries and timeout values to response headers + more testing

  #7926 merged Jan 23, 2025

• (Refactor) Langfuse - remove prepare_metadata, langfuse python SDK now handles non-json serializable objects

  #7925 merged Jan 23, 2025

• Add provider_specifc_header param

  #7932 merged Jan 23, 2025

• fix(utils.py): move adding custom logger callback to success event in…

  #7905 merged Jan 23, 2025

• (feat) - add deepseek/deepseek-reasoner to model cost map

  #7935 merged Jan 23, 2025

• (test) add e2e test for proxy with fallbacks + custom fallback message

  #7933 merged Jan 23, 2025

• build(deps): bump undici from 6.21.0 to 6.21.1 in /docs/my-website

  #7902 merged Jan 23, 2025

• (Testing) - Add e2e testing for langfuse logging with tags

  #7922 merged Jan 22, 2025

• Deepseek r1 support + watsonx qa improvements

  #7907 merged Jan 22, 2025

• Update MLflow calllback and documentation

  #7809 merged Jan 22, 2025

• (Feat - prometheus) - emit litellm_overhead_latency_metric

  #7913 merged Jan 22, 2025

• Litellm dev 01 21 2025 p1

  #7898 merged Jan 22, 2025

• (Code quality) - Ban recursive functions in codebase

  #7910 merged Jan 22, 2025

• (Feat) Add x-litellm-overhead-duration-ms and "x-litellm-response-duration-ms" in response from LiteLLM

  #7899 merged Jan 22, 2025

• (fix langfuse tags) - read tags from StandardLoggingPayload

  #7903 merged Jan 22, 2025

• (Bug fix) - Allow setting null for max_budget, rpm_limit, tpm_limit when updating values on a team

  #7912 merged Jan 22, 2025

• fix: add default credential for azure (#7095)

  #7891 merged Jan 21, 2025

• fix(proxy_server.py): fix get model info when litellm_model_id is set + move model analytics to free

  #7886 merged Jan 21, 2025

• (e2e testing + minor refactor) - Virtual Key Max budget check

  #7888 merged Jan 21, 2025

• Litellm dev 01 20 2025 p3

  #7890 merged Jan 21, 2025

• Litellm dev 01 20 2025 p1

  #7884 merged Jan 21, 2025

• (Feat) datadog_llm_observability callback - emit request_tags on logs

  #7883 merged Jan 21, 2025

• fix(fireworks_ai/): fix global disable flag with transform messages h…

  #7847 merged Jan 21, 2025

• fix: add default credential for azure

  #7095 merged Jan 21, 2025

18 Pull requests opened by 13 people

• (UI Internal users) - allow searching by user_id or user_email field

  #7885 opened Jan 21, 2025

• build(docker-compose.yml): add volumes for db and formart

  #7893 opened Jan 21, 2025

• fix: add o1 on supported_stream_models and support for "reasoning_effort"

  #7917 opened Jan 22, 2025

• Support running Aim Guard during LLM call

  #7918 opened Jan 22, 2025

• (perf - RPS improvement)

  #7930 opened Jan 22, 2025

• add a step to install the proxy extra

  #7944 opened Jan 23, 2025

• Fix 7955: allow datetime filtering on /spend/logs

  #7956 opened Jan 24, 2025

• Fix bedrock model pricing + add unit test using bedrock pricing api

  #7978 opened Jan 24, 2025

• refactor: update PromptLayerLogger to use new log-request endpoint

  #7987 opened Jan 24, 2025

• Created streaming feature for AioHttp_OpenAI

  #7998 opened Jan 25, 2025

• Refactor bedrock message transformation to use async httpx for image urls

  #8001 opened Jan 25, 2025

• Bedrock document processing fixes

  #8005 opened Jan 25, 2025

• Fix stream chunk builder for provider_specific_fields, concatenate reasoning content on streaming for deepseek r1

  #8009 opened Jan 26, 2025

• Bing Search Pass Thru

  #8019 opened Jan 27, 2025

• (doc) Add nvidia as provider

  #8023 opened Jan 27, 2025

• Add smolagents

  #8026 opened Jan 27, 2025

• fix(utils.py): handle failed hf tokenizer request during calls

  #8032 opened Jan 27, 2025

• feat(handle_jwt.py): initial commit adding custom RBAC support on jwt…

  #8037 opened Jan 27, 2025

38 Issues closed by 10 people

• [Bug]: Inconsistent Response Wrapper Fields from LiteLLM API

  #8022 closed Jan 27, 2025

• Getting 404 error while connecting to 'http://localhost:11434/api/chat' even when http://localhost:11434/ shows ollama running

  #8010 closed Jan 26, 2025

• [Bug]: add model definition command-r7b-12-2024 from cohere

  #7256 closed Jan 26, 2025

• [Bug]: DeepSeek reasoning_content stream response returns None always

  #7942 closed Jan 26, 2025

• [Bug]: AWS Bedrock Nova error

  #7181 closed Jan 26, 2025

• [Bug]: If you create a key tied to a user that does not belong to a team, and then edit the key to add it to a team (the user is still not a part of a team), using that key results in an unexpected error

  #7986 closed Jan 26, 2025

• [Bug]: AWS Nova tool calling doesn't work with LiteLLM

  #7186 closed Jan 26, 2025

• v1.59.5 issue with Tracing on Langfuse v3.13.0 + prompt

  #7938 closed Jan 25, 2025

• [Bug]: logs getting blown up by "`logging_obj` not found - unable to track `llm_api_duration_ms`"

  #7982 closed Jan 25, 2025

• [Bug]: Set Ollama API Base via `OLLAMA_API_BASE` environment variable.

  #7997 closed Jan 25, 2025

• [Bug]: LiteLLM call failed: litellm.APIConnectionError: OllamaException - litellm.Timeout: Connection timed out after 600.0 seconds.

  #7996 closed Jan 25, 2025

• [Feature]: Allow setting guardrails default on

  #5758 closed Jan 25, 2025

• [Bug]: prisma cannot connect to local postgres

  #7968 closed Jan 25, 2025

• [Feature]: Ability to have different api keys for different models, even in the same provider

  #7981 closed Jan 25, 2025

• [Feature]: Work around Azure o1 not accepting 'system' (requires 'developer')

  #7961 closed Jan 24, 2025

• [Bug]: How to disable ssl verification for ollama?

  #6499 closed Jan 24, 2025

• Replicate completion response with status=processing does not retry

  #7900 closed Jan 24, 2025

• [Feature]: Callback traces in response JSON

  #7921 closed Jan 24, 2025

• [Bug]: UI - Custom logo not working

  #7895 closed Jan 24, 2025

• [Bug]: Following the example to generate a custom docker image does not generate database schema

  #7949 closed Jan 24, 2025

• [Bug]: LiteLLM failed to connect to vLLM backend

  #7937 closed Jan 23, 2025

• [Bug]: Langfuse - Error trace when logging

  #7906 closed Jan 23, 2025

• [Feature]: How to set supports_function_calling when add model in console but not in config.yml

  #7876 closed Jan 23, 2025

• [Feature]: How do I add my own model service?

  #7909 closed Jan 23, 2025

• isn't gpt-4 a visual model?

  #7908 closed Jan 23, 2025

• Which callback hook should I use to modify API response values?

  #7915 closed Jan 23, 2025

• [Feature]: Stream Parsing via Configurable Rules for New Prompt Manager (XML aka Native Anthropic Prompt Generator Support)

  #7914 closed Jan 23, 2025

• request for reopening #7855

  #7867 closed Jan 22, 2025

• When I making api_key, It's user_id is null

  #7911 closed Jan 22, 2025

• [Feature]: DeepSeek-R1 support

  #7877 closed Jan 22, 2025

• Azure Chat OpenAI - api key issue

  #7878 closed Jan 22, 2025

• Setting `DOCS_FILTERED="True"` causes 'Internal Server Error /openapi.json'

  #7892 closed Jan 22, 2025

• Feature: Gemini Flash 2.0

  #7188 closed Jan 22, 2025

• [Bug]: Different infor out come for model_info

  #7873 closed Jan 21, 2025

• [Bug]: Internal Server Error, 'NoneType' object is not iterable

  #7695 closed Jan 21, 2025

• [Bug]: Broken Providers in OpenRouters through LiteLLM

  #7887 closed Jan 21, 2025

• [Bug]: stream_timeout and timeout doesn't work for watsonx models during stream=True

  #7870 closed Jan 21, 2025

• [Bug]: USE_DDTRACE embedding failure

  #7889 closed Jan 21, 2025

45 Issues opened by 37 people

• [Bug]: No matter what you do, the api key created upon user creation is not properly linked to the user's team (Vulnerability)

  #8036 opened Jan 27, 2025

• add groq/deepseek-r1-distill-llama-70b models

  #8035 opened Jan 27, 2025

• [Bug]: Internal Users able to update their existing keys to another user's user_id (Vulnerability)

  #8031 opened Jan 27, 2025

• [Bug]: Nitpick, low priority: When a user tries to delete another users key, it throws a 500 error instead of a 400 or 403

  #8030 opened Jan 27, 2025

• [Bug]: Users able to escalate their model privilege (Vulnerability)

  #8029 opened Jan 27, 2025

• [Feature]: Refresh screen when action is done

  #8027 opened Jan 27, 2025

• [Bug]: latency-based routing fails on openai embedders with "Object of type timedelta is not JSON serializable"

  #8025 opened Jan 27, 2025

• Literal is too restrictive

  #8024 opened Jan 27, 2025

• [Bug]: The model produced invalid content Consider modifying your prompt if you are seeing this error persistently.

  #8020 opened Jan 27, 2025

• [Bug]: deepseek-reasoner does not support successive user or assistant messages

  #8018 opened Jan 27, 2025

• [Bug]: More Detailed error message for custom models

  #8017 opened Jan 27, 2025

• [Feature]: Split large files into smaller ones in order to facilitate agentic coding with LiteLLM itself

  #8015 opened Jan 27, 2025

• [Bug]: Callbacks not executed when using context window fallback dict

  #8014 opened Jan 27, 2025

• [Bug]: Regression in failure callback handling

  #8013 opened Jan 27, 2025

• [Bug]: Inconsistent stream output between OpenAI and LiteLLM clients during tool calling

  #8012 opened Jan 26, 2025

• Parameter Incompatibility with Perplexity-Sonar Model in Litellm Integration

  #8011 opened Jan 26, 2025

• [Feature]: Add support for kimi by moonshot.

  #8004 opened Jan 25, 2025

• [help] how to config a self hosted file server for Files API in litellm?

  #7995 opened Jan 25, 2025

• [Bug]: In the UI, you are unable to edit the models accessible to a key that is assigned to the default team

  #7985 opened Jan 24, 2025

• [Bug]: max_token not always supplied to replicate using the parameter name replicate expects

  #7984 opened Jan 24, 2025

• [Bug]: UI incorrectly throwing "User is not assigned to team" error when trying to create API key in team user is definitely assigned to

  #7983 opened Jan 24, 2025

• [Feature]: Support for Snowflake LLMs and Embedding models

  #7979 opened Jan 24, 2025

• [Feature]: `litellm --version` not requiring `proxy` extra

  #7975 opened Jan 24, 2025

• [Bug]: logprobs not included in suggestion response

  #7974 opened Jan 24, 2025

• [Bug]: [DeepSeek R1] 400 error for 2 consecutive messages with role user

  #7972 opened Jan 24, 2025

• [Bug]: the > main-v1.57.0 It's not work In my k8s env

  #7971 opened Jan 24, 2025

• [Feature]: Support for Anthropic Citations API

  #7970 opened Jan 24, 2025

• [Bug]: Huggingface models not fully functional

  #7969 opened Jan 24, 2025

• [Bug]: Adding Internal User to Organization doesn't work

  #7966 opened Jan 24, 2025

• [Feature]: Allow spend /spend/logs endpoint to accept a datetime filter for full log list

  #7955 opened Jan 24, 2025

• [Feature]: Add supports_response_schema for deepseek/deepseek-chat

  #7951 opened Jan 23, 2025

• [Bug]: `input_cost_per_token` does not work

  #7950 opened Jan 23, 2025

• Langfuse client is disabled

  #7948 opened Jan 23, 2025

• [Bug]: drop_params works on individual models but not on global litellm_settings

  #7947 opened Jan 23, 2025

• Persistent error on litellm

  #7943 opened Jan 23, 2025

• [Bug]: OpenTelemetry does not work

  #7940 opened Jan 23, 2025

• [Bug]: Base Url in Ollama model is treated as password and not added

  #7939 opened Jan 23, 2025

• [Feature] MCP bridge support

  #7934 opened Jan 23, 2025

• [Feature]: LitServe support

  #7927 opened Jan 22, 2025

• [Feature]: Template system for UI

  #7924 opened Jan 22, 2025

• [Bug]:

  #7923 opened Jan 22, 2025

• [Bug]: Vertex Credentials Are Cached

  #7904 opened Jan 21, 2025

• [Bug]: Using ChatLiteLLM with LangChain and Vertex AI

  #7897 opened Jan 21, 2025

• [Bug]: Azure File Attachment Missing When Adding to Thread using Threads Proxy Endpoint

  #7896 opened Jan 21, 2025

• [Feature]: Code refactor - clean up __init__.py

  #7894 opened Jan 21, 2025

268 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [Feature]: Support to lof traces across mutiple projects in Langfuse

  #7875 commented on Jan 21, 2025 • 0 new comments

• text_completion output issues

  #7874 commented on Jan 21, 2025 • 0 new comments

• [Feature]: Support for OCIGenAI Models

  #5915 commented on Jan 21, 2025 • 0 new comments

• Server failing to start while setting for Redis Semantic Cache [Bug]:

  #3056 commented on Jan 21, 2025 • 0 new comments

• [Bug]: Deepseek support for JSON response format

  #7580 commented on Jan 22, 2025 • 0 new comments

• [Bug]: Files missing in docker Litellm

  #7649 commented on Jan 22, 2025 • 0 new comments

• New Models/Endpoints/Providers

  #4922 commented on Jan 22, 2025 • 0 new comments

• [Bug]: Context/Citations/Intent object is missing in Azure response model when using chat extensions like AzureSearchChatDataSource

  #7245 commented on Jan 22, 2025 • 0 new comments

• [Bug]: OpenAI Multimodal models Do not allow file uploads

  #6120 commented on Jan 22, 2025 • 0 new comments

• Error 400 when using pydantic objects with default options defined with Google models.

  #7808 commented on Jan 23, 2025 • 0 new comments

• [Feature]: Allow configuring db model sync job schedule

  #7841 commented on Jan 23, 2025 • 0 new comments

• [Bug]: Langfuse HTTP headers never reach Langfuse API

  #7604 commented on Jan 23, 2025 • 0 new comments

• Docker Database connection Issue

  #7450 commented on Jan 23, 2025 • 0 new comments

• [Feature]: Force enable fake streaming per model

  #5416 commented on Jan 23, 2025 • 0 new comments

• [Bug]: Inconsistent response_format handling between Fireworks AI models

  #7533 commented on Jan 23, 2025 • 0 new comments

• [Bug]: usage-based-routing-v2 router retry logic doesn't respect `retry_after` or do backoff causing immediate failure

  #7669 commented on Jan 24, 2025 • 0 new comments

• [Bug]: Valid config keys have changed in V2

  #7560 commented on Jan 25, 2025 • 0 new comments

• [Feature]: `aiohttp` migration - 10-100x Higher RPS Master ticket

  #7544 commented on Jan 25, 2025 • 0 new comments

• [Bug]: analytics-python queue is full

  #5934 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Cannot pass provider-specific parameters to Bedrock Anthropic models

  #7782 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Gemini response with streaming not returning with usage

  #7798 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Anthropic usage prompt cache details missing from logging callbacks when streaming

  #7790 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Bedrock Token Usage Reporting Streaming vs. Non-Streaming

  #7112 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Broken Ollama completion transformation for tool calling

  #7570 commented on Jan 26, 2025 • 0 new comments

• [Bug]: Gemini 1.5 Flash 8B Error in Model Config

  #7269 commented on Jan 26, 2025 • 0 new comments

• 🎅 I WISH LITELLM HAD...

  #361 commented on Jan 27, 2025 • 0 new comments

• [Feature]: Add support for YandexGPT

  #1254 commented on Jan 27, 2025 • 0 new comments

• [Bug]: No usage info internal user

  #6298 commented on Jan 27, 2025 • 0 new comments

• [Feature]: Support `metadata` on OpenAI

  #6022 commented on Jan 27, 2025 • 0 new comments

• Getting error while using pr-agent for azure(ado) pull requests.

  #7644 commented on Jan 27, 2025 • 0 new comments

• [Feature]: Bedrock latency-optimized inference

  #7606 commented on Jan 27, 2025 • 0 new comments

• [Bug]: File uploads with purpose=batch error out with 'Invalid value for purpose.'

  #6582 commented on Jan 27, 2025 • 0 new comments

• [Bug]: Error spam due to prometheus metrics trying to be updated for non-premium user

  #7817 commented on Jan 27, 2025 • 0 new comments

• add function call response parser for non openai models

  #768 commented on Jan 22, 2025 • 0 new comments

• Added CLOVA studio Hyperclova X API support

  #853 commented on Jan 22, 2025 • 0 new comments

• feat: add aphrodite support

  #1153 commented on Jan 22, 2025 • 0 new comments

• (feat) added experiemental guidance function calling

  #1258 commented on Jan 22, 2025 • 0 new comments

• fix(utils.py): support complete_response=true for text completion streaming

  #1358 commented on Jan 22, 2025 • 0 new comments

• Litellm user budget fix

  #1479 commented on Jan 22, 2025 • 0 new comments

• feat(proxy_server.py): new /user/export endpoint

  #1486 commented on Jan 22, 2025 • 0 new comments

• fix(ollama): metrics handling

  #1514 commented on Jan 22, 2025 • 0 new comments

• fix(caching.py): add more debug statements for caching

  #1858 commented on Jan 22, 2025 • 0 new comments

• allow users to create their own keys

  #1870 commented on Jan 22, 2025 • 0 new comments

• fix(proxy_server.py): add better debug logging for sso callbacks

  #1965 commented on Jan 22, 2025 • 0 new comments

• [WIP] fix claude alternating messages

  #2374 commented on Jan 22, 2025 • 0 new comments

• fix(proxy_server.py): more efficient verification_token GET request

  #2392 commented on Jan 22, 2025 • 0 new comments

• chore(helm-chart): use default environment variable for master key

  #2432 commented on Jan 22, 2025 • 0 new comments

• Fix base_url for replicate's http api

  #2434 commented on Jan 22, 2025 • 0 new comments

• Litellm update redisvl

  #2444 commented on Jan 22, 2025 • 0 new comments

• :wrench: add credentials parameter to completion

  #2463 commented on Jan 22, 2025 • 0 new comments

• Integration with Canonical Neural Cache

  #2504 commented on Jan 22, 2025 • 0 new comments

• support ZhipuAI models

  #2514 commented on Jan 22, 2025 • 0 new comments

• add Lite llm docker proxy (Gemini ver)

  #2574 commented on Jan 22, 2025 • 0 new comments

• fix(main.py): Correctly route to `/completions` (if supported) when called for openai-compatible endpoints

  #2595 commented on Jan 22, 2025 • 0 new comments

• (feat) allow users to opt out of message merge - Anthropic

  #2671 commented on Jan 22, 2025 • 0 new comments

• feat(main.py): support calling text completione endpoint for openai compatible providers

  #2709 commented on Jan 22, 2025 • 0 new comments

• [NEW-MODEL] Add Solar model

  #2717 commented on Jan 22, 2025 • 0 new comments

• fix(utils.py): default usage tokens to 0

  #2736 commented on Jan 22, 2025 • 0 new comments

• Fix bug where 'custom_llm_provider' argument is not passed correctly in 'acompletion'

  #2758 commented on Jan 22, 2025 • 0 new comments

• fix: fix embedding response to return pydantic object

  #2784 commented on Jan 22, 2025 • 0 new comments

• The Spark API supports the completion method from the Litellm

  #3058 commented on Jan 22, 2025 • 0 new comments

• fix(main.py): support 'custom_llm_provider' in acompletion

  #3121 commented on Jan 22, 2025 • 0 new comments

• Litellm fix async text completions

  #3215 commented on Jan 22, 2025 • 0 new comments

• fix(router.py): check cache hits before making router.completion calls

  #3227 commented on Jan 22, 2025 • 0 new comments

• OpenAI chat completion message type annotation

  #3284 commented on Jan 22, 2025 • 0 new comments

• Support dashscope API for Qwen models

  #3344 commented on Jan 22, 2025 • 0 new comments

• fix(main.py): use model_api_key determined from get_api_key

  #3348 commented on Jan 22, 2025 • 0 new comments

• Supporting api key from the headers as well

  #3418 commented on Jan 22, 2025 • 0 new comments

• fixes #3264 and adds team_alias to /global/spend/teams

  #3454 commented on Jan 22, 2025 • 0 new comments

• Fix exception handling gemini

  #3493 commented on Jan 22, 2025 • 0 new comments

• fix(router.py): fix default cooldown time to be 60s

  #3529 commented on Jan 22, 2025 • 0 new comments

• fix: remove --accept_data_loss flag

  #3565 commented on Jan 22, 2025 • 0 new comments

• Fixed JWT public key finding

  #3648 commented on Jan 22, 2025 • 0 new comments

• Adding multiple public keys test

  #3649 commented on Jan 22, 2025 • 0 new comments

• [Optimize] Optimize the code for remove time complexity in llms bedro…

  #3665 commented on Jan 22, 2025 • 0 new comments

• Clean up prod prints - Convert print to log

  #3667 commented on Jan 22, 2025 • 0 new comments

• Add support for upserting users automatically to a default team based on JWT key

  #3717 commented on Jan 22, 2025 • 0 new comments

• Fixes #542 allow system messages + chat for palm api

  #3718 commented on Jan 22, 2025 • 0 new comments

• Support DashScope Compatible API For Qwen Series Models

  #3758 commented on Jan 22, 2025 • 0 new comments

• Feature/improved semantic cache

  #3907 commented on Jan 22, 2025 • 0 new comments

• Fix function call arg

  #3917 commented on Jan 22, 2025 • 0 new comments

• Code duplication in Handling Responses

  #3960 commented on Jan 22, 2025 • 0 new comments

• fix(http_handler.py): fix async client ssl verify

  #3985 commented on Jan 22, 2025 • 0 new comments

• feat(router.py): set default priority

  #3998 commented on Jan 22, 2025 • 0 new comments

• Fix the workflow to update the price

  #4045 commented on Jan 22, 2025 • 0 new comments

• Improve prediction response method

  #4073 commented on Jan 22, 2025 • 0 new comments

• Added type hints for model_list parameter in RouterConfig

  #4074 commented on Jan 22, 2025 • 0 new comments

• Fix: Trim message break possible infinite loop

  #4090 commented on Jan 22, 2025 • 0 new comments

• `assistants.md`: Add `user_api_end_user_max_budget` metadata to `litellm_params`

  #4113 commented on Jan 22, 2025 • 0 new comments

• astra-assistants api support

  #4118 commented on Jan 22, 2025 • 0 new comments

• Fix black at circle ci

  #4161 commented on Jan 22, 2025 • 0 new comments

• Clarifai: Fixed model name error and streaming

  #4170 commented on Jan 22, 2025 • 0 new comments

• Use presigned urls for S3 cache

  #4190 commented on Jan 22, 2025 • 0 new comments

• Linting Refactor: New `ModelResponseChunk` for streaming

  #4219 commented on Jan 22, 2025 • 0 new comments

• ci(config.yml): add pytest-xdist

  #4343 commented on Jan 22, 2025 • 0 new comments

• fix(parallel_request_limiter.py): support spend tracking caching across multiple instances

  #4396 commented on Jan 22, 2025 • 0 new comments

• fix(azure.py): Allow using Cloudflare AI gateway for embedding

  #4629 commented on Jan 22, 2025 • 0 new comments

• Solving the return value format issue during multiple function calls with the LLaMA 3 model.

  #4636 commented on Jan 22, 2025 • 0 new comments

• Proxy (health endpoints): Add `/health/db endpoint` w/ Prisma metrics

  #4660 commented on Jan 22, 2025 • 0 new comments

• fix(factory.py): Filter out empty messages before making llm api call

  #4678 commented on Jan 22, 2025 • 0 new comments

• Update ollama.py

  #4752 commented on Jan 22, 2025 • 0 new comments

• Add `--config` arg to k8s Deployment example in docs

  #4795 commented on Jan 22, 2025 • 0 new comments

• Control running lakera prompt checks - pre api call OR in parallel

  #4832 commented on Jan 22, 2025 • 0 new comments

• Print each model only once on startup

  #4867 commented on Jan 22, 2025 • 0 new comments

• fix parsing multi tool calls in stream_chunk_builder

  #4936 commented on Jan 22, 2025 • 0 new comments

• Integrating Not Diamond with LiteLLM

  #4971 commented on Jan 22, 2025 • 0 new comments

• fix(spend_tracking): `/spend/logs` with no filter

  #4998 commented on Jan 22, 2025 • 0 new comments

• Add `extra_headers` support for Databricks completion requests

  #5006 commented on Jan 22, 2025 • 0 new comments

• Optimize Alpine Dockerfile by removing redundant apk commands

  #5016 commented on Jan 22, 2025 • 0 new comments

• fix: PII output parsing for multiple entities of same type

  #5068 commented on Jan 22, 2025 • 0 new comments

• Keywords AI Integration

  #5130 commented on Jan 22, 2025 • 0 new comments

• Fixes priority queue comparison to work with Redis cache enabled

  #5268 commented on Jan 22, 2025 • 0 new comments

• Update team_endpoints.py

  #5269 commented on Jan 22, 2025 • 0 new comments

• Fix regression ignoring SSL_VERIFY boolean values being set through e…

  #5361 commented on Jan 22, 2025 • 0 new comments

• [Feat] add google ai studio ft models

  #5373 commented on Jan 22, 2025 • 0 new comments

• Litellm current branch

  #5398 commented on Jan 22, 2025 • 0 new comments

• Use patch instead of apatch for instructor

  #5404 commented on Jan 22, 2025 • 0 new comments

• Litellm azure ad token common helper

  #5440 commented on Jan 22, 2025 • 0 new comments

• Solving budget info update if the budget id exists

  #5465 commented on Jan 22, 2025 • 0 new comments

• Bump cryptography from 42.0.7 to 43.0.1

  #5496 commented on Jan 22, 2025 • 0 new comments

• Fixed #5559 (asyncio tasks get detroyed while pending sometimes)

  #5561 commented on Jan 22, 2025 • 0 new comments

• Upgrade dependencies

  #5665 commented on Jan 22, 2025 • 0 new comments

• Log assistants API calls to cloudwatch

  #5761 commented on Jan 22, 2025 • 0 new comments

• Add REST API examples to Vision documentation

  #5844 commented on Jan 22, 2025 • 0 new comments

• Update some of the python dependencies

  #5864 commented on Jan 22, 2025 • 0 new comments

• Upgrade prism lib

  #5866 commented on Jan 22, 2025 • 0 new comments

• Upgrade python packages

  #5867 commented on Jan 22, 2025 • 0 new comments

• Upgrade poetry lock file

  #5868 commented on Jan 22, 2025 • 0 new comments

• Install curl to be used for AWS ECS health check

  #5869 commented on Jan 22, 2025 • 0 new comments

• Update model_prices_and_context_window.json

  #5887 commented on Jan 22, 2025 • 0 new comments

• [Fix] UI Usage tab for internal users

  #5895 commented on Jan 22, 2025 • 0 new comments

• fix(helm): make actual use of `image.dbReadyImage` and `image.dbReadyTag`

  #5961 commented on Jan 22, 2025 • 0 new comments

• refactor: cleanup root of repo

  #5972 commented on Jan 22, 2025 • 0 new comments

• Adding native support for Snowflake's Cortex LLM service

  #5974 commented on Jan 22, 2025 • 0 new comments

• When defaulting claude-3 to tiktoken, avoid failed network call in huggingface `from_pretrained`

  #6030 commented on Jan 22, 2025 • 0 new comments

• (fix) Unable to override max_retries for Azure and Groq (#6138)

  #6151 commented on Jan 22, 2025 • 0 new comments

• (fix) prometheus db / redis service logging / system health

  #6152 commented on Jan 22, 2025 • 0 new comments

• bug fix for calls to community models in Replicate

  #6205 commented on Jan 22, 2025 • 0 new comments

• Merge in security fixes

  #6222 commented on Jan 22, 2025 • 0 new comments

• (testing) add testing coverage for intializing custom logger class

  #6225 commented on Jan 22, 2025 • 0 new comments

• Langtrace Integration: adjust `endpoint` & add `oTel` documentation

  #6259 commented on Jan 22, 2025 • 0 new comments

• Decode message to string in _types.py to avoid TypeError exception when using fallbacks

  #6281 commented on Jan 22, 2025 • 0 new comments

• feat(azure): Bump default version to latest.

  #6282 commented on Jan 22, 2025 • 0 new comments

• (fix) Unable to override max_retries for Azure and Groq

  #6285 commented on Jan 22, 2025 • 0 new comments

• Create CODE_OF_CONDUCT.md

  #6317 commented on Jan 22, 2025 • 0 new comments

• (refactor) `convert_to_model_response_object` to be under 100 LOC and and unit testing

  #6328 commented on Jan 22, 2025 • 0 new comments

• [BUG Fix] Issue With IBM WatsonX integration for Date Format

  #6346 commented on Jan 22, 2025 • 0 new comments

• fix(proxy): allow background health check interval to be any number

  #6368 commented on Jan 22, 2025 • 0 new comments

• (testing) add basic smoke test for prod config

  #6399 commented on Jan 22, 2025 • 0 new comments

• fix #6422: function calling when content is a list

  #6425 commented on Jan 22, 2025 • 0 new comments

• Fix Codestral endpoints

  #6431 commented on Jan 22, 2025 • 0 new comments

• Add txt file type in GCS URIs as accepted file type for gemini 1.5

  #6451 commented on Jan 22, 2025 • 0 new comments

• Fix tool call without args in gemini

  #6485 commented on Jan 22, 2025 • 0 new comments

• (fix) Memory leak when using async streaming

  #6523 commented on Jan 22, 2025 • 0 new comments

• (draft) fix stream memory leak

  #6526 commented on Jan 22, 2025 • 0 new comments

• Fix docker img create

  #6547 commented on Jan 22, 2025 • 0 new comments

• VertexAI System Instruction support for finetuned Gemini models

  #6553 commented on Jan 22, 2025 • 0 new comments

• Flake8 check to ensure `load_dotenv` is not called

  #6571 commented on Jan 22, 2025 • 0 new comments

• Fix incorrect cost calculation when creation cached input tokens in Anthropic

  #6576 commented on Jan 22, 2025 • 0 new comments

• (refactoring fix) use standard_logging_payload for `_PROXY_track_cost_callback`

  #6585 commented on Jan 22, 2025 • 0 new comments

• update: renamed model from llama 3 to 3.1

  #6595 commented on Jan 22, 2025 • 0 new comments

• Jwt auth support ecdsa

  #6598 commented on Jan 22, 2025 • 0 new comments

• bedrock: add apac claude support for cross region inferencing

  #6630 commented on Jan 22, 2025 • 0 new comments

• feat: add default model region to proxy config

  #6639 commented on Jan 22, 2025 • 0 new comments

• (QOL improvement): Slack alerting - Add testing for alerts that use caching

  #6648 commented on Jan 22, 2025 • 0 new comments

• Generic sso support pkce

  #6651 commented on Jan 22, 2025 • 0 new comments

• (feat) Docker.non_root improvements for handing `nobody` user

  #6656 commented on Jan 22, 2025 • 0 new comments

• Custom Model Provider zhipu

  #6661 commented on Jan 22, 2025 • 0 new comments

• Litellm openai metadata param support v3

  #6665 commented on Jan 27, 2025 • 0 new comments

• fix(router.py): fix moderation pass through logic

  #6670 commented on Jan 22, 2025 • 0 new comments

• add scope3 logger and docs

  #6684 commented on Jan 22, 2025 • 0 new comments

• Litellm prometheus refactor - move to using standard logging payload

  #6689 commented on Jan 22, 2025 • 0 new comments

• removing max_retries param of custom vertex_ai model

  #6692 commented on Jan 22, 2025 • 0 new comments

• feat(prometheus_api.py): support reading model metrics from prometheu…

  #6695 commented on Jan 22, 2025 • 0 new comments

• Pass timeout to Sagemaker LLM/Databricks

  #6697 commented on Jan 22, 2025 • 0 new comments

• Fix open_text depreciation warning

  #6719 commented on Jan 22, 2025 • 0 new comments

• Replace depreciated class Config with model_config

  #6720 commented on Jan 22, 2025 • 0 new comments

• Add nebius ai studio support

  #6725 commented on Jan 22, 2025 • 0 new comments

• feat: add groq llama 3.2 models and update groq model feature list

  #6767 commented on Jan 22, 2025 • 0 new comments

• fix: duplicate exception_type for gemini

  #6768 commented on Jan 22, 2025 • 0 new comments

• docs: fix sample code for langfuse_integration

  #6780 commented on Jan 22, 2025 • 0 new comments

• chore: ruff.toml about ignore and select

  #6782 commented on Jan 22, 2025 • 0 new comments

• fix anthropic tool image content

  #6788 commented on Jan 22, 2025 • 0 new comments

• Update anthropic.md

  #6794 commented on Jan 22, 2025 • 0 new comments

• Allowing list as function call response to support computer use

  #6795 commented on Jan 22, 2025 • 0 new comments

• Bump aiohttp from 3.10.2 to 3.10.11

  #6800 commented on Jan 22, 2025 • 0 new comments

• Ensure cost per token is float.

  #6811 commented on Jan 22, 2025 • 0 new comments

• (fix) Pass `subject` with correct value to OpenMeter in OpenMeter integration

  #6816 commented on Jan 22, 2025 • 0 new comments

• add custom health probes in helm chart

  #6851 commented on Jan 22, 2025 • 0 new comments

• (stable nov 21st release)

  #6863 commented on Jan 22, 2025 • 0 new comments

• Create GPTLocalhost.md to showcase the integration between LiteLLM and Microsoft Word

  #6877 commented on Jan 22, 2025 • 0 new comments

• feat - add vision chat for ollama_chat

  #6880 commented on Jan 22, 2025 • 0 new comments

• Modifying exception classes that descend from BadRequestError …

  #6883 commented on Jan 22, 2025 • 0 new comments

• Update docs and models files to reflect Perplexity supported models (November 2024)

  #6888 commented on Jan 22, 2025 • 0 new comments

• (fix) auth checks - Add regex matching for `models` on virtual keys / teams

  #6901 commented on Jan 22, 2025 • 0 new comments

• Support retry policy for completion / acompletion

  #6916 commented on Jan 22, 2025 • 0 new comments

• fix(model prices & context window) - add claude-3-5-haiku-latest & cl…

  #6935 commented on Jan 22, 2025 • 0 new comments

• (stable release - nov 27th)

  #6949 commented on Jan 22, 2025 • 0 new comments

• NVIDIA fixes for chat completion impl

  #6981 commented on Jan 27, 2025 • 0 new comments

• Convert tool use arguments to string before counting tokens

  #6989 commented on Jan 22, 2025 • 0 new comments

• Add 4o-mini on openrouter

  #7012 commented on Jan 22, 2025 • 0 new comments

• Use litellm.client_session for ollama and ollama_chat

  #7028 commented on Jan 22, 2025 • 0 new comments

• fix: apply api_key param to amoderation factory

  #7049 commented on Jan 22, 2025 • 0 new comments

• feat,docs: instructions for using a runtime debugger with liteLLM

  #7055 commented on Jan 22, 2025 • 0 new comments

• refactor: add type annotations and overloads to completion functions

  #7057 commented on Jan 22, 2025 • 0 new comments

• Added a guide for users who want to use LiteLLM with AI/ML API.

  #7058 commented on Jan 22, 2025 • 0 new comments

• Update model json to add gemini-exp-1121

  #7061 commented on Jan 22, 2025 • 0 new comments

• Add AgentOps Integration Documentation

  #7062 commented on Jan 22, 2025 • 0 new comments

• Include error message if no error text

  #7072 commented on Jan 22, 2025 • 0 new comments

• Code QOL improvement - remove unused imports, attempt #2

  #7093 commented on Jan 22, 2025 • 0 new comments

• feat(ChatCompletionDeltaToolCall): add dictionary-like access methods

  #7100 commented on Jan 22, 2025 • 0 new comments

• feat: parameter added to use custom tokenizer with usage-based-routing strategies.

  #7106 commented on Jan 22, 2025 • 0 new comments

• Set Default custom_llm_provider = “openai”

  #7124 commented on Jan 22, 2025 • 0 new comments

• add extra_headers for ollama completion

  #7180 commented on Jan 22, 2025 • 0 new comments

• added gemini-2.0-flash-exp

  #7214 commented on Jan 22, 2025 • 0 new comments

• Update Debug Logging Instructions to reflect the new approach

  #7225 commented on Jan 22, 2025 • 0 new comments

• (stable dec 14th release)

  #7227 commented on Jan 22, 2025 • 0 new comments

• 1215同步

  #7243 commented on Jan 22, 2025 • 0 new comments

• Update map_finish_reason

  #7264 commented on Jan 22, 2025 • 0 new comments

• build(deps): bump next from 14.2.10 to 14.2.15 in /ui/litellm-dashboard

  #7272 commented on Jan 23, 2025 • 0 new comments

• build(deps): bump the github-actions group across 1 directory with 7 updates

  #7278 commented on Jan 22, 2025 • 0 new comments

• (fix proxy) fix logic for caching virtual keys in memory / redis

  #7285 commented on Jan 22, 2025 • 0 new comments

• Support vllm quantization

  #7297 commented on Jan 22, 2025 • 0 new comments

• NVIDIA: Tool calling, structured output, vlm models

  #7331 commented on Jan 22, 2025 • 0 new comments

• ci: remove '-s' from router testing

  #7340 commented on Jan 22, 2025 • 0 new comments

• build(pyproject.toml): remove 'click' and 'load-dotenv' from litellm sdk dep's

  #7354 commented on Jan 22, 2025 • 0 new comments

• Litellm fix get user info proxy admin

  #7362 commented on Jan 22, 2025 • 0 new comments

• Support gemini 2.0 thinking chat mode

  #7370 commented on Jan 22, 2025 • 0 new comments

• Add robust token counter with 0 default on failure for ollama_chat

  #7380 commented on Jan 22, 2025 • 0 new comments

• build(deps): bump jinja2 from 3.1.4 to 3.1.5

  #7387 commented on Jan 23, 2025 • 0 new comments

• fix: deployment_id for cooldown_handlers.py

  #7404 commented on Jan 22, 2025 • 0 new comments

• Code QA - Add Azure OpenAI to base llm testing

  #7405 commented on Jan 22, 2025 • 0 new comments

• Add double quotes to prevent view names from being lowercased

  #7409 commented on Jan 22, 2025 • 0 new comments

• (draft pr testing 0)

  #7562 commented on Jan 22, 2025 • 0 new comments

• Feature/openai modify assistant

  #7574 commented on Jan 22, 2025 • 0 new comments

• Add new model provider Novita AI

  #7582 commented on Jan 27, 2025 • 0 new comments

• Fix pydantic warning in prometheus integration

  #7583 commented on Jan 22, 2025 • 0 new comments

• feat: offline support for tiktoken

  #7588 commented on Jan 22, 2025 • 0 new comments

• fix(asr-groq): calculate Groq audio transcription cost

  #7610 commented on Jan 22, 2025 • 0 new comments

• Add nest_asyncio to fix ollama embed 'event loop closed' error

  #7625 commented on Jan 22, 2025 • 0 new comments

• refactor: polish tiktoken encoding_for_model

  #7626 commented on Jan 22, 2025 • 0 new comments

• Fix __init__ types on Message class

  #7632 commented on Jan 22, 2025 • 0 new comments

• Parallelize router initialization

  #7648 commented on Jan 22, 2025 • 0 new comments

• feat: add offline swagger docs

  #7653 commented on Jan 22, 2025 • 0 new comments

• (draft)

  #7694 commented on Jan 22, 2025 • 0 new comments

• Litellm use lru cache

  #7698 commented on Jan 22, 2025 • 0 new comments

• (router speedup)

  #7707 commented on Jan 22, 2025 • 0 new comments

• (perf proxy) - only read request body 1 time. (no need to read the same request body in Auth and on actual routes)

  #7710 commented on Jan 22, 2025 • 0 new comments

• fix_delta_get_default

  #7716 commented on Jan 22, 2025 • 0 new comments

• fix redis sentinel client handling to solve authentication error with password protected sentinel

  #7718 commented on Jan 22, 2025 • 0 new comments

• Adding openrouter llama 3.1 8b instruct models

  #7725 commented on Jan 22, 2025 • 0 new comments

• (docs) Update vertex.md old code example

  #7736 commented on Jan 22, 2025 • 0 new comments

• fix(utils): dangling async tasks

  #7737 commented on Jan 22, 2025 • 0 new comments

• fix: dangling async tasks

  #7741 commented on Jan 22, 2025 • 0 new comments

• Add response format option to hugging face

  #7747 commented on Jan 22, 2025 • 0 new comments

• Support arize phoenix on litellm proxy

  #7756 commented on Jan 22, 2025 • 0 new comments

• (fixes) updating `soft_budget` on virtual keys

  #7757 commented on Jan 22, 2025 • 0 new comments

• Feat/braintrust attributes

  #7759 commented on Jan 22, 2025 • 0 new comments

• feat(amberflo_metering.py): adding support for usage metering with amberflo

  #7766 commented on Jan 22, 2025 • 0 new comments

• Add beta header computer-use-2024-10-22 when the tools bash_20241022 or text_editor_20241022 are used

  #7805 commented on Jan 27, 2025 • 0 new comments

• fix: pydantic fields of UserAPIKeyLabelValues

  #7813 commented on Jan 27, 2025 • 0 new comments

• fix: now get_key_model_tpm and rpm limit looks for limits right place

  #7814 commented on Jan 22, 2025 • 0 new comments

• Add a `Tiltfile` for fast LiteLLM Proxy development with Tilt

  #7816 commented on Jan 22, 2025 • 0 new comments

• LiteLLM Minor Fixes & Improvements (01/16/2025) - p2

  #7828 commented on Jan 27, 2025 • 0 new comments

• (doc) - spec for using batches api with self hosted models

  #7837 commented on Jan 22, 2025 • 0 new comments

• add support for Amazon Nova Canvas model

  #7838 commented on Jan 22, 2025 • 0 new comments

• build(pyproject.toml): don't use orjson 3.10.15 - rust error

  #7858 commented on Jan 22, 2025 • 0 new comments

• (Testing / QA) - Add e2e langfuse logging tests

  #7862 commented on Jan 22, 2025 • 0 new comments