Enhancements to OpenAI Embedding: Batch Processing, Customer Support and Dynamic Model Selection #1844
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In this update to the `OpenAIEmbedding` class, we have introduced several significant enhancements to improve functionality and performance. 1. Batch Processing Support: The embedding method has been enhanced to handle multiple texts in a single request. This change allows for more efficient processing by reducing the number of API calls. Instead of processing one text at a time, you can now pass a list of texts to the `embed` method, which the class will handle in a single API request. This batch processing improves throughput and minimizes latency. 2. Asynchronous Support: We have incorporated asynchronous handling into the `embed` method to enable non-blocking operations. By using Python's `asyncio` library, the embedding requests can be processed concurrently, which is particularly beneficial for applications with high volumes of text or those requiring real-time responsiveness. 3. Dynamic Model Selection: The class now supports dynamic model selection based on input parameters. This allows users to choose different OpenAI models for embedding depending on their needs, such as performance versus accuracy trade-offs. The model selection is more flexible, making it easier to adapt to various use cases. 4. Enhanced Error Handling: I mproved error handling has been added to provide clearer and more informative error messages. This ensures that issues such as API rate limits or invalid inputs are handled gracefully, with appropriate feedback provided to users. 5. Optimized API Integration: The initialization process for the OpenAI client has been streamlined. Environment variables for API keys and base URLs are checked, and defaults are set if they are not provided. This ensures that the client is properly configured and reduces the likelihood of configuration errors. Overall, these updates enhance the usability, efficiency, and flexibility of the `OpenAIEmbedding` class, making it better suited for a variety of applications involving text embeddings.
Enhance OpenAIEmbedding Class with Batch Processing, Async Support, and Dynamic Model Selection
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
In the current update of the OpenAI embedding script there are new features such as batch processing, asynchronous support, dynamic model selection, and improved error handling. These enhancements are concerned with increasing the performance, flexibility, and the level of error shown to the user.
Related Issues
None on associated concerns, where proactive performance enhancement is being implemented.
Discussions
Some discussion focused on enhancing script throughput by means of batch operations and async capability.
QA Instructions
To validate the enhancements, one should perform the embedding requests in both batch and async modes.
Merge Plan
Before merge, make certain that all changes are good with different sizes of input.
Motivation and Context
This update is focused on enhancing the work of requests’ embedding and streamlines methods.
How we test:
Performance enhancements: Batch mode capability with asynchronous features.
Flexibility: Dynamic model selection.
Improved error handling: Saying the wrong thing at the wrong time.
Please delete options that are not relevant.
Checklist:
Maintainer Checklist