Milvus is an open-source vector database that is highly flexible, reliable, and blazing fast. It supports adding, deleting, updating, and near real-time search of vectors on a trillion-byte scale.
This package is an API Client for the Milvus v2.3.3 Restful API, and is built on the Saloon package.
Documentation about the Restful API is available on the Milvus website, and an OpenAPI spec is available here.
| Milvus Version | PHP Client Version |
|---|---|
| v2.3.x | v0.0.x |
| v2.2.x | Not supported (*) |
(*) But is mostly compatible, the only difference (that I can see) between them is the new Vector Upsert endpoint, and
new parameters (params.range_filter and params.radius) in the Vector Search endpoint.
You can install the package via composer:
composer require helgesverre/milvusYou can publish the config file with:
php artisan vendor:publish --tag="milvus-config"This is the contents of the published config/milvus.php file:
return [
'token' => env('MILVUS_TOKEN'),
'username' => env('MILVUS_USERNAME'),
'password' => env('MILVUS_PASSWORD'),
'host' => env('MILVUS_HOST', 'localhost'),
'port' => env('MILVUS_PORT', '19530'),
];For Laravel users, you can use the Milvus facade to interact with the Milvus API:
use HelgeSverre\Milvus\Facades\Milvus;
// NOTE: dbName is optional and defaults to 'default', this is only relevant if you have multiple databases.
// List all collections in the 'default' database
Milvus::collections()->list(
dbName: 'default'
);
// Create a new collection named 'documents' in the 'default' database with a specified dimension
Milvus::collections()->create(
collectionName: 'documents',
dimension: 128,
dbName: 'default',
);
// Describe the structure and properties of the 'documents' collection in the 'default' database
Milvus::collections()->describe(
collectionName: 'documents',
dbName: 'default',
);
// Drop or delete the 'documents' collection from the 'default' database
Milvus::collections()->drop(
collectionName: 'documents',
dbName: 'default',
);
// Insert a new vector into the 'documents' collection with additional fields like title and link
// Note "vector" is a reserved field name and must be used for the vector data
Milvus::vector()->insert(
collectionName: 'documents',
data: [
'vector' => [0.1, 0.2, 0.3 /* etc... */],
"title" => "Document name here",
"link" => "https://example.com/document-name-here",
]
);
// Search for similar vectors in the 'documents' collection using a provided vector
Milvus::vector()->search(
collectionName: 'documents',
vector: [0.1, 0.2, 0.3 /* etc... */],
);
// Delete a vector from the 'documents' collection using its ID
Milvus::vector()->delete(
id: '123129471497',
collectionName: 'documents'
);
// Query the 'documents' collection for specific documents using a filter condition and select specific output fields
Milvus::vector()->query(
collectionName: 'documents',
filter: "id in [443300716234671427, 443300716234671426]",
outputFields: ["id", "title", "link"],
);
// Retrieve a specific vector from the 'documents' collection using its ID
Milvus::vector()->get(
id: '123129471497',
collectionName: 'documents'
);
// Update or insert a vector in the 'documents' collection. If the ID exists, it's updated; if not, a new entry is created
Milvus::vector()->upsert(
collectionName: 'documents',
data: [
'id' => 123129471497,
'vector' => [0.1, 0.2, 0.3 /* etc... */],
"title" => "Document name here",
"link" => "https://example.com/document-name-here",
]
);If you are not using laravel, you will have to create a new instance of the Milvus class and provide a token or user/pass, the host and the port.
<?php
// use HelgeSverre\Milvus\Facades\Milvus;
use HelgeSverre\Milvus\Milvus;
$milvus = new Milvus(
token: "your-token",
host: "localhost",
port: "19530"
);
// Import the Milvus facade for easier access to Milvus functions
// NOTE: dbName is optional and defaults to 'default', this is only relevant if you have multiple databases.
// List all collections in the 'default' database
$milvus->collections()->list(
dbName: 'default'
);
// Create a new collection named 'documents' in the 'default' database with a specified dimension
$milvus->collections()->create(
collectionName: 'documents',
dimension: 128,
dbName: 'default',
);
// Describe the structure and properties of the 'documents' collection in the 'default' database
$milvus->collections()->describe(
collectionName: 'documents',
dbName: 'default',
);
// Drop or delete the 'documents' collection from the 'default' database
$milvus->collections()->drop(
collectionName: 'documents',
dbName: 'default',
);
// Insert a new vector into the 'documents' collection with additional fields like title and link
// Note "vector" is a reserved field name and must be used for the vector data
$milvus->vector()->insert(
collectionName: 'documents',
data: [
'vector' => [0.1, 0.2, 0.3 /* etc... */],
"title" => "Document name here",
"link" => "https://example.com/document-name-here",
]
);
// Search for similar vectors in the 'documents' collection using a provided vector
$milvus->vector()->search(
collectionName: 'documents',
vector: [0.1, 0.2, 0.3 /* etc... */],
);
// Delete a vector from the 'documents' collection using its ID
$milvus->vector()->delete(
id: '123129471497',
collectionName: 'documents'
);
// Query the 'documents' collection for specific documents using a filter condition and select specific output fields
$milvus->vector()->query(
collectionName: 'documents',
filter: "id in [443300716234671427, 443300716234671426]",
outputFields: ["id", "title", "link"],
);
// Retrieve a specific vector from the 'documents' collection using its ID
$milvus->vector()->get(
id: '123129471497',
collectionName: 'documents'
);
// Update or insert a vector in the 'documents' collection. If the ID exists, it's updated; if not, a new entry is created
$milvus->vector()->upsert(
collectionName: 'documents',
data: [
'id' => 123129471497,
'vector' => [0.1, 0.2, 0.3 /* etc... */],
"title" => "Document name here",
"link" => "https://example.com/document-name-here",
]
);If you are using the hosted version of Milvus, you will need to specify the following host and port along with your API token:
use HelgeSverre\Milvus\Milvus;
$milvus = new Milvus(
token: "db_randomstringhere:passwordhere",
host: 'https://in03-somerandomstring.api.gcp-us-west1.zillizcloud.com',
port: '443'
);This example demonstrates how to perform a semantic search in Milvus using embeddings generated from OpenAI.
First, create an array of data you wish to index. In this example, we'll use blog posts with titles, summaries, and tags.
$blogPosts = [
[
'title' => 'Exploring Laravel',
'summary' => 'A deep dive into Laravel frameworks...',
'tags' => ['PHP', 'Laravel', 'Web Development']
],
[
'title' => 'Exploring Laravel',
'summary' => 'A deep dive into Laravel frameworks, exploring its features and benefits for modern web development.',
'tags' => ['PHP', 'Laravel', 'Web Development']
],
[
'title' => 'Introduction to React',
'summary' => 'Understanding the basics of React and how it revolutionizes frontend development.',
'tags' => ['JavaScript', 'React', 'Frontend']
],
[
'title' => 'Getting Started with Vue.js',
'summary' => 'A beginner’s guide to building interactive web interfaces with Vue.js.',
'tags' => ['JavaScript', 'Vue.js', 'Frontend']
],
];Use OpenAI's embeddings API to convert the summaries of your blog posts into vector embeddings.
$summaries = array_column($blogPosts, 'summary');
$embeddingsResponse = OpenAI::client('sk-your-openai-api-key')
->embeddings()
->create([
'model' => 'text-embedding-ada-002',
'input' => $summaries,
]);
foreach ($embeddingsResponse->embeddings as $embedding) {
$blogPosts[$embedding->index]['vector'] = $embedding->embedding;
}Create a collection in Milvus to store your blog post embeddings, note that the dimension of the embeddings must match
the dimension of the embeddings generated by OpenAI (1536 if you are using the text-embedding-ada-002 model).
$milvus = new Milvus(
token: "your-token",
host: "localhost",
port: "19530"
);
$milvus->collections()->create(
collectionName: 'blog_posts',
dimension: 1536,
);Insert these embeddings, along with other blog post data, into your Milvus collection.
$insertResponse = $milvus->vector()->insert('blog_posts', $blogPosts);Generate a search vector for your query, akin to how you processed the blog posts.
$searchVectorResponse = OpenAI::client('sk-your-openai-api-key')
->embeddings()
->create([
'model' => 'text-embedding-ada-002',
'input' => 'laravel framework',
]);
$searchEmbedding = $searchVectorResponse->embeddings[0]->embedding;Use the Milvus client to perform a search with the generated embedding.
$searchResponse = $milvus->vector()->search(
collectionName: 'blog_posts',
vector: $searchEmbedding,
limit: 3,
outputFields: ['title', 'summary', 'tags']
);
// Output the search results
foreach ($searchResponse as $result) {
echo "Title: " . $result['title'] . "\n";
echo "Summary: " . $result['summary'] . "\n";
echo "Tags: " . implode(', ', $result['tags']) . "\n\n";
}To quickly get started with Milvus, you can run it in Docker, by using the following command
# Download the docker-compose.yml file
wget https://github.com/milvus-io/milvus/releases/download/v2.3.3/milvus-standalone-docker-compose.yml -O docker-compose.yml
# Start Milvus
docker compose up -dA healthcheck endpoint will now be available on http://localhost:9091/healthz, and the Milvus API will be available
on http://localhost:19530.
To stop Milvus, run docker compose down, to wipe all the data, run docker compose down -v.
For more details Installing Milvus Standalone with Docker Compose
For production workloads, consider checking out Zilliz.com, which are the developers behind Milvus and provides a hosted version of Milvus in the Cloud ☁️.
cp .env.example .env
## Start a local Milvus instance, it takes awhile to boot up
docker compose up -d
composer test
composer analyse srcThe MIT License (MIT). Please see License File for more information.
"Milvus®" and the Milvus logo are registered trademarks of the Linux Foundation (LF Projects, LLC). This package is not affiliated with, endorsed by, or sponsored by the Linux Foundation. It's developed independently and uses the "Milvus" name under fair use, solely for identification. All trademarks and registered trademarks, including "Milvus®", are the property of their respective owners. "Milvus®" is a registered trademark of the Linux Foundation.