Skip to content

jinyang1994/BentoML

ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

bentoml

BentoML: The Unified AI Application Framework

pypi_status CI Twitter Community

BentoML is a framework for building reliable, scalable, and cost-efficient AI applications. It comes with everything you need for model serving, application packaging, and production deployment.

๐Ÿ‘‰ Join our Slack community!

Highlights

๐Ÿฑ Bento is the container for AI apps

  • Open standard and SDK for AI apps, pack your code, inference pipelines, model files, dependencies, and runtime configurations in a Bento.
  • Auto-generate API servers, supporting REST API, gRPC, and long-running inference jobs.
  • Auto-generate Docker container images.

๐Ÿ„ Freedom to build with any AI models

๐Ÿญ Simplify modern AI application architecture

๐Ÿš€ Deploy Anywhere

  • One-click deployment to โ˜๏ธ BentoCloud, the Serverless platform made for hosting and operating AI apps.
  • Scalable BentoML deployment with ๐Ÿฆ„๏ธ Yatai on Kubernetes.
  • Deploy auto-generated container images anywhere docker runs.

Documentation

๐Ÿ› ๏ธ What you can build with BentoML

Getting Started

Save or import models in BentoML local model store:

import bentoml
import transformers

pipe = transformers.pipeline("text-classification")

bentoml.transformers.save_model(
  "text-classification-pipe",
  pipe,
  signatures={
    "__call__": {"batchable": True}  # Enable dynamic batching for model
  }
)

View all models saved locally:

$ bentoml models list

Tag                                     Module                Size        Creation Time
text-classification-pipe:kn6mr3aubcufโ€ฆ  bentoml.transformers  256.35 MiB  2023-05-17 14:36:25

Define how your model runs in a service.py file:

import bentoml

model_runner = bentoml.models.get("text-classification-pipe").to_runner()

svc = bentoml.Service("text-classification-service", runners=[model_runner])

@svc.api(input=bentoml.io.Text(), output=bentoml.io.JSON())
async def classify(text: str) -> str:
    results = await model_runner.async_run([text])
    return results[0]

Now, run the API service locally:

bentoml serve service.py:svc

Sent a prediction request:

$ curl -X POST -H "Content-Type: text/plain" --data "BentoML is awesome" http://localhost:3000/classify

{"label":"POSITIVE","score":0.9129443168640137}%

Define how a Bento can be built for deployment, with bentofile.yaml:

service: 'service.py:svc'
name: text-classification-svc
include:
  - 'service.py'
python:
  packages:
  - torch>=2.0
  - transformers

Build a Bento and generate a docker image:

$ bentoml build
...
Successfully built Bento(tag="text-classification-svc:mc322vaubkuapuqj").
$ bentoml containerize text-classification-svc
Building OCI-compliant image for text-classification-svc:mc322vaubkuapuqj with docker
...
Successfully built Bento container for "text-classification-svc" with tag(s) "text-classification-svc:mc322vaubkuapuqj"
$ docker run -p 3000:3000 text-classification-svc:mc322vaubkuapuqj

For a more detailed user guide, check out the BentoML Tutorial.


Community

BentoML supports billions of model runs per day and is used by thousands of organizations around the globe.

Join our Community Slack ๐Ÿ’ฌ, where thousands of AI application developers contribute to the project and help each other.

To report a bug or suggest a feature request, use GitHub Issues.

Contributing

There are many ways to contribute to the project:

  • Report bugs and "Thumbs up" on issues that are relevant to you.
  • Investigate issues and review other developers' pull requests.
  • Contribute code or documentation to the project by submitting a GitHub pull request.
  • Check out the Contributing Guide and Development Guide to learn more
  • Share your feedback and discuss roadmap plans in the #bentoml-contributors channel here.

Thanks to all of our amazing contributors!


Usage Reporting

BentoML collects usage data that helps our team to improve the product. Only BentoML's internal API calls are being reported. We strip out as much potentially sensitive information as possible, and we will never collect user code, model data, model names, or stack traces. Here's the code for usage tracking. You can opt-out of usage tracking by the --do-not-track CLI option:

bentoml [command] --do-not-track

Or by setting environment variable BENTOML_DO_NOT_TRACK=True:

export BENTOML_DO_NOT_TRACK=True

License

Apache License 2.0

FOSSA Status

Citation

If you use BentoML in your research, please cite using the following citation:

@software{Yang_BentoML_The_framework,
author = {Yang, Chaoyu and Sheng, Sean and Pham, Aaron and  Zhao, Shenyang and Lee, Sauyon and Jiang, Bo and Dong, Fog and Guan, Xipeng and Ming, Frost},
license = {Apache-2.0},
title = {{BentoML: The framework for building reliable, scalable and cost-efficient AI application}},
url = {https://github.com/bentoml/bentoml}
}

About

Build Production-Grade AI Applications

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 95.2%
  • Shell 2.4%
  • Starlark 0.8%
  • Jinja 0.5%
  • Dockerfile 0.3%
  • Swift 0.2%
  • Other 0.6%