Flux is one of the most advanced text-to-image model families yet. This demo exposes the Dev model via a simple HTTP server, thanks to Replicate Cog. It can be adapted to use the Schnell model which is faster and comes with an Apache-2.0 license. Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container. When you're up and running, you can generate images using the /predictions endpoint. Images are automatically stored in object-storage on Tigris (you'll need to make sure you add a bucket to the app).

fly apps create --name [APP_NAME]

fly storage create

fly secrets set HUGGING_FACE_HUB_TOKEN=<HUGGING_FACE_TOKEN>

cog push registry.fly.io/[APP_NAME]:latest --use-cuda-base-image false

Now replace image in your fly.toml, then:

fly deploy

curl --silent https://cog-flux.fly.dev/predictions/test \
    --request PUT \
    --header "Content-Type: application/json" \
    --data @- << 'EOF' | jq
{
    "input": {
        "prompt": "Starry night dreamscape with purple hot-air balloons",
        "aspect_ratio": "16:9",
        "guidance_scale": 3.5,
        "num_inference_steps": 50,
        "max_sequence_length": 512,
        "output_format": "png"
    }
}
EOF

Now view your image at https://fly.storage.tigris.dev/[BUCKET_NAME]/[OUTPUT_IMAGE_NAME]

Create an issue or ask a question here: https://community.fly.io/