This Streamlit app helps fine-tune and deploy LLMs using your cloud (AWS, GCP, Azure, Lambda Cloud, TensorDock, Vast.ai, etc.) via user interface and access them for inference.

To run workloads in the cloud, the app uses dstack.

pip install -r requirements.txt

If you have default AWS, GCP, or Azure credentials on your machine, the dstack server will pick them up automatically.

Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml. For further details, refer to server configuration.

Once clouds are configured, start it:

dstack server

Now you're good to run the app.

streamlit run Inference.py