LocalAI
LocalAI is a drop-in replacement for the OpenAI API that runs entirely on your own hardware. One binary serves text generation, image generation, speech-to-text and text-to-speech behind OpenAI-compatible endpoints, backed by a model gallery and pluggable inference backends. Heavier than llama.cpp, but multi-modal and self-managing.
LocalAI
In Development
This script is currently in active development and may be unstable or incomplete. Use in production environments is not recommended.
This script is currently in active development and may be unstable or incomplete. Use in production environments is not recommended.
Installation
Default install:
1
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVED/main/ct/localai.sh)"
Notes
No models are installed by default. Use the gallery in the web UI to pull one, or drop GGUF files into /opt/localai_data/models. Inference backends are fetched on first use and can be several GB each, which is what the 30 GB default disk is for.
The API is protected from the moment it starts. With user accounts off, a static API key is generated and saved to ~/localai.creds in the container; clients send it as 'Authorization: Bearer ' or an 'x-api-key' header, and it grants full admin access. With user accounts on, register in the web UI immediately after install - the first account created becomes the admin.</div> </div> ## Web Interface## Links - [Official Website](https://localai.io/) - [Documentation](https://localai.io/docs/) ---
GPU passthrough is offered during creation and driver libraries inside the container are matched to the host driver version automatically. Without a GPU everything still works on the CPU, just slowly.
Models, backends and the auth database live in /opt/localai_data and configuration in /opt/localai.env, both outside the application directory, so updates never touch them. A mount point on /opt/localai_data is worth considering rather than growing the root disk.
User accounts need a database. Upstream ships the release binary without the CGO auth tag, so the built-in SQLite store is unavailable and PostgreSQL is installed alongside LocalAI when the option is enabled. Leaving it off keeps the container to a single service.