# Open WebUI π
[](https://discord.gg/5rJgQTnV4s)
[](https://github.com/sponsors/tjbck)
**Open WebUI is an [extensible](https://docs.openwebui.com/features/plugin/), feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline.** It supports various LLM runners like **Ollama** and **OpenAI-compatible APIs**, with **built-in inference engine** for RAG, making it a **powerful AI deployment solution**.
Passionate about open-source AI? [Join our team β](https://careers.openwebui.com/)
> [!TIP]
> **Looking for an [Enterprise Plan](https://docs.openwebui.com/enterprise)?** β **[Speak with Our Sales Team Today!](mailto:sales@openwebui.com)**
>
> Get **enhanced capabilities**, including **custom theming and branding**, **Service Level Agreement (SLA) support**, **Long-Term Support (LTS) versions**, and **more!**
For more information, be sure to check out our [Open WebUI Documentation](https://docs.openwebui.com/).
## Key Features of Open WebUI β
| Feature | Description |
| - | - |
| π **Effortless Setup** | Install seamlessly using Docker or Kubernetes (kubectl, kustomize or helm) for a hassle-free experience with support for both `:ollama` and `:cuda` tagged images. |
| π€ **Ollama/OpenAI API Integration** | Effortlessly integrate OpenAI-compatible APIs for versatile conversations alongside Ollama models. Customize the OpenAI API URL to link with **LMStudio, GroqCloud, Mistral, OpenRouter, and more**. |
| π‘οΈ **Granular Permissions and User Groups** | By allowing administrators to create detailed user roles and permissions, we ensure a secure user environment. This granularity not only enhances security but also allows for customized user experiences, fostering a sense of ownership and responsibility amongst users. |
| π± **Responsive Design** | Enjoy a seamless experience across Desktop PC, Laptop, and Mobile devices. |
| π± **Progressive Web App (PWA) for Mobile** | Enjoy a native app-like experience on your mobile device with our PWA, providing offline access on localhost and a seamless user interface. |
| βοΈπ’ **Full Markdown and LaTeX Support** | Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction. |
| π€πΉ **Hands-Free Voice/Video Call** | Experience seamless communication with integrated hands-free voice and video call features, allowing for a more dynamic and interactive chat environment. |
| π οΈ **Model Builder** | Easily create Ollama models via the Web UI. Create and add custom characters/agents, customize chat elements, and import models effortlessly through [Open WebUI Community](https://openwebui.com/) integration. |
| π **Native Python Function Calling Tool** | Enhance your LLMs with built-in code editor support in the tools workspace. Bring Your Own Function (BYOF) by simply adding your pure Python functions, enabling seamless integration with LLMs. |
| π **Local RAG Integration** | Dive into the future of chat interactions with groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using the `#` command before a query. |
| π **Web Search for RAG** | Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, `serper`, `Serply`, `DuckDuckGo`, `TavilySearch`, `SearchApi` and `Bing` and inject the results directly into your chat experience. |
| π **Web Browsing Capability** | Seamlessly integrate websites into your chat experience using the `#` command followed by a URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions. |
| π¨ **Image Generation Integration** | Seamlessly incorporate image generation capabilities using options such as AUTOMATIC1111 API or ComfyUI (local), and OpenAI's DALL-E (external), enriching your chat experience with dynamic visual content. |
| βοΈ **Many Models Conversations** | Effortlessly engage with various models simultaneously, harnessing their unique strengths for optimal responses. Enhance your experience by leveraging a diverse set of models in parallel. |
| π **Role-Based Access Control (RBAC)** | Ensure secure access with restricted permissions; only authorized individuals can access your Ollama, and exclusive model creation/pulling rights are reserved for administrators. |
| ππ **Multilingual Support** | Experience Open WebUI in your preferred language with our internationalization (i18n) support. Join us in expanding our supported languages! We're actively seeking contributors! |
| 𧩠**Pipelines, Open WebUI Plugin Support** | Seamlessly integrate custom logic and Python libraries into Open WebUI using [Pipelines Plugin Framework](https://github.com/open-webui/pipelines). Launch your Pipelines instance, set the OpenAI URL to the Pipelines URL, and explore endless possibilities. [Examples](https://github.com/open-webui/pipelines/tree/main/examples) include **Function Calling**, User **Rate Limiting** to control access, **Usage Monitoring** with tools like Langfuse, **Live Translation with LibreTranslate** for multilingual support, **Toxic Message Filtering** and much more. |
| π **Continuous Updates** | We are committed to improving Open WebUI with regular updates, fixes, and new features. |
Want to learn more about Open WebUI's features? Check out our [Open WebUI documentation](https://docs.openwebui.com/features) for a comprehensive overview!
## Sponsors π
#### Emerald
| Logo | Description |
| - | - |
|  | [n8n](https://n8n.io/) β’ Does your interface have a backend yet? Try [n8n](https://n8n.io/) |
|  | [Tailscale](https://tailscale.com/blog/self-host-a-local-ai-stack/?utm_source=OpenWebUI&utm_medium=paid-ad-placement&utm_campaign=OpenWebUI-Docs) β’ Connect self-hosted AI to any device with Tailscale |
---
We are incredibly grateful for the generous support of our sponsors. Their contributions help us to maintain and improve our project, ensuring we can continue to deliver quality work to our community. Thank you!
## How to Install π
### Installation via Python pip π
Open WebUI can be installed using pip, the Python package installer. Before proceeding, ensure you're using **Python 3.11** to avoid compatibility issues.
1. **Install Open WebUI**:
Open your terminal and run the following command to install Open WebUI:
```bash
pip install open-webui
```
2. **Running Open WebUI**:
After installation, you can start Open WebUI by executing:
```bash
open-webui serve
```
This will start the Open WebUI server, which you can access at [http://localhost:8080](http://localhost:8080)
### Quick Start with Docker π³
> [!NOTE]
> Please note that for certain Docker environments, additional configurations might be needed. If you encounter any connection issues, our detailed guide on [Open WebUI Documentation](https://docs.openwebui.com/) is ready to assist you.
> [!WARNING]
> When using Docker to install Open WebUI, make sure to include the `-v open-webui:/app/backend/data` in your Docker command. This step is crucial as it ensures your database is properly mounted and prevents any loss of data.
> [!TIP]
> If you wish to utilize Open WebUI with Ollama included or CUDA acceleration, we recommend utilizing our official images tagged with either `:cuda` or `:ollama`. To enable CUDA, you must install the [Nvidia CUDA container toolkit](https://docs.nvidia.com/dgx/nvidia-container-runtime-upgrade/) on your Linux/WSL system.
### Installation with Default Configuration
| Command | Description |
| - | - |
| `docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main` | If Ollama is on your computer |
| `docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://example.com -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main` | If Ollama is on a Different Server |
| `docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda` | To run Open WebUI with Nvidia GPU support |
| `docker run -d -p 3000:8080 -e OPENAI_API_KEY=your_secret_key -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main` | If you're only using OpenAI API |
### Installation for OpenAI API Usage Only
| Command | Description |
| - | - |
| `docker run -d -p 3000:8080 -e OPENAI_API_KEY=your_secret_key -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main` | If you're only using OpenAI API |
### Installing Open WebUI with Bundled Ollama Support
This installation method uses a single container image that bundles Open WebUI with Ollama, allowing for a streamlined setup via a single command. Choose the appropriate command based on your hardware setup:
| Command | Description |
| - | - |
| `docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama` | With GPU Support |
| `docker run -d -p 3000:8080 -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama` | For CPU Only |
Both commands facilitate a built-in, hassle-free installation of both Open WebUI and Ollama, ensuring that you can get everything up and running swiftly.
After installation, you can access Open WebUI at [http://localhost:3000](http://localhost:3000). Enjoy! π
### Other Installation Methods
We offer various installation alternatives, including non-Docker native installation methods, Docker Compose, Kustomize, and Helm. Visit our [Open WebUI Documentation](https://docs.openwebui.com/getting-started/) or join our [Discord community](https://discord.gg/5rJgQTnV4s) for comprehensive guidance.
Look at the [Local Development Guide](https://docs.openwebui.com/getting-started/advanced-topics/development) for instructions on setting up a local development environment.
### Troubleshooting
Encountering connection issues? Our [Open WebUI Documentation](https://docs.openwebui.com/troubleshooting/) has got you covered. For further assistance and to join our vibrant community, visit the [Open WebUI Discord](https://discord.gg/5rJgQTnV4s).
#### Open WebUI: Server Connection Error
If you're experiencing connection issues, itβs often due to the WebUI docker container not being able to reach the Ollama server at 127.0.0.1:11434 (host.docker.internal:11434) inside the container . Use the `--network=host` flag in your docker command to resolve this. Note that the port changes from 3000 to 8080, resulting in the link: `http://localhost:8080`.
**Example Docker Command**:
```bash
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
### Keeping Your Docker Installation Up-to-Date
In case you want to update your local Docker installation to the latest version, you can do it with [Watchtower](https://containrrr.dev/watchtower/):
```bash
docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui
```
In the last part of the command, replace `open-webui` with your container name if it is different.
Check our Updating Guide available in our [Open WebUI Documentation](https://docs.openwebui.com/getting-started/updating).
### Using the Dev Branch π
> [!WARNING]
> The `:dev` branch contains the latest unstable features and changes. Use it at your own risk as it may have bugs or incomplete features.
If you want to try out the latest bleeding-edge features and are okay with occasional instability, you can use the `:dev` tag like this:
```bash
docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui --add-host=host.docker.internal:host-gateway --restart always ghcr.io/open-webui/open-webui:dev
```
### Offline Mode
If you are running Open WebUI in an offline environment, you can set the `HF_HUB_OFFLINE` environment variable to `1` to prevent attempts to download models from the internet.
```bash
export HF_HUB_OFFLINE=1
```
## What's Next? π
Discover upcoming features on our roadmap in the [Open WebUI Documentation](https://docs.openwebui.com/roadmap/).
## License π
This project is licensed under the [Open WebUI License](LICENSE), a revised BSD-3-Clause license. You receive all the same rights as the classic BSD-3 license: you can use, modify, and distribute the software, including in proprietary and commercial products, with minimal restrictions. The only additional requirement is to preserve the "Open WebUI" branding, as detailed in the LICENSE file. For full terms, see the [LICENSE](LICENSE) document. π
## Support π¬
If you have any questions, suggestions, or need assistance, please open an issue or join our
[Open WebUI Discord community](https://discord.gg/5rJgQTnV4s) to connect with us! π€
## Star History
---
Created by [Timothy Jaeryang Baek](https://github.com/tjbck) - Let's make Open WebUI even more amazing together! πͺ
# Ollama
Get up and running with large language models.
### macOS
[Download](https://ollama.com/download/Ollama.dmg)
### Windows
[Download](https://ollama.com/download/OllamaSetup.exe)
### Linux
```shell
curl -fsSL https://ollama.com/install.sh | sh
```
[Manual install instructions](https://github.com/ollama/ollama/blob/main/docs/linux.md)
### Docker
The official [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) `ollama/ollama` is available on Docker Hub.
### Libraries
| Library | Description |
| - | - |
| [ollama-python](https://github.com/ollama/ollama-python) | Python library for Ollama |
| [ollama-js](https://github.com/ollama/ollama-js) | JavaScript library for Ollama |
### Community
| Platform | Link |
| - | - |
| [Discord](https://discord.gg/ollama) | Discord community |
| [Reddit](https://reddit.com/r/ollama) | Reddit community |
## Quickstart
To run and chat with [Gemma 3](https://ollama.com/library/gemma3):
```shell
ollama run gemma3
```
## Model library
Ollama supports a list of models available on [ollama.com/library](https://ollama.com/library 'ollama model library')
Here are some example models that can be downloaded:
| Model | Parameters | Size | Download |
| - | - | - | - |
| Gemma 3 | 1B | 815MB | `ollama run gemma3:1b` |
| Gemma 3 | 4B | 3.3GB | `ollama run gemma3` |
| Gemma 3 | 12B | 8.1GB | `ollama run gemma3:12b` |
| Gemma 3 | 27B | 17GB | `ollama run gemma3:27b` |
| QwQ | 32B | 20GB | `ollama run qwq` |
| DeepSeek-R1 | 7B | 4.7GB | `ollama run deepseek-r1` |
| DeepSeek-R1 | 671B | 404GB | `ollama run deepseek-r1:671b` |
| Llama 4 | 109B | 67GB | `ollama run llama4:scout` |
| Llama 4 | 400B | 245GB | `ollama run llama4:maverick` |
| Llama 3.3 | 70B | 43GB | `ollama run llama3.3` |
| Llama 3.2 | 3B | 2.0GB | `ollama run llama3.2` |
| Llama 3.2 | 1B | 1.3GB | `ollama run llama3.2:1b` |
| Llama 3.2 Vision | 11B | 7.9GB | `ollama run llama3.2-vision` |
| Llama 3.2 Vision | 90B | 55GB | `ollama run llama3.2-vision:90b` |
| Llama 3.1 | 8B | 4.7GB | `ollama run llama3.1` |
| Llama 3.1 | 405B | 231GB | `ollama run llama3.1:405b` |
| Phi 4 | 14B | 9.1GB | `ollama run phi4` |
| Phi 4 Mini | 3.8B | 2.5GB | `ollama run phi4-mini` |
| Mistral | 7B | 4.1GB | `ollama run mistral` |
| Moondream 2 | 1.4B | 829MB | `ollama run moondream` |
| Neural Chat | 7B | 4.1GB | `ollama run neural-chat` |
| Starling | 7B | 4.1GB | `ollama run starling-lm` |
| Code Llama | 7B | 3.8GB | `ollama run codellama` |
| Llama 2 Uncensored | 7B | 3.8GB | `ollama run llama2-uncensored` |
| LLaVA | 7B | 4.5GB | `ollama run llava` |
| Granite-3.3 | 8B | 4.9GB | `ollama run granite3.3` |
> [!NOTE]
> You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
## Customize a model
### Import from GGUF
Ollama supports importing GGUF models in the Modelfile:
1. Create a file named `Modelfile`, with a `FROM` instruction with the local filepath to the model you want to import.
```
FROM ./vicuna-33b.Q4_0.gguf
```
2. Create the model in Ollama
```shell
ollama create example -f Modelfile
```
3. Run the model
```shell
ollama run example
```
### Import from Safetensors
See the [guide](docs/import.md) on importing models for more information.
### Customize a prompt
Models from the Ollama library can be customized with a prompt. For example, to customize the `llama3.2` model:
```shell
ollama pull llama3.2
```
Create a `Modelfile`:
```
FROM llama3.2
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system message
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
```
Next, create and run the model:
```
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
```
For more information on working with a Modelfile, see the [Modelfile](docs/modelfile.md) documentation.
## CLI Reference
### Create a model
`ollama create` is used to create a model from a Modelfile.
```shell
ollama create mymodel -f ./Modelfile
```
### Pull a model
```shell
ollama pull llama3.2
```
> This command can also be used to update a local model. Only the diff will be pulled.
### Remove a model
```shell
ollama rm llama3.2
```
### Copy a model
```shell
ollama cp llama3.2 my-model
```
### Multiline input
For multiline input, you can wrap text with `"""`:
```
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
```
### Multimodal models
```
ollama run llava "What's in this image? /Users/jmorgan/Desktop/smile.png"
```
> **Output**: The image features a yellow smiley face, which is likely the central focus of the picture.
### Pass the prompt as an argument
```shell
ollama run llama3.2 "Summarize this file: $(cat README.md)"
```
> **Output**: Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
### Show model information
```shell
ollama show llama3.2
```
### List models on your computer
```shell
ollama list
```
### List which models are currently loaded
```shell
ollama ps
```
### Stop a model which is currently running
```shell
ollama stop llama3.2
```
### Start Ollama
`ollama serve` is used when you want to start ollama without running the desktop application.
## REST API
Ollama has a REST API for running and managing models.
### Generate a response
```shell
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2",
"prompt":"Why is the sky blue?"
}'
```
### Chat with a model
```shell
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
```
See the [API documentation](./docs/api.md) for all endpoints.
## Community Integrations
### Web & Desktop
| Integration | Description |
| - | - |
| [Open WebUI](https://github.com/open-webui/open-webui) | Open WebUI integration |
| [SwiftChat (macOS with ReactNative)](https://github.com/aws-samples/swift-chat) | SwiftChat integration |
| [Enchanted (macOS native)](https://github.com/AugustDev/enchanted) | Enchanted integration |
| [Hollama](https://github.com/fmaclen/hollama) | Hollama integration |
| [Lollms-Webui](https://github.com/ParisNeo/lollms-webui) | Lollms-Webui integration |
| [LibreChat](https://github.com/danny-avila/LibreChat) | LibreChat integration |
| [Bionic GPT](https://github.com/bionic-gpt/bionic-gpt) | Bionic GPT integration |
| [HTML UI](https://github.com/rtcfirefly/ollama-ui) | HTML UI integration |
| [Saddle](https://github.com/jikkuatwork/saddle) | Saddle integration |
| [TagSpaces](https://www.tagspaces.org) | TagSpaces integration |
| [Chatbot UI](https://github.com/ivanfioravanti/chatbot-ollama) | Chatbot UI integration |
| [Chatbot UI v2](https://github.com/mckaywrigley/chatbot-ui) | Chatbot UI v2 integration |
| [Typescript UI](https://github.com/ollama-interface/Ollama-Gui?tab=readme-ov-file) | Typescript UI integration |
| [Minimalistic React UI for Ollama Models](https://github.com/richawo/minimal-llm-ui) | Minimalistic React UI integration |
| [Ollamac](https://github.com/kevinhermawan/Ollamac) | Ollamac integration |
| [big-AGI](https://github.com/enricoros/big-AGI) | big-AGI integration |
| [Cheshire Cat assistant framework](https://github.com/cheshire-cat-ai/core) | Cheshire Cat integration |
| [Amica](https://github.com/semperai/amica) | Amica integration |
| [chatd](https://github.com/BruceMacD/chatd) | chatd integration |
| [Ollama-SwiftUI](https://github.com/kghandour/Ollama-SwiftUI) | Ollama-SwiftUI integration |
| [Dify.AI](https://github.com/langgenius/dify) | Dify.AI integration |
| [MindMac](https://mindmac.app) | MindMac integration |
| [NextJS Web Interface for Ollama](https://github.com/jakobhoeg/nextjs-ollama-llm-ui) | NextJS Web Interface integration |
| [Msty](https://msty.app) | Msty integration |
| [Chatbox](https://github.com/Bin-Huang/Chatbox) | Chatbox integration |
| [WinForm Ollama Copilot](https://github.com/tgraupmann/WinForm_Ollama_Copilot) | WinForm Ollama Copilot integration |
| [NextChat](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web) | NextChat integration |
| [Alpaca WebUI](https://github.com/mmo80/alpaca-webui) | Alpaca WebUI integration |
| [OllamaGUI](https://github.com/enoch1118/ollamaGUI) | OllamaGUI integration |
| [OpenAOE](https://github.com/InternLM/OpenAOE) | OpenAOE integration |
| [Odin Runes](https://github.com/leonid20000/OdinRunes) | Odin Runes integration |
| [LLM-X](https://github.com/mrdjohnson/llm-x) | LLM-X integration |
| [AnythingLLM (Docker + MacOs/Windows/Linux native app)](https://github.com/Mintplex-Labs/anything-llm) | AnythingLLM integration |
| [Ollama Basic Chat: Uses HyperDiv Reactive UI](https://github.com/rapidarchitect/ollama_basic_chat) | Ollama Basic Chat integration |
| [Ollama-chats RPG](https://github.com/drazdra/ollama-chats) | Ollama-chats RPG integration |
| [IntelliBar](https://intellibar.app/) | IntelliBar integration |
| [Jirapt](https://github.com/AliAhmedNada/jirapt) | Jirapt integration |
| [ojira](https://github.com/AliAhmedNada/ojira) | ojira integration |
| [QA-Pilot](https://github.com/reid41/QA-Pilot) | QA-Pilot integration |
| [ChatOllama](https://github.com/sugarforever/chat-ollama) | ChatOllama integration |
| [CRAG Ollama Chat](https://github.com/Nagi-ovo/CRAG-Ollama-Chat) | CRAG Ollama Chat integration |
| [RAGFlow](https://github.com/infiniflow/ragflow) | RAGFlow integration |
| [StreamDeploy](https://github.com/StreamDeploy-DevRel/streamdeploy-llm-app-scaffold) | StreamDeploy integration |
| [chat](https://github.com/swuecho/chat) | chat integration |
| [Lobe Chat](https://github.com/lobehub/lobe-chat) | Lobe Chat integration |
| [Ollama RAG Chatbot](https://github.com/datvodinh/rag-chatbot.git) | Ollama RAG Chatbot integration |
| [BrainSoup](https://www.nurgo-software.com/products/brainsoup) | BrainSoup integration |
| [macai](https://github.com/Renset/macai) | macai integration |
| [RWKV-Runner](https://github.com/josStorer/RWKV-Runner) | RWKV-Runner integration |
| [Ollama Grid Search](https://github.com/dezoito/ollama-grid-search) | Ollama Grid Search integration |
| [Olpaka](https://github.com/Otacon/olpaka) | Olpaka integration |
| [Casibase](https://casibase.org) | Casibase integration |
| [OllamaSpring](https://github.com/CrazyNeil/OllamaSpring) | OllamaSpring integration |
| [LLocal.in](https://github.com/kartikm7/llocal) | LLocal.in integration |
| [Shinkai Desktop](https://github.com/dcSpark/shinkai-apps) | Shinkai Desktop integration |
| [AiLama](https://github.com/zeyoyt/ailama) | AiLama integration |
| [Ollama with Google Mesop](https://github.com/rapidarchitect/ollama_mesop/) | Ollama with Google Mesop integration |
| [R2R](https://github.com/SciPhi-AI/R2R) | R2R integration |
| [Ollama-Kis](https://github.com/elearningshow/ollama-kis) | Ollama-Kis integration |
| [OpenGPA](https://opengpa.org) | OpenGPA integration |
| [Painting Droid](https://github.com/mateuszmigas/painting-droid) | Painting Droid integration |
| [Kerlig AI](https://www.kerlig.com/) | Kerlig AI integration |
| [AI Studio](https://github.com/MindWorkAI/AI-Studio) | AI Studio integration |
| [Sidellama](https://github.com/gyopak/sidellama) | Sidellama integration |
| [LLMStack](https://github.com/trypromptly/LLMStack) | LLMStack integration |
| [BoltAI for Mac](https://boltai.com) | BoltAI for Mac integration |
| [Harbor](https://github.com/av/harbor) | Harbor integration |
| [PyGPT](https://github.com/szczyglis-dev/py-gpt) | PyGPT integration |
| [Alpaca](https://github.com/Jeffser/Alpaca) | Alpaca integration |
| [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT/blob/master/docs/content/platform/ollama.md) | AutoGPT integration |
| [Go-CREW](https://www.jonathanhecl.com/go-crew/) | Go-CREW integration |
| [PartCAD](https://github.com/openvmp/partcad/) | PartCAD integration |
| [Ollama4j Web UI](https://github.com/ollama4j/ollama4j-web-ui) | Ollama4j Web UI integration |
| [PyOllaMx](https://github.com/kspviswa/pyOllaMx) | PyOllaMx integration |
| [Cline](https://github.com/cline/cline) | Cline integration |
| [Cherry Studio](https://github.com/kangfenmao/cherry-studio) | Cherry Studio integration |
| [ConfiChat](https://github.com/1runeberg/confichat) | ConfiChat integration |
| [Archyve](https://github.com/nickthecook/archyve) | Archyve integration |
| [crewAI with Mesop](https://github.com/rapidarchitect/ollama-crew-mesop) | crewAI with Mesop integration |
| [Tkinter-based client](https://github.com/chyok/ollama-gui) | Tkinter-based client integration |
| [LLMChat](https://github.com/trendy-design/llmchat) | LLMChat integration |
| [Local Multimodal AI Chat](https://github.com/Leon-Sander/Local-Multimodal-AI-Chat) | Local Multimodal AI Chat integration |
| [ARGO](https://github.com/xark-argo/argo) | ARGO integration |
| [OrionChat](https://github.com/EliasPereirah/OrionChat) | OrionChat integration |
| [G1](https://github.com/bklieger-groq/g1) | G1 integration |
| [Web management](https://github.com/lemonit-eric-mao/ollama-web-management) | Web management integration |
| [Promptery](https://github.com/promptery/promptery) | Promptery integration |
| [Ollama App](https://github.com/JHubi1/ollama-app) | Ollama App integration |
| [chat-ollama](https://github.com/anan1213095357/ollama-chat-app) | chat-ollama integration |
| [SpaceLlama](https://github.com/tcsenpai/spacellama) | SpaceLlama integration |
| [YouLama](https://github.com/tcsenpai/youlama) | YouLama integration |
| [DualMind](https://github.com/tcsenpai/dualmind) | DualMind integration |
| [ollamarama-matrix](https://github.com/h1ddenpr0cess20/ollamarama-matrix) | ollamarama-matrix integration |
| [ollama-chat-app](https://github.com/anan1213095357/ollama-chat-app) | ollama-chat-app integration |
| [Perfect Memory AI](https://www.perfectmemory.ai/) | Perfect Memory AI integration |
| [Hexabot](https://github.com/hexastack/hexabot) | Hexabot integration |
| [Reddit Rate](https://github.com/rapidarchitect/reddit_analyzer) | Reddit Rate integration |
| [OpenTalkGpt](https://github.com/adarshM84/OpenTalkGpt) | OpenTalkGpt integration |
| [VT](https://github.com/vinhnx/vt.ai) | VT integration |
| [Nosia](https://github.com/nosia-ai/nosia) | Nosia integration |
| [Witsy](https://github.com/nbonamy/witsy) | Witsy integration |
| [Abbey](https://github.com/US-Artificial-Intelligence/abbey) | Abbey integration |
| [Minima](https://github.com/dmayboroda/minima) | Minima integration |
| [aidful-ollama-model-delete](https://github.com/AidfulAI/aidful-ollama-model-delete) | aidful-ollama-model-delete integration |
| [Perplexica](https://github.com/ItzCrazyKns/Perplexica) | Perplexica integration |
| [Ollama Chat WebUI for Docker](https://github.com/oslook/ollama-webui) | Ollama Chat WebUI for Docker integration |
| [AI Toolkit for Visual Studio Code](https://aka.ms/ai-tooklit/ollama-docs) | AI Toolkit for Visual Studio Code integration |
| [MinimalNextOllamaChat](https://github.com/anilkay/MinimalNextOllamaChat) | MinimalNextOllamaChat integration |
| [Chipper](https://github.com/TilmanGriesel/chipper) | Chipper integration |
| [ChibiChat](https://github.com/CosmicEventHorizon/ChibiChat) | ChibiChat integration |
| [LocalLLM](https://github.com/qusaismael/localllm) | LocalLLM integration |
| [Ollamazing](https://github.com/buiducnhat/ollamazing) | Ollamazing integration |
| [OpenDeepResearcher-via-searxng](https://github.com/benhaotang/OpenDeepResearcher-via-searxng) | OpenDeepResearcher-via-searxng integration |
| [AntSK](https://github.com/AIDotNet/AntSK) | AntSK integration |
| [MaxKB](https://github.com/1Panel-dev/MaxKB/) | MaxKB integration |
| [yla](https://github.com/danielekp/yla) | yla integration |
| [LangBot](https://github.com/RockChinQ/LangBot) | LangBot integration |
| [1Panel](https://github.com/1Panel-dev/1Panel/) | 1Panel integration |
| [AstrBot](https://github.com/Soulter/AstrBot/) | AstrBot integration |
| [Reins](https://github.com/ibrahimcetin/reins) | Reins integration |
| [Flufy](https://github.com/Aharon-Bensadoun/Flufy) | Flufy integration |
| [Ellama](https://github.com/zeozeozeo/ellama) | Ellama integration |
| [screenpipe](https://github.com/mediar-ai/screenpipe) | screenpipe integration |
| [Ollamb](https://github.com/hengkysteen/ollamb) | Ollamb integration |
| [Writeopia](https://github.com/Writeopia/Writeopia) | Writeopia integration |
| [AppFlowy](https://github.com/AppFlowy-IO/AppFlowy) | AppFlowy integration |
| [Lumina](https://github.com/cushydigit/lumina.git) | Lumina integration |
| [Tiny Notepad](https://pypi.org/project/tiny-notepad) | Tiny Notepad integration |
| [macLlama (macOS native)](https://github.com/hellotunamayo/macLlama) | macLlama integration |
| [GPTranslate](https://github.com/philberndt/GPTranslate) | GPTranslate integration |
| [ollama launcher](https://github.com/NGC13009/ollama-launcher) | ollama launcher integration |
| [ai-hub](https://github.com/Aj-Seven/ai-hub) | ai-hub integration |
| [Mayan EDMS](https://gitlab.com/mayan-edms/mayan-edms) | Mayan EDMS integration |
### Cloud
| Cloud | Link |
| - | - |
| [Google Cloud](https://cloud.google.com/run/docs/tutorials/gpu-gemma2-with-ollama) | Google Cloud integration |
| [Fly.io](https://fly.io/docs/python/do-more/add-ollama/) | Fly.io integration |
| [Koyeb](https://www.koyeb.com/deploy/ollama) | Koyeb integration |
### Terminal
| Terminal | Link |
| - | - |
| [oterm](https://github.com/ggozad/oterm) | oterm integration |
| [Ellama Emacs client](https://github.com/s-kostyaev/ellama) | Ellama Emacs client integration |
| [Emacs client](https://github.com/zweifisch/ollama) | Emacs client integration |
| [neollama](https://github.com/paradoxical-dev/neollama) | neollama integration |
| [gen.nvim](https://github.com/David-Kunz/gen.nvim) | gen.nvim integration |
| [ollama.nvim](https://github.com/nomnivore/ollama.nvim) | ollama.nvim integration |
| [ollero.nvim](https://github.com/marco-souza/ollero.nvim) | ollero.nvim integration |
| [ollama-chat.nvim](https://github.com/gerazov/ollama-chat.nvim) | ollama-chat.nvim integration |
| [ogpt.nvim](https://github.com/huynle/ogpt.nvim) | ogpt.nvim integration |
| [gptel Emacs client](https://github.com/karthink/gptel) | gptel Emacs client integration |
| [Oatmeal](https://github.com/dustinblackman/oatmeal) | Oatmeal integration |
| [cmdh](https://github.com/pgibler/cmdh) | cmdh integration |
| [ooo](https://github.com/npahlfer/ooo) | ooo integration |
| [shell-pilot](https://github.com/reid41/shell-pilot) | shell-pilot integration |
| [tenere](https://github.com/pythops/tenere) | tenere integration |
| [llm-ollama](https://github.com/taketwo/llm-ollama) | llm-ollama integration |
| [typechat-cli](https://github.com/anaisbetts/typechat-cli) | typechat-cli integration |
| [ShellOracle](https://github.com/djcopley/ShellOracle) | ShellOracle integration |
| [tlm](https://github.com/yusufcanb/tlm) | tlm integration |
| [podman-ollama](https://github.com/ericcurtin/podman-ollama) | podman-ollama integration |
| [gollama](https://github.com/sammcj/gollama) | gollama integration |
| [ParLlama](https://github.com/paulrobello/parllama) | ParLlama integration |
| [Ollama eBook Summary](https://github.com/cognitivetech/ollama-ebook-summary/) | Ollama eBook Summary integration |
| [Ollama Mixture of Experts (MOE) in 50 lines of code](https://github.com/rapidarchitect/ollama_moe) | Ollama Mixture of Experts integration |
| [vim-intelligence-bridge](https://github.com/pepo-ec/vim-intelligence-bridge) | vim-intelligence-bridge integration |
| [x-cmd ollama](https://x-cmd.com/mod/ollama) | x-cmd ollama integration |
| [bb7](https://github.com/drunkwcodes/bb7) | bb7 integration |
| [SwollamaCLI](https://github.com/marcusziade/Swollama) | SwollamaCLI integration |
| [aichat](https://github.com/sigoden/aichat) | aichat integration |
| [PowershAI](https://github.com/rrg92/powershai) | PowershAI integration |
| [DeepShell](https://github.com/Abyss-c0re/deepshell) | DeepShell integration |
| [orbiton](https://github.com/xyproto/orbiton) | orbiton integration |
| [orca-cli](https://github.com/molbal/orca-cli) | orca-cli integration |
| [GGUF-to-Ollama](https://github.com/jonathanhecl/gguf-to-ollama) | GGUF-to-Ollama integration |
| [AWS-Strands-With-Ollama](https://github.com/rapidarchitect/ollama_strands) | AWS-Strands-With-Ollama integration |
| [ollama-multirun](https://github.com/attogram/ollama-multirun) | ollama-multirun integration |
| [ollama-bash-toolshed](https://github.com/attogram/ollama-bash-toolshed) | ollama-bash-toolshed integration |
### Apple Vision Pro
| Integration | Link |
| - | - |
| [SwiftChat](https://github.com/aws-samples/swift-chat) | SwiftChat integration |
| [Enchanted](https://github.com/AugustDev/enchanted) | Enchanted integration |
### Database
| Integration | Link |
| - | - |
| [pgai](https://github.com/timescale/pgai) | pgai integration |
| [MindsDB](https://github.com/mindsdb/mindsdb/blob/staging/mindsdb/integrations/handlers/ollama_handler/README.md) | MindsDB integration |
| [chromem-go](https://github.com/philippgille/chromem-go/blob/v0.5.0/embed_ollama.go) | chromem-go integration |
| [Kangaroo](https://github.com/dbkangaroo/kangaroo) | Kangaroo integration |
### Package managers
| Package Manager | Link |
| - | - |
| [Pacman](https://archlinux.org/packages/extra/x86_64/ollama/) | Pacman integration |
| [Gentoo](https://github.com/gentoo/guru/tree/master/app-misc/ollama) | Gentoo integration |
| [Homebrew](https://formulae.brew.sh/formula/ollama) | Homebrew integration |
| [Helm Chart](https://artifacthub.io/packages/helm/ollama-helm/ollama) | Helm Chart integration |
| [Guix channel](https://codeberg.org/tusharhero/ollama-guix) | Guix channel integration |
| [Nix package](https://search.nixos.org/packages?show=ollama&from=0&size=50&sort=relevance&type=packages&query=ollama) | Nix package integration |
| [Flox](https://flox.dev/blog/ollama-part-one) | Flox integration |
### Libraries
| Library | Link |
| - | - |
| [LangChain](https://python.langchain.com/docs/integrations/chat/ollama/) | LangChain integration |
| [LangChain.js](https://js.langchain.com/docs/integrations/chat/ollama/) | LangChain.js integration |
| [Firebase Genkit](https://firebase.google.com/docs/genkit/plugins/ollama) | Firebase Genkit integration |
| [crewAI](https://github.com/crewAIInc/crewAI) | crewAI integration |
| [Yacana](https://remembersoftwares.github.io/yacana/) | Yacana integration |
| [Spring AI](https://github.com/spring-projects/spring-ai) | Spring AI integration |
| [LangChainGo](https://github.com/tmc/langchaingo/) | LangChainGo integration |
| [LangChain4j](https://github.com/langchain4j/langchain4j) | LangChain4j integration |
| [LangChainRust](https://github.com/Abraxas-365/langchain-rust) | LangChainRust integration |
| [LangChain for .NET](https://github.com/tryAGI/LangChain) | LangChain for .NET integration |
| [LLPhant](https://github.com/theodo-group/LLPhant?tab=readme-ov-file#ollama) | LLPhant integration |
| [LlamaIndex](https://docs.llamaindex.ai/en/stable/examples/llm/ollama/) | LlamaIndex integration |
| [LlamaIndexTS](https://ts.llamaindex.ai/modules/llms/available_llms/ollama) | LlamaIndexTS integration |
| [LiteLLM](https://github.com/BerriAI/litellm) | LiteLLM integration |
| [OllamaFarm for Go](https://github.com/presbrey/ollamafarm) | OllamaFarm for Go integration |
| [OllamaSharp for .NET](https://github.com/awaescher/OllamaSharp) | OllamaSharp for .NET integration |
| [Ollama for Ruby](https://github.com/gbaptista/ollama-ai) | Ollama for Ruby integration |
| [Ollama-rs for Rust](https://github.com/pepperoni21/ollama-rs) | Ollama-rs for Rust integration |
| [Ollama-hpp for C++](https://github.com/jmont-dev/ollama-hpp) | Ollama-hpp for C++ integration |
| [Ollama4j for Java](https://github.com/ollama4j/ollama4j) | Ollama4j for Java integration |
| [ModelFusion Typescript Library](https://modelfusion.dev/integration/model-provider/ollama) | ModelFusion Typescript Library integration |
| [OllamaKit for Swift](https://github.com/kevinhermawan/OllamaKit) | OllamaKit for Swift integration |
| [Ollama for Dart](https://github.com/breitburg/dart-ollama) | Ollama for Dart integration |
| [Ollama for Laravel](https://github.com/cloudstudio/ollama-laravel) | Ollama for Laravel integration |
| [LangChainDart](https://github.com/davidmigloz/langchain_dart) | LangChainDart integration |
| [Semantic Kernel - Python](https://github.com/microsoft/semantic-kernel/tree/main/python/semantic_kernel/connectors/ai/ollama) | Semantic Kernel - Python integration |
| [Haystack](https://github.com/deepset-ai/haystack-integrations/blob/main/integrations/ollama.md) | Haystack integration |
| [Elixir LangChain](https://github.com/brainlid/langchain) | Elixir LangChain integration |
| [Ollama for R - rollama](https://github.com/JBGruber/rollama) | Ollama for R - rollama integration |
| [Ollama for R - ollama-r](https://github.com/hauselin/ollama-r) | Ollama for R - ollama-r integration |
| [Ollama-ex for Elixir](https://github.com/lebrunel/ollama-ex) | Ollama-ex for Elixir integration |
| [Ollama Connector for SAP ABAP](https://github.com/b-tocs/abap_btocs_ollama) | Ollama Connector for SAP ABAP integration |
| [Testcontainers](https://testcontainers.com/modules/ollama/) | Testcontainers integration |
| [Portkey](https://portkey.ai/docs/welcome/integration-guides/ollama) | Portkey integration |
| [PromptingTools.jl](https://github.com/svilupp/PromptingTools.jl) | PromptingTools.jl integration |
| [LlamaScript](https://github.com/Project-Llama/llamascript) | LlamaScript integration |
| [llm-axe](https://github.com/emirsahin1/llm-axe) | llm-axe integration |
| [Gollm](https://docs.gollm.co/examples/ollama-example) | Gollm integration |
| [Gollama for Golang](https://github.com/jonathanhecl/gollama) | Gollama for Golang integration |
| [Ollamaclient for Golang](https://github.com/xyproto/ollamaclient) | Ollamaclient for Golang integration |
| [High-level function abstraction in Go](https://gitlab.com/tozd/go/fun) | High-level function abstraction in Go integration |
| [Ollama PHP](https://github.com/ArdaGnsrn/ollama-php) | Ollama PHP integration |
| [Agents-Flex for Java](https://github.com/agents-flex/agents-flex) | Agents-Flex for Java integration |
| [Parakeet](https://github.com/parakeet-nest/parakeet) | Parakeet integration |
| [Haverscript](https://github.com/andygill/haverscript) | Haverscript integration |
| [Ollama for Swift](https://github.com/mattt/ollama-swift) | Ollama for Swift integration |
| [Swollama for Swift](https://github.com/marcusziade/Swollama) | Swollama for Swift integration |
| [GoLamify](https://github.com/prasad89/golamify) | GoLamify integration |
| [Ollama for Haskell](https://github.com/tusharad/ollama-haskell) | Ollama for Haskell integration |
| [multi-llm-ts](https://github.com/nbonamy/multi-llm-ts) | multi-llm-ts integration |
| [LlmTornado](https://github.com/lofcz/llmtornado) | LlmTornado integration |
| [Ollama for Zig](https://github.com/dravenk/ollama-zig) | Ollama for Zig integration |
| [Abso](https://github.com/lunary-ai/abso) | Abso integration |
| [Nichey](https://github.com/goodreasonai/nichey) | Nichey integration |
| [Ollama for D](https://github.com/kassane/ollama-d) | Ollama for D integration |
| [OllamaPlusPlus](https://github.com/HardCodeDev777/OllamaPlusPlus) | OllamaPlusPlus integration |
### Mobile
| Integration | Link |
| - | - |
| [SwiftChat](https://github.com/aws-samples/swift-chat) | SwiftChat integration |
| [Enchanted](https://github.com/AugustDev/enchanted) | Enchanted integration |
| [Maid](https://github.com/Mobile-Artificial-Intelligence/maid) | Maid integration |
| [Ollama App](https://github.com/JHubi1/ollama-app) | Ollama App integration |
| [ConfiChat](https://github.com/1runeberg/confichat) | ConfiChat integration |
| [Ollama Android Chat](https://github.com/sunshine0523/OllamaServer) | Ollama Android Chat integration |
| [Reins](https://github.com/ibrahimcetin/reins) | Reins integration |
### Extensions & Plugins
| Integration | Link |
| - | - |
| [Raycast extension](https://github.com/MassimilianoPasquini97/raycast_ollama) | Raycast extension integration |
| [Discollama](https://github.com/mxyng/discollama) | Discollama integration |
| [Continue](https://github.com/continuedev/continue) | Continue integration |
| [Vibe](https://github.com/thewh1teagle/vibe) | Vibe integration |
| [Obsidian Ollama plugin](https://github.com/hinterdupfinger/obsidian-ollama) | Obsidian Ollama plugin integration |
| [Logseq Ollama plugin](https://github.com/omagdy7/ollama-logseq) | Logseq Ollama plugin integration |
| [NotesOllama](https://github.com/andersrex/notesollama) | NotesOllama integration |
| [Dagger Chatbot](https://github.com/samalba/dagger-chatbot) | Dagger Chatbot integration |
| [Discord AI Bot](https://github.com/mekb-turtle/discord-ai-bot) | Discord AI Bot integration |
| [Ollama Telegram Bot](https://github.com/ruecat/ollama-telegram) | Ollama Telegram Bot integration |
| [Hass Ollama Conversation](https://github.com/ej52/hass-ollama-conversation) | Hass Ollama Conversation integration |
| [Rivet plugin](https://github.com/abrenneke/rivet-plugin-ollama) | Rivet plugin integration |
| [Obsidian BMO Chatbot plugin](https://github.com/longy2k/obsidian-bmo-chatbot) | Obsidian BMO Chatbot plugin integration |
| [Cliobot](https://github.com/herval/cliobot) | Cliobot integration |
| [Copilot for Obsidian plugin](https://github.com/logancyang/obsidian-copilot) | Copilot for Obsidian plugin integration |
| [Obsidian Local GPT plugin](https://github.com/pfrankov/obsidian-local-gpt) | Obsidian Local GPT plugin integration |
| [Open Interpreter](https://docs.openinterpreter.com/language-model-setup/local-models/ollama) | Open Interpreter integration |
| [Llama Coder](https://github.com/ex3ndr/llama-coder) | Llama Coder integration |
| [Ollama Copilot](https://github.com/bernardo-bruning/ollama-copilot) | Ollama Copilot integration |
| [twinny](https://github.com/rjmacarthy/twinny) | twinny integration |
| [Wingman-AI](https://github.com/RussellCanfield/wingman-ai) | Wingman-AI integration |
| [Page Assist](https://github.com/n4ze3m/page-assist) | Page Assist integration |
| [Plasmoid Ollama Control](https://github.com/imoize/plasmoid-ollamacontrol) | Plasmoid Ollama Control integration |
| [AI Telegram Bot](https://github.com/tusharhero/aitelegrambot) | AI Telegram Bot integration |
| [AI ST Completion](https://github.com/yaroslavyaroslav/OpenAI-sublime-text) | AI ST Completion integration |
| [Discord-Ollama Chat Bot](https://github.com/kevinthedang/discord-ollama) | Discord-Ollama Chat Bot integration |
| [ChatGPTBox: All in one browser extension](https://github.com/josStorer/chatGPTBox) | ChatGPTBox integration |
| [Discord AI chat/moderation bot](https://github.com/rapmd73/Companion) | Discord AI chat/moderation bot integration |
| [Headless Ollama](https://github.com/nischalj10/headless-ollama) | Headless Ollama integration |
| [Terraform AWS Ollama & Open WebUI](https://github.com/xuyangbocn/terraform-aws-self-host-llm) | Terraform AWS Ollama & Open WebUI integration |
| [node-red-contrib-ollama](https://github.com/jakubburkiewicz/node-red-contrib-ollama) | node-red-contrib-ollama integration |
| [Local AI Helper](https://github.com/ivostoykov/localAI) | Local AI Helper integration |
| [vnc-lm](https://github.com/jake83741/vnc-lm) | vnc-lm integration |
| [LSP-AI](https://github.com/SilasMarvin/lsp-ai) | LSP-AI integration |
| [QodeAssist](https://github.com/Palm1r/QodeAssist) | QodeAssist integration |
| [Obsidian Quiz Generator plugin](https://github.com/ECuiDev/obsidian-quiz-generator) | Obsidian Quiz Generator plugin integration |
| [AI Summmary Helper plugin](https://github.com/philffm/ai-summary-helper) | AI Summmary Helper plugin integration |
| [TextCraft](https://github.com/suncloudsmoon/TextCraft) | TextCraft integration |
| [Alfred Ollama](https://github.com/zeitlings/alfred-ollama) | Alfred Ollama integration |
| [TextLLaMA](https://github.com/adarshM84/TextLLaMA) | TextLLaMA integration |
| [Simple-Discord-AI](https://github.com/zyphixor/simple-discord-ai) | Simple-Discord-AI integration |
| [LLM Telegram Bot](https://github.com/innightwolfsleep/llm_telegram_bot) | LLM Telegram Bot integration |
| [mcp-llm](https://github.com/sammcj/mcp-llm) | mcp-llm integration |
| [SimpleOllamaUnity](https://github.com/HardCodeDev777/SimpleOllamaUnity) | SimpleOllamaUnity integration |
| [UnityCodeLama](https://github.com/HardCodeDev777/UnityCodeLama) | UnityCodeLama integration |
| [NativeMind](https://github.com/NativeMindBrowser/NativeMindExtension) | NativeMind integration |
| [GMAI - Gradle Managed AI](https://gmai.premex.se/) | GMAI integration |
### Supported backends
| Backend | Link |
| - | - |
| [llama.cpp](https://github.com/ggml-org/llama.cpp) | llama.cpp integration |
### Observability
| Tool | Link |
| - | - |
| [Opik](https://www.comet.com/docs/opik/cookbook/ollama) | Opik integration |
| [Lunary](https://lunary.ai/docs/integrations/ollama) | Lunary integration |
| [OpenLIT](https://github.com/openlit/openlit) | OpenLIT integration |
| [HoneyHive](https://docs.honeyhive.ai/integrations/ollama) | HoneyHive integration |
| [Langfuse](https://langfuse.com/docs/integrations/ollama) | Langfuse integration |
| [MLflow Tracing](https://mlflow.org/docs/latest/llms/tracing/index.html#automatic-tracing) | MLflow Tracing integration |
# OpenedAI Speech
Notice: This software is mostly obsolete and will no longer be updated.
Some Alternatives:
* https://speaches.ai/
* https://github.com/remsky/Kokoro-FastAPI
* https://github.com/astramind-ai/Auralis
* https://lightning.ai/docs/litserve/home?code_sample=speech
An OpenAI API compatible text to speech server.
* Compatible with the OpenAI audio/speech API
* Serves the [/v1/audio/speech endpoint](https://platform.openai.com/docs/api-reference/audio/createSpeech)
* Not affiliated with OpenAI in any way, does not require an OpenAI API Key
* A free, private, text-to-speech server with custom voice cloning
Full Compatibility:
| Feature | Description |
| - | - |
| `tts-1` | `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable) |
| `tts-1-hd` | `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default) |
| `response_format` | `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm` |
| `speed` | 0.25-4.0 (and more) |
Details:
| Detail | Description |
| - | - |
| Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) | Very fast, runs on CPU |
| Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning | Fast, but requires around 4GB GPU VRAM |
| Custom cloned voices | Can be used for tts-1-hd |
| π Multilingual support | With XTTS voices, the language is automatically detected if not set |
| Custom fine-tuned XTTS model support | See: [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support) |
| Configurable generation parameters | See: [Generation parameters](#generation-parameters) |
| Streamed output | While generating |
| Occasionally, certain words or symbols may sound incorrect | Can be fixed with regex via `pre_process_map.yaml` |
| Tested with python | 3.9-3.11, piper does not install on python 3.12 yet |
If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults.
## Recent Changes
| Version | Date | Changes |
| - | - | - |
| 0.18.2 | 2024-08-16 | Fix docker building for amd64, refactor github actions again, free up more disk space |
| 0.18.1 | 2024-08-15 | Refactor github actions |
| 0.18.0 | 2024-08-15 | Allow folders of wav samples in xtts. Samples will be combined, allowing for mixed voices and collections of small samples. Still limited to 30 seconds total. Fix missing yaml requirement in -min image. Fix fr_FR-tom-medium and other 44khz piper voices (detect non-default sample rates). Minor updates |
| 0.17.2 | 2024-07-01 | Fix -min image (re: langdetect) |
| 0.17.1 | 2024-07-01 | Fix ROCm (add langdetect to requirements-rocm.txt). Fix zh-cn for xtts |
| 0.17.0 | 2024-07-01 | Automatic language detection |
| 0.16.0 | 2024-06-29 | Multi-client safe version. Audio generation is synchronized in a single process. The estimated 'realtime' factor of XTTS on a GPU is roughly 1/3, this means that multiple streams simultaneously, or `speed` over 2, may experience audio underrun (delays or pauses in playback). This makes multiple clients possible and safe, but in practice 2 or 3 simultaneous streams is the maximum without audio underrun |
| 0.15.1 | 2024-06-27 | Remove deepspeed from requirements.txt, it's too complex for typical users. A more detailed deepspeed install document will be required |
| 0.15.0 | 2024-06-26 | Switch to [coqui-tts](https://github.com/idiap/coqui-ai-TTS) (updated fork), updated simpler dependencies, torch 2.3, etc. Resolve cuda threading issues |
| 0.14.1 | 2024-06-26 | Make deepspeed possible (`--use-deepspeed`), but not enabled in pre-built docker images (too large). Requires the cuda-toolkit installed, see the Dockerfile comment for details |
| 0.14.0 | 2024-06-26 | Added `response_format`: `wav` and `pcm` support. Output streaming (while generating) for `tts-1` and `tts-1-hd`. Enhanced [generation parameters](#generation-parameters) for xtts models (temperature, top_p, etc.). Idle unload timer (optional) - doesn't work perfectly yet. Improved error handling |
| 0.13.0 | 2024-06-25 | Added [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support). Initial prebuilt arm64 image support (Apple M-series, Raspberry Pi - MPS is not supported in XTTS/torch). Initial attempt at AMD GPU (ROCm 5.7) support. Parler-tts support removed. Move the *.default.yaml to the root folder. Run the docker as a service by default (`restart: unless-stopped`). Added `audio_reader.py` for streaming text input and reading long texts |
| 0.12.3 | 2024-06-17 | Additional logging details for BadRequests (400) |
| 0.12.2 | 2024-06-16 | Fix :min image requirements (numpy<2?) |
| 0.12.0 | 2024-06-16 | Improved error handling and logging. Restore the original alloy tts-1-hd voice by default, use alloy-alt for the old voice |
| 0.11.0 | 2024-05-29 | π [Multilingual](#multilingual) support (16 languages) with XTTS. Remove high Unicode filtering from the default `config/pre_process_map.yaml`. Update Docker build & app startup. Fix: "Plan failed with a cudnnException". Remove piper cuda support |
| 0.10.1 | 2024-05-05 | Remove `runtime: nvidia` from docker-compose.yml, this assumes nvidia/cuda compatible runtime is available by default |
| 0.10.0 | 2024-04-27 | Pre-built & tested docker images, smaller docker images (8GB or 860MB). Better upgrades: reorganize config files under `config/`, voice models under `voices/`. Default listen host to 0.0.0.0 |
| 0.9.0 | 2024-04-23 | Fix bug with yaml and loading UTF-8. New sample text-to-speech application `say.py`. Smaller docker base image. Add beta [parler-tts](https://huggingface.co/parler-tts/parler_tts_mini_v0.1) support (you can describe very basic features of the speaker voice) |
| 0.7.3 | 2024-03-20 | Allow different xtts versions per voice in `voice_to_speaker.yaml`, ex. xtts_v2.0.2. Quality: Fix xtts sample rate (24000 vs. 22050 for piper) and pops |
## Installation instructions
### Create a `speech.env` environment file
Copy the `sample.env` to `speech.env` (customize if needed)
```bash
cp sample.env speech.env
```
#### Defaults
```bash
TTS_HOME=voices
HF_HOME=voices
#PRELOAD_MODEL=xtts
#PRELOAD_MODEL=xtts_v2.0.2
#EXTRA_ARGS=--log-level DEBUG --unload-timer 300
#USE_ROCM=1
```
### Option A: Manual installation
```shell
# install curl and ffmpeg
sudo apt install curl ffmpeg
# Create & activate a new virtual environment (optional but recommended)
python -m venv .venv
source .venv/bin/activate
# Install the Python requirements
# - use requirements-rocm.txt for AMD GPU (ROCm support)
# - use requirements-min.txt for piper only (CPU only)
pip install -U -r requirements.txt
# run the server
bash startup.sh
```
> On first run, the voice models will be downloaded automatically. This might take a while depending on your network connection.
### Option B: Docker Image (*recommended*)
#### Nvidia GPU (cuda)
```shell
docker compose up
```
#### AMD GPU (ROCm support)
```shell
docker compose -f docker-compose.rocm.yml up
```
#### ARM64 (Apple M-series, Raspberry Pi)
> XTTS only has CPU support here and will be very slow, you can use the Nvidia image for XTTS with CPU (slow), or use the piper only image (recommended)
#### CPU only, No GPU (piper only)
> For a minimal docker image with only piper support (<1GB vs. 8GB).
```shell
docker compose -f docker-compose.min.yml up
```
## Server Options
```shell
usage: speech.py [-h] [--xtts_device XTTS_DEVICE] [--preload PRELOAD] [--unload-timer UNLOAD_TIMER] [--use-deepspeed] [--no-cache-speaker] [-P PORT] [-H HOST]
[-L {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
OpenedAI Speech API Server
options:
-h, --help show this help message and exit
--xtts_device XTTS_DEVICE
Set the device for the xtts model. The special value of 'none' will use piper for all models. (default: cuda)
--preload PRELOAD Preload a model (Ex. 'xtts' or 'xtts_v2.0.2'). By default it's loaded on first use. (default: None)
--unload-timer UNLOAD_TIMER
Idle unload timer for the XTTS model in seconds, Ex. 900 for 15 minutes (default: None)
--use-deepspeed Use deepspeed with xtts (this option is unsupported) (default: False)
--no-cache-speaker Don't use the speaker wav embeddings cache (default: False)
-P PORT, --port PORT Server tcp port (default: 8000)
-H HOST, --host HOST Host to listen on, Ex. 0.0.0.0 (default: 0.0.0.0)
-L {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Set the log level (default: INFO)
```
## Sample Usage
You can use it like this:
```shell
curl http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" -d '{
"model": "tts-1",
"input": "The quick brown fox jumped over the lazy dog.",
"voice": "alloy",
"response_format": "mp3",
"speed": 1.0
}' > speech.mp3
```
Or just like this:
```shell
curl -s http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" -d '{
"input": "The quick brown fox jumped over the lazy dog."}' > speech.mp3
```
Or like this example from the [OpenAI Text to speech guide](https://platform.openai.com/docs/guides/text-to-speech):
```python
import openai
client = openai.OpenAI(
# This part is not needed if you set these environment variables before import openai
# export OPENAI_API_KEY=sk-11111111111
# export OPENAI_BASE_URL=http://localhost:8000/v1
api_key = "sk-111111111",
base_url = "http://localhost:8000/v1",
)
with client.audio.speech.with_streaming_response.create(
model="tts-1",
voice="alloy",
input="Today is a wonderful day to build something people love!"
) as response:
response.stream_to_file("speech.mp3")
```
Also see the `say.py` sample application for an example of how to use the openai-python API.
```shell
# play the audio, requires 'pip install playsound'
python say.py -t "The quick brown fox jumped over the lazy dog." -p
# save to a file in flac format
python say.py -t "The quick brown fox jumped over the lazy dog." -m tts-1-hd -v onyx -f flac -o fox.flac
```
You can also try the included `audio_reader.py` for listening to longer text and streamed input.
Example usage:
```bash
python audio_reader.py -s 2 < LICENSE # read the software license - fast
```
## OpenAI API Documentation and Guide
| Documentation | Link |
| - | - |
| [OpenAI Text to speech guide](https://platform.openai.com/docs/guides/text-to-speech) | OpenAI Text to speech guide |
| [OpenAI API Reference](https://platform.openai.com/docs/api-reference/audio/createSpeech) | OpenAI API Reference |
## Custom Voices Howto
### Piper
1. Select the piper voice and model from the [piper samples](https://rhasspy.github.io/piper-samples/)
2. Update the `config/voice_to_speaker.yaml` with a new section for the voice, for example:
```yaml
...
tts-1:
ryan:
model: voices/en_US-ryan-high.onnx
speaker: # default speaker
```
3. New models will be downloaded as needed, of you can download them in advance with `download_voices_tts-1.sh`. For example:
```shell
bash download_voices_tts-1.sh en_US-ryan-high
```
### Coqui XTTS v2
Coqui XTTS v2 voice cloning can work with as little as 6 seconds of clear audio. To create a custom voice clone, you must prepare a WAV file sample of the voice.
#### Guidelines for preparing good sample files for Coqui XTTS v2
| Guideline | Description |
| - | - |
| Mono (single channel) 22050 Hz WAV file | |
| 6-30 seconds long | Longer isn't always better (I've had some good results with as little as 4 seconds) |
| Low noise | No hiss or hum |
| No partial words, breathing, laughing, music or backgrounds sounds | |
| An even speaking pace with a variety of words is best | Like in interviews or audiobooks |
| Audio longer than 30 seconds will be silently truncated | |
You can use FFmpeg to prepare your audio files, here are some examples:
```shell
# convert a multi-channel audio file to mono, set sample rate to 22050 hz, trim to 6 seconds, and output as WAV file.
ffmpeg -i input.mp3 -ac 1 -ar 22050 -t 6 -y me.wav
# use a simple noise filter to clean up audio, and select a start time start for sampling.
ffmpeg -i input.wav -af "highpass=f=200, lowpass=f=3000" -ac 1 -ar 22050 -ss 00:13:26.2 -t 6 -y me.wav
# A more complex noise reduction setup, including volume adjustment
ffmpeg -i input.mkv -af "highpass=f=200, lowpass=f=3000, volume=5, afftdn=nf=25" -ac 1 -ar 22050 -ss 00:13:26.2 -t 6 -y me.wav
```
Once your WAV file is prepared, save it in the `/voices/` directory and update the `config/voice_to_speaker.yaml` file with the new file name.
For example:
```yaml
...
tts-1-hd:
me:
model: xtts
speaker: voices/me.wav # this could be you
```
You can also use a sub folder for multiple audio samples to combine small samples or to mix different samples together.
For example:
```yaml
...
tts-1-hd:
mixed:
model: xtts
speaker: voices/mixed
```
Where the `voices/mixed/` folder contains multiple wav files. The total audio length is still limited to 30 seconds.
## Multilingual
Multilingual cloning support was added in version 0.11.0 and is available only with the XTTS v2 model. To use multilingual voices with piper simply download a language specific voice.
Coqui XTTSv2 has support for multiple languages: English (`en`), Spanish (`es`), French (`fr`), German (`de`), Italian (`it`), Portuguese (`pt`), Polish (`pl`), Turkish (`tr`), Russian (`ru`), Dutch (`nl`), Czech (`cs`), Arabic (`ar`), Chinese (`zh-cn`), Hungarian (`hu`), Korean (`ko`), Japanese (`ja`), and Hindi (`hi`). When not set, an attempt will be made to automatically detect the language, falling back to English (`en`).
Unfortunately the OpenAI API does not support language, but you can create your own custom speaker voice and set the language for that.
1. Create the WAV file for your speaker, as in [Custom Voices Howto](#custom-voices-howto)
2. Add the voice to `config/voice_to_speaker.yaml` and include the correct Coqui `language` code for the speaker. For example:
```yaml
xunjiang:
model: xtts
speaker: voices/xunjiang.wav
language: zh-cn
```
3. Don't remove high unicode characters in your `config/pre_process_map.yaml`! If you have these lines, you will need to remove them. For example:
Remove:
```yaml
- - '[\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF\U0001F700-\U0001F77F\U0001F780-\U0001F7FF\U0001F800-\U0001F8FF\U0001F900-\U0001F9FF\U0001FA00-\U0001FA6F\U0001FA70-\U0001FAFF\U00002702-\U000027B0\U000024C2-\U0001F251]+'
- ''
```
These lines were added to the `config/pre_process_map.yaml` config file by default before version 0.11.0:
4. Your new multi-lingual speaker voice is ready to use!
## Custom Fine-Tuned Model Support
Adding a custom xtts model is simple. Here is an example of how to add a custom fine-tuned 'halo' XTTS model.
1. Save the model folder under `voices/` (all 4 files are required, including the vocab.json from the model)
```bash
openedai-speech$ ls voices/halo/
config.json vocab.json model.pth sample.wav
```
2. Add the custom voice entry under the `tts-1-hd` section of `config/voice_to_speaker.yaml`:
```yaml
tts-1-hd:
...
halo:
model: halo # This name is required to be unique
speaker: voices/halo/sample.wav # voice sample is required
model_path: voices/halo
```
3. The model will be loaded when you access the voice for the first time (`--preload` doesn't work with custom models yet)
## Generation Parameters
The generation of XTTSv2 voices can be fine tuned with the following options (defaults included below):
```yaml
tts-1-hd:
alloy:
model: xtts
speaker: voices/alloy.wav
enable_text_splitting: True
length_penalty: 1.0
repetition_penalty: 10
speed: 1.0
temperature: 0.75
top_k: 50
top_p: 0.85
```