OpenWebUI notes ...

Open WebUI installer: https://github.com/freeload101/SCRIPTS/blob/master/Bash/OpenWebUI_Fast.bash

Older Professor synapse prompt you can use: https://raw.githubusercontent.com/freeload101/SCRIPTS/refs/heads/master/Prof%20Synapse%20Old.txt

Fabric prompts you can import into openwebui !!! ( https://github.com/danielmiessler/fabric/tree/main/patterns ) https://github.com/freeload101/SCRIPTS/blob/master/MISC/Fabric_Prompts_Open_WebUI_OpenWebUI_20241112.json

Example AT windows task startup script to make it start and not die on boot https://github.com/freeload101/SCRIPTS/blob/master/MISC/StartKokoro.xml

Open WebUI RAG fail sause ... https://youtu.be/CfnLrTcnPtY

Open registration 

Model list / order

NAME                                                   ID              SIZE      MODIFIED

hf.co/mradermacher/L3-8B-Stheno-v3.2-i1-GGUF:Q4_K_S    017d7a278e7e    4.7 GB    2 days ago

qwen2.5:32b                                            9f13ba1299af    19 GB     3 days ago

deepsex:latest                                         c83a52741a8a    20 GB     3 days ago

HammerAI/openhermes-2.5-mistral:latest                 d98003b83e17    4.4 GB    2 weeks ago

Sweaterdog/Andy-3.5:latest                             d3d9dc04b65a    4.7 GB    2 weeks ago

nomic-embed-text:latest                                0a109f422b47    274 MB    2 weeks ago

deepseek-r1:32b                                        38056bbcbb2d    19 GB     4 weeks ago

psyfighter2:latest                                     c1b3d5e5be73    7.9 GB    2 months ago

CognitiveComputations/dolphin-llama3.1:latest          ed9503dedda9    4.7 GB    2 months ago

Disable Arena models

Documents WIP RAG is not good .

Discord notes;

https://discord.com/channels/1170866489302188073/1340112218808909875

• Abhi Chaturvedi:  @(Operat0r) try this To reduce latency and improve accuracy, modify the .env file: Enable RAG ENABLE_RAG=true
• Use Hybrid Mode (Retrieval + Reranking for better context)
• RAG_MODE=hybrid
• Reduce the number of retrieved documents (default: 5)
• RETRIEVAL_TOP_K=3
• Use a Fast Embedding Model (instead of OpenAI's Ada-002)
• EMBEDDING_MODEL=all-MiniLM-L6-v2 # Faster and lightweight . Optimize the Vector Database VECTOR_DB_TYPE=chroma CHROMA_DB_IMPL=hnsw # Faster search CHROMA_DB_PATH=/root/open-webui/backend/data/vector_db. Optimize Backend Performance # Increase Uvicorn worker count (improves concurrency) UVICORN_WORKERS=4
• Increase FastAPI request timeout (prevents RAG failures)
• FASTAPI_TIMEOUT=60
• Optimize database connection pool (for better query performance)
• SQLALCHEMY_POOL_SIZE=10
• So probably the first thing to do is increase the top K value in admin -> settings -> documents, or you could try the new "full context mode" for rag documents. You may also need to increase the context size on the model, but it will make it slower, so you probably don't want to do that unless you start seeing the "truncating input" warnings.
• @JamesK
• So probably the first thing to do is increase the top K value in admin -> settings -> documents, or you could try the new "full context mode" for rag documents. You may also need to increase the context size on the model, but it will make it slower, so you probably don't want to do that unless you start seeing the "truncating input" warnings.
• M]
• JamesK:  Ah, I see. The rag didn't work great for you in this prompt. There are three hits and the first two are duplicates, so there isn't much data for the model to work with
• [9:12 PM] JamesK:  context section
• I see a message warning that you are using the default 2048 context length, but not the message saying you've hit that limit (from my logs the warning looks like
• level=WARN source=runner.go:126 msg="truncating input prompt" limit=32768 prompt=33434 numKeep=5
• [6:06 AM] JamesK:  If you set the env var OLLAMA_DEBUG=1 before running ollama serve it will dump the full prompt being sent to the model, that should let you confirm what the rag has put in the prompt
• JamesK: Watch the console output from ollama and check for warnings about overflowing the context. If you have the default 2k context you may need to increase it until the warnings go away
• [8:58 PM] JamesK:  But also, if you're using the default rag, it chunks the input into small fragments, then matches the fragments against your prompt and only inserts a few fragments into the context, not the entire document. So it's easily possible for the information you want to not be present.

Auto updates

echo '0,12 */4 * * * docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui' >> /etc/crontab

Search

red note for API keys 

1. Go to Google Developers, use Programmable Search Engine , and log on or create account.
2. Go to control panel and click Add button
3. Enter a search engine name, set the other properties to suit your needs, verify you're not a robot and click Create button.
4. Generate API key and get the Search engine ID . (Available after the engine is created)
5. With API key and Search engine ID , open Open WebUI Admin panel and click Settings tab, and then click Web Search
6. Enable Web search and Set Web Search Engine to google_pse
7. Fill Google PSE API Key with the API key and Google PSE Engine Id (# 4)
8. Click Save

Note ​

You have to enable Web search in the prompt field, using plus ( + ) button. Search the web ;-)

Kokoro / Open Webui 

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

https://github.com/remsky/Kokoro-FastAPI?tab=readme-ov-file

apt update

apt upgrade

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg   && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list |     sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' |     sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

apt install docker.io -y

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2

http://localhost:8880/v1

af_bella

Import fabric prompts

https://raw.githubusercontent.com/freeload101/Python/46317dee34ebb83b01c800ce70b0506352ae2f3c/Fabric_Prompts_Open_WebUI_OpenWebUI.py