{"id":72,"date":"2026-10-10T18:45:05","date_gmt":"2026-10-10T18:45:05","guid":{"rendered":"https:\/\/ivand.eu\/?page_id=72"},"modified":"2026-10-11T19:16:27","modified_gmt":"2026-10-11T19:16:27","slug":"localmind","status":"publish","type":"page","link":"https:\/\/ivand.eu\/index.php\/localmind\/","title":{"rendered":"LocalMind"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

Privacy-first Agentic RAG knowledge assistant<\/strong><\/p>\n\n\n\n

LocalMind answers questions over your documents<\/strong> and the web<\/strong>.
An agent chooses the right tools, hybrid retrieval finds context, and answers include sources.
Weak local matches are blocked by a retrieval score threshold<\/strong> so the system does not sound confident when the docs do not support the answer.<\/p>\n\n\n\n

\n

Local inference uses GGUF models via llama-cpp-python<\/code>, with hardware detection and optional install of CUDA \/ Metal \/ ROCm builds from the Settings UI.<\/p>\n<\/blockquote>\n\n\n\n

\n\n\n\n

Features<\/h2>\n\n\n\n

<\/a><\/p>\n\n\n\n

Feature<\/th>Description<\/th><\/tr><\/thead>
Agentic routing<\/strong><\/td>Chooses local_rag<\/code>, web_search<\/code>, both<\/code>, or none<\/code> per query<\/td><\/tr>
Hybrid retrieval<\/strong><\/td>Dense embeddings + BM25, fused with tunable \u03b1<\/td><\/tr>
Score threshold<\/strong><\/td>Rejects weak local hits \u2192 web fallback or refuse<\/td><\/tr>
Citations<\/strong><\/td>Source chips and retrieved context in the UI<\/td><\/tr>
Document UI<\/strong><\/td>Upload \/ delete files and Build index<\/strong> in the browser<\/td><\/tr>
Settings UI<\/strong><\/td>Model path, GPU backend preference, install GPU llama-cpp-python<\/code><\/td><\/tr>
Hardware acceleration<\/strong><\/td>Detects CUDA, Metal, ROCm; Auto \/ CPU \/ custom GPU layers<\/td><\/tr>
Local GGUF<\/strong><\/td>Inference adapted from LocalBot<\/a><\/td><\/tr>
Easy start<\/strong><\/td>start.bat<\/code> \/ start.ps1<\/code> \/ start.sh<\/code> \/ python run.py<\/code><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n

Architecture<\/h2>\n\n\n\n

<\/a><\/p>\n\n\n

User query\n    │\n    ▼\n┌─────────────────────┐\n│  Agent (router)     │  propose tool\n└──────────┬──────────┘\n           │\n           ▼\n    Local hybrid search?\n           │\n     ┌─────┴─────┐\n     │ score OK? │\n     └─────┬─────┘\n        yes│    no → web fallback or refuse\n           ▼\n    Context (local and\/or web)\n           │\n           ▼\n    Local GGUF LLM\n           │\n           ▼\n    Answer + sources + route metadata\n<\/code><\/pre>\n\n\n\n
Retrieval score:<\/strong> after local search, if max_score < retrieval_min_score<\/code>, local chunks are not used for generation.<\/p>\n\n\n\n
\n\n\n\n
Project structure<\/h2>\n\n\n\n
<\/a><\/p>\n\n\n
LocalMind\/\n├── start.bat \/ start.ps1 \/ start.sh \/ run.py   # easy launchers\n├── requirements.txt\n├── .env                                        # LLM_MODEL_PATH, GPU settings\n├── src\/\n│   ├── config.py\n│   ├── llm.py\n│   ├── ingestion\/          # load, chunk, pipeline\n│   ├── retrieval\/          # embeddings, Chroma, hybrid search\n│   ├── inference\/          # GGUF runtime, hardware, backends\n│   ├── agent\/              # router, tools, score gate\n│   ├── services\/           # ingest, config, acceleration install\n│   └── ui\/app.py           # Streamlit UI\n├── data\/\n│   ├── raw\/                # your documents\n│   ├── processed\/          # chunks.jsonl\n│   └── vectorstore\/        # Chroma DB\n├── scripts\/ingest.py       # optional CLI ingest\n└── docs\/                   # architecture, WBS\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Quick start<\/h2>\n\n\n\n
<\/a><\/p>\n\n\n\n
Windows (easiest)<\/h3>\n\n\n\n
<\/a><\/p>\n\n\n\n
Double-click start.bat<\/code><\/strong><\/p>\n\n\n\n
Or in PowerShell:<\/p>\n\n\n\n
cd path\\to\\LocalMind\n.\\start.ps1<\/pre>\n\n\n\n
macOS \/ Linux<\/h3>\n\n\n\n
<\/a><\/p>\n\n\n\n
chmod +x start.sh\n.\/start.sh<\/pre>\n\n\n\n
Already in a venv<\/h3>\n\n\n\n
<\/a><\/p>\n\n\n\n
python run.py\n# or\nstreamlit run src\/ui\/app.py<\/pre>\n\n\n\n
The first run of start.bat<\/code> \/ start.sh<\/code> can create .venv<\/code> and install dependencies.<\/p>\n\n\n\n
\n\n\n\n
First-time setup in the UI<\/h2>\n\n\n\n
<\/a><\/p>\n\n\n
    \n
  1. \nSettings<\/strong>