{"id":78,"date":"2026-10-11T19:10:08","date_gmt":"2026-10-11T19:10:08","guid":{"rendered":"https:\/\/ivand.eu\/?p=78"},"modified":"2026-10-11T19:13:15","modified_gmt":"2026-10-11T19:13:15","slug":"localmind-updates-ui-first-rag-score-gates-and-real-gpu-installs","status":"publish","type":"post","link":"https:\/\/ivand.eu\/index.php\/2026\/10\/11\/localmind-updates-ui-first-rag-score-gates-and-real-gpu-installs\/","title":{"rendered":"LocalMind updates: UI-first RAG, score gates, and real GPU installs"},"content":{"rendered":"\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"506\" data-id=\"81\" src=\"https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49-1024x506.png\" alt=\"\" class=\"wp-image-81\" srcset=\"https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49-1024x506.png 1024w, https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49-300x148.png 300w, https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49-766x379.png 766w, https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49-1536x760.png 1536w, https:\/\/ivand.eu\/wp-content\/uploads\/2026\/10\/\u0421\u043d\u0438\u043c\u043a\u0430-\u043d\u0430-\u0435\u043a\u0440\u0430\u043d\u0430-\u043d\u0430-2026-10-11-21-35-49.png 1913w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">I shipped a solid round of updates to <strong>LocalMind<\/strong>, my privacy-first Agentic RAG assistant. The goal was simple: less terminal friction, clearer retrieval behavior, and hardware acceleration that you can actually turn on from the app.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">From \u201crun three commands\u201d to open and go<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Starting the app used to mean activating a venv and remembering a Streamlit path. Now you can:<\/p>\n\n\n<ul class=\"wp-block-list is-layout-flex wp-container-core-list-is-layout-c4cf3051 wp-block-list-is-layout-flex\" style=\"\">\n<li>Double-click <strong>start.bat<\/strong> on Windows<\/li>\n\n\n\n<li>Or run <strong>start.ps1<\/strong> \/ <strong>start.sh<\/strong> \/ <strong>python run.py<\/strong>\n<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The launcher can create the venv and install dependencies on first run. That sounds small, but it changes who can try the project.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Everything important is in the UI<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">LocalMind is no longer \u201cedit files, then chat.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Documents<\/strong><br>Upload PDF, DOCX, TXT, or Markdown, delete files, and <strong>Build index<\/strong> in one place. No separate ingest step required for normal use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Settings<\/strong><\/p>\n\n\n<ul class=\"wp-block-list is-layout-flex wp-container-core-list-is-layout-c4cf3051 wp-block-list-is-layout-flex\" style=\"\">\n<li>GGUF model path (saved to .env)<\/li>\n\n\n\n<li>Preferred backend: Auto \/ CUDA \/ Metal \/ ROCm \/ CPU<\/li>\n\n\n\n<li>GPU layers (&minus;1 auto, 0 CPU, or fixed)<\/li>\n\n\n\n<li>Status for what is actually loaded<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Chat<\/strong><br>Same agentic flow, with tool badges, sources, and more visible routing decisions.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The important RAG fix: retrieval must earn generation<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A common failure mode in agentic RAG: the router picks <em>local docs<\/em>, retrieval returns weak chunks, and the model still answers with confidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">LocalMind now applies a <strong>retrieval score threshold<\/strong> after hybrid search (dense + BM25):<\/p>\n\n\n<ul class=\"wp-block-list is-layout-flex wp-container-core-list-is-layout-c4cf3051 wp-block-list-is-layout-flex\" style=\"\">\n<li>If max_score is below the threshold &rarr; local context is <strong>rejected<\/strong>\n<\/li>\n\n\n\n<li>Then either <strong>fall back to web search<\/strong> or <strong>refuse<\/strong> (configurable in the sidebar)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Under each answer you can see proposed tool, max local score, and whether the route was overridden. Soft \u201cplease don\u2019t hallucinate\u201d prompts are not enough; this is a hard gate.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Hardware acceleration you can install from Settings<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Detecting a GPU is not the same as using one. Inference depends on how <strong>llama-cpp-python<\/strong> was built.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Settings now:<\/p>\n\n\n<ol class=\"wp-block-list is-layout-flex wp-container-core-list-is-layout-c4cf3051 wp-block-list-is-layout-flex\" style=\"\">\n<li>Detects CUDA, Metal, and ROCm hardware<\/li>\n\n\n\n<li>Offers <strong>Install matching llama-cpp-python<\/strong> for that backend<\/li>\n\n\n\n<li>Prefer prebuilt CUDA wheels when possible, with source builds for Metal \/ ROCm \/ fallback<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">After install: set backend to <strong>Auto<\/strong>, GPU layers to <strong>-1<\/strong>, reload the engine.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What this is for<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">LocalMind is still a focused demo of production-shaped GenAI work:<\/p>\n\n\n<ul class=\"wp-block-list is-layout-flex wp-container-core-list-is-layout-c4cf3051 wp-block-list-is-layout-flex\" style=\"\">\n<li>RAG + hybrid retrieval<\/li>\n\n\n\n<li>Agents with tool routing<\/li>\n\n\n\n<li>Evaluation-minded design (thresholds, visible overrides)<\/li>\n\n\n\n<li>Local inference with a path to real GPU use<\/li>\n\n\n\n<li>Clear project structure and a UI that stakeholders can actually try<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Next up: a proper golden evaluation set and metrics. A threshold is only as good as the numbers behind it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>GitHub:<\/strong> LocalMind \u00b7 related: <a href=\"https:\/\/github.com\/Ivan-Dambulov\/LocalBot\" target=\"_blank\" rel=\"noopener\">LocalBot<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I shipped a solid round of updates to LocalMind, my privacy-first Agentic RAG assistant. The goal was simple: less terminal friction, clearer retrieval behavior, and hardware acceleration that you can actually turn on from the app. From \u201crun three commands\u201d to open and go Starting the app used to mean activating a venv and remembering [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-78","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/posts\/78","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/comments?post=78"}],"version-history":[{"count":3,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/posts\/78\/revisions"}],"predecessor-version":[{"id":85,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/posts\/78\/revisions\/85"}],"wp:attachment":[{"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/media?parent=78"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/categories?post=78"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ivand.eu\/index.php\/wp-json\/wp\/v2\/tags?post=78"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}