기본 콘텐츠로 건너뛰기

AnythingLLM Not Working? Fixes for Docs, Ollama and Embeds

Summary: Most AnythingLLM problems fall into four buckets: the model ignores your documents, Ollama shows no models, embedding fails with "Fetch failed", or an agent never calls its tools. Each has a specific cause that AnythingLLM's own documentation explains, and most are fixed with one setting or one URL change rather than a reinstall. This guide walks through each symptom in the order you should check it, plus the October 2026 engine change that confuses people upgrading the Desktop app.

Before you start: find your version, logs and storage folder

AnythingLLM comes in three forms that fail in different ways: the free Desktop app (macOS, Windows, Linux), the free self-hosted Docker image, and the paid hosted Cloud. Know which one you are on before changing anything, because the fix for "Ollama not connecting" is completely different on Desktop and in Docker.

  1. Open the app info panel. In the Desktop app, click the version number in the top-right corner. The panel shows the installed version and the latest available one, the local ports for the API server and the document collector, and two buttons: Open logs and Open storage.
  2. Update if you are behind. Several behaviours in this guide depend on the version. Document attaching in chat changed in 1.8.5, Agent mode became a workspace option after 1.11.1, and Pro needs Desktop 1.15.0 or later.
  3. Know where the storage folder lives. On macOS it is /Users/<user>/Library/Application Support/anythingllm-desktop/storage, on Linux ~/.config/anythingllm-desktop/storage/, and on Windows C:\Users\<user>\AppData\Roaming\anythingllm-desktop\storage. Inside you will find logs, models, documents, lancedb (the default vector database) and anythingllm.db.
  4. Get verbose errors if the app crashes. Run the executable from a terminal so its logs print live. On Windows the usual path is C:\Users\<user>\AppData\Local\Programs\anythingllm-desktop\AnythingLLMDesktop.exe. For Docker, open Container Logs in Docker Desktop or print them with the docker logs command.

One easy-to-miss conflict: Desktop and the default Docker setup both use port 3001. If both run at once, the Desktop app can stop accepting text or opening its menu. Stop the container, or remap Docker to another host port (for example -p 3002:3001) and open it at http://localhost:3002.

Problem 1: "The AI ignores my documents"

This is the most common complaint, and AnythingLLM's docs answer it bluntly: the model does not read your whole library. When a document is embedded, it is split into chunks and stored as vectors. Your question is embedded the same way, and only the nearest chunks (usually four to six) are passed to the model. If the right chunk is not retrieved, the model never sees it. That is why "list the three highest-calorie recipes in my collection" fails even though each recipe is technically in the workspace.

Step 1: Loosen the similarity threshold

Hover over the workspace, click the gear icon, and open Vector Database. The Document similarity threshold filters out chunks below a score; the docs say the default minimum is 20% and that this setting is "likely the cause" of most missing-answer reports. Set it to No restriction, save with Update workspace, and ask again. The other options are Low (≥ .25), Medium (≥ .50) and High (≥ .75).

Step 2: Try Accuracy Optimized search

In the same tab, Search Preference can be switched from Default to Accuracy Optimized. It searches more chunks and re-ranks them. AnythingLLM's own testing puts the extra cost at roughly 100 to 500 ms per reply, and the reranking model downloads once on first use. This option is only available with the default LanceDB vector database.

Step 3: Adjust Max Context Snippets carefully

This controls how many chunks reach the model. The docs recommend 4 to 6 for most models. Going higher can help with large-context models, but more snippets also means more noise, and AnythingLLM trims context that would overflow the model.

Step 4: Use the document's own words, or pin it

Retrieval is mathematical (cosine distance), not a reading of intent, so phrase questions the way the document is written. If a file is critical and small enough to fit the context window, open the workspace's document list and click the pushpin icon. Pinning inserts the full text into every chat in that workspace. It is slower and costs more tokens with paid APIs, but the model sees everything. You can only pin a document that has already been embedded.

Attached vs embedded: a newer source of confusion

Since version 1.8.5, dragging a file into the chat box attaches it: the full text goes into that one thread only, not the rest of the workspace. If the attachments exceed the model's context window, AnythingLLM asks whether to Cancel, Continue Anyway or Embed. The docs advise against Continue Anyway because the app silently prunes text to fit, which produces inaccurate answers. Choose Embed if you want the file in every thread, and remember that in multi-user mode an embedded file becomes visible to everyone with access to that workspace.

Finally, check the chat mode under Chat Settings. Query mode answers only from your documents and says so when it finds nothing; Chat mode mixes documents with the model's general knowledge. If Query mode keeps replying that no relevant information was found, that is the similarity threshold again.

Problem 2: Ollama connected, but no models appear

AnythingLLM tries to detect Ollama automatically at http://127.0.0.1:11434, http://host.docker.internal:11434 and http://172.17.0.1:11434. If your Ollama is elsewhere, you have to type the URL into the LLM settings yourself.

  1. Confirm the Ollama server is running. Open http://127.0.0.1:11434 in a browser; you should see the plain text "Ollama is running". The docs point out that ollama run model-name alone does not start the API server, so use ollama serve (or the Ollama desktop app) if the page does not load.
  2. If AnythingLLM runs in Docker, stop using localhost. Inside a container, localhost, 127.0.0.1 and 0.0.0.0 point at the container itself, not your computer. Use http://host.docker.internal:11434 on Windows and macOS, or http://172.17.0.1:11434 on Linux. The same rule applies to LM Studio (port 1234), Chroma, PostgreSQL or anything else on the host.
  3. If Ollama is on another machine, use that machine's IP address and make sure its firewall allows the port. AnythingLLM cannot discover it for you.
  4. AnythingLLM Cloud cannot reach your home Ollama. The docs say it would require exposing your machine to the internet with a tunnel, which they explicitly do not recommend or support.

If Ollama itself is the problem (GPU not used, models failing to load), our Ollama troubleshooting guide covers that side.

The October 2026 engine change

Separate from "your own Ollama", the Desktop app used to ship a bundled copy of Ollama as its built-in engine. According to AnythingLLM's October 2026 blog post, new installs now use llama.cpp managed by AMD's open-source Lemonade server instead. Existing setups keep working until you click Engine upgrade in the top bar, and a future update will move everyone automatically. The catch: models must be downloaded again for the new engine, so do the upgrade when you have time and bandwidth. Connecting to an Ollama you installed yourself is unaffected. The same post says the new engine installs only the GPU libraries your machine can use, around 137 MB on a typical Windows PC without an NVIDIA card versus about 1.8 GB for the old bundled engine.

Problem 3: "Fetch failed" when embedding a file

The built-in embedder model is not bundled with the app; it downloads on first use. The docs list three causes, most likely first:

  1. Downloads are blocked. Look in your storage folder for models/Xenova. If it does not exist, a firewall is probably blocking the model download. Allow huggingface.co, api.huggingface.co and https://cdn.anythingllm.com/support/models/, then try again. Company laptops are the usual culprits.
  2. Windows is missing the Visual C++ Redistributable. Install Microsoft's Visual C++ Redistributable v14.x and embed again.
  3. Your CPU lacks AVX2. The default LanceDB vector database needs it. Virtual machines sometimes hide AVX2 from the guest. The fix is a different machine or a different vector database provider.

On the hosted Cloud, the equivalent failure is a 502 error after uploading something large. The built-in embedder runs on the instance's limited CPU, and a big document (measured in words, not file size) can kill the process. The help page suggests about 10,000 words per file on the smaller tier and 50,000 on the larger one, using the older tier names Starter and Professional. Re-uploading the same file will crash it again. Reboot the instance from the my.mintplexlabs.com dashboard, then split the file or switch to a cloud embedder.

Problem 4: The agent never uses its tools

Agent tool calls are just the model writing JSON that matches a tool's schema. Small, heavily quantized local models are bad at that, so the docs' advice centres on the model rather than AnythingLLM:

  • Check whether a tool actually ran. A real tool call shows a "thought" step in the chat. If you get an answer without one, the model probably made the result up.
  • Use a larger or less quantized model, or a cloud model just for agent calls while keeping a local model for normal chat.
  • Reset and trim. Type /reset to clear chat history, then ask again, and turn off tools you are not using so the prompt is smaller.
  • Know which mode you are in. If you see the @ symbol in the prompt box, you are not in Agent mode and need to start with @agent. If a provider's native tool calling misbehaves, the docs mention the PROVIDER_DISABLE_NATIVE_TOOL_CALLING environment variable to fall back to prompt-based calling.

On Cloud, custom agent skills and MCP servers are not supported at all for security reasons. If you need them, use Desktop or self-host with Docker.

Which AnythingLLM option avoids which problem

Option Price (official) Built-in local LLM Custom agents / MCP Typical trouble spot
Desktop app Free, no account needed; optional Pro license removes daily limits on Magic features Yes (llama.cpp via Lemonade on new installs) Yes Firewall blocking the embedder download; port 3001 clash with Docker
Self-hosted Docker Free Off by default; enable with ENABLE_BUILT_IN_LLM=true (releases after v1.18.0), or connect Ollama, LM Studio, etc. Yes Using localhost instead of host.docker.internal or 172.17.0.1
Cloud Basic $50/month; private instance, custom subdomain, bring an LLM API key No No 502 errors from large uploads with the built-in embedder
Cloud Pro $99/month; priority resources, 72-hour support SLA No No Same limits, more compute headroom

Enterprise (on-premise, SSO, RBAC, custom SLA) is quote-only. If you are weighing alternatives, see our Open WebUI playbook for small teams and the LM Studio tutorial, which covers chatting with files in another desktop app.

Editor's note

Good fit: people who want private document chat on their own computer without an account, and small teams willing to self-host. Poor fit: anyone expecting the model to "know" every file in a big library. Retrieval only ever returns a handful of chunks, so questions that need a full scan (totals, rankings across hundreds of files) will disappoint unless you pin small documents.

Gotchas we confirmed in the official docs:

  • Embedding a file shares it with every thread in the workspace, and with every user of that workspace in multi-user mode.
  • "Continue Anyway" on the context warning silently drops text. Choose Embed instead.
  • Upgrading to the new built-in engine means downloading your models again.
  • Cloud has no built-in LLM and cannot use your home Ollama.

Quick fixes: no relevant information found → similarity threshold to No restriction. No Ollama models in Docker → host.docker.internal:11434 (Linux: 172.17.0.1:11434). Fetch failed → check for models/Xenova and unblock Hugging Face. Desktop frozen → stop the Docker container using port 3001.

FAQ

Q: Is AnythingLLM free?
A: The Desktop app and the self-hosted Docker version are free. The optional Pro license only removes daily limits on the Magic features (Magic Echo, Magic Beacon, Magic Tab) and the watermark on generated documents; if it lapses, those features fall back to their free daily allowance. The hosted Cloud starts at $50 a month.

Q: Why does Query mode say it found nothing when the answer is in my PDF?
A: The chunk holding the answer was probably filtered out by the similarity threshold, or your wording doesn't match the document. Set the threshold to No restriction, reuse the document's own terms, and try Accuracy Optimized search. For background on how vector search behaves, see our Pinecone vs Qdrant vs Weaviate comparison.

Q: Do I still need Ollama to run local models?
A: No. New Desktop installs run models with the built-in llama.cpp engine and can download GGUF files from Hugging Face directly. Ollama remains a supported provider if you prefer to run it yourself. The Docker image also has a built-in engine on releases after v1.18.0, but it is off until you start the container with -e ENABLE_BUILT_IN_LLM=true; it cannot be switched on from the UI.

Check the version, the URL and the similarity threshold before anything else; that order solves most AnythingLLM problems. The official screenshots from AnythingLLM's documentation are at the end of this post.

Images: official screenshots from the AnythingLLM documentation (docs.anythingllm.com).

Last verified: 2026-11-05
Sources: Attaching vs RAG, RAG in AnythingLLM, Chat modes, Ollama connection debugging, Connecting to localhost (Docker), Built-in LLM for Docker, Fetch failed error on embed, AI agent not using tools, Desktop debugging and logs, Where is my data stored, Cloud limitations, 502 error on hosted, AnythingLLM Pro, AnythingLLM Cloud pricing, Moving from Ollama to Lemonade and llama.cpp.

Uploaded Image Uploaded Image Uploaded Image Uploaded Image

댓글

이 블로그의 인기 게시물

브루(Vrew) 요금제 정리: 무료 200크레딧으로 어디까지 쓸 수 있나

요약: 영상 자막 편집기 브루(Vrew)는 2026년 4월 22일부터 기능별 사용량 대신 '통합 크레딧' 하나로 요금을 매겨요. 무료 플랜은 매달 200크레딧이 들어오는데, 음성 분석(자동 자막)만 쓰면 최대 200분 분량이라 짧은 영상 몇 편은 충분하지만, 영상 초반의 Vrew 마크를 지우려면 유료 플랜이 필요해요. 이 글은 브루 공식 가격 페이지와 고객센터 공지를 2026년 10월 23일에 다시 확인해, 크레딧 계산법과 플랜 고르는 기준을 정리했어요. 목차 브루 요금 구조: 통합 크레딧이란 1크레딧으로 할 수 있는 일 무료 플랜으로 되는 것과 안 되는 것 유료 플랜 4종 비교표 월 구독·연 구독·기간 이용권, 무엇이 이득일까 상황별 추천: 학생·직장인·유튜버 크레딧이 모자랄 때: 사용 기록 지우개 업그레이드·환불 전에 알아둘 규칙 운영자 메모, 자주 묻는 질문 1. 브루 요금 구조: 통합 크레딧이란 예전 브루는 플랜마다 음성 분석 몇 분, AI 목소리 몇 자, 번역 몇 자처럼 기능별 한도를 따로 줬어요. 그런데 보이저엑스(브루 운영사)가 실제 이용 데이터를 보니 한두 가지 기능만 집중해서 쓰는 사람이 훨씬 많았다고 해요. 그래서 2026년 4월 22일 업데이트부터는 플랜마다 정해진 크레딧을 받고, 어떤 기능에 쓸지는 내가 정하는 방식으로 바뀌었어요. 플랜별 매달 지급량은 Free 200 · Light 2,000 · Standard 10,000 · Business 12,000크레딧 이에요. 공식 공지에 따르면 무료 사용자는 매달 1일에, 유료 사용자는 매월 결제일에 충전되고, 남은 크레딧은 다음 달로 넘어가지 않아요. 바뀐 뒤 무엇이 좋아졌나 공지에 나온 예시가 이해하기 쉬워요. Standard 플랜에서 AI 목소리만 쓰면 예전에는 최대 50만 자였지만, 지금은 1만 크레딧을 모두 AI 목소리에 몰아 쓸 수 있어 최대 100만 자가 돼요. 반대로 여러 기능을 골고루 쓰는 사람은 크레딧이 기능끼리 ...

에이닷 전화 vs 익시오: 통화 녹음·AI 요약, 내 통신사·폰에 맞는 앱 고르기

요약: 통화 녹음과 AI 요약을 쓰고 싶다면 먼저 내 통신사와 휴대폰 종류부터 확인해야 해요. LG유플러스 고객은 무료 앱 익시오, SKT 고객은 에이닷 전화가 가장 기능이 많고, 다른 통신사의 아이폰 사용자는 iOS에 들어 있는 기본 통화 녹음·전사 기능이 현실적인 선택이에요. 이 글은 2026-10-30에 SKT·LG유플러스·Apple 공식 페이지와 도움말을 직접 확인해 세 가지를 비교했어요. 왜 '앱 기능'보다 '내 조건'을 먼저 봐야 할까요 통화 요약 앱 비교 글을 보면 기능 목록부터 나열하는 경우가 많아요. 그런데 실제로 써 보려는 사람에게 가장 먼저 걸리는 건 기능이 아니라 이용 조건 이에요. 익시오는 LG유플러스 고객 전용이고, 에이닷 전화는 누구나 설치할 수 있지만 통역콜·baro 같은 기능은 SKT 고객에게만 열려 있어요. 아이폰은 운영체제 구조상 앱이 통화를 마음대로 녹음하기 어려워서, 통신사 앱이 안 되면 iOS 기본 기능에 기대야 해요. 그래서 이 글은 "어떤 앱이 제일 좋은가"보다 "내 통신사·내 폰에서 무엇을 쓸 수 있나" 순서로 정리했어요. 기능 차이는 그다음이에요. 한눈에 비교: 에이닷 전화 vs 익시오 vs 아이폰 기본 통화 녹음 구분 에이닷 전화 (SKT) 익시오 ixi-O (LG유플러스) 아이폰 기본 통화 녹음 (Apple) 누가 쓸 수 있나 통신사 구분 없이 앱 설치 가능. 단, 통역콜·baro·그룹통화 발신 등은 SKT 고객 전용 U+ 모바일 고객. 타 통신사, 만 14세 미만, 선불요금제 고객은 제한 (공식 사이트 FAQ상 알뜰폰 중 KB리브엠은 예외) 지원 국가·언어의 iPhone 사용자. 1명과 통화할 때만 녹음 가능 지원 OS·기기 Android 10 이상, iOS 16 이상 iOS 17 이상 iPhone 12 이상 / Android...

타입캐스트 vs 클로바더빙: 수익 나는 채널에 쓸 때 무료·유료 조건 비교

요약 타입캐스트와 클로바더빙은 둘 다 글만 입력하면 한국어 AI 목소리를 만들어 주는 서비스지만, 무료로 쓸 수 있는 범위가 꽤 다릅니다. 클로바더빙 무료는 매달 15,000자까지 쓸 수 있는 대신 수익이 생기는 채널에는 쓸 수 없고, 타입캐스트 무료는 처음에 주는 약 5분 분량의 다운로드로 끝나는 체험판에 가깝습니다. 광고 수익이 붙은 유튜브나 쇼핑몰 영상에 넣을 목소리라면 처음부터 유료 플랜 조건을 보고 고르는 편이 안전합니다. 왜 이 두 서비스를 같이 비교하나 쇼츠 내레이션, 수업 자료 더빙, 상품 소개 영상처럼 "내 목소리 대신 AI 목소리"가 필요할 때 한국 사용자들이 가장 많이 찾는 곳이 네오사피엔스의 타입캐스트 와 네이버의 클로바더빙 입니다. 기능 소개만 보면 비슷해 보이지만, 실제로 문제가 되는 건 음질보다 "이 목소리를 내 채널에 써도 되나" 입니다. 그래서 이 글은 기능 나열보다 무료 사용 범위, 출처 표기 의무, 유료 플랜 가격과 다운로드 한도를 중심으로 정리했습니다. 모든 수치는 2026년 10월 25일에 두 서비스의 공식 요금 페이지와 고객센터 도움말에서 직접 다시 확인한 내용입니다. 한눈에 보는 비교표 항목 타입캐스트 무료 타입캐스트 베이직 클로바더빙 Free 클로바더빙 Standard 월 요금 0원 9,900원 (연간 결제 시 10% 할인) 0원 19,900원 (네이버 클라우드 플랫폼에서 결제) 무료·포함 사용량 음성 생성·재생 무제한, 다운로드는 최초 3,000크레딧(약 5분) 1회 지급 월 30,000크레딧(약 60분) 월 15,000자, 다운로드 월 20회, 프로젝트 누적 5개 (매월 1일 갱신) 월 30,000자, 다운로드 월 40회, 프로젝트 누적 20개 쓸 수 있...