Use Nowledge Mem with fully local models
Keep memory processing, agents, embeddings, and enabled enrichment on your own machine with LM Studio, Ollama, or another local endpoint
Nowledge Mem can keep both your knowledge and the model calls that process it on your own machine. A local inference server such as LM Studio or Ollama exposes a loopback endpoint; Mem sends requests to that endpoint instead of a cloud model provider.
This guide gives you a practical first setup, a way to verify it, and the boundaries you need to check before calling the whole workflow fully local.
What fully local means here
The content processing you enable, including memory organization, agent responses, embeddings, and optional vision or OCR, must call only built-in models or endpoints on the same machine. No content is sent to a cloud model, and a local failure must not silently fall back to one.
Changing only the model used for memory processing does not, by itself, prove that every enabled workflow is local.
Local AI beyond the Mac since version 0.10
Since version 0.10, Nowledge Mem's zero-Python local runtime has covered desktop and headless Linux deployments. Official Docker images include a CPU build plus opt-in CUDA and Vulkan builds. You can run Mem on personal hardware or on your own Linux server.
The bundled Gemma 4 model is multimodal. Vision is the part integrated today: Library uses it for scanned PDFs, text in images, and other documents that need visual understanding. The underlying model also provides a foundation for audio, but production audio routing, PII filtering, and audio-meeting processing remain future directions rather than features delivered by this page today.
Beyond the model bundled with the app, Nowledge Mem gives Lemonade, Ollama, and LM Studio first-class local-provider support. Special thanks to community contributor and power user abn, who did substantial work on the Lemonade integration.
Before you start
You need:
- Nowledge Mem running in local desktop or local-server mode
- A local inference server, such as Lemonade, LM Studio, or Ollama
- A chat model loaded by that server
- Enough memory for the model and the rest of your daily applications
Lemonade's default OpenAI-compatible endpoint is http://127.0.0.1:13305/v1, LM Studio's is http://127.0.0.1:1234/v1, and Ollama's is http://127.0.0.1:11434/v1. If Mem and the model server run in different containers or on different machines, use an address that Mem can reach; inside a container, 127.0.0.1 refers to that container itself.
Start the local model server
Load a model in Lemonade, LM Studio, or Ollama, start the local server, and confirm that its OpenAI-compatible /v1 endpoint is running.
Verify that the server reports at least one model:
# Lemonade
curl http://127.0.0.1:13305/v1/models
# LM Studio
curl http://127.0.0.1:1234/v1/models
# Ollama
curl http://127.0.0.1:11434/v1/modelsThe model ID returned here is the value you should use in Mem. Do not guess it from the model's display name.
Add the provider in Mem
- Open Settings → Providers.
- Add Lemonade, LM Studio, or Ollama.
- Enter the loopback URL and the exact model ID reported by the server.
- Leave the API key empty unless your local server requires one.
- Test the connection, then save the provider.
If you use another local runtime, add an OpenAI-compatible custom endpoint. It must implement Chat Completions or the Responses API format you select.
Route every enabled model purpose
Provider setup and workload routing are separate. In the Providers screen, check each purpose:
| Purpose | Fully local choice | What it affects |
|---|---|---|
| Default | Your local provider | The model inherited by any purpose set to "Same as default" |
| Agents | Your local provider, or "Same as default" when Default is local | Background Intelligence, AI Now, Timeline, the browser-extension agent, and other agent workflows |
| Embedding | Built-in local | Search indexing and vector retrieval |
| Vision / OCR | Built-in local vision or a local vision-capable provider; otherwise leave those workflows unused | Image understanding and OCR requests used by Library |
Keeping Embedding on Built-in local is the simplest local setup. If you switch to a different embedding model or dimension, follow the reindex prompt before judging search results.
A text model is not automatically a vision model
If an external local model cannot process images, do not assign it to Vision or OCR and assume those features are covered. Use the built-in local vision model, another local vision-capable model, or keep image workflows disabled. Check the route again whenever you enable a new enrichment feature.
Verify the setup
Use a small, observable test:
- Disconnect or disable every cloud provider in Settings → Providers.
- Keep the local inference server's request log visible.
- Create a test memory with one durable preference and one piece of temporary noise.
- Run the relevant Background Intelligence action, then ask AI Now to retrieve the durable preference.
- Confirm that requests appear only in the local server log and that the result is saved or returned as expected.
- Stop the local server and repeat once. The action should fail visibly; it should not succeed through a cloud fallback.
Success means the local server handled the request, Mem produced the expected output, and no remote provider was used.
Model sizing: one user's Apple Silicon benchmark
Community member Versun tested three 4-bit models on a Mac mini with an M4 Pro and 64 GB of unified memory. All three passed the same controlled memory-organization task three times. The differences were mainly latency and memory use, not a broad measure of intelligence.
| Model | Cold load | Resident memory task | Long-input TTFT | LM Studio total peak RSS |
|---|---|---|---|---|
| Gemma 4 12B GGUF | 3.82 s | 7.38 s | 36.79 s | 10.65 GiB |
| Gemma 4 26B-A4B MLX | 8.17 s | 2.68 s | 13.83 s | 17.69 GiB |
| Qwen3.6 35B-A3B MLX | 11.55 s | 2.64 s | 9.87 s | 17.72 GiB |
His practical choices were:
- Under 32 GB: the 12B GGUF model leaves more room for other applications.
- Between 32 and 64 GB: the 26B-A4B MLX model is a safer balance.
- 64 GB and a resident model: the 35B-A3B MLX model gave the best overall latency in this test.
- Frequent load and unload: the 26B-A4B model had the lower combined load-plus-task time.
These results come from one machine, one runtime version, and one fixed prompt. Shared memory pages can also make summed RSS larger than the exact reduction in free system memory. Treat the table as a sizing reference, then test your own workload.
Read Versun's original Chinese benchmark or the formatted republication on the Nowledge Labs blog.
Privacy boundaries to review
A local model endpoint keeps inference local, but other product features can still use the network:
- Sync and remote access send the data required for those features to the service you configured.
- Browser connectors and imports fetch content from their source services.
- Updates, sign-in, and external links may contact their respective services.
- A custom endpoint on another device is self-hosted, but it is no longer on the same machine.
If your requirement is an offline machine rather than simply local inference, disable sync and remote providers, disconnect the network, and rerun the verification above.
Troubleshooting
Mem cannot list models
Confirm that the model server is running and that the URL ends in /v1. Use curl against /v1/models before debugging Mem.
The connection works, but agent tasks fail
The model may not support the chat or tool-calling behavior used by that workflow. Try a stronger instruction model and verify that Agents points to it.
The first request is much slower
That is usually a cold load. Keep the model resident if your runtime and available memory allow it. Measure cold-load time separately from steady-state requests.
Search changed after switching embeddings
Embedding vectors from different models or dimensions are not interchangeable. Complete the requested reindex and test search again.
Next, see LLM Providers for provider-specific settings and context-window guidance.