jankyai

Directory

In use now

Some of these I have actually run. The rest I would hand to a friend, and I have not tested them. Quality varies by model, task, and runtime.

Ollama

Personally used

Best forOne command, a model, and a chat box.

The catchOllama normally unloads idle models after about five minutes unless configured otherwise.

  • macos
  • linux
  • windows
easy

llama.cpp

Personally used

Best forThe runtime on the IronClaw machine. llama-server is what actually serves GPT-OSS.

The catchThe flags. There are a lot of flags, and two of them set the context.

  • macos
  • linux
  • windows
fiddly

AnythingLLM

Personally used

Best forA document chat in front of a local model. It sat in the early stack, between Ollama and llama.cpp.

The catchAnother program between you and the weights. I did not keep it as the front door.

  • macos
  • linux
  • windows
moderate

LibreChat

Personally used

Best forThe browser I actually talk to Lucy through.

The catchIt is the front door. The model and AgentGate are still doing the work behind it.

  • linux
  • macos
  • windows
moderate

LM Studio

Recommended, not personally tested

Best forA desktop app when you do not want a flag in sight.

The catchI have not run it. The model browser is the thing people notice first, and downloads add up.

  • macos
  • windows
  • linux
easy

GPT4All

Recommended, not personally tested

Best forA local chat that installs like a normal app.

The catchI have not run it. The model list tends to lag whatever just came out.

  • macos
  • windows
  • linux
easy

Jan

Recommended, not personally tested

Best forAn open desktop client with a local server under it.

The catchI have not run it. Extensions are a second project. Start with the chat.

  • macos
  • windows
  • linux
moderate

Open WebUI

Recommended, not personally tested

Best forA browser front end in front of Ollama.

The catchI have not run it. You would be running two things, and both of them need an update.

  • linux
  • macos
  • windows
moderate

ComfyUI

Recommended, not personally tested

Best forImage graphs you can actually see.

The catchI have not run it. The first graph is a project of its own, and custom nodes will fight.

  • windows
  • linux
  • macos
fiddly

whisper.cpp

Recommended, not personally tested

Best forSpeech to text on the machine you already have.

The catchI have not run it. Long audio is where a small model usually starts guessing.

  • macos
  • linux
  • windows
moderate

Can I run it?

The 16GB note is a measurement from this machine. The 8GB line is a starting point, not a test I ran on an 8GB card. Quality varies by model, task, and runtime.