tl; dr: AI is a no-go for use in my mud or in the sort of writing where it would be most helpful. It's so-so for code, but pretty good at mediocre code that dosn't matter when it's done in an automated fashion. With more hand holding, it can be pretty good with code, with slightly better trade-offs than taking the time to do it right without AI.

I tried repeatedly to use models in ollama to be stand-in personalities for NPCs in my mud. It failed horribly every time. I've been trying for over a year now. I just kind of load up a few models every few months and try them out. I start with RPG specific models from Ollama's web site. Then try general ones. If I keep this nonsense up, I'll probably also try huggingface.

I start with a model, I feed it lore from my notes, and it does an okay job of answering questions.

But then I face two main issues...

• it hallucinates on being asked for more information

• it deviates from character as soon as requested

At least it doesn't sprial into converging on a single answer or repeated phrase like it used to. But try as I may, I cannot solve the issue.

I tried (older) versions of Codellama to try and get a decent git commit message, and failed every time.

Claude Haiku, Claude Sonnet, and DeepSeek all did pretty well in creating git commit messages. But that was based on activity they carried out which was summarized in the session. I need to circle back and try comment generation based on just code diffs.

It does a decent job of sticking to common commit format. Sometimes it needs reminding that it must include a summary and a body, but otherwise it does fine.

Generating code is somewhat hit or miss with both for-pay and entirely open models.

Claude Haiku, Claude Sonnet, and DeepSeek are all decent here if there's a readily available test harness. Basically doing test-driven development with AI forces it to adhere to the contract of the test. So at the very least, it is writing code that does what the test demands of it.

Various models through ollama do okay, provided it's a common language, standard API, or generally well known. And to combat errors I have to know the content well enough to spot stupid mistakes because the dumb mistakes are heavier in the smaller parameter size models. (7b and smaller)

For a model to work as an agent for code development and driving a shell editor, it must support tools.

Ironically, codellama does not support tools. So codellama, (and cogito,) while it can read, interpret and chat about code, it cannot actually do the coding.

And some models that support tools are not putting their responses in the return message properly. Instead, they're being embedded inside the content. I tried to modify code companion for neovim to also parse these messages, but I was missing pieces.

Curiously, not all llama models can do tools. It's an odd mix. I've had luck with some llama 3.2 models, but I can't run the big ones, and the quality has been poor where I've tried to use them.

Meanwhile, Claude Haiku was awesome. And Claude Sonnet was some kind of rock star. While Haiku uses considerably fewer tokens than Sonnet, they both still use a lot of tokens when tools are being used from an editor. There's a lot of back and forth and redundant context sent back and forth. This is where DeepSeek shines. DeepSeek is an order of magnitude cheaper for tokens and it doesn't require as much back and forth as haiku and sonnet tend to require in working out and solving a problem. However; DeepSeek is slow compared to Haiku or Sonnet. And the quality of the solution isn't quite as good. Haiku and Sonnet tend to have much more recent information and a broader context for information. I was getting stellar suggestions relating to nix when using Haiku and Sonnet. I had to hold DeepSeek's hand with stuff here. And then using open models was even worse.

Agent based editing was absolutely awesome for writing programs in languages I don't know that well. Unit tests kept them on track, and programs generally worked as advertised. This is how I threw together my markdown navigate plugin for neovim.

My goal here was to feed my notes into a given model and have it write something that I could read to learn about what I'd been taking notes about in context.

I would not, nor will I ever, use AI to write a post or an article for the purpose of sharing that with others. Not only is it multiple levels of plagiarism, but what it produces reads like garbage. But I digress.

How did it do? Horrible. Non-starter. It can summarize, paraphrase, but I can't tell it to write an article for me on a topic because everything comes out looking like some shitty medium article, but worse, because it will hallucinate garbage in addition to the sources and reference material I give it.

The separate task of proof-reading is so/so. It's too expensive to use as a spell checker or grammar checker. It's cheaper to use a spell checker and proselint.

It did a pretty terrible job. Laughably bad. But to be fair, I had only tried Claude. When I realized what it was costing me I gave up. Originally I had wanted some ascii art to use as a background for admonitions in latex. I was going to use a tiny font so it would look sharp, but the ascii art was terrible and I never got as far as putting it into a document.

AI is pretty good for code if you can recognize the quality of the output and know where to look for the pitfalls. In many cases it can produce something that's good enough to get a task done. Beyond this, I haven't been able to find a practical use for it.

I have been using AI to critique code, maybe I should have it critique my notes, maybe it could point out when I'm being vague in something. Then perhaps when I review notes later there will be less of a chance of not knowing what the note is for due to poor wording.

Commercial AI (except for DeepSeek) is too expensive. I can't run a decent model locally on my hardware, and all the commercial offerings are too expensive, with the exception of DeepSeek.

Tags: index, ai

created: 2026-01-30

(re)generated: 2026-07-17