Finding a Local Japanese LLM That's Good at Writing Blog Posts

This post included, I do most of my blog writing together with an LLM. High-end models like Claude and GPT can write long text fast. But that speed comes with a lot of hallucination, which means a lot of rework. The more back-and-forth it takes, the more it costs — and more than that, it eats my time.

So I figured a model that is free, that I can hand a job to and leave alone, and that needs little rework might actually suit blog writing better — even if it is slower — as long as I have gathered the facts first. That is why I decided to try local Japanese LLMs running on my own machine.

I compared local Japanese LLMs that run on a MacBook Air (M3, 24GB memory) via Ollama. The task I measured was not the usual short question-and-answer accuracy, but whether a model could write long Japanese text like the posts on this blog — in a specified style, and faithful to the facts I gave it.

This post covers the results of that comparison, plus the actual Ollama API calls and the prompts I used, and a prompting tip I picked up along the way. To give away the conclusion: I went with Shisa V2.1 unphi4 14B. The faster 8B models were plenty capable too, but since I run several tasks in parallel, I weighted hands-off operation and low rework heavily, and chose the 14B. I will walk through why.

My criteria: hands-off operation and low rework

I always have several tasks running in parallel. So what I want is a model I can hand a job to and leave alone, and one whose output I do not have to heavily rework. If a model has both, then even when generation is slow, I can work on something else while I wait. That is why, ahead of raw speed, these two come first.

Concretely, hands-off operation and low rework show up as “staying faithful to the facts I gave, and writing in the style I asked for.” Conversely, if a model rewrites facts or drifts from the style, the touch-ups before publishing pile up. I will always do a final editing pass myself — that is a given — but how small I can keep that pass is what makes the difference. The question is whether a model can write a long piece like this post, over 3,000 characters, in a style close to my own and faithful to the facts I provided.

The models I compared, and downloading them

Here are the three models I chose to compare. I downloaded all of them with ollama pull (Ollama 0.32.0 this time). All are quantized to Q4_K_M.

ollama pull qwen3:8b
ollama pull hf.co/mradermacher/shisa-v2.1-qwen3-8b-GGUF:Q4_K_M
ollama pull hf.co/mradermacher/shisa-v2.1-unphi4-14b-GGUF:Q4_K_M

Shisa V2.1 is not in the official library, so I pulled mradermacher’s GGUF conversion directly from Hugging Face. On disk they are 5.2GB / 5.0GB / 8.9GB. The 70B would be around 40GB at Q4 and will not fit in memory at run time on a 24GB machine, so I left it out this time.

Calling the Ollama API

Rather than the interactive ollama run, I POSTed directly to /api/chat. I fixed the temperature and the seed so that the same input returns the same output.

curl -s http://localhost:11434/api/chat -d '{
  "model": "hf.co/mradermacher/shisa-v2.1-unphi4-14b-GGUF:Q4_K_M",
  "stream": false,
  "think": false,
  "options": {
    "temperature": 0.8,
    "seed": 42,
    "num_ctx": 8192,
    "num_predict": 3072
  },
  "messages": [
    {"role": "system", "content": "(the style spec; see below)"},
    {"role": "user",   "content": "(the situation, facts, and request; see below)"}
  ]
}'

The message.content in the returned JSON is the body. You can read the generation time from total_duration and the token count from eval_count. With this setup the 14B took 13.5 minutes for 3,000 characters (about 4.5 tok/s); the 8B models took roughly half that.

Prompt design: give it context and enough facts

Across the three, what moved output quality the most was not the model — it was what I fed in. This was the single most useful thing I learned for actual writing, so I am writing it down as a technique.

If you hand a model only a bullet list of facts and demand a character count, it invents facts it does not have in order to fill the gap. That is hallucination. And the larger the model, the more likely it is, because it dutifully tries to hit the length. It starts volunteering opinions you never gave it — “I think X is good” — and spinning up a story just to reach the word count.

Flip that around and the fix is simple: before you send anything, gather it all up first — who is writing, in what situation and for what purpose, a sample of the style, and enough facts to match the target length. Do that and the model no longer needs to pad with its own invention; the fact-fabrication stops, and the writing moves toward the style you wanted.

First, specify the style in system. (The prompt is in Japanese, since the goal is to produce Japanese. It sets a first-person, polite-form (desu/masu) base with short plain-form sentences mixed in, no hype, honest and concrete, and gives a real excerpt from an existing post as a style sample.)

あなたは技術ブログ「あそぴテックのごった煮ブログ」を書いている「私」本人です。
一人称。基本は敬体（です・ます）ですが、要所で常体を短く落として本音を出します。
誇張・宣伝文句・スピリチュアルな断定はしません。実際に手を動かして感じたことを、
具体的な事実とともに、正直に書きます。

文体の見本（実際の記事の一節）:
「今さら、新しいデータベースを作っているなんて、俺は何をしてるんだろう。
Alopex DBの作業中、そう思うことが何度でもある。世の中には、すでに優れた
データベースがいくつもあります。（中略）ローカルで軽く動かしたいならSQLiteは強い。
手元で分析したいならDuckDBは便利です。」
——このように、敬体を基調にしつつ、短い常体を混ぜて呼吸を作ります。

Then in user, pass the situation, your stance, the facts, and the request.

## 状況
この週末、私はこのブログの記事執筆を手伝わせるローカルLLMを1つ選ぶために、
手元のMacBook Air（M3・メモリ24GB）で日本語モデルをいくつか動かして比べていました。

## 私の立場（ここが大事）
応答速度は重視していません。むしろ遅くてもかまいません。生成を待っているあいだ、
私は別の作業に手を動かせるからです。見たかったのは、どのモデルが一番、
自分の言葉に近い、自然な日本語を書くかということでした。

## 材料（この範囲の事実で書く。数字や製品名を新たに作らないこと）
- 環境は Ollama 0.32.0、量子化はすべて Q4_K_M
- 比べたのは qwen3:8b、Shisa V2.1 Qwen3 8B、Shisa V2.1 unphi4 14B、それと古い llama3
- llama3は6タスク中3つ（要約・敬語メール・二十四節気）で日本語ではなく英語で返した
- 二十四節気「芒種」の読みは「ぼうしゅ」。14Bだけが正しく読み、8Bの2つは外した
- 敬語のメールでは、14Bが拝啓・敬具を使い丁寧だった
- 速度は8Bが14Bのおよそ倍。14Bは3000字に13分半
- 事実だけ渡して800字で書かせると、14Bは材料にない事実を1つも足さなかった

## 依頼
この週末に実際に手を動かした「私」として、上の材料をもとに本文を書いてください。
1500字程度、一人称、敬体を基調に短い常体を混ぜる。材料にない事実は書かない。
見出しは付けず、本文のみ。

With this approach the 14B did not rewrite the facts I gave it, and wrote as instructed — a polite-form base with short plain-form sentences mixed in. Given enough facts and a clear purpose, the model stops compensating and focuses on composition. When you want to suppress hallucination, adding more facts works better than switching models.

Missing knowledge can be supplied the same way — just write it into the prompt. For example, the reading of the seasonal term 芒種 (bōshu): some models return it correctly, others get it wrong, but add one line to the prompt — “芒種 = bōshu” — and any model writes it correctly. Since knowledge can be supplied from outside, it was more practical to choose a model by how well it composes the facts I give into natural Japanese than by how much it already knows. So in the end I decided on style.

Comparing the three models’ long-form output

I sent the same system and user to each of the three models once, with a fixed seed, and read the returned text side by side. Whether they rewrote any facts I checked mechanically by matching the numbers; the naturalness of the style I judged by reading.

Shisa V2.1 14B was the best at natural style. It writes on a polite-form base, mixing in short plain-form sentences at key moments.

応答速度は正直どうでもよかった。待ってる間も他の作業が出来るから、それは歓迎。一番気になっていたのは、どのモデルが自分の言葉に最も近い、自然な日本語を書けるか、ということでした。

(Response speed honestly did not matter. I can do other work while I wait, so I welcome it. What I cared about most was which model writes the most natural Japanese, closest to my own words.)

It came closest to the polite/plain mix in my sample (”…は強い。 …便利です。”), and wrote all 1,326 characters with no errors in the model names or numbers I had given it.

qwen3:8b had no misreadings or fabrications and was faithful to the facts, with speed to match. Its writing ran a bit short and leaned toward plain form, drifting from the polite-form base I had asked for, but its balance of accuracy and speed is high. Shisa V2.1 Qwen3 8B had the most distinct first-person voice — it even picked up the Alopex DB topic from my style sample to close the piece — and reproduced the style most naturally of the three. However, it rewrote the model version as “V20.1,” so it would need a fix before publishing.

The model I chose, and how I use it

Given all this, I am adopting Shisa V2.1 unphi4 14B to assist with writing this blog.

It writes long Japanese in the style I specify, faithful to the facts I give, and does not rewrite them — so there is little to touch up before publishing. Generating 3,000 characters takes 13.5 minutes, but I can spend that time on other work, so hands-off operation and low rework come together. That combination is exactly what I wanted for running several tasks in parallel.

Both 8B models, averaging speed and accuracy, were plenty good. That said, for sit-down blog writing what I weight most is low rework. Reading all three side by side, the one that was most reproducible for a given prompt, and expressed what I wanted to say most directly, was the 14B.

So going forward, I plan to write my posts mainly with this model.