Through Part 4, the conclusion
still wasn’t fully wrapped up: I couldn’t reach a final conclusion until I checked
Laptop 2’s actual openaiReasoningEffort value. But while that check was put off, a
completely different question changed the direction of the investigation.
Why won’t these two notes merge? — merging duplicate Korean and English notes
The wiki-cleanup feature (refactor) described in Part 1 finds two notes covering the same thing and proposes merging them. But this vault had a pattern that it kept missing. A Korean note “심리적 안전감” and an English note “Psychological Safety” covered exactly the same concept (psychological safety), yet no matter how many times I ran refactor, they never merged into one.
Digging into the code, the plugin catches duplicate notes in two layers. The first relies
on the rule “when two files get exactly the same title, add the suffix _dup2.” When the
title strings are literally identical, as with “클로드.md” and “클로드_dup2.md” (“Claude”),
it forces them into the same group and reviews them first as merge candidates. On top of
that, in case the first layer slips, there was already a second safety net that rescans
for the pattern in code alone, without calling the LLM.
The problem was that both layers only work when the title strings are exactly the same. “심리적 안전감” and “Psychological Safety” are the same concept, but because the titles are in different languages, they never trip either safety net. In that case everything rests on a much looser judgment, “embedding similarity,” which turns each note’s meaning into a vector and computes how close they are. How loose that judgment was, I couldn’t know until I measured it.
Measuring it on real data — cosine similarity of Korean and English embeddings
So I pulled the vault’s actual embedding cache file (the file that stores the vector
computed for each note). The vectors were computed with OpenAI’s text-embedding-3-small
and stored truncated to 256 dimensions. With this file I could compute not made-up
examples but the real similarity of notes that actually exist in this vault.
Here are the results (cosine similarity: a number from 0 to 1 for how much two vectors point in the same direction; the closer to 1, the closer in meaning).
| Pair | Similarity |
|---|---|
| Same-language duplicates (Anthropic ↔ Anthropic_dup2, 심리적 안전감 ↔ 심리적 안전감_dup2, etc.) | 0.84 – 0.95 |
| 심리적 안전감 ↔ Psychological Safety (Korean/English, same concept) | about 0.77 |
| Two completely unrelated notes | 0.35 – 0.46 |
The numbers split clearly into three bands. Real duplicates within the same language were 0.84 or higher; completely unrelated notes were 0.46 or lower. In between, the same-concept pair that differed only in language sat clearly lower, at about 0.77. Definitely higher than unrelated pairs, but noticeably lower than same-language duplicates: an “ambiguous middle zone.”
The mechanism already existed; only the threshold was set wrong — embedding similarity threshold
Here another interesting fact came out. The mechanism to solve this was actually already in the code. A verification pipeline that finds merge candidates by embedding similarity even among notes with different titles, then asks the LLM once more “are these two really the same concept?” before confirming, was already wired into the real refactor path. But the threshold that filters those candidates was set at 0.80, a value calibrated only on same-language duplicates. The measured Korean/English pair (0.77) was below that threshold, so it was filtered out before even becoming a “candidate,” never getting a chance to be asked about.
I also checked the code that actually performs the merge. Merging three or more notes at once (for example, “심리적 안전감,” “심리적 안전감_dup2,” and “Psychological Safety” together) was already fully supported. The only thing blocking it was whether those three ever met each other as “the same candidates.”
Fixed and released in about 20 minutes
I lowered the threshold from 0.80 to 0.70: 0.07 below the measured Korean/English pair (0.77), and a full 0.24 above the upper bound of unrelated pairs (0.46). This threshold doesn’t make the final call; it only decides “how widely to cast the net for candidates to ask the LLM about,” so lowering it carried no risk of more false positives (wrong merges). Final accuracy is still the job of the LLM verification step after it. The only cost was a few more verification API calls. About 20 minutes after recording the measurement, I made the fix, ran the tests, and released it.
This finding came from a completely different branch from the investigation I’d set out to do (chunk size vs. reasoning effort), but it ended up proving one real weakness of this vault with actual data and fixing it on the spot. And while verifying this fix, I would find one more corner where the same language gap still remained.
Continued in Part 6.