Tag: Testing
All the articles with the tag "Testing".
-
I cut LLM note ingest time by 44%, and proved the results were the same by replaying responses
An Obsidian plugin's ingest went from 554 to 313 seconds once LLM dedup stopped waiting in line. Replaying saved LLM responses proved the notes were unchanged.
-
I had an LLM fix the notes its judge flagged, and a loss the judge couldn't see got through
An Obsidian plugin had an LLM fix dedup edits its judge flagged, not roll them back. Over 96 runs it mostly restored the original, and an unseen loss passed.
-
An LLM judge blocked every broken note, but half of the dedup got rolled back
An LLM judge rechecked notes edited by LLM dedup and rolled back flagged ones. No broken note survived five test runs, but 13 of 28 rewrites were undone.
-
When an LLM removed duplicates from my notes, the remaining sentences lost what they pointed to
Removing note duplicates with an LLM left 'this technology stack' and 'also' pointing at nothing. I stopped adding code rules and had an LLM judge each edit.
-
My LLM note-merge test passed, but the fixed code never ran once
I fixed an LLM note-merge bug in my Obsidian plugin and reran it on real notes. It passed, but the fixed code never ran until I restored the pre-merge state.
-
I deleted the wrong example from my LLM note-merge rule, but the rule stayed
My Obsidian plugin's LLM rule for merging duplicate notes blocked even same-product notes. I traced it, tested real merges, and adopted Karpathy's LLM Wiki.
-
In LLM summaries, a claim made once disappeared and one made twice survived
Chasing an LLM summary that dropped a source's causal claim, I ruled out grammar and counterintuitiveness with controls. A claim stated once vanished.
-
I found a rule missing from the LLM merge prompt, and filling it in changed nothing
Chasing an LLM note merge that dropped one source's content, I found the merge prompt lacked an information-density rule. Adding it changed nothing.
-
The code I carefully built to catch LLM language mixing turned out to have done nothing
I built code to detect and fix Korean sentences leaking into LLM-written English notes. It fixed zero. The real cause was a prompt missing a language rule.
-
I tested my Obsidian plugin by remote-controlling Obsidian on another PC
To test an Obsidian plugin in the real app before each release, I drove Obsidian on another PC over the Chrome DevTools Protocol (CDP), deploy to button clicks.