Anyone working today on multi-agent systems and autonomous AIs will eventually hit the same dead end: either you feed every trivial user query through a Large Language Model (LLM)—which generates noticeable latencies, blocks valuable VRAM, and inevitably leads to hallucinations—or you build static Regex and if/else walls in front of it that collapse at the first unexpected punctuation mark.
In the development of L.U.N.A. Prime, we landed precisely on this breaking point when managing large document corpora—and we found a mathematical solution demonstrating how true autonomy functions beyond prompt wrappers.
1. The Trap of Static Stopword Lists
The classic reflex of many developers when pre-filtering texts is the notorious Stopword List: You hardcode a list of words such as ["der", "die", "das", "und", "haben", "gelesen", "buch"] into the code to filter out irrelevant words before a search begins.
This sounds pragmatic, but in reality, it is digital dogmatism. Language is dynamic and context-dependent.
The Practical Shock
For us, this led directly to a parsing error. When asked:
"How many books have you already read and in the system?"
L.U.N.A. should actually query the library inventory in the SQLite database. Instead, the full-text search was triggered. Why? The word "gelesen" (read) was not on the list of filtered filler words. The algorithm treated "gelesen" as a highly specific search term, searched the database for it, and consequently outputted a text excerpt from a single chapter—where the word happened to appear in the running text.
One gets caught in an endless maintenance spiral: If you add "read" to the stopword list, the query works, but if a user later searches for a book review on the topic of "read works", the system filters out the term. Static lists don't heal symptoms; they just shift them.
2. Information Theory Instead of Probability Dice
To make L.U.N.A. independent of such rigid crutches, we have completely banished the stopword lists from the code and replaced them with dynamic information theory (Shannon Entropy & TF-IDF / BM25).
The principle is simple, but radical: Words do not have an absolute value, but only a relative information density within a corpus.
- Words that appear in almost every chapter or text segment (high Document Frequency, DF), contribute statistically minimal specificity—they are structural noise.
- Terms that appear sporadically and rarely ("The Stranger", "Light Column", "Mephisto"), have a high IDF Score (Inverse Document Frequency) and carry the actual semantic load.
Autonomous Evolution
During every ingestion process—whether a novel, a textbook, or source code—L.U.N.A. Prime automatically recalculates the information density of all terms in the mathematical memory core:
Formula for Information Weighting:
IDF(t) = ln((N + 1) / (DF(t) + 1)) + 1
As soon as a word appears in more than 40% of all segments, the system deterministically classifies it as temporary noise for free-text searches. No one needs to write manual rules anymore. The system learns what is important from the texts it has read itself.
3. The Multi-Document Stress Test (N > 1)
A corpus consisting of only one text merely proves the internal word density. The true stress test for a knowledge-based kernel only emerges in multi-document operation, when heterogeneous works meet and the IDF matrix must be rebalanced across documents.
Transparency Note & Methodology: As the initial test corpus, my published Dark Fantasy novel “When the Light Vanished” (18 chapters, ~11,500 words) was used to validate deterministic entities and plot arcs within a structurally known, complete work. In the next step, L.U.N.A. independently incorporated an external classic (Goethe's Faust I, ~34,000 words) to demonstrate selective cross-document routing and inter-book entropy live.
The result in the production dashboard:
21:40:42 > How many books have you read and stored in the system?
21:40:42 📚 L.U.N.A. Prime Archive and Library Inventory
(Deterministic SQLite Query from Vault/document_corpus.db | 0% Inference)
• Registered Works / Documents: 2
• Total Text Volume Captured: 45,616 Words
Stored Documents:
1. 📖 Faust - The Tragedy Part One Goethe.txt [TEXT]
• Sections/Chapters: 1 | Words: 34,117 | Captured: 07.10.2026 21:40
2. 📖 WHEN THE LIGHT VANISHED - CorrectionTemplate.txt [TEXT]
• Sections/Chapters: 18 | Words: 11,499 | Captured: 06.10.2026 22:45
The Metrics
- 0 Millisecond Inference Latency: The query was directed neither to a local language model nor to a cloud API. The deterministic parser recognized the relational information pattern and delivered the SQLite aggregation values without delay .
- 0% Hallucination: No fabricated numbers, no invention of third-party works—exact facts at the byte level.
- Resource Efficiency: The vGPU inference dispatcher remained completely untouched. VRAM and computational power remain free for genuine generative tasks.
Conclusion: True Autonomy Emerges Beyond the LLM
Autonomy is not achieved merely by feeding a language model with more parameters or presenting it with gigantic system prompts. True autonomy arises from a deterministic, neuro-symbolic foundation.
Compact models are entirely sufficient for formulating ideas. However, the logic, memory, and understanding of the corpus structure must reside within the mathematical kernel . L.U.N.A. Prime proves anew every day: Whoever mathematically understands language does not have to guess it.