the sentence is solved
ask any modern AI for a sentence and you'll get something clean. fluid. maybe even beautiful. ask it for 2,000 words and watch it forget what it said three paragraphs ago.
this is not a prompt problem. it is not a scale problem. it is an architecture problem. and until you understand the shape of it, you will keep asking the model to do something it was never built to do.
next-token prediction is a local objective
transformer language models achieve near-perfect word-level prediction while completely missing higher-level transition structures [33]. the training objective ... next-token prediction ... rewards making the next sentence sound natural. it does not reward maintaining a document-level theme [34].
increasing model size improves perplexity. it does not improve latent coherence [33]. you cannot prompt your way past an objective function that was never asked to plan.
the sentence is solved because a sentence is a local problem. the paragraph is an architecture problem because a paragraph is a structural one. and no amount of parameter scaling fixes a structural deficit.
the context window mirage
you probably think a bigger context window is the answer. it feels like it should be. 200K tokens should be enough to hold a novella.
it is not.
LongGenBench shows models ace long-context understanding benchmarks but fail long-context generation benchmarks as output grows [14]. understanding is not the same as producing. a model can read 100,000 tokens. it cannot write 100,000 tokens without drifting.
the mechanism is ugly. context compression, positional under-training, encoding attenuation, softmax crowding ... all of these shrink the effective context far below the nominal size [8]. even 200K-token windows suffer information loss, entity tracking failure, and coherence degradation on extended narratives [13].
here is the number that should keep you up: over 96% of cumulative attention weight in Llama-3 concentrates on just 1,000 of 100,000 tokens [38]. the model is not reading its own context. it is reading a sliver of it. the rest is noise it learned to ignore.
the 2,000-word wall
there is a measurable collapse point. IS-CoT identifies a phenomenon called "Length Collapse" ... quality and length compliance degrade sharply past 2,000 words, with near-total failure at 16K-32K targets [17]. users independently report hitting a coherence wall around 1,000 words, with repetition and instruction drift [21].
the failure is not random. it has a shape.
contradictions cluster in the 40-60% range of narratives, while facts are established early at 15-30% [29]. hallucinations concentrate at the end of long outputs, a pattern researchers call "Hallucinate at the Last" [11]. the model starts confident, stays coherent for a while, then begins contradicting itself and fabricating details as it runs out of structural scaffolding.
if you have ever watched an AI write a great intro, a decent middle, and a conclusion that contradicts the opening ... you have watched this exact pattern.
scale is not the variable. structure is.
here is where it gets interesting. an 8B model with dynamic planning beats a 671B model without it [17].
read that again. an 8-billion parameter model outperforms DeepSeek-V3.2-671B and Gemini-2.5-Flash [17]. not because it is smarter. because it has a structure the larger model lacks.
every model that closes the coherence gap shares one thing: it plans, and it plans dynamically. not a fixed outline generated once and abandoned. recursive decomposition that adapts mid-generation.
- WriteHERE uses heterogeneous recursive planning ... decomposing into retrieval, reasoning, and composition subtasks dynamically. its gains grow with output length ... bigger advantage at 8K words than 2K [9].
- DOME uses dynamic hierarchical outlining with a temporal knowledge graph memory. it reduces contextual conflicts by 87.61% [26].
- CogWriter runs planning, monitoring, and reviewing agents on a Qwen-14B backbone. it beats GPT-4o by 22% on instruction accuracy [37].
- IS-Writer-8B uses dynamic interleaved planning. it outperforms DeepSeek-V3.2-671B and Gemini-2.5-Flash [17].
the pattern is unmistakable. a 14B model with structure beats a frontier model without it. an 8B model with planning beats a 671B model without it. the industry is spending billions chasing scale when the actual lever is architecture.
errors are not uniform. they are surgical.
one more finding that changes how you think about this. errors are not spread evenly across a generated document. they concentrate on 5-10% of tokens that represent critical decision junctions [38]. non-key tokens become more predictable with context, with error rates approaching zero. error-bearing segments show 12-19% higher token-level entropy than the whole text [29].
coherence failure is not a diffuse fog. it is a series of specific wrong turns at specific moments.
you do not need to fix every token. you need to catch the key ones.
what this means for LUNARI
most AI writing tools are wrappers. they take a single model, send one long prompt, and hope for the best. when the output drifts past 1,000 words, they blame the model. when users complain about repetition and contradiction, they suggest switching to a larger context model [21]. it helps. it does not solve.
LUNARI's seven-agent crew is already the right shape. seven agents are not seven calls to the same prompt. they are a structural decomposition ... planning, research, writing, governance, memory, composition, execution. each one owns a layer the single-pass model has to hold all at once and inevitably drops.
but having the right shape is not the same as implementing the right patterns. the research points at four specific build directions:
- dynamic outline regeneration. RAVEN should not plan once and hand off. it should re-plan between sections, adapting the outline to what has actually been written. fixed outlines fail because the model forgets the initial plan during extended generation [17]. WriteHERE, IS-CoT, and DOME all converge on the same fix ... fuse planning and writing so the plan breathes [9][17][26].
- reflection-gated section passing. after NOVA writes a section, a separate pass checks it against the plan, the entity registry, and the prior section before the next section generates. if it fails, regenerate. do not pass forward. IS-CoT's ablation is the proof ... removing the reflection module cuts the gain in half. removing interleaved planning nearly eliminates improvement entirely [17].
- cross-agent entity memory. a shared store that tracks entities, claims, and decisions across all agents, injected into every call. DOME's temporal knowledge graph stores subject-action-object triples and retrieves them during generation [26]. right now, LUNARI agents share context through prompts. that is the context mirage. an explicit memory layer is the difference between a crew that remembers what it said three sections ago and one that hallucinates a contradiction in the 40-60% zone [29].
- entropy-spawned verification. log token entropy during generation. when it spikes ... 12-19% higher at error-bearing segments [29] ... flag that moment for review before continuing. cheaper than reviewing everything. more targeted than fixed checkpoints. the Anthropic API supports logprobs. the detection logic is straightforward.
none of these require a bigger model. none of them require more parameters. they require structure. and structure is what a crew gives you that a single prompt never will.
the takeaway
AI mastered the sentence because a sentence is a local optimization. it never learned the paragraph because a paragraph is a structure. you cannot scale your way past an architecture problem. you can only rebuild the architecture.
an 8B model with dynamic planning beats a 671B model without it. a 14B model with monitoring agents beats GPT-4o. the evidence is not ambiguous. structure beats scale.
the future of AI writing is not a bigger model writing a longer document in one pass. it is a crew of smaller, specialized agents that plan, write, reflect, remember, and verify ... each one holding a piece of the structure that no single model can hold alone.
that is what we are building. not because it is fashionable. because the research says it works, and we have been reading the research.