Skip to main content

Research

Papers & notes

Papers and technical notes from the lab. Where we can, we release weights and code alongside.

RSS feed

Featured

#efficient-reasoning

Tessera-27B-B-CJK: teaching a 27B agent to think less per turn

Tessera-27B-B-CJK is Qwen3.8-27B with a rank-32 LoRA trained on Tinker. It thinks in a compressed CJK shorthand that mixes English, CJK words and symbols, and we release it as an agentic model rather than a general reasoner. On 44 frozen states from 11 held-out coding-agent tasks it uses about 0.50-0.64x the base model's thinking tokens per agentic turn, from the typical turn to the token-weighted ratio. It made a valid tool call on all 88 of its draws, with no repetition loop in any of them, and on 6 long-context states with inputs of 101K-131K tokens it made a valid tool call 6 times out of 6. The safety smoke test raised no flag against the base. On hard math and competitive code it is worse than the base, and it loops more often there.

Read paper

2026

  2. #reinforcement-learning#rl-stability

    Per-rollout weighting in RL on Tessera-27B-B-CJK: a pre-registered ablation

    Our first RL run on the Tessera line lost log-probability on its own tokens and rambled and looped more. We suspected per-token weighting, which lets a rollout's weight grow with its length, so long failures weigh the most. In a pre-registered ablation from the released Tessera-27B-B-CJK, weighting each rollout by its advantage and not its length avoided that drift in both seeds we ran. In an exploratory check on 24 hard math and code problems from earlier evaluations, it got 83 of 144 rollouts right against 71 and used 0.78x the tokens per correct answer. The planned control contrast did not replicate, a blinded read could not confirm the held-out gain, and on 24 easier problems we had not used before, the efficiency gain did not appear.

  3. #compressed-reasoning#chain-of-thought

    Tessera-Preview-9B: Compressed Reasoning at 18x Fewer Tokens, and What It Costs

    Tessera-Preview-9B is a proof of concept: a 9B coding model fine-tuned on a 10K-example corpus to reason internally in a dense CJK notation minted by a compressor we own, while emitting ordinary code and tool calls. On the full LiveCodeBench set (1,055 problems, matched 16K greedy budget, paired harness) it reaches 34.9% against its base's 39.5% while spending a median 17.7x fewer output tokens per problem, and the two models fail in entirely different ways: the base almost never writes wrong code (95.9% of its completions pass) but thinks into the token ceiling on 58.8% of problems, while Tessera completes 78% of problems cheaply and errs by writing wrong code. The compressed channel is causally load-bearing (in an archived 50-problem ablation, forcing an empty think collapses accuracy to 4%), and the register itself accounts for only ~1.7x of the saving; the rest is a trained brevity curriculum. The measured costs are equally plain: -4.6 points of coding accuracy at matched greedy budget, a large instruction-following deficit on prompts far from the 100%-code training distribution, and none of the base's sampling headroom. The failure pattern tracks training-data coverage closely enough that the small, narrow corpus is likely a major cause, which is what the scaled successor now in preparation is designed to test.

  4. #titans#memoryNegative result

    Parametric Memory Cannot Replace Retrieval: Why Titans-Style Neural Memory Fails for Cross-File Symbol Resolution

    We integrate Titans-style parametric memory into Qwen 3.5-9B and evaluate on CrossCodeEval for cross-file code completion. Across four architecture iterations, parametric memory produces zero measurable improvement over a memory-ablated baseline (1.0% EM vs 1.0% EM, an underpowered comparison given the base model's 2-7/200 solve rate), while explicit context concatenation yields 2.5x higher exact match. Per-sample analysis points to a mechanism-task mismatch: inner-loop SGD on a low-dimensional MLP compresses lossily, but cross-file symbol resolution requires lossless recall of specific token sequences.

  7. #methodology#mechanismNegative result

    Brain-Embedded LLMs: A Research Arc in Methodology, Encoder Mechanisms, and Negative Results

    We injected CorticalNet brain states into Qwen3.5-27B as a universal LLM enhancer. Then we retracted three early wins under matched-pairs sampling, found that a 13MB static encoder beat transformer encoders inside our specific stack, and confirmed the brain pipeline actively hurts technical generation work. Follow-up experiments narrowed the claim twice: the technical harm is input-conditioning-specific (any input-conditioned prefix loses to OFF); the Mid recall benefit is prefix-tuning-generic (brain, keyword-extract, static soft prompts, and raw-encoder prefixes all tie each other and all beat OFF). A research arc in why methodology matters more than mechanism, and what survives after each methodology pass narrows the claim further.