Ward Plunet engagement report

Best tweets

• Jun 25, 202625x their median

  We recently obtained the highest-resolution 3D images of the human brain ever taken from outside the skull. This is the first look. Introducing Aleph, a research lab building brain interfaces for the telepathic future. (1/n) https://t.co/wW9xag34zy

  6.7K9902233481.6M viewsView on X
• May 27, 20268.5x their median

  Introducing DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation https://t.co/c9AvsRKybj What if we didn’t have to hold an entire neural network in memory to train it? Standard neural net training optimizes all parameters jointly. As a result, the memory required during training grows linearly with the depth of the network. In our #ICLR2026 paper, we propose DiffusionBlocks, a principled framework to train networks one block at a time, drastically reducing memory requirements while matching end-to-end performance. With DiffusionBlocks, we split the network into blocks and train them one at a time, so you only need memory for a single block. How? We explicitly assign each block a role: to move the representation a little closer to the target than the block before it did. That role turns out to be precisely what a diffusion model does, step by step. Each block only needs to optimize its own objective and can be trained independently. We validated this across five different architectures: • ViT • DiT • Masked diffusion • Autoregressive transformers • Recurrent-depth transformers In each case, performance is competitive with end-to-end training while using a fraction of the memory. This perspective also extends naturally to recurrent-depth (Looped) transformers, which apply the same network iteratively and normally require expensive backpropagation through time (BPTT). Viewed through DiffusionBlocks, we can replace those multiple iterations with a single forward pass during training. Read our paper and code, to learn more. Paper: https://t.co/CRj96VGYQn GitHub: https://t.co/eNW0K9Xh8E 🐟

  2.3K35256100883K viewsView on X
• Jun 25, 20265.0x their median

  We removed an LM's ability to speak German by fine-tuning on only 4 German tokens. As part of a 1-day hackathon with our product Silico, we removed a 67M-parameter language model's ability to predict German text, by tuning only a scalar factor on one subcomponent of the weights. (1/6)

  1.4K1295649307K viewsView on X
• Jul 13, 20264.5x their median

  Moving from conventional ML to continual learning requires revisiting even the fundamental concepts such as “test”/“train” time. LLMs Need Sleep and Dreaming! We introduce a phase, where the model consolidates its fragile short-term memories into stable long-term memories, and then dreams to recursively self-improve over time. For memory consolidation, we introduce a new form of distillation, called Knowledge Seeding (KS), where a small model(s) distills its knowledge to a larger model. Our experiments on continual learning and reasoning tasks show that this new phase can help the model to perform better and relatively better mitigates catastrophic forgetting.

  1.2K1874830123K viewsView on X
• Jul 9, 20263.0x their median

  This week in @TheEconomist I wrote about the incredible progress that we have been making against dementia. Though total numbers are rising (because we are living longer), the actual rate at every age has been coming down sharply for decades. A short 🧵 1/ https://t.co/xMv0KLMl3A

  8121392125247K viewsView on X
• May 27, 20261.7x their median

  New paper: We present a "Unified Neural Scaling Law" functional form that accurately models & extrapolates the multivariate scaling behaviors of artificial neural networks as the variables listed in this attached video are varied. (1/N)

  4836312350K viewsView on X
• Sep 16, 2026

  looped transformer unlocks a new scaling dimension - recurrent depth; i think it received less love (other than the rumors that frontier ai labs use it) than it deserves, here are few notes / observations fundamentally the method lets the model think deeper without increasing either its parameter count or its verbalized chain of thought length (the other 2 important scaling dimensions) i find the constrast with moe interesting: > looped transformer: keep params const, increase total FLOPs (more computation = reasoning) > moe: keep FLOPs const (through active params), increase total params (more param capacity = memory/knowledge) the architectural modification is straight forward, instead of: L1 → L2 → … → L_n (where n = num layers) we do: P → R → R → … → R → C where: * P - prelude, few transformer layers that we apply only once at the beginning (i.e. L1 -> L2 -> ... -> L_p) * R - recurring core, normally this would constitute the middle layers of the transformer, but in constrast with regular transformer R is repeated/looped an arbitrary number of times (r) * C - postlude or "coda", similar to P just at the exit so the computation proceeds as following: 1. e = P(x) 2. apply s_i = R(e, s_i-1) r times, where s0 ~ N(0, sigma^2 * I), and s and e are concat and mapped/"projected" back to hidden size 3. apply C on the final s (state) (the particularly important design decision is that e is re-injected at every recurrence) now the beautiful part is that more r at inference improves perf, and harder / more context-heavy problems benefit from more latent compute. e.g. on ARC-C they demonstrate: * 0-shot -> perf saturates around 8-12 iterations * 1-shot -> 20 iters * few shot (25-50 examples) -> 32 iterations (the recurrent reasoning is like a learned optimization in the latent space during inference; grad-descent during inference unlocked?) -- the above generalization is downstream of how we train the model, r is sampled from heavy tailed log-normal-Poisson distribution during training thus the model is trained across randomized recurrent depths (they use truncated BPTT - backprop through depth in this case - to save on memory) -- few systems notes: 1. they synchronize one r per microbatch across workers, otherwise GPUs with small r would be idle waiting for workers that happened to sample large r 2. the model has a natural specdec capability -> draft model == low recurrence count (r), verifier model == high recurrence count. states computed during drafting can be reused when verifying! e.g. s0→s1→s2→s3 from draft (r=4) can be fed into verifier => s3→s4→…→s11 (r=12) 3. aggresive KV cache sharing is possible because every recurrent iteration uses the same weights, they show that caches from different recurrence depths can be aggressively reused zero-shot 4. latents converge quicker for "easier" tokens, early looping exit is thus possible per token, depending on how "hard" the token is 5. takes up less memory in training/inference (tnx to truncated BPTT and KV sharing respectively) few mechinterp notes: 1. might be harder to do J-space/J-lens type of analysis (the deeper the model the harder it is to trust the jacobians that map intermediate activations to the output); in general this leads to more "neauralese" and makes it harder to solely use cot to monitor the models (i suspect that's what we've been seeing from oai/ant recently) 2. they find evidence for interesting structures in latent space, such as the tendency of the latent representation to follow "orbitals" to compute arithmetic tasks and for reasoning in general (the shape rotators win again?) -- in general there is less need for long cot data, but it’s fully complementary one can and will do both in practice another example you can do great work at academia! soon we’ll have incredibly strong OSS models that can run locally, we’ve decoupled num of params from the perf of the model! intelligence will be abundant, aligned to you, and democratized :)

  2852317415K viewsView on X
• Jul 7, 2026

  A tiny 1-millimeter cube of human brain tissue can contain tens of thousands of neurons & other structures. Our new animation explores Neuroglancer data from @Google & @AllenInstitute, using visualization techniques to reveal the complexity packed into microscopic brain tissue 🧠 https://t.co/1NoCGHIVuZ

  136412311K viewsView on X
• Jul 10, 2026

  1/6 New preprint! 🧠🔬 We gave Vision-Language Models the AI equivalent of anhedonia. https://t.co/8COD07ts9X

  50131310K viewsView on X

Ranked by total interactions across everything we have tracked for this account, which is a longer history than the 30-day window the rates above use. The multiple compares each post to this account's own median.

Keep going