As of: 2026-10-17 22:10 CEST.
Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/
Canonical page: https://postcutoff.com/p/2026-10-16-eliebakouch-reflection-beam-architecture/

# Elie Bakouch: Reflection Beam architecture details (sliding window 512, 5:1, gated attention, no shared experts)

elie (@eliebakouch), X, 2026-10-16.
Source: https://x.com/eliebakouch/status/2111117557516120304

## Why it matters

Most-viewed technical read (~30k views) of Beam's released weights.

## Summary

Posted Fri Oct 16 15:30:23 +0000 2026 by elie (@eliebakouch); 30,036 views, 303 likes, 21 reposts when read via api.fxtwitter.com on 2026-10-17. Sliding-window attention with window 512 and a 5:1 local:global pattern (same as Thinking Machines' Inkling); global-attention scaling like Llama 4; residual depth scaling; gated attention, QK-norm, sandwich norm, logit softcapping; no shared experts. Links huggingface.co/reflection/Beam-1.0-501B-A23B.

## Archived text

> reflection beam oss architecture details
> 
> IT'S SLIDING WINDOW AGAIN, same window size and pattern as inkling (window 512 and 5:1, meta is 2k and 3:1). they use a similar global attention scaling as llama4 (https://arxiv.org/abs/2501.19399) which still makes it position dependent even if they don't use RoPE
> 
> they use a residual depth scaling, quite similar to muse open but "fixed", tho it's not really fixed because you have a learned norm nearby. you can actually plot both the effective residual contributions and you see that it's not super different from muse
> 
> they use gated attention (with a per head bias for the full attention layer), QK Norm, Sandwich norm, similar softcapping before the LM head (like gemma and also muse), no shared experts
> 
> the shape is pretty similar to other american oss open weight models (but tbf, mimo v2 flash and stepfun 3.5 and 5 also have this shape, it's not only us labs 👀)
> 
> model weight: https://huggingface.co/reflection/Beam-1.0-501B-A23B
> 
> > Quoting @reflection_ai: Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
> > 
> > We're releasing:
> > - Weights, tokenizer, and reference implementation
> > - BF16, FP8 and NVFP4 weights and a DFlash2 drafter
> > - The open source harness we built and trained Beam on: Mirror CLI
> > 
> > Weights and model card on Hugging Face: https://huggingface.co/reflection

Archived 2026-10-17 via fxtwitter (unofficial).
Counts: 30,205 views, 303 likes, 21 reposts, 11 replies (at fetch time)
Media: https://pbs.twimg.com/media/HUwyBmcW8AA4M12.jpg?name=orig

## Cited in

- 2026-10-05: [Reflection AI unveils Beam, a 501B-parameter Apache-2.0 open-weight MoE](https://postcutoff.com/e/2026-10-05-reflection-beam-501b-open-weight/)