Treebeard

Treebeard research notes / Qwen3.6-35B-A3B

MoE routing notes

Qwen picks 8 of 256 experts per token. Treebeard keeps the choice and reorders the work for the GPU.

256Routed experts
Top 8Chosen per token
+1Shared expert
40Decoder layers
2048Hidden width

The flow

Top half: Qwen's computation. Bottom half: Treebeard's scheduling. Nothing above the line changes.

Detailed flowchart of top-8 Qwen mixture-of-experts routing followed by Treebeard Intel SYCL and NVIDIA CUDA scheduling
Open legacy PNG Open editable SVG Read technical notes

One token

Two parallel branches consume the same normalized token state, then merge before the residual connection.

Routed branch

  1. 1Project the 2,048-wide token state into 256 router logits.
  2. 2Apply softmax across every routed expert to get probabilities.
  3. 3Take the eight highest-probability expert IDs with TopK.
  4. 4Renormalize those eight weights so the selected weights sum to one.
  5. 5Run only the selected experts' 2,048 → 512 → 2,048 SwiGLU MLPs.
  6. 6Weight and sum the eight expert outputs into y_routed.

Shared branch

  1. 1Compute one sigmoid gate from the same normalized token state.
  2. 2Always run the single shared SwiGLU expert. It sits outside the top-8 race.
  3. 3Scale that output with the sigmoid gate to produce y_shared.
  4. 4Add y_routed + y_shared, then add the FFN residual for the next block state.

What Treebeard does

The selected IDs and weights are fixed by the time the backend receives the operation. Treebeard optimizes how those token/expert pairs reach the matrix kernels.

Intel SYCL

Bucket and reuse

At four or more tokens, the device groups selected pairs by expert and handles up to four tokens per weight pass. Smaller batches use a fused per-pair device route. Shape-aware workgroups use one subgroup for 512-row gate/up projections and four for the 2,048-row down projection.

NVIDIA CUDA

Stay on the quantized fast path

Selected-expert projections remain in device-routed MMVQ/MMQ kernels. On Blackwell with Q8_0 and twelve concurrent token columns, Treebeard uses the direct 12-column MMVQ path. Those twelve columns are tokens—not twelve experts.

The rule

Every token runs exactly eight routed experts plus one shared branch. Treebeard never rescores, replaces, drops, or adds experts. It reschedules execution after Qwen has made the routing decision.