Train models with JAX and Flax.
Language models, diffusion and JEPA, on one device or a mesh.
curl -LsSf https://dewml.dev/install.sh | sh Trained with Dew
176M hybrid DiT · Prompts and settings


















Latest tutorials and posts
- A racing drone that learns to fly from its cameraA new tutorial, A racing drone that flies from its camera, trains a first-person racing drone from scratch.
- Continual learning in contextA trained language model is frozen: whatever it reads, its weights stay where training left them.
- How Dew works, from a model to kernels on the GPUA new page, How Dew works, follows one training run through Dew, from the Flax model and data you write down to the kernels that run on the GPU, and checks each step against the code.
- World models: acting and imagining in one loopSuppose you have taught a program to balance a pole on a cart by letting it practise in a simulator, pushing the cart left and right millions of times until it stops dropping the pole.
Video
Wan 2.1 1.3B's released weights, sampled with Dew's JAX implementation · Prompts and settings
Training
A small decoder trains on Tiny Shakespeare on one A100, then writes one sample from each of the prompts ROMEO: and JULIET:.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 10.7M parameters · on 1 × NVIDIA A100-SXM4-40GB · batch 64 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1000/1000 100% 33.5 step/s 0:00:46 │ │ data 64,192 rows · 0 rejected │ │ │ │ train │ │ loss 1.201 █▅▄▃▃▂▂▁▂▁ ↓ 40% │ │ ce 1.201 █▅▄▃▃▂▂▁▂▁ ↓ 40% │ │ perplexity 3.322 █▅▄▂▃▁▂▁▂▁ ↓ 56% │ │ token_accuracy 62.4% ▁▄▅▇▆█▇███ ↑ 21.9 pt │ │ data_wait_ms 0.01949 ▄▅▅█▇▂▅▁▂▅ ↑ 3.4% │ │ grad_norm 0.2868 █▄▂▂▂▁▁▁▁▁ ↓ 72% │ │ cache_misses 3.000 ▁█████████ ↑ 50% │ │ compiles 1.000 cache_hits 9.000 peak_memory_gib 4.016 │ │ throughput │ │ step_time_ms 29.77 █▅▁▂▂▅▂▂▂▂ ↓ 2.6% │ │ samples_per_sec 2,150 ▁▄█▇▇▄▇▇▇▇ ↑ 2.6% │ │ tokens 16,076 ▄▁▅▃█▄▇▄█▄ → │ │ tokens_per_s 539,918 ▁▃█▇█▄█▇█▇ ↑ 2.6% │ │ mfu 11.7% ▁▃█▇█▄█▇█▇ ↑ 0.3 pt │ │ hlo_flops_util ≥ 12.6% ▁▄█▇▇▄▇▇▇▇ ↑ 0.3 pt │ │ hfu ≥ 12.6% ▁▄█▇▇▄▇▇▇▇ ↑ 0.3 pt │ │ uncounted kernels 50.00 │ │ optimizer │ │ learning_rate 1.000e-04 ██▇▇▆▅▄▃▂▁ ↓ 90% │ │ accepted 100.0% │ │ │ │ val · step 1000 │ │ perplexity 3.567 █▁ ↓ 6.6% │ │ loss 1.272 █▁ ↓ 5.1% │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────────────────────────╮ │ trained 1000 steps in 0:00:50 │ │ first step after 19.90 s │ │ then 33.5 step/s │ │ goodput 59.9% of the wall time in steps │ │ final loss 1.201 │ │ val at 1000 perplexity 3.567 ↓ 6.6% █▁ loss 1.272 ↓ 5.1% █▁ │ ╰────────────────────────────────────────────────────────────────╯ ROMEO: And therefore, he shall be the whole of the sea, Of her that makes the warm of such a fair court, As as the proclaims to my brother's love, Or make my executioner than the common That he be so much a JULIET: Then she is God's name, come, that say the people Is the requirest of it.
Language modeling
The objective creates the model's variables, computes the loss and runs evaluation. Trainer.fit steps the optimizer, logs progress and saves checkpoints, and every model on this page trains through that one call.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 18,560 parameters · on 1 × cpu · batch 8 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100% 0:00:03 │ │ data 48 rows · 0 rejected │ │ │ │ train │ │ loss 5.874 ▁ │ │ ce 5.874 ▁ │ │ perplexity 355.7 ▁ │ │ token_accuracy 0.4% ▁ │ │ data_wait_ms 0.02289 ▁ │ │ grad_norm 1.819 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 83.48 ▁ │ │ samples_per_sec 95.83 ▁ │ │ tokens 512.0 ▁ │ │ tokens_per_s 6,133 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────╮ │ trained 3 steps in 0:00:04 │ │ first step after 4.23 s │ │ then 29.3 step/s │ │ goodput 1.6% of the wall time in steps │ │ final loss 5.874 │ ╰────────────────────────────────────────────╯
Representation learning
A JEPA encoder sees part of an image and learns to predict the representations of the patches it cannot see.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · JepaEncoder ────────────────────────────────────────────────────────────────────────╮ │ 18,736 parameters · on 1 × cpu · batch 64 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100% 0:00:09 │ │ data 384 rows · 0 rejected │ │ │ │ train │ │ loss 1.489 ▁ │ │ repr_cov_offdiag 0.1102 ▁ │ │ repr_std 0.3723 ▁ │ │ data_wait_ms 0.02631 ▁ │ │ grad_norm 2.365 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 207.3 ▁ │ │ samples_per_sec 308.8 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ JepaEncoder ──────────────────────────╮ │ trained 3 steps in 0:00:13 │ │ first step after 12.66 s │ │ then 17.9 step/s │ │ goodput 0.9% of the wall time in steps │ │ final loss 1.489 │ ╰────────────────────────────────────────────╯
Fine-tuning
Load a Hugging Face checkpoint as a Flax model, fine-tune it with the same trainer, and save it back in the Hugging Face layout with its original tensor names. The first cell fine-tunes SmolLM2-135M on Tiny Shakespeare and compares its text before and after.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 134.5M parameters · on 1 × cpu · batch 1 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20/20 100% 1.4 step/s 0:00:35 │ │ data 23 rows · 0 rejected │ │ │ │ train │ │ loss 3.218 █▅▆▁ ↓ 27% │ │ ce 3.218 █▅▆▁ ↓ 27% │ │ perplexity 24.97 █▄▆▁ ↓ 69% │ │ token_accuracy 40.6% ▁▄▇█ ↑ 14.1 pt │ │ data_wait_ms 1.059 ▁▁██ ↑ 35× │ │ grad_norm 9.280 █▁▂▁ ↓ 17% │ │ cache_misses 12.00 ▁███ ↑ 9.1% │ │ compiles 1.000 cache_hits 0 │ │ throughput │ │ step_time_ms 767.5 █▂▁▂ ↓ 29% │ │ samples_per_sec 1.303 ▁▇█▇ ↑ 40% │ │ tokens 64.00 ██▁█ → │ │ tokens_per_s 83.39 ▁█▇█ ↑ 40% │ │ optimizer │ │ accepted 100.0% learning_rate 1.000e-04 │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ─────────────────────╮ │ trained 20 steps in 0:00:37 │ │ first step after 22.22 s │ │ then 1.3 step/s │ │ goodput 39.5% of the wall time in steps │ │ final loss 3.218 │ ╰─────────────────────────────────────────────╯ Before: I'm glad you're interested in learning more about the history of the Eiffel Tower. The tower was built in 1889 as a After: BUCKINGHAM: Hastings: BUCINGHAM: BUCINGHAM:
A larger model: Qwen3-0.6B, one step, then saved in its original layout.
This cell needs more memory than a shared host has, so Run in Colab opens it on a GPU.
Paris. The capital of France is also the capital of the ╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 596.0M parameters · on 1 × cpu · batch 1 · bfloat16 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 100% 0:00:39 │ │ data 4 rows · 0 rejected │ │ │ │ train │ │ loss 3.576 ▁ │ │ ce 3.576 ▁ │ │ perplexity 35.73 ▁ │ │ token_accuracy 45.3% ▁ │ │ data_wait_ms 0 ▁ │ │ grad_norm 65.00 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 13.00 ▁ │ │ throughput │ │ step_time_ms 9,160 ▁ │ │ samples_per_sec 0.1092 ▁ │ │ tokens 64.00 ▁ │ │ tokens_per_s 6.987 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 1.000e-05 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────╮ │ trained 1 steps in 0:00:41 │ │ first step after 40.70 s │ │ goodput 0.1% of the wall time in steps │ │ final loss 3.576 │ ╰────────────────────────────────────────────╯
Supported families: Qwen, Llama, Gemma, Mistral, DeepSeek, Mamba and multimodal decoders.
Inference
A TextGeneration task holds the model, its weights and the sampling settings. Run the cell to generate on a shared CPU host.
Paris. Paris is the largest city in France and the capital of the French department of the Espace de Paris.
Recorded on a 4-vCPU Cloudflare container with Dew aee614a on 2026-10-08, in 40 s. Run the cell on a live CPU kernel.
Server runs many requests through one task in shared batches, and each request keeps its own random key and KV cache:
This cell needs more memory than a shared host has, so Run in Colab opens it on a GPU.
[' Paris. The capital of France is also the capital of the Republic of France. The capital of Fr ance is also the capital', ' Tokyo. The capital of the United States is Washington, D.C. The cap ital of France is Paris. The capital of']
Store weights in int8 or fp8:
This cell needs more memory than a shared host has, so Run in Colab opens it on a GPU.
int8 Paris. The capital of Italy is Rome. The capital of fp8 Paris. The capital of France is also the capital of the
The serving guide covers the paged KV cache, prefix sharing, constrained decoding, and the error each quantized format adds.
Decision models
A decision model answers typed questions about a state in one forward pass: each question names its options, and the answer is a probability distribution over exactly those options. Laya, a ModernBERT-large decision model, loads as a Dew model, and systemone takes and returns Jev's wire format.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
{
"department": {
"type": "choice",
"choice": "billing",
"probabilities": {
"billing": 0.9897,
"technical": 0.0103
},
"confidence": 0.9794
},
"urgency": {
"type": "score",
"score": 1.7091,
"legend": {
"0": "not urgent",
"1": "soon",
"2": "blocking"
},
"probabilities": {
"0": 0.0483,
"1": 0.1942,
"2": 0.7574
},
"confidence": 0.5636
},
"churn_risk": {
"type": "noul",
"noul": 0.0818
}
}
The decision models guide covers calibration, abstention and fine-tuning a decision head on any backbone.
Diffusion
The process sets how training adds noise and what the model learns to predict. At generation time a solver runs the process in reverse, and you can switch solvers without retraining.
Run sends the cell's Dew calls to a process on the host that already holds the model. The same cell runs as is with Dew installed.

Run opens an isolated Python context backed by the shared Dew model on a 4-vCPU CPU host.
Recorded on one A100 with Dew 7575f1a on 2026-10-22, in 93 s.
Kernel output
Or open it in Colab to run it on a GPU and change anything.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · SimpleDiT ──────────────────────────────────────────────────────────────────────────╮ │ 26,320 parameters · on 1 × cpu · batch 64 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100% 0:00:06 │ │ data 384 rows · 0 rejected │ │ │ │ train │ │ loss 0.6171 ▁ │ │ data_wait_ms 0.03363 ▁ │ │ grad_norm 0.1143 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 147.9 ▁ │ │ samples_per_sec 432.8 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ SimpleDiT ────────────────────────────╮ │ trained 3 steps in 0:00:08 │ │ first step after 7.89 s │ │ then 9.9 step/s │ │ goodput 2.5% of the wall time in steps │ │ final loss 0.6171 │ ╰────────────────────────────────────────────╯
A 176M-parameter text-to-image model at 256×171, trained with Dew. Weights: dewml/hybrid-dit-176m.



A style LoRA
Stable Diffusion 1.5 with a rank-16 LoRA on its attention projections, trained with Dew for 160 steps on 20 public-domain Hiroshige prints from the Library of Congress, on one A100. Each pair shares its prompt and seed; drag the slider to compare. Prompts, seeds and settings.


Seed 1042


Seed 2073


Seed 3104


Seed 4135


Seed 5166


Seed 6197
Distributed
To use more devices, give Trainer a mesh; the objective and data stay as they are. FSDP shards parameters and optimizer state, tensor parallelism splits each layer's weight matrices, and an expert axis puts an MoE's experts on different devices. The devices left over form the data-parallel axis.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 18,560 parameters · on 4 × cpu · mesh fsdp 2 × tensor 2, 0% of the parameters' bytes spl │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100% 0:00:03 │ │ data 48 rows · 0 rejected │ │ │ │ train │ │ loss 5.874 ▁ │ │ ce 5.874 ▁ │ │ perplexity 355.7 ▁ │ │ token_accuracy 0.4% ▁ │ │ data_wait_ms 0.03056 ▁ │ │ grad_norm 1.819 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 117.4 ▁ │ │ samples_per_sec 68.15 ▁ │ │ tokens 512.0 ▁ │ │ tokens_per_s 4,362 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────╮ │ trained 3 steps in 0:00:04 │ │ first step after 4.16 s │ │ then 15.0 step/s │ │ goodput 3.1% of the wall time in steps │ │ final loss 5.874 │ ╰────────────────────────────────────────────╯
Placement and sharding · Several hosts · Cloud TPU
Expert routing
With the experts split over devices, each MoE layer's router sends tokens to experts on other devices through all-to-all exchanges. This is the routing examples/moe_mesh.py recorded over 300 training steps.
training step 1 of 300
Post-training
GRPO samples a group of completions for each prompt and scores them with a reward function. The reward here is the fraction of adjacent tokens that count up by one, modulo 13.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 83,136 parameters · on 1 × cpu · batch 8 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 150/150 100% 27.5 step/s 0:00:12 │ │ data 1,224 rows · 0 rejected │ │ │ │ train │ │ loss 0.04121 █▅▆▃▄▇▅▃▅▅▅█▄▂▄▄█▃▂▃▂▁▃▄▃▃▅▅▆▅▅▅▂▅▅▁▅▆▆▃█▂▆ ↓ 7.0% │ │ actor/ppo_kl 4.220e-08 ▅▃█▃▄▄▆▅▂▆▅▇▄▆▃█▃▄▅▇▆▅▆▆▄▆▄▇▅▂▅▄▄▆▇▆▁▄▇▆▅▅▇ ↑ 12× │ │ kl 2.061 █▅▆▃▄▇▅▃▅▅▅█▄▂▄▄█▃▂▃▂▁▃▄▃▃▅▅▆▅▅▅▂▅▅▁▅▆▆▃█▂▆ ↓ 7.0% │ │ pg 3.353e-08 █▅▄▄▄▇▁▅▄▄▆█▆▄▄▃▅▄▅▄▆▄▆▄▄▁▅▄▄▂▅▄▄▄▄▄▃▄▄▄▃▄▆ ↓ 62% │ │ data_wait_ms 0.02828 ▃▂▃█▃▄██▃█▇▃▅▁▄▄▃▂▂▁▃▇▃▁▃▆█▃▄▁▁▁▃▂▂▃▁▄▂▃▄▃▃ ↓ 4.5% │ │ grad_norm 0.3026 ▇▆▂▂▂▆▆▆▆▂▇█▇▂▁▇▇▁▆▁▆▃█▁▂▆▇▁▁▆▆▁▁▁▁▁▇▂▁▂▇▃▇ ↓ 45% │ │ actor/pg_clipfrac 0 actor/pg_clipfrac_lower 0 mismatch/ess 1.000 mismatch/k3_kl 0 │ │ mismatch/kl 0 compiles 1.000 cache_hits 0 cache_misses 21.00 │ │ throughput │ │ step_time_ms 11.05 ▅▅▄▅▃█▆█▅▅▅▇▅▅▄▇▅▄▅▄▇▄▅▄▄▇▅▅▅▆▅▄▅▅▆▅▆▅▇▅▅▆▁ ↓ 63% │ │ samples_per_sec 2,896 ▄▄▅▄▆▁▃▁▄▄▄▂▄▄▅▂▄▅▄▅▂▅▄▅▅▂▄▄▄▃▄▅▄▄▃▄▃▄▂▄▄▃█ ↑ 169% │ │ rollout_seconds 0.03291 ▆▂▂▁▂▂▂▅█▃▃▃▆▂▃▁▆▂▁▂▁▆▁▂▁▁▅▁▂▂▅▂▁▂▂▄▃▄▃▆▂▃▃ ↓ 34% │ │ optimizer │ │ accepted 100.0% learning_rate 0.001000 │ │ rollout │ │ latency/max 0.01380 █▂▂▁▂▄▂▄▇▂▃▃▇▃▃▂▃▃▂▃▂█▂▃▂▂▃▂▂▂▇▂▁▃▂▁▃▄▃▇▂▃▃ ↓ 58% │ │ latency/p50 0.01340 █▂▂▁▂▃▂▄▇▂▃▃▇▃▃▂▃▃▂▃▂█▂▂▂▂▃▂▂▂▇▂▁▃▂▂▃▄▃▇▂▃▃ ↓ 59% │ │ latency/p90 0.01375 █▂▂▁▂▃▂▄▇▂▃▃▇▃▃▂▃▃▂▃▂█▂▃▂▂▃▂▂▂▇▂▁▃▂▁▃▄▃▇▂▃▃ ↓ 58% │ │ latency/p99 0.01379 █▂▂▁▂▄▂▄▇▂▃▃▇▃▃▂▃▃▂▃▂█▂▃▂▂▃▂▂▂▇▂▁▃▂▁▃▄▃▇▂▃▃ ↓ 58% │ │ reward/mean 0.8527 ▄▆▃▃▇▇▃▆▅▅▅▄▅▅▅▃▃▅▆▇▁▅▃▇▃▃▅▃██▄▅▇▅▅▅▄█▃▂▇▅█ ↑ 5.5% │ │ reward/observed 0.8527 ▄▆▃▃▇▇▃▆▅▅▅▄▅▅▅▃▃▅▆▇▁▅▃▇▃▃▅▃██▄▅▇▅▅▅▄█▃▂▇▅█ ↑ 5.5% │ │ waited_seconds 0.01533 █▃▂▁▂▄▂▄▇▃▃▃▇▃▃▂▃▃▂▃▂█▂▃▂▂▄▂▂▂▇▂▁▃▂▁▃▄▃▇▂▃▄ ↓ 55% │ │ groups 8.000 lag 0 lag/max 0 lag/mean 0 length/mean 8.000 masked/cancelled 0 │ │ masked/infra_error 0 merge/calls_per_chain 1.000 pack/fill 100.0% resubmitted 0 │ │ status/agent_error 0.0% status/cancelled 0.0% status/completed 0.0% status/infra_error │ │ 0.0% status/truncated 100.0% │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ─────────────────────╮ │ trained 150 steps in 0:00:14 │ │ first step after 8.67 s │ │ then 27.8 step/s │ │ goodput 38.1% of the wall time in steps │ │ final loss 0.04121 │ ╰─────────────────────────────────────────────╯
Trainer.Reinforcement learning from pixels
A racing drone learns to fly through gates from a 64-pixel camera image and what its sensors read, with no demonstrations and no privileged state: a Dreamer world model learns from its own flights, and the policy learns inside the model's imagination. The film shows four drones of the trained policy, each through its own camera.
A racing drone that flies from its camera · Recording and scores
Reliability
A checkpoint stores the training state and the position in the data stream. When the job receives SIGTERM, fit stops at the next step every process agrees on, writes a checkpoint and exits with status 143. Run the same command again to resume.
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 18,560 parameters · on 1 × cpu · batch 8 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2/2 100% 3.5 step/s 0:00:03 │ │ data 40 rows · 0 rejected │ │ │ │ train │ │ loss 6.108 ▁ │ │ ce 6.108 ▁ │ │ perplexity 449.6 ▁ │ │ token_accuracy 0.4% ▁ │ │ data_wait_ms 0.02359 ▁ │ │ grad_norm 1.805 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 236.4 ▁ │ │ samples_per_sec 33.85 ▁ │ │ tokens 512.0 ▁ │ │ tokens_per_s 2,166 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────╮ │ trained 2 steps in 0:00:04 │ │ first step after 3.39 s │ │ then 5.9 step/s │ │ goodput 4.3% of the wall time in steps │ │ final loss 6.108 │ ╰────────────────────────────────────────────╯ Resumed from step 2 in /tmp/cell-reliability-0p43xk5u/checkpoints ╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 18,560 parameters · on 1 × cpu · batch 8 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3 100% 0:00:01 │ │ data 48 rows · 0 rejected │ │ │ │ train │ │ loss 5.874 ▁ │ │ ce 5.874 ▁ │ │ perplexity 355.7 ▁ │ │ token_accuracy 0.4% ▁ │ │ data_wait_ms 0 ▁ │ │ grad_norm 1.846 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 1.000 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 62.06 ▁ │ │ samples_per_sec 128.9 ▁ │ │ tokens 512.0 ▁ │ │ tokens_per_s 8,250 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ─────────────────────╮ │ trained 1 steps in 0:00:02 │ │ first step after 1.39 s │ │ goodput 10.5% of the wall time in steps │ │ final loss 5.874 │ ╰─────────────────────────────────────────────╯
Run runs the cell on a shared 4-vCPU host; training stops after 20 steps.
Wrote profile for 2 steps to profile ╭─ dew · CausalTransformer ──────────────────────────────────────────────────────────────────╮ │ 18,560 parameters · on 1 × cpu · batch 8 · float32 │ │ │ │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5/5 100% 10.5 step/s 0:00:03 │ │ data 64 rows · 0 rejected │ │ │ │ train │ │ loss 5.883 ▁ │ │ ce 5.883 ▁ │ │ perplexity 358.9 ▁ │ │ token_accuracy 0.7% ▁ │ │ data_wait_ms 0.04664 ▁ │ │ grad_norm 1.758 ▁ │ │ compiles 1.000 ▁ │ │ cache_hits 0 ▁ │ │ cache_misses 5.000 ▁ │ │ throughput │ │ step_time_ms 113.2 ▁ │ │ samples_per_sec 70.68 ▁ │ │ tokens 500.2 ▁ │ │ tokens_per_s 4,419 ▁ │ │ optimizer │ │ accepted 100.0% ▁ │ │ learning_rate 0.001000 ▁ │ ╰─────────────────────────────────────────────────────────────────────────────── ✓ finished ─╯ ╭─ ✓ CausalTransformer ────────────────────╮ │ trained 5 steps in 0:00:05 │ │ first step after 4.56 s │ │ then 11.3 step/s │ │ goodput 7.2% of the wall time in steps │ │ final loss 5.883 │ ╰────────────────────────────────────────────╯
Try the 176M model
Tutorials
- 01Diffusion from scratchGPU
- 02Train a diffusion modelGPU
- 03Text to image with guidanceGPU
- 04Samplers and schedulesGPU
- 05Train a language modelGPU
- 06Representations with I-JEPAGPU
- 07Scale across devicesCPU
- 08Continue a pretrained decoderGPU
- 09SFT for LLMsGPU
- 10RL with verifiable rewardsGPU
- 11LLMs in an agent harnessGPU
- 12World modelsGPU
- 13Continual learning in contextGPU
- 14Vision-based drone racingGPU
How each example above was recorded
- lm: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 34.7 s. - jepa: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 46.4 s. - finetune: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 121.5 s. - decide: a 4-vCPU Cloudflare container, JAX 0.11.2.post3, Dew
aee614a, 2026-10-08, 61.8 s. - diffusion: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 36.9 s. - mesh: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 34.5 s. - grpo: a 4-vCPU Cloudflare container, JAX 0.11.2.post16, Dew
8b4627a, 2026-10-24, 32.9 s. - reliability: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 35.3 s. - profile: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 34.7 s. - pretrained: a 4-vCPU Cloudflare container, JAX 0.11.2.post11, Dew
d2adda0, 2026-10-21, 138 s. - serving: a 4-vCPU Cloudflare container, JAX 0.11.2.post3, Dew
aee614a, 2026-10-08, 64.4 s. - formats: a 4-vCPU Cloudflare container, JAX 0.11.2.post4, Dew
a758210, 2026-10-19, 79.1 s.