the path · 0/15
start the path

the kernel path · jaxpr · lesson 04 of 5

Params, and the drills that fix them

Params are the configuration a primitive needs before it can run. Fluency is repetition, and the gym exists for exactly that.

the goal Read dimension_numbers, axes, and friends on real equations without a lookup, at drill speed.

mastery work · this chapter0/2
  1. go →
manual items are your word; auto items complete from your streaks, labs, and can-you ticks · stored in your browser only
§ 01

Params, and the drills

Params are where a primitive keeps whatever isn't itself a traced value, the configuration that has to be fixed before the primitive can run rather than computed while it runs. dot_general's params hold dimension_numbers, the exact statement of which axes contract against which and which axes broadcast, the information that turns two arrays into a matmul, a batched matmul, or a plain contraction, depending on what's in there. A reduction like sum or max holds axes in its params, the list of dimensions being collapsed. A type conversion holds new_dtype, the target type values get cast into. None of these are variables flowing through the jaxprThe traced program: one equation per primitive in single-assignment form, every shape and dtype stated.taught in /l/jaxpr →; they are settings the equation carries alongside it.

Reading params fluently is less about memorizing a table and more about repetition against real jaxprsThe traced program: one equation per primitive in single-assignment form, every shape and dtype stated.taught in /l/jaxpr →, enough of them that dimension_numbers or axes stops requiring a lookup. That's what the drills are for: decoder exercises where you predict what an equation does from its params alone, oracle checks that confirm or correct the guess, and spot-the-decision exercises that isolate exactly which param changed the behavior. None of it is complicated once you've done it a dozen times. The difficulty is entirely in not having done it yet.

The same primitives keep showing up once you move into kernel material. A Pallas kernel body still lowers through dot_general, still carries dimension_numbers, still reduces along explicit axes. Nothing about the grammar changes there, only the shapes and the register tiling underneath it. That's the payoff of getting fluent with params here: by the time you're staring at a kernel jaxprThe traced program: one equation per primitive in single-assignment form, every shape and dtype stated.taught in /l/jaxpr → with a grid and a block spec wrapped around it, the params inside are already familiar, and what's new is only the pipelining around them.

§ 02

Exercises

exercise Before opening the vmap gallery entry: predict how many equations dense gains under jax.vmap and what happens to each shape. Then check. The answer to the first question is the lesson.
exercise Trace any three-line function of your own with jax.make_jaxpr, close over one array from outside the function, and point at where it surfaces: invars or constvars?
exercise In the grad gallery entry, find the equations computing the tanh derivative and name the variable holding the incoming cotangent. The mul and sub pattern around tanh is the fingerprint.
exercise Run the gym's dot_generalThe general matmul primitive. Its dimension_numbers state which axes contract and which batch; decoding them on sight is the core IR-reading skill.taught in /l/stablehlo → decoder until a streak of 5, then come back and read the dimension_numbers param on the gallery's matmul equation without the drill's help.
before you move on

Check yourself

01 What kind of thing belongs in an equation’s params rather than its operands?

Anything that must be fixed before the primitive can run and is not itself a traced value: dimension numbers for a dot_general, the axes of a reduction, a scan’s body and lengths.

02 The same primitives keep appearing under Pallas kernels. What changes about the grammar there, and what does not?

Nothing about the grammar changes; a kernel body still lowers through dot_general with dimension_numbers and reductions with explicit axes. What changes is the shapes and the memory the values live in.

assigned

Readings