Start with installation and the first call. This guide maps research questions to the existing public API; the complete walkthrough supplies a runnable experiment. You do not need to understand the Rust engine to use these tools.
Observe the right quantity
| Question | Use | Meaning of the result |
|---|---|---|
| What does the model answer? | llm_generate() |
Generated text; async mode resolves to the same result shape |
| What numerical representation does each document get? | llm_embed() |
One pooled, optionally normalized vector per input |
| What happens inside a forward pass over my prompt? | llm_trace() |
Activation values at selected layers, components and token positions |
| What happens while an answer is generated? | llm_generate(on_state = ...) |
Bounded observations delivered to an R callback during async generation |
| How do two recorded live states differ? | llm_compare() |
Matched coordinates, with differences withheld for incompatible histories |
| Which coefficients were applied during generation? | llm_timeline() |
Recorded additive coefficient history, not a static projection log |
A token is a model-specific piece of text. An
activation is a numerical value produced while
processing it. The residual stream passes values
between model blocks; attn_out and mlp_out
identify particular component outputs. They are different quantities,
and not all architectures expose every component. R-facing layer, token
and neuron indices are 1-based.
as.matrix(trace, layer = ..., component = ...) selects
one activation slice. Rows represent captured
(prompt_id, token_pos) pairs; columns represent neurons.
This matrix is ready for ordinary R analysis. It is not a matrix of
next-token probabilities or a measure of causal importance.
Keep prompts and observations aligned
llm_trace() traces the literal supplied prompt. It does
not capture a previous call’s generated answer and does not apply a chat
template. With generation, chat = FALSE continues literal
text; chat = TRUE applies the model’s template. Keep
formatting consistent when the experimental question requires it.
llm_compare() consumes two live
observations, not arbitrary traces or text strings. Record the
model, generation settings and preceding sampled token IDs. The first
state can have the same prompt history even if the newly sampled tokens
differ. Once prior generated tokens diverge, activation differences are
withheld: the model is then processing different inputs. Do not fill
those gaps with zero.
Token events, sampled states and committed text are distinct,
especially with cancellation or stop sequences. Use the meanings in the
streaming
guide, not interchangeable counts. To distinguish a token cap from a
stop, retain the finish_reason on the actual
prompt_end token event; the first-use recipe shows this
alongside the text. Callbacks execute on R’s main thread; keep them
short.
Design an intervention experiment
- State the question and observable outcome. Define prompts, output limits, settings and a concrete criterion before observing results. Read the text as well as counting characters or tokens.
- Separate construction from evaluation. Capture paired construction inputs and reserve different prompts for evaluation. Choose coefficients before the final comparison; do not retune on that comparison’s outputs.
- Keep controls. Include the original model and a zero-coefficient derived handle. Match generation settings and record any failed conditions.
-
Apply the appropriate operator.
llm_direction()builds a model-bound artifact from paired captures.llm_apply_direction()creates a derived model that shares weights while owning a separate context. Residual addition adds a vector; component projection removes its scaled contribution at a supported site.llm_ablate()fixes selected values. These operations are not equivalent. -
Observe through generation. Ordinary tracing and
embeddings are refused on intervened handles; their separate contexts
would not inherit the intervention. Use live
on_stateobservations when studying the actual intervention. - Export and interpret. Retain all texts, failures, settings, plots and trusted local RDS objects. Close each derived handle and then the original.
The direction is computed from paired target-minus-control values. A name such as “uncertainty direction” records the construction hypothesis; it does not prove the vector uniquely represents that concept. A two-prompt illustration cannot establish generalized efficacy. Keep construction, coefficient selection and held-out evaluation separate in a larger study.
The current artifact schema requires nonempty construction, selection and evaluation split manifests even for a fixed coefficient. In the small first-use example, the selection prompt is declared and reserved but never executed: there is no selection procedure or selection evidence. This is a schema requirement, not a reason to pretend that tuning occurred. The older complete walkthrough uses a practice prompt in that slot; its coefficient is also fixed in advance.
The portable first-use recipe puts prompt texts and its recorded settings at the top. For a new question, change the texts before the first run while preserving their IDs, row order and roles, and use a new output directory. Keep the supplied CPU/context-512/no-stop configuration: these fields are fixed in the execution body, not general configuration switches. The example’s README lists the limits of adaptation. The record of the original study remains unchanged.
Read the graphics correctly
-
plot(model)describes the model and configured interventions. It is not an activation heatmap or evidence that a component causes an outcome. - A comparison plot depicts the returned
llm_compare()table. Matching input history permits a coordinate difference; causal claims need additional design. - A timeline depicts additive revisions. A static projection can be active while the additive history is empty.
Export at a readable size instead of photographing a narrow RStudio plot pane. The graphics vignette includes PNG/PDF and CSV/RDS examples. Saved observations can be replotted without running the model again. CSV does not preserve all R attributes; retain the trusted RDS alongside it, and never load an untrusted RDS file.
Bound the work
Start with one small model, one derived handle at a time, one or a
few layers, and positions = "last". Prompt tracing runs a
forward pass without sampling. Async generation uses optional
later and promises; a noninteractive R script
must service the event loop and wait for settlement, as the walkthrough
does. Calling llm_cancel() requests cancellation; it does
not forcibly terminate an R process or guarantee a wall-clock/memory
ceiling.
Use the support matrix before combining model types, components and operators. For a record of what a less-contextualized agent could actually accomplish, read the first-use assessment.