Skip to contents

Builds an ordinary data frame from two delivered on_state observations. No inference is performed. Differences are withheld when the complete sampled input histories diverge. Caller-recorded context is validated, not authenticated.

Usage

llm_compare(
  reference,
  intervention,
  context,
  layer,
  component = "residual",
  neurons = NULL,
  max_bytes = 64 * 1024^2
)

# S3 method for class 'relm_comparison'
plot(x, ...)

Arguments

reference, intervention

Single observations from llm_generate(on_state=).

context

A list with reference and intervention entries, each holding model_sha256, settings, and integer generated_tokens through its state. Settings contain, in order, seed, chat, temperature, top_p, max_tokens, stop, context_length, backend, relm_version, engine_revision.

layer

One 1-based captured transformer layer.

component

One of residual, attn_out, mlp_out.

neurons

NULL for all captured coordinates, or unique 1-based indices.

max_bytes

Working materialization budget, whole bytes from 64 KiB to 256 MiB.

x

A relm_comparison returned by llm_compare().

...

Named main, cex or col (one or three colors).

Value

A relm_comparison data frame with layer, component, neuron, reference, intervention and difference. Attributes retain bounded provenance, alignment, applied steering and a truncated logit comparison, never the original traces.

The plot method returns the comparison invisibly.

Details

The input for state k excludes its just-sampled token. Equal model/settings/ prompt records establish prompt identity; every earlier sampled token must also match. A divergent history keeps the observations but returns NA differences. Missing top-k entries are NA, not zero probabilities. A sampled token may never appear in committed output after cancellation or stop removal. Returning a coefficient to zero does not erase prior effects in KV history.

Examples

# Synthetic schema example: no model, inference or download.
source(system.file("examples", "graphics-states.R", package = "relm"), local = TRUE)
comparison <- llm_compare(reference, intervention, context, layer = 2)
comparison$difference
#> [1]  0.5 -1.0  1.0  0.0
figure <- tempfile(fileext = ".pdf")
grDevices::pdf(figure, width = 10, height = 7)
tryCatch(plot(comparison), finally = grDevices::dev.off())
unlink(figure)