Skip to contents

Runs a forward pass over each prompt and returns the model's distribution for the single token that would come next: the top most likely candidates, ranked from most to least probable. This is the raw material behind generation — the first token llm_generate() would pick (greedily) is exactly this table's rank-1 token.

Usage

llm_logits(m, prompt, top = 20)

Arguments

m

An llm handle from llm().

prompt

A character vector of prompts; the result has top rows per prompt, and prompt_id is the 1-based index into prompt.

top

Single positive integer: how many top candidates to return per prompt (clamped to the vocabulary size). Default 20.

Value

A base data.frame with top rows per prompt and columns prompt_id (int, 1-based index into prompt), rank (int, 1 = most likely), token_id (int, 1-based), token (chr, the token piece), logit (dbl), and prob (dbl, softmax over the full vocabulary).

Details

Each prompt is treated as raw text (no chat template is applied — wrap it yourself if you want the chat framing) and tokenized the way chat = FALSE generation is. The distribution is read at the final position, so it answers "what token does the model expect immediately after this text?". Probabilities are the softmax over the whole vocabulary, computed before the top candidates are selected, so each prob is the token's true share of the full distribution: the returned probabilities are correct but (being only the head of the distribution) sum to less than 1.

The result is one long-format data.frame with top rows per prompt, stacked in prompt order. Within each prompt the rows are ordered by descending logit, so rank == 1 is the most likely next token; logit and prob are therefore non-increasing down the ranks. Token ids are 1-based (like every index in relm, and like llm_tokens()), so token_id round-trips through llm_tokens(m, token_id, decode = TRUE).

Active interventions on m (from llm_steer()/llm_ablate()) apply, so llm_logits() is the direct way to read how a steer or an ablation reshapes the next-token distribution.

The model must carry a tokenizer; a no_vocab model (such as the in-repo synthetic model) raises relm_error_tokenize. No model ships inside the package yet, so the runnable example is guarded by the RELM_TEST_MODEL_QWEN environment variable — point it at a local Qwen2.5 GGUF to run it.

Examples

if (FALSE) { # nzchar(Sys.getenv("RELM_TEST_MODEL_QWEN"))
m <- llm(Sys.getenv("RELM_TEST_MODEL_QWEN"))
llm_logits(m, "The capital of France is", top = 5)
close(m)
}