Skip to contents

Loads a GGUF model file and returns an llm handle: an external pointer to the native model plus its metadata. All arguments are validated in R before the native boundary is crossed, so a bad request is reported as a classed condition and never reaches the engine.

Usage

llm(
  path,
  context_length = 4096,
  gpu_layers = NULL,
  backend = c("auto", "metal", "cuda", "cpu"),
  mmap = TRUE,
  projector = NULL,
  threads = NULL
)

# S3 method for class 'llm'
print(x, ...)

Arguments

path

Single string: path to a GGUF model file.

context_length

Positive integer: the active context window in tokens (llama.cpp: n_ctx). Default 4096.

gpu_layers

NULL (auto: offload every layer that fits) or a single non-negative integer count of layers to offload (llama.cpp: n_gpu_layers). Ignored on the CPU backend.

backend

One of "auto", "metal", "cuda", "cpu".

mmap

Logical: memory-map the model file (default TRUE).

projector

NULL (default: text-only, unchanged) or a single string: path to the model's mmproj GGUF, enabling image input (see the Image input section).

threads

NULL (default: preserve native defaults) or a single positive whole number of CPU threads for text generation and prompt processing. Derived generation, embedding, trace and intervention contexts inherit it. R check-time core limits take precedence. This does not set the separate vision projector's threads or guarantee faster inference.

x

An llm handle.

...

Ignored.

Value

An object of class llm (see the package's class documentation).

x, invisibly.

Details

The handle owns native memory (potentially several gigabytes). Free it deterministically with close() when done; a garbage-collection finalizer frees it as a safety net otherwise (see close.llm()).

backend = "auto" selects available compiled backends in priority order: Metal, then CUDA, then CPU. It does not benchmark their speed. Requesting a backend the build was not compiled with raises relm_error_backend.

Supply your own compatible GGUF, or a pinned file fetched by llm_download(). No pretrained user model ships inside the package; tiny test fixtures are not conversational models. The runnable example is guarded by the RELM_TEST_MODEL_QWEN environment variable (point it at a local Qwen2.5 GGUF). Loading a model does not guarantee support for every observation or intervention API; unsupported capabilities are explicitly refused.

Image input (vision models)

projector enables image input for a vision-language model: point it at the model's companion mmproj GGUF (the vision projector published alongside the model, e.g. mmproj-*.gguf). The projector is a session property fixed at load — it is bound to the loaded model — so it belongs on llm(), not on each call; a handle loaded with a projector then accepts the images argument of both llm_generate() and llm_embed(). A projector whose input embedding size does not match the model raises relm_error_image naming both sizes.

Trust note: the projector file is engine-trusted input, exactly like path — a corrupt or hostile GGUF is parsed by native code. Prefer files whose SHA256 you can verify (e.g. fetched with llm_download()); treat a projector from an unknown source with the same care as a model file.

See also

Examples

if (FALSE) { # nzchar(Sys.getenv("RELM_TEST_MODEL_QWEN"))
m <- llm(Sys.getenv("RELM_TEST_MODEL_QWEN"))
print(m)
summary(m)
close(m)
}
if (FALSE) { # nzchar(Sys.getenv("RELM_TEST_MODEL_VLM")) && nzchar(Sys.getenv("RELM_TEST_MMPROJ_VLM"))
# A vision-language model: pass its companion mmproj as the projector
# (e.g. llm_download("qwen2-vl-2b-instruct-q4_k_m") and
# llm_download("qwen2-vl-2b-instruct-mmproj-f16")).
m <- llm(Sys.getenv("RELM_TEST_MODEL_VLM"),
  projector = Sys.getenv("RELM_TEST_MMPROJ_VLM")
)
print(m)
close(m)
}