Loads a GGUF model file and returns an llm handle: an external pointer to
the native model plus its metadata. All arguments are validated in R before
the native boundary is crossed, so a bad request is reported as a classed
condition and never reaches the engine.
Arguments
- path
Single string: path to a GGUF model file.
- context_length
Positive integer: the active context window in tokens (llama.cpp:
n_ctx). Default 4096.- gpu_layers
NULL(auto: offload every layer that fits) or a single non-negative integer count of layers to offload (llama.cpp:n_gpu_layers). Ignored on the CPU backend.- backend
One of
"auto","metal","cuda","cpu".- mmap
Logical: memory-map the model file (default
TRUE).- projector
NULL(default: text-only, unchanged) or a single string: path to the model's mmproj GGUF, enabling image input (see the Image input section).- threads
NULL(default: preserve native defaults) or a single positive whole number of CPU threads for text generation and prompt processing. Derived generation, embedding, trace and intervention contexts inherit it. R check-time core limits take precedence. This does not set the separate vision projector's threads or guarantee faster inference.- x
An
llmhandle.- ...
Ignored.
Details
The handle owns native memory (potentially several gigabytes). Free it
deterministically with close() when done; a garbage-collection
finalizer frees it as a safety net otherwise (see close.llm()).
backend = "auto" selects available compiled backends in priority order:
Metal, then CUDA, then CPU. It does not benchmark their speed. Requesting a backend the build was
not compiled with raises relm_error_backend.
Supply your own compatible GGUF, or a pinned file fetched by llm_download().
No pretrained user model ships inside the package; tiny test fixtures are not
conversational models. The runnable example is guarded by the
RELM_TEST_MODEL_QWEN environment variable (point it at a local Qwen2.5 GGUF).
Loading a model does not guarantee support for every observation or
intervention API; unsupported capabilities are explicitly refused.
Image input (vision models)
projector enables image input for a vision-language model: point it at the
model's companion mmproj GGUF (the vision projector published alongside
the model, e.g. mmproj-*.gguf). The projector is a session property fixed
at load — it is bound to the loaded model — so it belongs on llm(), not on
each call; a handle loaded with a projector then accepts the images
argument of both llm_generate() and llm_embed(). A projector whose input
embedding size does not match the model raises relm_error_image naming
both sizes.
Trust note: the projector file is engine-trusted input, exactly like
path — a corrupt or hostile GGUF is parsed by native code. Prefer files
whose SHA256 you can verify (e.g. fetched with llm_download()); treat a
projector from an unknown source with the same care as a model file.
See also
close.llm(), print.llm(), summary.llm()
Examples
if (FALSE) { # nzchar(Sys.getenv("RELM_TEST_MODEL_QWEN"))
m <- llm(Sys.getenv("RELM_TEST_MODEL_QWEN"))
print(m)
summary(m)
close(m)
}
if (FALSE) { # nzchar(Sys.getenv("RELM_TEST_MODEL_VLM")) && nzchar(Sys.getenv("RELM_TEST_MMPROJ_VLM"))
# A vision-language model: pass its companion mmproj as the projector
# (e.g. llm_download("qwen2-vl-2b-instruct-q4_k_m") and
# llm_download("qwen2-vl-2b-instruct-mmproj-f16")).
m <- llm(Sys.getenv("RELM_TEST_MODEL_VLM"),
projector = Sys.getenv("RELM_TEST_MMPROJ_VLM")
)
print(m)
close(m)
}