Guides on Llamatik Documentation

Embeddings

Mon, 01 Jan 0001 00:00:00 +0000

Embeddings convert text into vectors that preserve semantic meaning. They are a foundation for search, retrieval, similarity ranking, clustering, and RAG pipelines.

Initialization#

val ok = LlamaBridge.initEmbedModel(modelPath)
check(ok)

Use an embedding model here, not your general chat model.

Generating vectors#

val v1 = LlamaBridge.embed("What is Llamatik?")
val v2 = LlamaBridge.embed("Kotlin llama.cpp bindings")

The returned FloatArray can be stored directly or passed into your vector search library.

Text generation

Mon, 01 Jan 0001 00:00:00 +0000

Text generation is the core LlamaBridge workflow.

Basic one-shot generation#

Use generate(...) when each request is independent.

val ok = LlamaBridge.initGenerateModel(modelPath)
check(ok)

val answer = LlamaBridge.generate("Explain Llamatik in one sentence.")
println(answer)

This is the right choice for:

one-off prompts
utility screens
simple summarization tasks
commands that do not need chat history

Context-aware generation#

Use generateWithContext(...) when you want to separate:

Streaming

Mon, 01 Jan 0001 00:00:00 +0000

Streaming lets your app consume generation output incrementally. This usually feels much better than waiting for the full answer, especially on-device where responses may take time.

Plain streaming#

LlamaBridge.generateStream(
 prompt = "Stream a short poem.",
 callback = object : GenStream {
 override fun onDelta(text: String) = print(text)
 override fun onComplete() = println("\nDone")
 override fun onError(message: String) = println("Error: $message")
 }
)

Streaming with context#

LlamaBridge.generateStreamWithContext(
 systemPrompt = "You are concise.",
 contextBlock = "Topic: on-device LLMs.",
 userPrompt = "Give me 3 bullet points.",
 callback = object : GenStream {
 override fun onDelta(text: String) = print(text)
 override fun onComplete() = println("\nDone")
 override fun onError(message: String) = println("Error: $message")
 }
)

Streaming JSON#

LlamaBridge.generateJsonStream(
 prompt = "Return a JSON object for one task.",
 jsonSchema = schema,
 callback = object : GenStream {
 override fun onDelta(text: String) = print(text)
 override fun onComplete() = println("\nDone")
 override fun onError(message: String) = println("Error: $message")
 }
)

Streaming with KV cache continuity#

Every streaming function on LlamaBridge (generateStream, generateStreamWithContext, etc.) resets the KV cache before each call by design. To keep the model’s memory across turns while streaming, you have two options.

Generation parameters

Mon, 01 Jan 0001 00:00:00 +0000

Llamatik exposes runtime generation parameters through updateGenerateParams(...).

LlamaBridge.updateGenerateParams(
 temperature = 0.7f,
 maxTokens = 256,
 topP = 0.95f,
 topK = 40,
 repeatPenalty = 1.1f,
 contextLength = 4096,
 numThreads = 4,
 useMmap = true,
 flashAttention = false,
 batchSize = 512,
 gpuLayers = 0,
)

Sampling parameters#

temperature#

Controls randomness.

lower values: more deterministic
higher values: more varied and creative

maxTokens#

Sets the maximum number of tokens the model may generate. Use this to control response length and latency.

Schema-constrained JSON

Mon, 01 Jan 0001 00:00:00 +0000

Llamatik can ask the model to return JSON and optionally constrain it to a JSON Schema. This is one of the most useful features when you want AI output to plug into typed application logic.

Basic JSON generation#

val json = LlamaBridge.generateJson(
 prompt = "Return a JSON object with fields: name (string), year (int)."
)
println(json)

JSON Schema constrained generation#

val schema = """
{
 "type": "object",
 "additionalProperties": false,
 "properties": {
 "name": { "type": "string" },
 "year": { "type": "integer" }
 },
 "required": ["name", "year"]
}
""".trimIndent()

val json = LlamaBridge.generateJson(
 prompt = "Generate an example object.",
 jsonSchema = schema
)

With context#

val json = LlamaBridge.generateJsonWithContext(
 systemPrompt = "You output only JSON.",
 contextBlock = "The product is Llamatik.",
 userPrompt = "Return product metadata.",
 jsonSchema = schema
)

Streaming JSON#

LlamaBridge.generateJsonStream(
 prompt = "Return JSON for a grocery list with 3 items.",
 jsonSchema = schema,
 callback = object : GenStream {
 override fun onDelta(text: String) = print(text)
 override fun onComplete() = println("\nDone")
 override fun onError(message: String) = println("Error: $message")
 }
)

Why this matters#

Schema-constrained output reduces brittle post-processing and makes it easier to parse results with Kotlin serialization libraries. It is especially useful for:

KV cache and sessions

Mon, 01 Jan 0001 00:00:00 +0000

KV cache reuse is one of the most important performance features for chat-like experiences. Instead of rebuilding the full prompt state from scratch for every turn, the model can continue from what it has already processed.

There are three ways to use the KV cache:

LlamaSession — for streaming multi-turn conversations with an in-memory KV cache that persists across stream() calls. Recommended for chat UIs that do not need cross-process persistence.
generateContinueStream — streaming multi-turn generation with the global KV cache. Supports sessionSave/sessionLoad for persistence across restarts. The correct choice when you need both streaming and disk-serialized session state.
generateContinue — blocking (non-streaming) continuation. Use when you do not need incremental token delivery.

Core methods (global)#

LlamaBridge.sessionReset()
LlamaBridge.sessionSave(path)
LlamaBridge.sessionLoad(path)
LlamaBridge.generateContinue(prompt) // blocking
LlamaBridge.generateContinueStream(prompt, callback) // streaming

Fresh turn vs continued turn#

Fresh generation#

val answer = LlamaBridge.generate("Explain Kotlin coroutines.")

Use this when the request is independent and you do not care about previous turns.

Stable Diffusion image generation

Mon, 01 Jan 0001 00:00:00 +0000

StableDiffusionBridge lets you generate images locally from text prompts. The bridge returns raw RGBA pixel data so your app can convert it into the most appropriate image type for the current platform.

Initialize the model#

val modelPath = StableDiffusionBridge.getModelPath("dreamshaper.safetensors")
val ok = StableDiffusionBridge.initModel(modelPath, threads = 4)
check(ok)

Generate an image#

val rgba = StableDiffusionBridge.txt2img(
 prompt = "A cinematic illustration of a llama explorer",
 negativePrompt = "blurry, low quality, extra limbs",
 width = 512,
 height = 512,
 steps = 20,
 cfgScale = 7.0f,
 seed = 1234L,
)

Understanding the result#

The output is an RGBA ByteArray. To display it, convert the bytes into your platform image type using the same width and height you requested.

Whisper speech-to-text

Mon, 01 Jan 0001 00:00:00 +0000

WhisperBridge is the speech-to-text entry point in Llamatik. The library exposes two transcription methods: transcribeWav for a simple flat string, and transcribeWavSegments for structured per-segment output with timestamps, detected language, and speaker-turn boundaries.

Initialize the model#

val modelPath = WhisperBridge.getModelPath("ggml-base.en.bin")
val ok = WhisperBridge.initModel(modelPath)
check(ok)

Transcribe a file#

val text = WhisperBridge.transcribeWav(
 wavPath = "/path/to/audio.wav",
 language = "en"
)
println(text)

Language hint#

The language parameter is optional, but supplying it can improve reliability when you already know the input language.

Multimodal vision (VLM)

Mon, 01 Jan 0001 00:00:00 +0000

MultimodalBridge lets you analyze images on-device using a vision-language model (VLM). The model can describe scenes, answer questions about images, extract text, and reason about visual content.

Required models#

You need two files from the same VLM release:

A GGUF language model (e.g. llava-v1.6-mistral-7b.Q4_K_M.gguf)
A multimodal projector (e.g. llava-v1.6-mistral-7b-mmproj-model-f16.gguf)

The projector must match the language model — mixing files from different releases will produce incorrect or empty output.

Concurrent sessions

Mon, 01 Jan 0001 00:00:00 +0000

LlamaSession lets you run multiple independent generation streams at the same time. Each session has its own KV cache and cancel flag, so they do not interfere with one another.

When to use sessions#

The global LlamaBridge.generateStream(...) works well for single-threaded chat flows. Use LlamaSession when:

you want to compare responses to the same prompt from different system prompts side by side
you have multiple users sharing a single JVM process
you need a background task (summarization, classification) while the main chat is active

Create a session#

Sessions are created from LlamaBridge after the generation model is loaded. Pass a name string to give the session a human-readable label:

Multi-Token Prediction (MTP)

Mon, 01 Jan 0001 00:00:00 +0000

Multi-Token Prediction (MTP) is a speculative decoding technique where a lightweight draft head — embedded in the same GGUF file as the main model — predicts several tokens ahead of the trunk. The trunk model then verifies all draft tokens in one batched forward pass and accepts the ones that match.

The result is faster generation with no change in output quality.

Building from source

Mon, 01 Jan 0001 00:00:00 +0000

The pre-built Maven Central artifacts include CPU-only native binaries everywhere except macOS, where Metal is enabled by default. If you need a GPU backend such as Vulkan or CUDA you must build Llamatik from source and pass the appropriate CMake flags.

Extra CMake flags#

The llamatik.cmake.args Gradle property (or the LLAMATIK_CMAKE_ARGS environment variable as a fallback) lets you append flags to the CMake configure step without editing the build script.