Getting started#
This page walks through installing Llamatik Server, running it, and making your first inference request.
Requirements#
- JVM 21+
- A GGUF model file (e.g.
tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf) - Docker (optional, for containerised deployment)
Clone and build#
git clone https://github.com/ferranpons/Llamatik-Server.git
cd Llamatik-Server
./gradlew buildRun locally#
./gradlew runThe server starts on http://localhost:8080 by default.
Minimal environment#
export JWT_SECRET="change-me-in-production"
export JDBC_DRIVER="org.h2.Driver"
export JDBC_DATABASE_URL="jdbc:h2:mem:llamatik;DB_CLOSE_DELAY=-1"JWT_SECRET signs user tokens. JDBC_* configure the user/profile database (H2 in-memory is fine for local development; use PostgreSQL for production). See Configuration for all variables.
Load a model and generate text#
Step 1 — initialize the generation model:
curl -s -X POST http://localhost:8080/v1/generation/init \
-H "Content-Type: application/json" \
-d '{"modelPath": "/models/tinyllama.gguf"}' | jq{ "ok": true }Step 2 — generate text:
curl -s -X POST http://localhost:8080/v1/generation/generate \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain Kotlin Multiplatform in one sentence."}' | jq{ "text": "Kotlin Multiplatform lets you share business logic across Android, iOS, and Desktop from a single codebase." }Step 3 — stream tokens:
curl -N -X POST http://localhost:8080/v1/generation/stream \
-H "Content-Type: application/json" \
-d '{"prompt": "Write a haiku about a llama."}'data: {"event":"delta","text":"A "}
data: {"event":"delta","text":"llama "}
data: {"event":"delta","text":"wanders."}
data: {"event":"done"}Load a Whisper model and transcribe audio#
curl -s -X POST http://localhost:8080/v1/whisper/init \
-H "Content-Type: application/json" \
-d '{"modelPath": "/models/ggml-base.en.bin"}' | jq
curl -s -X POST http://localhost:8080/v1/whisper/transcribe \
-H "Content-Type: application/json" \
-d '{"wavPath": "/audio/recording.wav", "language": "en"}' | jqLoad an image model and generate an image#
curl -s -X POST http://localhost:8080/v1/stablediffusion/init \
-H "Content-Type: application/json" \
-d '{"modelPath": "/models/dreamshaper.safetensors", "threads": 4}' | jq
curl -s -X POST http://localhost:8080/v1/stablediffusion/txt2img \
-H "Content-Type: application/json" \
-d '{"prompt": "A watercolor llama in the mountains", "width": 512, "height": 512}' | jqThe imageBase64 field in the response is a Base64-encoded RGBA byte array.
Next steps#
- Configure path security and authentication: Configuration
- Explore all endpoints: API Reference
- Run in Docker or as a systemd service: Deployment