Fine tune Llama with Unsloth on weightroom
October 9, 2026
weightroom's Unsloth workspace includes a notebook that takes you from loading a model to saving a trained LoRA adapter. The Python environment and GPU libraries are already installed, so you can start with the training code.
This walkthrough uses Llama 3.2 3B Instruct with QLoRA: a 4-bit base model plus a small set of trainable adapter weights. The base weights stay frozen while the adapter learns from your examples. Unsloth's fine-tuning guide explains the approach.
Open the included notebook
Launch the Unsloth workspace on weightroom, choose an available GPU, and keep the JupyterLab add-on enabled. Once the workspace is ready, open JupyterLab from its access links and select:
/workspace/unsloth-quickstart.ipynb
Run the cells in order. The first model download can take a few minutes; subsequent cells use that downloaded model.
Load the model and attach LoRA
The first cell loads the model with these settings:
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/Llama-3.2-3B-Instruct",
max_seq_length=2048,
load_in_4bit=True,
)
The next cell attaches rank-16 LoRA adapters to the attention and MLP projections. Keep the supplied settings for your first run. The notebook prints the trainable parameter count so you can see how much of the model you're updating.
Train on the sample dataset
The dataset cell loads 1,000 examples from yahma/alpaca-cleaned. Each example becomes an instruction, an optional input, and a response. The notebook handles this formatting for you.
The training cell runs 60 optimizer steps with a batch size of 2 and gradient accumulation of 4. On a single GPU, that combines eight examples per optimizer update. It prints training loss when the run finishes.
For your own dataset, preserve the instruction, input, and output fields or adapt the formatting function to your schema. Start with a small dataset that represents the behavior you want, and keep separate examples for evaluating the result.
Save and inspect the adapter
The final notebook cell saves the adapter and tokenizer:
model.save_pretrained("/workspace/my-finetuned-model")
tokenizer.save_pretrained("/workspace/my-finetuned-model")
Open that directory in JupyterLab. It should contain adapter_model.safetensors, adapter_config.json, and tokenizer files. These adapter weights are loaded alongside the original base model.
To inspect the saved weights, run:
from safetensors.torch import load_file
import torch
weights = load_file(
"/workspace/my-finetuned-model/adapter_model.safetensors"
)
assert any(
"lora_B" in name and torch.count_nonzero(value).item() > 0
for name, value in weights.items()
)
print("Saved adapter contains nonzero LoRA weights")
We verified this path on a weightroom GPU workspace using a shortened, two-step training run. It completed training and saved an adapter with nonzero LoRA weights. That checks the training and export path; model quality still depends on your data and evaluation.
Reload the adapter for inference
In a fresh notebook kernel, load the saved directory using Unsloth's adapter inference workflow:
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="/workspace/my-finetuned-model",
max_seq_length=2048,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
prompt = "### Instruction:\nGive three tips for staying focused.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
response = outputs[0, inputs.input_ids.shape[1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
The prompt follows the same instruction format used during training. Try held-out prompts from your intended task, compare responses with the base model, and keep the adapter directory together with its tokenizer and configuration files.
Explore GPU Workspaces.
Choose a pre-configured environment for development, training, or inference.