When should you fine-tune? (Usually: not first.)

Fine-tuning has an aura of being "the real AI work," but it's the last lever you should reach for, not the first. Walk down this ladder and stop at the first rung that solves the problem:

1. Better prompting — clearer instructions, a system prompt, few-shot examples. Free, instant, and fixes a surprising share of "the model is dumb" problems.
2. RAG / tool use — if the issue is missing knowledge (your docs, live data, private facts), retrieve it into the context. Covered in the RAG track.
3. Fine-tuning — only when you need to change the model's behavior: a consistent format, a tone, a domain style, or a narrow task the base model does inconsistently even with good prompts.

Good fits for fine-tuning:

• Enforcing a strict output format or schema the model keeps drifting from.
• A consistent brand voice or persona across thousands of calls.
• A narrow, repetitive classification/extraction task where you have labelled examples.
• Distilling a big expensive model's behavior into a smaller, cheaper one (cost/latency win).

Poor fits: up-to-date facts, private knowledge bases, anything you'd want a citation for, or anything that changes weekly. Those are retrieval problems.

What SFT actually changes

The dominant technique is Supervised Fine-Tuning (SFT): you show the model many (input → ideal output) pairs and nudge its weights so its outputs move toward your examples. It's the same next-token training the base model had, but continued on your curated data.

What that buys you and what it doesn't:

• It shifts behavior/style — after SFT on 500 support transcripts in your voice, the model defaults to that voice without you re-prompting it every time.
• It does not reliably add facts — a fact seen a handful of times in training is easily overwritten and never citable.
• It can cause forgetting — over-training on a narrow task degrades general ability ("catastrophic forgetting"). More data and fewer epochs is usually safer than the reverse.

Full fine-tuning vs. LoRA / PEFT

Updating all of a model's billions of weights ("full fine-tuning") is expensive, needs lots of GPU memory, and produces a full-size copy of the model per task. PEFT (Parameter-Efficient Fine-Tuning) avoids that. The most common form is LoRA (Low-Rank Adaptation):

• Freeze the original weights. Train only a small pair of low-rank "adapter" matrices injected into each layer — often under 1% of the parameters.
• The adapter is tiny (megabytes), so you can keep many task adapters for one base model and swap them in on demand.
• Quality is usually close to full fine-tuning for typical tasks, at a fraction of the cost. QLoRA goes further by training the adapter on top of a quantized (compressed) base model, so it fits on a single consumer GPU.

The data is the job

Fine-tuning quality is dominated by dataset quality, not hyperparameters. A few hundred clean, consistent, representative examples usually beat tens of thousands of noisy ones. The format is the same chat structure you already know — written as JSONL, one training example per line:

import json                                   # standard-library JSON module

# Each training example is one conversation showing the model the behavior you want it to default to.
examples = [
    {"messages": [
        {"role": "system",    "content": "You are a support agent. Reply in 2 sentences, warm and concrete."},
        {"role": "user",      "content": "My invoice is wrong."},
        {"role": "assistant", "content": "I'm sorry about that — I've flagged your latest invoice for review. "
                                          "You'll get a corrected copy by email within one business day."},
    ]},
    # …hundreds more, all demonstrating the SAME target behavior/format/voice…
]

# Write JSONL: one JSON object per line (NOT a JSON array). This is the format hosted APIs expect.
with open("train.jsonl", "w") as f:           # open for writing; the `with` block auto-closes the file
    for ex in examples:                        # for each training example…
        f.write(json.dumps(ex) + "\n")         # dump it to a JSON string (like JSON.stringify) + a newline

Data rules that matter more than any knob:

• Consistency beats volume. Every example should demonstrate the same target behavior. One off-style example teaches the model the wrong lesson.
• Match production inputs. Train on examples that look like the real requests you'll receive, edge cases included.
• Hold out a validation split. Keep ~10–20% of examples unseen to measure whether the fine-tune actually generalizes rather than memorizes.
• Balance the classes. For classification/extraction, don't let one label dominate.

Fine-tuning vs. RAG vs. embeddings — the confusion table

These three get conflated constantly. Keep them straight:

Tool	Changes…	Reach for it when…	Updatable?
Prompting	nothing (inference-time steering)	you haven't tried clear instructions/few-shot yet	instantly
RAG	what the model knows at query time	the gap is missing / fresh / private knowledge	instantly (re-index)
Fine-tuning	how the model behaves by default	the gap is consistent format / tone / task behavior	slowly (retrain)
Embeddings	nothing — they represent meaning for search	you need semantic similarity / retrieval	n/a (a component of RAG)

The trap is treating fine-tuning as "teaching the model new information." Embeddings and RAG handle knowledge; fine-tuning handles behavior. Many production systems fine-tune for voice/format and use RAG for facts — they compose cleanly.

A minimal hosted workflow

On a managed platform the loop is: prepare JSONL → upload → create a fine-tune job → get a new model id → evaluate → deploy. You never touch a GPU.

from openai import OpenAI
client = OpenAI()

f = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")   # upload the dataset
job = client.fine_tuning.jobs.create(training_file=f.id, model="gpt-4o-mini-2024-07-18")  # start the job
# When it finishes, job.fine_tuned_model is a NEW model id you call exactly like any other model:
#   client.chat.completions.create(model=job.fine_tuned_model, messages=[...])

Pitfalls

• Fine-tuning for facts. The number-one mistake — use RAG for knowledge.
• Skipping the ladder. Fine-tuning before trying a good prompt or RAG wastes time and money.
• Dirty, inconsistent data. The model learns exactly what you show it, including your mistakes.
• No validation split / no eval. You can't tell memorization from generalization, or improvement from regression.
• Over-training. Too many epochs on a narrow set causes catastrophic forgetting of general skills.
• Ignoring maintenance cost. A fine-tune is a frozen artifact; prompts and RAG indexes update in seconds. Factor the retraining treadmill into the decision.

Recap

• Fine-tuning is the last rung: try prompting → RAG → fine-tuning, stopping at the first that works.
• SFT changes behavior (format, tone, task style) by continuing training on your (input → output) pairs; it does not reliably add knowledge.
• LoRA/PEFT is the practical default — tiny swappable adapters, near-full quality, a fraction of the cost.
• Data quality dominates: a few hundred clean, consistent, representative examples in messages JSONL, with a held-out split.
• Keep the tools straight — RAG changes knowledge, fine-tuning changes behavior, embeddings power search — and remember they compose.
• Worked material maps to the exercises: the JSONL format (Ex 1), data-quality audit (Ex 2), and the prompting-vs-RAG-vs-fine-tuning decision (Ex 3).

"My invoice is wrong."          -> "I'm sorry about that — I've flagged it for review. You'll get a corrected copy within one business day."
"How do I export my data?"      -> "You can export from Settings → Data → Export. The file lands in your inbox within a few minutes."
"Do you offer student pricing?" -> "Yes — students get 40% off with a valid .edu email. Just reply here and I'll apply it."

import json

SYSTEM = "You are a support agent. Reply in 2 sentences, warm and concrete."

raw = [
    ("My invoice is wrong.",
     "I'm sorry about that — I've flagged it for review. You'll get a corrected copy within one business day."),
    ("How do I export my data?",
     "You can export from Settings → Data → Export. The file lands in your inbox within a few minutes."),
    ("Do you offer student pricing?",
     "Yes — students get 40% off with a valid .edu email. Just reply here and I'll apply it."),
]

with open("train.jsonl", "w") as f:
    for user, assistant in raw:
        ex = {"messages": [
            {"role": "system",    "content": SYSTEM},
            {"role": "user",      "content": user},
            {"role": "assistant", "content": assistant},
        ]}
        f.write(json.dumps(ex) + "\n")   # one JSON object per line — JSONL, not a JSON array

{"messages":[{"role":"system","content":"Reply in 2 sentences."},{"role":"user","content":"My invoice is wrong."},{"role":"assistant","content":"Sorry! Flagged for review — fixed copy within one business day."}]}
{"messages":[{"role":"system","content":"You are a pirate."},{"role":"user","content":"How do I export data?"},{"role":"assistant","content":"Arrr, click Settings then Export, matey!"}]}
{"messages":[{"role":"user","content":"Student pricing?"},{"role":"assistant","content":"Students get 40% off with a valid .edu email. This has been our policy since 2019 and we also offer nonprofit discounts and annual billing and a referral program and a 14-day trial and..."}]}
{"messages":[{"role":"system","content":"Reply in 2 sentences."},{"role":"user","content":"Refund?"},{"role":"assistant","content":"Refunds are issued within 30 days."}]}
{"messages":[{"role":"system","content":"Reply in 2 sentences."},{"role":"user","content":"Refund?"},{"role":"assistant","content":"Refunds are issued within 30 days."}]}