Preparing Training Data for Fine-Tuning: Format, Size & Splits

Ask anyone who has shipped a fine-tune what the hard part was and you will get the same answer. Not the GPU. Not the hyperparameters. It was the six weeks of arguing about what a correct example even looks like, and then producing three thousand of them without contradicting themselves.

Training is days. Data is weeks. This lesson is the weeks.

What one example looks like

For chat models, one training example is a conversation: the system message, the user input, and the assistant response you want. Usually one JSON object per line — JSONL.

{"messages": [
  {"role": "system",
   "content": "You convert facilities emails into maintenance tickets. Return JSON only."},
  {"role": "user",
   "content": "The AC in meeting room 3 has been dripping onto the carpet since Monday."},
  {"role": "assistant",
   "content": "{\"category\":\"HVAC\",\"location\":\"Meeting room 3\",\"urgency\":\"high\",\"summary\":\"AC unit leaking water onto carpet\"}"}
]}

One detail here matters more than it looks: the system message is present, and it is the same one production will send. Train without it and serve with it, and you have a train-serve mismatch — the model was taught a slightly different task than the one you run. It shows up as a model that scores beautifully on your eval harness and behaves oddly in the app.

The cardinal rule

Every example's response must be fully derivable from that example's input.

Break it and you are training the model to invent. Here is the bug in its natural habitat:

user:      "where is my order"
assistant: "Your order ORD10024 is in transit, arriving 5 Sep."

# The input contains no order id, no status, no date.
# What the model learns is not "ORD10024 is in transit".
# What it learns is: "when asked about an order, confidently
# produce an order id, a status and a date."
#
# You have trained a hallucination machine, deliberately.

system: "You are ShopKart support. Use only the ORDER block."
user:   "where is my order

         ORDER: {id: ORD10024, status: in_transit, eta: 2026-09-05}"
assistant: "Your order ORD10024 is in transit and should reach you
            by 5 September."

# Now the response is derivable from the input, and the model
# learns to READ the order block rather than to invent one.

The same rule generates the examples people always forget: what should the model do when the information isn't there? If your dataset never contains an example where the right answer is "that detail isn't in the record", the model will never learn to say it.

How many examples?

Providers accept surprisingly few — often tens. That is the floor for the job running, not the bar for it being worth doing.

The practical method beats any table: label 300, train, measure. Then label 300 more and measure again. The curve tells you whether more data is still buying anything before you commit six weeks to it. Doubling a dataset that has already plateaued is the most common wasted effort in these projects.

💡 500 clean, diverse, correctly-labelled examples beat 50,000 scraped ones — and the scraped ones actively teach mistakes, so they are worse than nothing.

What the mix should be

Mirror production, then deliberately over-represent the hard parts:

• ~55–60% typical traffic, sampled to match the real category mix. If 40% of tickets are delivery complaints, roughly 40% of your data should be.
• ~20% awkward inputs — vague, multi-issue, wrong language, missing fields, forwarded threads with history quoted.
• ~10% "none of the above" — inputs where the right output is a refusal, an empty result, or "not stated". Without these the model learns that a confident answer is always correct.
• ~10% edge cases that matter — the high-severity ones the business cares about most.

Watch class balance. A dataset that is 90% one label teaches the model to over-predict it, and aggregate accuracy will look fine because 90% is achievable by always guessing the majority. Report per-class recall.

Splitting without leaking

Split into train / validation / test — roughly 70 / 15 / 15. The test set stays untouched until the very end.

The trap is how you split. A random split puts near-duplicates on both sides: two variants of the same product, two emails from the same building, two chunks of the same document. Then your validation score measures memorisation and looks wonderful.

Split by a grouping key instead — by customer, by source document, by product category — and, where behaviour changes over time, split by time as well, training on the past and validating on the future. That is what production actually looks like.

Wait — how do I know the labels are right?

You don't, unless you check. And a systematically wrong label is the hardest failure in this entire course to diagnose, because the model learns it faithfully and it looks like a model problem.

The process that prevents it:

1. Write the labelling guide before labelling anything. Decide the ambiguous cases in advance — what counts as urgent, how to handle two issues in one sentence, what to do with an empty input.
2. Have two people label the first 200 independently and measure agreement. Below roughly 85%, the guide is unclear — fix the guide, not the labels.
3. Spot-check 10% of the rest.
4. De-duplicate on normalised text, not exact match.

If two careful humans can't agree on the right answer, the model has no chance — and neither does your evaluation.

🎯 Selection-round radar: "How would you prepare data for fine-tuning?" is asked far more often than any hyperparameter question. Cover four things: format matching production exactly including the system prompt → responses derivable from inputs, never containing facts the input didn't supply → composition that mirrors real traffic plus edge and "no answer" cases → and splits grouped by source and time so near-duplicates can't leak.

Synthetic data

Using a larger model to generate training data for a smaller one is common and legitimate — often called distillation — with two caveats you should state together.

Licensing. Many providers restrict using their outputs to train competing models. Check the terms; this is a real constraint, not a theoretical one.

Error inheritance. You inherit the teacher's mistakes, biases and blind spots — and unlike human labelling errors, they arrive at scale and look plausible. Synthetic data needs human review, and it works best for augmenting coverage of rare cases rather than as the whole dataset.

Privacy

Training data goes into the weights. Two things follow.

Scrub PII before training — names, phone numbers, account numbers, addresses. The task almost never needs them, and once they are in the weights they can potentially surface in outputs.

And note the asymmetry with retrieval: with RAG, deleting a document removes it from the system instantly. With fine-tuning, a deletion request is a retraining trigger. If you operate under data-deletion obligations, that is a design constraint, not a detail.

Common mistakes

• Responses containing facts the input never supplied.
• Training format that differs from what production sends.
• Random splits that leak near-duplicates into validation.
• No examples where the correct answer is a refusal or "not stated".
• Inconsistent style across examples, so the model learns a blend.
• Labelling before writing the labelling guide.
• Committing to 3,000 examples before measuring whether 300 helped.
• PII left in the data.

Quick recap

Practice Zone — PYQs from real selection rounds

Six MCQs, then two tasks: review a real training file for bugs, and plan a dataset end to end.

Q1Training dataeasy

Roughly how many examples does a typical task-specific fine-tune need?

Asked in

a10–20bHundreds to a few thousand high-quality examples for most task-specific tuning — quality and coverage matter far more than raw countcAt least ten milliondExactly 50,000

Q2Training datamedium

What does a supervised fine-tuning example usually look like?

Asked in

aA single unlabelled paragraphbA conversation: an optional system message, the user input, and the exact assistant response you want — in the same format your application will use at inferencecA vector embeddingdA prompt with no answer

Q3Training datamedium

Your dataset is 90% one category and 10% another. What is the likely outcome?

Asked in

aNo effect — models handle imbalance automaticallybThe model over-predicts the majority class and may nearly never produce the minority one, while overall accuracy still looks finecTraining will fail with an errordThe model will refuse to answer

Q4Training datahard

Why must the validation set be held out before any tuning decisions?

Asked in

aTo make training fasterbBecause a model evaluated on data it trained on will look excellent and generalise poorly — you need unseen data to measure real performance and to pick the right checkpointcIt is only a formalitydSo the data can be reused later

Q5Training datamedium

Is using a larger model to generate training data for a smaller one legitimate?

Asked in

aNever — it is always forbiddenbIt's a common technique (distillation), subject to the source model's terms of use, and it requires human review because you inherit the teacher's mistakes and biasescYes, and no review is needed since the output is high qualitydOnly for image models

Q6Training datahard

Which dataset problem is hardest to detect after training?

Asked in

aA file that fails to parsebConsistent labelling errors from one annotator — the model learns them faithfully, and they look like model failures rather than data failurescExamples that are too shortdA missing file extension

Hands-on tasks:

{"messages":[{"role":"user","content":"where is my order"},
 {"role":"assistant","content":"Your order ORD10024 is in transit
  and will arrive on 5 Sep."}]}

{"messages":[{"role":"user","content":"WHERE IS MY ORDER???"},
 {"role":"assistant","content":"Your order ORD10024 is in transit
  and will arrive on 5 Sep."}]}

{"messages":[{"role":"user","content":"refund status?"},
 {"role":"assistant","content":"Refund processed."}]}

{"messages":[{"role":"user","content":"track order"},
 {"role":"assistant","content":"I'll check that for you right
  away! Let me look into it. One moment please..."}]}

FAQ

Next lesson: the technique that made fine-tuning affordable for everyone — Lesson 4: LoRA & PEFT →