rapidfire.

LLM Fine-Tuning Cost: What You'll Actually Pay in 2026

Written By:

Matt Tanner

Published on

For an illustrative LoRA fine-tune of Llama 3.1 8B, assume 4,440,705 training tokens per epoch and three epochs. At Together AI's November 13, 2026 rate, the training charge works out to $4.53. Count the configurations you tried and threw away, the evaluation runs, and a GPU serving the result around the clock, and the first month of that same project passes $4,000. In that budget, the training run is the smallest line.

This guide prices both sides of LLM fine-tuning with rates pulled on November 13, 2026, shows the math behind every number, and ends with a formula you can copy. If your bigger worry is per-request spend after launch, our guide to LLM cost optimization covers the inference side.

How much does LLM fine-tuning cost?

As of November 2026, hosted APIs charge $0.34 to $7.99 per million training tokens for 8B to 70B open models. Rented 48 GB to 80 GB GPUs cost $1.09 to $8 per GPU-hour. In our worked examples, an 8B LoRA run costs $2 to $20, a 70B fine-tune $81 to $320, and a month of serving costs more.

The cost of fine-tuning an LLM splits three ways depending on how you buy compute:

Approach

What you pay for

Range today (Nov 2026)

Our 8B or 70B example

Hosted API (Together AI, Fireworks, AWS Bedrock)

Training tokens ร— epochs, with per-job minimums

$0.34 to $7.99 per 1M tokens

8B LoRA: $4.53 to $19.85

LoRA or QLoRA on rented GPUs

GPU-hours, billed while the machine is up

$1.09 to $8 per GPU-hour

8B LoRA on one A100: $1.98 to $6.23

Full fine-tuning

About 18 bytes of GPU memory per parameter, so more GPUs or a pricier API tier

70B self-hosted: at least 16 GPUs, $63.84/hr on demand

70B, 40M tokens: $89.60 on Together AI

Each figure comes from the provider tables and scenarios below. The ranges are wide because the method, the model size, and the provider each move the price by several multiples.

What drives the cost

Six inputs set the training cost of a fine-tune, and they multiply rather than add:

  • Model size. Per-token API prices step up with parameter count. Together charges $0.34 per million tokens for LoRA on Llama 3.1 8B and $2.03 on Llama 3.3 70B. On your own GPUs, LLM training compute scales with parameters too: roughly 6 FLOPs per parameter per token (Kaplan et al.).

  • Method. Full fine-tuning updates every weight. Parameter-efficient fine-tuning methods like LoRA train small adapter matrices on a frozen pre-trained base, which cuts memory far more than it cuts compute.

  • Tokens, not examples. Billing counts tokens, and a system prompt repeated in every example is part of every example, so you pay for it every epoch. For illustration, assume a classifier's repeated system prompt makes each example average 130.5 tokens instead of 35.5. That's about 3.7 times the token usage, though a per-job minimum can leave the actual bill unchanged. Count your own formatted examples with the target model's tokenizer before budgeting.

  • Epochs and evaluations. Each epoch reprocesses the whole dataset, and Together also bills validation tokens each time it evaluates.

  • GPU type and hours. The cheapest GPU per hour isn't always the cheapest per job. In the fine-tuning cost model from Xia et al., the same Mixtral job was estimated at $32.70 on an A40 billed at $0.79 an hour and $17.90 on an H100 at $2.10, because the H100 fit batches of 17 instead of 4 and processed nearly five times as many queries per second.

  • Number of experiments. Few teams ship the first configuration they try. Eight candidates means roughly eight times the training cost unless you stop the weak ones early.

The last two inputs are the ones LLM development teams underestimate. Picking the base model is its own decision, covered in our guide to LLM model selection, but every candidate you keep in play multiplies the fine-tuning jobs you'll pay for.

Hosted API fine-tuning pricing

Hosted fine-tuning bills per training token. The last column shows what a 10M-token job costs at each provider's listed rate, pulled November 13, 2026:

Provider and model

Method

Price per 1M training tokens

Cost for 10M tokens

Together AI, Llama 3.1 8B

LoRA / full

$0.34 / $0.38

$3.40 / $3.80

Together AI, Llama 3.3 70B

LoRA / full

$2.03 / $2.24

$20.30 / $22.40

Fireworks, models up to 16B

LoRA / full

$0.50 / $1.00

$5.00 / $10.00

Fireworks, 16.1B to 80B

LoRA / full

$3.00 / $6.00

$30.00 / $60.00

AWS Bedrock, Llama 3.1 8B / 70B, US West (Oregon)

Customization

$1.49 / $7.99

$14.90 / $79.90

OpenAI, gpt-4.1-mini / gpt-4.1

Existing users only

$5.00 / $25.00

$50.00 / $250.00

Three billing rules matter more than the headline rate:

  1. Per-job minimums. Every Together job is billed at least $4.00 for these models. At $0.34 per million, a LoRA job on Llama 3.1 8B has to process 11.76M tokens before the minimum stops applying, so small experiments pay $4 each no matter how few tokens they use.

  2. Packing. Together concatenates short examples into full-length windows by default. Turn packing off, and each example is billed at max_seq_length tokens, padding included. For a dataset averaging 165 tokens per example, a 2,048-token window bills more than 12 times the real tokens.

  3. Stopped and failed jobs. Canceled or early-stopped Together jobs pay for completed steps only, and jobs that fail on Together's side or on invalid input are fully refunded. The pricing docs also describe a CLI and API estimate you can run before launching.

OpenAI is winding down self-serve fine-tuning. It's closed to new organizations. Since July 2, 2026, job creation is limited to orgs that ran inference on a fine-tuned model in the past 60 days. On January 6, 2027, active customers lose job creation too. Existing fine-tuned models keep serving until their base model is deprecated, and the deprecation schedule lists GPT-4o, GPT-4.1, and their mini versions for January 14, 2027. If you're starting now, budget on an open model.

The inference markup shows up more as a change in billing model than as a higher per-token rate. Together serves fine-tuned adapters on dedicated endpoints billed by the minute, not through the per-token serverless API. Bedrock prices inference on a custom Llama 3.1 model at $24.00 per model unit per hour without a commitment. Fireworks doesn't add a charge for serving a trained model, but you pay for the deployment's GPU time.

Renting GPUs: LoRA vs. QLoRA vs. full fine-tuning

On rented hardware, the method decides how many GPUs you need, and the GPU count times the hourly rate is your burn rate. Memory per parameter is the quickest way to size it. Mixed-precision full fine-tuning with Adam needs about 18 bytes per parameter (6 for weights, 8 for optimizer states, 4 for gradients) before activations, per Hugging Face's memory breakdown.

LoRA can keep the frozen base in 16-bit (2 bytes per parameter) and train only the adapters. QLoRA quantizes the frozen base to 4 bits, which lets its authors fine-tune a 65B model on a single 48 GB GPU.

Method, Llama 3.1 8B

Memory before activations

Example setup

Rate (Nov 2026)

QLoRA

~4 GB base + adapters

One 48 GB GPU (Runpod L40S)

$1.09/hr

LoRA

~16 GB base + adapters

One 80 GB GPU (Runpod A100 SXM)

$2.19/hr, from $1.39 on Community Cloud

Full fine-tuning

~144 GB

Several 80 GB GPUs, such as one Together H100 node

8 ร— $3.99 = $31.92/hr on demand

Billing granularity changes the math. Runpod bills pods per second. Together's GPU clusters come in whole nodes of 8 GPUs, and on-demand clusters bill hourly until you delete them.

A worked example: hours ร— rate

Take the illustrative LoRA job from the intro: Llama 3.1 8B, an assumed 4,440,705 tokens per epoch, three epochs, one A100 80 GB.

  1. Compute: 6 ร— 8B parameters ร— 13.32M tokens = 6.4 ร— 10^17 FLOPs. That uses the full fine-tuning rule, so it errs on the high side for LoRA, which skips most weight gradients. The LoRA paper measured a 25% training speedup over full fine-tuning on GPT-3 175B.

  2. Throughput: NVIDIA rates the A100 at 312 dense BF16 TFLOPS. The share of that peak a job reaches is its model FLOPs utilization (MFU). PaLM's 540B pre-training run reported 46.2%. We assume 20% to 40% for a short-sequence fine-tune.

  3. Hours: 6.4 ร— 10^17 รท (312 ร— 10^12 ร— 0.40) รท 3,600 = 1.42 hours at 40% MFU, or 2.85 hours at 20%.

  4. Cost: 1.42 to 2.85 hours ร— $2.19 = $3.12 to $6.23 on Runpod's Secure Cloud, or $1.98 to $3.96 at the $1.39 Community Cloud rate.

That lands close to Together's $4.53 for the same tokens. The difference is that the API price is known before you start, while the GPU estimate depends on an MFU you only learn by running the job.

Costs people forget

The training line is the easiest to estimate and often the smallest. The rest of the LLM development budget hides in these:

  • Data collection, cleaning, and labeling. Engineer or annotator hours spent writing examples, fixing formats and checking labels. In the example below, it's the second-largest line of month one.

  • Evaluation. Held-out generation runs after every candidate, plus validation tokens on hosted APIs. On Together, 1,000 validation examples at 165 tokens each, evaluated 10 times, add 1.65M billed tokens.

  • Failed runs and hyperparameter sweeps. Every configuration you discard was paid for in full unless you stopped it early.

  • Storage. Bedrock charges $1.95 per custom model per month. Runpod charges $0.07 per GB per month for standard network storage under 1 TB, and a Together cluster's shared volume keeps billing after you delete the cluster. LoRA helps here too: the LoRA paper's GPT-3 checkpoint shrank from 350 GB to 35 MB.

  • Serving the tuned model. A dedicated H100 on Together costs $5.49 an hour, which Together's docs put at roughly $3,950 for a 30-day month of continuous running. Replicas scaled to zero cost nothing, and Fireworks scales idle deployments to zero after an hour by default. If you plan to run the model on your own hardware, our guide to the self-hosted LLM decision walks through those costs.

  • Retraining. Data drifts, and pre-trained base models get deprecated, so plan on repeating the LLM training and evaluation lines over the long term.

For the 8B LoRA project from the worked example, with a sequential eight-configuration sweep at 20% MFU and one month of serving, the lines stack up like this:

[Image placeholder: Single stacked horizontal bar titled "Month-one budget, 8B LoRA project (estimate, Nov 13, 2026 prices)". Segments left to right with dollar labels and share: "Data prep (assumed 20 h ร— $75): $1,500 (27.2%)", "Training runs kept, 2 finalists: $12.47 (0.2%)", "Failed or abandoned runs, 6 of 8: $37.40 (0.7%)", "Evaluation (assumed 2 GPU-hours): $4.38 (0.1%)", "Serving, Together 1ร— H100 for 30 days: $3,952.80 (71.8%)". Total label at right end: "$5,507.05". Callout arrow pointing at the three tiny middle segments: "Training, abandoned runs and evals: 1.0% of the month". Alt text: Stacked bar showing where a month-one fine-tuning budget goes: serving 71.8%, data prep 27.2%, training, failed runs and evaluation about 1%.]

The two assumed lines are placeholders for your own rates. Training and abandoned runs come to 0.9% of the month here, which flips at larger scale: a single abandoned 70B run costs more than this whole sweep.

Worked scenarios

These are estimates built from the stated assumptions and the November 13, 2026 rates above, not quotes. All three dataset sizes and token counts below are illustrative inputs, not reported tokenization measurements or completed training runs.

Small classifier

7B-class LoRA

Larger full fine-tune

Model and method

Qwen3.5 0.8B, LoRA

Llama 3.1 8B, LoRA

Llama 3.3 70B, full

Data

5,000 rows, 652,577 tokens per epoch

26,872 rows, 4.44M tokens per epoch

Assumed 20M tokens per epoch

Epochs

3

3

2

One run

$4.00 on Together (usage $0.67, minimum applies)

$4.53 on Together, $6.66 on Fireworks, $19.85 on Bedrock; $3.12 to $6.23 on one A100

$89.60 on Together; at least 16 ร— 80 GB GPUs to self-host

Eight-config sweep

$32.00

$36.24 on Together; 11.4 to 22.8 A100-hours ($24.94 to $49.87)

$716.80 on Together

Small classifier. Training compute is trivial: by the 6-FLOPs rule, three epochs take 75 to 151 seconds on one A100. On Together, the job uses 16.6% of the $4.00 minimum, so a sweep of eight costs $32 regardless of how short you make the data. On a rented GPU, most of what you'd pay for is setup and evaluation time.

7B-class LoRA. Provider choice moves a single run by more than 4x, from $4.53 on Together to $19.85 on Bedrock, yet the whole eight-config sweep still costs less than one day of serving on a dedicated H100 ($131.76).

Larger full fine-tune. At 18 bytes per parameter, full fine-tuning a 70B model means 1,260 GB of weights, gradients, and optimizer states. That's at least sixteen 80 GB GPUs before activations, or two Together nodes at $63.84 an hour on demand. Together's $89.60 equals 1.4 hours of those two nodes.

For the same 40M tokens, LoRA costs $81.20 on Together and $120.00 on Fireworks, and Bedrock's Llama 3.1 70B customization costs $319.60. Each discarded configuration at this size costs another run's worth, which is when sweep discipline starts paying off.

How to cut fine-tuning costs

Most savings come from running fewer, shorter, cheaper jobs, and from not finishing the ones that are already losing:

  • Start with a smaller model. By the 6-FLOPs rule, a 0.8B model needs a tenth of the training compute per token of an 8B one, though some APIs, Together included, charge the same per-token rate for both. Our SLM vs. LLM comparison covers when small models hold up.

  • Use parameter-efficient fine-tuning before full fine-tuning. On Together, LoRA trains 0.1% to 1% of the parameters full fine-tuning would. Merged adapters add no inference latency.

  • Shorten sequences. Remove unnecessary repeated system-prompt text and keep packing on. If you turn packing off, count your formatted examples first and choose a supported max_seq_length that accommodates them without excessive padding.

  • Use early stopping. Together's early stopping needs a validation file and evaluations, and refunds the unused steps.

  • Compare configurations in parallel, and stop weak runs early. This is the lever for the sweep line.

The sweep is where fine-tuning budgets leak. In a sequential sweep, later candidates have to wait their turn before producing comparable early curves. If you let all eight finish before comparing them, you pay for every full run.

RapidFire's open-source engine changes the order of operations. Its shard-based scheduler runs many configurations concurrently, even on a single GPU, so every candidate shows early loss and eval curves on the same data. The trainer and LoRA configs are thin wrappers around Hugging Face TRL and PEFT, so you can declare learning rates or LoRA ranks as lists and get a run per combination.

From the dashboard, you stop laggards, which takes effect at the end of the current shard, and clone-modify promising runs with an optional warm start. Our post on hyperparallel SFT and lower GPU costs walks through this with Bitext customer-support data.

Here's the arithmetic for the 7B-class sweep. Say each run is three epochs with the data split into four shards, so a run stopped after its first shard has used 1/12 of a full run. Keep two finalists, stop the other six after one shard, and you pay for 2.5 run-equivalents instead of 8: 3.6 to 7.1 A100-hours ($7.79 to $15.59) instead of 11.4 to 22.8.

Switching between runs isn't free, though RapidFire caches models and adapters to keep that overhead low, and the savings depend on how early you can trust the curves.

At larger scale, the same mechanics compound. In a RapidFire case study, a Fortune 500 team fine-tuning a small model on four T4 GPUs used to manage a few dozen configurations per sweep, and full sweeps took 5 to 6 days. On the same hardware, they got to more than 2,000.

In the open-source engine, you make the stop and clone calls yourself. RapidFire Converge sits on top of the engine, reads interim metrics, and automatically sends those IC Ops. Weak versions stop, strong ones are cloned, every decision is logged with its reason, and it never exceeds the time or spend you set.

To try the manual version first, the RapidFire SFT tutorial compares four LoRA configurations of Llama 3.1 8B and Mistral 7B on 5,000 Bitext examples. It also has a lite version for GPUs under 80 GB and a Colab notebook.

Fine-tune, RAG, or prompt? Count the cost first

Before any of this, check whether you need to fine-tune at all. Prompting adds no training bill, and retrieval mostly adds indexing and longer prompts. Fine-tuning adds the training line, the evaluation line and, usually, a dedicated serving bill. It earns that when the problem is behavior, format, or a narrow task the base model keeps getting wrong, and it's the wrong tool when the model is missing facts that change. Our guide to RAG vs. fine-tuning works through that choice with examples.

Cost calculator or formula

For hosted fine-tuning jobs, the formula is:

Job cost = max(((training tokens ร— epochs) + (validation tokens ร— evaluations)) รท 1,000,000 ร— price per 1M tokens, job minimum)

For rented GPUs:

Wall-clock hours = (6 ร— parameters ร— training tokens ร— epochs) รท (peak FLOPS per GPU ร— MFU ร— GPUs) รท 3,600. Total GPU-hours = wall-clock hours ร— GPUs, then cost = total GPU-hours ร— hourly rate per GPU. This assumes the chosen MFU captures multi-GPU scaling losses.

Then multiply by the number of configurations you'll run, discounted by how early you stop the losers, to get the total cost of the experiment. In Python, with the 7B-class numbers from above:

def per_token_job(train_tokens, epochs, price_per_m, minimum=0.0,
                  val_tokens=0, n_evals=0):
    """Hosted API job: tokens processed x price, floored at the job minimum."""
    billed = train_tokens * epochs + val_tokens * n_evals
    return max(billed / 1e6 * price_per_m, minimum)


def gpu_job(params, train_tokens, epochs, peak_flops, mfu,
            n_gpus, price_per_gpu_hour):
    """Return wall-clock hours and cost; assumes MFU captures scaling losses."""
    flops = 6 * params * train_tokens * epochs
    hours = flops / (peak_flops * mfu * n_gpus) / 3600
    return hours, hours * n_gpus * price_per_gpu_hour


def experiment(cost_per_run, n_configs, losers_fraction_run=1.0, n_kept=1):
    """Sweep: kept runs finish, the rest stop after a fraction of a run."""
    return cost_per_run * (n_kept + (n_configs - n_kept) * losers_fraction_run)


# Illustrative tokens per epoch for the Llama 3.1 8B scenario
tokens = 4_440_705
# Together LoRA on Llama 3.1 8B, rate pulled Nov 13, 2026
print(round(per_token_job(tokens, 3, 0.34, minimum=4.00), 2))
# One A100 80 GB on Runpod Secure Cloud at 20% MFU
hours, cost = gpu_job(8e9, tokens, 3, 312e12, 0.20, 1, 2.19)
print(round(hours, 2), round(cost, 2))
# Eight configs run to the end vs. two kept and six stopped at 1/12
print(round(experiment(cost, 8), 2),
      round(experiment(cost, 8, 1 / 12, n_kept=2), 2))
def per_token_job(train_tokens, epochs, price_per_m, minimum=0.0,
                  val_tokens=0, n_evals=0):
    """Hosted API job: tokens processed x price, floored at the job minimum."""
    billed = train_tokens * epochs + val_tokens * n_evals
    return max(billed / 1e6 * price_per_m, minimum)


def gpu_job(params, train_tokens, epochs, peak_flops, mfu,
            n_gpus, price_per_gpu_hour):
    """Return wall-clock hours and cost; assumes MFU captures scaling losses."""
    flops = 6 * params * train_tokens * epochs
    hours = flops / (peak_flops * mfu * n_gpus) / 3600
    return hours, hours * n_gpus * price_per_gpu_hour


def experiment(cost_per_run, n_configs, losers_fraction_run=1.0, n_kept=1):
    """Sweep: kept runs finish, the rest stop after a fraction of a run."""
    return cost_per_run * (n_kept + (n_configs - n_kept) * losers_fraction_run)


# Illustrative tokens per epoch for the Llama 3.1 8B scenario
tokens = 4_440_705
# Together LoRA on Llama 3.1 8B, rate pulled Nov 13, 2026
print(round(per_token_job(tokens, 3, 0.34, minimum=4.00), 2))
# One A100 80 GB on Runpod Secure Cloud at 20% MFU
hours, cost = gpu_job(8e9, tokens, 3, 312e12, 0.20, 1, 2.19)
print(round(hours, 2), round(cost, 2))
# Eight configs run to the end vs. two kept and six stopped at 1/12
print(round(experiment(cost, 8), 2),
      round(experiment(cost, 8, 1 / 12, n_kept=2), 2))
def per_token_job(train_tokens, epochs, price_per_m, minimum=0.0,
                  val_tokens=0, n_evals=0):
    """Hosted API job: tokens processed x price, floored at the job minimum."""
    billed = train_tokens * epochs + val_tokens * n_evals
    return max(billed / 1e6 * price_per_m, minimum)


def gpu_job(params, train_tokens, epochs, peak_flops, mfu,
            n_gpus, price_per_gpu_hour):
    """Return wall-clock hours and cost; assumes MFU captures scaling losses."""
    flops = 6 * params * train_tokens * epochs
    hours = flops / (peak_flops * mfu * n_gpus) / 3600
    return hours, hours * n_gpus * price_per_gpu_hour


def experiment(cost_per_run, n_configs, losers_fraction_run=1.0, n_kept=1):
    """Sweep: kept runs finish, the rest stop after a fraction of a run."""
    return cost_per_run * (n_kept + (n_configs - n_kept) * losers_fraction_run)


# Illustrative tokens per epoch for the Llama 3.1 8B scenario
tokens = 4_440_705
# Together LoRA on Llama 3.1 8B, rate pulled Nov 13, 2026
print(round(per_token_job(tokens, 3, 0.34, minimum=4.00), 2))
# One A100 80 GB on Runpod Secure Cloud at 20% MFU
hours, cost = gpu_job(8e9, tokens, 3, 312e12, 0.20, 1, 2.19)
print(round(hours, 2), round(cost, 2))
# Eight configs run to the end vs. two kept and six stopped at 1/12
print(round(experiment(cost, 8), 2),
      round(experiment(cost, 8, 1 / 12, n_kept=2), 2))

It prints 4.53, then 2.85, 6.23, then 49.87, 15.59. Swap in your own token count, current rates, and an MFU measured from your first few hundred steps, since that one input moves the GPU estimate the most.

FAQ

Which LLM is best for fine-tuning?

The best LLM for fine-tuning is the smallest open model that passes your evaluation after training, because every cost line scales with size. Pick one your provider supports: Together's fine-tuning list includes Llama 3.1 8B, Qwen3.5 models from 0.8B up, Gemma 4, and gpt-oss. Shortlist two or three and fine-tune each on the same data before deciding.

Can LLMs be fine-tuned?

Yes. Open-weight models can be fine-tuned on your own GPUs with libraries like Hugging Face TRL and PEFT, or through hosted APIs from Together AI, Fireworks, and AWS Bedrock. OpenAI's self-serve fine-tuning is closed to new organizations and stops accepting jobs from existing customers on January 6, 2027.

How expensive are LLMs to run?

For a fine-tuned model, serving often costs more than training. A dedicated H100 runs about $3,950 for a 30-day month on Together. A no-commitment Bedrock model unit for a custom Llama 3.1 model costs $24 an hour, or $17,280 over 720 hours. Per-token serverless pricing is usually cheaper at low or bursty traffic, but on Together, fine-tuned adapters can't use it.

How much does LLM cost?

For open-weight models, what you pay for is compute: training tokens on a hosted API or GPU-hours on rented hardware. As of November 13, 2026, a minimum-size LoRA job on Together AI costs $4, and the illustrative 8B LoRA run on 13.32M tokens costs $4.53 to $19.85 across Together, Fireworks, and Bedrock. A 70B full fine-tune on 40M tokens costs $89.60 on Together.