Use case · Fine-tuning (SFT + LoRA)
Ecommerce and retail
Retail chatbots handle everything from product discovery and sizing to order tracking, returns, and promotions, and have to answer in a consistent brand voice.
The experiment
Fine-tune a small open-source chat model to answer retail questions more consistently, in a fixed support style.
- Dataset
- Bitext retail and ecommerce chatbot training dataset.
- Agent
- A fine-tuned causal LLM trained with SFT and LoRA (PEFT) as a retail support chatbot.
- Knobs tested
- Prompt formatLoRA rankTarget modulesBase model size
What the experiment showed
- 01Giving the model enough adaptation room (LoRA rank and module coverage) was the biggest driver of improvement.
- 02Prompt and format changes alone were not a guaranteed win with very little data.
- 03With small eval sets, treat results as directional and check tone, correctness, and format by hand.
How RapidFire helped
Every configuration in one run
Compared the knobs side by side on the same hardware, instead of one run after another.
Live results
Scored each version on slices of the eval set as it ran, so decisions came early.
Weak runs stopped early
Cut directions that were not working and put the compute behind the ones that were.
Winners cloned
Cloned the strongest run and changed one factor at a time to find what drove the gain.
Apply it to your data
- 1Turn tickets, macros, and help articles into question and best-answer pairs.
- 2Start from a lightweight base model and sweep format, LoRA capacity, target modules, and model size.
- 3Pick winners on held-out evals plus human review for helpfulness, policy, tone, and refusals.