How to Fine-Tune Sarvam-30B on Your Own Enterprise Data: A Practical Guide for European Teams

Step 1: Decide Between LoRA, QLoRA, and Full Fine-Tuning

For most enterprise use cases, you do not need to fully fine-tune Sarvam-30B. Low-Rank Adaptation (LoRA) or QLoRA will get you 90 to 95 percent of the performance of a full fine-tune at roughly one-twentieth of the compute cost.

The practical decision rule is simple. Choose LoRA if you have a single L40S or a couple of A100 GPUs and your training data is fewer than around 200,000 high-quality examples. Choose QLoRA if you are running on consumer or prosumer hardware and want to keep memory under 24 GB. Choose full fine-tuning only if you have a multi-GPU cluster and a specific reason, such as catastrophic forgetting concerns or a major domain shift, to update all weights.

The economics are hard to argue with. As Thomas Wolf, Chief Science Officer at Hugging Face in Paris, has consistently noted in public presentations, supervised fine-tuning with LoRA has become the sensible default for enterprise teams deploying open-source models precisely because the 90 percent result at five percent of the cost is almost always the right trade-off for scoped production use cases.

Step 2: Prepare Your Data Correctly

This is where most fine-tuning projects fail. The model is a commodity. The data is the moat. You want between 5,000 and 50,000 training examples, clean, deduplicated, and structured in the chat format Sarvam expects.

The format you need uses a messages array of role and content pairs. Roles should be system, user, and assistant. Set the system prompt once per conversation, keep content as plain text, and stay under 28,000 tokens total per example. Optional metadata labels are useful for evaluation slicing later.

A common mistake is loading raw PDFs or email threads without pre-processing. Sarvam, like any fine-tuned model, amplifies patterns in your training data. If your data contains inconsistent formatting, conflicting answers, or unresolved ambiguity, the fine-tuned model will reproduce those defects at inference time. European enterprise datasets frequently carry additional complexity: multilingual content, GDPR-redacted fields, and legacy document formats. Budget time for this cleaning step. It is the unglamorous work that determines whether the fine-tune succeeds.

Step 3: Set Up the Training Environment

The most reliable environment is a Hugging Face Transformers 4.45 or later stack with PEFT, TRL, accelerate, and bitsandbytes for QLoRA.

pip install transformers>=4.45 peft trl accelerate bitsandbytes datasets
pip install vllm  # for inference later

Download the Sarvam-30B weights from Hugging Face. For teams running on European cloud infrastructure, whether on OVHcloud, Hetzner, or a hyperscaler region inside the EU, Hugging Face Hub is the pragmatic default. Verify checksums before you start training, particularly if your organisation has supply-chain security requirements under frameworks such as ISO 27001 or the EU Cyber Resilience Act.

Step 4: Configure the LoRA Adapter

Keep adapter rank modest at first. A rank of 16 or 32 with alpha equal to two times the rank is a sensible starting point. Target the attention projection layers (q_proj, k_proj, v_proj, o_proj) and the MLP projection layers (gate_proj, up_proj, down_proj).

The single most important hyperparameter for LoRA stability is learning rate. Start at 1e-4 with cosine decay and linear warmup for the first three percent of steps. If loss diverges, halve the rate. Researchers at ETH Zurich's Data Analytics Lab have published extensively on learning rate sensitivity in parameter-efficient fine-tuning, and the consensus is consistent: warmup schedule matters more than most practitioners expect, and rank is far less decisive than learning rate choice.

Step 5: Monitor Training Runs Like a Professional

Track three things per epoch: training loss, validation loss, and a small held-out benchmark suite that reflects your actual production requirements. Do not rely only on loss curves. A fine-tune can show falling loss while silently degrading on out-of-distribution reasoning because the model is over-fitting to your data's style.

Build a small evaluation set of 100 to 300 examples representing real production queries. Score on this set every epoch. If it degrades for two consecutive epochs while training loss still falls, stop training. This discipline is especially important for regulated sectors: a model that performs well on loss curves but poorly on domain-critical edge cases is a compliance risk, not just a technical disappointment.

Step 6: Deploy Inference That Actually Scales

Once the adapter is trained, merge it back into the base model or serve it as a hot-swappable adapter using vLLM with LoRA support. Sarvam-30B's 2.4 billion active parameters mean you can run roughly 8 to 16 concurrent requests on a single L40S at enterprise latency targets.

For multi-tenant deployments, vLLM's adapter hot-swapping allows you to serve multiple fine-tuned versions from a single base-model instance. That architecture suits European financial services firms that need to isolate model versions per regulatory entity or per jurisdiction, serving, for instance, a UK FCA-compliant variant and an EU AI Act-compliant variant from the same base.

Set your production SLO before you deploy. A reasonable target for interactive chat workloads is a p95 first-token latency under 500 ms and inter-token latency under 80 ms. Background document-processing workflows can tolerate under one second first-token. Fix these numbers before deployment, not after your first incident review.

Step 7: Evaluate and Iterate

Build a comprehensive evaluation suite that includes:

• Your production benchmark set (100 to 300 examples).
• A safety and refusal set, including adversarial prompts in your production languages.
• A regression set covering tasks the base Sarvam-30B already handles well, to detect catastrophic forgetting.
• A language quality set for any European languages your users communicate in, including languages that are lower-resourced in most LLM training corpora.

Score your fine-tuned model against the base Sarvam-30B on all four dimensions. The fine-tune should win on your production set, tie or win on safety, tie on regression, and tie or win on language quality. Any pattern outside this suggests a problem in your training data, not your training configuration.

Step 8: Documentation and Governance

For regulated sectors, document model provenance, training data sources, fine-tune configuration, evaluation methodology, and deployment approvals. This is no longer optional hygiene: it is increasingly a legal requirement. The EU AI Act, which applies to high-risk AI systems deployed in the EU from August 2026, requires technical documentation of training data, model architecture, and performance metrics. The UK's AI Safety Institute has published analogous guidance for frontier and fine-tuned models used in critical national infrastructure.

Kris Shrishak, a technology policy adviser who has briefed the European Parliament's AI Committee on model governance, has argued publicly that organisations fine-tuning open-source models on proprietary data need to treat the resulting weights as a regulated artefact equivalent to a software product, complete with version control, change logs, and audit trails. That framing is the right one for 2026 European deployments.

Build a standardised documentation template and apply it to every fine-tune run. The cost of doing this retrospectively is always higher than doing it from day one.

Common Pitfalls to Avoid

• Treating fine-tuning as a substitute for retrieval. Most enterprise use cases are better served by retrieval-augmented generation with a smaller prompt budget than by fine-tuning on private knowledge. Fine-tune for style, tone, and task format; use retrieval for facts.
• Ignoring evaluation tooling. Teams that set up proper evals in week one ship better models than teams that spend month one training and month two discovering their model regressed on key tasks.
• Training on data that cannot be disclosed. If your training data contains personal data, customer conversations, or anything regulatory-sensitive under GDPR, you need to either fully anonymise it or treat the fine-tuned weights as a regulated artefact with restricted access.
• Underestimating inference cost. Fine-tune once, serve millions of times. Optimise serving before optimising training. The GPU hours you spend shaving 10 percent off your training run are almost never worth more than the GPU hours you save by quantising your serving stack properly.