Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO

📝 Paper Summary

Small Language Models Multilingual Alignment Code-Switching Mitigation

PureTC-1B stabilizes Traditional Chinese generation in small language models by eliminating unintended code-switching through a parameter-efficient pipeline of continual pre-training, supervised fine-tuning, and direct preference optimization.

Core Problem

Open-weight Small Language Models (SLMs) frequently exhibit token-level instability in Traditional Chinese (TC), unpredictably emitting Simplified Chinese, English, or other languages.

Why it matters:

Unintended code-switching undermines the reliability required for professional AI applications in law, government, and education
Model scale alone does not guarantee linguistic fidelity; even state-of-the-art open-source models suffer from spontaneous foreign-language token generation
Prompt engineering is insufficient to prevent this instability, necessitating robust fine-tuning solutions that fit on commodity hardware

Concrete Example: Under a Traditional Chinese prompt, a standard open-source SLM might spontaneously insert English words or Simplified Chinese characters into its response, compromising the linguistic purity required for official documents.

Key Novelty

Adapter-Only Stabilization Pipeline (PureTC-1B)

Continual Pre-Training (CPT) shifts the model's language priors toward Traditional Chinese using broad and domain-specific corpora
Supervised Fine-Tuning (SFT) aligns the model to follow instructions while enforcing single-language output
Direct Preference Optimization (DPO) directly penalizes code-switched outputs by optimizing preferences for linguistically pure responses

Architecture

A conceptual text-based flowchart of the three-stage overall fine-tuning flow (CPT, SFT, DPO) applied to the base model using LoRA adapters.

Evaluation Highlights

Reduces non-Traditional Chinese contamination (Other-Language Rate) by 51.3% relative (micro-average) compared to the Llama-3.2-1B-Instruct base model
Increases strict perfect-purity rate (Pass@TC) from 9.5% to 29.9% (+20.4 pp) on the macro-average benchmark
Outperforms larger baselines on Named Entity Translation, reducing incorrect-language tokens by 77.2% relative to Llama-3B and 57.2% relative to Qwen-1.5B

Breakthrough Assessment

7/10

Provides a highly practical, hardware-friendly recipe for stabilizing specific language generation in SLMs without full-parameter retraining, though noise robustness slightly degrades.

⚙️ Technical Details

Problem Definition

Setting: Text generation enforcing strict character-level adherence to Traditional Chinese without unintended code-switching

Inputs: Instruction and input text in Traditional Chinese (or mixed languages for translation/purge tasks)

Outputs: Linguistically pure Traditional Chinese text

Pipeline Flow

Continual Pre-Training (CPT)
Supervised Fine-Tuning (SFT)
Direct Preference Optimization (DPO)

System Modules

CPT Adapter

Shifts the model's language priors toward Traditional Chinese (TC) using general and specialized corpora

Model or implementation: LoRA adapter on Llama-3.2-1B-Instruct (rank=512, alpha=1024)

SFT Adapter

Aligns the model with specific instruction formats ensuring contextually faithful and structured outputs

Model or implementation: LoRA adapter continuing from CPT (rank=512, alpha=1024)

DPO Adapter

Optimizes preference for linguistically pure TC over semantically equivalent answers with code-switching

Model or implementation: Low-rank LoRA adapter on SFT-merged base (rank=12, alpha=24)

Novel Architectural Elements

Sequential LoRA adapter merging strategy: high-rank CPT/SFT adapters are progressively merged into the base model before training a thin (low-rank) DPO adapter, preserving capabilities while steering preferences

Modeling

Base Model: meta-llama/Llama-3.2-1B-Instruct

Training Method: Sequential CPT, SFT, and DPO using LoRA

Objective Functions:

Purpose: Shift language priors toward Traditional Chinese.

Formally: Causal language modeling objective on frozen base model.
Purpose: Optimize preference for linguistically pure outputs over code-switched answers.

Formally: Direct Preference Optimization (DPO) loss.

Adaptation: LoRA (rank=512 for CPT/SFT, rank=12 for DPO)

Training Data:

CPT: 1.78M documents (2B tokens) from Taiwan-Text-Excellence-2B + 300MB legal judgments
SFT: 1,500 synthetic examples generated via GPT-4-mini across 5 tasks
DPO: Pairwise preference corpus derived from the 5 SFT tasks (GPT-4-mini pure TC vs CPT+SFT model rejected outputs)

Key Hyperparameters:

learning_rate: 6e-6 (CPT step 1), 1e-5 (CPT step 2), 7e-7 (SFT), 6e-6 (DPO)
batch_size: 1 (per-device with gradient accumulation)
lora_dropout: 0.4

Compute: Trainable on a single 12 GB consumer GPU in bf16 precision

Comparison to Prior Work

vs. CSCL: Focuses on eliminating code-switching entirely for strict monolingual adherence rather than deliberately incorporating it to improve cross-lingual transfer
vs. Synthetic code-switched fine-tuning: Uses DPO to actively penalize code-switched outputs rather than training on artificially mixed datasets to elevate low-resource performance
vs. Llama-3B / Qwen-1.5B baselines: Achieves higher strict purity through a targeted stabilization pipeline despite a smaller parameter count

Limitations

Degraded performance on Noise Robustness tasks, as the fine-tuned system tends to over-translate noisy inputs (URLs, code, emojis) instead of ignoring them
Strict elimination of all English tokens may penalize valid usage of technical acronyms or proper nouns necessary in professional applications
Evaluation dataset is imbalanced across task families, requiring dual reporting of macro and micro averages
Larger-scale baselines (7B+ models) were not evaluated due to hardware constraints
No statistical significance tests reported for the benchmark gains

Reproducibility

Model weights and datasets are published on Hugging Face (referenced in paper). Training scripts are not explicitly provided, but the paper states it uses the open-source LLaMA-Factory framework. GPT-4-mini was used for synthetic SFT/DPO data generation.

📊 Experiments & Results

Evaluation Setup

Zero-shot text generation under fixed deterministic decoding settings (Temperature 0.2, Top-p 0.9, Max New Tokens 1024)

Benchmarks:

Custom 605-prompt TC Stability Benchmark (Translation, bilingual purge, conversion, structured JSON, summarization, noise robustness) [New]

Metrics:

Other-Language Rate (OLR)
Pass@TC
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Macro-average and micro-average results demonstrate that PureTC-1B significantly reduces contamination and increases strict purity compared to the base model.
Custom 605-prompt TC Stability Benchmark (Macro-Avg)	Other-Language Rate (OLR)	0.214	0.098	-0.116
Custom 605-prompt TC Stability Benchmark (Macro-Avg)	Pass@TC	9.5	29.9	20.4
Custom 605-prompt TC Stability Benchmark (Micro-Avg)	Other-Language Rate (OLR)	0.231	0.104	-0.127
Task-specific comparisons show PureTC-1B outperforming even larger baselines on stability metrics like Named Entity Translation.
Named Entity Translation Task	Other-Language Rate (OLR)	0.250	0.057	-0.193
Named Entity Translation Task	Other-Language Rate (OLR)	0.133	0.057	-0.076

Main Takeaways

PureTC-1B substantially improves Traditional Chinese (TC) stability at the 1B scale, reducing contamination by over 50% and tripling the strict-purity rate
Gains are robust across 7 out of 8 task families, with Noise Robustness being the only task where the model regressed (by over-translating noise)
Linguistic stability is not solely a function of scale; the specialized 1B model outperforms larger open-weight baselines (Llama-3B, Qwen-1.5B) on most stability-oriented tasks

📚 Prerequisite Knowledge

Prerequisites

Basic understanding of language model fine-tuning
Familiarity with parameter-efficient training methods like LoRA
Understanding of human preference alignment algorithms

Key Terms

SLMs: Small Language Models—compact AI models (around 1B to 3B parameters) designed for cost-effective, on-device deployments

TC: Traditional Chinese—the primary script targeted for stable generation in this paper

code-switching: The spontaneous generation of foreign-language tokens (e.g., English or Simplified Chinese) within an otherwise Traditional Chinese response

OLR: Other-Language Rate—a metric quantifying the fraction of characters in the generated string that are not compliant with the Traditional Chinese-only policy

Pass@TC: A strict metric measuring the frequency of perfectly clean outputs (where OLR exactly equals 0)

LoRA: Low-Rank Adaptation—a parameter-efficient fine-tuning method that trains small adapter modules while keeping the base model frozen

CPT: Continual Pre-Training—further training a pre-trained model on domain-specific or language-specific text to shift its internal priors

SFT: Supervised Fine-Tuning—training a model on specific instruction-output pairs to teach it how to respond appropriately

DPO: Direct Preference Optimization—an algorithm that aligns language models to human preferences by directly penalizing rejected outputs and rewarding preferred ones without a separate reward model

NFKC normalization: A Unicode standard process that standardizes character forms, such as converting full-width digits to standard ASCII digits