Design a machine translation system for a product like Google Translate: text translation between 100+ language pairs, from high-resource pairs like English-Spanish down to low-resource languages with little parallel data, served interactively at web scale where users expect answers in a fraction of a second.
How would you design this system? Cover the model architecture, how you get training data for low-resource languages, the serving design, and how you measure translation quality.
The Problem: 100 languages mean roughly 10,000 directed pairs, most with almost no parallel data, and users still expect every pair to answer in milliseconds. Design one system that covers them all.
Answer
The design is a single massively multilingual encoder-decoder model trained on mined bitext plus back-translated monolingual text, with a target-language token steering generation (the NLLB recipe), behind a dual-track serving layer: a fast NMT track for short, latency-critical, terminology-sensitive text, and a fine-tuned translation LLM for long, nuanced, document-level input, which is exactly the split Google shipped in its Cloud Translation API (2024) and DeepL shipped as its next-gen model. The two pivotal decisions are the multilingual pooling (high-resource pairs teach low-resource ones through a shared representation, enabling zero-shot pairs) and the honest admission that one model family does not win everywhere: NMT still wins on latency, cost, and determinism, the LLM wins on context and fluency.
(1) One Multilingual Model: shared encoder-decoder with a target-language token; transfer from high-resource pairs gives zero-shot coverage of unseen pairs instead of 10,000 separate models.
(2) Data Flywheel: mine bitext from web crawls with multilingual sentence embeddings (LASER), then back-translate monolingual text to synthesize pairs for low-resource languages.
(3) Dual-Track Serving: NMT for short/interactive/terminology-critical requests, translation LLM for long-form and idiomatic text; a router picks per request.
(4) Quality Measurement: automatic metrics (COMET, BLEU) for regression, human MQM error marking for truth, A/B tests for product impact, all reported per language pair.
(5) Low-Resource Strategy: zero-shot transfer first, then few-example adaptive conditioning (Google’s Adaptive Translation shows as few as 5, and up to 30k, example pairs can steer output at inference time).
Figure 1: Dual-track serving over one multilingual training base: the router sends short, latency-critical text to NMT and long, context-heavy text to the translation LLM; terminology constraints apply on both tracks.
Clarify Before Designing:
(1) Scope: text only, or also speech and image translation; and which language pairs at launch?
(2) Latency Budget: interactive typing (sub-300 ms) versus document translation (seconds) imply different model families.
(3) Quality Bar: gist-level understanding for users, or publication-grade output with human post-editing?
(4) Terminology: must product names, legal terms, or brand glossaries be honored exactly?
(5) Customization: do enterprise customers need domain adaptation (medical, legal) with their own example pairs?
Leave a Reply