You are building a writing-assistance product like Grammarly. As a user types in a browser extension, email client, or mobile keyboard, the system underlines mistakes and offers corrections in near real time: spelling, punctuation, subject-verb agreement, tense, and for premium tiers, style and clarity rewrites.
How would you design this system? Cover the model architecture, the latency budget, the training data, and how you evaluate suggestion quality.
The Problem: suggestions must arrive faster than the next keystroke, and a wrong correction costs more trust than a missed error. Design the system behind the wavy underline.
Answer
The design is a model ladder behind a per-sentence router: cheap deterministic rules and dictionaries catch spelling and punctuation on-device, an edit-tagging Transformer (one parallel pass over the tokens emitting KEEP, DELETE, INSERT, REPLACE tags) handles most grammatical error correction in the cloud, a seq2seq model takes the sentences that need reordering, and an LLM serves style and clarity rewrites only on demand. The two pivotal decisions are routing each sentence to the cheapest model that can fix it (latency is a product feature), and tuning every error type to a precision-first operating point because false corrections erode trust faster than misses.
(1) Error Taxonomy Drives Architecture: mechanical errors (spelling, punctuation, agreement) and open-ended style rewrites are different problems and deserve different models.
(2) Model Ladder With Routing: rules → edit-tagger → seq2seq GEC → LLM, cheapest first; the router reads error type, confidence, and latency budget.
(3) Precision-First Operating Points: per-error-type thresholds tuned on F0.5, since a wrong correction damages trust more than a miss.
(4) Latency Engineering: score only the current sentence, cache unchanged text, and keep the inline path under 100 ms end to end.
(5) Personalization & Dialect: locale (US/UK), register (casual vs formal), and per-user dismiss history modulate which suggestions fire.
Figure 1: The inline path: each finished sentence is routed to the cheapest sufficient model, and a merger dedupes overlapping suggestions before rendering underlines. Unchanged text is served from cache.
Clarify Before Designing:
(1) Scope: which languages and locales, and is style or clarity rewriting in scope or only mechanical correctness?
(2) Latency Budget: how many milliseconds for inline suggestions, and is there an async path for long-form checks?
(3) Deployment: what runs on-device versus in the cloud, given browser-extension and mobile constraints?
(4) Privacy: may user text leave the device, and are there regulated contexts (legal, medical) to special-case?
(5) Personalization: can we learn from per-user accept and dismiss behavior, and where may that profile live?
Leave a Reply