The clearest change AI has made to everyday communication is machine translation. Neural systems read a whole sentence and produce a translation word by word, and in 2016 Google reported that its neural system cut translation errors by an average of 60 percent against its earlier phrase-based system in a human evaluation of simple sentences. Even so, the Government of Canada's Translation Bureau warns that AI translation tools make mistakes and recommends professional review.
Key takeaways
- Neural machine translation replaced phrase-based systems in major products after 2016, and the 2017 Transformer design raised benchmark scores further.
- The 60 percent error reduction reported for Google's system applied to simple, isolated sentences, not to whole documents.
- BLEU, the usual automatic score, counts matching word sequences against reference translations and cannot judge meaning on its own.
- Canada's Translation Bureau advises a professional review of AI translations and advises against putting confidential text into online tools.
- GCtranslate, the Bureau's in-house tool, runs only on the Government of Canada network.
How a neural translation system works
Older statistical systems translated by matching short phrases from large tables of examples. A neural system instead converts the source sentence into numbers and generates the target sentence one piece at a time, using the whole source sentence as context. The system described in Google's 2016 paper used a deep network with eight encoder layers and eight decoder layers, an attention mechanism that lets the decoder look back at relevant source words, and a method of splitting rare words into smaller pieces so that unfamiliar names and compounds can still be handled.
In June 2017, eight Google researchers posted "Attention Is All You Need," which dropped the older recurrent layers and relied on attention alone. On the WMT 2014 benchmark the model reached 28.4 BLEU for English to German and 41.8 BLEU for English to French, and the English-to-French model trained in 3.5 days on eight GPUs. That architecture, the Transformer, is the basis of most current language models, which is why translation and chat assistants now share so much technology.
What the 60 percent figure does and does not say
The 2016 Google paper was submitted on September 26, 2016. Its abstract says the system was competitive with the best published results on the WMT 2014 English-to-French and English-to-German benchmarks, and that in a side-by-side human evaluation on simple sentences it reduced errors by an average of 60 percent relative to Google's production phrase-based system. Three limits follow from that wording. The test sentences were simple and isolated, so long documents, idioms, and specialized vocabulary were not what was measured. The comparison was against one earlier system, not against professional translators. And the figure is an average across language pairs, so individual pairs differ.
A worked example of how BLEU scores a sentence
BLEU, introduced at the Association for Computational Linguistics meeting in 2002, compares a machine translation with one or more human reference translations by counting matching word sequences. The example below is illustrative and uses only single words.
| Item | Text |
|---|---|
| Reference translation | the cat sits on the mat |
| Machine output | the cat sat on the mat |
| Matching words | five of six (the, cat, on, the, mat) |
| Single-word precision | 5 / 6, about 0.83 |
The full metric also checks longer sequences and penalizes outputs that are too short. The weakness is visible even here: a wrong verb tense costs one word, while a translation that says the opposite with the same words could score well. Automatic scores are therefore useful for comparing systems during development, and human review is needed for deciding whether a text is safe to publish.
What Canada's Translation Bureau advises
Official-language duties make translation a practical question in Canada. The Translation Bureau's page on using artificial intelligence for translation states that AI tools make mistakes and lists consequences that include endangering health or safety, misinforming readers, prompting complaints or legal action, and violating language rights. It recommends that a professional translator review any AI translation, or translate the whole text from the start.
The page also advises against using online AI tools for confidential material, because many internet-based tools store data on servers outside Canada. For federal departments, the Bureau supports GCtranslate, which it describes as trained on Canadian data that the Bureau manages, using Government of Canada terminology, and running in a Protected B cloud environment. It translates between English and French for routine texts, is available only on the Government of Canada network, and is being rolled out gradually. The Bureau says it was developed in response to the Artificial Intelligence Strategy for the Federal Public Service 2025-2027.
A practical checklist for translated text
- Is the text routine, such as an internal note, or does it carry legal, medical, or safety content?
- Does the text contain confidential or personal information that should not go into an online tool?
- Can someone who reads both languages check names, numbers, dates, and negations?
- Is a published version required in two official languages, which brings in professional translation duties?
- Does the tool state where data is stored?
FAQ
Is machine translation as good as a human translator?
Not in general. Benchmarks and human evaluations show strong results on many sentences, but they do not cover every domain or document length, and the Translation Bureau still recommends professional review.
What does BLEU measure?
It measures overlap between machine output and reference translations, based on matching word sequences. It does not verify meaning, tone, or factual accuracy.
Is it safe to paste confidential documents into an online translator?
The Translation Bureau advises against it, noting that many online tools store data on servers outside Canada. Organizations typically follow their own privacy and security policies on this point.
Why did translation improve so much after 2016?
Neural networks with attention could use the full sentence as context, and the Transformer design later improved quality and training speed on standard benchmarks.
Related reading
More in AI Basics.
Last verified: October 2026
Sources
- Wu Y. et al., Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, arXiv, 2016
- Vaswani A. et al., Attention Is All You Need, arXiv, 2017
- Papineni K. et al., Bleu: a Method for Automatic Evaluation of Machine Translation, ACL, 2002
- Using artificial intelligence for translation purposes, Translation Bureau, Government of Canada, 2026