Translation memories and machine translation

André Guyon
(Language Update, Volume 9, Number 2, 2012, page 26)

For our partners at the National Research Council of Canada (NRC), there’s nothing new about what I discuss below. But as far as I know, it’s not yet on the radar of language professionals.

Machine translation will often produce sentences that are just as good as, or even better than, high-quality translation memory fuzzy matches* (with a high match percentage).

The researchers told us that the above statement applies to fuzzy matches in the 85% to 98% range, or something like that, if I recall correctly. I’ll come back to these figures later.

They had also often told us that the texts that a machine translates well are typically those for which a translation memory produces a large number of hits.

My usual reaction was: “All right, but why should I bother with machine translation if the translation memory already serves my purposes well?” If the quality of a machine translation is the same as or poorer than memory results, then sorry, but I’m really not interested!

So when they mentioned last year that in many cases, the quality is better, I realized that machine translation had gone beyond the field of pure research! In fact, it fell squarely within my mandate: try to find the optimal use of the best tools.

Was it true that translation engines could produce better draft sentences than good fuzzy matches? And if so, then how often? Was it simply because translation engines, unlike translation memories, can piece together sentence fragments that are repeated?

So I asked for examples of output from the NRC researchers, who unfortunately had not kept any data. Although I kept an open mind, I wasn’t going to spend my time trying to confirm or refute their findings.

Then, when I was looking for ways to improve our translation memories and out of curiosity compared translation memory results with output from our translation engines (developed using NRC’s translation engine, PORTAGE, and our corpora), I was very surprised:

1. I found many cases where the machine translation required less revision than the results from a gigantic translation memory I use (nearly a billion words).
2. Although the parameters were not what NRC would have recommended as ideal for a test, i.e. the translation memory and the translation engine did not contain exactly the same data, the results were still good, despite the fact that our engines are not perfectly optimized (groupings of similar texts only).
3. This was often the case even for sentences whose fuzzy factor** was 70%.

I’m not saying that translation engines produce better results in all cases. But I’m not saying the opposite either. I’m simply saying that at this time, we would be depriving ourselves of useful output if we didn’t use translation engines.

Below are a few representative examples of both cases. I have not yet compiled statistics and don’t really plan on doing so for the time being. I’ve seen enough cases where the machine translation output was as good as, if not better than, the memory results.

I was even able to verify that combining segments using a simple algorithm in the memory would not always produce better results either.

Here’s a case where the memory produced a better result than our engines, but note that the engine also produced excellent output.

Sentence to be translated:

In the 3rd quarter of 2011-2012, the performance of Real Property projects over $30M is close to the Performance Measure Target of 90% of the projects meeting or exceeding their approved project parameters.

This is the closest match that the gigantic translation memory found (in the left column, I’ve highlighted the differences, and in the right column, what I would change):

English	French
In the 2nd quarter of 2011-2012, the performance of Real Property projects over $30M is in line with the Performance Measure Target of 90% of the projects meeting or exceeding their approved project parameters.	Pour le deuxième trimestre de 2011-2012, le rendement des projets immobiliers de plus de 30 M$ est conforme à l’objectif de mesure du rendement voulant que 90 % des projets respectent leurs paramètres approuvés ou les dépassent.

This is what the Bureau’s translation engine (developed using PORTAGE) produced (what I would change to make the translation correct is highlighted):

Au cours du 3^e trimestre de 2011-2012, l’exécution de projets immobiliers de plus de 30 M est près de la mesure du rendement cible de 90 % des projets en respectant ou en dépassant les paramètres de projet approuvés.

And here’s the Google Translation output:

Au 3ème trimestre de 2011-2012, la performance des projets immobiliers sur 30 millions de dollars est proche de la cible Mesure du rendement de 90% des projets atteignent ou dépassent leurs paramètres des projets approuvés.

Now, let’s look at an example where the machine translation would require fewer changes than the memory result:

Four of the ‘At Risk’ projects have had no change in overall performance rating since September 30th, 2011.

These are the two closest hits found by the translation memory:

English	French
These three ‘At Risk’ projects have had no change in overall performance rating since March 31st 2011.	La note de rendement de ces trois projets, qui sont considérés comme étant « à risque », n’a pas changé depuis le 31 mars 2011.
Three of the ‘At Risk’ projects have had no change in overall performance rating since June 30th, 2011.	La cote de rendement de trois projets considérés comme étant « à risque » n’a pas changé depuis le 30 juin 2011.

This is the Bureau’s translation engine output:

Quatre des projets « à risque » n’ont aucun changement dans la cote de rendement globale depuis le 30 septembre 2011.

All you’d have to do to the above sentence is add the word connu between the two highlighted words—it’s that easy. This therefore seems like the “best” solution to me, although I know I’m no specialist in the field.

And here’s the Google Translation output:

Quatre «à risque» des projets n’ont pas eu changement de notation de la performance globale depuis Septembre 30th, 2011.

This final example seriously impressed me:

Moreover, the revised Client Feedback Questionnaire (CFQ) includes the eight core questions of the Common Measurement Tool (CMT) recommended by the Treasury Board Secretariat.

This is the Bureau’s translation engine output:

De plus, la version révisée du questionnaire de rétroaction des clients (CFQ) comprend les huit questions fondamentales de l’outil de mesures communes (OMC) recommandée par le Secrétariat du Conseil du Trésor.

Here’s the Google Translation output:

En outre, le questionnaire du client révisé Commentaires (CFQ) comprend les questions fondamentales huit de l’outil de mesures communes (OMC) recommandés par le Secrétariat du Conseil du Trésor.

And this is what I found using the translation memory:

English	French
In addition, the Client Feedback Questionnaire (CFQ) is being revised to include the eight core questions of the Common Measurement Tool (CMT) recommended by the Treasury Board Secretariat.	De plus, nous sommes à réviser le questionnaire de rétroaction de la clientèle afin d’y inclure les huit questions fondamentales de l’Outil de mesures communes (OMC) recommandées par le Secrétariat du Conseil du Trésor (SCT).

Although I’m no specialist in the field, I can tell that the sentence produced by the translation engine is nearly perfect.

And there were more like it, many more.

• Back to remark 1* For a given sentence to be translated, a source sentence already translated that contains 80% or 90% of the same words approximately in the same position.
• Back to remark 2** 70% of the words in the sentence to be translated are the same as the words of an already translated sentence in source language.

© Services publics et Approvisionnement Canada, 2024
TERMIUM Plus^®, la banque de données terminologiques et linguistiques du gouvernement du Canada
Outils d'aide à la rédaction – Favourite Articles
Un produit du Bureau de la traduction