From Computer Assisted Translation to Human Assisted Translation. Session 3 - Machine and Human Translation: Finding the Fit?

HAL Id: hal-02497341

https://hal.archives-ouvertes.fr/hal-02497341

Submitted on 3 Mar 2020

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

From Computer Assisted Translation to Human Assisted Translation. Session 3 - Machine and Human

Translation: Finding the Fit?

Fiorenza Mileto

To cite this version:

Fiorenza Mileto. From Computer Assisted Translation to Human Assisted Translation. Session 3 - Machine and Human Translation: Finding the Fit?. Tralogy II. Trouver le sens : où sont nos manques et nos besoins respectifs ?, Jan 2013, Paris, France. 8p. �hal-02497341�

Fiorenza Mileto

LUSPIO fiorenza.mileto@luspio.it

TRALOGY II - Session 3 Date d’intervention : 17/01/2013

Automated translation systems have been developed over the past 50 years and in the latest years (thanks to the evolution from rule-based machine translation to statistical machine translation) they are penetrating the translation industry in a very massive way. It seems that they are ready to revolution the world of translation.

Finally corpora (after years of use and misuse) found a good place to be used in a fruitful way: statistical machine translation needs huge quantities of words to work properly and to return an understandable sentence.

At this stage of development, machine translation and assisted translation are working together not only because the post-editing process is performed mainly inside CAT tools, but also because translation memories are more and more used as corpora for specialized translation and specific subjects. Unfortunately, translation memories were not always created and used in a proper way in the translation industry: they are often “dirty” and badly managed because they are fed and used by so many translation specialists in the various stages of translation projects, they are exchanged among so many different tools with different default settings often unknown to creators and users, and, most of all, they are mixed and shared but rarely cleaned.

From a CAT tool point of view, translation memories created for a specific subject are to be considered an asset, but a dirty translation memory is not a good starting point for machine translation: if you have to tell to the machine how to translate and you provide an inconsistent resource, the output of machine translation can hardly be good.

Machine translation aims at reducing the efforts and the intervention on the translation side, maybe substituting the translator or reducing the traditional competences requested to the translator. Is it really so ?

The initial effort to clean the linguistic data on which a machine translation engine is based is considerable and expensive. It requires all the traditional professional abilities and competencies that a translator acquires after years of studies and experience.

After the revolution introduced by CAT tools, translation industry is preparing to face a new revolution. Universities may play a pivotal role in this revolution: they may represent a bridge between linguistic data available on the market and the traditional linguistic competencies required to prepare them for machine translation, teaching students how to clean and maintain TMs, how to post-edit a translation generated with machine translation and how to help linguists and developers of machine translation engines to improve them.

Maybe machine translation is only an intermediate step between computer assisted

2

FROM COMPUTER ASSISTED TRANSLATION TO HUMAN ASSISTED TRANSLATION - FIORENZA MILETO

1. From Computer Assisted Translation to Human Assisted Translation

(Note: The ideas reported in this paper represent my own opinions and arise from my personal experiences)

When learning to use computers, we are taught that a computer does what you tell it to do, and that garbage in cannot give but garbage out. This means that when something goes wrong...

2. Machine translation and translation professionals

In the latest years, automated translation systems have been penetrating the translation industry in a very massive way. They seem promising and threatening at the same time, and translation professionals are still trying to ‘measure’ the extent to which they will reduce times and costs of the translation process granting the same quality standards The aim would be to obtain the same translation correctness, free of formal, formatting and style issues, with the very same fluent readability in less time at a lower cost.

3. Pre-editing and post-editing

Despite its advance, generally raw machine translation is still not of publishable quality, and preparation (also pre-editing) and post-editing is required to make source text suitable and polish the output.

A translation project involving machine translation, linguistic and technical skills are required for data preparation (building bicorporal, building and maintaining dictionaries, refining patterns and rules, keeping data clean).

For machine translation to be effective, the source text should also be edited to make it suitable. This consists in converting the text format, correcting any spelling or grammar errors, removing any formatting issues or unnecessary hard returns, tagging non-translatable items, and possibly simplifying and shortening sentences. For example, as a general rule, any machine- translation friendly text should contain sentences no longer than 25 words, no passive sentences, no gerunds or idiomatic expressions. To this end, many are (re-)exploring controlled languages as a means to remove or reduce ambiguities and intricacies thus helping machine translation systems return better outputs.

After processing, the output must be polished. Again, this is just another task in the multifaceted workflow of a typical translation project involving machine translation, and it is not be confused with translation or review: it is called post-editing and is a discrete task, which needs ad hoc training.

To exploit machine translation at its whole capacity, experienced, skilled, and knowledgeable professionals should be hired.

For cost-effectiveness, pre- and post-editing could be — and in fact usually are — assigned to one person, who would ideally be a tech-savvy subject-matter expert, with at least a basic knowledge of the machine translation engine. Needless to say, s/he must have a very good knowledge of his/her mother tongue.

This person needs to be watchful and keen to details, and be definitely unbiased with respect to machine translation. His/her work will be used to (re)train the machine translation engine, that is cleaning linguistic data, reshaping and improving the translation corpus for better customization.

4. Machine translation or assisted translation ? SMT or RBMT ?

A translation project involving machine translation is made up of different steps:

f Initial evaluation

First of all the source material should be checked for suitability to machine translation according to a few parameters:

• Volumes: they should be huge enough to be worth any customization effort;

• Data availability: if the client doesn’t have bilingual material or has never translated its material before, the implementation of machine translation may take longer and initial investment may be more considerable;

• Content type: machine translation is best suited for technical documentation and manuals; legal, marketing or too creative text are not recommended;

Formats and formatting;

• Terminology: it should be consistent, in the source text too, to prevent any issues and may greater post-editing efforts or ex-post dictionary customization, with both statistical and rule-based engines;

• Style: the source text should be well-written, with no errors or typos.

f SMT or RBMT: the choice between a statistical (SMT) or rule-based (RBMT) machine translation engine depends on content type, and should be made once the material has been analyzed.

• In SMT, the engine is based on a large corpus (millions of words) of terms retrieved from various reliable sources from which it is generated a basic output that it is then customized using a specific corpus and client’s requirements: this is the engine training and it is done for each language, because to get optimal results the corpus should be bilingual and quite huge (as a general rule the client’s aligned corpus or translation memories of at least 1 million of words are used). Since no syntactical or lexical data are used, the better is the corpus, the better the quality of output because the system learns how to translate by analysing the statistical relationships between large volumes of aligned source and target data. That’s why the role of the linguist in charge of the clean up of the translation memory and of the customization of the engine is fundamental:

data have to be consistent from a terminological and stylistic point of view. After the initial step performed by a linguist, the training will be done periodically exploiting the material produced during post-editing, but only when a sufficient quantity of data will be produced in order to feed the statistical system (million of words, not after every single small translation project).

• In RBMT, the system is based on a series of grammar rules for each language combined with a basic core dictionary containing detailed grammar data about source and target terms. Text is analyzed according to these rules, and the addition of terms and grammar rules to customize the system is defined as encoding. In this system, domain dictionaries are required and dictionary maintenance and linguistic feedback provided by the post-editor on each terms returned by machine translation is fundamental.

Both approaches have their pros and cons: a choice between the two is to be based on content type and on available linguistic data.

4

FROM COMPUTER ASSISTED TRANSLATION TO HUMAN ASSISTED TRANSLATION - FIORENZA MILETO

Setting up a RbMT engine could take less, provided that a language pair is available together with a large dictionary. Tuning the rules for any specific usage could take a little longer, but as long as source texts are simple and consistently written RbMT could prove cost-effective and almost immediately profitable.

SMT requires very large bilingual corpora, which take long to be gathered. No tuning is required, though, for rules. Data cleaning, however, could be time-consuming and the initial setup could be difficult. A termbase is helpful also in this case.

In any case, an adequate timeline should be traced for engine customization and data preparation and/or cleaning, as well as for any post-editing effort.

5. Translation memories and statistical machine translation

When implementing a statistical machine translation system, a large amount of linguistic data is required to feed the engine. For this reason, a baseline corpus of generic terms is used to be then associated with subject-specific corpora provided by the customer, possibly coming from its translation memories. In fact, translation memories could be a real asset for customers willing to implement machine translation, but they must be clean to be effective.

This is a major problem with translation memories. Maintenance best practices are rarely followed, for example for lack of instructions or time. Errors stratify from one project to another for the many people involved (translators, reviewers, project managers, language specialists, engineers, etc.). If we add the fact that each tool for creation of translation memories has its own settings and functionalities (often obscure to those who create or use the TM), a really dangerous combination of factors is threatening consistency and reliability of a TM. For example, a typical source of inconsistency and propagation of errors come from project managers who do not pay attention to specific options in the translation environment tool allowing for multiple translations for a segment. In addition, if for sequential projects a project manager pretranslates files and locks 100 % matches in order not to have them reviewed, the problem is amplified.

Another example could come from terminology updates applied only to single project files rather than consistently to a whole TM; sometimes TMs are not updated after a simple spell check or a grammar check; sometimes client’s instructions require to confirm and store in TM all translated segments (even when for languages like Italian or French an inversion of strings is required creating a mismatch) and in some cases unedited high fuzzy matches could lead to serious mistranslations (such in the case of opposites): the “dirty” translation memory is ready.

A few automatic checks (spell check, grammar check, terminology check, number check, translation inconsistencies, untranslated segments, formatting issues etc.), though, could be performed using any TM maintenance tool or with commercial quality assurance (QA) tool like ApSIC Xbench.

Style guides are crucial even for keeping translation memories clean and consistent. When working with translation memories it would be advisable to plan every single step of a translation project and set the various option of tools taking into consideration the whole translation cycle and any potential re-usage of every single project, trying to “protect” as much as possible the consistency and correctness of linguistic data in a long term perspective. For projects involving machine translation any specific instruction on settings for TMs (e.g. QA checks settings, filter mismatch penalty setting or formatting setting) would be invaluable, while usually default settings are used and not specified. Sometimes it may happen, for example, that to retrieve more leverage a project manager acts on some options changing TM settings for a particular project (let’s image a scenario with fuzzies generated by formatting issues or badly segmented files, for example), but it may also happen that for the following project may be performed after

6 months, the project manager may be no more the same, the tool used may be upgraded… and what about the TM ? Well, it is simply reused !

6. Evaluation of post-editing: time, quality, effort

Even if the most appropriate machine translation method is chosen, post-editing will be necessary. Many factors should be taken into consideration (source text, length of sentences, experience of translators, etc.) and they are so relevant and difficult to define in details that sometime in the same project the results are different from file to file.

In the last few years many academic research and case studies related to different metrics applied to post-editing process have been published to provide the basic element to assess a machine translation output, the post-editing and data maintenance efforts.

A major result is the need for a mid-to-long-term effort for implementation and customization and for a clear goal.

Ana Guerberof’s doctoral thesis is a little gem in this respect, for the thorough analysis of the many studies on measuring machine translation output and the post-editing effort. The experiments conducted by many companies (like Autodesk, Adobe, Microsoft, SAP, Symantec) are really interesting because they are conducted under “real” conditions, where many factors (such as customized training material, post-editing guidelines, QA cycle, times and costs) can be taken into consideration, but no detailed data are available of such studies to perform a satisfactory evaluation. Unfortunately, although fairly exhaustive, even Ana Guerberof’s work is not conclusive.

One of the reasons for which it is difficult to evaluate and “measure” the post-editing effort and quality is that it should be performed on real projects, with the engine algorithms customized for the client and the machine trained with the linguistic data and the translation memories prepared ad hoc for the machine and the client, but all these data are customer sensitive data that have to be protected.

At present, the best advice on post-editing is to develop a collection of basic rules to prevent at least the typical faults and defects of a (RbMT or SMT) machine translation engine. It would also be advisable to develop guidelines to write for machine translation to be effectively post- edited as well as few for the creation, maintenance and cleaning of translation memories to be used as a base for statistical machine translation.

This is difficult to implement at this initial stage because of the lack of data evaluation of the post-editing activity (i.e., productivity, quality, competences required for the translation etc.) and the need for a single evaluation standard.

The world is getting faster and faster, companies produce more and more content and run their businesses in more and more countries. In many a situation, MT with post-editing is the only viable alternative, for the tight combination on deadline, budget and volume. Much of the content to be translated simply do not deserve the highest quality of translation.

The history of the translation industry is rich of examples of boycotting any technical advance, from the introduction of typewriters to PCs to CAT tools. And yet, the tools that once arch enemies soon became a translator’s closest friends.

On the basis of what has happened in the past, for the near future we may expect that further developments in machine translation will make it even more ’translator-friendly’, and, if the translation industry decides that machine translation needs to be used (as increasing volumes

6

FROM COMPUTER ASSISTED TRANSLATION TO HUMAN ASSISTED TRANSLATION - FIORENZA MILETO

and reducing time will steadily require), translators will accept this change and start sharing information and influencing producers.

This trend may spur and urge the requirement for shared standards and metrics to evaluate quality, productivity and compensation criteria for post-editing to eventually prevent any further misuse of machine translation. In the end, human translators using automated tools will replace human translators who do not use such tools.

Quantitative data provided by some private consultancy companies (such as TAUS, Translation Automation User Society) may help in defining good practices, methods for quality assessment and creation of processes that may support companies in the adoption of machine translation and the penetration of MT in translation industry. While the reports on machine translation implementation and post-editing provided by De Palma and Kelly on Common Sense Advisory (research and consulting firm) could foster the sharing of experiences and ideas to create best practices.

7. Post-Editing of Machine Translation (PEMT)

After selecting the proper machine translation system, post-editing becomes crucial. To this end, many elements should be considered:

1. post-editors should be hired on a long-term basis to be really effective in improving machine output with clear data, and accumulate the necessary experience to meet requirements and discriminate between essential and preferential changes, between corrections and improvements (i.e avoiding over-editing, and under-editing);

2. post-editing requires ad hoc training (for example, the first thing he should learn is to read source first, then machine translation output: it seems a mean thing, but reading the target first could increase the possibility to introduce errors suggested by the machine. Contrastive analysis is not always needed/useful and it should be applied in a more ‘effective’ way), to detect known faults (type of errors introduced by the machine translation), and learn about the automatic assessing approach (WER, BLEU, etc.) of choice.

3. post-editors are essential elements in a machine translation process and should be able to interact with the people in charge of the engine’s optimization to improve the effectiveness of the system and the process.

Unfortunately, in some cases, the urge to implement machine translation in order to obtain the results promised by the potential of such systems (less time, lower costs) is leading to poorly trained machine, due to lack of guidelines, uncertainty on metrics and standards, limited time provided for post-editing step and inadequate evaluation of feedback and results. In this way, data that are being generated to retrain the machine and improve results are not adequate and won’t lead to development of the machine translation. This seems the very same phenomenon that happened with the implementation of assisted translation processes and the reuse of 100

% matches: in order to exploit the leverage obtained with the translation memories, 100 % matches were often reused without any further review, relying on dirty TMs that were not cleaned and maintained properly.

There is still much to do about metrics, data and requirements to assess MT output, and post- editing effort, but the urge for a standardization has been impellent in the latest period. In an article for Localization Focus, Celia Rico Pérez advocates the development of a flexible decision tool for implementing post-editing guidelines taking into account customer’s requirements, translation volumes, quality expectation, time constraints, and end usage of translation to help post-editors to decide when to discard a segment or avoid any amendments, how to deal with terminology, and the typical errors to correct.

In this perspective, introducing machine translation in academic curricula on a large scale may be another natural bridge between academic education and professional development, in the same way as for translation memory tools in the last decade. Jaap van der Meer recently stated that at last corpus linguistics may be used to develop new professional profiles.

Another essential element in the implementation of post-editing is the selection of resources.

In this early phase of machine translation history, many experienced translators refuse to work as post-editor mainly for two reasons: remuneration and motivation.

Translators don’t see the introduction of machine translation as a new opportunity, but as a way to further reduce remuneration and professional capabilities. Post-editing task seems less creative and satisfactory compared with traditional translation and translators are not so happy to provide their experience and capabilities to ‘train’ a machine that maybe could substitute them. The psychological aspect is fundamental and what it should be taken into account is that:

first of all machine translation wouldn’t exist without the linguistic data and inputs provided by translators; there are so many subjects and fields of translation that often are interwoven in the very same text, that it is difficult to train a machine with all the information it needs to return a good quality translation (there is no algorithm that could teach the machine to act beyond the rule and use ‘creativity’ to solve certain translation issues); skills and abilities required to a translator are not reduced but enhanced by machine translation system, the only difference is that they will be used in different ways and in different project phases if compared with the past.

Essentially the computer needs someone who tells it what to do !

Bibliography

Autodesk, (2011), “Machine Translation at Autodesk”. http://translate.autodesk.com/index.

html.

Belham, Judith (2001), “Transferable skills in an MT course”, MT Summit VIII Workshop on Teaching Machine translation, Santiago de Compostela. www.dlsi.ua.es/tmt/proceedings.html

Belam, Judith (2003), “’Buying up to falling down’; a deductive approach to teaching post- editing”. Paper presented to the MT Summit IX, New Orleans. www.dlsi.ua.es/es/t4/belam.pdf

Bowker, Lynne (2005), “Productivity vs. Quality ? A pilot study on the impact of translation memory systems”. Localisation Focus 4:1. 13‐20.

EMT Expert Group, (2009), “Competences for professional translators, experts in multilingual and multimedia communication”. http://ec.europa.eu/dgs/translation/programmes/emt/

key_documents/emt_competences_translators_en.pdf.

García, Ignacio (2010), “Is Machine Translation Ready Yet ?” Target 22 (1): 7‐21.

Guerberof, Ana (2009), “Productivity and Quality in the Post‐editing of Outputs from Translation Memories and Machine Translation.” Localisation Focus 7(1): 11‐21

Guerberof, Ana (2012), “Productivity and Quality in the Post‐editing of Outputs from Translation Memories and Machine Translation.” Doctoral Thesis, Universitat Rovira i Virgili, Spagna

Kenny, Dorothy and Andy Way (2001), “Teaching Machine Translation & Translation Technology:

A Contrastive Study”. Machine Translation Summit VII, Teaching MT Workshop, Santiago de Compostela. http://www.dlsi.ua.es/tmt/proceedings.html.

8

FROM COMPUTER ASSISTED TRANSLATION TO HUMAN ASSISTED TRANSLATION - FIORENZA MILETO

Lee, Jason, and Posen Liao (2011), “A Comparative Study of Human Translation and Machine Translation with Post--‐editing”. Compilation and Translation Review 4(2): 105‐149. http://ej.nict.

gov.tw/CTR/v04.2/ctr040215.pdf.

O’Brien, Sharon (2002), “Teaching Post-editing, a proposal for course content”. Paper presented to the 6th International Workshop of the European Association for Machine Translation, Manchester, UK www.co.umist.ac.uk/harold/teachingMT/OBrien.doc

O’Brien, Sharon (2007), “An Empirical Investigation of Temporal and Technical Post-Editing Effort.” Translation And Interpreting Studies, II, I

Perez, Celia Rico (2012), “A Flexible Decision Tool for Implementing Post-editing Guidelines.”

Localization Focus, vol. II Issue I

Plitt, Mirko, and François Masselot (2010), “A Productivity Test of Statistical Machine Translation.

Post‐Editing in a Typical Localisation Context”. The Prague Bulletin of Mathematical Linguistics 93 ufal.mff.cuni.cz/pbml/93/art-plitt-masselot.pdf

Pym, Anthony (2011), “What technology does to translating”. Translation & Interpreting 3(1):

1‐9. http://www.trans--‐int.org/index.php/transint

Yuste Rodrigo, Elia (2001), “Making MT commonplace in translation training curricula ‐ too many misconceptions, so much potential”. Paper presented to the Machine Translation Summit VII, Teaching MT Workshop, Santiago de Compostela. www.dlsi.ua.es/tmt/proceedings.html

Wiggins, Dion (2012), “Top 10 Trends and Predictions for the Professional Translation Industry – 2013 and Beyond.” Asia Online