To MT or not to MT? That is the (unavoidable) question

(And you might be surprised by our answer…)

In 2023, it is virtually impossible to get through a day without some interaction with the great AI debate. Regardless of industry, seniority, or (frankly) interest, the average professional is obliged to engage with the subject of AI and machine learning on a daily basis – with how it will revolutionize your workload, your budget, your life… This may seem new to many, and it has certainly gone mainstream since the release of ChatGPT to the masses. But, with large language models (LLMs) hitting our sector – logically enough – long before they did many others, we in the translation industry have been discussing, disputing and/or implementing machine learning for many years now.

What is MT?

In the context of translation and language services, this debate largely centers around machine translation (or “MT”). For those who do not know, machine translation does exactly what its name suggests. It uses software to automate translation of content from one language to another. Most often, this is based on large language models or neural networks, i.e., computer systems that mimic the human brain. These machines can be trained by humans, and usually content is subsequently reviewed and “post-edited” by a linguist. Advocates of machine translation cite time savings, cost reductions and general efficiency as reasons for use, while opponents point to poor-quality translations, excessive time spent on corrections, and the undermining of the role and expertise of professional linguists. It’s a controversial topic, where everyone involved has varying degrees of skin in the game. Which brings us to the elephant in the room. Namely, on which side of the debate does Conversis stand?

To MT or not to MT?

To answer this question, it’s important to first provide some context around what we actually do day in, day out at Conversis, and where that work intersects (or not) with MT. As our tagline proudly states, we are Life Sciences translation experts. This is everything we do, every day. Primarily, this focuses around clinical trial documentation, patient recruitment and retention, and regulatory submissions. We work mostly with CROs, Medical Communications agencies and Pharmaceutical companies to help them do everything from advertise their trials internationally, to successfully meeting regulatory requirements in other countries. We help them keep their patients informed and safe, and their international projects running. Conversis translations are used in clinical contexts; in describing how to take a medication, explaining the risks and benefits of taking part in a trial, attesting to a national body that a drug meets certain standards…This is not content with a tolerance for inaccuracies!

And so to the question of where MT fits in. As with everything we do, we’ve approached machine translation with an open mind, keen to assess benefits and incorporate whichever elements best serve our Life Sciences clients. What we have found is that – for us – MT can be a useful tool in very specific scenarios with clear parameters. For example, say a client has vast amounts of content in a language they don’t speak where they simply need to understand the gist. MT is a low-cost, high-speed way to make that information understandable for them. Similarly, content that will be used internally for information purposes, such as market research, can be processed quickly and cost-effectively via MT and post-editing. However, this type of work makes up a very small proportion of what we do; most of our projects, as we’ve noted above, are patient- or authority-facing. And for that kind of work, our overwhelming findings up to this point inform our very clear current stance on machine translation. That is: we don’t recommend it. Not for our clients and not for the majority of the work we do.

How did we come to this conclusion?

This is not due to a fear of the new or unknown, and it’s certainly not due to a mistrust of technology. On the contrary, Conversis is always quick to embrace, create and customize technology to improve the way we work. We do not recommend use of machine translation because none of the MT options currently available do as good a job as our human expert linguists. And because working with medical content involves a series of checks and balances that make the benefits MT does bring to the table relatively insignificant.

By way of explanation, let’s come back to the arguments in favor of machine translation – that it is quicker, more cost-effective, and generally more efficient. In our experience, for Life Sciences content, these arguments just don’t hold water. Given the stakes we are working with every day, absolute accuracy is our primary concern.

“Faulty translations may entail the “failure of the participant to act as instructed […]” not to mention physical or emotional damage, misconduct of the experiment, time, and money.”*

And to achieve absolute accuracy, even those on the technology side will tell you human intervention (e.g. post-editing) is essential. Once you account for the cost of this intervention (fixing inaccuracies, plus the efforts involved in making content drawn from a variety of sources internally consistent), you can already see your cost savings start to erode. In terms of time saved, our findings are that – given the expertise and experience of our human linguists – it can often take longer to check and fix something that’s not quite there than it does to get it right first time. Add to that the many linguist, client and board review rounds dictated by medical regulations, and your savings and efficiencies are often dwarfed to the point of irrelevance.

What, then, about the arguments against? Namely, poor quality, level of corrections required, and the harmful effect on linguists. We’ve already dealt with the corrections issue above, but it’s worth noting that we are speaking to our own Life Sciences context here. There may well be contexts where there is a tolerance for less than 100%; where some sacrifices can be made in favor of cost and turnaround times and the general greater good. The context in which Conversis works is not one of them! As to quality, we know from experience that our linguists bring a level of expertise, nuance and accuracy to their work that is unmatched by any MT solution currently available. In the medical realm, human translators still provide a level of confidence that machine translation cannot guarantee. Take, for example, a recent study completed at Columbia University, which pitted humans against LLMs to test their ability to detect nonsense. They found that “every model exhibited blind spots, labeling some sentences as meaningful that human participants thought were gibberish”. You can read more on this study here. Or the example an industry colleague told us about recently, where a term in their medical documentation meaning “fetal” had been translated by machine as “fatal” – quite a significant difference in meaning (and possible outcome)!

Human vs machine

But surely, the received wisdom goes, a machine should be more reliable in terms of quality and consistency than any fallible human? And here is where having a little bit of knowledge is, as they say, a very dangerous thing. Because if you scratch the surface of how MT actually works; the premise on which it functions, it becomes very clear, very quickly how much more limited it is than a live, human expert. An MT tool does not generate original thoughts and ideas. In a translation context, what MT actually does is try to mimic the series of human decisions involved in translating a sentence. MT makes decisions based on the past data it can access and is non-deterministic, i.e., ask it to translate the same sentence in the same context three times and you will likely get three different translations. Even the experts don’t understand why. So far, not so dissimilar to how a human works – right? But what about a context – not unusual in the work we do – where that past data simply does not exist? Where there is no vocabulary for a certain illness in a certain country or culture; no data on which basis to make predictions? In a sector like ours – where new developments are a regular occurrence and communicating them quickly and accurately is key – machine learning is, somewhat ironically, and certainly unhelpfully, trapped in the past.

Another element to take into consideration is context – both of the content and of the cultures you are translating for. Context is key to clear communication – whether that’s knowing which of three possible translations to use for a given word in a given sentence, or how best to land a point for a specific community of people. Large language models are not yet nuanced enough to translate for different contexts and readerships, and are certainly not immersed in local culture, regional regulations, or expectations in terms of format, tone, and presentation. Just look at these examples from US asylum cases, where, among other things, names are translated as months of the year, cities and states are rendered incorrectly, and content is – generally – translated based on a completely and uniquely Anglo-centric context.

This is to say nothing of the inherent biases present in LLMs and the difficulties this poses for creating inclusive content. For instance, how to incorporate non-gendered terminology into languages that are inherently gendered is a very hot topic in our work at the moment. This is something we humans need to find solutions for before the machines can try to mimic them!

Last, but not least, is the question of our linguists. This may not seem like a relevant consideration for some. But we would argue that those “some” do not work with the caliber of linguist we do. These are individuals with medical qualifications and language qualifications. People who have honed their skills to not only comprehend and translate into their native language – but also to do so with specific medical expertise. People who have spent their lives building up a skill set so specific and valuable, they can advise our expert clients on their own work! Based on what we’ve seen, machine translation has a long way to go before we will deem it worthy of comparison to our linguist network and the work that they do.

So, there it is! Our answer (for now!) to where we stand on the great AI debate. But that part in parentheses is crucial to note! There are no definites in the world of AI, where everything is in such flux and developing rapidly. So we would like to be clear: we will always strive to provide best-in-class solutions and services to our clients. And we know from experience that technology can be a great tool to facilitate human work in a variety of contexts. As such, we will be constantly checking and testing, and keeping on top of progress. And, if and when MT can rival our experts; if and when regulations evolve to allow meaningful use of AI in a Life Sciences context, we will, without question, be the first in line to put it to work for our clients.

*https://www.clinicalleader.com/doc/the-role-of-language-translation-in-clinical-trials-0001