Machine translation is advancing very quickly. Vera helps turn model evaluation into useful evidence to decide which system to use, improve, or deploy.
Machine translation evaluation can no longer depend on a single metric
Machine translation has advanced extraordinarily in recent years. Neural models, customised systems, and, more recently, large language models (LLMs) have greatly expanded the possibilities for automating translation processes in companies, public administrations, media, universities, and multilingual organisations.
But this progress also raises an increasingly important question: how do we know which machine translation model performs best?
The answer isn't always simple. A model may score well on an automatic metric and still make significant errors in terminology, style, omissions, fluency, or contextual adequacy. Another may appear worse on an overall metric but perform better in a specific domain. And when models are updated or adapted to new data, improvements can emerge, but so can regressions that are hard to detect.
In this scenario, evaluating machine translation quality is no longer just about getting a score, it's about making better decisions.
The problem: many metrics, little actionable evidence
Machine translation model evaluation is often fragmented. Many teams juggle independent scripts, spreadsheets, automatic metrics, human reviews, manual reports, and comparisons that are hard to reproduce.
This creates several problems. On one hand, automatic metrics make it possible to analyse large volumes of data, but they don't explain why a model fails. On the other, human evaluation provides linguistic judgment, but it can be costly, difficult to organise, and hard to compare without a structured workflow.
The result is that many organisations have data but not always a clear view to answer key questions: which model offers better quality for a specific domain, whether a new version genuinely improves on the previous one, which errors occur most frequently, whether automatic metrics align with human evaluation, or whether a system is reliable enough before moving it to production.
Answering these questions requires more than a single metric. It requires a platform that enables evaluation, comparison, analysis, and decision-making.
Vera: automatic and human evaluation in a single workflow
Vera is a web platform to evaluate, compare, and improve machine translation models.
Its aim is to bring together the main elements of the evaluation process into a single environment: automatic metrics, MQM-based human review, model comparison, error analysis, correlations, and reporting.
With Vera, a team can upload a dataset, compare the outputs of different models, calculate automatic metrics, review segments through human evaluation, filter errors, and analyse the results visually. All within a more organised, reproducible workflow geared towards decision-making.
The idea is simple: move from scattered evaluation to structured evaluation.
Automatic evaluation to compare models at scale
The first layer of Vera is automatic evaluation. The platform makes it possible to calculate metrics that help compare models, versions, or machine translation configurations quickly and consistently.
This evaluation is especially useful when working with large volumes of data or when an initial, objective approximation of the performance of several systems is needed. It makes it possible to detect differences, observe trends, and select which cases require more detailed review.
In addition, Vera incorporates statistical analysis to help determine whether differences between models are genuinely significant. This avoids making decisions based solely on small variations in score that may not be relevant.
Human evaluation to understand errors
Automatic metrics are necessary, but not sufficient. That's why Vera incorporates human evaluation based on the MQM framework, which makes it possible to identify and classify errors with greater linguistic precision.
Through human annotation, evaluators can review segments, flag and classify errors, and assign severity levels. This makes it possible to better understand what's happening: whether a model struggles with terminology, omits information, produces unnatural translations, or shows specific problems in certain domains or language pairs.
In addition to MQM evaluation, Vera also allows each evaluator to assign a score from 1 to 10 to each segment, adapting the Direct Assessment (DA) method to a more manageable scale.
This layer of evaluation is key to moving from "which model scores better" to "why one model is more reliable than another".
Analysis to turn results into decisions
The value of Vera doesn't lie only in calculating metrics or enabling annotations. It lies in connecting all that information to support decision-making.
The platform makes it possible to analyse correlations between automatic and human evaluation, filter errors by type or severity, compare models visually, and generate results that can be used in research, validation, continuous improvement, or production deployment processes.
This is especially useful when an organisation needs to decide which model to use, justify a technical choice, monitor the evolution of a system, or detect regressions after an update.
Rather than simply stating which model achieves a higher score, Vera helps to understand which model can be trusted in a given context.
Who is Vera for?
Vera is designed for teams working with machine translation who need to evaluate quality rigorously.
It can be useful for research groups, universities, translation teams, technology companies, public administrations, language service providers, or organisations deploying machine translation in real-world workflows.
It is also especially relevant in contexts where linguistic quality, traceability, and the ability to justify decisions are critical: software localisation, media, institutional documentation, public services, specialised translation, or the evaluation of multilingual systems.
Vera at EAMT 2026: interest from the research community and industry
Vera was presented at the EAMT (European Association for Machine Translation) 2026 conference, one of the leading international events specialising in machine translation and translation technologies. The presentation provided an opportunity to share the platform's proposal with an audience made up of researchers, universities, technology companies, translation teams, and professionals working directly with the evaluation and deployment of machine translation systems.
The interest generated during the event confirmed an increasingly visible need in the sector: tools that go beyond calculating metrics, and that help to compare models, validate results, detect errors, and make decisions with greater confidence. Both from the research community and from industry, Vera sparked conversations around hybrid evaluation, the combination of automatic metrics and human review, the traceability of results, and the need to control the quality of continuously evolving models.
This reception reinforces Vera's approach as a platform designed to connect research, linguistic validation, and practical application. Its value lies not only in evaluating models, but in helping teams better understand their behaviour, justify their decisions, and move towards continuous improvement processes in machine translation.
From evaluation to continuous improvement
Machine translation is no longer a static technology. Models change, data changes, and users' needs change too. That's why evaluation shouldn't be understood as a one-off action, but as an ongoing process.
Vera helps build that process. It makes it possible to compare, validate, and analyse models more systematically, combining the speed of automatic evaluation with the depth of human review.
Ultimately, Vera makes it possible to answer an essential question with more evidence: which machine translation model can we trust for this use case?
Book a demo of Vera
If your organisation works with machine translation and needs to compare models, validate quality, or improve its systems, visit Vera's page, request a demo, and discover how Vera can help you turn evaluation into more reliable decisions.
