Aurix/Case studies/AI Data Pipelines
case / /02

The model is only as good as its data.

DataForge is a suite of annotation platforms purpose-built for training and evaluating AI translation models. Binary preference collection, HITL feedback loops, summary evaluation, and ecommerce-domain pipelines each fit a specific evaluation methodology and share the same infrastructure.

4
Annotation platforms in the suite, shared infrastructure
HITL
Continuous feedback loops, not one-shot labeling
Multi
Binary, preference, scoring, and summary evaluation
Domain
Ecommerce-specialised data pipelines
The Challenge · 01

An AI translation model is a function of the data it was trained on. The data is the product.

Most teams building AI translation models talk about the model. The model is the wrong thing to talk about. The model is the output. The data is the product, and the annotation platform is the factory.

Generic web-scraped corpora get you a generic translation model. They will not get you a model that handles ecommerce product titles correctly, or one that preserves the structure of a long-form summary, or one that knows when a binary “is this acceptable?” judgement is the right unit of evaluation. For that, you need labelled data produced under the methodology that matches the evaluation question. You need it produced at volume, by annotators who can move quickly because the tool fits the task.

The methodologies are not interchangeable. Binary judgement (is this translation acceptable?) needs a different UI from preference comparison (translation A vs translation B). Summary evaluation (does this preserve meaning?) needs a different flow again. Ecommerce-domain validation (is this product title correct for the locale?) needs its own pipeline because the data shape is different. Forcing all of them into a single off-the-shelf labelling tool produces inconsistent data, which produces inconsistent models.

“The model is the output. The data is the product. The annotation platform is the factory.”
the design constraint

The challenge was to build a suite rather than a tool. Each annotation platform needed to fit a specific evaluation methodology while sharing the same infrastructure. HITL feedback loops would connect them, so the data could improve over time instead of being labelled once and forgotten.

The Problem · 02

Two ways teams handle annotation. Both produce mediocre data.

Teams training AI translation models tend to default to one of two patterns when they need labelled data. Both compromise the dataset before the model ever sees it.

Failure 01

Generic labelling tool, forced methodology.

Off-the-shelf labellers handle eighty percent of annotation work fine. When the task calls for binary preference, summary evaluation, or ecommerce product validation, the team starts working around the tool's assumptions. Annotators get confused. Inter-annotator agreement drops. The data comes back inconsistent and has to be cleaned before it can train anything.

Observed
Drift in agreement
Failure 02

One-shot labelling without feedback loops.

Most annotation projects treat labelling as a one-time step. Annotators tag the data, the model gets trained, the dataset is filed away. When the model drifts in production or hits an edge case nobody anticipated, there is no path to feed corrections back into the dataset. The training data is fossilised. The model degrades silently against a moving distribution.

Observed
Static dataset

Neither produces what a team training serious AI translation models actually needs: a methodology-specific annotation surface for each evaluation type, plus a HITL feedback loop that keeps the dataset alive as the model meets reality.

The Solution · 03

A suite of annotation platforms, methodology-specific surfaces, shared infrastructure.

DataForge is not a single tool. Each annotation platform fits a specific evaluation methodology. They run on a shared backend that handles task queuing, data versioning, audit trail, and HITL feedback into model training.

dataforge.suite / live
4 platforms · shared backend
Platforms
  • P1
    Binary annotation
    accept/reject
  • P2
    Preference comparison
    A vs B
  • P3
    Summary evaluation
    long-form
  • P4
    Ecommerce pipeline
    domain data
Infrastructure
  • I1
    Celery
    async pipeline orchestration
  • I2
    PostgreSQL
    structured + versioned
  • I3
    HITL
    prod drift back to training
  • I4
    REST API
    model integration e2e

01 · Binary & preference annotation

Two of the most useful evaluation methodologies for translation work: accept/reject judgements and head-to-head A/B preference. Each gets a UI designed around the decision being made, with built-in disagreement-resolution flows.

02 · Summary evaluation platform

Long-form translation output cannot be evaluated by spot-checking strings. The summary platform presents the full source and target, asks structured questions about meaning preservation and fluency, and produces scores annotators agree on.

03 · Ecommerce-domain pipelines

Ecommerce data has its own shape: product titles, descriptions, attribute values, locale-specific units and currencies. The ecommerce pipeline is built around that shape rather than treating it as generic text, which is what produces ecommerce-grade training data.

04 · HITL feedback loops, end to end

When a model misclassifies in production, the error becomes an annotation task. The corrected label flows back into the dataset, gets versioned, and is available for the next training run. Labelling stops being a one-time cost and becomes a continuous improvement loop.

The Result · 04

Annotation became infrastructure, not a one-off project.

DataForge is operating as the data factory behind multiple AI translation model lines. Methodology-specific platforms produce consistent labelled data. HITL loops keep the datasets current. The model team's bottleneck moved off "we need data" and onto "what should we train next."

result.01
4
Annotation platforms running on shared infrastructure
result.02
Continuous
HITL feedback flow from production back into training data
result.03
0
Methodologies forced into the wrong UI
01

Methodology-fit annotation produced consistent labelled data.

Each platform is built around the evaluation question annotators are answering: binary, preference, summary, or domain validation. Inter-annotator agreement improved because the UI stopped fighting the methodology, and the data downstream of it became trainable without a cleaning pass.

02

HITL feedback loops kept training data current with production drift.

Production errors became annotation tasks automatically. Corrected labels flowed back into the dataset, got versioned, and were available for the next training cycle. The dataset stopped being a static artifact and became a live record of how the model meets reality.

03

Multiple evaluation methodologies, one shared backend.

Four annotation platforms use the same backend. Adding an evaluation type starts with its review interface. Task queues, data versioning, and auditing are already available, so the team can focus on the new methodology.

04

Ecommerce pipelines unlocked ecommerce-grade model quality.

Ecommerce-specific data, including product titles, attributes, and locale-specific patterns, gets annotated through a pipeline that knows the data shape. The resulting models handle ecommerce content meaningfully better than the same architectures trained on generic web corpora, because the training signal is domain-specific by construction.

05

Annotation became a capability, not a cost centre.

The team that owns data generation moved from "we need to build another labeller" to "which methodology fits this evaluation question." Most of the answers are off-the-shelf within the suite. The ones that are not become new platforms on the same backend. They ship faster because the hard parts are already solved.

Built withReactDjangoPostgreSQLPythonCelery
Get Started

Need annotation infrastructure that fits the methodology?

Talk to us about how DataForge can plug into your model training workflow. We will walk you through the suite, run a sample annotation flow on your data shape, and tailor a deployment to the evaluation methodologies your team actually uses.

30-min walkthrough · No commitment
Sample annotation · Bring your own data shape
Pilot-ready · Methodology-fit from day one