ML Associate New Version: Sapior Ships Model-to-Data Association v2

The new version of ML Associate is now available for all Sapior workspaces. We rebuilt the association engine to handle how teams actually work with models in production: multiple datasets, prompt templates, evaluation sets, and deployment artifacts that change independently.

If you used v1 to link models to datasets, v2 changes the unit of association from a static table to a versioned relationship graph. That shift matters because it lets you trace a model prediction back to the exact dataset version, feature definition, and prompt template that produced it.

Why we shipped a new association engine

V1 was built around a simple model-to-dataset link. It worked for demos and small projects, but production ML teams kept asking the same questions:

Which dataset version was this model actually fine-tuned on?

Did the evaluation set drift since the last training run?

Are the embeddings in this model associated with a different retrieval corpus?

Can we link a prompt template to a model and a retrieval dataset in one view?

Those questions require more than a foreign key. They require a graph with version awareness, confidence scoring, and semantic matching.

What changed in ML Associate v2

Automatic version linking

ML Associate v2 now watches your model registry, feature store, dataset registry, and prompt store. When a new model version is registered, it automatically proposes associations to the dataset versions, prompt templates, and evaluation sets that were active at training time.

The proposal is not just timestamp-based. Sapior uses a combination of git metadata, pipeline run IDs, and data fingerprints to build candidate links. You can accept or reject proposals from the UI or CLI.

Drift-aware association scoring

Every association now has a drift score. The score reflects how much the associated dataset, prompt, or feature distribution has shifted since the model version was created.

If a training dataset has drifted by more than your workspace threshold, the association is flagged. The model card and deployment view show the risk level. This is the difference between an association existing and an association being trustworthy.

Semantic lineage for multi-modal data

V1 assumed tabular or text datasets. V2 adds first-class associations for image datasets, audio datasets, retrieval corpora, and vector indexes. You can now link a model to a chunking strategy, an embedding schema, and a retrieval corpus in the same relationship graph.

Semantic lineage means you are not only recording that a model used a dataset. You are recording what the dataset meant in that context: the query patterns, prompt templates, and retrieval settings.

Similar lineage work in Azure ML and Google’s Model Cards Toolkit focuses on metadata; v2 goes one step further by scoring whether the association still holds in production.

Association as code

ML Associate v2 supports association-as-code through a new YAML spec. You can define associations in your repository and let Sapior reconcile them with your workspace state. The spec includes explicit version constraints, drift thresholds, and approval policies.

How migration works

Existing v1 associations are imported automatically. Sapior maps the old model-to-dataset link to a basic association edge with a neutral drift score and full version history for the linked dataset.

You can keep v1 links as-is, but we recommend enabling automatic version linking for any model that is actively retrained or fine-tuned. The new proposals start appearing within minutes after the first pipeline run.

What this unlocks for platform teams

The new version is not just a UI update. It unlocks several downstream workflows:

Compliance reviews that need exact training data lineage.

Incident response that traces a bad prediction to a specific dataset distribution shift.

Retraining decisions based on drift score rather than elapsed time.

Evaluation set coverage reporting across model versions.

Model cards that stay current because associations are updated from the pipeline.

What's next

We are working on association-based alerting, public association APIs, and a GraphQL endpoint for building custom lineage views. We are also testing an experimental feature that suggests retraining windows based on drift trajectories across your association graph.

ML Associate v2 is available now. Open your Sapior workspace and select ML Associate from the model catalog. The new CLI commands are available in the Sapior CLI v0.19.0.