The expertise a styling product needed already existed, and it was locked inside hours of video of experts talking. This is the pipeline that turned it into attributes a system can query, and the dataset that made the attributes checkable.
The expertise is spoken out loud and stored as video
Knowing which fabric drapes badly on which cut, or which colours fight each other, is real expertise, and the people who have it explain it out loud rather than writing it into a schema. Video is the worst possible storage format for it: unsearchable, unstructured, and full of the things speech contains that data cannot use. Meanwhile the product needed to reason about style, construction, fabric and colour compatibility as fields, not as anecdotes.
The pipeline
Three stages, each with a different failure mode.
- Transcription
- Whisper turns the video into text. This is the easy stage and the one people assume is the hard one.
- Extraction into a schema
- Multi-modal models pull garment attributes out of the transcript and the imagery, and the output is validated against a Pydantic schema rather than accepted as prose. An attribute that does not fit the schema is caught at the boundary, which is the difference between a knowledge pipeline and a pile of summaries.
- Labelling
- The first labelled garment-image dataset for the product, covering style, construction, fabric and colour compatibility.
What was already there, and wrong
The feature-extraction pipeline was not new. It existed, it ran, and its logic had drifted from what it was supposed to be measuring. Fixing legacy extraction is less satisfying than building extraction, and it is worth more: every downstream model was learning from whatever the old logic produced, so the errors were not sitting in one place where someone would trip over them. They were distributed into everything trained afterwards.
The dataset is the durable part
Pipelines get rewritten and models get replaced. A labelled dataset outlives both, and it is the only thing on this project that makes the next version measurable against this one. Before it existed, improving attribute extraction meant looking at some outputs and forming an impression.
Two shortcuts past the schema, and what each one loses
The schema is the slow part, which is why both alternatives aim at removing it.
- Buy a fashion attribute dataset
- Labelled fashion data exists and can be licensed. Its taxonomy is somebody else's, built for somebody else's product, and the attributes this one needed — how a fabric drapes on a cut, which colours fight — are the judgements a general dataset does not carry. Correcting them afterwards means building the schema anyway, and then reconciling it with someone else's.
- Ask a model to watch the video and answer directly
- Passing the footage to a multimodal model and querying it works in a demo and cannot be checked. With no schema there is no field to fill, so there is nothing to score against — and the dataset, which is the part that outlives every pipeline built on it, never comes into existence.
The work is not the retrieval, it is deciding what the fields are
Most of what a company knows is in this shape - held by people, expressed in speech, never written down in a form anything can query. The work is not the retrieval. It is deciding what the fields are, and then proving the extraction filled them correctly.
What it proves
Expert video turned into structured, queryable data, backed by the product's first labeled garment-image dataset.
Category
AI Engineering
Built with
Python · Multi-modal LLMs · Whisper · Prompt Engineering · Pydantic · Dataset Labeling · Video-to-Text Pipelines · FastAPI · LangSmith · Langfuse · Docker
Need something similar?
The cheapest way in is two weeks. The first days work out which task would pay for itself in your processes; the rest builds that agent on your own data and measures it.