Wheron: Your Navy Jumper Is Black Under a Kitchen Light

Wheron reads a photo of a garment and works out what colour it is. This is one of those features that is ninety percent right immediately and spends a long time getting to ninety-five, and the residual errors are not random — they cluster on exactly the colours people care about distinguishing.

Finding the garment before finding the colour

Averaging the pixels in a photograph gives you the colour of the room. Before there is any colour question, there is a segmentation question: which pixels are the clothing.

We run subject segmentation to separate the garment from its background, then work only within that mask. This handles the common cases well — a jumper on a bed, a jacket on a hanger against a wall — and it fails in the way you would expect: a white shirt on white bedding, or a black coat photographed against a dark wardrobe interior, gives the segmenter very little to work with.

That is the same class of problem as page detection in a scanner app, and it has the same fix from the user's side: photograph the item against something that is not the same colour as the item. A dark bedspread for light clothes, a light floor for dark ones.

Within the mask, we cluster the pixels rather than averaging them, because averaging destroys the answer for anything patterned. A striped shirt averaged is a muddy mid-tone that matches neither stripe. Clustering gives you the dominant colours as separate values and lets a garment legitimately be two colours.

The colour is not the colour

Here is the part that dominates the error rate, and it happens before any of our code runs.

A camera decides white balance automatically. It looks at the scene, guesses the colour of the light, and corrects the image so that things it believes are white come out white. Under warm indoor lighting — a typical living room bulb — this correction is substantial. Under mixed lighting, daylight from a window plus a warm lamp, the guess is a compromise that is wrong for both.

The image you receive has already been shifted. There is no undo.

The result is systematic rather than random. Navy under warm light comes back nearly black. Black under warm light comes back warm dark brown. White under fluorescent picks up a green cast. Grey picks up whatever the light was and stops being grey.

So the errors are concentrated exactly where they hurt: navy and black, cream and white, charcoal and brown. Nobody minds if the app is slightly wrong about the shade of a red jumper. Everybody minds if their navy suit files under black, because that is a distinction they organise their wardrobe by.

Naming, and why RGB distance is the wrong tool

Once you have a colour, you have to give it a name, and the naive approach compares the extracted value against a table of named colours by straight numerical distance in RGB.

RGB distance does not match perception. Two colours the same numerical distance apart can be obviously different in one part of the space and indistinguishable in another. Doing colour naming this way produces confident, weird answers — a garment that any person would call burgundy filed as purple.

The fix is to work in a perceptually uniform colour space, where numerical distance corresponds roughly to how different two colours look. Nearest-name in that space produces answers that agree with people much more often.

It does not fix the white balance problem, because that is an error in the input rather than in the naming. Nothing downstream recovers a colour the camera has already shifted.

Which is why there is a review step

Every automatic result in Wheron is a suggestion presented for confirmation, and this is a deliberate design position rather than a lack of confidence in the models.

The review happens while the garment is in front of you. That is the only moment when you can look at the actual jumper, see that the app said black, and know it is navy. Ten minutes later you are looking at a photograph too, and the photograph has the same white balance problem the app did.

So the flow is: add the photo, look at what the app worked out, fix what is wrong, save. It takes a few seconds per item and it is the difference between a closet you can filter and one where the filters return the wrong things and you stop using them.

The colours are the field most worth checking. Category detection is more reliable, because the shape of a garment does not depend on the lighting. Season is not detected at all — a jacket's warmth is not visible in a photograph — so it is yours to set, and it is the attribute that makes the closet useful for packing and for weather.

Building the closet

Do not photograph everything. The most common way this app gets abandoned is an ambitious first session cataloguing an entire wardrobe, which takes hours and produces a database of clothes you do not wear.

Start with what is in rotation. The things you actually reach for in a normal week. Fifteen or twenty items is enough to make the app useful, and useful is what makes you keep adding.

Add in batches of similar items. All the tops, then all the shoes. The review step is faster when consecutive items share a category and a background, and you can set up the lighting once.

Photograph against contrast. Dark items on a light surface, light items on a dark one. This makes segmentation work and it is the single thing you control that most affects the quality of the result.

Use even, neutral light where you can. Near a window during the day is much better than under a warm bulb at night. This is the white balance problem, and it is the one you can actually do something about.

Fix the colour while the garment is in front of you. Especially for anything navy, charcoal, cream, or grey.

Set the season yourself. It is the attribute that does most of the work later and none of it can be inferred.

What it is for

A wardrobe database earns its place in a small number of specific situations rather than as a daily habit.

Packing is the strongest one: filtering by season and category against a trip's actual requirements beats standing in front of a wardrobe trying to remember what you own. Shopping is second — a phone that can tell you that you already have three grey jumpers prevents the fourth. Seasonal changeover is third, when half your clothes are in storage and you cannot see them.

Daily outfit choice is the case people expect and the one where it helps least, because you can already see what is in front of you. Where it does help is with the clothes that are physically hidden — at the back, in a box, in another room. Those are invisible to you and visible to a filter.