karthikbi.dev
← Writing

Copilot Ignored a Real Field Twice. A Description Didn't Help. Synonyms Did.

·4 min read

Power BI, Copilot

Extra is a genuinely ambiguous column name — it's a real field on the NYC taxi data (rush-hour and overnight surcharges), but no human would guess that without context, and neither did Copilot. I used it to find out which piece of model metadata actually changes Copilot's field selection.

Attempt 0 — raw field, no metadata

I asked: “what's the total extra charges.” Copilot skipped Extra entirely, picked totalAmount instead, and returned 170,582,317.45 labelled “Sum of totalAmount.” Its own explanation gave it away — it said the breakdown was “missing,” even though Extra had been in the model the whole time, just unrecognised.

Original attempt — Copilot missed Extra entirely, returned Sum of totalAmount
Original attempt — Copilot missed Extra entirely and returned Sum of totalAmount (170,582,317.45).

Attempt 1 — add a description

I added a field description: “Extra charges applied outside the standard fare, such as rush hour or overnight surcharges.” Asked the identical question again.

Still wrong. This time Copilot grabbed Total SUMX, an unrelated measure, and returned 136,181,159.92. The description didn't move the field selection at all.

A field description added to Extra
Attempt 1 — a field description added to Extra.
Attempt 1 result — still wrong, Copilot grabbed Total SUMX
Attempt 1 result — still wrong, Copilot grabbed Total SUMX (136,181,159.92).

Attempt 2 — add synonyms

Same field, added explicit synonyms: surcharge, extra fee, additional charge, extra amount. Asked the same question a third time.

Correct. Copilot identified Extra directly and returned 3,414,531.09 labelled “Sum of Extra,” with an accurate written summary.

Attempt 2 result — synonyms added, Copilot correctly identified Extra
Attempt 2 result — synonyms added, Copilot correctly identified Extra (Sum of Extra, 3,414,531.09).

What that tells me

Not all metadata is equal to Copilot. A description didn't change which field got picked. Synonyms — mapping the actual business phrase (“extra charges”) to the field — did. Microsoft's guidance points to AI data schema and AI instructions as higher-priority levers still; that's the next thing I want to test.

If Copilot is missing an obviously relevant field, try synonyms before assuming a longer description will fix it. In this test it was the only lever that worked.

What I take from this

  • Description and synonyms are not interchangeable — only synonyms changed the outcome here
  • Synonyms work best when they carry the exact phrases users actually type
  • Worth testing next: AI data schema and AI instructions, which Microsoft ranks higher than either
← Back to Writing