In a first-match catalog, the order of the rules is part of the logic

When an Unreal Android build fails on Ubriot, we try to name the cause before anyone reads a log. The worker quotes the first error line in the failure message, and the API checks that message against a list of known failures: a missing keystore, an out of memory kill, a Git LFS pointer that was never replaced by the real file, a toolchain path that does not exist. Each entry has a pattern, a plain title, a likely cause and a suggested fix. No AI model is involved in this step. It is a list of regular expressions, and that is the point: it is fast, it costs nothing and it gives the same answer every time.

The lookup is one line. It walks the list in order and returns the first entry whose pattern matches the message.

const hit = catalog?.find((failure) => failure.pattern.test(errorMessage));

That makes the order of the list a behaviour, even though nothing in the code says so. Two entries that can both match the same message do not tie. The one written earlier wins, silently, and the other never gets a chance.

We found this by reading the catalog next to the way the failure messages are built, not from a bad diagnosis on a customer build. The cook failure entry, which matches a LogCook error line, sat at the bottom of the Unreal list. The toolchain entry for the Android SDK and NDK sat above it, and its pattern includes the name of the NDK_ROOT setting. A cook error is an asset or Blueprint that failed to load or save, and the line that reports it can quote almost anything, including a path or a setting name that happens to appear in the asset's own message. If that line mentions NDK_ROOT, the toolchain entry matches first. The build gets told to check its Android NDK installation when the real problem is a texture in the project.

The two causes send people in opposite directions. A wrong NDK means a change on the build machine or in the project settings. A broken asset means opening the editor and fixing or deleting that asset. A confident wrong label costs more than no label, because someone follows it.

The fix was to move the cook failure entry up, to just after the keystore entry and before the engine version and toolchain entries. The reasoning is about specificity. A line that begins with a cook error marker is a stronger signal than a line that merely contains a setting name, so the narrower, more reliable pattern goes first and the broad keyword patterns go after it.

Moving an entry is easy to undo by accident. Someone adds a new rule at the bottom, or tidies the list into alphabetical order, and the behaviour changes with no failing test. So we added one that targets the collision directly. It feeds the catalog a message in which a cook error line contains NDK_ROOT, using a made-up asset path, and asserts that the result is the cook failure entry. The existing test for the catalog only checked that each known message produced its own tag, which cannot notice an overlap, because each sample message was written to hit exactly one entry.

That gap is worth stating on its own. If every sample in your test table was written to match one rule, the table proves that each rule can match. It does not prove that the right rule wins when two apply. The cases that matter for a first-match list are the ones that are deliberately ambiguous, and they have to be written on purpose.

If you keep a list like this in your own tooling, three habits help. Put the most specific patterns first and the broad keyword ones last. Write a test for each pair of rules whose patterns can overlap, asserting which one wins. And treat a reorder in review as a behaviour change, not formatting.

The catalog is still short and still returns no match when nothing fits, in which case the build keeps the original error text and the usual diagnosis path. We would rather say nothing than name a cause we are not sure of.