Regex Discipline Convention

When to reach for a regex/heuristic vs. when to let the model do the judgment.

The rule: the model doing knowledge work judges FIRST. A regex is earned ONLY after you have seen enough real data (small sample first — see skills/conventions/test-before-bulk.md) to confirm the signal is rote, repetitive, and 100% deterministic. A regex is a compression of a pattern you already verified by looking — never a substitute for looking. Premature regex (writing a pattern off the bat, before reading the data, to do work that requires judgment) is the anti-pattern.

The One Question

Before writing ANY regex / keyword-score / pattern-filter, answer:

Is this signal 100% deterministic and rote — or does it require judgment?

• Deterministic & rote → regex is the right tool. (ISO timestamp extraction, \.mp3$ file filtering, splitting on a known delimiter, magic-byte detection, a URL shape, a YAML frontmatter fence, an ID format you have confirmed is consistent.)
• Requires judgment → the model does it. ("Is this clip a highlight," "is this message important," "does this paragraph contain the thesis," "is this person a real contact," "is this a good title," sentiment / theme / quality.) A regex here rewards surface features — keyword density, length, punctuation — and misses the actual thing.

If you can't answer the question, you have not seen enough data yet. Go look first.

A sharper restatement of the same test: did a MACHINE emit this exact string, or could a HUMAN phrase it a hundred ways? A machine-emitted string in one shape (a calendar prefix, an exact domain, a bot template, a URL/token shape) is a regex tell. A phrase a human writes — and especially one an adversary could imitate — is judgment. The two phrasings agree: "100% deterministic and rote" and "a machine emitted it in one shape" are the same bar.

The Earned-Regex Sequence

A regex is earned, not assumed:

1. Do the work as the model on a small real sample. This is the test-before-bulk discipline (skills/conventions/test-before-bulk.md). Read the actual data. Make the judgments yourself.
2. Notice a tell that is genuinely mechanical — a pattern that holds 100% across the sample, with no judgment in the loop, that you can state precisely. ("Every message from that system is noreply@acme-example.com." "Every transcript segment line starts with **[mm:ss]**.")
3. THEN write the regex to compress that confirmed-deterministic step — to save tokens on the rote part, NOT to make the judgment.
4. Keep the judgment with the model. The regex pre-filters or post-formats; the model still decides anything that isn't mechanical.

Skipping steps 1–2 and jumping to step 3 is premature regex. That's the bug.

Division of Labor

Layer	Tool	Why
Find/format the rote, deterministic part	regex	Cheap, exact, no judgment needed
Decide anything requiring taste/meaning/quality	model	Judgment doesn't compress to a pattern
Confirm a tell is actually rote before trusting regex	model + small-sample test	You must SEE the data first

Regex is a scalpel for parsing, not a brain for judging. Use it to carry out a decision the model already made, never to make the decision.

Red Flags (you are about to write premature regex)

• You're writing a score() function with keyword lists and weights to rank quality.
• You haven't read a representative sample of the source data yet.
• The pattern is meant to decide something a smart human would call a judgment call.
• You're reaching for regex because it's faster than reading, not because the signal is rote.
• The thing you're matching has exceptions you're already hand-waving ("mostly it's…").
• The thing you're matching is exactly what an adversary would imitate (phishing keywords, spoofed brand names, urgency language). A regex on adversary-controlled phrasing is a hole, not a filter.
• You'd be embarrassed to defend the pattern against the 10 counterexamples you haven't looked for.

If any fire: stop, read a small sample, let the model judge, and only regex the mechanical residue — if any.

Green Lights (regex is the right call)

• Extracting a format you've confirmed is consistent (timestamps, file extensions, IDs, URLs).
• Splitting/tokenizing on a known, stable delimiter.
• Magic-byte / binary-shape detection.
• Post-formatting a value the model already chose (slugify a title, normalize whitespace).
• A pre-filter that narrows candidates for the model — explicitly NOT the final decision, and only after you've verified the filter doesn't drop real positives.

Never Regex What an Attacker Can Imitate

When the input is adversary-influenced (inbound messages, webhook payloads, anything a stranger can send), the bar is higher than "rote": the tell must be something the adversary cannot forge cheaply. "Action required," "verify your account," "sign this document" are precisely what a credential-harvesting attacker writes on purpose — a keyword regex that acts on those words is a regex the attacker can drive. "Is this a real request or a spoof" requires checking sender-domain-vs-claimed-identity, thread state, and account context — exactly the judgment the model does and a subject-line regex cannot. When the thing you're matching is what an adversary would imitate, a regex isn't just imprecise — it's a hole.

Cautionary Tales

1. The audio-clip ranking pipeline (scoring as judgment)

A pipeline tried to pick highlight clips from long recordings with a regex score() that counted topic-vocabulary keywords. Result: every clip scored 99–100 (useless for ranking), titles grabbed the first throwaway sentence, themes were incoherent, and it missed nearly every genuine highlight. When a model pass read the transcripts directly and judged, the scores spread 73–92 and the real highlights surfaced. "Is this a good clip" is judgment. It was never a regex job. The regex's only legitimate use would have been finding rough candidate windows for the model to consider — and even that wasn't worth it; reading the transcript was faster and better.

2. The inbox classifier (classification as judgment) — the adversarial twist

An inbound-message pipeline ran deterministic regex rules FIRST and only let the residue fall through to the model classifier. That ordering is correct only for machine-emitted tells. The trap: keyword regexes crept in to make judgment calls before the model ever looked — a school-mail filter matching parent|birthday|grade|library (which match a huge slice of non-school mail), a press-inquiry phrase soup (can you talk|following up.*story — reporters phrase it a hundred ways, newsletters trip it constantly), a newsletter heuristic keying on team@/hello@ localparts (real humans use those), and a financial-action subject regex (action required|sign.*document) that matched exactly what phishing imitates. The classifier prompt could be perfect and still be bypassed by a brittle pattern upstream.

The earned tells in that same pipeline prove the rule by contrast: calendar Accepted:/Declined: prefixes (the calendar system emits them verbatim), exact machine senders (noreply@acme-example.com), a bot's fixed message template, unsubscribe-URL/token shapes. Every one is a string a machine emitted in one shape — not a phrase a human (or an attacker) could write a hundred ways.

The unifying test across both failures: could a human phrase this a hundred ways, and could an adversary imitate it? If yes → judgment, model. Only a string a machine emitted in exactly one shape is a regex tell.

Where This Bites in GBrain

The shipped surfaces this convention protects:

• Enrichment (skills/enrich/SKILL.md) — notability, compiled truth, and which facts matter are judgment calls. Don't keyword-score entity relevance.
• Signal detection (skills/signal-detector/SKILL.md) — "is this original thinking" is the audio-clip failure shape. Score signals with the model, not keyword lists.
• Webhook transforms (skills/webhook-transforms/SKILL.md) — inbound external events are adversary-influenced input. Classify with machine tells
  • model judgment; never with keyword regexes an outsider can imitate.

Relationship to Other Conventions

• Test before bulk (skills/conventions/test-before-bulk.md) is the mechanism for "seeing enough data first." You cannot legitimately decide a signal is deterministic without it. The two are two halves of one rule: look before you compress, compress only the rote.
• Cross-modal review (skills/cross-modal-review/SKILL.md) catches premature regex after the fact: a heuristic-scored output shows no spread (everything maxed). If your scores don't spread, suspect a regex doing a judge's job.