A Regex is not a linguist: What building African Language Correction Model Taught Me About Context

Okay, I have a confession.

I used to think multilingual NLP was mostly about getting the language into the tokenizer... Till I worked with Gikuyu, Swahili and isiZulu.

During AI-BRIDGE Engineering process, one of the most interesting parts of the Bias Correction Layer was realizing that 'multilingual' does not always mean translating the same strategy multiple times... (It would have made it so easy though, right?... but we can't always have it the easy way).

The rule engine had to behave differently because the languages themselves behave differently. And some of those differences were architectural.

Six languages, Six Different Personalities

The active lexicons were not symmetrical.

The project contained approximately:

• 432 active Swahili rules

• 1,326 active Gikuyu rules

• 106 English rules

• 145 French rules

• 162 Hausa rules

• 153 isiZulu rules

That asymmetry is be design. It reflects how we had to model the linguistic risks of each language. For example, Swahili had specific handling for possessives, occupational gender markers, Sheng registers, proverbs and role descriptions.

Gikuyu leaned heavily into morphology, polarity and long phrase matching.

isiZulu used gender linked concordial suffix patterns

French had inclusive writing, and phrase level alternatives

Hausa had dedicated neutral context handling around terms that are frequently neutral in real reporting.

With all these, The engine ended up becoming a small linguistic policy system.

Longest-First matching sounds boring until it saves you

The rule loader sorts biased phrases longest-first. This becomes particularly important for Gikuyu.

Suppose the lexicon contains:

mũtumia wa mũciĩ
mũtumia wa mũciĩ ndangĩhõta
mũtumia wa mũciĩ ndangĩhõta wĩra wa ũtongoria

• Linguistic breakdown:

  • mũtumia wa mũciĩ → "Woman of the household / Housewife"

  • mũtumia wa mũciĩ ndangĩhõta → "A housewife cannot manage..."

  • mũtumia wa mũciĩ ndangĩhõta wĩra wa ũtongoria → "A housewife cannot manage leadership work"

• Why shortest-first fails here: Matching mũtumia wa mũciĩ first triggers a generic gender-role flag or rewrite, destroying the input string before the engine can catch the specific high-severity policy rule targeting leadership discrimination (...ndangĩhõta wĩra wa ũtongoria).

The loader therefore treats phrase length as part of the disambiguation strategy. I prepared a simple illustration to help describe this better.

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'background': '#050505',
      'primaryTextColor': '#e0ffff',
      'lineColor': '#00e5ff'
    }
  }
}%%
flowchart TD
    %% Global Link Styling (Cyan, Dashed for the flow effect)
    linkStyle default stroke:#00e5ff,stroke-width:2px,stroke-dasharray: 5 5;

    %% Nodes
    A[(Load lexicon)]
    B(Normalize rows)
    C(Sort by biased phrase length)
    D{{Longest phrase first}}
    E(Regex / literal matching)
    F{Context gating}
    G([Rewrite or warn])

    %% Flow Connections
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G

    %% Neon Cyberpunk Color Palette for Black Backgrounds
    %% Cyan: #00e5ff | Electric Blue: #0077ff | Neon Green: #39ff14 | Purple: #b000ff | Pink: #ff007f
    classDef dataNode fill:#001a1a,stroke:#00e5ff,stroke-width:2px,color:#00e5ff,font-weight:bold;
    classDef processNode fill:#0a0a14,stroke:#0077ff,stroke-width:2px,color:#80bfff;
    classDef matchNode fill:#051a05,stroke:#39ff14,stroke-width:2px,color:#b3ffb3;
    classDef gateNode fill:#1a0a1a,stroke:#b000ff,stroke-width:2px,color:#e6b3ff;
    classDef actionNode fill:#1a000d,stroke:#ff007f,stroke-width:3px,color:#ffb3d9,font-weight:bold;

    %% Assigning Classes
    class A dataNode;
    class B,C processNode;
    class D,E matchNode;
    class F gateNode;
    class G actionNode;

This is one of those engineering choices that looked trivial during code review and became extremely important once real text arrived.

Swahili required more than word matching

Some of the most interesting rules were not obvious gendered nouns. We realzed that possessive words such as: yake, wake, zake can be completely neutral depending on what they refer to.

Nyumba yake is not the same linguistic problem as daktari wake

So the context checker introduced a human-referent gate and an inanimate-noun gate.

That is the kind of logic that almost disappeared when we tried to treat different languages as a bag of strings.😅😅😅

And then there was counter-stereotype context

This is where things became even more interesting. A phrase may contain the same gendered term, but the surrounding sentence may be challenging the stereotype rather than expressing it (aha!... you see where I'm going with this?)

For example, a sentence may contain the same gendered term but the surrounding sentence may be challenging the stereotype rather than expressing it.

A sentence may mention women in a sentence about women succeeding in an occupation. A context-blind replacement system could correct the sentence that was already doing the correcting.

That is why the context checker contained dedicated logic for counter stereotypes, advocacy language, celebration, biographies, statistics and other situations.

So rather than our rule-engine asking itself Does this word occur? instead, it asks itself should this word be changed in this context?

The deeper engineering lesson

Low resource language engineering forces you to stop hiding behind averages.

When the conventional English tools we are used to are slightly wrong, someone may call it a quality issue. But when an African language system is wrong, the engineering mistake can be much harder to diagnose because there may be less training data ...(I'll definitely talk about this and other challenges we faced in a dedicated post).. ,fewer benchmarks, fewer reviewers and more morphological complexity.

That makes explicit policy very valuable.

You need to know:

Which rules are deterministic?
Which are warnings?
which contexts suppress edits?
Which substitutions are allowed?
Which languages require human review?
Which metrics actually reflect runtime behaviour?

This level of explicitness is what makes AI-BRIDGE debuggable.

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'background': '#050505',
      'primaryTextColor': '#e0ffff',
      'lineColor': '#00e5ff',
      'edgeLabelBackground': '#0a0a14'
    }
  }
}%%
flowchart LR
    %% Global Link Styling (Cyan, Dashed for the flow effect)
    linkStyle default stroke:#00e5ff,stroke-width:2px,stroke-dasharray: 5 5;

    %% Nodes with Semantic Shapes
    A([Matched phrase])
    B{What is around it?}

    C([Do not rewrite])
    D([Usually preserve])
    E([Usually preserve])
    F([Preserve / suppress])
    G([Rewrite])
    H([Warn / review])

    %% Flow Connections
    A --> B
    B -->|Name / biography| C
    B -->|Quotation| D
    B -->|Statistics| E
    B -->|Counter-stereotype| F
    B -->|Known harmful stereotype| G
    B -->|Ambiguous| H

    %% Neon Cyberpunk Color Palette
    %% Cyan (Input) | Purple (Decision) | Neon Green (Preserve) | Pink (Rewrite) | Yellow (Warn)
    classDef startNode fill:#001a1a,stroke:#00e5ff,stroke-width:2px,color:#00e5ff,font-weight:bold;
    classDef gateNode fill:#1a0a1a,stroke:#b000ff,stroke-width:2px,color:#e6b3ff;
    classDef safeNode fill:#051a05,stroke:#39ff14,stroke-width:2px,color:#b3ffb3;
    classDef actionNode fill:#1a000d,stroke:#ff007f,stroke-width:3px,color:#ffb3d9,font-weight:bold;
    classDef warnNode fill:#1a1500,stroke:#ffcc00,stroke-width:2px,color:#ffe680;

    %% Assigning Classes
    class A startNode;
    class B gateNode;
    class C,D,E,F safeNode;
    class G actionNode;
    class H warnNode;

Alright, we have come to the end of this article.

Adios! See you on the next one!