Master Data Cleansing

Master data cleansing (also data cleansing) is the systematic correction, standardisation and enrichment of master data — typically supplier, customer, item or account master records. The goal is a consistent, duplicate-free and complete dataset that accounting, procurement and reporting can rely on. AI-powered cleansing also catches duplicates and matches that rule-based tools miss.

Key takeaways

• Poor data quality costs organisations an average of 12.9 million US dollars per year (source: Gartner, “How to Improve Your Data Quality”, 2021).
• Typical problems: duplicates (the same supplier created several times), inconsistent spellings, outdated or missing fields (IBAN, VAT ID, addresses).
• AI cleansing works semantically rather than character-based: “Müller GmbH & Co. KG” and “Mueller GmbH” are recognised as one supplier.
• Cleansing only becomes sustainable with continuous matching: every new document is automatically checked against the master data (master data matching).

What errors typically hide in master data?

Master data degrades gradually — through time pressure at record creation, system migrations, acquisitions and simply the passage of time. The most common error classes:

• Duplicates: The same supplier or customer exists multiple times — with slightly different spelling, a different legal form or an old address. Consequences: split purchasing volumes, duplicate vendor accounts and, at worst, duplicate payments.
• Inconsistent normalisation: “Str.” vs. “Straße”, country codes vs. spelled-out names, different formats for phone numbers, IBANs or VAT IDs.
• Outdated details: renamings, relocations, changed bank accounts that were never updated.
• Gaps: missing mandatory fields such as payment terms, tax numbers or contacts.

How does AI-powered master data cleansing work?

The process runs in four steps:

1. Inventory analysis. The full dataset is ingested and measured: duplicate candidates, normalisation deviations, field gaps — with metrics per error class.
2. Duplicate detection. An AI model compares records semantically, not just character by character. It considers names, addresses, IBAN, VAT ID and — where available — document history: two vendors with the same bank account are very likely the same supplier.
3. Normalisation & enrichment. Spellings are unified, formats standardised, missing fields — where possible — completed from documents or registers.
4. Approval & write-back. Clear-cut corrections run automatically; uncertain matches are presented for human approval with a confidence score. The cleansed result flows back into the ERP.

The decisive part is the connection to daily operations: with master data matching, every new document is checked against the cleansed dataset — new duplicates never arise in the first place.

Worked example: what does cleansing actually save?

An illustrative calculation for a vendor master with 20,000 suppliers:

• In grown datasets, several percent of records are typically duplicates — at an assumed 5%, that is 1,000 duplicated vendors.
• Manual clarification (search, compare, merge, document) easily takes 15 minutes per case: 1,000 × 15 min = 250 hours of pure duplicate work — before any normalisation or gap-filling.
• With AI support, human work shrinks to approving uncertain matches — typically a fraction of the cases; the rest runs automatically.

Add the avoided downstream damage: every undetected duplicate can lead to duplicate payments, wrong procurement analytics and lost early-payment discounts — part of the 12.9 million US dollars per year that Gartner attributes to poor data quality.

Frequently asked questions (FAQ)

What is master data cleansing?

The systematic correction, standardisation and enrichment of master data — such as suppliers, customers, items or accounts. It includes duplicate detection, normalisation, completion of missing fields and validation.

Why is poor master data expensive?

Gartner puts the cost of poor data quality at an average of 12.9 million US dollars per organisation per year (2021). Concrete consequences: duplicate payments, misdirected postings, wrong analytics, heavy clarification effort.

How does AI find duplicates that classic software misses?

Classic tools compare character strings; AI compares meaning — and adds context signals such as IBAN, VAT ID and invoice history. Different spellings and legal forms are recognised as the same record.

Is master data cleansing a one-off project?

No. The sensible setup is an initial cleanse plus continuous matching, where every new document is automatically checked against the master data.

Does cleansing run automatically or with human approval?

Both: clear-cut cases automatically, uncertain matches with a confidence score for approval (human-in-the-loop).

---

Further reading: The solution page Master Data Cleansing with an AI Agent shows how feld.ai detects duplicates, matches records and keeps data quality high — EU-hosted on our own servers in Austria.