“Clean the product file” may produce neat rows and consistent names while selling units remain wrong and codes lose leading zeros. A tidier appearance is insufficient acceptance evidence. Define corrections, review flags and who approves the meaning of each change.
This guide concerns commissioning defined data-cleaning outputs before system loading or analysis. It does not assume complete migration or customer-identity merging. One file and a few documented rules can be a suitable start if repeatability and remaining limitations are understood.
Let purpose define required quality
A fictional maintenance-parts distributor stores codes, descriptions, units and manufacturers for an internal search catalogue. It may need standardized manufacturer names and interpreted units while retaining old descriptions for reference. Text variation need not be deleted when it carries useful meaning.
The UK Government Data Quality Framework distinguishes completeness, accuracy and fitness for purpose. Populated cells need not be correct, and technically valid values may misrepresent reality. Use these distinctions to frame requirements without treating government guidance as company certification or a mandatory process.
A rule register before tool execution
| Observation | Approvable rule | Review case |
|---|---|---|
| Codes with leading zeros. | Preserve text according to product-code definitions. | Lost zeros without an authoritative recovery source. |
| Variant unit labels. | Map to approved units without implicit quantity changes. | A symbol meaning a pack or piece depending on supplier. |
| Inconsistent manufacturer names. | Apply a reference mapping approved by the data owner. | Similar names without evidence of the same manufacturer. |
| Missing required values. | Flag gaps and refer to an identified source. | Unrecoverable information rather than plausible invented values. |
Add rule identifiers, approvers, scope and whether execution corrects or merely suggests. Do not apply blanket whitespace removal or numeric conversion before understanding affected fields: a transformation suitable for descriptions may damage codes. Review a sample before broad processing.
Retain the source and explanations of differences
Request a preserved source under agreed access policy, a separate result and a register linking old and new records and changes to rules. Review reports need not contain secrets or personal details. Data owners should explain individual changes, and implementers should reproduce results from the same inputs.
If stable identifiers are absent, agree a traceable reference before sorting or merging rows. Changing row numbers should not be the sole explanation mechanism. Record rule versions: rerunning modified rules does not demonstrate reproduction of earlier results without explaining differences.
Separate reference suggestions from established facts
A tool might suggest a manufacturer name resembling the source text. OpenRefine describes reconciliation as semi-automated matching requiring review and judgment. Similarity alone is insufficient identity evidence, and references may not cover every item. Separate confirmed, suggested and unmatched values rather than hiding them in one tidy column.
Before using an external reference service, identify necessary transmitted data and whether sharing is authorized and appropriate. Internal references or local processing may be more suitable than sending complete files. Naming a tool establishes neither universal suitability nor a requirement for external services.
Read quality reports with denominators and exceptions
Request before-and-after measurements for each included rule: rows checked and cases passed, corrected or unresolved. Separate improved completeness from accuracy verified against approved sources. One percentage must not conceal difficult-row exclusions or changed success definitions.
Exercise reporting with known synthetic cases: a valid zero-prefixed code, ambiguous unit, similar but unconfirmed name and missing value. Inspect results and exceptions. Agree who corrects future source errors and how recurrence in subsequent files is addressed; repeated cleaning alone does not repair input causes.
The customer-duplication guide handles identity and merge decisions, while Power BI readiness examines analytical relationships. Send the rule register and sample when requesting data services from Nextwave, making cleaning reviewable without promising absolute accuracy.
Quality references and example limits
The GOV.UK Data Quality Framework and OpenRefine reconciliation guide were opened on 4 November 2026. Rules and examples are illustrative rather than actual processing of company files. AI assisted preparation, with information and translation review.
