Data Migration: What to Carry, What to Abandon
The instinct is to bring everything across. It is the wrong instinct, and it is why so many migrations end with a clean system full of dirty data. What to carry, what to archive, what to deliberately abandon — and the four hours of deduplication that decide whether your reports mean anything.
Every migration begins with somebody saying "we have eight years of history in this workbook" as though that were an asset. Usually it is eight years of a naming convention that changed four times, three people's abbreviations, and a column that meant one thing until 2023 and something else afterwards. Carrying that across does not preserve history. It launders it into a system that will now report it confidently.
The useful question is not what can be migrated. It is what a report will be wrong about if you migrate it.
Three categories, and only one of them travels
Bring it across
- The master lists — items you still stock, customers you still sell to, suppliers you still buy from, assets you still own.
- Open positions — what customers owe you, what you owe suppliers, purchase orders placed and not yet received.
- Current stock, established by counting it, not by copying a figure.
- Anything a regulator or auditor could ask for in the next seven years — as files in a document store, not as records.
Leave it behind
- Closed transactions. Last year's sales, settled invoices, completed orders. Keep the workbook; do not import it.
- Items you have not moved in two years. They are the reason your new item list will be unusable on day one.
- Customers with no activity since 2023. A CRM full of dead contacts is a CRM nobody searches.
- Anything you cannot explain the meaning of. If nobody in the room knows what the column means, importing it does not recover the knowledge.
The middle category is where the real argument happens, because "open positions" sounds simple and is not. An invoice showing KES 40,000 outstanding since 2022 is not an open position; it is a decision you have been avoiding. Migration forces the decision, which is one of its genuine benefits and one of the reasons people resist it.
The [deduplication](/glossary/deduplication) pass nobody budgets for
This is four to twelve hours of unglamorous work and it determines whether every report you run for the next five years is meaningful. Do it in the spreadsheet, before any import, because it is far easier there.
The item list clean-up, in order
- Sort alphabetically and read it. You will find the same thing three times — "Cement 50kg", "Cement 50 kg", "CEMENT-50KG" — and you will find them in the first two minutes.
- Decide the naming convention before you merge, not while merging. Noun first, then the distinguishing attribute, then the pack size: "Cement, Portland, 50kg". Boring is correct.
- Pick a survivor for each duplicate group and note which records need to point at it. Do not merge quantities yet — the count in week five will settle quantities.
- Strip anything you have not moved in two years into a separate tab called "archive". You are not deleting it; you are keeping it out of the live list.
- Fill in the unit of measure on every surviving row. This is the field people skip, and it is the one that later makes "we bought 20" ambiguous between cartons and pieces.
- Add a barcode column, even if it is empty for most rows. Retrofitting barcodes across three thousand items is a project; capturing them as you go is a habit.
- Give every row a category, from a list of no more than fifteen. Forty categories is the same as none.
What each import template actually needs before it will load
Template The column that stops it
Items Name and an inventory account — not the price
Customers Company name. Everything else is optional
Vendors Name and email, because a supplier with no email cannot be invited
Warehouses A stable code — other templates reference it by that code
Locations A warehouse code that already exists or is in the same batch
Assets Name, asset type, and whether it is tracked individually or as a pool
Chart of accounts Account names — there is no account number column to fill
Load warehouses and locations first, then items, then customers and vendors, then assets. The order is not arbitrary: later templates reference earlier ones by code, and a warehouse code that does not exist yet is the most common reason a location file fails on row one.
Where imports go wrong, in descending order of frequency
- A changed heading. The template ships with the exact column names the importer looks for. Renaming "Buying Price" to "Cost" does not make it clearer; it makes it invisible.
- Numbers stored as text. Excel will happily keep "1,200" as a string with a comma in it. So will the file you export from your old system.
- Trailing spaces on names. "Cement " and "Cement" are two items, and you will not see the difference on screen.
- A code referenced before it exists — a location pointing at a warehouse in a file you have not uploaded yet.
- One heroic file. Three thousand rows in one upload means one bad row can stop the batch and you have no idea which. Split it: fifty rows first, check what arrived, then the rest.
- Dates in the wrong order. 05/03/2026 is ambiguous and your source system and your spreadsheet may disagree about it. Use the format the template shows.
Do the fifty-row test on every file
Upload fifty rows. Open five of the records that were created. Look at every field, not just the name — particularly the unit of measure, the category and the cost. Then upload the rest. This takes ten minutes and it is the difference between fixing one file and fixing three thousand records.
What our own importers do and do not do
What AWRA OpsHub does today
- Thirteen templates as real spreadsheets with a sample row, the required columns marked, and per-template notes on the accepted values.
- Row import on eight of them: items, customers, vendors, assets, asset custodians, warehouses, locations, chart of accounts.
- Created / updated / skipped counts reported after an inline upload, so a file that half-worked tells you it half-worked.
- Files over 256 KB processed in the background, with a notification when they finish, so a large item list does not time out a browser.
- Any custom fields you have defined for a module appear as extra columns on that module's template automatically.
- Employees and attendance imported through a genuine preview-then-commit flow.
What it does not do
- No preview on the eight general templates. You find out what a file did after it did it.
- No undo. Correcting a bad import means correcting records.
- No importer at all for stock adjustments, stock transfers, requisition lines, RFQ lines or opening balances — those five templates download only.
- No column mapping screen. The headings have to match the template rather than being matched to it.
- No reader for any specific competing product's export format.
The absence of a preview is the one that changes behaviour: it is exactly why the fifty-row test above is a rule here rather than a suggestion.
The archive decision, which is a compliance decision
What you leave behind still has to be findable. In Kenya that means keeping records available for the periods the Tax Procedures Act and your sector regulator expect, and it means being able to produce them if someone asks — which nobody can do from a workbook that lives on a laptop that left with an employee.
The practical answer is boring and works: export the old system to spreadsheets and PDFs, name them by year and module, put them somewhere with controlled access and a backup, and write down where they are in a one-page note that survives the person who made it. Then migrate only what the new system needs to operate. Data exit, retention and deletion covers the same problem from the other end — the day you leave a system rather than the day you join one — and the two are worth reading together because the second one tells you what to demand of your new vendor while you still have leverage.
Our take
Migrate master lists and open positions; archive everything else as files and stop feeling guilty about it. Spend the four hours on deduplicating the item list — it is the highest-return work in the entire rollout and the only part nobody can do for you. And test every file with fifty rows first, because no importer in this price bracket, including ours, will show you a diff before it commits.
Send us your item list before you commit to anything
We will tell you what state it is in, roughly how long the clean-up will take, and which of your columns will not survive the trip. It is a more useful first conversation than a demo.
See plans & pricingFrequently asked questions
Should we migrate our full transaction history?
Almost never. Closed transactions belong in an archive of spreadsheets and PDFs, organised by year and module, kept somewhere with controlled access. Migrate the master lists, the open positions, and the current stock. Importing years of settled history mainly imports years of inconsistent naming, and it makes your new reports confidently wrong rather than usefully empty.
How long does data cleaning take?
Four to twelve hours for the item list of a typical SME, depending on how many duplicates there are and whether anyone has ever agreed a naming convention. Customers and suppliers are quicker. It is the single highest-return task in a rollout and the one part a vendor cannot do for you, because most of the decisions require knowing which of two similar items you actually still stock.
What order should we import in?
Warehouses, then locations, then items, then customers and vendors, then assets and custodians. Later templates reference earlier ones by code — a location row pointing at a warehouse code that does not exist yet is the most common first-row failure.
Can we import stock quantities?
Not through an importer. The stock-adjustments template downloads so you can see the layout, but there is no upload behind it, because a bulk load would bypass the approval and posting rules that make an adjustment trustworthy. Opening stock is established by a physical count entered as a count session — which is better practice anyway, since a copied figure inherits whatever was already wrong.
What if our column headings do not match the template?
Change your headings to match. There is no column-mapping screen, and the importer looks for the template's exact names — a renamed column is simply not read, which shows up as a field silently arriving empty rather than as an error.
Can a bad import be undone?
No. There is no rollback, so a bad file is corrected by correcting records. This is exactly why the fifty-row test matters: upload fifty rows, open five of the created records and check every field, then upload the rest.
