6 points worth checking on software migration reviews

Accounts of migrations are written by two groups: people selling the service, and people who had a bad time. The middle, which is the ordinary migration that mostly worked with four annoying surprises, is barely written down anywhere, and that is exactly the experience you are about to have.

So this page does two things. It sets out what outside accounts are actually good for, and it describes the review you should run on your own project, because the only migration report that will ever be written about your estate is the one you write.

What to take away

What outside accounts are good for

Three things, and they are worth having.

Surprises. Somebody describing the specific thing that caught them out, such as a permission model that mapped differently than the documentation implied, is telling you what to test. That is useful regardless of whether their conclusion matches yours.

Sequencing. Accounts that describe the order of operations, especially where the author says they would do it differently, save real time. Order matters enormously in this work and it is the part that is hardest to reason about from first principles.

Recovery. What people did when something went wrong, which is the part no supplier will describe, and which tells you what your fallback needs to cover.

What they are not good for is deciding. The author moved a different estate between a different pair of systems with different obligations. Their verdict is about their pairing.

Ask a reference customer the right questions

Suppliers will offer a reference. The wasted version of that call asks whether they were happy.

Ask instead: what did you have to rebuild by hand? Then work through the list.

Ask for a reference of roughly your size doing roughly your kind of work. A reference ten times your size had a named engineer and a project team; you will have a queue and somebody's afternoons.

The review you run on yourself

Within two weeks of cutover, while it is fresh. One meeting, one document, no blame.

Check the destination against a baseline, not your memory of the source. Configuration drifts during a migration: settings get relaxed to make something work at two in the morning and never get put back.

Working through published secure configuration baselines, such as the secure cloud business applications project, catches the sharing default somebody widened and forgot.

Prove you still have what you had

The quiet failure of a migration is not that something broke. It is that something is missing and nobody notices for a year.

About a month after the move, run three checks. Pick twenty random items from the pre migration inventory; confirm each exists, is readable, and is reachable by the right people.

Open something in the oldest format you hold, and confirm anything kept for obligation reasons is still identifiable as such, since retention labels often fail to travel.

Being able to demonstrate that content is still present, still readable, and still what it claims to be is the whole discipline of digital preservation, and a migration is exactly the moment when those three properties are most at risk.

Write it for the next person

The review's audience is whoever runs the next move, in three years, when nobody currently involved is still in post. Write for them: what the estate looked like, what the pairing was, what the tool did badly, what we would test first.

Store it somewhere it will be found. A document in a project folder that gets archived is a document that does not exist. Put it where migrations are discussed, and link it from wherever your inventory lives.

Where this fits

The category overview sets out what generally survives a move. The preparation work is where most of the review's findings will point, since almost every surprise traces back to something the dry run did not include. The tool questions are worth revisiting with your own experience attached, and the outstanding items list will mostly be about the connection rebuild.

Common questions

Is a supplier case study worth reading?

As a description of what is possible, yes. As evidence of what is typical, no. The selection of which projects become case studies is the entire point of them, and the projects that went badly are not in the set.

Should the review be shared beyond the project team?

A short version, yes, because people who lived through the disruption deserve to know what happened and what is still outstanding. Keep the detailed version internal so that it can be honest about what went wrong without anyone managing their reputation in it.

What if the migration went well?

Write the review anyway, and be suspicious. A migration with no findings usually means nobody looked hard, particularly at permissions and at monthly processes. Run the twenty item check regardless.

How long should we keep the old system after the review?

Long enough to cover one full monthly cycle after the review, then close it on an announced date. The review itself will usually identify the one thing still needed from it, and that thing should be exported deliberately rather than left as a reason to keep paying.