Skipping software integrations altogether: manual re-entry and nightly files

Every connection you build is a small piece of software that runs unattended, depends on two systems you do not control, and belongs to whoever built it until they leave. That is the honest description, and it is why the useful question is not how to connect two products but whether to.

This page is about the options that are not a connection, and about what each real option costs to keep running rather than to build.

What to take away

Six options, including the ones nobody proposes

The first row is the one to argue for hardest. A great deal of connection work exists to keep two copies of the same list in step, and the copies exist because two teams each wanted their own. That is a decision, not a technical requirement.

Manual re-entry, done deliberately

There is a threshold below which automating something costs more than doing it. It is higher than engineers like and lower than managers assume.

Work it out instead of guessing. Count records per week, minutes each, and the error rate. Estimate the build honestly, then add maintenance: an hour every few months when something changes, plus a day when it breaks during a holiday.

Twenty records a week at two minutes each is forty minutes. A connection that takes three days to build and half a day a quarter to maintain does not pay that back quickly.

What makes manual re-entry acceptable is that it is visible. When it stops, somebody knows immediately, because the person doing it is the alarm. Automated connections fail silently, which is the whole problem.

The shared reference file

For lists that change rarely and are read by several processes, one agreed file in one agreed place is a genuine architecture. Cost centers, office locations, product codes, holiday dates.

Two rules make it work. One named owner, the only person who edits it. A format that outlives the tool that made it, so plain, boring, and openly specified.

The reasoning for preferring open formats in shared use is set out in the open standards principles. It applies to a file three teams read as much as to a national service.

Scheduled export and import

Underrated, and the right answer more often than a platform. A file is produced on a schedule, moved, and loaded. It is slow, it is honest, and when it fails there is a missing file, which anybody can see.

Three things decide whether it works. Both ends must read the format the same way, because a date or decimal separator interpreted differently quietly breaks it.

The schedule must match the need, and most needs are hourly at most, while failures must reach a person, so check the file arrived rather than hope it did.

On formats, a useful habit is to prefer ones that are documented and widely readable rather than whatever the tool exports by default. The published tables of acceptable file formats used for records transfer are a practical shortlist of formats that survive, and something that survives a records transfer will survive your nightly job.

What each one costs to keep

Build cost is the number people compare. Keeping cost is the number that decides.

For each option, answer four questions. Who is told when it fails, and by what mechanism? Who is allowed to fix it, and are they available in August? What happens when one of the two systems is upgraded? And how would anybody discover that it has been quietly producing wrong results for a fortnight?

The fourth is the worst case in this whole subject. A connection that stops is annoying. A connection that keeps running and drifts is expensive, and it is discovered by someone noticing that two numbers do not match, months later.

Choosing between them

Work down the list rather than up it. Can the duplication be removed? If not, is the volume low enough for a person? If not, does a scheduled file do it? If not, does a native connector exist? Only then a platform, and only then custom code.

That order is unfashionable because it starts with the least impressive answers. It is also the order that produces the fewest unowned pieces of software running in the background of your organization. The category overview covers how those accumulate, and the selection method is the right read once you have concluded that a platform genuinely is the answer.

Where this fits

If work is scattered across products, the tracker configuration may remove the need instead of automating it. If documents live in two places, the storage conventions matter more than any connection.

Every option above needs a credential from something, not someone, settled in the administration layer.

Common questions

Our two products advertise an integration. Is it enough?

Check what it actually synchronizes before relying on it. Advertised integrations are often narrower than the word suggests: sign in only, or a notification, or a one way copy of a single object. Test the specific fields you care about.

Is a nightly export really acceptable in 2027?

For most internal reporting, yes. Ask who is waiting for the data. If the answer is a person watching a screen, you need something faster. If the answer is a weekly meeting, nightly is generous.

How do we stop connections accumulating?

Keep one register with an owner and a review date per connection, and require a named owner before anything is built. The register is more effective than any policy, because it makes the count visible, and the count is what shocks people.

What if the person who built our connections has left?

Treat every one of them as unknown until proven otherwise. Find what runs, what credential it uses, and what breaks if it stops. Then rebuild the ones that matter in something a second person can read, and switch off the ones nobody can justify.