The short version: between late October and mid-November we ran an annual health check on 37 systems still in production, covering backups, certificates and dependencies, accounts and permissions, capacity and logs, and emergency plans. The result was better than expected and worse than expected: no system was close to failing, but three problems turned up almost everywhere — and none of them fail immediately.
The trigger was mundane. On a project two years into production, the client's IT lead changed and nobody could say which services ran on the server or when the certificate expired. Nothing was broken; nobody was confident either. Handover documents, we realised, are rarely the current version two years later. Hence a full sweep.
1 · Five checks, each with a pass criterion
| Check | Pass criterion | If it fails |
|---|---|---|
| Backup and restore | Daily full plus incremental; at least one real restore in the past three months | Restore to a scratch database on the spot and reconcile row counts and totals |
| Certificates and domains | More than 90 days remaining; expiry alerts sent to a current employee | Renew on the client's behalf or configure automatic alerts |
| Accounts and permissions | No shared logins; leavers disabled; at most three administrators | Produce a permission matrix, adjust after client confirmation |
| Capacity and logs | Disk below 80%; logs kept 90 days or more; errors reviewed daily | Add storage, archive, configure rotation and alerting |
| Emergency plan | Written steps for power loss, network loss, device failure; phone numbers answered | Rewrite a one-page plan and walk through it on site |
2 · Three recurring problems, all of them silent
- Certificates nobody watches. Eleven systems expire within 90 days. An expired certificate does not stop the server; it makes every browser show a red warning and stops the business — usually on a Monday morning. The root cause is ownership: registered in whose name, renewal notices to whose mailbox, and after handover nobody owns it.
- Backups run every night and have never been restored. Nine systems. Files exist and sizes look right, but when we ran restores, two backups were incomplete — the script had errored and only part of the tables had been written. An unverified backup is not a backup.
- Shared accounts and leavers still enabled. Fourteen systems, typically a shared admin login with the password taped to the monitor. Not a technical problem but a management one — and when an audit asks, the system owner answers for it.
All three share one trait: they never fail immediately, and when they fail, they fail big. Operations is not about fighting fires; it is about making sure they do not start.
3 · What we changed in the handover package
- The checklist ships with the deployment documents, so client IT can run the same five checks every quarter.
- Certificate expiry reminders 30 days ahead — we register the date and notify, instead of waiting for a red browser bar.
- Restore drills written into operations contracts — at least one real restore a year, with a written record, which also serves as audit evidence.
- A role-to-person permission sheet at go-live, updated when staff change, so accounts do not travel with people.
4 · A five-minute self-check
You do not need a vendor for this. Four questions tell you whether there is latent risk:
- When did you last actually restore a backup? If nobody can answer, that is the risk.
- How many days until your certificate and domain expire, and whose mailbox gets the notice?
- How many shared accounts exist, and were leavers' accounts disabled?
- If the server does not start tomorrow morning, who is the first call — and does that number still get answered?