SQL Server after disk failure — how to secure the database and company environment

SQL Server failure after the disk problem rarely ends with one error message. Often there are slowdowns, inaccessibility of shares, unstable virtual machine or controller errors, and then the company loses access to databases and stops running the circulation of documents, ERPs or accounting modules.

How to distinguish a logical problem from a physical media failure

If SQL Server stops starting after power loss, host reboot or array errors, it cannot be assumed that a standard database repair is sufficient. When there are I/O errors, RAW, datastore problems, unstable array or damaged SSD, the base is only a victim of a larger infrastructure problem.

This is why in B2B environments it is so important to think about a layer of data and a layer of media. In some cases, the right path will be database repair, and in others, you have to perform a diagnostic of the disk, RAID, NAS or VM.

What not to do when SQL does not start after disk failure

• Do not run any repair/attach procedures on the same damaged volume.
• Do not perform uncontrolled migration from damaged MDF/LDF files without protecting them.
• Don't rebuild VM or matrix if you don't know if they are the source of errors.
• Do not update the host, controller or system just to force the environment to start.
• Do not restore backup to production without consistency test and rollback plan.

How to secure the environment from further data loss

First stop the write-generating actions on the damaged environment. Then determine whether you have current backups, what error logs look like, where exactly the database files are located and whether the problem concerns a single instance or the entire stack: host, VM, mass storage and SQL layers.

In practice, it is worth preparing an orderly package of information: system name, SQL version, database location, user symptoms, backup status, last known moment of correct operation and scope of influence on business. This shortens the diagnosis and organizes communication in the team.

Where the most unnecessary risks arise

The most common damage occurs when different people carry out parallel, uncoordinated actions: the administrator rebuilds the host, accounting launches old copies, and someone else exports catalogues from the damaged volume. Without one person responsible for the incident, it is easy to overwrite the data, dispersal the version and lose the material needed for the correct diagnosis.

Backup, logs and test environment — what to prepare before contact

If the company has copies, it is necessary to check not only the existence of them, but also the usefulness: where they come from, whether they contain all the necessary components and whether they can be reproduced outside production. It is also a good practice to secure SQL logs, system logs and storage layers, because they often show whether the failure started with the layer of the database or infrastructure.

In environments based on NAS, RAID or virtualization, information about changes made just before the incident is equally important: disk replacement, array rebuild, VM migration, host update or reboot after power loss. See the guides to a RAID failure in a company and the first 24 hours after a server or NAS failure.

What a good B2B path looks like in such an incident

First you limit damage, then you clean up the evidence, and then you decide if the priority is to recover the environment, repair the base, or switch to backup. If there are customer data, financial documents or employee data in the databases, access, confidentiality and the NDA must be arranged at the start.

For accounting departments and accounting offices, it is also important to decide whether it is a full return to production, recovery of a specific range of data or safe preparation of the environment to be restored on the new server. These are different scenarios and should not be followed by one emergency procedure.

FAQ before escalation

Can you do DBCC CHECKDB or repair immediately after disk failure?

Only if you are sure the media and file system are stable. With a physical or logical failure of the storage layer, such movement may increase the damage range.

Does the backup solve the case automatically?

Not always. The backup may be old, incomplete or inconsistent, so first you have to assess its real usefulness.

When to report a case as a B2B incident rather than an incident involving one base?

When a failure affects the work of several people, environments or business processes, and the source may lie outside SQL itself — e.g. in the storage layer, RAID, NAS or VM.

When to go to service instead of further diagnosing yourself

If the production environment is unstable and the team is uncertain about the consistency of the bases and backups, further attempts in the working production environment increase the risk. In such a situation it is best to go to laboratory contact, indicate business priority and indicate whether the problem is mainly about the base, storage layer or the entire environment. For strictly database issues the right side is database repair and, for a wider corporate incident — B2B path for companies.

Suspect, MDF/LDF and safe next step

If after disk failure, RAID, NAS or VM, the database is running in mode suspect, waits for consistency to be restored or reports MDF/LDF errors, do not try to solve the problem with the service reboot itself. First secure a copy of the database files, SQL logs and media status. Only on this basis select the database diagnostic, media analysis or B2B procedure.

• Do not run DBCC CHECKDB with the option REPAIR_ALLOW_DATA_LOSS on the only copy of the database without first securing the MDF/LDF files, logs and storage-device health information.
• Do not overwrite the production environment with a fast attach/repair if you do not know if the problem started in SQL, file system, VM, RAID or disk itself.