RAID Levels
Combining disks for performance, redundancy, or both
What RAID Is
RAID — Redundant Array of Independent Disks (originally "Inexpensive" in the 1988 Berkeley paper by Patterson, Gibson, and Katz) — combines multiple physical block devices into a single logical device. The goal is one or more of:
- Performance — parallelise I/O across spindles (striping)
- Fault tolerance — survive disk failures via redundant copies or parity
- Capacity aggregation — present multiple disks as one large volume
The Core Abstraction
RAID sits between the raw block devices and whatever consumes them (filesystem, LVM, database). The consumer sees a single device (/dev/md0, a hardware controller's logical drive, a ZFS vdev). The RAID layer transparently splits reads/writes across members and reconstructs data on failure.
Note
RAID is not backup. It protects against disk failure, not against filesystem corruption, ransomware, rm -rf /, fire, theft, or simultaneous multi-disk failure. Backups are an independent concern.
RAID 0 — Striping
Block-level striping across N disks. A chunk of a logical block is written to disk 1, the next to disk 2, and so on, wrapping around. No redundancy, no parity.
Layout (4 disks)
- Capacity: N × smallest_disk
- Throughput: up to N× sequential read/write (IOPS also scale)
- Fault tolerance: zero. Any single disk failure destroys the entire array (each stripe is split across all disks).
- Minimum disks: 2
Warning
Failure math: Annual failure rate of N disks is roughly N × per-disk AFR. A 4-disk RAID 0 is ~4× more likely to lose all data than a single disk. Use only for scratch space, caches, or throwaway compute.
RAID 1 — Mirroring
Every write goes to every member disk. All disks hold an identical copy. Reads can be served from any member.
Layout (2 disks)
- Capacity: 1 × smallest_disk (i.e. 1/N of raw)
- Read: scales with N (reads can be dispatched to any member)
- Write: bottlenecked by the slowest member — every write hits every disk
- Fault tolerance: survives N−1 failures (any single disk is a complete copy)
- Minimum disks: 2
Tip
Classic use: boot drives, OS volumes, small critical datasets. Simple to reason about — rebuild is a straight block-for-block copy, no parity math.
RAID 5 — Distributed Parity
Striping plus one parity block per stripe, rotating the parity position across disks so no single disk is a parity bottleneck. Parity is XOR of the data blocks in the stripe.
Layout (4 disks) — P rotates across stripes
- Capacity: (N−1) × smallest_disk
- Fault tolerance: survives 1 disk failure. Reconstruct missing block as XOR of the survivors.
- Read: scales ~N−1×
- Write penalty: a small write requires read-modify-write (RMW) — read the old data + old parity, compute new parity (
new_parity = old_parity XOR old_data XOR new_data), write new data + new parity. 4 I/Os per logical small write. - Minimum disks: 3
The Read-Modify-Write penalty
Small-write workflow on RAID 5/6
Read old data
from the target disk
Read old parity
from the parity disk
XOR compute
new parity
Write data
+ write parity
Full-stripe writes avoid RMW: if the filesystem or cache can coalesce writes to a full stripe, parity is computed from the new data directly without reading anything first. This is why large stripe-aligned workloads (streaming backup, video) do well on RAID 5 while small random writes (databases, VMs) do not.
RAID 6 — Double Parity (P + Q)
Same pattern as RAID 5, but with two parity blocks per stripe. P is ordinary XOR parity; Q is a second, linearly-independent syndrome computed using Reed-Solomon coding over GF(2⁸). Both rotate across disks.
Layout (5 disks) — P and Q rotate
- Capacity: (N−2) × smallest_disk
- Fault tolerance: survives 2 simultaneous disk failures
- Write penalty: heavier than RAID 5 — small writes become 6 I/Os (read old data, old P, old Q; write new data, new P, new Q). Q computation is CPU-heavier than XOR.
- Minimum disks: 4
Tip
Why bother with RAID 6? On large modern arrays, rebuild time after a single failure can run for days. RAID 6 lets the array tolerate a second failure during that rebuild window. Combined with the URE issue (see below), RAID 6 is the pragmatic default for big arrays.
RAID 10 — Striped Mirrors (1+0)
Mirror pairs first, then stripe across the mirrors. A "nested" level. Do not confuse with RAID 01 (stripe then mirror), which is rare and has worse failure characteristics.
Layout (4 disks) — 2 mirrored pairs, striped
- Capacity: 50% of raw
- Fault tolerance: survives multiple disk failures as long as not both halves of the same mirror die. A 4-disk RAID 10 can survive 2 failures if they're in different mirrors, but loses everything if a single mirror pair fails together.
- Read: scales with N (any member of any mirror can serve a read)
- Write: full-bandwidth stripe writes, no RMW, no parity math
- Rebuild: straight block copy from the surviving mirror member — fast, and only stresses one other disk
- Minimum disks: 4 (always even)
Tip
The database default. RAID 10 sacrifices capacity for predictable latency and graceful rebuilds. For random-write OLTP workloads it's usually the right answer.
RAID 50 & RAID 60 — Striped Parity Groups
Take multiple RAID 5 sets (or RAID 6 sets) and stripe across them (RAID 0 on top). Each parity group is independent: a failure in one group doesn't touch the rebuild of another.
RAID 50
Stripe across N RAID 5 sets. Tolerates 1 failure per set (up to N total, never 2 in the same set).
Rebuild is scoped to one RAID 5 group, so the rebuild window only stresses that group's disks.
RAID 60
Stripe across N RAID 6 sets. Tolerates 2 failures per set. Preferred for very large arrays (20+ disks) where rebuild time makes RAID 6 alone risky.
Note
When to reach for 50/60: 16+ disk JBODs. Breaking one giant RAID 6 into, say, three RAID 6 groups of 8 disks each keeps rebuild windows bounded and reduces the blast radius of any single event.
Comparison Across Levels
| Level | Min disks | Usable capacity | Fault tolerance | Read perf | Write perf | Typical use |
|---|---|---|---|---|---|---|
| RAID 0 | 2 | N | None | N× | N× | Scratch, caches |
| RAID 1 | 2 | 1 | N−1 disks | N× | 1× (slowest member) | Boot, OS, small critical data |
| RAID 5 | 3 | N−1 | 1 disk | (N−1)× | Penalty on small writes (RMW) | Read-heavy, archive (small arrays only) |
| RAID 6 | 4 | N−2 | 2 disks | (N−2)× | Heavier RMW than RAID 5 | Large bulk storage, backup targets |
| RAID 10 | 4 | N/2 | 1 per mirror pair | N× | (N/2)×, no RMW | Databases, VMs, latency-sensitive |
| RAID 50 | 6 | N − (groups) | 1 per group | High | Better than RAID 5 alone | Large arrays, balanced perf/capacity |
| RAID 60 | 8 | N − 2×(groups) | 2 per group | High | Heaviest write cost | Very large arrays with bounded rebuild windows |
The Write Hole
A fundamental vulnerability of parity RAID (5, 6, 50, 60). Consider a stripe update:
- Controller issues writes for data and parity to different disks.
- Power fails after the data write completes but before the parity write completes (or vice versa).
- On reboot, data and parity are inconsistent — but neither disk reports an error; each one just has "its" block.
- If a different disk in the stripe now fails, the rebuild reconstructs the missing block using the stale parity. The resulting data is silently wrong.
Warning
Write hole = silent corruption on rebuild. This is not a theoretical problem; it's the single biggest reason parity RAID has a bad reputation with careful operators.
Mitigations
| Approach | Mechanism | Caveat |
|---|---|---|
| BBWC / FBWC | Controller cache backed by battery or flash/supercap — outstanding writes survive power loss and complete on next boot | Battery health must be monitored; dead battery = cache disabled = massive perf drop |
| UPS | Prevent the power event in the first place | Doesn't help against kernel panic or controller failure |
| Write-intent bitmap + journal (mdadm) | Bitmap tracks dirty stripes; journal device (SSD) persists stripe updates before committing | Journal device becomes a SPOF unless mirrored |
| Avoid parity RAID | RAID 10 has no write hole (no parity to go stale) | 50% capacity cost |
| Copy-on-write RAID | ZFS raidz / btrfs raid5/6 write new stripes to free space; never overwrite live data | Different paradigm — replaces mdadm entirely |
UREs and Why Large RAID 5 Arrays Are Dangerous
Every consumer/enterprise HDD has a published Unrecoverable Read Error rate. Typical specs:
- Consumer SATA: ~1 URE per 1014 bits read (≈ 12.5 TB)
- Enterprise SATA/SAS: ~1 URE per 1015 bits read (≈ 125 TB)
- Best enterprise SAS: 1 per 1016 bits
A URE is a sector the drive cannot read back — even after its internal retry logic. The drive returns a read error rather than bad data.
RAID 5 rebuild math — worked example
Suppose an 8-disk RAID 5 of 18 TB consumer drives. One disk fails. To rebuild, every byte of the 7 surviving disks must be read: 7 × 18 TB = 126 TB = 1.008 × 1015 bits.
With URE rate 1 per 1014 bits, the expected number of UREs during rebuild is roughly 10. Even with enterprise drives at 1015, you're in the neighbourhood of 1 URE per rebuild.
A URE during a RAID 5 rebuild = second data-source failure = rebuild aborts = array lost.
Warning
RAID 5 on large modern disks is considered unsafe for exactly this reason. For any array above ~10 TB total, use RAID 6, RAID 10, or a ZFS-style raidz2. A single-parity scheme with 18 TB consumer drives is playing the lottery.
Bit Rot (Silent Data Corruption)
A disk occasionally returns wrong data without indicating any error. Causes: cosmic-ray bit flips in the drive's buffer, firmware bugs in the write path, mis-directed writes (data written to the wrong sector), cable/controller ECC misses, failing NAND cells returning plausible-but-wrong data.
RAID trusts the disk. If a read succeeds, the data is assumed correct. Parity is only consulted when a disk reports failure.
If one mirror member silently returns corrupt data, RAID 1 has no way to decide which copy is correct. Scrubbing can detect mismatches but not fix them.
Result: silent corruption propagates to the filesystem and to backups.
Every block carries an end-to-end checksum stored in the parent block's metadata. On read, the checksum is verified.
Mismatch → fetch from parity/mirror → verify → repair the bad copy in place ("self-healing"). Every read path validates.
Result: bit rot is detected and corrected transparently.
Warning
Hardware RAID does not protect against bit rot. Neither does mdadm by itself. If bit rot matters to you (it should for long-lived data), use a filesystem with per-block checksums — ZFS, btrfs, or bcachefs — either on top of mdadm or replacing it.
Hardware vs Software RAID
Hardware RAID
A dedicated PCIe controller card (LSI/Broadcom MegaRAID, Dell PERC, HPE Smart Array, Adaptec). The OS sees only logical drives exposed by the controller; the raw member disks are hidden (unless the controller is configured for HBA/passthrough mode).
- On-card cache (typically 512 MB – 8 GB DRAM) backed by battery (BBWC) or flash/supercap (FBWC). Absorbs RMW penalty and hides write latency.
- ASIC/IOC offloads XOR and Reed-Solomon calculations from the host CPU.
- Closed firmware with proprietary on-disk format. Configuration is done via
storcli,perccli,megacli, or the controller's UEFI/BIOS menu.
Software RAID (Linux mdadm)
The kernel MD (Multiple Device) subsystem implements RAID in software. The host CPU handles parity math; the host RAM handles caching (via the page cache).
- Portable — pull the disks out, plug them into any Linux machine,
mdadm --assemble --scan, array comes up. - Open format — superblock layout is documented; the same member disks can be read by any mdadm version that supports the superblock version.
- CPU cost is trivial on modern hardware — XOR runs at many GB/s via SSE2/AVX2 on a single core; Reed-Solomon uses dedicated instructions (PCLMULQDQ). Parity calculation is not a bottleneck.
- No on-card battery cache — but modern NVMe with power-loss protection plus a properly-sized UPS + write-intent bitmap gives equivalent safety.
- Controller = SPOF. Card dies → array is stuck until a replacement card with compatible firmware arrives. Cross-vendor rescue is generally impossible.
- Cost: $500–$2000+ per card; BBU replacements on a schedule.
- Opaque: SMART counters, individual disk errors, and failure reasons are hidden behind the controller's own view of reality.
- Firmware bugs have historically eaten arrays. Your fault-tolerance is only as good as a vendor's closed binary.
- Advantages remain: lower CPU overhead for embedded or low-spec hosts; on-card cache hides write latency on spinning rust; BMC integration in servers.
- Portable: any Linux host can import the array. No vendor lock-in.
- Free, fully open source, fully visible (
/proc/mdstat,mdadm --detail). - CPU cost is negligible on modern hardware.
- No dedicated write cache — needs UPS + bitmap (+ optional SSD journal) to match HW RAID's write-hole protection.
- Scales with the host: NVMe + mdadm on a modern CPU outperforms most HW RAID cards.
Trade-offs at a glance
| Dimension | Hardware RAID | Software RAID (mdadm) |
|---|---|---|
| Cost | $500–$2000+ per card + BBU | $0 |
| Portability | Tied to controller model/firmware | Any Linux host |
| CPU overhead | None (ASIC offload) | Negligible on modern CPUs |
| Write cache | BBWC/FBWC on card | Page cache + optional SSD journal |
| Visibility | Vendor CLI, opaque | /proc/mdstat, fully open |
| Recoverability | Needs same/compatible controller | Assemble anywhere |
| Bit-rot protection | None | None (use ZFS/btrfs on top) |
| Monitoring | storcli, SNMP via controller |
mdmonitor + SMART on each disk |
Tip
Industry drift: over the last decade, the default choice for new deployments has shifted from HW RAID to software — mdadm for general-purpose Linux, ZFS/raidz for anything requiring end-to-end integrity, and distributed storage (Ceph, Longhorn) above a single node. HW RAID persists in legacy environments and in vendor-sealed appliances.
