How do I troubleshoot slow backup speeds?

Slow backups usually mean something in the chain — network, storage, source, or software — isn’t keeping up. Here’s how to find the bottleneck.

Step 1: Assess the Backup Environment

Understand the workload. What type of data? Large files, small files, databases? Is the backup incremental, differential, or full?
Check backup frequency. Are jobs running during peak hours and contending with production traffic?
Identify sources and targets. Where is data coming from (servers, VMs)? Where is it going (local disk, NAS/SAN, cloud)?

Step 2: Identify Bottlenecks

Network Connectivity
- Bandwidth: Measure throughput between source and target with iperf or Wireshark.
- Latency: High latency kills remote backup performance, especially with chatty protocols.
- Congestion: Verify the network isn’t saturated during the backup window.
Storage Performance (Target)
- Disk throughput: Measure IOPS and sequential write speed where backups land. Slow disks are a common bottleneck.
- RAID configuration: RAID 5/6 rebuilds tank write performance. Confirm no rebuild is running.
- Filesystem fragmentation: Check fragmentation on spinning disks; it hurts sequential writes.
Backup Software Configuration
- Compression & encryption: Both add CPU overhead. If enabled, verify the backup server has headroom.
- Parallel streams: Ensure the software uses multiple streams/threads to saturate the pipe.
- Throttling: Check for intentional bandwidth or resource limits in the application.
Source Server Performance
- CPU & RAM: Source shouldn’t be starved during backup.
- Disk read speed: Test read performance of source volumes.
Backup Target Specifics
- Cloud storage: Watch for slow upload speeds, API rate limits, or egress costs.
- NAS/SAN: Confirm the array is tuned for sequential write workloads (e.g., write cache enabled, proper RAID stripe size).

Step 3: Analyze Backup Logs

Check software logs for errors, warnings, retries, or skipped files.
Look for patterns: same VMs/files slowing things down, locks causing retries.
Verify no files or VMs are locked during backup (VSS/VMware snapshot issues).

Step 4: Test and Isolate Components

Run a test backup on a small, representative dataset. Measure speed without interference.
Compare speeds across different targets (local vs. NAS vs. cloud).
Isolate the network: transfer large files from source to target with scp, rsync, or a raw dd over SSH. This removes backup software from the equation.
Isolate storage: write/read large files directly to the target filesystem (e.g., fio or dd).

Step 5: Optimize Configuration

Hardware upgrades
- Upgrade NICs/switches to 10/25/40 GbE if you’re on 1 GbE.
- Add SSD/NVMe cache or tier for backup target writes.
Backup software tuning
- Enable deduplication if CPU headroom exists; it reduces data moved but costs compute.
- Increase thread/stream count to match available cores and network paths.
- Shift from full to incremental or synthetic full backups where possible.
Network optimization
- Implement QoS to prioritize backup traffic during its window.
- Enable jumbo frames (9000 MTU) end-to-end — NICs, switches, routers, and targets must all support it.
Storage optimization
- Align partitions/volumes to underlying stripe/block boundaries.
- Use a write-back cache tier (SSD/NVMe) on spinning-disk targets.

Step 6: Monitor and Adjust

Track backup duration, throughput, and error rates over time with Nagios, Zabbix, SolarWinds, or vendor tools.
Correlate performance dips with changes (new VMs, firmware updates, network changes).
Schedule backups off-peak to minimize contention — but verify the window is long enough.

Step 7: Engage Vendors

If you’ve isolated the bottleneck and can’t resolve it:

Open a case with your backup software vendor. Provide logs, iperf/fio results, and topology.
Contact storage/network vendors for firmware updates, known issues, or tuning guides specific to your hardware.