Your hard disk fails on Linux - now what?

I happened to me a few times in the last few weeks. A filesystem check for a hard disk would take for ever - my system won't boot. One of the harddisks was reporting bad block errors. At this time i have four different hard disks that are giving me this kind of trouble. All broken harddisks have been taken out of the system and I am analyzing the problem using a virtual machine and a USB-SATA bridge.

First step: remove harddisk from system and put into SATA USB docking station

Hopefully your disk is not needed to boot or fully operate your system. In that case you might want to boot of a USB stick or other media. In any case my procedure is to remove the disk from the original system and use a different system for analysis. In my case i am using an Ubuntu based virtual machine and connect the drive via a USB-SATA bridge. With USB 2 devices the performance is poor. I am not using my old Logilink QP002 Sata Docking Station anymore for this reason.

The USB 3 device i bought in 2015: 611EMR%2BPqWL._AC_SL1001_.jpg still serves me well.

Second step - check problems

Tools needed

A Linux virtual machine
smartctl
hdparam
debugfs
mount

Examples

disk Digda

Where is it mounted?

sudo mount | grep Digda
/dev/sdb1 on /media/wf/Digda type ext2 (rw,nosuid,nodev,relatime,uhelper=udisks2)

get basic info

using fixbad -i
using smartctl -i
using hdparm -I

fixbad

fixbad /dev/sdb -i

smartctl

sudo smartctl -i /dev/sdb
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.0-72-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     SAMSUNG SpinPoint F4 EG (AF)
Device Model:     SAMSUNG HD204UI
Serial Number:    S2H7J9EZC04171
LU WWN Device Id: 5 0024e9 203f40fb7
Firmware Version: 1AQ10001
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    5400 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 2.6, 3.0 Gb/s
Local Time is:    Mon Oct  5 11:41:44 2020 CEST

==> WARNING: Using smartmontools or hdparm with this
drive may result in data loss due to a firmware bug.
****** THIS DRIVE MAY OR MAY NOT BE AFFECTED! ******
Buggy and fixed firmware report same version number!
See the following web pages for details:
http://knowledge.seagate.com/articles/en_US/FAQ/223571en
http://www.smartmontools.org/wiki/SamsungF4EGBadBlocks

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

hdparm

sudo hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
	Model Number:       SAMSUNG HD204UI                         
	Serial Number:      S2H7J9EZC04171      
	Firmware Revision:  1AQ10001
	Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6
Standards:
	Used: unknown (minor revision code 0x0028) 
	Supported: 8 7 6 5 
	Likely used: 8
Configuration:
	Logical		max	current
	cylinders	16383	16383
	heads		16	16
	sectors/track	63	63
	--
	CHS current addressable sectors:    16514064
	LBA    user addressable sectors:   268435455
	LBA48  user addressable sectors:  3907029168
	Logical  Sector size:                   512 bytes
	Physical Sector size:                   512 bytes
	device size with M = 1024*1024:     1907729 MBytes
	device size with M = 1000*1000:     2000398 MBytes (2000 GB)
	cache/buffer size  = unknown
	Form Factor: 3.5 inch
	Nominal Media Rotation Rate: 5400
Capabilities:
	LBA, IORDY(can be disabled)
	Queue depth: 32
	Standby timer values: spec'd by Standard, no device specific minimum
	R/W multiple sector transfer: Max = 16	Current = ?
	Advanced power management level: disabled
	Recommended acoustic management value: 254, current value: 0
	DMA: mdma0 mdma1 *mdma2 udma0 udma1 udma2 udma3 udma4 udma5 udma6 
	     Cycle time: min=120ns recommended=120ns
	PIO: pio0 pio1 pio2 pio3 pio4 
	     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
	Enabled	Supported:
	   *	SMART feature set
	    	Security Mode feature set
	   *	Power Management feature set
	   *	Write cache
	   *	Look-ahead
	   *	Host Protected Area feature set
	   *	WRITE_BUFFER command
	   *	READ_BUFFER command
	   *	NOP cmd
	   *	DOWNLOAD_MICROCODE
	    	Advanced Power Management feature set
	    	Power-Up In Standby feature set
	   *	SET_FEATURES required to spinup after power up
	    	SET_MAX security extension
	    	Automatic Acoustic Management feature set
	   *	48-bit Address feature set
	   *	Device Configuration Overlay feature set
	   *	Mandatory FLUSH_CACHE
	   *	FLUSH_CACHE_EXT
	   *	SMART error logging
	   *	SMART self-test
	   *	General Purpose Logging feature set
	   *	64-bit World wide name
	   *	WRITE_UNCORRECTABLE_EXT command
	   *	{READ,WRITE}_DMA_EXT_GPL commands
	   *	Segmented DOWNLOAD_MICROCODE
	   *	Gen1 signaling speed (1.5Gb/s)
	   *	Gen2 signaling speed (3.0Gb/s)
	   *	Native Command Queueing (NCQ)
	   *	Host-initiated interface power management
	   *	Phy event counters
	   *	NCQ priority information
	    	DMA Setup Auto-Activate optimization
	    	Device-initiated interface power management
	   *	Software settings preservation
	   *	SMART Command Transport (SCT) feature set
	   *	SCT Read/Write Long (AC1), obsolete
	   *	SCT Write Same (AC2)
	   *	SCT Error Recovery Control (AC3)
	   *	SCT Features Control (AC4)
	   *	SCT Data Tables (AC5)
Security: 
	Master password revision code = 65534
		supported
	not	enabled
	not	locked
	not	frozen
	not	expired: security count
		supported: enhanced erase
	344min for SECURITY ERASE UNIT. 344min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50024e9203f40fb7
	NAA		: 5
	IEEE OUI	: 0024e9
	Unique ID	: 203f40fb7
Checksum: correct

Links

fixbad script

Bad Block Howto

Contents