Category: Hard disks

Why update the firmware !

My answer here is a bit unconventional, and certainly not a fact.. I even think I am wrong, but it can’t hurt, so here it goes

Seagate recommends you update the disk’s firmware to improve performance and longevity of the hard drive, I on the other hand have an extra mission…

The firmware on a hard drive is stored partially on a chip on the PCB, and partially on the disk itself ! I know that disk platters have a data retention life of 10 years, the area where the firmware is written is never refreshed since it is read only when the disk boots up, So i am hoping (even though doubtful) that the firmware update might re-write this area of the disk and breath new life into it.

A disk certain application claims to refresh the data on that area of the disk, After testing that application I will come here and add my findings accordingly

Getting the firmware

Let us start by downloading the firmware ! To download firmware from seagate’s website, you will need to know your hard drive serial number, to do just that, open the command prompt elevated, and run the following command

wmic diskdrive get model,serialnumber

The result of me running that command is as follows

The results of executing the command above show that I have a hard drive of the model (ST2000DM001-1CH164) as you can see with a serial number that I now have (Masking serial numbers just in case seagate has a problem with me publishing them as it allows you to download the firmware using the serial number)…..

Now that I have the serial number, I can go to seagate’s firmware download page here and grab the firmware… once done, I unzip it, and the following folders appear

Creating a bootable USB flash disk

Now, inside the Bootable tools, there is a file (SeaChest_RC_2.7.4_10-18-2018.usbBootMaker.exe) that will create a bootable flash stick for you, insert a flash stick of any size that is to be deleted by this application, run the application, and now you have a bootable drive, but without the firmware, so copy the firmware folder you see above into the flash stick, and now you are ready to boot from it, for instructions how to boot from a flash stick, you will need to check with your motherboard’s manufacturer documentation, it is usually a simple thing such as hitting F11 at boot time.

Updating the firmware

Once booted, you should be presented with a linux command prompt, where we can run commands to update the firmware

To see what disks are on your system, run the following command

SeaChest_Firmware --scan

The scan should give you the handle for the drive, if you have never used linux before, the handle started with /dev/ (Short for device), and sata disks usually start with sdX (Where X starts with A and ends with a letter corresponding to the last disk you have in your system) old PATA disks usually start with hdX… but that is usually not something you need as PATA disks are virtually non existent at this stage

Now, execute the firmware update command like so

SeaChest_Firmware -d /dev/sg3 --downloadFW /firmware/filename.LOD

Now, if you want to know whether the update was successful or not, just run the scan command again, and note the firmware on it !

My Problem !

As you can see from the image below, I have 3 firmware files, named 1TB, 2TB, and 3TB, when i ran the command above, The system claimed that the update was successful, but didn’t really update the firmware, I was still stuck with 26 rather than 29 !

So i decided to use Seagates own configuration file to do the update with the command

SeaChest_Firmware -d /dev/sg3 --fwdlConfig GPCC2949.CFS

Surprise was that I got the following error

model matched but the current firmware version does not match the available updates

So, I went back in time and remembered that for this particular disk, I had changed the PCB before (Trying to get a 3TB disk to work by moving a certain chip from one board to the other, diagnosis turned out the problem is not the PCB)… So instead of flashing the 2TB firmware file, I flashed the 3TB, and what do you know, It worked.

Anyway, I will come back with screenshots of the whole thing… and more data for those who are having trouble updating their firmware, until then, hang in there

Whenever i get the following message

mount /dev/sdd1 /hds/sgt2tb
The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an
unsafe state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.)
Could not mount read-write, trying read-only

The command

ntfsfix /dev/sdd1

resolves the issue, and produces the following message

Mounting volume... The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sdd1 was processed successfully

The same mount command you see here will now work flawlessly

mount /dev/sdd1 /hds/sgt2tb

I am still unsure what process from the mentioned above is responsible, as this oftentimes pops up on drives that were never system drives, so there is no hibernation file problem

The answer to this should be simple, I initiated the test with

badblocks -nsv /dev/sdb

, first, interrupt bad blocks with ctrl+c, the output should be

Checking for bad blocks in non-destructive read-write mode
From block 0 to 1953514583
Checking for bad blocks (non-destructive read-write test)
Testing with random pattern:   0.92% done, 49:38 elapsed. (0/0/0 errors)
 21.32% done, 18:49:24 elapsed. (0/0/0 errors)

Interrupted at block 416437376

Interrupt caught, cleaning up

Okay, so we know what blocks it was supposed to check (1 through 1953514583), and where it was interrupted (416437376)

So i will ask it to resume testing from where it finished (-1), up to the end

badblocks -nsv /dev/sdb 1953514583 416437375

n = Non destructive
s = Show progress
v = tell us about what you find !

The new run should tell you the percentage correctly, but the time counter will be reset to zero, as it is only counting how long this run has been running for

One thing to note is that bad blocks can be used to instruct the filesystem to avoid the bad blocks, but it also allows the disk’s firmware to substitute bad blocks with spare blocks, so that the disk works again with no intervention from your end !

So for my 2TB hard drive…

416437375 = 21% (13 hours)
619014719 = 31.6% (+23:22)
627995199 = 32.15% (+1:04)
667782398 = 34.18% (+4:46)
715469885 = 36.62% (+5:44)
827834875 = 42.38%

While running the tests, you might want to keep an eye on the hard drive temperature with a command like

hddtemp /dev/sdb

To create a log file of the bad blocks, every run should have it’s own file !

badblocks -nsv -o /root/badblocks3.txt /dev/sdb 1953514583 627995198

The concatenation of those files you are creating is very useful in creating a file system if you ever decide to format the drive later !, but the recommended way is using badblocks with the other disk tools directly

while the test is running, you will see 3 numbers that correspond to readerror/writeerror/corruptionerror

1- Before using dd, ddrescue, or dd_rescue, you need to know which disk is which, you can do that by simply using the command “fdisk -l” in my case, the old disk turned out to be /dev/sdb and the new un-partitioned disk is /dev/sdc.

So, i have been cloning a 2TB hard drive ( WD20EARS ) to a WD20EARX, same disk, but with a few differences

WD20EARS is sata 2 and the other is sata 3, another difference is that using “hdparm -I /dev/sdb” the older WD20EARS reports (And should not be true)

WD20EARS

Logical/Physical Sector size:           512 bytes

wile with “hdparm -I /dev/sdc” the newer WD20EARX reports

        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes

The first clone did not work for a reason unknown to me, i cloned my NTFS disk with ddrescue (gddrescue) on a linux (because i don’t know how to clone on windows) and then plugged it into windows, where it simply did not work, and in disk management reported the disk as un-partitioned space, so now i want to do the thing again, but i don’t want that slow performance, so i increased block size to 4KiB. (UPDATE: THE NEW COPY WITH 4KiB DID WORK BUT I DONT KNOW IF THE 4KiB SIZE IS RELEVANT, MAYBE YOU SHOULD TAKE A LOOK AT THE SECOND DIFFERENCE BETWEEN THE DISKS UP AT THE BEGINNING OF THE POST)

For now, i will try the cloning with the command (Only change the block level for advanced format hard drives)

ddrescue --block-size=4KiB /dev/sdb /dev/sdc rescue2.log

And if all of your data is important, you can ask ddrescue to retry every bad block 3 times (or as many times as you wish) with the -r command

ddrescue --block-size=4KiB -r3 /dev/sdb /dev/sdc rescue2.log

And what do you know, the disk now works on my WINDOWS machine 😀 no errors and no nothing, great, so now to some details about the copy

The result up to now is that i am reading at a maximum of 129MB while the average (in the first 60 GBs is 93018 kB/s), if this continues, i will be done in less than 6 hours.

The part that does not make any sense to me is that western digital states clearly in the specs that the maximum (Host to/from drive (sustained)) is 110 MB/s for both drives, it must be that i need to wait a bit more and see what that actually means.

rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    74787 MB,  errsize:       0 B,  current rate:     119 MB/s
   ipos:    74787 MB,   errors:       0,    average rate:   93018 kB/s
   opos:    74787 MB,     time from last successful read:       0 s
Copying non-tried blocks...

Now, once done, you can have the OS reload the partition table without having to restart, you can simply use the command partprobe

partprobe
or
partprobe /dev/sdc

To use partprobe, you need to install parted

apt-get install parted

If it were a linux drive, an advanced format drive would not have it’s first sector on sector 63 but rather on sector 2048, which is at exactly 2KiB, it could (but usually does not) start at any other value divisible by 8.

Windows probably does something similar for our AF Disk, so asking parted about our ntfs disk, this is what parted says

Model: ATA WDC WD20EARS-00M (scsi)
Disk /dev/sdb: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  2000GB  2000GB  primary  ntfs

1049kB is 1074176 bytes, Which when divided by 8 is 134272 (divisible by 8).

NOTES:
-There is a tool specifically for cloning ntfs volumes called ntfsclone, i am not sure what extra features it provides that are specific to ntfs, i have never used it before, with my disk that has bad blocks, i can only rely on gddrescue.
-A block is 512 on regular drives, and 4096 on newer ones, if you want to backup the hard drive’s geometry, you can do one of the following
Backup the first 63 blocks (MBR + Bootloader). on a “non advanced format” drive

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=512 count=63

On an advanced format drive, we can try

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=4096 count=63

Which, will make us read 258048 bytes rather than the traditional 32256 bytes (around 250K rather than 32K)

It is simple, here is what you need to know

You can format it EXT4, but ext2 and ext3 are also OK ! ext2 and ext3 allow up to 16TB disks, and file sizes of up to 2TB, ext4 allows much more.

Any linux kernel newer than 2.6.31 should work just fine with “Advanced format” drives using the exact same steps in this article.

MBR only supports 2TB drives, you need GPT, so let us get started

1- apt-get update
2- apt get install parted
3- parted /dev/sdc
4- mklabel gpt
5- Answer yes to: Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
6- mkpart primary ext4 0% 100% (to make a partition as big as the disk (will occupy starting from first megabyte (for alignment) to the end of disk))
7- quit

FYI, if you want multiple partitions, here are the 2 lines that should replace step 6
6- mkpart primary ext4 0% 40%
6- mkpart primary ext4 40% 100%

and remember to format both (sdc1 and sdc2) when you are done with parted

Now to formatting the drive

mkfs.ext4 /dev/sdc1

Before mounting it, i like ext4, but i don’t want a journaling OS on this drive that is not the system drive, so i will need do a few things to the drive first

Lazy writeback
tune2fs -o journal_data_writeback /dev/sdc1

No Journaling
tune2fs -O ^has_journal /dev/sdc1

Now to check what we have

dumpe2fs /dev/sdc1 |grep ‘Filesystem features’
Or maybe if you want the whole thing on the screen
dumpe2fs /dev/sdc1 |more

if has_journal option exist when executing the first – you have journal on the file system

And there we are, Now we need to mount it at boot time by adding it to fstab, to do that, we will need the disk’s unique ID !

8- Now executing the following command will give you the unique ID of this new partition for use with fstab (The disk list we will edit below in step 10)
blkid /dev/sdc1
9- create the directory where you want to mount your hard disk, for example
mkdir /hds
mkdir /hds/3tb
10- Now, we add the following line to fstab, notice that noatime increases performance, but some applications might need or rely on it. postfix does not and i have verified that.

UUID=b7a491b1-a690-468f-882f-fbb4ac0a3b53       /hds/3tb            ext4     defaults,noatime                0       1

11- Now execute
mount -a

You are done,. if you execute
df -h
You should see your 2+TB hard drive in there !

To make sure the drive is aligned correctly, i like to write a file on it and see how fast that goes… so let us use a 2GB file

dd if=/dev/zero of=/hds/WD2000_3/deleteme.img bs=1M count=2000

Outcome came out (for a western digital black 2TB)
First run: 2097152000 bytes (2.1 GB) copied, 5.94739 s, 353 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 11.1405 s, 188 MB/s
Outcome came out for a western digital green 3TB
First run: 2097152000 bytes (2.1 GB) copied, 8.32337 s, 252 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 14.376 s, 146 MB/s

the consecutive runs give close results, what i printed here is the average

Section 1: My hard drive has bad sectors / Blocks / area

Do i need to change it ?
Not necessarily, but If it is in warranty, and they allow you to replace it, a new one is not a bad idea, otherwise read on

it all depends on whether the bad sectors are expanding or not, if they are not, they are probably caused by shock to the hard drive, usually, it is enough to mark them as bad using “chkdsk /r” on windows and leave the drive working.

To find out if your bad sectors are Spreading or not spreading, do a “chkdsk /r” four times, make sure the same number appears in the second and third and fourth time (Forget the first time), then, if the second is different but the third and fourth are the same, then do the test 2 more times, and make sure you get the same number of bad sectors for trials 3, 4, 5, 6, if so, your bad sectors are not spreading.

You did not mention backup in the answer before, do we need to backup ?
People would typically ask you to backup just in case, i say you should always have backup of your most important files, non spreading sectors of the hard drive, in my humble experience do not contribute negatively to reliability, so my answer is, backup should be done regardless

How do i know how many bad sectors are marked on an NTFS hard drive ?
There is a tool called nfi.exe that comes with a bundle Microsoft makes available here http://support.microsoft.com/kb/253066/en-us this tool is part of (OEM Support Tools), it can tell you everything about a disk formatted in NTFS

I accedientally pulled the power plug of a PC from the socket, and that PC was just starting to boot. The seagate hard drive inside stopped working, and the bad sectors turned out to affect the partition table, in any case, i slaved it on a windows vista PC, then into the Computer management, disk management panel, and what do you know, as if it has no paritions…

The solution to detect the boundaries of the 4 partitions it had is software called XXXX

Ran the software (The analyze option) , and what do you know, my partions exactly, 100MB made by Windows 7, a 479 GB partition for Windows, a 1GB swap partition for linux and an EXT3 partition for Linux…

So happily i asked the software to write the partitioning info to the disk, but the disk won’t hold the data, the bad sectors are where Windows writes the partition information

So, i ran down to the computer shop (In our building), and got the same exact drive (Seagate 500GB Model number xxx)

Mounted both on a Linux machine as slaves, both the damaged and the target.

To find out which one is SDC and which one is SDB, i watched as the linux machine booted, and as it booted, it threw in errors saying SDB all the time, so i know that SDB is the busted drive !

Installed gddrescue (apt-get install gddrescue), and ran it with the following command

ddrescue /dev/sdb /dev/sdc resumelog.log

(The additional log file helps us resume in case of interruption)

Once that is done, i put the new hard drive in a Windows machine, still can not see any partition info

1- Ran xxxx, it can see the 4 partitions, write changes… and what do you know, the partitions stick, we are good to go, i restart, but still, Windows can now see the partitions, but thinks drive G is not formated !

So i opened the command prompt (Elevated), then ran the command

chkdsk g: /f

the /f stands for fix, the thing took some time, but after the restart drive G works fine, all files are in there, and no one wants to kill me no more 🙂