Problem solve Get help with specific problems with your technologies, process and projects.

Troubleshooting, finding lost files in Linux servers

Just because a server crashes doesn't mean your data is lost forever. Find out how to retrieve your data from damaged Linux server environments where ext3 is used.

Sometimes a server crashes or an error appears on a hard drive, and you can't access your file system. It's a hard fact of IT life, but it doesn't have to be terribly difficult to access data you thought was lost forever. If it is an ext2 or ext3 file system in a Linux environment, there is a way to access it anyway, even if at first sight you can't.

More on troubleshooting Linux:
How to get copy, conversion power with dd

How to set up a cluster file system

This tip will help you troubleshoot and then access damaged partitions in a Linux environment where the ext3 file system is used, as it is in about 80% of all Linux servers in use today. You'll learn how to benefit from advanced mount options that allow you to access data that you may have considered lost.

Accessing a file system with superblock

In order to access a file system, you need the superblock. This is a 1 kilobyte block that contains all metadata about the file system. It normally is the second 1 K block on an ext3 file system. In the listing below, you can see a part of the contents of the superblock as displayed with the debugfs utility:

Filesystem volume name:   
Last mounted on:          
Filesystem UUID:          09979101-96e0-4533-a7f3-0a2db9b07a03
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr filetype needs_recovery sparse_super large_file
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              5248992
Block count:              10486428
Reserved block count:     524321
Free blocks:              3888202
Free inodes:              4825213
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16352
Inode blocks per group:   511


The problem arises when, due to some error, the superblock isn't accessible anymore. Fortunately, some backup copies of the superblock are written on the ext3 file system by default. Using these backup copies, you can still mount a file system that you may have considered lost otherwise.

The actual position on disk of the first backup of the superblock depends on the size of the file system. On modern large file systems, you will always find it at block 32768. To access it, you can use the mount option -o sb.

The problem arises when the mount expects you to specify the position of the superblock in 1024 byte blocks. The default block size for a modern ext3 volume or partition is 4096 bytes. Therefore, to tell the mount command where it can find the superblock, you have to multiply the position of the superblock by 4; which would result in the block value 131072 in most cases. If, for example, your /dev/sda5 file system had a problem, you could try mounting it with the command

mount -o sb=131072 /dev/hda5 /somewhere. 

Now that you have mounted the problematic file system and thus limited the scope of the problem to the superblock, it is time to fix the problem. You can do this by copying the backup superblock back the the location of the old superblock. You can do that using

dd if=/dev/hda5 of=/dev/hda5 bs=1024 skip=131072 count=1 seek=1. 

Once you've finished, your file system is accessible again in the way it was before.

Problems mounting ext3 file systems often occur because of a problem in the administrative information at the beginning of the file system. If the problem comes from the superblock, you can mount the file system anyway, using a backup superblock and the mount option -o sb. Using dd, you can even restore the superblock to its original location.

Dig Deeper on Linux servers

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.