How to Recover Corrupted Partition From A Bad Superblock

-->
Q. How can I Recover a bad superblock from a corrupted ext2/3/4 partition to get back my data? I'm getting following error:

/dev/sda2: Input/output error
mount: /dev/sda2: can't read superblock

How do I fix this error?

A. Linux ext2/3/4 filesystem stores superblock at different backup location so it is possible to get back data from corrupted partition.

WARNING! Make sure file system is UNMOUNTED.

If your system will give you a terminal type the following command, else boot Linux system from rescue disk (boot from 1st CD/DVD. At boot: prompt type command linux rescue).

Mount partition using alternate superblock

Find out superblock location for /dev/sda2:

# dumpe2fs /dev/sda2 | grep superblock

Sample output:

  Primary superblock at 0, Group descriptors at 1-6
  Backup superblock at 32768, Group descriptors at 32769-32774
  Backup superblock at 98304, Group descriptors at 98305-98310
  Backup superblock at 163840, Group descriptors at 163841-163846
  Backup superblock at 229376, Group descriptors at 229377-229382
  Backup superblock at 294912, Group descriptors at 294913-294918
  Backup superblock at 819200, Group descriptors at 819201-819206
  Backup superblock at 884736, Group descriptors at 884737-884742
  Backup superblock at 1605632, Group descriptors at 1605633-1605638
  Backup superblock at 2654208, Group descriptors at 2654209-2654214
  Backup superblock at 4096000, Group descriptors at 4096001-4096006
  Backup superblock at 7962624, Group descriptors at 7962625-7962630
  Backup superblock at 11239424, Group descriptors at 11239425-11239430
  Backup superblock at 20480000, Group descriptors at 20480001-20480006
  Backup superblock at 23887872, Group descriptors at 23887873-23887878

Now check and repair a Linux file system using alternate superblock # 32768:

# fsck -b 32768 /dev/sda2
Sample output:

fsck 1.40.2 (12-Jul-2007)
e2fsck 1.40.2 (12-Jul-2007)
/dev/sda2 was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #241 (32254, counted=32253).
Fix? yes
Free blocks count wrong for group #362 (32254, counted=32248).
Fix? yes
Free blocks count wrong for group #368 (32254, counted=27774).
Fix? yes
..........
/dev/sda2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sda2: 59586/30539776 files (0.6% non-contiguous), 3604682/61059048 blocks

Now try to mount file system using mount command:

# mount /dev/sda2 /mnt

You can also use superblock stored at 32768 to mount partition, enter:

# mount sb={alternative-superblock} /dev/device /mnt
# mount sb=32768 /dev/sda2 /mnt

Try to browse and access file system:

# cd /mnt
# mkdir test
# ls -l
# cp file /path/to/safe/location
You should always keep backup of all important data including configuration files.

How to manage log files using logrotate

-->

Managing log files effectively is an essential task for Linux sysadmin.

In this article, let us discuss how to perform following log file operations using UNIX logrotate utility.

• Rotate the log file when file size reaches a specific size
• Continue to write the log information to the newly created file after rotating the old log file
• Compress the rotated log files
• Specify compression option for the rotated log files
• Rotate the old log files with the date in the filename
• Execute custom shell scripts immediately after log rotation
• Remove older rotated log files

1. Logrotate Configuration files

Following are the key files that you should be aware of for logrotate to work properly.

/usr/sbin/logrotate – The logrotate command itself.

/etc/cron.daily/logrotate – This shell script executes the logrotate command everyday.

$ cat /etc/cron.daily/logrotate
#!/bin/sh

/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0

/etc/logrotate.conf – Log rotation configuration for all the log files are specified in this file.

$ cat /etc/logrotate.conf
weekly
rotate 4
create
include /etc/logrotate.d
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}

/etc/logrotate.d – When individual packages are installed on the system, they drop the log rotation configuration information in this directory. For example, yum log rotate configuration information is shown below.

$ cat /etc/logrotate.d/yum
/var/log/yum.log {
    missingok
    notifempty
    size 30k
    yearly
    create 0600 root root
}

2. Logrotate size option: Rotate the log file when file size reaches a specific limit

If you want to rotate a log file (for example, /tmp/output.log) for every 1KB, create the logrotate.conf as shown below.

$ cat logrotate.conf
/tmp/output.log {
        size 1k
        create 700 ranjith ranjith
        rotate 4
}

This logrotate configuration has following three options:

• size 1k – logrotate runs only if the filesize is equal to (or greater than) this size.
• create – rotate the original file and create the new file with specified permission, user and group.
• rotate – limits the number of log file rotation. So, this would keep only the recent 4 rotated log files.

Before the logrotation, following is the size of the output.log:

$ ls -l /tmp/output.log
-rwx------- 1 ranjith ranjith 25868 2010-06-09 21:19 /tmp/output.log

Now, run the logrotate command as shown below. Option -s specifies the filename to write the logrotate status.

$ logrotate -s /var/log/logstatus logrotate.conf

Note : whenever you need of log rotation for some files, prepare the logrotate configuration and run the logroate command manually.
After the logrotation, following is the size of the output.log:

$ ls -l /tmp/output*
-rwx------- 1 ranjith ranjith 25868 2010-06-09 21:19 output.log.1
-rwx------ 1 ranjith ranjith        0 2010-06-09 21:20 output.log

Eventually this will keep following setup of rotated log files.

• output.log.4.
• output.log.3
• output.log.2
• output.log.1
• output.log

Please remember that after the log rotation, the log file corresponds to the service would still point to rotated file (output.log.1) and keeps on writing in it. You can use the above method, if you want to rotate the apache access_log or error_log every 5 MB.

Ideally, you should modify the /etc/logrotate.conf to specify the logrotate information for a specific log file.

3. Logrotate copytruncate option: Continue to write the log information in the newly created file after rotating the old log file.

$ cat logrotate.conf
/tmp/output.log {
         size 1k
         copytruncate
         rotate 4
}

copytruncate instruct logrotate to creates the copy of the original file (i.e rotate the original log file) and truncates the original file to zero byte size. This helps the respective service that belongs to that log file can write to the proper file.