A Newly Installed Kernel Fails to Boot with Kernel Panic "Not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

Asset ID:	1-72-2096573.1
Update Date:	2017-10-18
Keywords:

Solution Type Problem Resolution Sure

Solution 2096573.1 : A Newly Installed Kernel Fails to Boot with Kernel Panic "Not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

Applies to:

Linux OS - Version Oracle Linux 6.0 with Unbreakable Enterprise Kernel [2.6.32] and later
Oracle VM - Version 3.3.3 to 3.3.3 [Release OVM33]
Private Cloud Appliance X5-2 Hardware - Version All Versions to All Versions [Release All Releases]
Linux x86-64
Linux x86

Symptoms

On Oracle Linux, after installing a new kernel by running a command such as :

# yum update kernel

The server no longer boots when using this new kernel, it stops booting at :

kernel panic - Not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
Pid: 1, comm: swapper Not tainted
Call trace :

? panic +0xa7/0x16f
? mount_block_root +0x216/ox2cb
? gunzip +0x0/0x34e
[...]

When booting from a former kernel version and looking at disk space usage using e.g. :

# df -h

the /boot mount point is 100% full.

When attempting to run :

# yum reinstall kernel

it fails with the following kind of error messages :

# yum reinstall kernel
Loaded plugins: security
Setting up Reinstall Process
ol6_u5_base | 3.1 kB 00:00
ol6_u5_patch | 2.5 kB 00:00
Skipping the running kernel: kernel-2.6.32-220.el6.x86_64
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-431.29.2.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Reinstalling:
kernel x86_64 2.6.32-431.29.2.el6 ol6_u5_patch 28 M

Transaction Summary
========================================================================================================================================================================
Reinstall 1 Package(s)

Total download size: 28 M
Installed size: 121 M
Downloading Packages:
kernel-2.6.32-431.29.2.el6.x86_64.rpm | 28 MB 00:01
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : kernel-2.6.32-431.29.2.el6.x86_64 1/1
Error unpacking rpm package kernel-2.6.32-431.29.2.el6.x86_64
error: unpacking of archive failed on file /boot/System.map-2.6.32-431.29.2.el6.x86_64;56962b6a: cpio: write
Verifying : kernel-2.6.32-431.29.2.el6.x86_64 1/1

Failed:
kernel.x86_64 0:2.6.32-431.29.2.el6

The relevant error being :

error: unpacking of archive failed on file /boot/System.map-2.6.32-431.29.2.el6.x86_64;56962b6a: cpio: write

Cause

The /boot partition is full and it was coming to full during the new kernel installation attempt, hence the init ramfs of new installed kernel is corrupted.

Solution

Run the following commands to determine which RHCK and UEK kernels are installed :

# rpm -qa kernel
# rpm -qa kernel-uek

and determine older kernel versions that can possibly be removed from the system before running again :

# yum reinstall kernel

or alternatively, for the UEK kernel :

# yum reinstall kernel-uek

Another option is to use an external boot partition but the most direct workaround for this issue is simply to remove older kernels and reinstall the most recent one.

Do not manually delete files from the /boot mount point, you can remove the unnecessary packages using yum command.

References

<NOTE:1927958.1> - Alert: Verify Sufficient Space in /boot on Your Hypervisor/dom0 Before Upgrading to Oracle VM 3.3.x

Attachments

This solution has no attachment