Exadata X4-2 Hardware
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>Oracle Exadata>DB: Exadata_EST
 

In this Document

Symptoms

Cause

Solution

Created from <SR 3-9462913831>

Applies to:

Symptoms

You are executing patchmgr from DB node as root user.

./patchmgr -cells /opt/oracle.SupportTools/cell_list -patch_check_prereq -rolling

It hangs on this stage ":Working: DO: Initialize files, check space and state of cell services. Up to 1 minute ..."
There is no update in any logfiles on DB nodes or Cell nodes. Looks like it just waiting for something, if not touched it can wait indefinitely.
But if you press Ctrl-C then it immediately gives the message that ":SUCCESS: DONE: Initialize files, check space and state of cell services." and continues until the end without problems.

Cause

 patchmgr executes several commands on cells, one of them gets stuck on cell02.

Next command:
/bin/bash ./dostep.sh prechk:no:1618 patch_prereq -no-auto-rollback is_rolling 900 3600 900 36000 default noforce

After tracing we see that it executes:
find / -name MegaSAS.log

Tracing of this fine command shows that it hangs when tries to read the content of:
/proc/5326/net

Process 5326 is
root      5325 19894  0 Jul13 ?        00:00:00 sh -c /bin/netstat -a -i -n 2>/dev/null >> /opt/oracle.ExaWatcher/archive/Netstat.ExaWatcher/2014_07_13_02_59_56_NetstatExaWatcher_exa02cell02-adm.ctlm.de.dat
root      5326  5325  0 Jul13 ?        00:00:00 /bin/netstat -a -i -n

It is actually executed from ExaWatcher, netstat command which had hung

After stopping exawatcher ExaWatcher using next command:
./ExaWatcher.sh --stop

All processes gone, except netstat

ps -aef | grep 5326
root      5326     1  0 Jul13 ?        00:00:00 /bin/netstat -a -i -n

It is still here, kill -9 can not clear it.

Solution

 Unfortunately the only way to clear this is to reboot the cell.

This issue is the subject of Enhancement Request Unpublished Bug 19618662 - EXADATA CELL PRE-UPGRADE MODE ISSUE 

This solution has no attachment