FS System: How to Use nvdtool to Dump the NVDIMM Registers on a Controller

Asset ID:	1-71-1962910.1
Update Date:	2018-04-30
Keywords:

Solution Type Technical Instruction Sure

Solution 1962910.1 : FS System: How to Use nvdtool to Dump the NVDIMM Registers on a Controller

Applies to:

Oracle FS1-2 Flash Storage System - Version All Versions and later
Information in this document applies to any platform.

Goal

This document provides the instructions on how to use the utility nvdtool to extract the registers for the NVDIMMs on the Flash Storage Controllers.

Note: There is the existing unpublished Bug 20312168 (type Enhancement request) to request an option for the CLI "fscli" in order to collect these registers. Starting from the future release 060300-025900, it will be possible to use the command "fscli controller -command -commandstring NVDTOOL -controller /CONTROLLER-01" (this example is for Controller 1) to dump the NVDIMM registers.

Solution

The nvdtool is a command found in /usr/bin of an FS1-2 Controller. In addition to the bare command itself, there are four valid options in which the command can be executed:

-d option, dumps a "short set" of registers (0f, 30-34, 37-3a & 3f)
-s option, dumps the temperature sensor register (34)
-t option, dumps the NVDIMM Table
-x option,dumps the PSM registers (32)

Note: Since all of the options also dump the same set of registers (00-0f, 30-3f & 40-4c) the -d, -s & -x options do not appear to offer anything that cannot be obtained using either the no option or -t option. As such, the most effective use of this command is when it is executed with -t.

Connect to the active Pilot:

# ssh root@<FS1 IP address>

Note: Refer to KM Documents 2029847.1 FS System: How to Enable ssh Access to the Pilot and 2046703.1 FS System: Passwords Associated with the Oracle FS1-2 Flash Storage System.
Determine the IP addresses for Controllers 1 & 2 by viewing the contents of /etc/nodenames. Controller 1's WWN will always end in 0, Controller 2 will always end in 1:

[root@pilot1 ~]# cat /etc/nodenames
172.30.80.2 WN2008fffffffffff2 WN2008000101000000 mgmtnode
172.30.80.128 WN508002000158ba50 WN2008000101000001
172.30.80.129 WN508002000158ba51
172.30.80.3 WN2009fffffffffffa
[root@pilot1 ~]#
Connect to the Controller via ssh (this example connects to Controller 2):

[root@pilot1 ~]# ssh 172.30.80.129
WN508002000158BA51 #
Extract the NVDIMM registers:

WN508002000158BA51 # nvdtool -t

Mon Aug 21 15:12:51 2017

(nvdimm_get_vregs)

vendor reg: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
-----------------------------------------------------------
nvdimm 12: 68 81 00 05 00 00 78 00 00 00 00 00 03 9f 00 d9
nvdimm 13: 68 81 00 05 00 00 78 00 00 00 00 00 03 9f 00 d9
nvdimm 14: 68 81 00 05 00 00 78 00 00 00 00 00 03 9f 00 d9
nvdimm 15: 68 81 00 05 00 00 78 00 00 00 00 00 03 9f 00 d9

vendor reg: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
-----------------------------------------------------------
nvdimm 12: 94 00 58 60 1c 00 52 da 00 28 0a 96 58 00 00 03
nvdimm 13: 94 00 58 60 1b 00 54 e5 00 91 0a 96 58 00 00 06
nvdimm 14: 8f 00 58 60 19 00 54 d8 00 a5 09 96 58 00 00 05
nvdimm 15: 8f 00 38 60 1b 00 54 d8 00 6a 08 96 58 00 00 07

vendor reg: 40 41 42 43 44 45 46 47 48 49 4a 4b 4c
--------------------------------------------------
nvdimm 12: 10 29 80 00 00 00 00 00 ff 00 00 00 00
nvdimm 13: 10 29 80 00 00 00 00 00 ff 00 00 00 00
nvdimm 14: 10 29 80 00 00 00 00 00 ff 00 00 00 00
nvdimm 15: 10 29 80 00 00 00 00 00 ff 00 00 00 00

NVDIMM Table Hex dump:
---------------------------------------------------------------------
[0x7f65c59f0000] 0x0000: 0xba06cc78 0x00000418 0x00000003 0x00530000
[0x7f65c59f0010] 0x0010: 0x80000000 0x00000030 0x00000000 0x00000004
[0x7f65c59f0020] 0x0020: 0x0c020202 0x01000404 0x00000101 0x00000000
[0x7f65c59f0030] 0x0030: 0x00000000 0x00000000 0x00000000 0x00000000
[0x7f65c59f01f0] 0x01f0: 0x00100000 0x00000000 0x00000010 0x00100000
[0x7f65c59f0200] 0x0200: 0x00000000 0x00000010 0x00100000 0x00000000
[0x7f65c59f0210] 0x0210: 0x00000010 0x00100000 0x00000000 0x00000010
[0x7f65c59f0220] 0x0220: 0x00100000 0x00000000 0x00000010 0x00100000
[0x7f65c59f0230] 0x0230: 0x00000000 0x00000010 0x0104d902 0xd9029403
[0x7f65c59f0240] 0x0240: 0x94060104 0x0104d902 0xd9028f05 0x8f070104
[0x7f65c59f0250] 0x0250: 0x00000000 0x00000000 0x00000000 0x00000000
[0x7f65c59f0370] 0x0370: 0x00000fff 0x00000000 0x0000f000 0x00000000
[0x7f65c59f0380] 0x0380: 0x00000000 0x00000000 0x0000f000 0x00000000
[0x7f65c59f0390] 0x0390: 0x0000f000 0x00000000 0x0000f000 0x00000000
[0x7f65c59f03a0] 0x03a0: 0x00000000 0x00000000 0x0000f000 0x00000000
[0x7f65c59f03b0] 0x03b0: 0x00000000 0x00000000 0x00000000 0x00000000

NVDIMM Table:
----------------------------------------------------------------------
cksum            = 0xba06cc78
size             = 0x00000418
version          = 0x00000003
nvdsc_vers       = 0x00530000
baseaddr         = 0x0000003080000000
allsize          = 0x400000000
ncpu             = 0x00000002
ncpctrl          = 0x00000002
dpchan           = 0x00000002
reg_dimms        = 0x0000000c
disc_nvdimms     = 0x00000004
sup_nvdimms      = 0x00000004
unsup_nvdimms    = 0x00000000
sup_enabled      = 0x00000001
post_enabled     = 0x00000001
boot_flag        = 0x00000001
disabled_nvdimms = 0x00000000
disabled_dimms   = 0x00000000
slidx            = 0x00000000

bitmaps.disc_dimm    = 0xfff
bitmaps.disc_nvdimm = 0xf000
bitmaps.unsup        = 0x0
bitmaps.sup          = 0xf000
bitmaps.contig_mask = 0xf000
bitmaps.init         = 0xf000
bitmaps.valid        = 0x0
bitmaps.rest_img     = 0xf000
bitmaps.train_err    = 0x0
bitmaps.test_err     = 0x0
bitmaps.disabled_nvdimms = 0x0
bitmaps.release      = 0x0
bitmaps.as_pend      = 0x0
bitmaps.restore_pend = 0x0
bitmaps.charge_pend = 0x0
bitmaps.not_ready    = 0x0
bitmaps.vers_failed = 0x0
bitmaps.esm_eol      = 0x0
bitmaps.esm_htf      = 0x0
bitmaps.disabled_dimms = 0x0
bitmaps.pad[0]       = 0x0

slog[0].code=0, info0=0, info1=0, info2=0, info3=0, info4=0, info5=0

entry[00].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[01].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[02].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[03].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[04].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[05].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[06].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[07].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[08].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[09].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[10].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[11].type=00 ver=00 size=16 present=0 bay=0 esm_ver=0
entry[12].type=02 ver=d9 size=04 present=1 bay=3 esm_ver=94
entry[13].type=02 ver=d9 size=04 present=1 bay=6 esm_ver=94
entry[14].type=02 ver=d9 size=04 present=1 bay=5 esm_ver=8f
entry[15].type=02 ver=d9 size=04 present=1 bay=7 esm_ver=8f

----------------------------------------------------------------------

WN508002000158BA51 #

Once the data has been obtained, there are a couple of items to examine for known issues:

Register 30:

vendor reg: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
-----------------------------------------------------------
nvdimm 12: 94 00 58 60 1c 00 52 da 00 28 0a 96 58 00 00 03
nvdimm 13: 94 00 58 60 1b 00 54 e5 00 91 0a 96 58 00 00 06
nvdimm 14: 8f 00 58 60 19 00 54 d8 00 a5 09 96 58 00 00 05
nvdimm 15: 8f 00 38 60 1b 00 54 d8 00 6a 08 96 58 00 00 07

this register reports the PSM revision number. At this time there is no known issues with different revisions.
Register 32:

vendor reg: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
-----------------------------------------------------------
nvdimm 12: 94 00 58 60 1c 00 52 da 00 28 0a 96 58 00 00 03
nvdimm 13: 94 00 58 60 1b 00 54 e5 00 91 0a 96 58 00 00 06
nvdimm 14: 8f 00 58 60 19 00 54 d8 00 a5 09 96 58 00 00 05
nvdimm 15: 8f 00 38 60 1b 00 54 d8 00 6a 08 96 58 00 00 07

This register reports the status of the ESM itself. Typically expect a value of 0x58 here with bits 6 (Passed Diagnostics), 4 (ESM Ready) and 3 (ESM Present) set. In the example above, NVDIMM 15 is set to 0x38. With bit 5 (ESM End of Life Warning) set instead of bit 6. This would appear to indicate an upcoming failure of that ESM. However the FS1-2 does not monitor this bit. As such, it can be ignored at this time.
Register 34:

vendor reg: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
-----------------------------------------------------------
nvdimm 12: 94 00 58 60 1c 00 52 da 00 28 0a 96 58 00 00 03
nvdimm 13: 94 00 58 60 1b 00 54 e5 00 91 0a 96 58 00 00 06
nvdimm 14: 8f 00 58 60 19 00 54 d8 00 a5 09 96 58 00 00 05
nvdimm 15: 8f 00 38 60 1b 00 54 d8 00 6a 08 96 58 00 00 07

a value of 0xff in register 34 indicates a temperature out of range condition.
Register 3f:

vendor reg: 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f
-----------------------------------------------------------
nvdimm 12: 94 00 58 60 1c 00 52 da 00 28 0a 96 58 00 00 03
nvdimm 13: 94 00 58 60 1b 00 54 e5 00 91 0a 96 58 00 00 06
nvdimm 14: 8f 00 58 60 19 00 54 d8 00 a5 09 96 58 00 00 05
nvdimm 15: 8f 00 38 60 1b 00 54 d8 00 6a 08 96 58 00 00 07

can be mapped to their respective ESMs as follows:

NVDIMM 12 at P1/D0 is tied to ESM0 and should display 03
NVDIMM 13 at P1/D1 is tied to ESM2 and should display 06
NVDIMM 14 at P1/D2 is tied to ESM1 and should display 05
NVDIMM 15 at P1/D3 is tied to ESM3 and should display 07

if the values in register 3f do not match the above, that indicates that there is a problem between that NVDIMM and its ESM.

References

<BUG:20312168> - "FSCLI" COMMAND REQUIRED TO COLLECT THE NVDTOOL OUTPUT

Attachments

This solution has no attachment