Exadata Flashdisk in Poor Performance Status After Replacement

I recently encountered a bad flashdisk on an Exadata X2-2 machine and had it replaced. That is simple enough, just open a ticket with Oracle and schedule the hardware replacement. However, after the replacement two separate modules went into a “poor performance” state. Oracle support then advised me to follow the note listed below to fix it.

Flash Disks may report ‘Not Present’ or ‘Poor Performance’ after FDOM/Flash Disk Replacement [ID 1306635.1]

The note mentions that it covers Version 11.2.1.2.1 to 11.2.2.2.0 [Release 11.2], but I was on Version 11.2.3.2.1 .

A couple of notes about 1306635.1 as I encountered two errors with the statements.

1) “drop celldisk all flashdisk” command is out of order; “drop flashlog” needs to come first.
2) “drop celldisk all flashlog” is not a valid command

Please be sure to read the entire note as some additional commands may be required depending on the storage software version and whether or not you are in WriteBack or WriteThrough mode.

Here’s the original error with the part that was replaced; FLASH_4_1 reported as critical.

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
FLASH_1_0 1206M0CY4H normal
FLASH_1_1 1206M0CYL2 normal
FLASH_1_2 1206M0CY43 normal
FLASH_1_3 1206M0CY44 normal
FLASH_2_0 1202M0CN7C normal
FLASH_2_1 1202M0CRLX normal
FLASH_2_2 1202M0CN6D normal
FLASH_2_3 1202M0CQXA normal
FLASH_4_0 1202M0CR8P normal
FLASH_4_1 1202M0CR7P critical
FLASH_4_2 1202M0CRLQ normal
FLASH_4_3 1202M0CMWU normal
FLASH_5_0 1202M0CRJT normal
FLASH_5_1 1202M0CRL1 normal
FLASH_5_2 1202M0CRLT normal
FLASH_5_3 1202M0CRLP normal

The flashdisk in question was successfully replaced ( FLASH_4_1 ) and it reported normal; however, two more issues popped up on flashdisks in the same PCI slot 4 immediately after replacing the flashdisk.

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
FLASH_1_0 1206M0CY4H normal
FLASH_1_1 1206M0CYL2 normal
FLASH_1_2 1206M0CY43 normal
FLASH_1_3 1206M0CY44 normal
FLASH_2_0 1202M0CN7C normal
FLASH_2_1 1202M0CRLX normal
FLASH_2_2 1202M0CN6D normal
FLASH_2_3 1202M0CQXA normal
FLASH_4_0 1202M0CR8P normal
FLASH_4_1 1101M067DW normal
FLASH_4_2 1202M0CRLQ warning - poor performance
FLASH_4_3 1202M0CMWU warning - poor performance
FLASH_5_0 1202M0CRJT normal
FLASH_5_1 1202M0CRL1 normal
FLASH_5_2 1202M0CRLT normal
FLASH_5_3 1202M0CRLP normal

Following the note the support gave me, here’s the fix. As I mentioned before, please be sure to read the entire note as some additional commands may be required depending on the storage software version and whether or not you are in WriteBack or WriteThrough mode.

[root@exaxcel07 ~]# cellcli
CellCLI: Release 11.2.3.2.1 - Production on Tue Jun 04 13:22:11 CDT 2013

Copyright (c) 2007, 2012, Oracle.  All rights reserved.
Cell Efficiency Ratio: 6,291

CellCLI> alter lun 4_2 reenable force
LUN 4_2 on physical disk FLASH_4_2 successfully marked to status normal.LUN 4_2 successfully reenabled.

CellCLI> alter lun 4_3 reenable force
LUN 4_3 on physical disk FLASH_4_3 successfully marked to status normal.LUN 4_3 successfully reenabled.

CellCLI> drop flashcache
Flash cache exaxcel07_FLASHCACHE successfully dropped

CellCLI> drop flashlog
Flash log exaxcel07_FLASHLOG successfully dropped

CellCLI> drop celldisk all flashdisk
CellDisk FD_00_exaxcel07 successfully dropped
CellDisk FD_01_exaxcel07 successfully dropped
CellDisk FD_02_exaxcel07 successfully dropped
CellDisk FD_03_exaxcel07 successfully dropped
CellDisk FD_04_exaxcel07 successfully dropped
CellDisk FD_05_exaxcel07 successfully dropped
CellDisk FD_06_exaxcel07 successfully dropped
CellDisk FD_07_exaxcel07 successfully dropped
CellDisk FD_08_exaxcel07 successfully dropped
CellDisk FD_09_exaxcel07 successfully dropped
CellDisk FD_10_exaxcel07 successfully dropped
CellDisk FD_11_exaxcel07 successfully dropped
CellDisk FD_12_exaxcel07 successfully dropped
CellDisk FD_13_exaxcel07 successfully dropped
CellDisk FD_14_exaxcel07 successfully dropped
CellDisk FD_15_exaxcel07 successfully dropped

CellCLI> create celldisk all flashdisk
CellDisk FD_00_exaxcel07 successfully created
CellDisk FD_01_exaxcel07 successfully created
CellDisk FD_02_exaxcel07 successfully created
CellDisk FD_03_exaxcel07 successfully created
CellDisk FD_04_exaxcel07 successfully created
CellDisk FD_05_exaxcel07 successfully created
CellDisk FD_06_exaxcel07 successfully created
CellDisk FD_07_exaxcel07 successfully created
CellDisk FD_08_exaxcel07 successfully created
CellDisk FD_09_exaxcel07 successfully created
CellDisk FD_10_exaxcel07 successfully created
CellDisk FD_11_exaxcel07 successfully created
CellDisk FD_12_exaxcel07 successfully created
CellDisk FD_13_exaxcel07 successfully created
CellDisk FD_14_exaxcel07 successfully created
CellDisk FD_15_exaxcel07 successfully created

CellCLI> create flashlog all
Flash log exaxcel07_FLASHLOG successfully created

CellCLI> create flashcache all
Flash cache exaxcel07_FLASHCACHE successfully created

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
	 FLASH_1_0	 1206M0CY4H	 normal
	 FLASH_1_1	 1206M0CYL2	 normal
	 FLASH_1_2	 1206M0CY43	 normal
	 FLASH_1_3	 1206M0CY44	 normal
	 FLASH_2_0	 1202M0CN7C	 normal
	 FLASH_2_1	 1202M0CRLX	 normal
	 FLASH_2_2	 1202M0CN6D	 normal
	 FLASH_2_3	 1202M0CQXA	 normal
	 FLASH_4_0	 1202M0CR8P	 normal
	 FLASH_4_1	 1101M067DW	 normal
	 FLASH_4_2	 1202M0CRLQ	 normal
	 FLASH_4_3	 1202M0CMWU	 normal
	 FLASH_5_0	 1202M0CRJT	 normal
	 FLASH_5_1	 1202M0CRL1	 normal
	 FLASH_5_2	 1202M0CRLT	 normal
	 FLASH_5_3	 1202M0CRLP	 normal

All is good now.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s