If a Commvault HyperScale X node fails due to a problem with a disk, you can replace the failed disk with a new, empty disk. Ransomware protection is available in the new disk.
Tip
To physically identify the failed drive, look for the drive with a lit drive light. When a disk fails, its drive light is automatically lit.
Before You Begin
Caution
-
Before you proceed with the disk replacement, please verify that the serial number of the disk matches with the serial number that appears in the IPMI console for the disk.
-
If you need to replace multiple disks on a node, then you must replace all the failed disks with new disks.
-
You should not replace a disk when the node is powered down. Node must be powered on and CVFS services (hedvigswatch, hedvighblock) must be running during the disk replacement.
-
If the node cannot access the replaced disk, check if the status of the disk needs to be set to NON-RAID at BIOS or IRMC. This setting can differ with each hardware vendor.
If the hardware vendor does not identify the failed disk, to locate the failed disk, run the following commands:
-
Identify the CVFS version on the node.
/usr/local/hedvig/scripts/whichCommit.sh
-
Run the lsblk command and identify the original disk
# lsblk | grep 'hedvig/d*' | grep -v 'lvm'
-
Turn on the drive light on the disk using the following steps:
Note
This step is supported only on CVFS version 4.7.8 and higher.
-
Determine the physical slot for the disk
# /usr/local/hedvig/scripts/spm/disk-helper.py --device sd[X] list_slot
-
Identify the serial number of the disk.
# /usr/local/hedvig/scripts/spm/disk-helper.py --device sd[X] show_serial
-
Turn on the drive light to easily locate the disk.
# /usr/local/hedvig/scripts/spm/disk-helper.py --device sd[X] led_on
-
To turn off the drive light, run the following command:
# /usr/local/hedvig/scripts/spm/disk-helper.py --device sd[X] led_off
-
-
Verify that the original (failed) disk is not mounted, by running both the following commands:
# lsblk
# mount
The original disk should not be mounted in the output of both the commands. (The MOUNTPOINT for the original disk should appear blank.)
If the original (failed) disk is mounted, unmount it by running the following command:
umount -l /dev/sdIdHere
-
Before attaching the new disk, verify the following:
-
The capacity of the new disk is equal to the disk that you want to replace.
-
The new disk does not contain a file system.
-
Attach the new disk to the node that has the failed disk.
-
Verify that the entry in
/etc/fstab
for the failed disk is unchanged.
-
-
Run the following command to verify that the new disk appears in the unmounted state:
# lsblk
The new disk will probably be listed as the final device in the list. The new disk should not be mounted in the output. (The MOUNTPOINT for the new disk should appear blank.)
-
Enable root access, if root access is disabled in the storage pool. For more information, see Enabling or Disabling Root Access.
Procedure
Caution
You must replace the disk using the Command Center, as shown in the following steps. Attempting to manually replace a disk might make the disk ineligible for replacement.
-
From the navigation, click Storage > HyperScale.
The HyperScale page appears.
-
In the Name column, click the <storage pool> that has the disk that you want to replace.
The <storage pool> page appears.
-
On the Nodes tab, click Column Settings () and then click Columns > Actions.
The Actions column will be displayed.
-
In the node that has the disk that needs to be replaced, click the Actions button , and then click Replace drives.
The Confirm replace dialog box is displayed.
-
Verify the MediaAgent name and make sure that it is associated with the node that has the disk that needs to be replaced., and then click Yes.
Result
-
The new disk appears with the Online (Drive replacement is in progress) status, initially, while the data recovery from the failed disk is in progress.
-
After successful data recovery, the new disk appears with the Online status.
-
Verify that the sestatus reports "Enforcing" and all Commvault and Hedvig processes run under cvstorage context. You can run the following command to ensure that the processes are running under cvstorage context.
pstree -Z | grep -i cv
-
Enable root access, if root access is disabled in the storage pool. For more information, see Enabling or Disabling Root Access.