---
title: NVMe Drive Firmware Upgrade Process
slug: -I9FJ0PUXTPrzHLfe2ZhU
description: Upgrade NVMe drive firmware in Ocient, enhancing system stability and performance.
docTags: 
createdAt: Thu Jul 21 2022 17:05:13 GMT+0000 (Coordinated Universal Time)
updatedAt: Mon Mar 30 2026 18:52:28 GMT+0000 (Coordinated Universal Time)
---

The Ocient® software uses NVMe drives for storage because of their higher input or output (I/O) throughput than spinning media drives. These drives are referred to as data disks or data drives. When you run Ocient software, the Ocient System accesses these data drives using the User Mode I/O (UIO) driver during normal operation.

This procedure guides a System Administrator through binding the drives back to the NVMe driver and upgrading the drive firmware. The procedure includes steps for upgrading the firmware of only Intel® NVMe drives. For other drive models, follow a similar process using the firmware upgrade process from the manufacturer.

:::hint{type="danger"}
Stop the `rolehostd` process prior to working on the NVMe drives. Otherwise, when the `rolehostd` process is running, the process becomes unstable because it has open file handles on the drives.
:::

Execution of this process requires the sudo privilege on the node where the drive firmware is updated.

## Prerequisites

Perform the firmware upgrade on one node of any specified type (Foundation, Loader, SQL) at a time. You can execute the firmware upgrade in parallel on different types of nodes.

:::hint{type="info"}
Perform the upgrade during a maintenance window because issues with the upgrade might prevent the node from returning to service.
:::

The prerequisites for this procedure are:

1. The following packages are installed on the node undergoing firmware upgrade:
   - `smartmontools`
   - `nvme-cli`
   - `intelmas`
   - `sedutil`
2. The target firmware version for the upgrade is known and documented.
3. If you upgrade the data drive firmware on a Loader Node, prior to beginning the procedure, take the node out of the loading path by updating the LAT configuration. Before you shut down the `rolehostd` process, drain the pages that are held by the node.

The following example illustrates the firmware update commands used for an Intel NVMe SSD. Please substitute the appropriate command for your drive manufacturer in each of the steps that use the `intelmas` command.

## Firmware Upgrade Procedure

The following steps require `sudo` permissions on the node.

### Prepare for the Firmware Upgrade

1\. Log in to the node as an administrator user.

2\. If you upgrade the drive firmware on a Foundation Node or Loader Node, save the status of the drives for later in the procedure. After you replace `<IP_ADDRESS_OF_THE_NODE>` with the IP address of your node, save the output of this command.&#x20;

```curl
curl -s http://<IP_ADDRESS_OF_THE_NODE>:9090/v1/stats | jq -r '.[] | select (.name ==
 "localStorageService.device.status" ) | "(.device) (.value)" '
```

Alternatively, you can run a query on the SQL Node to display the status of the individual storage device and firmware version. After you replace `<NODE_NAME>` with the name of your node, connect to any SQL Node on the system and run this query.

```sql
SELECT
  s.id AS serial_number,
  s.pci_address,
  s.device_status,
  s.firmware_version
FROM
  sys.nodes n
JOIN sys.storage_device_status s
  ON n.id = s.node_id
WHERE
  n.name = '<NODE_NAME>';
```

Output

```none
serial_number     |pci_address |device_status |firmware_version
------------------+------------+--------------+-----------------
CVMD6356009M800GGN|0000:3b:00.0|ACTIVE        |VDV10184
CVMD636400BZ800GGN|0000:3c:00.0|ACTIVE        |VDV10184
CVMD636400JA800GGN|0000:5e:00.0|ACTIVE        |VDV10184
CVMD6365003C800GGN|0000:5f:00.0|ACTIVE        |VDV10184
CVMD6365003P800GGN|0000:86:00.0|ACTIVE        |VDV10184
CVMD63650092800GGN|0000:87:00.0|ACTIVE        |VDV10184
PHLJ0431019J8P0HGN|0000:88:00.0|ACTIVE        |VDV10184
PHLJ106500018P0HGN|0000:89:00.0|ACTIVE        |VDV10170
```

3\. If you upgrade the drive firmware on a SQL Node, save the output of the `df` command.

### Stop the `rolehostd` Process and Attach the Data Drives

1\. Stop the `rolehostd` process to ensure that the drive replacement does not impact system stability.

```shell
sudo systemctl stop rolehostd
```

2\. Prevent the `rolehostd` process from starting after reboot.

```shell
sudo systemctl disable rolehostd
```

3\. Confirm that the output of this command indicates that the `rolehostd` process is not running and that the process auto start on reboot setting is disabled.

```shell
sudo systemctl status rolehostd
```

:::hint{type="info"}
When you upgrade the drive firmware, you can impact data reliability while the `rolehostd` process is running on the node.
:::

4\. Attach the data drives to the NVMe driver.

```shell
sudo /opt/ocient/scripts/nvme-driver-util.sh bind-nvme
```

5\. Ensure that all drives are not the operating system drive and are attached to the NVMe driver in the output of this command. Any drives that did not bind to the NVMe driver in the previous step do not upgrade, so you must investigate any failure to bind to the NVMe driver.

```shell
/opt/ocient/scripts/nvme-driver-util.sh
```

**INCORRECT**: Eight data drives are attached to the UIO driver. This is the incorrect system state for performing a firmware upgrade.

```none
./opt/ocient/scripts/nvme-driver-util.sh
NVMe device status

use option 'bind-uio' to bind uio_pci_generic driver to Ocient Payload/unpartitioned drives
use option 'bind-nvme' to bind nvme driver to drives

BDF             Numa Node       Driver name             Device name
0000:01:00.0    0               nvme                    nvme0n1
0000:3b:00.0    0               uio_pci_generic         -
0000:3c:00.0    0               uio_pci_generic         -
0000:5e:00.0    0               uio_pci_generic         -
0000:5f:00.0    0               uio_pci_generic         -
0000:86:00.0    1               uio_pci_generic         -
0000:87:00.0    1               uio_pci_generic         -
0000:88:00.0    1               uio_pci_generic         -
0000:89:00.0    1               uio_pci_generic         -
```

**CORRECT**: The data drives are attached to the NVMe driver. This is the correct system state for performing a firmware upgrade.

```none
./opt/ocient/scripts/nvme-driver-util.sh
NVMe device status

use option 'bind-uio' to bind uio_pci_generic driver to Ocient Payload/unpartitioned drives
use option 'bind-nvme' to bind nvme driver to drives

BDF             Numa Node       Driver name             Device name
0000:01:00.0    0               nvme                    nvme0n1
0000:3b:00.0    0               nvme                    nvme1n1
0000:3c:00.0    0               nvme                    nvme2n1
0000:5e:00.0    0               nvme                    nvme3n1
0000:5f:00.0    0               nvme                    nvme4n1
0000:86:00.0    1               nvme                    nvme5n1
0000:87:00.0    1               nvme                    nvme6n1
0000:88:00.0    1               nvme                    nvme7n1
0000:89:00.0    1               nvme                    nvme8n1
```

### Perform Firmware Upgrade

1\. Run this command and review the `FirmwareUpdateAvailable` field to identify the drives that need a firmware upgrade. Save the results of this step for comparison later in the process.

:::CodeblockTabs
Example

```shell
sudo intelmas show  -d Index, SerialNumber, DeviceStatus, Capacity, FirmwareUpdateAvailable -intelssd
```
:::

Drive 8 needs the firmware update, but drive 7 does not need the update.

:::CodeblockTabs
Output

```none
- 7 Intel SSD DC P4510 Series PHLJ0431019J8P0HGN -

Capacity : 8001.56 GB
DeviceStatus : Healthy
FirmwareUpdateAvailable : The selected drive contains current firmware as of this tool release.
Index : 7
SerialNumber : PHLJ0431019J8P0HGN

- 8 Intel SSD DC P4510 Series PHLJ106500018P0HGN -

Capacity : 8001.56 GB
DeviceStatus : Healthy
FirmwareUpdateAvailable : Firmware=VDV10170 Bootloader=VB1B0172
Index : 8
SerialNumber : PHLJ106500018P0HGN
```
:::

2\. Execute this command for every drive in the previous step that indicates an old firmware in the `FirmwareUpdateAvailable` field. Replace `<DRIVE_INDEX>` with the number shown in the `Index` field from the output of the previous command for all drives that require the update.

```shell
sudo intelmas load -intelssd <DRIVE_INDEX>
```

3\. Ensure that the output shows `Firmware updated successfully.` This is the sample log output for the upgrade.

```none
- intelmas load -intelssd 8
WARNING! You have selected to update the drives firmware!
Proceed with the update? (Y|N): Y
Checking for firmware update...

- Intel SSD DC P4510 Series PHLJ106500018P0HGN -

Status : Firmware updated successfully. Please reboot the system.
```

4\. After you upgrade all drives that require the firmware upgrade, reboot the node using the `reboot` command. After the node completes the reboot, log in to the node as an administrator.

5\. Attach the data drives to the NVMe driver using this command.

```shell
sudo /opt/ocient/scripts/nvme-driver-util.sh bind-nvme
```

6\. Compare the output of this command with the output of the same command captured earlier in this process. Ensure that the reported drive status and capacity are the same as before the upgrade and that the firmware shown is current.

```shell
sudo intelmas show -d Index, SerialNumber, DeviceStatus, Capacity, FirmwareUpdateAvailable -intelssd
```

7\. Reattach the data drives to the UIO driver.

```shell
sudo /opt/ocient/scripts/nvme-driver-util.sh bind-uio
```

### Enable the `rolehostd` Process and Perform Final Checks

1\. Enable the `rolehostd` process to start after reboot, and then start the process.

```shell
sudo systemctl enable rolehostd && sudo systemctl start rolehostd
```

2\. Ensure that the `rolehostd` process is enabled and started.

```shell
sudo systemctl status rolehostd
```

3\. If you upgrade the firmware of the drive on a Foundation Node or Loader Node, ensure that the output of this command matches the output of the same command captured earlier in this process.

```curl
curl -s http://<IP_ADDRESS_OF_THE_NODE>:9090/v1/stats | jq -r '.[] | select (.name == "localStorageService.device.status" ) | "\(.device) \(.value)" '
```

Alternatively, you can run a query on the SQL Node to capture the individual device status and firmware version. Ensure that the drives and statuses in this query match the results from the query you executed earlier in this process. After you replace the node name `<NODE_NAME>` with the name of your node, connect to any SQL Node on the system and execute this query.

```sql
SELECT
  s.id as serial_number,
  s.pci_address,
  s.device_status,
  s.firmware_version
FROM
  sys.nodes n
JOIN sys.storage_device_status s
  ON n.id = s.node_id
WHERE
  n.name = '<NODE_NAME>';
```

Output&#x20;

```none
serial_number     |pci_address |device_status |firmware_version
------------------+------------+--------------+-----------------
CVMD6356009M800GGN|0000:3b:00.0|ACTIVE        |VDV10184
CVMD636400BZ800GGN|0000:3c:00.0|ACTIVE        |VDV10184
CVMD636400JA800GGN|0000:5e:00.0|ACTIVE        |VDV10184
CVMD6365003C800GGN|0000:5f:00.0|ACTIVE        |VDV10184
CVMD6365003P800GGN|0000:86:00.0|ACTIVE        |VDV10184
CVMD63650092800GGN|0000:87:00.0|ACTIVE        |VDV10184
PHLJ0431019J8P0HGN|0000:88:00.0|ACTIVE        |VDV10184
PHLJ106500018P0HGN|0000:89:00.0|ACTIVE        |VDV10184
```

4\. If you upgrade the firmware of the drive on a SQL Node, ensure that the output of the `df` command shows the same mount points as those captured earlier in this process.

## Related Links

[System Catalog](https://docs.ocient.com/system-catalog)

