Perform manual upgrades via the CLI

Manual upgrade mode lets you control CubeCOS upgrades directly through the command line. With this approach, you log in to nodes remotely and run system commands to manage the update process.

Use manual upgrade only if:

• One-click upgrades have failed
• You need precise control over specific nodes
• You're running CubeCOS version 3.0.0

note

If you're running CubeCOS version 3.1.0 or later, try the one-click upgrade first.

Prerequisites

Before you start, review the prerequisites and upgrade workflow sections in the one-click upgrade guide.

Upload the firmware files

Before you upgrade, verify your cluster health and upload the necessary files.

Check cluster health

Before starting the upgrade, check the health status of your cluster to ensure all services are running properly.

1. Enter the cluster management mode.

   cluster

2. Check cluster health:

   check

3. Verify all services show ok status.

Verify all nodes are healthy

1. Open the CubeCOS UI.
2. Go to Home > Nodes.
3. Confirm all nodes show a healthy status.

Upload firmware files

1. From your local machine, upload the firmware and MD5 checksum files to the /var/update/ directory:

   scp <local-file-path> <your-username>@<your-cluster-ip>:/var/update/

   Replace:

   • <local-file-path>: Local path to the firmware or checksum file
   • <your-username>: Your cluster username
   • <your-cluster-ip>: Your cluster IP address

2. When prompted, enter your SSH password.

3. Repeat the upload operation for all hosts in the cluster.

Verify uploaded firmware files

Before starting the upgrade, make sure that all the firmware files have been uploaded to each host in the cluster.

1. Enter the update management mode.

   update

2. List uploaded firmware images:

   list

3. When prompted, select local. This option specifies the location of your uploaded files.

Install the firmware

Install the firmware onto all the hosts in the cluster.

1. Enter the update management mode.

   update

2. Start the interactive install process.

   update

3. When prompted, select local. This option specifies the location of your uploaded files.

4. Select the image file that you want to install.

5. Enter YES to confirm the installation.

6. Wait for the installation process to return complete.

Migrate VMs before reboot

To ensure service continuity, migrate all virtual machines (VMs) from the node to other nodes before upgrading and rebooting.

danger

A cluster upgrade can affect performance, so make sure you perform this task during a maintenance window.

Run the following command on the master node only.

1. Enter the IAAS management mode.

   iaas

2. Access compute management:

   compute

3. Evacuate VMs from the node:

   upgrade_host_evacuation

Reboot the cluster

Firmware upgrades require a reboot.

One time operation

Only a manual migration is required on the master node of the cluster. Subsequent hosts will automatically migrate VMs and reboot sequentially.

Run this command on the master node:

1. Start rolling reboot operation for the entire cluster by executing the following command on the master node.

   reboot

2. Once the master node has completed the bootstrap process, the second node automatically reboots.

3. Before the reboot, the VM is automatically migrated to other nodes. After the reboot, auto-bootstrap is triggered automatically.

4. This will be performed for all nodes in the cluster.

Verify the upgrade

After the upgrade, check the health status of your cluster to ensure all services are running properly.

1. Enter the cluster management mode.

   cluster

2. Check the health status of the cluster.

   check

3. Ensure that the health status is OK. The following example shows the expected output:

             Service  Status  Report
          IaasDb      ok  [ mysql(v) ]
       Baremetal      ok  [ ironic(v) ]
          K8SaaS      ok  [ rancher(v) ]
      ObjectStor      ok  [ swift(v) ]
   ... # output abbreviated

Check firmware version on all nodes

After the upgrade, check the firmware version of each node matches the version you uploaded.

1. Enter the firmware management mode.

   firmware

2. Check the version on the active firmware partition.

   list

3. Ensure that the version of the active firmware matches the version you uploaded. The following example shows the expected output:

   1: CUBE_3.1.0_20260203-2300_be0cb70 [ACTIVE]
   Firmware Version: Cube Appliance 3.1.0
   Installation Date: Feb  5, 2026 11:33:20 AM
   Installation Type: Re-install USB
   Last Boot: Feb  5, 2026 11:46:38 AM
   Comment:
   2: CUBE_3.1.0_20260203-2300_be0cb70
   Firmware Version: Cube Appliance 3.1.0
   Installation Date: Feb  5, 2026 11:35:05 AM
   Installation Type: Re-install USB
   Last Boot: Never
   Comment:

Check all nodes after upgrade

Check the status of all nodes to ensure they are up and in a healthy state.

1. Open CubeCOS UI.
2. Navigate to Home > Nodes.
3. Check the status of all nodes from the node list.

Check the status of all the VMs

Check the status of all VMs to ensure they are all active.

1. Open skyline UI.
2. Navigate to Home > Compute > Instances.
3. Check the status of all VMs from the VM list.