Skip to main content
Version: 2.2

Update CubeOS

Update CubeOS#

USB drive#

copy CUBE_2.0.0_20201125-2132_a4d7645.pkg into usb drive (format: fat or exfat)

Plug the usb to the server

SCP#

upload CUBE_2.0.0_20201125-2132_a4d7645.pkg to your server over scp, directory: /var/update

$ scp CUBE_2.0.0_20201125-2132_a4d7645.pkg [email protected]:/var/update/

Connect to console#

$ ssh [email protected]Warning: Permanently added '10.32.2.220' (ECDSA) to the list of known hosts.Password:

Run Health Check#

CLI: cluster> health

sky1> clustersky1:cluster> health
[ ClusterLink ]link:Ping 10.32.2.220 ... OK
[ ClusterSettings ]etcd:http://10.32.2.220:12379 is healthy: successfully committed proposal: took = 1.020481ms
[ HaCluster ]hacluster:Cluster name: cube-9aid0G8SFk65gm93Cluster Summary:* Stack: corosync* Current DC: sky1 (version 2.0.3-5.el8_2.1-4b1f869f0f) - partition with quorum* Last updated: Mon Dec  7 16:19:11 2020* Last change:  Sun Nov  8 23:33:16 2020 by root via cibadmin on sky1* 1 node configured* 0 resource instances configured
Node List:* Online: [ sky1 ]
Full List of Resources:* No resources
Daemon Status:corosync: active/disabledpacemaker: active/disabledpcsd: active/disabled
[ MsgQueue ]rabbitmq:Cluster status of node rabbit@sky1 ...Basics
Cluster name: rabbit@localhost
Disk Nodes
rabbit@sky1
Running Nodes
rabbit@sky1
Versions
rabbit@sky1: RabbitMQ 3.8.3 on Erlang 22.3.4.1
Alarms
(none)
Network Partitions
(none)
Listeners
Node: rabbit@sky1, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communicationNode: rabbit@sky1, interface: 10.32.2.220, port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Feature flags
Flag: implicit_default_bindings, state: enabledFlag: quorum_queue, state: enabledFlag: virtual_host_metadata, state: enabled
Timeout: 60.0 seconds ...Listing queues for vhost / ...name    pid synchronised_slave_pidscompute_fanout_953e54e3b2544147805392b69a61028d <[email protected]>engine_worker.afe7a13c-d807-4ceb-9526-2dc9263c9828  <[email protected]>scheduler_fanout_1d604aeb42f14b2daad6e4b7c90da789   <[email protected]>
[ IaasDb ]mysql:wsrep_cluster_status    Disconnectedwsrep_cluster_size  0
[ VirtualIp ]vip:non-HAhaproxy_ha:non-HA
[ Storage ]ceph:cluster:    id:     c6e64c49-09cf-463b-9d1c-b6645b4b3b85    health: HEALTH_OK
services:    mon: 1 daemons, quorum sky1 (age 4w)    mgr: sky1(active, since 4w)    mds: cephfs:1 {0=sky1=up:active}    osd: 20 osds: 20 up (since 4w), 20 in (since 4w)    rgw: 1 daemon active (sky1)
task status:    scrub status:        mds.sky1: idle
data:    pools:   23 pools, 720 pgs    objects: 93.53k objects, 387 GiB    usage:   379 GiB used, 30 TiB / 30 TiB avail    pgs:     720 active+clean
io:    client:   121 KiB/s rd, 696 KiB/s wr, 214 op/s rd, 111 op/s wr

[ ApiService ]haproxy:(sky1)# pxname           svname     check_statusopenstack_horizon  localhost  L4OKgrafana_backend    localhost  L4OKkibana_backend     localhost  L4OKcube_lmi           localhost  L4OKswift_radosgw      localhost  L4OKhttpd:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+lmi:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+memcache:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+
[ Compute ]nova:+----+----------------+------+----------+---------+-------+----------------------------+| ID | Binary         | Host | Zone     | Status  | State | Updated At                 |+----+----------------+------+----------+---------+-------+----------------------------+| 10 | nova-scheduler | sky1 | internal | enabled | up    | 2020-12-07T08:19:23.000000 || 16 | nova-conductor | sky1 | internal | enabled | up    | 2020-12-07T08:19:18.000000 || 20 | nova-compute   | sky1 | nova     | enabled | up    | 2020-12-07T08:19:16.000000 |+----+----------------+------+----------+---------+-------+----------------------------+
[ Network ]neutron:+--------------------------------------+------------------------------+------+-------------------+-------+-------+-------------------------------+| ID                                   | Agent Type                   | Host | Availability Zone | Alive | State | Binary                        |+--------------------------------------+------------------------------+------+-------------------+-------+-------+-------------------------------+| aca5c90d-d014-4435-9f5b-e59822e046b3 | OVN Controller Gateway agent | sky1 | n/a               | :-)   | UP    | ovn-controller                || a1bdab2a-630e-4b8e-95b2-7cec8498544d | OVN Metadata agent           | sky1 | n/a               | :-)   | UP    | networking-ovn-metadata-agent |+--------------------------------------+------------------------------+------+-------------------+-------+-------+-------------------------------+
[ Image ]glance:+------------+--------------+------------+| Status     | API Endpoint | Images     |+------------+--------------+------------+| healthy    | available    | 10         |+------------+--------------+------------+
[ BlockStor ]cinder:+------------------+-----------+------+---------+-------+----------------------------+| Binary           | Host      | Zone | Status  | State | Updated At                 |+------------------+-----------+------+---------+-------+----------------------------+| cinder-backup    | sky1      | nova | enabled | up    | 2020-12-07T08:19:30.000000 || cinder-volume    | cube@ceph | nova | enabled | up    | 2020-12-07T08:19:33.000000 || cinder-scheduler | sky1      | nova | enabled | up    | 2020-12-07T08:19:29.000000 |+------------------+-----------+------+---------+-------+----------------------------+
[ FileStor ]manila:+----+------------------+--------------+------+---------+-------+----------------------------+| Id | Binary           | Host         | Zone | Status  | State | Updated_at                 |+----+------------------+--------------+------+---------+-------+----------------------------+| 1  | manila-scheduler | sky1         | nova | enabled | up    | 2020-12-07T08:19:31.000000 || 2  | manila-share     | sky1@generic | nova | enabled | up    | 2020-12-07T08:19:38.000000 |+----+------------------+--------------+------+---------+-------+----------------------------+
[ ObjectStor ]swift:+------------+--------------+--------+------------+------------+| Status     | API Endpoint | Tenant | Containers | Objects    |+------------+--------------+--------+------------+------------+| healthy    | available    | admin  | 0          | 0          |+------------+--------------+--------+------------+------------+
[ Orchestration ]heat:+------------+--------------+------------------+| Status     | API Endpoint | Engine (Up/Down) |+------------+--------------+------------------+| healthy    | available    |              5/0 |+------------+--------------+------------------+
[ LBaaS ]octavia:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ DNSaaS ]designate:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------++--------------------------------------+----------+--------------+--------+-------+--------------+| id                                   | hostname | service_name | status | stats | capabilities |+--------------------------------------+----------+--------------+--------+-------+--------------+| 3b3d3977-df4d-4842-8072-c03c5949706d | sky1     | api          | UP     | -     | -            || 69a20406-a2f7-4c7e-a5a7-b9dbc99142d4 | sky1     | central      | UP     | -     | -            || 82b68fc4-e7ff-4c50-bb5c-3fd0ffaa381d | sky1     | worker       | UP     | -     | -            || 9c7113c9-fec9-4ad5-87fa-9772569934ba | sky1     | mdns         | UP     | -     | -            || dbb4cf77-7a1b-4c43-96d7-3132050393bc | sky1     | producer     | UP     | -     | -            |+--------------------------------------+----------+--------------+--------+-------+--------------+
[ K8SaaS ]k3s:NAME   STATUS   ROLES    AGE   VERSION        INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                 CONTAINER-RUNTIMEsky1   Ready    master   28d   v1.18.8+k3s1   10.32.2.220   <none>        CentOS Linux 8 (Core)   4.18.0-193.19.1.el8_2.x86_64   containerd://1.3.3-k3s2rancher:NAMESPACE       NAME                                         READY   STATUS    RESTARTS   AGEkube-system     pod/local-path-provisioner-6d59f47c7-8wsz6   1/1     Running   0          28dkube-system     pod/metrics-server-7566d596c8-zpddc          1/1     Running   0          28dkube-system     pod/coredns-7944c66d8d-nrdq7                 1/1     Running   0          28dcattle-system   pod/rancher-5c5fb59466-rmmr7                 1/1     Running   1          28dcattle-system   pod/rancher-5c5fb59466-hh8nl                 1/1     Running   0          28dcattle-system   pod/rancher-5c5fb59466-4wks6                 1/1     Running   0          28d
NAMESPACE       NAME                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGEdefault         service/kubernetes         ClusterIP   10.43.0.1       <none>        443/TCP                  28dkube-system     service/kube-dns           ClusterIP   10.43.0.10      <none>        53/UDP,53/TCP,9153/TCP   28dkube-system     service/metrics-server     ClusterIP   10.43.177.106   <none>        443/TCP                  28dcattle-system   service/rancher            ClusterIP   10.43.236.92    <none>        80/TCP                   28dcattle-system   service/rancher-nodeport   NodePort    10.43.92.136    <none>        443:30443/TCP            28d
NAMESPACE       NAME                                     READY   UP-TO-DATE   AVAILABLE   AGEkube-system     deployment.apps/local-path-provisioner   1/1     1            1           28dkube-system     deployment.apps/metrics-server           1/1     1            1           28dkube-system     deployment.apps/coredns                  1/1     1            1           28dcattle-system   deployment.apps/rancher                  3/3     3            3           28d
NAMESPACE       NAME                                               DESIRED   CURRENT   READY   AGEkube-system     replicaset.apps/local-path-provisioner-6d59f47c7   1         1         1       28dkube-system     replicaset.apps/metrics-server-7566d596c8          1         1         1       28dkube-system     replicaset.apps/coredns-7944c66d8d                 1         1         1       28dcattle-system   replicaset.apps/rancher-5c5fb59466                 3         3         3       28d
[ InstanceHa ]masakari:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ DisasterRecovery ]freezer:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ DataPipe ]zookeeper:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+kafka:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+
[ Metrics ]monasca:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+telegraf:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+grafana:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ LogAnalytics ]filebeat:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+auditbeat:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+logstash:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+es:+------------+------------+| Status     | Error Code |+------------+------------+| healthy    |          0 |+------------+------------+kibana:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ Notifications ]influxdb:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+kapacitor:+------------+--------------+------------+| Status     | API Endpoint | Error Code |+------------+--------------+------------+| healthy    | available    |          0 |+------------+--------------+------------+
[ Node ]node:+------------+-------+---------------+---------------+|            |  CPU  |      Disk     |     Memory    |+------------+-------+-------+-------+-------+-------+| Host       | Usage | Usage | Avail | Usage | Avail |+------------+-------+-------+-------+-------+-------+| sky1       | 27.8% |   44% |   99G |   85% |  14Gi |+------------+-------+-------+-------+-------+-------+

Update CubeOS#

sky1> updatesky1:update> update1: local2: usb3: serverEnter index: 11: CUBE_2.0.0_20201125-2132_a4d7645.pkgEnter index: 1Firmware update will require an appliance reboot.Enter 'YES' to confirm: YESFormatting partition 2Installing CUBE_2.0.0_20201125-2132_a4d7645Running install scriptInstalling postinstall scriptFinished updating. Please reboot appliance.

Reboot#

sky1:update> rebootEnter 'YES' to confirm: YESConnection to 10.32.2.220 closed by remote host.Connection to 10.32.2.220 closed.

Run bootstrap_cube#

Welcome to the Cube ApplianceEnter "help" for a list of available commandssky1> bootsky1:boot> bootstrap_cubebootstraping cube...bootstrap successfullyif this is a single node start, just run "boot> cluster_sync"if this is a cluster start, wait until bootstrap_done is done in all nodes and run "cluster> start" to sync cluster data for all nodes

Run cluster_sync#

sky1:boot> cluster_synccluster_sync successfully

Check CubeOS version#

sky1:boot> backsky1> updatesky1:update> showCurrent: 2.0.0Rollback: 1.3.5sky1:update> backsky1> firmwaresky1:firmware> get_info1: CUBE_1.3.5_20200426-0942_6c7ef1e2: CUBE_2.0.0_20201125-2132_a4d7645Enter index: 2Firmware Version: Cube Appliance 2.0.0Installation Date: Nov 27, 2020 08:35:34 PMInstallation Type: UpgradeLast Boot: Nov 27, 2020 08:48:54 PMsky1:firmware>

Run Service check & repair#

sky1> clustersky1:cluster> check_repair          Service  Status  Report      ClusterLink      ok  [ link(v) clock(v) dns(v) ]       ClusterSys      ok  [ bootstrap(v) license(v) ]  ClusterSettings      ok  [ etcd(v) ]        HaCluster  FIXING  [ hacluster(3) ]                       ok  [ hacluster(f) ]         MsgQueue      ok  [ rabbitmq(v) ]           IaasDb      ok  [ mysql(v) ]        VirtualIp      ok  [ vip(v) haproxy_ha(v) ]          Storage      ok  [ ceph(v) ceph_mon(v) ceph_mgr(v) ceph_mds(v) ceph_osd(v) ceph_rgw(v) rbd_target(v) ]       ApiService      ok  [ haproxy(v) httpd(v) lmi(v) memcache(v) ]     SingleSignOn      ok  [ keycloak(v) ]          Compute      ok  [ nova(v) ]        Baremetal      ok  [ ironic(v) ]          Network      ok  [ neutron(v) ]            Image      ok  [ glance(v) ]        BlockStor      ok  [ cinder(v) ]         FileStor      ok  [ manila(v) ]       ObjectStor      ok  [ swift(v) ]    Orchestration      ok  [ heat(v) ]            LBaaS      ok  [ octavia(v) ]           DNSaaS      ok  [ designate(v) ]           K8SaaS      ok  [ k3s(v) rancher(v) ]       InstanceHa      ok  [ masakari(v) ] DisasterRecovery      ok  [ freezer(v) ]    BusinessLogic      ok  [ mistral(v) murano(v) cloudkitty(v) senlin(v) watcher(v) ]       ApiManager      ok  [ tyk(v) redis(v) mongodb(v) ]         DataPipe      ok  [ zookeeper(v) kafka(v) ]          Metrics      ok  [ monasca(v) telegraf(v) grafana(v) ]     LogAnalytics      ok  [ filebeat(v) auditbeat(v) logstash(v) es(v) kibana(v) ]    Notifications      ok  [ influxdb(v) kapacitor(v) ]sky1:cluster>
Last updated on by Roy Tan