Update CubeOS
SCP#
upload CUBE_2.3.0_20220707-1705_fd11e121.pkg to your server over scp, directory: /var/update
$ scp CUBE_2.3.0_20220707-1705_fd11e121.pkg [email protected]:/var/update/Connect to console#
$ ssh [email protected]Warning: Permanently added '192.168.100.100' (ECDSA) to the list of known hosts.Password:Run Health Check#
CLI: cluster> health
sky1> cluster health [ ClusterLink ] link: Ping 192.168.100.100 ... OK clock: time difference to sky1 is 0ms dns: sky1 DNS lookup took 0m0.125s
[ ClusterSys ] bootstrap: sky1 cube services... [ready] license: controller: sky1 serial: 1H2ZLG2 ---------------------------------------------------------------- license.type=trial issue.by=Bigstack Ltd. issue.to=roy issue.hardware=* issue.date=2022-04-14 06:17:26 UTC expiry.date=2022-10-11 06:17:26 UTC ---------------------------------------------------------------- state: License (type: trial) is valid for 91 days
[ ClusterSettings ] etcd: http://192.168.100.100:12379 is healthy: successfully committed proposal: took = 995.8µs
[ HaCluster ] hacluster: Cluster name: cube-pSXpo2S8ttbP4a7P Cluster Summary: * Stack: corosync * Current DC: sky1 (version 2.1.2-4.el8-ada5c3b36e2) - partition with quorum * Last updated: Tue Jul 12 10:13:59 2022 * Last change: Fri Jul 8 15:41:32 2022 by root via cibadmin on sky1 * 1 node configured * 0 resource instances configured
Node List: * Online: [ sky1 ]
Full List of Resources: * No resources
Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/disabled
[ MsgQueue ] rabbitmq: warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell) Cluster status of node rabbit@sky1 ... Basics
Cluster name: rabbit@sky1
Disk Nodes
rabbit@sky1
Running Nodes
rabbit@sky1
Versions
rabbit@sky1 : RabbitMQ 3.8.3 on Erlang 22.3.4.1
Alarms
(none)
Network Partitions
(none)
Listeners
Node: rabbit@sky1 , interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@sky1 , interface: 192.168.100.100, port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0 Node: rabbit@sky1 , interface: [::], port: 15672, protocol: http, purpose: HTTP API
Feature flags
Flag: drop_unroutable_metric, state: enabled Flag: empty_basic_get_metric, state: enabled Flag: implicit_default_bindings, state: enabled Flag: quorum_queue, state: enabled Flag: virtual_host_metadata, state: enabled
warning: the VM is running with native name encoding of latin1 which may cause Elixir to malfunction as it expects utf8. Please ensure your locale is set to UTF-8 (which can be verified by running "locale" in your shell) Timeout: 60.0 seconds ... Listing queues for vhost / ... name pid synchronised_slave_pids octavia_provisioning_v2 <rabbit@sky1 .1.3041.0> conductor.sky1 <rabbit@sky1 .1.2067.0> q-reports-plugin.sky1 <rabbit@sky1 .1.1807.0>
[ IaasDb ] mysql: wsrep_cluster_status Disconnected wsrep_cluster_size 0
[ VirtualIp ] vip: non-HA haproxy_ha: non-HA
[ Storage ] ceph: cluster: id: c6e64c49-09cf-463b-9d1c-b6645b4b3b85 health: HEALTH_WARN noout flag(s) set
services: mon: 1 daemons, quorum sky1 (age 3d) mgr: sky1 (active, since 3d) mds: 1/1 daemons up osd: 46 osds: 46 up (since 3d), 46 in (since 3d) flags noout rgw: 1 daemon active (1 hosts, 1 zones)
data: volumes: 1/1 healthy pools: 25 pools, 753 pgs objects: 5.06k objects, 21 GiB usage: 24 GiB used, 13 TiB / 13 TiB avail pgs: 753 active+clean
io: client: 1.2 KiB/s rd, 1.3 KiB/s wr, 1 op/s rd, 0 op/s wr
[ ApiService ] haproxy: (sky1 ) # pxname svname check_status openstack_horizon localhost L4OK grafana_backend localhost L4OK kibana_backend localhost L4OK prometheus_backend localhost L4OK ceph_dashbaord_backend localhost L6OK cube_lmi localhost L4OK swift_radosgw localhost L4OK httpd: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+ lmi: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+ memcache: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+
[ SingleSignOn ] keycloak: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/keycloak-0 1/1 Running 2 3d23h 10.42.0.60 sky1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/keycloak-headless ClusterIP None <none> 80/TCP 3d23h app.kubernetes.io/instance=keycloak,app.kubernetes.io/name=keycloak service/keycloak-http ClusterIP 10.43.101.126 <none> 80/TCP,8443/TCP,9990/TCP 3d23h app.kubernetes.io/instance=keycloak,app.kubernetes.io/name=keycloak
NAME READY AGE CONTAINERS IMAGES statefulset.apps/keycloak 1/1 3d23h keycloak localhost:5080/bigstack/keycloak:11.0.3
[ Compute ] nova: +----+----------------+---------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+----------------+---------------+----------+---------+-------+----------------------------+ | 1 | nova-conductor | sky1 | internal | enabled | up | 2022-07-12T02:14:04.000000 | | 4 | nova-scheduler | sky1 | internal | enabled | up | 2022-07-12T02:14:01.000000 | | 13 | nova-compute | sky1 | nova | enabled | up | 2022-07-12T02:14:04.000000 | | 14 | nova-compute | sky1-ironic | nova | enabled | up | 2022-07-12T02:14:09.000000 | +----+----------------+---------------+----------+---------+-------+----------------------------+
[ Baremetal ] ironic: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ +---------------------+----------------+ | Supported driver(s) | Active host(s) | +---------------------+----------------+ | ipmi | sky1 | +---------------------+----------------+
[ Network ] neutron: +--------------------------------------+----------------------+--------+-------------------+-------+-------+----------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+----------------------+--------+-------------------+-------+-------+----------------------------+ | 35eeb37f-0ecd-427d-b9a6-bb0a6c6e5327 | OVN Controller agent | sky1 | | :-) | UP | ovn-controller | | 60903474-cdb6-5a8b-8cf6-1c4fd4a4019f | OVN Metadata agent | sky1 | | :-) | UP | neutron-ovn-metadata-agent | | 7aaa52cf-c58d-5d78-a95c-139f8edc8502 | VPN Agent | sky1 | | :-) | UP | neutron-ovn-vpn-agent | +--------------------------------------+----------------------+--------+-------------------+-------+-------+----------------------------+
[ Image ] glance: +------------+--------------+------------+ | Status | API Endpoint | Images | +------------+--------------+------------+ | healthy | available | 2 | +------------+--------------+------------+
[ BlockStor ] cinder: +------------------+-----------+------+---------+-------+----------------------------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+-----------+------+---------+-------+----------------------------+ | cinder-backup | sky1 | nova | enabled | up | 2022-07-12T02:14:29.000000 | | cinder-volume | cube@ceph | nova | enabled | up | 2022-07-12T02:14:23.000000 | | cinder-scheduler | sky1 | nova | enabled | up | 2022-07-12T02:14:27.000000 | +------------------+-----------+------+---------+-------+----------------------------+
[ FileStor ] manila: +----+------------------+----------------+------+---------+-------+----------------------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | +----+------------------+----------------+------+---------+-------+----------------------------+ | 1 | manila-scheduler | sky1 | nova | enabled | up | 2022-07-12T02:14:24.874495 | | 2 | manila-share | sky1@generic | nova | enabled | up | 2022-07-12T02:14:31.265623 | +----+------------------+----------------+------+---------+-------+----------------------------+
[ ObjectStor ] swift: +------------+--------------+--------+------------+------------+ | Status | API Endpoint | Tenant | Containers | Objects | +------------+--------------+--------+------------+------------+ | healthy | available | admin | 0 | 0 | +------------+--------------+--------+------------+------------+
[ Orchestration ] heat: +------------+--------------+------------------+ | Status | API Endpoint | Engine (Up/Down) | +------------+--------------+------------------+ | healthy | available | 3/0 | +------------+--------------+------------------+
[ LBaaS ] octavia: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ DNSaaS ] designate: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+ +--------------------------------------+----------+--------------+--------+-------+--------------+ | id | hostname | service_name | status | stats | capabilities | +--------------------------------------+----------+--------------+--------+-------+--------------+ | ce221951-bf8d-4f3b-b447-f84aeb6ac90e | sky1 | central | UP | - | - | | 311e56af-b112-4e3b-9adf-3f483a5dd6ef | sky1 | api | UP | - | - | | b78c2c81-d02a-4f98-8fa9-b43af656f92f | sky1 | worker | UP | - | - | | c3ec2588-b3ea-4b7d-a61b-8daeb8de3712 | sky1 | producer | UP | - | - | | d9a8d377-687e-4869-92e3-bd724da343ae | sky1 | mdns | UP | - | - | +--------------------------------------+----------+--------------+--------+-------+--------------+
[ K8SaaS ] k3s: NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME sky1 Ready control-plane,etcd,master 3d23h v1.20.4+k3s1 192.168.100.100 <none> CentOS Stream 8 4.18.0-348.el8.x86_64 containerd://1.4.3-k3s3
rancher: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/rancher-5d7d796496-k2mpj 1/1 Running 2 3d23h 10.42.0.66 sky1 <none> <none> pod/rancher-webhook-b5b7b76c4-hpn6j 1/1 Running 2 3d23h 10.42.0.63 sky1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/rancher ClusterIP 10.43.161.254 <none> 80/TCP,443/TCP 3d23h app=rancher service/rancher-webhook ClusterIP 10.43.144.204 <none> 443/TCP 3d23h app=rancher-webhook
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR deployment.apps/rancher 1/1 1 1 3d23h rancher localhost:5080/bigstack/rancher:v2.5.7 app=rancher deployment.apps/rancher-webhook 1/1 1 1 3d23h rancher-webhook rancher/rancher-webhook:v0.1.0-beta9 app=rancher-webhook
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR replicaset.apps/rancher-5d7d796496 1 1 1 3d23h rancher localhost:5080/bigstack/rancher:v2.5.7 app=rancher,pod-template-hash=5d7d796496 replicaset.apps/rancher-webhook-b5b7b76c4 1 1 1 3d23h rancher-webhook rancher/rancher-webhook:v0.1.0-beta9 app=rancher-webhook,pod-template-hash=b5b7b76c4
[ InstanceHa ] masakari: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ DisasterRecovery ] freezer: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ BusinessLogic ] mistral: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ murano: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ cloudkitty: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ +------------------+------+--------------------------------------------------------+ | Metric | Unit | Metadata | +------------------+------+--------------------------------------------------------+ | vcpu | vcpu | ['hostname', 'zone', 'component', 'device', 'service'] | | memory | GiB | ['hostname', 'zone', 'component', 'device', 'service'] | | network.inbound | GB | ['hostname', 'component', 'service', 'zone'] | | network.outbound | GB | ['hostname', 'component', 'service', 'zone'] | +------------------+------+--------------------------------------------------------+ senlin: +-----------------------+--------+---------+-------+---------------------+-----------------+ | binary | host | status | state | updated_at | disabled_reason | +-----------------------+--------+---------+-------+---------------------+-----------------+ | senlin-engine | sky1 | enabled | up | 2022-07-12T02:14:52 | None | | senlin-conductor | sky1 | enabled | up | 2022-07-12T02:14:37 | None | | senlin-health-manager | sky1 | enabled | up | 2022-07-12T02:14:53 | None | +-----------------------+--------+---------+-------+---------------------+-----------------+ watcher: +----+-------------------------+--------+--------+ | ID | Name | Host | Status | +----+-------------------------+--------+--------+ | 1 | watcher-applier | sky1 | ACTIVE | | 2 | watcher-decision-engine | sky1 | ACTIVE | +----+-------------------------+--------+--------+
[ ApiManager ] tyk: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/dashboard-tyk-6t9lf 1/1 Running 2 3d23h 10.42.0.67 sky1 <none> <none> pod/gateway-tyk-7m5vs 1/1 Running 5 3d23h 192.168.100.100 sky1 <none> <none> pod/pump-tyk-76c5565f8b-km7j5 1/1 Running 2 3d23h 10.42.0.57 sky1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/dashboard-tyk ClusterIP 10.43.122.228 <none> 3000/TCP 3d23h app=dashboard-tyk
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR daemonset.apps/dashboard-tyk 1 1 1 1 1 <none> 3d23h dashboard-tyk localhost:5080/bigstack/tyk-dashboard:latest app=dashboard-tyk daemonset.apps/gateway-tyk 1 1 1 1 1 <none> 3d23h gateway-tyk-headless docker.io/tykio/tyk-gateway:v3.2.1 app=gateway-tyk,release=tyk
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR deployment.apps/pump-tyk 1/1 1 1 3d23h pump-tyk-headless docker.io/tykio/tyk-pump-docker-pub:v1.4.0 app=pump-tyk,release=tyk
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR replicaset.apps/pump-tyk-76c5565f8b 1 1 1 3d23h pump-tyk-headless docker.io/tykio/tyk-pump-docker-pub:v1.4.0 app=pump-tyk,pod-template-hash=76c5565f8b,release=tyk
redis: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/redis-node-0 2/2 Running 4 3d23h 10.42.0.54 sky1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/redis ClusterIP 10.43.235.172 <none> 6379/TCP,26379/TCP 3d23h app.kubernetes.io/component=node,app.kubernetes.io/instance=redis,app.kubernetes.io/name=redis service/redis-headless ClusterIP None <none> 6379/TCP,26379/TCP 3d23h app.kubernetes.io/instance=redis,app.kubernetes.io/name=redis
NAME READY AGE CONTAINERS IMAGES statefulset.apps/redis-node 1/1 3d23h redis,sentinel docker.io/bitnami/redis:6.2.4-debian-10-r13,docker.io/bitnami/redis-sentinel:6.2.4-debian-10-r14
mongodb: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/mongodb-0 1/1 Running 2 3d23h 10.42.0.65 sky1 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/mongodb ClusterIP 10.43.172.89 <none> 27017/TCP 3d23h app.kubernetes.io/component=mongodb,app.kubernetes.io/instance=mongodb,app.kubernetes.io/name=mongodb service/mongodb-headless ClusterIP None <none> 27017/TCP 3d23h app.kubernetes.io/component=mongodb,app.kubernetes.io/instance=mongodb,app.kubernetes.io/name=mongodb
NAME READY AGE CONTAINERS IMAGES statefulset.apps/mongodb 1/1 3d23h mongodb docker.io/bitnami/mongodb:4.4.6-debian-10-r29
[ DataPipe ] zookeeper: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ kafka: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+
[ Metrics ] monasca: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ telegraf: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ grafana: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ LogAnalytics ] filebeat: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ auditbeat: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ logstash: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ es: +------------+------------+ | Status | Error Code | +------------+------------+ | healthy | 0 | +------------+------------+ kibana: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ Notifications ] influxdb: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+ kapacitor: +------------+--------------+------------+ | Status | API Endpoint | Error Code | +------------+--------------+------------+ | healthy | available | 0 | +------------+--------------+------------+
[ Node ] node: +------------+-------+---------------+---------------+ | | CPU | Disk | Memory | +------------+-------+-------+-------+-------+-------+ | Host | Usage | Usage | Avail | Usage | Avail | +------------+-------+-------+-------+-------+-------+ | sky1 | 8.4% | 10% | 225G | 32% | 84Gi | +------------+-------+-------+-------+-------+-------+Check Services#
Make sure all services is up and running
sky1:cluster> check Service Status Report ClusterLink ok [ link(v) clock(v) dns(v) ] ClusterSys ok [ bootstrap(v) license(v) ] ClusterSettings ok [ etcd(v) ] HaCluster ok [ hacluster(v) ] MsgQueue ok [ rabbitmq(v) ] IaasDb ok [ mysql(v) ] VirtualIp ok [ vip(v) haproxy_ha(v) ] Storage ok [ ceph(v) ceph_mon(v) ceph_mgr(v) ceph_mds(v) ceph_osd(v) ceph_rgw(v) rbd_target(v) ] ApiService ok [ haproxy(v) httpd(v) lmi(v) memcache(v) ] SingleSignOn ok [ keycloak(v) ] Compute ok [ nova(v) ] Baremetal ok [ ironic(v) ] Network ok [ neutron(v) ] Image ok [ glance(v) ] BlockStor ok [ cinder(v) ] FileStor ok [ manila(v) ] ObjectStor ok [ swift(v) ] Orchestration ok [ heat(v) ] LBaaS ok [ octavia(v) ] DNSaaS ok [ designate(v) ] K8SaaS ok [ k3s(v) rancher(v) ] InstanceHa ok [ masakari(v) ] DisasterRecovery ok [ freezer(v) ] BusinessLogic ok [ mistral(v) murano(v) cloudkitty(v) senlin(v) watcher(v) ] ApiManager ok [ tyk(v) redis(v) mongodb(v) ] DataPipe ok [ zookeeper(v) kafka(v) ] Metrics ok [ monasca(v) telegraf(v) grafana(v) ] LogAnalytics ok [ filebeat(v) auditbeat(v) logstash(v) es(v) kibana(v) ] Notifications ok [ influxdb(v) kapacitor(v) ]Important : Ceph#
Please make sure all osd is added to cluster, if there is unused disk please make sure the osd partition is wiped out
Update CubeOS#
sky1> update update 1: local 2: usb 3: server Enter index: 1 1: CUBE_2.3.0_20220707-1705_fd11e121.pkg Enter index: 1 Firmware update will require an appliance reboot. Enter 'YES' to confirm: YES Formatting partition 2 Installing CUBE_2.3.0_20220707-1705_fd11e121 Running install script Installing postinstall script Finished updating. Please reboot appliance.Reboot#
sky1:> rebootEnter 'YES' to confirm: YESConnection to 192.168.100.100 closed by remote host.Connection to 192.168.100.100 closed.Run bootstrap_cube#
Welcome to the Cube ApplianceEnter "help" for a list of available commandssky1> bootsky1:boot> bootstrap_cubebootstraping cube...bootstrap successfullyif this is a single node start, just run "boot> cluster_sync"if this is a cluster start, wait until bootstrap_done is done in all nodes and run "cluster> start" to sync cluster data for all nodesRun cluster_sync#
sky1:boot> cluster_synccluster_sync successfullyCheck CubeOS version#
sky1:firmware> list 1: CUBE_2.2.4_20220616-2227_cc7bf129 Firmware Version: Cube Appliance 2.2.3 Installation Date: Jul 8, 2022 10:39:15 AM Installation Type: Re-install USB Last Boot: Jul 8, 2022 11:43:10 AM Comment: 2: CUBE_2.3.0_20220707-1705_fd11e121 [ACTIVE] Firmware Version: Cube Appliance 2.2.4 Installation Date: Jul 8, 2022 02:59:11 PM Installation Type: Upgrade Last Boot: Jul 8, 2022 03:07:47 PM Comment: sky1:firmware>Run Service check & repair#
sky1> cluster check_repair Service Status Report ClusterLink ok [ link(v) clock(v) dns(v) ] ClusterSys ok [ bootstrap(v) license(v) ] ClusterSettings ok [ etcd(v) ] HaCluster FIXING [ hacluster(3) ] ok [ hacluster(f) ] MsgQueue ok [ rabbitmq(v) ] IaasDb ok [ mysql(v) ] VirtualIp ok [ vip(v) haproxy_ha(v) ] Storage ok [ ceph(v) ceph_mon(v) ceph_mgr(v) ceph_mds(v) ceph_osd(v) ceph_rgw(v) rbd_target(v) ] ApiService ok [ haproxy(v) httpd(v) lmi(v) memcache(v) ] SingleSignOn ok [ keycloak(v) ] Compute ok [ nova(v) ] Baremetal ok [ ironic(v) ] Network ok [ neutron(v) ] Image ok [ glance(v) ] BlockStor ok [ cinder(v) ] FileStor ok [ manila(v) ] ObjectStor ok [ swift(v) ] Orchestration ok [ heat(v) ] LBaaS ok [ octavia(v) ] DNSaaS ok [ designate(v) ] K8SaaS ok [ k3s(v) rancher(v) ] InstanceHa ok [ masakari(v) ] DisasterRecovery ok [ freezer(v) ] BusinessLogic ok [ mistral(v) murano(v) cloudkitty(v) senlin(v) watcher(v) ] ApiManager ok [ tyk(v) redis(v) mongodb(v) ] DataPipe ok [ zookeeper(v) kafka(v) ] Metrics ok [ monasca(v) telegraf(v) grafana(v) ] LogAnalytics ok [ filebeat(v) auditbeat(v) logstash(v) es(v) kibana(v) ] Notifications ok [ influxdb(v) kapacitor(v) ]sky1:cluster>