Version: 3.0

Add Hard Disks

Add a new hard disk to storage pool.

Accessing desired Node via SSH

Connect to the console of the node having a new hard disk.

ssh [email protected]

Warning: Permanently added '192.168.1.x' (ECDSA) to the list of known hosts.
Password:

Verify storage status before performing

Use command storage > status.

Demonstration shows the cluster has 16 osds.

6 for cc1, 4 for cc2, 6 for cc3

cc2> storage status
    cluster:
      id:     c6e64c49-09cf-463b-9d1c-b6645b4b3b85
      health: HEALTH_OK

    services:
      mon: 3 daemons, quorum cc1,cc2,cc3 (age 24h)
      mgr: cc1(active, since 24h), standbys: cc3, cc2
      mds: 1/1 daemons up, 1 standby, 1 hot standby
      osd: 16 osds: 16 up (since 18h), 16 in (since 18h)
      rgw: 3 daemons active (3 hosts, 1 zones)

    data:
      volumes: 1/1 healthy
      pools:   25 pools, 945 pgs
      objects: 90.07k objects, 458 GiB
      usage:   1.3 TiB used, 4.5 TiB / 5.8 TiB avail
      pgs:     945 active+clean

    io:
      client:   101 KiB/s rd, 522 KiB/s wr, 24 op/s rd, 82 op/s wr

  ID  HOST   USED  AVAIL  WR OPS  WR DATA  RD OPS  RD DATA  STATE
  0  cc1   55.4G   316G      9     52.7k      0        0   exists,up
  1  cc1   91.9G   280G      9     39.1k      1        0   exists,up
  2  cc1   70.9G   301G      9     52.0k      1        0   exists,up
  3  cc1   64.9G   307G      1     15.1k      0        0   exists,up
  4  cc1   67.5G   304G      1     16.7k      1        0   exists,up
  5  cc1   72.6G   299G      4      133k      1     5824   exists,up
  8  cc2   94.3G   277G      0     6553       1        0   exists,up
  9  cc2    115G   256G      2     7782       0        0   exists,up
  10  cc2    107G   264G      1     8192       0        0   exists,up
  11  cc2   95.1G   277G      5     28.8k      1        0   exists,up
  12  cc3   88.1G   284G      2     10.3k      1        0   exists,up
  13  cc3   59.2G   313G      0        0       0        0   exists,up
  14  cc3   70.5G   301G     14     68.0k      5      107k  exists,up
  15  cc3   64.4G   307G      1     28.0k      3     22.4k  exists,up
  16  cc3   70.4G   301G      0        0       1       16   exists,up
  17  cc3   83.9G   288G      7     36.7k      0        0   exists,up

List all disks avilable for adding

Use command storage > list_avil.

cc2> storage list_avail
    index          name      size  on_hours     error              serial
  --
        1      /dev/sdb    745.2G     69618         0  BTWA632602Q3800HGN

  --
  Found 1 available disk.

Add a New Disk with Raw mode

Use command storage > add_disk.

cc2> storage add_disk
    index          name      size  on_hours     error              serial
  --
        1      /dev/sdb    745.2G     69618         0  BTWA632602Q3800HGN

  --
  Found 1 available disk
  Enter the index to add this disk into the pool: 1
  Disk protection mode:
  1: raw
  2: encrypt
  Enter index: 1
  No disk encryption (default mode).
  Enter 'YES' to confirm: YES
  Added disk(raw) /dev/sdb.

Verify storage status

The Cluster will provision two new OSDs to the new disk. The process would take some time. Verify by using command storage > status.

Demonstration shows the cluster has 16 + 2 (new) osds.

6 for cc1, 6 for cc2, 6 for cc3

cc1> storage status
    cluster:
      id:     c6e64c49-09cf-463b-9d1c-b6645b4b3b85
      health: HEALTH_OK

    services:
      mon: 3 daemons, quorum cc1,cc2,cc3 (age 2h)
      mgr: cc1(active, since 2h), standbys: cc3, cc2
      mds: 1/1 daemons up, 1 standby, 1 hot standby
      osd: 18 osds: 18 up (since 29s), 18 in (since 29s)
      rgw: 3 daemons active (3 hosts, 1 zones)

    data:
      volumes: 1/1 healthy
      pools:   25 pools, 945 pgs
      objects: 87.83k objects, 449 GiB
      usage:   1.3 TiB used, 5.2 TiB / 6.5 TiB avail
      pgs:     1418/251600 objects degraded (0.564%)
              35909/251600 objects misplaced (14.272%)
              783 active+clean
              104 active+remapped+backfill_wait
              52  active+remapped+backfilling
              6   active+undersized+degraded+remapped+backfilling

    io:
      client:   1.3 MiB/s rd, 494 KiB/s wr, 59 op/s rd, 62 op/s wr
      recovery: 858 MiB/s, 1 keys/s, 161 objects/s

  ID  HOST   USED  AVAIL  WR OPS  WR DATA  RD OPS  RD DATA  STATE
  0  cc1   47.3G   324G      7     49.5k      0        0   exists,up
  1  cc1   89.2G   282G      3     13.5k      7      193k  exists,up
  2  cc1   77.1G   295G     10     74.3k      5      164k  exists,up
  3  cc1   69.2G   302G      1     9216       0        2   exists,up
  4  cc1   67.2G   304G      7     31.1k      2     2459   exists,up
  5  cc1   67.0G   305G      1     28.7k      1     2547   exists,up
  6  cc2   5668M   366G      0        0       0        0   exists,up
  7  cc2   5575M   366G      0        0       0        0   exists,up
  8  cc2    100G   272G      0        0       1        0   exists,up
  9  cc2    112G   259G      7     32.7k      4      120k  exists,up
  10  cc2    104G   267G      1     15.1k      1     1638   exists,up
  11  cc2   88.2G   283G      0      819       2     28.8k  exists,up
  12  cc3   89.3G   282G      0        0       4     62.4k  exists,up
  13  cc3   52.0G   320G      4     20.0k      2     71.2k  exists,up
  14  cc3   61.8G   310G      4     44.0k     26      824k  exists,up
  15  cc3   53.0G   319G      0     1638       2        0   exists,up
  16  cc3   69.4G   302G      0        0       1        0   exists,up
  17  cc3   87.9G   284G      4     60.0k      2     37.6k  exists,up

Add a New Disk with Encrypt mode

Similar to Add a New Disk with Raw mode, also use the command storage > add_disk to perform.

info

Disks added with raw mode would each have 2 OSDs.

On the other hand, disks added with encrypt mode would each have only 1 OSD.

cc2>　storage add_disk
  index          name      size
--
      1      /dev/sdj      3.7T
--
Found 1 available disk
Enter the index to add this disk into the pool: 1
Disk protection mode:
1: raw
2: encrypt
Enter index: 2
Encrypt disk(s) to protect physical disk loss (beware of performance impacts).
Enter 'YES' to confirm: YES
Added disk(encrypt) /dev/sdj.

Add all Available Disks on a Node

Use command storage > add_avail to perform.

Two modes of adding disks works same as single disk addition.

cc2> storage add_avail
    index          name      size  on_hours     error              serial
  --
        1      /dev/sdb    745.2G     69618         0  BTWA632602Q3800HGN

  --
  Disk protection mode:
  1: raw
  2: encrypt
  Enter index: 1
  No disk encryption (default mode).
  Enter 'YES' to confirm: YES
  --
  Added disk(raw) /dev/sdb.
  --
  Processed 1 disk out of 1.

Accessing desired Node via SSH​

Verify storage status before performing​

List all disks avilable for adding​

Add a New Disk with Raw mode​

Verify storage status​

Add a New Disk with Encrypt mode​

Add all Available Disks on a Node​