# Remove OSD from Storage Pool

> Scenario: One of the nodes has failed to power up with no reason. The OSDs that host by the failed node went offline, so we have to recover the storage pool ASAP from the `health_warn` status. We could do it from any host of the cluster.

Scenario: One of the nodes has failed to power up with no reason. The OSDs that host by the failed node went offline, so we have to recover the storage pool ASAP from the `health_warn` status. We could do it from any host of the cluster.

## Accessing desired Node via SSH {#accessing-desired-node-via-ssh}

```bash
ssh admin@192.168.1.x

Warning: Permanently added '192.168.1.x' (ECDSA) to the list of known hosts.
Password:
```

## Check Storage Status {#check-storage-status}

Use command `storage status`.

As shown below, we have 1 OSD down due to a failed hard disk, OSD No.7 on node hostname (cc2).

The health status is showing `HEALTH_WARN`.

```bash
cc1> storage status
    cluster:
      id:     c6e64c49-09cf-463b-9d1c-b6645b4b3b85
      health: HEALTH_WARN
            1 osds down
            Degraded data redundancy: 1418/251600 objects degraded (0.564%), 72 pgs degraded

    services:
      mon: 3 daemons, quorum cc1,cc2,cc3 (age 101m)
      mgr: cc1(active, since 101m), standbys: cc3, cc2
      mds: 1/1 daemons up, 1 standby, 1 hot standby
      osd: 18 osds: 17 up (since 60s), 18 in (since 2m)
      rgw: 3 daemons active (3 hosts, 1 zones)

    data:
      pools:   25 pools, 945 pgs
      objects: 87.83k objects, 449 GiB
      usage:   1.2 TiB used, 5.3 TiB / 6.5 TiB avail
      pgs:     1418/251600 objects degraded (0.564%)
              728 active+clean
              145  active+undersized
              72  active+undersized+degraded
    io:
      client:   61 KiB/s rd, 503 KiB/s wr, 18 op/s rd, 76 op/s wr

  ID  HOST   USED  AVAIL  WR OPS  WR DATA  RD OPS  RD DATA  STATE
  0  cc1   46.6G   325G     15      376k      0        0   exists,up
  1  cc1   85.7G   286G      4     23.2k      1       20   exists,up
  2  cc1   75.4G   296G     16     72.7k      1        0   exists,up
  3  cc1   70.8G   301G      5     60.7k      0        0   exists,up
  4  cc1   62.5G   309G      7     54.3k      1        0   exists,up
  5  cc1   66.9G   305G      0        0       1       90   exists,up
  6  cc2   73.2G   298G      6     46.3k      0        0   exists,up
  7  cc2   5575M   366G      0        0       0        0   exists
  8  cc2   77.5G   294G      0     5734       1        0   exists,up
  9  cc2   97.4G   274G     13      136k      0        0   exists,up
  10  cc2   80.5G   291G      6     55.1k      0        0   exists,up
  11  cc2   68.4G   303G      5     39.1k      1        0   exists,up
  12  cc3   84.1G   288G      0        0       1        0   exists,up
  13  cc3   52.1G   320G      1     8210       1     48.0k  exists,up
  14  cc3   62.8G   309G     13      116k      2     44.0k  exists,up
  15  cc3   51.1G   321G      5     56.7k      2        0   exists,up
  16  cc3   69.0G   303G      3     37.7k      2      135   exists,up
  17  cc3   87.1G   285G      8      372k      0        0   exists,up
```

## Remove OSDs {#remove-osds}

Use command `storage remove_osd` to remove down OSDs.

```bash
cc1> storage remove_osd
  Enter osd id to be removed:
  1: osd.7
  Enter index: 1
  Enter 'YES' to confirm: YES
cc1>
```

## Verify Storage Status {#verify-storage-status}

After removing failed OSDs, check the storage health with command `storage status`

The health status should be showing `HEALTH_OK`.

```bash
cc1> storage status
    cluster:
      id:     c6e64c49-09cf-463b-9d1c-b6645b4b3b85
      health: HEALTH_OK

    services:
      mon: 3 daemons, quorum cc1,cc2,cc3 (age 93m)
      mgr: cc1(active, since 92m), standbys: cc3, cc2
      mds: 1/1 daemons up, 1 standby, 1 hot standby
      osd: 17 osds: 17 up (since 60s), 17 in (since 2m); 10 remapped pgs
      rgw: 3 daemons active (3 hosts, 1 zones)

    data:
      volumes: 1/1 healthy
      pools:   25 pools, 945 pgs
      objects: 87.82k objects, 449 GiB
      usage:   1.2 TiB used, 5.0 TiB / 6.2 TiB avail
      pgs:     1250/251643 objects misplaced (0.497%)
              935 active+clean
              10  active+remapped+backfilling

    io:
      client:   340 KiB/s rd, 428 KiB/s wr, 49 op/s rd, 74 op/s wr
      recovery: 151 MiB/s, 21 objects/s

  ID  HOST   USED  AVAIL  WR OPS  WR DATA  RD OPS  RD DATA  STATE
  0  cc1   46.6G   325G     15      376k      0        0   exists,up
  1  cc1   85.7G   286G      4     23.2k      1       20   exists,up
  2  cc1   75.4G   296G     16     72.7k      1        0   exists,up
  3  cc1   70.8G   301G      5     60.7k      0        0   exists,up
  4  cc1   62.5G   309G      7     54.3k      1        0   exists,up
  5  cc1   66.9G   305G      0        0       1       90   exists,up
  6  cc2   73.2G   298G      6     46.3k      0        0   exists,up
  8  cc2   77.5G   294G      0     5734       1        0   exists,up
  9  cc2   97.4G   274G     13      136k      0        0   exists,up
  10  cc2   80.5G   291G      6     55.1k      0        0   exists,up
  11  cc2   68.4G   303G      5     39.1k      1        0   exists,up
  12  cc3   84.1G   288G      0        0       1        0   exists,up
  13  cc3   52.1G   320G      1     8210       1     48.0k  exists,up
  14  cc3   62.8G   309G     13      116k      2     44.0k  exists,up
  15  cc3   51.1G   321G      5     56.7k      2        0   exists,up
  16  cc3   69.0G   303G      3     37.7k      2      135   exists,up
  17  cc3   87.1G   285G      8      372k      0        0   exists,up
```
