A pool is a logical group for storing objects. It holds a collection of objects,
known as Placement Groups (PG
, pg_num
).
You can create and edit pools from the command line or the web-interface of any Proxmox VE host under Ceph → Pools.
When no options are given, we set a default of 128 PGs, a size of 3 replicas and a min_size of 2 replicas, to ensure no data loss occurs if any OSD fails.
Do not set a min_size of 1. A replicated pool with min_size of 1 allows I/O on an object when it has only 1 replica, which could lead to data loss, incomplete PGs or unfound objects.
It is advised that you either enable the PG-Autoscaler or calculate the PG number based on your setup. You can find the formula and the PG calculator [17] online. From Ceph Nautilus onward, you can change the number of PGs [18] after the setup.
The PG autoscaler [19] can
automatically scale the PG count for a pool in the background. Setting the
Target Size
or Target Ratio
advanced parameters helps the PG-Autoscaler to
make better decisions.
Example for creating a pool over the CLI.
pveceph pool create <name> --add_storages
If you would also like to automatically define a storage for your pool, keep the ‘Add as Storage’ checkbox checked in the web-interface, or use the command line option --add_storages at pool creation.
The following options are available on pool creation, and partially also when editing a pool.
3
.
warn
, it produces a warning message when a pool
has a non-optimal PG count. Default: warn
.
true
(only visible on creation).
Advanced Options
2
.
128
.
target size
if both are set.
Further information on Ceph pool handling can be found in the Ceph pool operation [20] manual.
To destroy a pool via the GUI, select a node in the tree view and go to the Ceph → Pools panel. Select the pool to destroy and click the Destroy button. To confirm the destruction of the pool, you need to enter the pool name.
Run the following command to destroy a pool. Specify the -remove_storages to also remove the associated storage.
pveceph pool destroy <name>
Pool deletion runs in the background and can take some time. You will notice the data usage in the cluster decreasing throughout this process.
The PG autoscaler allows the cluster to consider the amount of (expected) data stored in each pool and to choose the appropriate pg_num values automatically. It is available since Ceph Nautilus.
You may need to activate the PG autoscaler module before adjustments can take effect.
ceph mgr module enable pg_autoscaler
The autoscaler is configured on a per pool basis and has the following modes:
warn |
A health warning is issued if the suggested |
on |
The |
off |
No automatic |
The scaling factor can be adjusted to facilitate future data storage with the
target_size
, target_size_ratio
and the pg_num_min
options.
By default, the autoscaler considers tuning the PG count of a pool if it is off by a factor of 3. This will lead to a considerable shift in data placement and might introduce a high load on the cluster.
You can find a more in-depth introduction to the PG autoscaler on Ceph’s Blog - New in Nautilus: PG merging and autotuning.
[18] Placement Groups https://docs.ceph.com/en/nautilus/rados/operations/placement-groups/
[19] Automated Scaling https://docs.ceph.com/en/nautilus/rados/operations/placement-groups/#automated-scaling
[20] Ceph pool operation https://docs.ceph.com/en/nautilus/rados/operations/pools/