mgr/dashboard: Add cache pool support
Ceph provides a concept of cache pools. It should be possible for an administrator to define such a cache pool and assign it to an existing pool via dashboard:
Use a pool of fast storage devices (probably SSDs) and use it as a cache for an existing slower and larger pool.
Use a replicated pool as a front-end to service most I/O, and destage cold data to a separate erasure coded pool that does not currently (and cannot efficiently) handle the workload.
We should be able to create and add a cache pool to an existing pool of data, and later remove it, without disrupting service or migrating data around.