On Wed, 2024-10-16 at 17:06 +0300, Onni Hakala wrote:
> I have a large dataset of > 100TB which would be very expensive to store solely into SSD drives.
>
> I have access to a server which has 2x 3.84TB NVME SSD disks and large array of HDD drives 8 x 22TB.
>
> Most of the data that I have in my dataset is very rarely accessed and is stored only for archival purposes.
>
> What would be the de-facto way to use both SSD and HDD together in a way use where commonly used
> data would be fast to access and old data would eventually only be stored in compressed format in the HDDs?
>
> I was initially looking into building zpool using zfs with raidz3 and zstd compression for my HDDs
> but I’m unsure how to add the SSDs into this equation and I thought that this is probably a common
> scenario and wanted to ask opinions from here.
The traditionalway to handle that would be to create an extra tablespace on a file system built
on the slow disks. You'd move tables that are no longer hot to that slow tablespace.
If you cannot move the whole table, use partitioning and move old partitions to the slow
tablespace.
Yours,
Laurenz Albe
