Split ZFS in two at two locations?
https://lemmy.nowsci.com/post/12710922
=> More informations about this toot | More toots from fmstrat@lemmy.nowsci.com
I can’t think of a way off hand to match your scenario, but Ive heard ideas suggested that come close. This is exactly the type of question you should ask at practicalzfs.com.
If you don’t know it, that’s Jim Salter’s forum (author of sanoid and syncoid) and there are some sharp ZFS experts hanging out there.
=> More informations about this toot | More toots from tvcvt@lemmy.ml
Great, thank you.
=> More informations about this toot | More toots from fmstrat@lemmy.nowsci.com
what is your goal with this?
do you still want to keep all the data in a single pool?
if so, you could make datasets in the pool, and move the top directories into the datasets. datasets are basically dirs that can have special settings on how they are handled
=> More informations about this toot | More toots from ReversalHatchery@beehaw.org
Yea your edit is the problem unfortunately. Moving across datasets would incur disk reads/writes and sending of terabytes of data.
The goal in separating them out is because I want to be able to independently zfs send folder 1 somewhere without including folder 2. Poor choice of dataset layout when I built the array.
=> More informations about this toot | More toots from fmstrat@lemmy.nowsci.com
hmm I see. and why do you want that? balancing storage usage between backup sites? one of them is too little for the whole pool?
for now I don’t have a better idea, sorry. maybe this is the second best time to think up a structure for the datasets, and move everything into it.
but if the reason is the latter, one backup site cant hold the whole pool, you may need to reorganize it again in the future. and that’s not an easy thing, because now you’ll have the same data (files of the same category) scattered around the FS tree even locally. maybe you could ease that with something like mergerfs, and having it write each file to the dataset with lower storage usage.
if you are ready to reorganize, think about what kinds (and subkinds) of files will you be likely to store in a larger amount, like media/video, media/image, and don’t forget to take advantage of per-dataset storage settings, like for compression, recordsize, maybe caching. not everything needs its own custom recordsize, but for contiguously read files a higher value might be better, also if its not too often accessed and want better compression ratio as compression (and checksumming!) happens per records. video is sometimes compressible, or rather some larger data blob inside the container
=> More informations about this toot | More toots from ReversalHatchery@beehaw.org This content has been proxied by September (ba2dc).Proxy Information
text/gemini