full image - Repost: Help Struggling with Storage/RAID options and performance (from Reddit.com, Help Struggling with Storage/RAID options and performance)
Mining:
Exchanges:
Donations:
For full disclosure, I originally posted this to /r/proxmox yesterday but didn't get any answers. Maybe my post was too long (I tried to make it as brief as possible) or maybe it indicated I'm just too stupid to help? Either way, I'm hoping to have better luck here.I've been trying to plan out a new proxmox cluster for a few weeks now and, try as I might, I just can't wrap my head around the multitude of storage options. I'm sorry for the length of this, I've tried to organize things as well as I can.My Goals:An HA cluster for redundancy so I can start self hosting all of my important stuff.I also do a lot of homelab stuff and I want speed and performance for spinning up new VMs to try things out (think cloning an Ubuntu VM template, for example). Basically I want to be able to spin up and tear down linux VMs as quickly as possible.Reliable/fault-tolerant storage for Media (Plex etc), photos, backups etc (RAID 6 or Raidz2).Use 4TB SAS drives from ebay (open to other suggestions) to help keep costs lowMy Hardware2 x R620s, I would like the one with a faster CPU to be the primary member (2 votes) and the slower one to the be the secondary (only 1 vote). Each of these have a 10gbe nic, currently I have not attempted to flash the 710 mini RAID card2 x Supermicro disk shelves - SYS-937R-E2JB (16 disks each)One with just a compute module (dual xeon, 24 gigs ram)One with just a JBOD module.These don't support SATA drives without some interposer card, and I can't find it in stock anywhere. For this reason, I don't think SSDs are practical here (SAS SSD seems pretty expensive still, even on ebay)Custom built PC currently running Unraid, fast CPU (i9 @ 4 ghz, 16 gigs ram) and 10 gig NIC. Would consider repurposing this to a high speed NAS/SAN running TrueNAS with SSDsAs best I can figure, I have a few options for basic hardware setups...I'm hoping you guys can help me understand if any one of these would be the clear choice to proceed with based on my goals above.Option1: Run TrueNas on compute module of Supermicro disk shelf, connect it to the other disk shelf as JBOD (32 drives total), serve everything via 10 gig networkingSetup 2 x Z-Pools1 "high speed" (3 (or 4) x mirror vdev? I don't fully understand the terminology, but basically raid 10 vdevs)? Basically, whatever would give me the best setup to host all of the vm disks on from the R620s1 Z-Pool for bulk storage, made up of raidz2 z-pools for storing MediaOption 2: Use the onboard raid controllers and setup RAID 10 on each R620 for VM disksIdeally find some fast (10K) HDDs and load things upUse GlusterFS to keep things in sync between the twoSetup truenas on the Supermicro disk shelves and just use that whole thing for bulk storageOption 3: Repurpose PC for high end NAS/SAN for storing VM disksPut something like 6 x SSDs into PC and setup RAID 10 (or zfs equivalent)Run TrueNAS and share things out via iSCSI or NFSSetup truenas on the Supermicro disk shelves and just use that whole thing for bulk storageOption 4: Connect disk shelves directly to R620s?I don't know if this is possible since one of the shelves doesn't have a jbod module (just the compute)? If I was to pursue this option, I assume both R620s would need access to the disk shelves, and I'm not sure I can make that happen? I think I'd need to find a second JBOD module so both shelves have one?If you've made it this far, thank you for taking the time to try and help me out. I'd like to figure out if anyone of these is the clear choice for moving forward with. I then have a lot to learn about ZFS and optimizing my setup for speed and redundancy, but that's a different post for a different day.
Social Media Icons