Thanks for the feedback guys

Haven't looked into latency, but right now I've tested pinging my router and it takes 0.5ms (so 500 microseconds).
Found on Wiki about 10GbE: "10GBASE-T has latency in the range 2 to 4 microseconds compared to 1 to 12 microseconds on 1000BASE-T".
Here, it says that 10GbE has got a mean latency of 9usec (http://www.extremenetworks.com/librar ... _Network_Applications.pdf)
Here, it says that GbE (in full-duplex mode) has a latency on 0.5usec with a 64B packet (http://www.datalinkcom.net/How%20Fast ... %20Gigabit%20Ethernet.pdf).
Finally, here it says about 50-125usec for GbE and 5-50 for 10GbE when using TCP/IP (http://www.qlogic.com/Resources/Docum ... n_to_Ethernet_Latency.pdf).

I guess using full-duplex, offloading the load of the NIC (and not the CPU) and using Jumbo-Frames can help a bit with the latency issue. If ever there's any delay, I think the server or the client will adapt itself and wait until the working data set is processed before taking another one.

I could also switch to Fibre Channel but the max speed is 8Gbit per SFP slot. :/ That way I get a little lest bandwidth but light-speed latency.

Hi guys


As I've said on Bean's NAS topic, I'm going to build one for myself and it has to be upgradeable and it must have the very best networking solution.


I've decided to go with ZFS and RAIDz2 (some sort of RAID6) for multiple reasons, which you can find here: https://calomel.org/zfs_raid_speed_capacity.html


First of all, the most important: networking. Any NAS will be transferring data back and forth through its network interface. Therefore, considering the fact that ZFS provides two levels of high-speed cache (ARC=RAM, L2ARC=SSD), I need a networking solution that is capable of taking advantage of these. A single Gigabit Ethernet link is useless, same goes for two. I could use two 4GbE NICs but it wastes so many ports on the switch... So I'll be going with 10GbE (in the future, too expensive for now and haven't got enough $$ ). The NAS will mainly be serving my own computer, so a direct 10GbE link between the two is perfect (10GbE switches cost a lot). Two or three Gigabit links will serve other computers, in rare occasions. Each 10GbE card runs at PCI-e 8x so let’s consider one port used for that. I could also use double 10GbE NIC for a 20GbE link, but I have time to think about this.





RAM runs at 1333 9-9-9-27 2T (Rampage III Extreme with 990X and ValueRAM DIMMs, shame on me for those timings), and still 10GbE isn’t enough to fully take advantage of the RAM cache (obviously). But anyway, 1GiB/s is still pretty darn fast. The SSD won’t even use that much bandwidth so that’s fine.


Now let’s get down to storage. As I’ll use 2-3 GbE links for now, while also taking into count a dedicated PCI-E 8x port for the future NIC, I still need another 8x port for the disk controller. This one will provide me eight SATA3 ports, so up to 12TB with RAIDz2, with each drive being 2TB. I believe that’s enough space for a quite a few years. When 12TB becomes not enough, I’ll be switching to newer drives anyway.


So in this regard, I only need two PCI-E 8x ports (16 lanes). No need for quad PCI-E 8x capable motherboards; two 8x ports are enough, some 4x are welcome.To have decent network speeds, I’ll be throwing in a PCI GbE NIC to have 384MiB/s until I get the 10GbE stuff.


With this in mind, I chose the following components:


PSU: Seasonic G-450 => Seasonic rocks, solid power units and Gold certification tops it up
Case: BitFenix Shinobi Core => 8 disk bays, two fans for HDD cooling and good cable management. What else ?
CPU: i3 3220 => ECC support (mandatory) and cheapest 2C/4T (good for compression) socket-1155 processor
MOBO: ASUS P8B WS => Good PCI-E layout, PCI slot, ECC support (mandatory)
RAM: ValueRAM 4GB ECC 1333 => cheapest ECC memory module at 1333 or more
SSD: Kingston SSDNow V300 60GB => cheap SSD used only for caching, no need for 2K$ Intel crap
Disk controller: Highpoint RocketRAID 2720SGL => best option after LSI, decent performance and JBOD
Disks: WD RE 2TB => WD hasn’t failed on me yet, and RE drives are solid
Fans: Noiseblocker XL2 Rev.3 => silent fan with decent airflow


I think that's the best way to go. I'll start with 4 drives, so 4TB usable and 2 drives for parity. I'll be using a 1810G-24 switch from HP (24 ports) with dual GbE and one PCI NIC for the server and three out of four ports from my Intel server NIC for my personal computer. For now, that's enough and a bit faster that my normal Caviar Black.

What do you guys think ?

Mining BTC with GPUs is completely useless now. Only LTC mining is still worth it.

@Bean: I'll do as soon as I get my stuff. First I need to get my money from some website, then I'll be able to pay the actual hardware.

Homemade NASes rock simply because it costs much less than a 10-bay QNAP NAS for example but also because it is highly modular. You can change almost anything in it to achieve what you need. That's why we build custom PCs and we should also build custom NASes.

Quote:

I think I'll order my stuff by the end of the week. Everyday I find something new/to change so I prefer taking my time since I'm putting quite some money on the table. I also have a 24-port switch coming in soon, and a 4GbE NIC from Intel already.

RAID10 wastes 1/2 of your disk space, so when you have a few disks it's fine. When you begin scaling, RAIDz2 become much more efficient (space/$$, reliability, etc).

I'm planning on doing a custom NAS which will run nas4free (ZFS). If you want, you can inspire yourself from my shopping list: Digitec Shoppinglist

The base system costs about 550CHF (PSU, PU, mobo, RAM and CPU). Else is for convenience and better performance. The SSD is for caching and the disk controller provides better performance than the on-board cheap controller by Intel. In the future, you can add up to 2 disk controllers and two 4GbE NIC (so total of 16 disks and 10GbE of network connectivity). Also, I don't really like Red drives so I'd recommend to go with Se at least (if RE are too expensive for you).

EDIT:

What I recommend is RAID10 if you data is not so valuable, and RAID6/RAIDz2 for better reliability.

Quote:

This place is heaven on Earth.

Same for me, nice to see new faces and meet fellow Swiss overclockers. I'll try doing some scores next time.

Crap, Christian's video "Gangnam style" is private.

Yes, a lot of crazy benchers. It will be fun, for sure.