32 replies to this topic

[40H3X]

Does anyone have experience with the filesystems ZFS and UFS (Unix native), be it positive/negative?

 

Thanks for any thoughts

[WanWizard]

We use ZFS in our business.

 

We have two FreeNAS systems with SATA disks (ZFS on FreeBSD) which we use for high-volume, low speed NFS storage, and a ZFS on Linux (CentOS 7) cluster with SAS/15k disks and 10G network for high speed NFS storage which amongst others is used as datastore for our VMWare farm...

 

edit:

 

As to the second part of your question, I find FreeNAS as stable as a rock, ZFSonLinux is lagging a bit behind, but if you know the few bugs that bother you, you can take them into account, and then there are no issues as well. Took us a while to get the cluster stable, but that was more of a linux issue than a ZFS issue.

 

If you're looking for the particular ZFS features like snapshots, you might also have a look at btrfs...

[40H3X]

Thanks for your professional input

 

The main reason why I consider to move over from UFS to ZFS is the "auto" repair,  as the system it concerns has no UPS and power failure might lead to filesystem damage.

[40H3X]

Found this one, big, but really complete...

 

https://en.wikipedia...of_file_systems

[WanWizard]

Ideally, ZFS requires a system with ECC memory, otherwise you're still not 100% safegarded against bit errors...

 

I can't comment on its behaviour under your circumstances, our systems are DC grade: end-to-end redundant (diesel + battery UPS, A and B power feeds in the racks, two power supplies per server, two servers, two HBA's per server, two 10G NIC's to two switches, two disk shelfs, both with two power supplies as well.

[40H3X]

Thanks for pointing that out to me, in my case (FW using APU2) ECC is enabled and in this case it not industrial/enterprise equipment

Edited by 40H3X, 1 November 2019 - 08:25.

[40H3X]

https://blog.3mdeb.c...c-engines-apu2/

 

So moving over from UFS to ZFS, even using HDD with single disk, would be upgrading safety against power surges. I am thinking about

ZFS set copies=2 tank0 

...although some advocate against it https://jrs-s.net/20...opies-equals-n/

 

From what I read TRIM is default enabled (I am using mSATA), nice, as I had some troubles enabling it on UFS.

 

BTW I am considering to move over to FreeNAS when time comes, although Synology is nice, ZFS is not possible as there is no ECC and to less RAM, from what I have read it should be above 2Gb. Than maybe BtrFS on Synology seems nice, but I don't think it will be on home user systems https://www.synology...nl-nl/dsm/Btrfs.

 

...for anyone who wants to read more about it

[WanWizard]

At home, I run Xpenology as a VMware VM, which emulates a DS3615xs, and which does have Btrfs. I think the main reason for not having it available is that the low-end boxes lack the performance to deal with ZFS / Btrfs. 

 

I notice a big difference on the box between my old DS414 and my new setup, I always had to wait until directories on the NAS were loaded, now they are there virtually instantly.  

 

As to your other point, ZFS, like btrfs, is copy-on-write, so power surges are never a problem.

 

The problem with a single disk and "copies" is the same as creating an mdadm raid-1 using two partitions of the same disk: you have data redundancy, but not disk redundancy, as disk failure will cause the loss of both data sets. The question is if that is worth it, given the price difference between lets say 1 x 4TB disk vs 2 x 2TB disks....

[40H3X]

Why do you use XPEnology on a virtual manager and not a bare metal install? Although your install gives you BTrFS, why not use FrreeNAS with ZFS, is it because of the DSM GUI?

[WanWizard]

I've switched to XPEnology because I needed at least 10 disk bays, which means a very expensive system to buy.

 

I've chosen for a VM instead of bare metal because I have quite a few other physical machines that I am going to consolidate on this server. 

 

I currently also use my laptop for development, so it runs nginx, apache, postgres, mariadb, etc. Which means it's high end, expensive, and not really portable. It needs replacing, and I don't want to spend the money.

So I am going to move this to VM's as well, so I can switch to an Ultrabook and work with a remote desktop. The need to be able to work offline isn't really there anymore today.

 

I am indeed not very charmed by the FreeNAS interface, but another reason is that I prefer Linux over FreeBSD, amd "stuff" I run on it, or that needs connecting to it, doesn't really work on FreeBSD.

 

Also, ZFS is allergic for different (size) disk, while Synology's mdadm + lvm layering handles that very well, and ZFS performance reduces when a volume gets fuller. Things I don't want in a home situation.

[40H3X]

Thanks for the insight

 

You prefer Linux over FreeBSD, I understand , I myself get mixed up on the syntax when using the CLI when I switch between Linux and FreeBSD

 

XPEnology seems nice, considering it, as I have 2 x 3TB WD RED lying around and a Intel Core i5-2300CPU 2,80GHZ with 8,00 GB. Also stumbled over a "DS3615xs 6.1 Jun's Mod V1.02-alpha ISO", as you are (much) more experienced on XPE, could you advise on a version, currently using DSM 6.2.2 on a DS215+ and for now I think looking at a bare metal install...

[WanWizard]

That ISO is a 50MB boot disk, which is basically contains the stuff that is in flash in a Synology box, a patched bootloader, and the web interface that allows you to install DSM (from the downloaded .pat file). It is not visible / accessable once DSM is running.  You write the ISO to a USB stick, and boot the server (or in my case the VM) to install DSM.

 

I have DSM 6.2.2-24922 Update 3 at the moment. You do have to be careful with updates, you have to check for compatibility with the patched bootloader before you do.

 

Look here for a 6.2 loader, there are different ones that boot different DS versions, you can also pick a DS918 is you don't need a 12-bays...

[40H3X]

So far no luck, it boots, but no ip is given, looking into it.

[WanWizard]

What do you mean with "no ip given"? 

[40H3X]

I tried to boot from usb (pid, vid, mac and serial edited), but can access it as it has no ip.

[WanWizard]

It should dhcp? You have to be careful with changing the MAC btw, it won't start if it is not correct for the DS model "emulated", it's one of the security features in the DSM bootloader.

[40H3X]

First checked if the NIC, and Intel 82579V, was supported and it was.

 

Then I tried the 6.2 loader for models 918+/3615xs/3617xs, as I read that the usb port used could make a difference, I tried ....all nine of them... resulting in a total of 27 reboots, but none gave an IP. If at first you don't succeed so I moved over to "DS3615xs 6.1 Jun's Mod V1.02-alpha ISO" and bingo an IP. As I was curious why this loader version gave me an IP, I wanted to test the additional compiled modules with 6.2 loader (extra.lzma and extra2.lzma) https://xpenology.co...5xs-3617xs-918/ but no IP.

 

Although I had an IP  with 6.1 I encountered  another error "we've detected errros on the hard drives (3,4) and the sata ports have also been disabled", but that's the next hurdle

Edited by 40H3X, 11 November 2019 - 20:15.

[WanWizard]

The only thing I've noticed is that the MAC set on the grub cmdline differs from the MAC address of the NIC used, the bootloader won't start, after which it's obvious it won't do a DHCP request...

[40H3X]

 
I notice a big difference on the box between my old DS414 and my new setup, I always had to wait until directories on the NAS were loaded, now they are there virtually instantly...

Why is that?

[WanWizard]

Because the new setup is al awful lot faster?