EDIT: SOLUTION:
Nevermind, I am an idiot. As @ClickyMcTicker pointed out, it’s the client side that is causing the trouble. His comment gave me thought so I checked my testing procedure again. Turns out that, completely by accident, everytime I copied files to the LVM-based NAS, I used the SSD on my PC as the source. In contrast, everytime I copied to the ZFS-based NAS, I used my hard drive as the source. I did that about 10 times. Everything is fine now. Maybe this can help some other dumbass like me in the futere. Thanks everyone!
Hello there.
I’m trying to setup a NAS on Proxmox. For storage, I’m using a single Samsung Evo 870 with 2TB (backups will be done anyway, no need for RAID). In order to do this, I setup a Debian 12 container, installed Cockpit and the tools needed to share via SMB. I set everything up and transfered some files: about 150mb/s with huge fluctuations. Not great, not terrible. Iperf reaches around 2.25Gbit/s, so something is off. Let’s do some testing. I started with the filesystem. This whole setup is for testing anyway.
- Storage via creating a directory with EXT4, then adding a mount point to the container. This is what gave me those speeds mentioned above. Okay, not good. –> 150mb/s, speed fluctuates
- a Let’s do ZFS, which I want to use anyway. I created a ZFS pool with ashift=12, atime=off, compression=lz4, xattr=sa and 1MB record size. I did “some” research and this is what I came up with, please correct me. Mount to container, and go. –> 170mb/s, stable speed
- b Tried OpenMediaVault and used EXT4 with ZFS as base for the VM-Drive. –> around 200mb/s
- LVM-Thin using Proxmox GUI, then mount to container. –> 270mb/s, which is pretty much what I’m reaching with Iperf.
So where is my mistake when using ZFS? Disable compression? A different record size? Any help would be appreciated.
You must log in or register to comment.
I had to learn the hard way that SMB is single threaded, meaning that you’re probably butting up against one core of your system. I bet if you look at who is sending the data you’ll see one core is pinned at 100%
I don’t think it’s the CPU as I am able to reach max speed, just not using ZFS…
For storage, I’m using a single Samsung Evo 870 with 2TB (backups will be done anyway, no need for RAID). In order to do this, I setup a Debian 12 container, installed Cockpit and the tools needed to share via SMB.
I wonder who owns the storage harddisk: the proxmox host or the Debian VM?
Maybe that’s worth another try in your test setup: pass this physical device to the Debian VM, in order to eliminate possible losses from the virtualization.
https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)
The disk is owned by to PVE host and then given to the container (not a VM) as a mount point. I could use PCIe passthrough, sure, but using a container seems to be the more efficient way.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters LVM (Linux) Logical Volume Manager for filesystem mapping NAS Network-Attached Storage PCIe Peripheral Component Interconnect Express RAID Redundant Array of Independent Disks for mass storage SATA Serial AT Attachment interface for mass storage SSD Solid State Drive mass storage ZFS Solaris/Linux filesystem focusing on data integrity
7 acronyms in this thread; the most compressed thread commented on today has 13 acronyms.
[Thread #355 for this sub, first seen 14th Dec 2023, 21:45] [FAQ] [Full list] [Contact] [Source code]
