full image - Repost: What is ZIL and how does it affect Write Coalescing performance (from Reddit.com, What is ZIL and how does it affect Write Coalescing performance)
Mining:
Exchanges:
Donations:
The short answer is ZIL SSD can help Write coalescing happen much faster for Synchronous writes.If you don’t do synchronous writes, ZIL speed should not matter. In most cases, if you use SMB to map network drives, ZIL speed should not matter because SMB is usually used for Asynchronous writes rather than Synchronous writes.ZIL is often thought of as being like standard write Cache, but it works differently. Standard Write cache can add some write performance at the cost of increasing risk to your data (this risk can be mitigated by a UPS but not eliminated). ZIL is there to lower the performance cost of synchronous writes. And Synchronous writes are a kind of write that offers a higher level of data protection.So you can think of standard write cache as a way to trade some data safety for more performance. And ZIL is a way lower the performance cost when you trade some performance for more data safety.Write Coalescing:If you do an Asynchronous write, ZIL is not used. If you have many small writes, you write them to the RAM. When enough small writes build up in the RAM, the small writes are combined or Coalesced into larger writes. Then the larger writes are written to the drives, and because they are in larger chunks now, they are written more sequentially, which can greatly increase the write speed when writing to a RAID of drives.But the issue with holding small writes in the RAM to combine into larger groups is that if there is a power outage, the data in the RAM is lost. RAM is not persistent storage.If you are using your NAS as a file server through SMB, losing the data in the RAM from a power outage may not be so terrible because the NAS should know the file was lost. The NAS can request the file to be sent again. Corruption should be avoidable.But in some cases, with iSCSI for example, the NAS may not know where 1 file ends and the next begins. If part of a file is lost in the RAM, the NAS may not know how to request the file to be sent again because the NAS may not be aware of files in that iSCSI LUN. And in some cases, corruption can be the result.To prevent data corruption in such situations, Synchronous writes can be required by the application.A Synchronous write adds the requirement that you can’t send the next block until the first block was fully written to “Persistent Storage” (meaning you can lose power to the NAS and the data will “Persist”).What does this have to do with Write Coalescing?Write Coalescing still happens with Synchronous writes, but slower.A block is written to the RAM and to ZIL space at the same time. But this time the next block won’t be sent to the RAM until the first block was completed to the ZIL space in persistent storage. When the first block is in the ZIL, then next block is sent to the RAM and the ZIL at the same time. Again the RAM waits for the ZIL write to be complete. This keeps happening until there is enough data in the RAM to combine to a larger chunk it can write to the RAID group.If the ZIL space is fast enough, the time the RAM waits on the ZIL can be small. It is possible for Synchronous writes to be almost as fast as Asynchronous writes if the ZIL is fast enough.But the worst-case scenario is if you do a Synchronous write to an HDD pool while the ZIL space is on the HDD pool as well. If you write a block to an HDD and don’t even send the next block to the RAM before the first block is on the HDD, that can be quite slow.But if you have an SSD for ZIL, this can be much faster.If you have an SSD with power loss protection, this can be even faster because even the DRAM cache on the SSD can often be considered “persistent” if the SSD has power loss protection. And if you can write a block to the DRAM of an SSD, and then request the next block, the low latency of DRAM can help make ZIL fast.A great ZIL on QuTS Hero NAS is likely Intel Optane SSDs.Optane drives may cost more per TB, but ZIL can be very small and still work well. And a small Optane drive need not cost too much.Our Dual controller NAS have an even faster ZIL; NVRAM with battery backup and SSD copy to flash. What that means is the RAM has a portion of it that has battery backup and a feature to automatically copy its data to an SSD if the NAS loses power. The battery backup only needs to keep the RAM going for a short time while it copies its contents to the SSD. This is similar in principle to power loss protection on an SSD where the SSD still has power for a short time after the NAS loses power so the SSD’s DRAM cache can copy to the NAND flash.Some things to consider:Should you put L2ARC and ZIL on the same SSDs?With L2ARC much of what you read gets written to the SSD so it can be cached. The massive amounts of data that can be written to L2ARC SSDs can make ZIL have less performance if the same SSDs are used for ZIL. If an SSD write speed is limited, the more you use up that write speed with writes to L2ARC, the less of that write speed is left for the ZIL.Also, L2ARC SSD can likely wear out faster because of how much data is written to them, and you don’t want to lose your ZIL space, as that can be dangerous to your data. Our ZIL is kept in RAID1 so 1 SSD failure should not cause you to lose the ZIL data. You can decide how worried you are about the possibility of both ZIL SSDs failing at the same time.What if you don’t have enough SSDs for ZIL, SSD System pool, and L2ARC?The SSD system pool is usually important to performance. So I tend to prioritize that first.Because QuTS Hero uses RAM ARC cache, the SSD L2ARC cache is often not as important. In many cases, you can do without it without losing much performance.There are also many cases where you don’t need fast ZIL because you don’t use the ZIL space.In most cases, if you use SMB you are doing Asynchronous writes and not using ZIL.But NFS is often Synchronous by default. iSCSI is often synchronous or a mixture of Asynchronous and Synchronous. So, in those cases, having a low latency ZIL can be important for good performance. You can then consider adding a QM2 card with 2 Optane M.2 SSDs or using Fast NVMe SSDs with power loss protection.
Social Media Icons