Skip to content
Advertisement

Decrease in Random read IOPs on NVME SSD if requests issued over small region

(TL;DR) On NVME SSDs (Intel p3600 as well as Avant), I am seeing decrease in the IOPS if I issue random reads over a small subset of the disk instead of the entire disk.

While reading the same offset over and over, the IOPS are about 36-40K for 4k blocksize. The IOPS gradually increase as I grow the region over which random reads are being issued. The program (seen below) uses asynchronous IO on Linux to submit the read requests.

JavaScript

OS : Linux 4.2.0-35-generic

SSD : Intel P3600 NVME Flash

What could be causing this problem ?

The program can be run as follows

JavaScript

and validate if you also see the increasing pattern of IOPS seen above

JavaScript

Advertisement

Answer

Surely it is to do with the structure of the memory. Internally this drive is built from many memory chips and may have multiple memory buses internally. If you do requests across a small range all the requests will resolve to a single or few chips and will have to be queued. If you access across the whole device then the multiple request are across many internal chips and buses and can be run asynchronously so will provide more throughput.

User contributions licensed under: CC BY-SA
5 People found this is helpful
Advertisement