Cold-cache Sequential IO Benchmark

Discussion in 'backup, imaging & disk mgmt' started by angstrom, Jan 8, 2017.

  1. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    So I was building myself a new rig as a Christmas present. One thing led to another and I ended up writing an little, but very pedant benchmarking tool that measures maximum read/write rate for any given file system location.

    Cold-Cache Sequential IO Benchmark -- https://ccsiobench.com

    Unlike other benchmarking tools it automatically searches for the best combination of IO buffer size and count, and it iterates over several different IO modes (direct, sequential, etc.). In its default setup it runs around 180 different tests, so what comes out is a pretty damn accurate estimation of IO rates.

    You basically tell it "C:\Foo" and it tells you "Maximum read rate is 328.8 MBps if using 8 x 64 KB buffers in direct, sequential mode".

    CCSIO Bench can also be used to test virtual volumes, e.g. TrueCrypt mounts, and network shares. Former is pretty eye-opening as to how much TC costs in terms of IO performance and latter is very useful when profiling NAS performance.

    Here's a mugshot of it in action:

    ccsio-bench-screenshot.png

    A bit of technical background:
    This is very much work in progress, so please consider it an early beta and cut it some slack :)

    What do you guys think?
     
  2. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @angstrom This seems like quite a gem of a tool. Would this also be good for testing RAM Disk performance as well?
     
  3. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Ah, good idea. Yes, it should work well for that purpose too.
     
  4. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    It was rather interesting to see that my RAM Disk was performing slightly slower in comparison to my SSD drive. Not significantly slower, but enough of a difference to be somewhat interesting. I'm not certain whether or not the results would differ much if the test file size was smaller but I am definitely curious to continue to watch your new program continue to develop.
     
  5. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Update is out - added write speed testing and new Graph mode:

    graphs-and-write-tests.png

    @WildByDesign - Interesting. I've too been testing all I can find and got some unexpected results too, e.g. apparently having a Storage Spaces volume on a single-disk pool eats as much as 10-15% of read performance.
     
  6. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Update's out. Grab it here - https://ccsiobench.com

    1. Added an option to choose between Quick, Standard and Extended tests. Press Alt to get to pop open the menu.

    menu-options.png

    2. Added an option of viewing/saving full text report.

    3. Added an option for sharing a screenshot, just plain PNG, Imgur-style. For example, https://ccsiobench.com/s/mpFja.png

    share-screenshot.png

    ---

    Anyone's up to sharing some results?
     
  7. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
  8. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
  9. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @angstrom
    RAM Disk on low-powered Windows 10 netbook: https://ccsiobench.com/s/mNa8g
    SSD on same low-powered Windows 10 netbook: https://ccsiobench.com/s/mNgeo

    If possible, I would definitely like to have the ability to run the test with various sizes for the test file. Maybe not necessarily custom sizes. But something similar to pre-set values (like you did for test length: Quick, Standard, Extended) so something like Small, Standard, Large so that it remains relatively easy to have comparisons of user test scores. Therefore several pre-determined test file sizes as opposed to entirely custom sizes. Anyway, just a suggestion. Regardless, I really appreciate this tool (among all of your tools) and I think that this tool will absolutely take off in popularity and utility as development goes on. Keep up the great work! :thumb:

    I have a technical question if you have a moment. Do you know why my write speed on my RAM Disk is limited to the same as my SSD? Would this be due to the general low-powered netbook chipset being the bottleneck? (EDIT: I should note that my RAM Disk write speed is nearly identical to the read speed when using CrystalDiskMark program.)
     
  10. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
  11. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
    WBD, that SSD looks like it's connected to a SATA II port (even though it's a SATA III device). How old is that NetBook (make'n'model)?

    Even if that was a SATA II port being utilized, the WRITE speeds are all wrong on both tests. Wish I had some RAMdisk software to do some testing... I think I have enough RAM to play.
     
  12. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
    Since CCSIO can't even identify your NetBook's chipset, there's a good chance that ChipSet may not be very robust to begin with.
     
  13. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
    I'll pickup one of the FREE RAMdisks and play a bit... will report back.
     
  14. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @TheRollbackFrog Thank you for your time.
    From CPU-Z report output:
     
  15. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    For comparison sake, here is output from CrystalMarkDisk (different test file size though):
    So I was wrong, the write speed was actually faster in comparison to read speed. Although of course these two benchmarking programs quite likely work differently so it could be like an apples to oranges comparison here. But my main concern is regarding write speed so I am still not sure why it is reporting quite a bit differently.
     
  16. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
    WBD, here's my RAMdisk Run (4gB device) using an i7@3.4/3.9mhz desktop running on an Intel H51 chipset (C220... a few generations back). Notice the massive speed diffs but also the very slow WRITE speeds (comparatively). It may have to do with how a RAMdisk is being WriteCached by the generating software (I'm using SoftPerfect), if at all, and, for sure, your dual-core Celeron @2.16mhz vs the 8-threaded i7@3.7MHZ should have some effect.

    Here's my SSD Run (about same SSD specs as yours) but connected to a real SATA III port in above System hardware config. The READ speed is close to what it's done on this configuration before and notice the WRITE speed compared to the RAMdisk... just about the same (your anomaly as well). This SSD is not being WriteCached so that isn't part of the equation for this test.

    Maybe @angstrom can give us some prophetic guidance/knowledge here :D

    Your CpuZ report only gives me the processor specs, not what the underlying chipset really is. I believe the full report does have that info.
     
    Last edited: Jan 22, 2017
  17. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
    After playing around with some other benchmarks (ATTO, CrystalDiskMark, etc.), that low WRITE rate seen by CCSIO seems to be the anomaly... as in it's not very consistent with other benchmarks at all.

    @angstrom (Alex)... go for it!
     
  18. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Coming up shortly. The way it works is that it limits every test run with both the size of a test file and the run-time. Default test file size is 2 Gigs and run time cap is 1 second. It is possible to remove the latter, so the test will run based until the file is fully read or written. However this is a fairly unproductive way to test if we are testing with a low-performance IO buffer/count combination, e.g. with a single 2 KB buffer. It will take ages to go through 2 Gigs and there's really not much point in doing that to begin with. Hence the time cap.

    I'll make another reply below, one sec.
     
  19. Infected

    Infected Registered Member

    Joined:
    Feb 9, 2015
    Posts:
    1,134
  20. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    I'm guessing that it's due to how the test file is created and, possibly, due to what data is used to fill it.

    CrystalDiskMark is a UI wrapper around diskspd utility. I didn't know that before looking just now, nor have I heard about diskspd.

    The way diskspd works is very similar to ccsiobench. For sequential write test CDM spawns diskspd in (32 buffers x 128KB x 1 thread, sequential unbuffered access) configuration. CCSIO doesn't go as high as 32 buffers, but it does check (16 x 128 KB) combo with the same access (and it's using a single thread exclusively). However the write speed it gets is indeed around ~170 MBps.

    The only difference I can think of is that for local non-SSD drives CCSIO creates its test file by using SetFileValidData() function, which extends the file on disk without initializing its contents. Conversely, for SSDs and remote locations it actually writes things into file to grow it to needed size. SetFileValidData() is much faster, but I suspect it may mess with RamDisk internal caching logic or what-have-you. It certainly causes abnormally high results when used with SSDs, because these buggers seem to remember which flash blocks were ever written to and which are pristine, so they quickly return a bunch of zeroes if we are trying to latter.

    Give me 10 minutes, I'll put together a version that creates test file via writing in all cases. Let's see if it makes any difference.
     
  21. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
  22. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    Thank you, sir. With 0.9.6, the following is what I have got in a Quick test on RAM Disk.
    Link: https://ccsiobench.com/s/m1AzO

    I will try a Standard test shortly to see if there is any difference.

    EDIT: No difference with Standard test, unfortunately.
     
  23. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
  24. TheRollbackFrog

    TheRollbackFrog Imaging Specialist

    Joined:
    Mar 1, 2011
    Posts:
    4,945
    Location:
    The Pond - USA
  25. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    OK, found it!

    When write-testing ccsio-bench defaulted to using random data, but being overly pedantic it also re-generated the random fill for every write and that's an expensive thing to do. So the prep for a write request took longer than the request itself, making testing code the bottleneck. I've reworked the code to not do anything by IO in the main testing loop.

    Softperfect RamDisk - http://ccsiobench.com/s/nlDvu

    NVMe now also shows higher write speeds:
    Updated version (0.9.7) - https://ccsiobench.com/get

    PS. Thanks for critical feedback and not taking numbers for granted.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.