Diskovery (beta)

Discussion in 'backup, imaging & disk mgmt' started by angstrom, Mar 31, 2016.

  1. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Can I get some opinionated, but quality feedback on my new baby?


    In short it's a tool that scans the machine looking for all storage devices - both drives and arrays - extracts all info it can from them, including SMART data, then does the same for all logical volumes and finally presents it all neatly in the UI.

    Something like this:

    diskovery.png

    Of some note:
    1. Diskovery does a decent job talking to directly accessible drives, including those in USB enclosures.

    2. It can look into RAID arrays, but this is strictly a per-vendor affair, so the app currently supports just the CSMI-capable controllers and HPSA cards (SmartArrays). MegaRAID and PERC support is coming up.

    3. The app detects both maximum and effective USB drive connection type. That is, it can point out USB3 drives plugged into USB2 ports (screenshot).

    4. The app talks to the Virtual Disk Service, so it displays the type, health and status of so-called dynamic volumes. You know, the Windows Software RAID contraptions (screenshot).

    5. It also understands sub-partition slicing - that is when a single partition is being used to host two or more volumes.

    6. It has a nice UI, but that's obviously subjective and some are bound to disagree. However, it is the only app of its kind that actually bothers to show which physical drives are attached to which RAID volumes.

    7. Fits on an floppy, 225KB in size at the moment. Single exe with no external dependencies and no bloat.
    What else... built-in update checks, built-in scan log submission, auto-refresh on both device arrival/departure and on volume mounting/dismounting.

    This used to be a test runner for the SMART module for Bvckup 2. That was in November last year. As I was tweaking and growing it, it started to shape into something useful, so I ended up spinning it off into a separate program. There are no plans to turn this into a paid product, so it will remain free even after it leaves beta.

    So there you have it. Everything you wanted to know about your storage and then some.
     
    Last edited: Mar 31, 2016
  2. trott3r

    trott3r Registered Member

    Joined:
    Jan 21, 2010
    Posts:
    1,283
    Location:
    UK
    Looks useful.
    How about adding the ability to change drive letters?
     
  3. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    It's meant to be an inspection tool rather than a management utility, so it's ultimately read-only.
     
  4. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    tried running it and after about 10 minutes I used Process Hacker to check and it shows no cpu or I/O disk activity so I terminated it.
     
  5. donaddams

    donaddams Registered Member

    Joined:
    Jul 5, 2008
    Posts:
    99
    Location:
    mojave Desert
    Very nice works great, Thanks
     
  6. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Did the UI show up?
     
  7. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    No, just the screen showing the dictionary note.
    I'll try again now and time it accurately.

    Tried the second time and the UI popped up immediately.

    So I don't understand, but it's working now.

    I tried to send the log file but:
    when I tried to View the log I was told that it's in use by another app - so not viewable, and then when I clicked to send it, it crashed and I had to "end" it.
     
    Last edited: Mar 31, 2016
  8. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Did you click on the blue link with a witty joke at the bottom of that screen?

    Yes, sorry about the crash. It was a brain fart on my end, resolved now.

    The log viewing thing - it means that your default .log viewer tries to open files in read/write sharing mode, which will fail if the owner of a file is working with it and has it opened in the read sharing mode only. No other app should be writing into the Diskovery log, so the app doesn't enable write sharing on it. In other words - fix your log viewer :)

    I've also changed Diskovery to default to Notepad for log viewing unless Ctrl is held down when the log is opened. In this case it will use the default .log viewer.

    0.9.0.8 is out - adds vendor detection for M4-xxx Crucial drives, ADATA and HGTS drives, adds a couple of new SSD SMART attributes, now also shows logical volumes that don't have physical extents (such as TrueCrypt and RamDisk mounts).
     
  9. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    No, guess I should have but after waiting 10 minutes I just 'X'd it, sorry. But I didn't see that screen the next time, did it somehow remember that it had been presented before?


    I'm pretty sure that's Notepad and I don't know how to 'fix' that.

    I'll try the new version... & see what happens...
     
  10. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    OK, the new version worked and the log was sent.
    I didn't attempt to use the icon in the lower left before this but I did this time just to see what it was.
    It may be a standard icon to many but it's new to me. And it's in a very lonely spot and just grey, maybe a bit of colour in the top left corner where most menus are?

    I did get the 'dictionary' screen again and clicked on the link and the UI opened right away. It might be better to have the text for the link just text and add a button [Continue] which is more obvious...
     
  11. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @angstrom Very nice program. I like the portability and efficiency. The program brings up thorough information and yet is able to display that information in a pleasant layout/design. Very fast and snappy as well. By the way, the web site layout and design is nice as well.
     
  12. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    @jwcca - got the log, thanks. Made few changes based on it, including vendor detection for your drive vendors and a couple of new SMART attributes from their list. 0.9.0.9 is out, change log is here.

    @WildByDesign - thanks, glad you like it.
     
  13. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    And I like it too. I'll spend some time now actually looking at all the detail that you've provided.
    Thanks,
    J
     
  14. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    @jwcca - thanks, glad to hear that!

    0.9.0.10 is out, adds support for RAM drives and volumes and for unformatted/corrupted volumes. Also adds vendor detection for Samsung NMVe drives.
     
  15. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @angstrom I definitely appreciate the addition of RAM drive support. This is quite a thorough and fantastic tool. Keep up the great work.

    I suppose my only gripe at the moment is not being able to resize the GUI and having that dynamically scale the contents within the UI. The reason being is because on my main daily systems is has a resolution of 1366x768 which cuts off part of the UI.
     
  16. jwcca

    jwcca Registered Member

    Joined:
    Dec 6, 2003
    Posts:
    772
    Location:
    Toronto
    It's interesting to see the different drive/port numbers assigned by Intel Storage Mgmt, W7 and Diskover, columns K - M in the screenshot.
    Then compare those to the motherboard socket number in column F. Intel only reports on the Intel chip (K) with the Marvel left blank.
    The Diskover values are 1 higher than the W7 values except for the Marvel chip where one set of values, drive H, is the same and the values for drive I are 'radicaly different.

    The one item that might be beneficial would be to add the firmware value, column Q.

    Otherwise the app is very thorough and 'good to have'. Previously I had to run Perfect Disk to check the serial numbers so I now no longer need Power Desk (all drives are SSDs).

    Thanks,

    J
     

    Attached Files:

  17. Zero3K

    Zero3K Registered Member

    Joined:
    Mar 28, 2004
    Posts:
    380
    Location:
    Louisville, KY
    I have some suggestions for it:

    1. It should auto-update the SMART status of the drives every so often.
    2. It should be able to be minimized to the system tray.
    3. It should be able to warn the user when the monitored drives aren't working as well as they should be (such as when the temperature of one of them gets too high, etc).
    4. Maybe it could use a section where it shows what's being read/written to the displayed drive.
     
  18. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Thanks, appreciate the pat on the back :)

    OK, noted. The column resizing is on the ToDo list, but perhaps there might be another way to handle smaller resolutions. Let me think about it.

    This information is readily available and it was shown originally, but then I couldn't think of any case where this can be actually useful, so I removed it. It's still available in the raw scan log if needed.

    All makes sense and all noted, thanks!

    Ideally I would want to have a version that's just a single tray icon. You click on it and it just says "Everything's OK", and then when there's a problem with a drive, it will light up like a Xmas tree. But we'll get to that a bit later I think.

    ---

    New version - 0.9.0.11 - is out, adds detection of the effective SATA signaling rate, including that of the drives in arrays. It worked out to be not as straight-forward as I thought it would be. Details are here.
     
  19. MerleOne

    MerleOne Registered Member

    Joined:
    Mar 6, 2006
    Posts:
    1,336
    Location:
    France
    Got a strange warning regarding my SSD. Will try with latest version (0.9.0.11 or 13 ?) ?
     
  20. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Can you elaborate?
     
  21. MerleOne

    MerleOne Registered Member

    Joined:
    Mar 6, 2006
    Posts:
    1,336
    Location:
    France
    I will, as soon as I can access again that PC (currently moving...).
     
  22. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    A little update, anyone?

    To date ~200 people have kindly submitted their scan logs. Tallied up they yielded:

    1. Around 30 different vendors, some of which I haven't even seen before:

    diskovery-vendors-wilders.png

    2. Top three are Seagate, WD and Samsung - responsible for 19.1%, 18.1% and 12.6% of the drives.

    3. 219 HDDs, 58 SSDs - 79% and 21% respectively - adding up to 277 devices in total.

    Next, the SMART data... now that's where it get interesting.

    3. The only SMART attribute present in all SMART reading was ... the power cycle count (#12 decimal).

    4. All but one reading also contained power on hours and reallocated sector count. Which is excellent news, because the latter is one of the main predictors of impending drive failures.

    5. The temperature attribute (#194) was reported by a mere 71% of HDDs and 51% of SSDs. That said, some devices report the temp as a different attribute, so these %ages aren't totally accurate.

    6. With an exception of one device the attribute range of 17 - 159 is unused.

    7. Lots and lots undocumented attributes. Some vendors have incomplete specs and others (*ahem*Seagate*ahem*) just don't release SMART specs at all. This means that we can't even label the attributes.

    8. Lots of inaccuracies in the specs too, meaning that it's hard to reliably interpret the attribute's raw values. Raw value is a 6-byte sequence of vendor-specific data, so you *must* have a spec to understand them. And you want to understand them, because that's where the actually interesting data is.

    Regrettably some specs would say that it's just a 48-bit counter, whereby in reality it'll be a 16-bit something followed by 32-bit counter. Furthermore, certain vendors (*ahem*Kingston*ahem*) tend to change the format not just between the product lines, but between sibling products that are just a model away from each other. It's really quite nuts.

    --

    With all this lovely mess in mind, here's how Diskovery is going to deal with it. The app will use gradually refined interpretation. It will first check for known common attributes that aren't media-specific and that are widely supported (power on data, temperature, etc.). Second, it will consult common HDD or SSD attribute tables. Third, it'll do a per-vendor lookup and, finally, it'll descend to the model-specific interpretation if there's one available.

    This is by no means a revolutionary approach, but I mention it, because it *will* result in some attributes left either uninterpreted or tagged as "undocumented". If this happens, hopefully you now understand why (and you won't forget to send the scan log in for the record).

    --

    New release will go out early next week. The one after should add the NVMe health interpretation, and then we'll move to adding support for LSI/MegaRAID controllers and such.

    Have a nice Friday everyone!
     
    Last edited: Apr 22, 2016
  23. MerleOne

    MerleOne Registered Member

    Joined:
    Mar 6, 2006
    Posts:
    1,336
    Location:
    France
    Hi, this reminds me I still have to re-test this gem on my SSD laptop and report (how ?) the strange readings I get.
     
  24. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    126
    Location:
    Switzerland
    Please wait until next week. Current version incorrectly interprets #231 as "Temperature" whereby it's typically means "SSD lifetime remaining".
     
    Last edited: Apr 22, 2016
  25. MerleOne

    MerleOne Registered Member

    Joined:
    Mar 6, 2006
    Posts:
    1,336
    Location:
    France
    Sorry, can't wait : I just sent the log. What I find strange is the warning regarding "Command timeouts" , the line is yellow and I see 68 as raw value for a metrics of 100. ID is 188.

    Regards,
    .merle1.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.