Bvckup 2

Discussion in 'backup, imaging & disk mgmt' started by angstrom, Jul 7, 2014.

  1. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Hello everyone,

    I thought I'd formally introduce a little creation of mine called Bvckup 2




    In short


    It's a simple data backup software that is meant for mirroring directories. If you are currently doing your backups by copying files from your primary machine to an external drive or a NAS, the app will help you do it better, faster and in altogether more pleasant format.

    What it is not

    It's not a system backup, it doesn't create a system image that can be used to restore a bootable system after a failure. It is not a all-in-one backup tool, it doesn't compress things, it doesn't FTP or encrypt files nor does it have a restore function.

    It merely copies things from A to B, as is, and it is exceedingly good at it.

    Speed

    1. It uses delta copying to update file modified between the backup runs, so that it copied only modified blocks of files. This speeds things up, in many cases dramatically. I run a backup that includes 256 GB TrueCrypt container and straight copying to an external USB3 drive takes about 4 hours. With delta copying a day worth of changes takes under an hour to propagate.

    2. It uses fully asynchronous I/O with adaptive buffering, meaning that the app never waits for a read/write request to complete before issuing the next one. This eliminates unnecessary idling when copying and helps keeping the copying pipeline busy at all times. It has a higher bulk copying speed than newer robocopy versions and other specialized 3rd party programs that I have tested.

    3. It uses parallel disk scanning, when building the file lists for source/destination locations. Not a rocket science by any means, but this can speed things up by a factor of 10 when scanning slower NASes over laggy networks.

    4. It defaults to maintaining a local snapshot of the destination, instead of re-scanning it on every run. It basically scans destination once. remembers what's there and the incrementally updates this snapshot as it works its way through the backup. So at the end of the backup it has the exact picture of what's at the destination, so on the next run it has no need to scan it.

    5. It's heavily optimized through out in terms of mundane code optimization. I come from the system programming background and writing a lean code is a habit. The app is written pretty much in C, though it does use several C++ features and it is compiled as a ++ code. There are no dependencies on any external frameworks - not even MFC/ATL - it's all done over raw Win32 API with a bit of COM where it's unavoidable.

    Convenience


    1. Removable device tracking - it can make a note of a device fingerprint or its volume label and then track it across all drive letters, should the drive get a new one. This also prevents a backup from going on some other removable device that may just show up under the same letter.

    2. There backup scheduling options - periodic, real-time and manual

    3. Volume shadow copying

    4. Symlink support

    5. Move/rename detection at source

    6. Archiving of deleted items with age-based auto-pruning

    The UI

    It has the best UI in business. Guaranteed.

    Screenshots are here.

    I have killed about 18 months on the app and a good half of it went into the UI design. There's also quite a bit of interesting stuff in the development log.

    Status

    The app went through 12 months of beta, from May 2013 to May this year. During beta it saw about 15,000 installations and it went through 67 interim releases. It is now routinely used to handle multi-TB backups in production environments as well as a ton of smaller home and personal backup. In other words, it is pretty damn stable.

    Misc

    My personal site is at http://swapped.cc, just in case.

    Any thoughts, questions or comments - I'd be happy to hear/answer them.

    Thanks!

    Alex
     

    Attached Files:

    Last edited: Jul 7, 2014
  2. TheRollbackFrog

    TheRollbackFrog Registered Member

    Joined:
    Mar 1, 2011
    Posts:
    3,059
    Location:
    The Pond - USA
    Greetings, Angstrom! Having participated in your BETA testing program for this file replication tool, I can only say it is one of the easiest set-and-forget file backup tools I have ever used, bar none. Sure, there are a few "features" not yet implemented that some will require (versioning, compression) but this release is excellent and I really look forward to additional features as they are developed.
     
  3. kennyboy

    kennyboy Registered Member

    Joined:
    Oct 4, 2006
    Posts:
    404
    Agree, a superb and simple program that does exactly what I need, and reliably too, even though still a beta. Many thanks to you.
     
  4. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Thanks, great to hear this! You helped quite a bit along the way too, so thanks go both ways :)

    The core premise of the app is that it is intentionally feature-restricted. It does just one thing, well. There's no support for compression and I have no plans for adding it. It is very easy to add, but doing so will dilute the focus for the app and that's something that I'm keen on avoiding.

    The versioning I do plan to add, but it's a major feature, it takes time to plan and to implement properly and as such it will be in the next major version (V3). I firmly believe that the backup archive should be kept in an open format, so that people won't get locked into a specific app. Details are here if anyone's interested.
     
  5. TheRollbackFrog

    TheRollbackFrog Registered Member

    Joined:
    Mar 1, 2011
    Posts:
    3,059
    Location:
    The Pond - USA
    Kenny, the v2 of the product is out of BETA and available as an Official Release.
     
  6. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Oh, right. Yes, it's been out of beta for two months now.

    You still can get the last beta if you want an older free version, but it's neither supported nor is it maintained. It's basically the "you are on your own" version. Kennyboy, perhaps it's the one you were referring to?
     
  7. kennyboy

    kennyboy Registered Member

    Joined:
    Oct 4, 2006
    Posts:
    404

    Yes, sorry. It was of course the Beta that I was referring to.
     
  8. Brian K

    Brian K Imaging Specialist

    Joined:
    Jan 28, 2005
    Posts:
    8,652
    Location:
    NSW, Australia
    angstrom,

    Nice program. Fast.

    One question following my brief exposure to this app. Is it possible to use the root directory as a source rather than a folder?

    For example, D:\ instead of D:\source_folder

    Edit.... It was there all along. Thanks.

    Edit2... Any problems with a partition containing 500 GB of data running with "When files or folders are modified, in real time"?
     
    Last edited: Jul 7, 2014
  9. TheRollbackFrog

    TheRollbackFrog Registered Member

    Joined:
    Mar 1, 2011
    Posts:
    3,059
    Location:
    The Pond - USA
    Brian, my experience during BETA says no, not at all... especially if the partition is primarily a DATA partition and the changes aren't constant. I tried this at the folder level by selecting a very busy folder on the system partition and replicated it in real time. It gets busy but its effect on the system was minimal. I also selected that very busy folder and used the INCLUSION feature of only 1 or 2 sub-folders in real time. When you do this, the function is fired off constantly (it's looking at the top level foder for changes) but only data changing in the included sub-folders is replicated.

    It really doesn't have a lot of effect on the system. And I think he plans on adding some bandwidth throttling if users feel they need it.
     
  10. Brian K

    Brian K Imaging Specialist

    Joined:
    Jan 28, 2005
    Posts:
    8,652
    Location:
    NSW, Australia
    TRF,

    Thanks. I appreciate your speaking from experience with this issue.

    It's my kind of app. Simple. Copying A to B in native format. I don't have a need for compression or versioning.
     
  11. Stigg

    Stigg Registered Member

    Joined:
    Jun 13, 2012
    Posts:
    322
    Location:
    Brisbane, Australia
    Where can I get that from to try?
     
  12. newbino

    newbino Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    377
    angstrom,

    thanks, it looks like an interesting program and will trial it soonest. A question about licensing: on the site, it states that the license includes "Maintenance releases". Am I correct to assume that it will cover the various iterations of 2.x, and when you release 3.0 it will require a new license? And if yes, any idea how you'll handle this with existing license holders (ie, discounts)?
     
  13. Enternal

    Enternal Registered Member

    Joined:
    Apr 21, 2009
    Posts:
    42
    https://bvckup2.com/ at the bottom of the page where it have a big blue button that say "DOWNLOAD", there's a little link Looking for the last beta? that you can click and it will provide you the link for the beta. Basically:
    https://bvckup2.com/get/beta :)

    Anyway, excellent software. Very lightweight and fast. What's amazing about it is that it's obviously the developer, Alex (angstrom), is a graphic designer of sorts since every single little graphical detail has been thought too. Not only that, he's also a superb programmer it seems since the entire software is very lightweight on system resources, does a great job at doing what it's supposed to do, very fast, and very easy to use. Plus delta copying without relying on rsync and cygwin.

    If you use it long enough, you can also tell that a lot of thought has been put into the output to let you know exactly what has been done, anything that went wrong, and other miscellaneous information that may be important. Completely worth it. I wasn't one of the beta testers and found it much later (sometime in May) but so far, it's definitely one of my top favorites for file-based backups along with FreeFileSync (for portable usage) and perhaps zback (as a backup).
     
  14. beethoven

    beethoven Registered Member

    Joined:
    Dec 27, 2004
    Posts:
    1,044
    Could you please clarify something - I am currently using Cobian to keep a number of copies of data backups. Am I right in understanding that with Bvckup 2 I can "only" do synching? Is that what you mean by no versioning at the moment? The reason I ask is that I have had several experiences where I realised I had messed up a file and could go back to an earlier backup to fix this. If there is "only" a synched file, I would need to consider how to deal with this issue as any "user errors" will immediately be mirrored in the backup/synched file.
     
  15. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    This is probably not a good idea, because the backup will end up running pretty much all the time.

    This is because on a large disk the scan for changes will take quite a bit even with a warm cache, so the backup will end up spending most of its time scanning.

    And the reason for scanning for changes is that there are basically three ways for an app to track file system changes. First is to subscribe to the high-level "something changed in this tree" notifications, second is to subscribe to discrete "item Xyz changed" events and third is parse NTFS change journal. Third is not really an option because the journal can be flushed with no notice and it requires admin privileges to read. The problem with second is that the event queue is lossy, meaning that the system may start dropping events if there's too much going on. More importantly, it maps to an uncontrolled kernel memory usage. Imagine you just touched 100,000 files - now, you have a large chunk of memory allocated by the kernel to hold the notifications on behalf of the app, and the only way to flush it is to process notifications. And what if the app can't.

    Long story short - the app needs to rescan the source on every backup run. All other options have unpleasant marginal cases.

    With regards to the effect of the file count on backups - I've done a considerable amount of work to ensure that run-time memory usage is flat. The app still uses memory to store the source tree in memory (at a cost of about 100 bytes per item), but once it gets past scanning, the memory usage remains constant. See here for details.

    Correct, 2.x releases are included. I plan to add several things - (1) Email notifications for end-of-backup events (2) Backup verification (i.e. bitwise comparison of src and dst) (3) I/O throttling (4) backup run ETA

    Moving to 3.x releases - as premature to talk about this as it is - an upgrade pricing will be in place. It's a standard practice I believe and I have no intention to reinvent the wheel there.
     
  16. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Correct, Bvckup 2 is not meant to protect against human errors, this is not its primary purpose. Files are mirrored and their previous copies are not archived (unless a file is deleted, in which case the app can archive its backup copy).

    You can set several periodic backups, staggered in time, to implement a rolling backup and it will give you an approximation of archiving.

    As I said, I don't argue that archiving is not useful and I plan to implement it, but I want to get it right. See here for details if interested.
     
  17. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Thanks, much appreciate all the kind words :)
     
  18. sdmod

    sdmod Shadow Defender Expert

    Joined:
    Oct 28, 2010
    Posts:
    781
    On trying to install the beta today, it tried to make a connection out and then Avast came up with 'Sign of "Win32 Dropper-gen [Drp]" has been found in "C:\Program Files\Common Files\Windows Driver Foundation\WUDFHost.exe" file.'
     
  19. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Did you get the beta off bvckup2.com? What does it digital signature say?

    I run the setup package and both .exe inside of it (the app and the uninstaller) through VirusTotal as a part of the release procedure. They are all clean.
     
  20. sdmod

    sdmod Shadow Defender Expert

    Joined:
    Oct 28, 2010
    Posts:
    781
    I installed it again in Shadow mode to see...
    File: bvckup2-setup-unsupported-beta.exe

    downloaded from https://bvckup2.com/get/beta
    File size: 1.51 MB (1,584,744 bytes)
    MD5 checksum: B4CE25C0DC215A29020E5F696CBEA120
    SHA1 checksum: F4B1E337FCF0AE06A19FE564183DA3273D82B1E4

    It still insisted on some online verification or it showed as corrupted
    I didn't get a Trojan alert this time.
     
  21. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Yep, that's kosher.
    It doesn't do any online checks per se, however the first thing it does it checks the digital signature on its own executable. This is handled by Windows security subsystem, which may pull down missing intermediate certificates and/or it may run a CRL (revocation) check. This is what you are seeing, I have no control over this behavior, it comes from the guts of wintrust.dll. See DNS requests section on Behavioral Info tab of the VirusTotal scan.
    I've seen false positives before when AVs got weak knees seeing that app's exe is UPX compressed. Though I have no idea why Avast would decide to point at WUDFHost.exe. This issue is between you and Avast I'm afraid.
     
  22. sdmod

    sdmod Shadow Defender Expert

    Joined:
    Oct 28, 2010
    Posts:
    781
    Thanks for the information.
    It probably was a false positive. Just to be on the safe side I virus vaulted and deleted file WUDFHost.exe and replaced from a scanned backup.
    I have an aversion to software that instigates authentication check behaviours. I prefer software to be dumb on my machine and not start connections without being asked.

     
  23. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    I don't like software that phones out either, but as I said these are connections made by Windows itself, not the app. I'll see if I can try and suppress this somehow.

    With regards to the authentication checks - this helped detecting corrupted downloads on more than one occasion. It also helps mitigating basic installer mangling and 3rd party wrapping.

    Edit - And it is possible to suppress over-the-network checks. I've made the change and it'll be in the next production release (not in the beta though).
     
    Last edited: Jul 9, 2014
  24. sdmod

    sdmod Shadow Defender Expert

    Joined:
    Oct 28, 2010
    Posts:
    781
    Thanks angstrom, much appreciated :)
     
  25. angstrom

    angstrom Registered Member

    Joined:
    Oct 27, 2013
    Posts:
    42
    Location:
    Switzerland
    Just a quick note to say that a new version - Release 70 - is out and it adds support for network share logons, command-line control of the backups and few other things.
    Full change log is over here - http://www.bvckup2.com/support/forum/#!/topic/416
     
Loading...