Macrium Reflect

Thanks for the confirmation. After I remembered Image Guardian it seemed obvious to me that there is no other way these files disappeared. I hope they fix it quickly or I will have to find a replacement.

 

@xxJackxx @warrior99 @stapp
I am wondering, if you look through Reflects logs page and inspect the logs for the last few runs, can you spot any retention activity that would indicate that one of the jobs that ran since updating has indeed culled your backup sets, and for what reason?
The fact that the folder is gone is interesting.

Can you post your logs here? Would be interesting to see what had happened!

 

Just another Macrium user, lurk mostly, but figured I would add my 2 cents on the situation to help figure out the problem.
I cant say I have seen Reflect delete a folder during its retention run....
AFAIK, when retention runs, Reflect will print out a line in the log file for the current job to explain the operation; so I wanted to point out that the clue to the situation lies there!

 

I agree, it would be, but there was nothing in the log to indicate it deleted anything. It was due to purge the previous backup set. The first thing that happened after updating was it ran an incremental backup on the existing current set, logged doing so, and then everything was gone. Without posting any private information or a bunch of junk, this part of the report makes it look like it deleted nothing, but everything was gone, including the backup it just ran:

Retention Rules
Rules will be applied to all matching backup sets in the destination folder
Full: Retain full backups for 5 Weeks
Linked incremental and differential backups will also be deleted
Backup Sets: 2 sets found
Nothing to delete
Incremental: Retain incremental backups for 35 Days
The oldest incremental may be consolidated
Incremental Backups: 19 found
Destination
Backup Type: Incremental
File Name: Append to recent backup in directory 'E:\Macrium\'
E:\Macrium\5341BCEADD2F7557-04-04.mrimg

After which it proceeded to log the backup process, a bunch of text with no interesting information. And to answer the next question, there was 450 GB free space on that drive so it wasn't a space issue.

@stapp Thanks for reporting it.

 

@xxJackxx
Interesting, so at the time the job ran and when it finished, Reflect reported two sets (the current set your last inc backup was appended to, and the old set) and stated there is nothing to delete yet.
You mentioned that you only have one external media, and the backup folder on it is now gone... really interesting, perhaps concerning.

Apologies for asking stupid questions, but have you got any other drives mapped on the PC, or is E:\ the only other letter on your PC outside of C:\?

If you're 100% sure that you haven't missed anything I would strongly advise you to message Macrium support and present this issue, as they would have a much easier time figuring out the fault (be it bug in the software or user error) and addressing it when working directly with the person experiencing the issue rather than getting a report from 3rd party that they have to chase down.

From my previous experience with them, even if you're out of support, bug reports are taken seriously and get escalated no matter the support status.

 

I have an internal D: data drive but there are no Macrium folders there. I have a single 2 TB external SSD I use for backup plus some other files that are undisturbed. I find it odd that it would run that incremental, log doing so, finish the job, and then the files and folder are gone with no indication in the log of doing so..

With Image Guardian on no other process could have even tried to remove the files without me getting a popup from Image Guardian. I have run a full backup and 2 incrementals since. If this is repeatable I am probably fine until the next purge when this set is 5 weeks old. If it was a random 1 off we may never know what happened. I am out of support but will go ahead and reach out as this will need to be fixed in any case if it is broken.

 

I have emailed their support as I don't seem to be able to log into my Macrium account. I will post updates if I get any that aren't intended to be private communication.

 

@stapp
I cant say I am having the same issue as well, I just thought that the target backup folder deletion was a little bit odd.
I mostly run Full + Diffs, having retention delete my oldest Diff when a new one gets made. While Inc backups are great for their speed, I don't like dealing with large sets where only one item is a point of failure for the rest of the Inc set.
@xxJackxx it would be great to know how this investigation turns out, looking forward to seeing you find a resolution (or an explanation) to this problem.
It would be great to hear from @warrior99 to see if they have experienced the same behavior where logs don't report deletion.

 

That was the thing that got my attention. If just the files were missing I would have thought it odd but would have been less surprised. I would have assumed a purge of outdated files with an incorrect date check or something. The folder as well? That was concerning.

I agree that incrementals are more fragile but since I only run a set for 5 weeks with another started at the first of the month I've never had an issue. Even at that a single fail is only from that file forward and if the base image gets corrupted doesn't matter if it's differential or incremental. Differential is a lot of redundant writes to a SSD.

So far I sent them the log for the event in question and they requested additional logs that I sent to them. No more updates yet since doing so. I'm not sure what hours their staff works but I think they're in the UK so I am not necessarily expecting any more responses today.

 

Ok where to start
I have not got Macrium Reflect logs
I had an issue with my pc and had to do a restore with Macrium Reflect to the only backup file I had left after the update,
And Macrium Reflect failed to complete the restore, this came as a shock as I have done many restore's so with my only backup not working I have to wipe and format and reinstall,
I wont be installing Macrium for the now.

 

I'd try the restore again. And again.

 

How good that we have the Wilders Forums. I was about to start the updating procedure, and before doing so I just wanted to inform myself here about the changes coming with the update. But reading your posts, of course I will not update for the moment. - Within the next days anyway the migration to my new PC will be finished. I thought about transferring my Macrium Reflect licence but maybe it is better to stay with the Aomei Backupper which I have installed there and which works until now without any issues. (But I admit that Macrium for many years did its work also for many years perfectly on the computer I will leave now.)

 

I have a desktop that I do not intend to update. For the laptop it's too late.

The last email I got from them is a request to schedule time for me to show them the problem. Unless someone can clue me in as to how I would do that (showing the problem) I'm not sure how that would go. I've sent logs which didn't log what happened. I can't show them the files that are now gone. The new files are no clue. I don't expect that my employer would be good with me doing this during work hours. I'm really not sure what to do with this unless I can find a repeatable way to duplicate it. Which I may try to do but I'm not there yet if it is even possible.

 

Are you sure you’re looking at the log of the correct job? There’s no chance some other job log contains records of deletions? And are you sure you’re not omitting any portions of the log that might be relevant? I only ask because I still haven’t see a full job log here, and I’ve been involved in a number of cases where an element that was crucial to understanding a problem remained unknown FAR longer than it needed to be because someone posted an overly snipped log or overly cropped screenshot, totally unaware that the content they were omitting in fact contained key information.

 

This is just a suggestion, I don't have experience with this, just heard it from a colleague of mine.

Use any data recovery software. You do not need to recover the files, just be able to find any trace of any one of the deleted files. The files will show up in the recovery software with deleted time stamps showing the date and the exact time the files were deleted. Compare it with Macrium logs to see if Macrium was running at the time the files were deleted. That will at least prove that Macrium was running when the files were deleted.

 

I had thought about doing that but now that someone else has suggested it I'm going to give it a try.

 

It get weirder. I scanned the drive for deleted files. None of the backups were found. What I did find was that all of the missing backups were on the drive, with the folder containing them, in another folder. It would be easy to assume I dragged and dropped them there by mistake. Based on the time stamps the backup started at 9:00:00. These files were last accessed on that date at 9:00:31. The backup completed in 1:46. Based on that these files were last accessed while the backup was running. I was not using the laptop at the time, and I certainly did not have File Explorer open at that time. I opened it after the backup was done to see how much space was being used and the folder was gone.

I just attempted to cut and paste one of the files to yet another folder and Image Guardian stopped me, which I expected it to do. Based on that it could not have been something I did and the files had to have been moved while the backup was running. We could assume that somehow the destination folder got changed, but on the next run it created the folder where the old one had been and has worked correctly since then. Now I need to get this info back to them. This all makes even less sense than the idea that they were deleted.

 

There is a flaw in the Image Guardian logic that it will prevent the actual backup files from being moved or deleted, but if you cut/paste the entire backup directory containing the backup files to another location on the drive, Image Guardian will not block that move. Unfortunately it does not even log such a move, otherwise all Image Guardian blocks are logged under the "Events" tab in Image Guardian setting dialog.

But it is good that your files were not deleted, just moved. So you did not lose any data. I am not sure why or how this could have happened. Maybe check the permissions on that backup directory, if Macrium could not write to that directory, it may have created a new directory to write the new backup files to, but that still does not explain why the original backup directory got moved to a new location!

 

@xxJackxx I am glad you got to the bottom of the issue.
MIG will allow you to move files on the same file system, so you just have accidentally moved the backup target folder into a different folder erroneously.
This is great news, I trust Macrium for my data protection needs and have recommended Reflect to nearly everyone I know that is also a PC nerd, and I would hate to see a bug that would delete folders at random.

 

From the testing I just did it won't let you move individual files but it looks like it will let you move the folder. Seems like a big enough hole to give a ransomware author a way to get around it, or at least hide the files. I was pretty sure I didn't move it but I can't prove anything either way at this point so I'm going to let it go for now.

 

@xxJackxx Apologies, I have misspoke, I meant to say folders and not files. And no this behaviour doesn't open any avenues for ransomware....if a malicious actor is trying to hide someones files on a computer by moving them to a different folder on the same disk/FS, I think they might need a career outside of IT, don't you agree?
MIG will allow you to move folders containing backup files to another folder on the same filesystem but not the file itself, this is just how NTFS works.
I haven't tried using other filesystems that it supports like exFAT or the still experimental REFS.
The way MIG works according to macriums KB is that it blocks any writes done to the file. MIG won't block creation (you can see yourself by copy pasting a backup file to a different folder, tada you have made a copy while MIG is on). When moving a folder on NTFS, the location of the files on the disk in that folder dont change, just the location of the folder on NTFS, while the files themselves remain unmodified. This is why MIG doesn't block this operation as the file is never modified, and you can rest easy that this is not a bug or a security hole.

 

Though I haven't tried it with these files... they could hide them in an Alternate Data Stream. It would be enough that the end user would think they were gone. This is based on assumption as I don't like to claim anything I have not personally verified but based on previous experience I would think it would be possible.
Otherwise I agree with what you said. it would take extraordinary effort to block this, more effort than it would be worth.