VoodooShield/Cyberlock

Interesting thread here: https://malwaretips.com/threads/1-0...ution-efficacy-test-malware-test.60856/page-2

@DardiM

Actually, there was not a single default setting that was changed with VS during the test that made any difference at all... if anything some worked to our disadvantage (eg, reducing the time between executions via the countdown timer and number of flashes). The main option that was disabled was the "Allow by parent process" feature, which had to be disabled since the test app was spawning the malware as child processes.

As I have mentioned, the blacklist scanner is overwhelmed when that many samples are thrown at it at once, and when this happens, the composite VoodooAi score can change, so it depends on a lot of factors. The easiest and best way to test is just by using the stand alone VoodooAi app, that way you can see all of the raw algorithm data, and you know for certain that the blacklist scan was not involved in any way.

And actually, all you are doing is proving that VS is even more accurate under real world conditions with VS in default settings... since it is properly classifying the handful of samples that may not be that malicious. Please read post #23 on the same page for more details.

You can download the stand alone VoodooAi app... it is in my signature. It has the same raw algorithm scores as VS does, and it will tell you everything you need to know. From what I remember, it should detect 998 out of the 1,000 as suspicious or unsafe... mostly unsafe.

Please feel free to conduct your own tests and post the videos, and you will see what I have been seeing for a very, very long time now . Thank you!

 

IMA... As I mentioned before, I felt like a total idiot claiming 100% efficacy, but that is how the test turned out... if the blacklist scanner would have scanned every single file, the results would have been 998 / 1000 (99.8%), which I would have been MUCH happier with... but I cannot change the video. As far as the 2 (or possibly up to 5) samples go... honestly, I have no idea if they are malware or not, and it would take a ninja malware researcher to tell us for certain.

I certainly appreciate you taking the time to test VS, thank you! Please continue to do so, and if you find anything that slips through, please let me know!

 

Sorry I don't like the MalwareTips Community they are not trustworthy!

Daniel

 

It's all good though... I can see where he would have questions on some things (there are a lot of factors and obstacles to consider), and I am happy to clarify. And I am the first to admit that the test was not absolutely perfect, although I did my best to make it as fair as possible. But then again, the entire point of the test was to answer the question "What happens when you execute 1,000 random malware samples on a computer protect by the leading Antivirus Software?"... which I believe we did.

In the end, he saw for himself that VS was 99.8-100% accurate, depending on if the 2 questionable samples were malware or not. The funny thing is... we will never know about those 2 samples. I will keep them just in case I run into a malware researcher that can tell us for sure.

Thank you TH!

 

Are you simply stating that about the entire community/forum just because you don't like 1 or 2 threads?

 

Just because you dont like them or ppls there doesnt mean they are not trustworthly.. be carefull with what u say. malwaretips a a nice play too, like wilderssecurity.

 

Here is the latest version with the 2 new options in Settings / Advanced to disable both the blacklist and VoodooAi. Obviously, it would not be smart to disable either for daily use, but it will be really helpful for testing and for the AV test labs to have the option to test VS how they see fit.

There was a lot more involved in disabling both of these then I ever imagined, so there might be a few small bugs, like on some labels, verbiage or whatever, so if you see anything, please let me know!

Also, keep in mind, this version is kind of experimental, so if VS 3.28 is running great for you, and you do not have the time to deal with potential new small bugs, by all means, please keep running 3.28 until we are sure there are not any new bugs from these new changes.

Also, the freeze issue is not fixed... Krusty13 will appreciate this... after I posted earlier that there is a chance that it is fixed, honest to God, 10 seconds later VS froze on me... and it had not froze for at least 10 days. This happens to Krusty a lot... he will post on wilders that VS is not freezing, then next thing he knows, it will freeze.

Anyway, this is the plan (for those who experience the freeze issue). If you can please run VS with both the blacklist and VoodooAi enabled until it freezes, then send me the logs. After that, please disable VoodooAi and see if it freezes. I added a couple of log entries... one right before the VoodooAi analysis, and one right after, that logs the path, command line and parent process name. And they will either say "Before VoodooAi:" or "After VoodooAi:", and there should always be two of them. If we get a freeze and only have the "Before VoodooAi:", then at least we know what process is causing it to freeze. I think Kees is right, I think it has to do with VoodooAi and some kind of protected process.

http://www.voodooshield.com/artwork/InstallVoodooShield329.exe
Thank you!

 

I will not argue my point in Public it's my and many that I will not mention or to start a flame war or even threaten me. So please do not start one.

Thanks,

Daniel

 

Hey Dan any news on the Website fix for uploading our Whitelist to the Cloud? How depended is the Cloud Whitelisting back up for VoodooShield? Info please....!

Thanks,

Daniel

 

Not yet... I emailed Dywayne, the original web developer for that issue and to build out the admin console a little more, but I have not heard back from him yet. I hope he can get to it soon, because I am not good with web development. Thank you!

 

Thanks!

 

Installed the new version 3.29 over the top of 3.28. Will let you know if it freezes...

 

I was just asked a question that I would like to make public so that everyone is on the same page.

Question:

DardiM on MT is testing VS with 166 samples (166 samples not detected by Kaspersky) from your 1000 malware.
He mentioned got popups for few samples with details safe & recommendation allow, details FP & recommendation allow, etc...
So how the detection for VS was 100% in the test?

My Answer:

Simple, here is a good example... please look at sample # 9 out of 1000 at 39:41 in the video (which I have already discussed on the wilders VoodooShield? thread). The blacklist false positive adjusted the composite VoodooAi score to 0.3608, which is just over 0.3333... the threshold for VoodooAi to allow an item when VS is on AutoPilot.

There are probably 4-5 samples that are similar to this, but there will be a logical explanation for them too.

Remember, VS has different thresholds depending on the mode, so if you test in a different mode, then you might receive a different result.

But that is why I always recommend people test with the stand alone version of VoodooAi, if they are interested in seeing how VoodooAi performs on its own. That is actually the main reason why I created the stand alone version... so people can test the efficacy of only VoodooAi, and so they know there is no funny business going on with the blacklist scan. Fire up fiddler and test until your heart is content .

It will not be an issue either way now, since you can disable both the blacklist scan and VoodooAi, and you can test however you like.

 

BTW, if sample # 9 would have been less than 0.3333, it would have executed. I am under the impression that sample #9 is not super clean, but it is not that bad either... hence the score of 0.3608.

The same applies to the other 4-5 or so samples. I do wish the blacklist scanner could have tested all of the files, but that is just the way it goes. And again, that is what the stand alone version of VoodooAi is for... and it is free .

 

Actually, now that I think about it, false positives do not affect the composite VoodooAi score, but true positives do. So the score was probably 0.3608, and over the 0.3333 safe threshold for AutoPilot on its own.

BTW, 1-5 or so true positives will not affect the composite VoodooAi score that much at all, but the higher the number or positives and the higher the VoodooAi score, the more the composite VoodooAi score is affected.

 

BTW, it is starting to become clear where we should set the thresholds... remember how it has been a struggle to determine exactly what to set them to?

I have been playing it safe (especially with AutoPilot)... but I think we can probably bump up the .3333 to .5000 for AutoPilot... I think those 2 files were relatively safe. Sound good to everyone?

I think the stand alone version of VoodooAi has a safe threshold of .5000... and same with Smart mode when VS is OFF.

And we still need to figure out the other thresholds . This stuff just takes time.

 

Go for it buddy!

Daniel

 

VoodooShield v329.Beta running well on Windows 10 X64 O.S. So far, I will let everyone
if something chances.

Moose

 

Yep! That's happened a few times now.

I'll install 3.29 for a few days and perhaps disable Ai and see how it goes. This time I won't put VS in training mode when installing or updating so hopefully if it is going to freeze it will do so a bit sooner.

 

No issues with VoodooShield v3.29......

 

Running great as always on my 2 rigs! Both Windows 10 x64.

Daniel

 

3.29 is OK on my Win-7 even alongside some other heavyweight apps.

 

Hey Daniel,nice to see I am not the only one without problems with VS

 

I never have with VS! Now I would like to see it for myself but I can't replicate even on my 4 VM's 2 Win 7 x64 and 2 Win 10 x64.... so it could be a conflict with certain other security software that others are using with VS so I wonder what the common denominator is so i could test it.

Daniel

 

I use Smart mode and I have never tried Autopilot maybe I should?

Daniel