MRG - new test results

[Dark Star 72]

MRG have published their new Browser Security - Financial Malware tests results this morning.

http://malwareresearchgroup.com/?page_id=2

[ViVek]

Prevx,SpyShelter,Zemana,Defensewall.........great apps

[3x0gR13N]

Would be nice if they tested with products' advanced settings as well (via simple checkbox or slider leveraging, where available) and added screenshots of the warnings (if any) of the products that passed the test.
And any reason why the results of one of the tested products (Comodo presumably) is censored/darkened out?

[trjam]

Kudos to Prevx, very good work.:thumb:

[Dark Star 72]

Will be interesting to see this series of tests unfold over the next few weeks.
I was surprised to see that OA++ failed, and it seems that just about all of the actual security suite's failed except for Norton. Nice to see that DW passed this one after the confusion in the last set of tests.
The different tests to follow may throw up wildly different results.
One things for sure, some people will be getting hot under the collar ;D

[chinook9]

The report indicates that there were 28 applications tested and one of the lines is blacked out. I expect that one vendor didn't like the results.

Anyone have any idea which one this would be?

[ViVek]

-{ Quote: "

Anyone have any idea which one this would be?" }-


Comodo

[Dark Star 72]

Here is the original list and info on the testing procedure:

http://malwareresearchgroup.com/?p=1517#more-1517

[CloneRanger]

Congrats to those that passed :thumb:

Interesting to note that all these products - Prevx, SafeCentral, Spyshelter, TrustDefender, Zemana - that are specifically designed to securely browser protect, did so.

It would be nice to know the attempted attack details, but they are arn't saying, for now anyway.

[CogitoTesting]

To me the surprise winner is NIS 2010 and congratulation to all those who won. :thumb:

Thanks

[Sveta MRG]

To confirm, Comodo was excluded from this test, we will give out details on our site later on.


The testing continues and when this round finishes we will release all the materials that we have, images, videos....

Just to give you a little heads up for DefenseWall HIPS, we tested version 3.00 not 2.56, it will be changed in the report in few minutes.

Regards,
Sveta

[Noob]

Wow, this time this was FAST :P

[Franklin]

Was Sandboxie setup where only your browser has start/run access and able to use internet resources?

[JerryM]

Norton just keeps plugging along and pretty well near the top in all the test I have seen latel.
Congratulations to Norton for a great AV.

Regards,
Jerry

[trjam]

Jerry, you should try Prevx, very light and et and forget. Plus it works.

[JerryM]

-{ Quote: "Jerry, you should try Prevx, very light and et and forget. Plus it works." }-

Thanks, Jeff. Not sure why I have never done so. I guess I have always thought I had such good security I did not need it.

Do I detect a new Avatar? Surely not.;D

Regards,
Jerry

[trjam]

Only this one. Personally, I am using it by itself.

[Kees1958]

-{ Quote: "Was Sandboxie setup where only your browser has start/run access and able to use internet resources?" }-

Franklin.

I would not think so. To me independant tests should be primarely intended for people orientating on a product to buy. Since you new to a product, it is 99% likely such a new user start at the bottem of the leaning curve, most likely with a default config. This is the reason most tests are performed with all default settings.

For testing software launched by a competitor or simular products, it may be a nice suggestion to open up a sticky where experienced users like yourself post additional settings to protect against these Proof of Concepts/Malware. In this way the knowledge of highly configurable software will be shared (from power users to medium expereinced and new users).

Regards Kees

[Mongol]

Hmmm...I don't see Avast in the test...??? :blink: :)

[whitedragon551]

-{ Quote: "Was Sandboxie setup where only your browser has start/run access and able to use internet resources?" }-

I was wondering the same thing. Sandboxie isnt meant to detect keyloggers or anything for that matter and seems out of place in that test.

[Dr who]

-{ Quote: "To confirm, Comodo was excluded from this test, we will give out details on our site later on." }-

Snipped from your site

-{ Quote: "Well the technical issue means that Comodo team was not willing to except the fact that their product gave no clear warning which would suggest blocking the threat. We didn't want to give them a pass and we decided to exclude them from the test as their behavior was going in the wrong direction.

There is also a Copyright issue involved as Comodo used our tool without permission.

One of the Comodo team representatives made some very hurtful and unprofessional remarks, until we get an official apology, Comodo will be excluded from all tests.

Regards,
Sveta" }-

Maybe you are too kind if they have become absusive.

A fail is a fail except when you remove them from the test group results.

This is not very professional Sveta if you are trying to establash yourself as a professional testing service:ouch:

[AvinashR]

-{ Quote: "Comodo team was not willing to except the fact that their product" }-

I guess its accept...

Anyways it seems that Comodo is going way to un-professional..This is not good for them..>:(

[andyman35]

-{ Quote: "I was wondering the same thing. Sandboxie isnt meant to detect keyloggers or anything for that matter and seems out of place in that test." }-
Nothing new about MRG comparing apples and oranges in truly unrepresentative tests.:P

[Sveta MRG]

-{ Quote: "Nothing new about MRG comparing apples and oranges in truly unrepresentative tests.:P" }-

We included SandboxIE in the test for two main reasons:
1)We received a number of requests to include it.
2)Because we have seen posts made in several security forums where individuals have stated that SandboxIE is all they use for online security – and some have specifically suggested that this will protect you from financial malware.

It is clear from this that many people do not understand SandboxIE and may use it thinking it will protect them from FM downloaded during a browsing session – when it will not.

We contacted Ronen before we published and discussed our reasons for including SandboxIE and we also agreed with Ronen that after giving the overview of SandboxIE, it would not be included in further testing in this project.

Regards,
Sveta

[andyman35]

-{ Quote: "We included SandboxIE in the test for two main reasons:
1)We received a number of requests to include it.
2)Because we have seen posts made in several security forums where individuals have stated that SandboxIE is all they use for online security – and some have specifically suggested that this will protect you from financial malware.

It is clear from this that many people do not understand SandboxIE and may use it thinking it will protect them from FM downloaded during a browsing session – when it will not.

We contacted Ronen before we published and discussed our reasons for including SandboxIE and we also agreed with Ronen that after giving the overview of SandboxIE, it would not be included in further testing in this project.

Regards,
Sveta" }-
My issue with the testing of Sandboxie is that average users that leave it set as default would not use it as a standalone product but as a supplement to an AV.Those power users that would just use SBIE would have configured it as per Franklin's post and set tight restrictions on what exactly can run and which resources it can access.

[Kees1958]

-{ Quote: "
It is clear from this that many people do not understand SandboxIE and may use it thinking it will protect them from FM downloaded during a browsing session – when it will not.

Regards,
Sveta" }-

Sveta I agree with you. I once posted "The damage of fanboys advises" in which I critised the near mythical support some applications got at Wilders.

From the applications I mentioned the SBIE and Comodo users responded most fiercely

Expect some critism and emotional reponses (when you not in favour you are against them) ;D

Regards Kees

[Scoobs72]

-{ Quote: "My issue with the testing of Sandboxie is that average users that leave it set as default would not use it as a standalone product but as a supplement to an AV.Those power users that would just use SBIE would have configured it as per Franklin's post and set tight restrictions on what exactly can run and which resources it can access." }-

So Sandboxie shouldn't be included because a small number of users may configure it in a way which would prevent FM from executing? Ermm...that is not a reason to exclude it.

[andyman35]

-{ Quote: "So Sandboxie shouldn't be included because a small number of users may configure it in a way which would prevent FM from executing? Ermm...that is not a reason to exclude it." }-
No you misunderstood my point there.
I'm saying that the test scenario isn't a likely real-world one,where an average user would have SBIE and nothing else on their system.In it's default configuration it certainly won't restrict the malware from running,however the bulk of users that would be using it alone will be techie types that'll configure it for higher security.With restrictions in place it would prevent FM from even executing.

At risk of inciting the wrath of Kees I'd just like it tested in a manner that most mirrors it's likely usage.

[Scoobs72]

-{ Quote: "No you misunderstood my point there.
I'm saying that the test scenario isn't a likely real-world one,where an average user would have SBIE and nothing else on their system.In it's default configuration it certainly won't restrict the malware from running,however the bulk of users that would be using it alone will be techie types that'll configure it for higher security.With restrictions in place it would prevent FM from even executing.
" }-

I understand what you're saying, but by following that logic Bufferzone, DW, Geswall, Prevx, Safecentral, Spyshelter, Trusteer and Trustdefender also shouldn't have been tested. Or is it ok that they were tested?

[whitedragon551]

-{ Quote: "I understand what you're saying, but by following that logic Bufferzone, DW, Geswall, Prevx, Safecentral, Spyshelter, Trusteer and Trustdefender also shouldn't have been tested. Or is it ok that they were tested?" }-

I dont think DW and Geswall should have been tested in this group. I dont know what the others are. But DW and Geswall are HIPS not keyloggers. If they were going to toss HIPS in they should have tested Threatfire and Mamutu as well.

[Scoobs72]

-{ Quote: "I dont think DW and Geswall should have been tested in this group. I dont know what the others are. But DW and Geswall are HIPS not keyloggers. If they were going to toss HIPS in they should have tested Threatfire and Mamutu as well." }-

DW and Geswall are policy based HIPS, but DW has specific keylogger, clipboard and screengrabber protection. Threatfire and Mamutu are behaviour blockers, not HIPS.

It seems everybody can make a case for why an app should or shouldn't be included.

[mvario]

-{ Quote: "I understand what you're saying, but by following that logic Bufferzone, DW, Geswall, Prevx, Safecentral, Spyshelter, Trusteer and Trustdefender also shouldn't have been tested. Or is it ok that they were tested?" }-

Well those other products make claims regarding the prevention of private information being sent or accessed. Sandboxie is pretty much designed for one thing, especially in default configuration, and that is to protect the system from changes made by programs running in the sandbox. Period.

[Scoobs72]

-{ Quote: "Well those other products make claims regarding the prevention of private information being sent or accessed. Sandboxie is pretty much designed for one thing, especially in default configuration, and that is to protect the system from changes made by programs running in the sandbox. Period." }-

And yet if you use Sandboxie in a particular manner, even in its default configuration it can prevent FM activity. So it can protect from this FM, but unless you specifically use it in a manner to protect you, it doesn't.
We can all keep arguing the toss on this issue, but it doesn't matter. It has been tested, it didn't prevent the FM activity, so now people know...just to enlighten those that didn't previously know that it wouldn't protect them.

[whitedragon551]

-{ Quote: "And yet if you use Sandboxie in a particular manner, even in its default configuration it can prevent FM activity. So it can protect from this FM, but unless you specifically use it in a manner to protect you, it doesn't.
We can all keep arguing the toss on this issue, but it doesn't matter. It has been tested, it didn't prevent the FM activity, so now people know...just to enlighten those that didn't previously know that it wouldn't protect them." }-

Ok so next time lets toss in Mamutu and Threatfire in. Even though they are behaviour blockers, they should stop FM activity or anything else that "could" stop it if configured properly.

[tzuk]

-{ Quote: "It is clear from this that many people do not understand SandboxIE and may use it thinking it will protect them from FM downloaded during a browsing session – when it will not." }-

It's true that Sandboxie does not try to detect key or data logging. However, prudent use of Sandboxie involves the "delete sandbox" function, especially when switching from casual browsing to banking.

Your test therefore would seem to represent someone who is using Sandboxie, but not it to its full extent. As such, it is probably not an adequate response to those who praise the robustness of Sandboxie, as they probably take a more rigorous approach to using Sandboxie, and delete their sandboxes often.

[andyman35]

-{ Quote: "I understand what you're saying, but by following that logic Bufferzone, DW, Geswall, Prevx, Safecentral, Spyshelter, Trusteer and Trustdefender also shouldn't have been tested. Or is it ok that they were tested?" }-
Bufferzone definitely not,DW and Geswall probably shouldn't IMO.Prevx should since Safeonline is specifically designed to prevent browser hijacking,etc.As for the others I'm not familiar with their workings so couldn't say.

[Kees1958]

Sveta,

Was DefenseWall tested in normal mode or the special anti-keylogger go banking/shopping mode

Regards

Kees

[Sveta MRG]

-{ Quote: "Sveta,

Was DefenseWall tested in normal mode or the special anti-keylogger go banking/shopping mode

Regards

Kees" }-


DefenseWall HIPS was tested in Normal Mode (default settings).

Regards,
Sveta

[JW Clements]

-{ Quote: "We included SandboxIE in the test for two main reasons:

Regards,
Sveta" }- But not Avast.
Was Avast deliberately excluded or just not included? Just another curious poster......???

[Sveta MRG]

-{ Quote: "But not Avast.
Was Avast deliberately excluded or just not included? Just another curious poster......???" }-

Avast will be included in our next round of testing.

Regards,
Sveta

[Noob]

I want to see the more detailed aspects ;D

[Franklin]

-{ Quote: "
It is clear from this that many people do not understand SandboxIE and may use it thinking it will protect them from FM downloaded during a browsing session – when it will not.

We contacted Ronen before we published and discussed our reasons for including SandboxIE and we also agreed with Ronen that after giving the overview of SandboxIE, it would not be included in further testing in this project
Sveta" }-
If you don't understand the concept or won't read about how to use Sandboxie then it won't protect against the test and all your test has managed to do is make those that dislike Sandboxie salivate with utter glee.

Yes you contacted Tzuk and he was decent enough to reply:
-{ Quote: "I was kindly contacted by the person running this test who informed they found that Sandboxie failed the test.

I explained that the concept of Sandboxie is that it doesn't try to detect threats, but rather it contains the threats in the sandbox, making it possible get rid of any threats very easily, by simply deleting the sandbox, before one proceeds to do a sensitive activity like logging on to your bank.

I asked them to run a second test for Sandboxie, one where they use "delete sandbox" to show that Sandboxie does pass the test, when it is used properly. Regretfully, the testers did not care to use the "delete sandbox" function when testing Sandboxie." }-
So you refused to use the simple measure of deleting the sandbox before proceeding with the test.

Maybe Tzuk should bring out a default Sandboxie where only browsers have start/run/internet access just to accommodate testers that have no idea of Sandboxie's capabilities.

[MikeNash]

A few people have contacted me asking "Why did OA fail?"

The simple answer is "I don't know." The tests results are vague.

It would be interesting if MRG were to say why they failed an application at least. This would increase the credibility of the results, as well as let vendors know - is there a useability issue, or a security issue, or both.

OA could have failed for two reasons:

a) OA simply failed, nothing detected, no warning.
b) OA threw up a prompt - which was deemed not informative enough - and it failed.

I'm hoping for case (b).

This sort of testing is a nice idea, but the methodology is a little weak.


Mike


Edit: Just read comments over at the MRG forum...

-{ Quote: "
Spycop can block this simulator, but the description in the alert to the user falls in to what we feel is a grey area.

If an application simply alerts the user with something like “code.exe wants to run with elevated privileges, do you want to allow it to run” – this is a fail as our methodology requires that a specific threat be identified and some specific statement given as to whether the user should run it or not – such as “code.exe is trying to capture keystrokes / inject a process (etc) this presents a security risk and we suggest you block it”

Spycop did identify a risk and did advise the user it presented a risk in some detail, but the risk it identified was for a global hook – which in itself, is not necessarily an indication of malicious intent / activity
" }-

Seems to answer the question.

[codylucas16]

Do people really care about these tests anymore? They've shown that they are not capable of properly testing applications and that they simply do not understand how the programs they are testing work. They also publish biased tests and change testing methodology during and after tests to suit their biases. I've looked at some of the tests they've performed and honestly cannot help but laugh. This recent test and the handling of Comodo really shows how "professional" they really are.

[Peter2150]

A few years back with the earlier concept of Prevx they found that 50% of the users failed to do the correct thing with pop ups. Now we find testers doing the same thing.

I am waiting for one of the test groups to fail a program because it failed to warn the user it needed to be installed.

[Triple Helix]

-{ Quote: "A few years back with the earlier concept of Prevx they found that 50% of the users failed to do the correct thing with pop ups. Now we find testers doing the same thing.

I am waiting for one of the test groups to fail a program because it failed to warn the user it needed to be installed." }-

Sorry Peter I don't understand what do you mean by need to be installed? The programs needs to be installed first no way! :gack: :wacko:

TH

[MikeNash]

-{ Quote: "Do people really care about these tests anymore? They've shown that they are not capable of properly testing applications and that they simply do not understand how the programs they are testing work. They also publish biased tests and change testing methodology during and after tests to suit their biases. I've looked at some of the tests they've performed and honestly cannot help but laugh. This recent test and the handling of Comodo really shows how "professional" they really are." }-

Yes, I think they do. Most people are not qualified to test applications and have neither the time, nor resources to do so. MRG, Matousec, Creer, Escalader and everyone else who makes good-faith attempts to test software perform a service for the community at large.

Reviewing the methodology, we have first of all the good:

1 - Testing on consistent platform.
2 - Not reusing the platform (install, uninstall, install).
3 - Testing with out of box settings - realistic. We have people who have used OA for some time, for example that didnt know about certain features.


The bad:

1 - There is no detail in the results. An application that completely fails is treated the same way as an application that fails due to a badly worded prompt.

2 - The prompts that may be badly worded introduces an element of subjectivity which is not open to scrutiny or comparison.

3 - The vendor feedback didn't happen in our case at least.

I can understand not wanting to circulate the test app (not sure how Comodo got hold of it, and that should probably be explained to avoid accusations of bias) - however, what this now comes down to is that we have a claim from a party to have tested a product, and gotten a result where there is no possible way to analyse or verify it in any way at all.

This could be fixed quite simply by providing more details of prompts that were, or were not issued by the security application.

I haven't dealt much with the MRG guys so I can't comment on arguments you raise about bias, but in my limited dealings with them on prior tests they seemed to be ok guys.

As for how they handle Comodo: Having previously had a few run-ins with Comodo myself I would say ignoring them is a pretty sound strategy.

I am curious however, why Comodo results were pulled after a disgreement with them on details that nobody else seems to have.


Mike

[Page42]

-{ Quote: "Sorry Peter I don't understand what do you mean by installed? " }-
I think he's being facetious, TH. ;)

[Triple Helix]

-{ Quote: "I think he's being facetious, TH. ;)" }-

So was I. ;D Can't help but laugh!

TH

[bellgamin]

-{ Quote: "Sorry Peter I don't understand what do you mean by need to be installed?" }-It was a satirical statement.

Inference (in my opinion) was: If a tester is too inept to use security software properly, he is probably too inept to even INSTALL it properly. I heartily agree.

IMO, this tester lacks the technical expertise needed to properly test these kinds of security apps.

[Triple Helix]

-{ Quote: "It was a satirical statement.

Inference (in my opinion) was: If a tester is too inept to use security software properly, he is probably too inept to even INSTALL it properly. I heartily agree.

IMO, this tester lacks the technical expertise needed to properly test these kinds of security apps." }-

I agree with you a was just taking the joke alittle further! ;) I fixed my original post sort of :argh:

TH

[EscapeVelocity]

I guess to pass this test you have to make your standard pop up warning sound a little more menacing.

[Triple Helix]

-{ Quote: "I guess to pass this test you have to make your standard pop up warning sound a little more menacing." }-

Like this: If you bypass this warning your system will be hosed! Is that about right?

TH

[MikeNash]

-{ Quote: "I guess to pass this test you have to make your standard pop up warning sound a little more menacing." }-

Now is the time to reveal our new OA popups:

tada!!!!

217510

[EscapeVelocity]

So I learned real fast that you must google the flagged file/process/service/registry item, etc....what you dont know. Then you will have learned something to boot.

I also noticed that there are 4 or so lists/databases, that come up in google again and again, when you google these. Perhaps you could put a link in your popup message that searches those databases and brings them up in your browser automatically. Or just links to one, or something. Like bleepingcomputer is one, I believe.

[EscapeVelocity]

If that is already been implemented in some products then pardon me, I am just a noob.

[Kees1958]

Well,

Testing standards transparency helps to understand the scores. Same as with business audits, when you do not tell before auditing what the reference standards are to meet, you will get a lot of discussion when presenting the audit/benchmark.

Regards Kees

[MaB69]

-{ Quote: "Now is the time to reveal our new OA popups:

tada!!!!

217510" }-

;D Mike, i think that i will infect my system myself to see again this cute notification

[andyman35]

If these tests are targetted at non-technical users,as seems to be the implication,then perhaps they'd be better served if it was explained to them that while 'Product A' failed 'Test A' with it's default settings it can be configured to block said threat.That would lend them merit as an educational resource at least.

[MikeNash]

-{ Quote: "If these tests are targetted at non-technical users,as seems to be the implication,then perhaps they'd be better served if it was explained to them that while 'Product A' failed 'Test A' with it's default settings it can be configured to block said threat.That would lend them merit as an educational resource at least." }-

That's a good idea - especially if told how to do it.

[Sveta MRG]

Hi, I have looked at some of the comments here and think there are a number of distinct issues. Firstly, the Comodo results and the rumours and accusations surrounding this.

Comodo failed as when originally tested the only alert CIS gave was that the simulator required elevated privileges. This is NOT an adequate warning.
CIS gave no other alerts and the simulator was able to compromise the system and capture all the user data entered in to the test site.
We liaised with Comodo and ran numerous tests for them with CIs in various configurations. We ran so many tests that in the end, they detected the simulator with the AV component of CIS.

We ran tests for them, ignoring the AV detection and our simulator was able to bypass CIS even when run in the sandbox.

We have all the chat logs for the discussions we had with Comodo and these prove they admit we bypassed CIS – even when isolated in the sandbox.
If you are at all interested in seeing the truth of the matter you can see some of the chat log in Chris’ post here:

http://forums.malwareresearchgroup.com/viewtopic.php?f=29&t=390&p=1408#p1408
(http://forums.malwareresearchgroup.com/viewtopic.php?f=29&t=390&p=1408#p1408)
In terms of us providing detailed results for each application tested, we will be doing this shortly.

This is a “project” and not a one off snapshot test. The results published are a baseline, testing starts on Monday and will be repeated every day. Details of alerts etc for each application will be provided in daily results and we will liaise directly with vendors.

Regards,
Sveta

[Dr who]

Hi Sveta,

Great to see that you are on a learning curve at MRG:thumb:

Can you please just answer one question with a streight honest yes or no ?

Comodo claim you (MalwareResearchGroup.com) changed the testing citeria wording( Clause.8 ) after running the tests.Is this correct or is Melih(Comodo) lying ?

http://forums.comodo.com/news-announcements-feedback-cis/why-is-comodo-not-included-in-mrg-test-merged-t55743.0.html;msg392718#msg392718

-{ Quote: "Look at their Methodology Used in the Test section on page 4 for their current version (clause eight)


8. It says: If the security application requests input from the user, it must provide a specific notice to block an action and or identify it as a specific threat. (an alert stating something is simply "unknown" or requires "elevated Priviliges" fails to identify specific threats or risks and so is ignored)


and the original one (attached) that we were sent along with testing result the very same clause eight said:

8. Testing is conducted with all systems having internet access. (this is the one we were provided originally )

So as per their testing methodology we passed....but in their minds we failed...so they went ahead and changed the testing methods "AFTER" the tests to justify their subjective result....now that is a disgrace, somehow they think its ok to test, fail then write the methodology to match the results!!" }-

[Sveta MRG]

-{ Quote: "Hi Sveta,

Great to see that you are on a learning curve at MRG:thumb:

Can you please just answer one question with a streight honest yes or no ?

Comodo claim you (MalwareResearchGroup.com) changed the testing citeria wording( Clause.8 ) after running the tests.Is this correct or is Melih(Comodo) lying ?

http://forums.comodo.com/news-announcements-feedback-cis/why-is-comodo-not-included-in-mrg-test-merged-t55743.0.html;msg392718#msg392718" }-

-{ Quote: "Where have we changed the methodology? They failed on the original test on the 18th, they were failed when we discussed the matter with them and they were failed when we published. The only thing we have done is add some explanatory detail in the report as to what is a pass and what is a fail." }-

Regards,
Sveta

[Saraceno]

No disrespect to Comodo users, and helpers, but Melih is under the impression he has a red cape attached to his back.

Get over it, you win some you lose some.

Comodo rep: "I still say Sveta, terming a product failure on the basis of alert interpretation is not great idea".

I disagree. Not every user out in 'internet world' has the brains to interpret alerts correctly. Using prevx for example, alerts are as clear as day whether you're an advanced user or a novice. If you're marketing your program to all users, alerts should be able to be deciphered/understood by all users.

[Cvette]

I'm willing to change to Outpost RC when it comes out. I love the COMODO Firewall and D+, but I have lost respect for them... :(

[Sveta MRG]

-{ Quote: "No disrespect to Comodo users, and helpers, but Melih is under the impression he has a red cape attached to his back.

Get over it, you win some you lose some.

Comodo rep: "I still say Sveta, terming a product failure on the basis of alert interpretation is not great idea".

I disagree. Not every user out in 'internet world' has the brains to interpret alerts correctly. Using prevx for example, alerts are as clear as day whether you're an advanced user or a novice. If you're marketing your program to all users, alerts should be able to be deciphered/understood by all users." }-

Yes, exactly. Alerts need to be clear so that all users are able to understand them.

I don't pay much attention at what Melih is doing or saying, I've seen it many times before:-\

[Cvette]

Sveta, I cannot believe the way he his speaking...

-{ Quote: "Own up to your mistakes Sveta, you will be a better man for it!" }-
-{ Quote: "You guys have messed up in a big way.

Apologise to the community and tell them how you will improve, then you will gain respect." }-
>:( COMODO is gone from my PC. And I certainly feel like leaving a piece of my mind with Melih.

[Kees1958]

Ahh well,

Anyone feeling hurt when addressed by Melih should watch his movies/blog.

They are a cross-over of mister bean and charlie chaplins modern times and really funny and entertaining.

Helps to put things into perspective.

Unless you are afraid that the empire will strike back

[Cvette]

-{ Quote: "Ahh well,

Anyone feeling hurt when addressed by Melih should watch his movies/blog.

They are a cross-over of mister bean and charlie chaplins modern times and really funny and entertaining.

Helps to put things into perspective.

Unless you are afraid that the empire will strike back" }-

You can always trust Kees to lighten the moment :argh: LOL!

[CJsDad]

Saw the thread over on the Comodo forum but things like IM's between two parties should be kept in private and not posted all over a message board for people to read just to prove a point but considering the source it's not suprising, definitely not a professional way of discussing the matter at hand, just my opinion.

[MikeNash]

-{ Quote: "Saw the thread over on the Comodo forum but things like IM's between two parties should be kept in private and not posted all over a message board for people to read just to prove a point but considering the source it's not suprising, definitely not a professional way of discussing the matter at hand, just my opinion." }-

Agree. But when one party publishes a damaging part, I think it implicitly gives the other party permission to publish too.

[CJsDad]

Mike

You are correct but two wrongs dont make a right. Anyway, that thread has went way down hill, it's useless at this point, just a bunch of bickering back and forth.

[Saraceno]

-{ Quote: "If these tests are targetted at non-technical users,as seems to be the implication,then perhaps they'd be better served if it was explained to them that while 'Product A' failed 'Test A' with it's default settings it can be configured to block said threat.That would lend them merit as an educational resource at least." }-

Didn't see your post, was at work and quickly glanced at the comodo posts. Good point.

[chennemann]

Wouldn't it be nice if both sides would learn from this instance and state publicly how they are going to improve. I am not following this close, but something similar to:

MRG - We feel that the warning message needs to show the possible threat clearly so that the average user would not continue, etc... Add any educational information they could add to help Comodo and the readers

Comodo - We feel that our software is working as intended and did pass the test. We are taking MRG's feedback and will adjust the warning or whatever other solution is appropriate. Educate their customers about how the warnings work and how to set up the system so they will be safe.

This seems so petty on both sides, considering they both want the public to respect them.

Chuck

[jmonge]

good advise che;)

[ALiasEX]

-{ Quote: "Wouldn't it be nice if both sides would learn from this instance and state publicly how they are going to improve. I am not following this close, but something similar to:

MRG - We feel that the warning message needs to show the possible threat clearly so that the average user would not continue, etc... Add any educational information they could add to help Comodo and the readers

Comodo - We feel that our software is working as intended and did pass the test. We are taking MRG's feedback and will adjust the warning or whatever other solution is appropriate. Educate their customers about how the warnings work and how to set up the system so they will be safe.

This seems so petty on both sides, considering they both want the public to respect them.

Chuck" }-Did you read this? http://forums.malwareresearchgroup.com/viewtopic.php?f=29&t=390&p=1408#p1408

If that really is how it went down then all onus is on Comodo, in my opinion.

[Noob]

Another big mess for Comodo ::)

Though, i like their software :)

[chennemann]

-{ Quote: "Did you read this? http://forums.malwareresearchgroup.com/viewtopic.php?f=29&t=390&p=1408#p1408

If that really is how it went down then all onus is on Comodo, in my opinion." }-

I did read MRG side. Even if one side is 100% right, they are both hurting their image IMHO.

Both want respect from the general public. It seems to me they care more about who is right instead of helping improve their product. It reminds me of two kids fighting over something really minor, just to prove their point.

Most professional companies that I have been associated with would not be involved in small issues like this. They would state their point and say how they are going to improve.

I really hope both companies continue improve.

Chuck

[ALiasEX]

I have read more at the Comodo forum. I have to agree "It reminds me of two kids fighting over something really minor, just to prove their point." (I was probably involved with something similar the other day, though.)

Melih of Comodo is by far the most childish, in my opinion.

[CJsDad]

The problem with this scenario is where it's all taking place, at the Comodo forums.
Technically its Melih's place where he can do and say what he wants with no retributions, who is going to stop him from posting or close the thread? Any mod over there going to step in and say "O.k. enough this thread is closed"? What they seem to be doing is acting like the loyal followers that they are and letting that childish, immature, thread run it's course.
If that thread was to start say here or another security forum it would have already been locked.

[adik1337]

I don't care about what Melih says ... I proven it myself ...sorry Comodo fanboys but for real, Comodo sucks bigtime!

[kjdemuth]

Now why would you say such things. Comodo doesn't "suck bigtime" as you say. I'm not a fanboy of comodo either. I like what they are doing and its unfortunate that some people are making it look bad. If you do have some solid proof as to why its 'sucks" please let me know.

[adik1337]

-{ Quote: "Now why would you say such things. Comodo doesn't "suck bigtime" as you say. I'm not a fanboy of comodo either. I like what they are doing and its unfortunate that some people are making it look bad. If you do have some solid proof as to why its 'sucks" please let me know." }-
start reading from page 1 :isay:

[kjdemuth]

I have been reading since the thread started. You still haven't given me a concrete answer as to why, in your very small knowledge, that comodo "sucks". Clearly you haven't been around for the discussions of why you can't judge a piece of software on one test. If that was the case then Online Armor sucks too. I would think that alot of people would disagree with that. So please don't throw around comments loosely.

[Noob]

-{ Quote: "I have been reading since the thread started. You still haven't given me a concrete answer as to why, in your very small knowledge, that comodo "sucks". Clearly you haven't been around for the discussions of why you can't judge a piece of software on one test. If that was the case then Online Armor sucks too. I would think that alot of people would disagree with that. So please don't throw around comments loosely." }-
You know some people just hate things :P

And i'm not a exception, but not in the security world, for me here everyone has a chance ;D

[sded]

I thought this https://forums.comodo.com/general-discussion-off-topic-anything-and-everything/the-reason-for-leaving-experience-t55867.0.html was an interesting thread from one of the Comodo mods who used to post here. Sounds like we might see him again-good luck to him. :)

[kjdemuth]

-{ Quote: "You know some people just hate things :P

And i'm not a exception, but not in the security world, for me here everyone has a chance ;D" }-

Thanks noob,
I hate to lash out at people but come on....
I'm a die hard a-squared fan and it didn't start trashing it when MRG test had it missing 945 wild results. I've used comodo firewall in ver 3 and it was great. I would use it again, even though these tests say otherwise.

[AvinashR]

Guys cool down, you guys are creating Global Warming here !!:D

[JRViejo]

Removed several Off Topic posts. If personal attacks continue, this thread will be closed!

JR

[AvinashR]

Here he steeped in !!

Thanks JR. BTW i don't know why you guys getting so much personal on just a test results? We should take every tests positively. I never take these test seriously, every security product or company can't be 100% right all the time. Always take every test in a positive manner. Why bashing all the time?

Grow up guys !!:D

[Narxis]

-{ Quote: "I have been reading since the thread started. You still haven't given me a concrete answer as to why, in your very small knowledge, that comodo "sucks". Clearly you haven't been around for the discussions of why you can't judge a piece of software on one test. If that was the case then Online Armor sucks too. I would think that alot of people would disagree with that. So please don't throw around comments loosely." }-

The problem is not the test, the problem is how the guys at comodo react to the test.

I think Comodo is still a great free product, but they should do less whining...

[kjdemuth]

I wasn't bashing anyone. I was defending myself from a personal attack. There is no reason for someone to be coming here and throwing around inaccurate comments. If you can back up your comments with answers then yes by all means comment. To start talking jive and not being able to give someone some reasons in childish.

[AvinashR]

-{ Quote: "I wasn't bashing anyone. I was defending myself from a personal attack. There is no reason for someone to be coming here and throwing around inaccurate comments. If you can back up your comments with answers then yes by all means comment. To start talking jive and not being able to give someone some reasons in childish." }-

??? ??? ???

Saying all to me? :(

I am sorry if i did a mistake....:-[

[kjdemuth]

No just commenting in general. I don't want people thinking that I go around causing problems. I like to think that I do contribute positivly to the forum.

[hayc59]

-{ Quote: "To confirm, Comodo was excluded from this test, we will give out details on our site later on.
Regards,
Sveta" }-

Sveta, thank you and this should be interesting read!! *puppy*

[hayc59]

-{ Quote: "I have been reading since the thread started. You still haven't given me a concrete answer as to why, in your very small knowledge, that comodo "sucks". Clearly you haven't been around for the discussions of why you can't judge a piece of software on one test. If that was the case then Online Armor sucks too. I would think that alot of people would disagree with that. So please don't throw around comments loosely." }-
I can give you several reason why Comodo and Melih suck decieves and cheats folks and show you several threads why! but they have been posted over and over again..just look at the first thread in 'time out room'
bottom of wilders forum!! just for starters

[kjdemuth]

Yes I should be very interesting. I had followed one of the links to the comodo forums. Apparently there are 2 mods that were leaving. They sited that comodo wasn't responsive to when they pointed out bugs. They also said that the realeased version that they knew were flawed.

[kjdemuth]

Yeah I noticed that too. Decieves is a better way of describing it. Its too bad that they choose to be that way. CIS is a good piece of software. It could be great if it were developed properly.

[hayc59]

eXPerience dude actually wins
in this, he is free to help others with better
morals and better softwares! I am sure there is not a forum around
that would not welcome him!!

[kjdemuth]

Maybe he will find his way over here. Its too bad that they treated him like that.

[adik1337]

-{ Quote: "Yes I should be very interesting. I had followed one of the links to the comodo forums. Apparently there are 2 mods that were leaving. They sited that comodo wasn't responsive to when they pointed out bugs. They also said that the realeased version that they knew were flawed." }-
so at last you found the reason why I said comodo "sucks"? ... I wasn't bashing you personally ... I was bashing comodo and then you said this:
-{ Quote: "Originally Posted by kjdemuth
I have been reading since the thread started. You still haven't given me a concrete answer as to why, in your very small knowledge, that comodo "sucks". Clearly you haven't been around for the discussions of why you can't judge a piece of software on one test. If that was the case then Online Armor sucks too. I would think that alot of people would disagree with that. So please don't throw around comments loosely" }-
but anyway ... if you like comodo that much goodluck to you.

[AvinashR]

You will face such troubles when you do something good...I took MRG report in positive manner.

May be i am bit late but one thing i would like to say "Good Work Sveta"..:D

[adik1337]

good work to MRG team indeed ... thanks for not being the next matousec

[jmonge]

this tests are very helpfull to see how security developers are doing under the hood;)