Should we do a security competition?

I see a lot of tests get scrutinized here and I see a lot of people putting their own security setups to the test.

I've never seen a proper test of combined antimalware. I think it would be interesting to look at the entire "suites" that people have put together.

Here are my ideas on how it would be:

I personally think Windows, Linux, and OSX should be in separate categories OR we should simply have this as a Windows competition.

Any ideas? I figure with the cumulative knowledge on here we can come up with a fairly good test.

We would also need some impartial judges with moderate/ very high computer knowledge.

 

I think we would be judging in categories, not just by "who is protected the most."

1. Protection against malware
2. Ease of use/ maintenance
3. Ease/ Time to set up
4. System effect/ "weight" on your resources

Can't think of anything else to judge it on. But each setup would get points in the other categories and be the winner of the category and then one setup would be the overall winner.

 

And all the videos should be uploaded to the same site, like YouTube or similar?

I'm not going to participate I am just wondering

 

Well it wouldn't really matter. But youtube would probably be the easiest. If we did youtube we could have a public vote on it.

 

That's what I meant, that it would be much easier for everyone who would like to watch them to have all videos uploaded in one place instead of several sites.

 

Well if people on here wanted to have the competition there would be a topic in which we got it all set up/ the vids would be posted.

But youtube would probably make thigns easier.

 

Hungry Man,

You've hit upon a very interesting idea here, imo.
From my perspective, I would be willing to put my setup to the test... I would welcome the results... not due to arrogance, but to curiosity.

Problem is for me, twofold (at least).
One, the time involved and
two, the expertise.
By expertise I mean setting up my machine and whatever else it takes to pull it off.

I humbly offer myself as a judge, however, based upon my qualifications on the lower end of your prescribed scale...

 

I guess judges should be nominated/ elected by other users. I really don't know lol not that far ahead. I just want to see if people are interested in the general idea first.

 

I know. I was just going for the laugh.
I hope your idea can get some legs, though.
It would be neat.

 

=p I think it would be a fun way to see which security setups are best for which reasons.

 

With a default-deny security policy in place, there wouldn't be much to see. Just an "access denied" message.

 

Default-deny would give you a great score in terms of protection but not necessarily for ease of use.

 

For "ease of use", normal usage will need to be defined. In daily usage, it doesn't behave any different than another setup. It's only when you try to do something that changes the system that the policy becomes inconvenient. Mine would lose out under "ease/time to set up". The initial setup was time consuming, but after that, it's done until you change or update something, which I seldom do.

 

Ease of use would have to be "benchmarked" in a certain way. Like running some random portable application, which is unlikely to be whitelisted. Installing a well known piece of software. Simple opening of browser/ word document.

At this point I'm just curious to see whether or not people actually want to do this

 

I'd suggest there are two category:

one for FREE setup
one for PAID setup

 

Interesting, lots here test their set-ups against malware every day so I'm sure more than a few would upload results but to what end? The endless debates here would more likely be inflamed than resolved I would imagine.

The 'judges' would have to have criteria. 'What is an infection' (what constitutes malware even!) could cause more debate than the tests are worth. For example is a machine running an active malware sample under Defensewall protection, in the Comodo/Avast/KIS sandbox or otherwise restricted actually infected? Different views on that I'd imagine.

Similarly, what about traces left behind, are they infections? Who has the expertise/authority to establish what is/is not malicious detritus from malware? Who decides a) what the malware is designed to do and b) whether it achieved that?

It is fairly straightforward to run some samples and if they are blocked give a pass and if not a fail but simply allowing malware to run or the presence of excutables or leftovers on your system does not automatically mean infection.

If things like that are resolved it could work. Interesting what the mods would think though.

Cheers

 

@ Hungryman

Sorry for my late reply, I've been busy. If life allows me some time I could upload my own results on youtube for everybody to view soon.

Thanks.

 

I agree.

These are exactly the questions I want to put to Wilders members. We need clear cut rules and definitions for tests.

But all of that can come after we actually decide on whether this is happening or not, and then we can find judges who everyone agrees are competent and well established users.

Thanks for the support.

I know I personally test my setup. I believe others do as well. All this would be is testing your setup but with predetermined malware and while running a screen recorder.

 

I think it's a great idea,but to be fair I think the test should be seperated.Example, how could one compete using standalone AV vs another using hips or sandboxing. just some food for thought.

 

Well the idea is to compare whole setups. If someone is only using a single antivirus... that's all there is to it, their scores will be determined by how well that single AV works.

 

so you do your test with your av today on 10 viruses and i will do mine 10 days from now (`cause i don`t have the time to do it right now). your av will catch, let`s say 7 viruses, and my av will catch them all, `cause in this 10 days that passed the virus database of my av will grow and this viruses will be in the definitions....is this a fair test??
EDIT: the viruses must be the same, right?

 

All tests have to be recorded and submitted within 24 hours of the released malware list, which will be PM'd to all listed participants. This will negate, at least partially, the fact that an AV will catch more malware a week later than it did a week prior.

Yes, everyone uses the same malware/ exploits.

 

@ Hungryman

If I have to do my test let say this coming Saurday evening or early Sunday morning when should I expect my own MDL list since these malware lists are being constantly updated?

Thanks.

 

got it,thats cool.who going first.

 

No idea yet. I still want to make sure the mods/ admins are ok with it so we don't get shut down halfway through and I'd like to see more users supporting the idea.