Should we do a security competition?

Discussion in 'polls' started by Hungry Man, Aug 2, 2011.

Thread Status:
Not open for further replies.
  1. Ranget

    Ranget Registered Member

    Joined:
    Mar 24, 2011
    Posts:
    846
    Location:
    Not Really Sure :/
    :) :rolleyes: :rolleyes: :rolleyes: :rolleyes:
    :eek: :eek:

    i will Join the comp
     
  2. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    Well that's good to know =p when there's enough support we can make a discussion thread.
     
  3. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    The malware samples could be a problem. They don't allow links to real malware here. Without a legitimate way to link to the test samples, a 24 hour time frame will be hard to work with.

    If this doesn't conflict too much with real life, I'm interested to put my default setup to another round of tests. Been a few years since it's been tested hard. Can we aim for a weekend? In order to test a default-deny setup properly, we'll need malware that exploits applications that are normally allowed to run. Clicking on an infected installer is pointless with default-deny as it won't be able to execute. Samples that target applications will need to exploit something besides Internet Explorer or those of us who have stripped it out will have quite an advantage.

    I assume we can't run a virtual system (Virtual PC, etc) and judge our defenses based on what escapes the virtual guest and alters the host system. That in itself will defeat almost everything.

    Regarding testing the results, we could start with before and after system snapshots of the file system, registry and services. AFAIK, Inctrl5 works on XP and can create reports that list all of the changes. No idea what will do the equivalent job on Vista/Win-7 or on 64 bit systems.

    As for categories, lets keep all of the Windows versions and all of the different packages together. Since the OS is just as much a part of the security package, keeping them all together keeps the playing field level and allows each OS and security package to stand on its own abilities and strengths. I suspect that the results will be an eye opener, especially to the "get with the times" crowd. I also don't see a point to separating free and paid packages. There's free security software available that's every bit as good as the paid software. This would also leave a big "gray area". Example, the paid version of SSM is now freely available. Which category would it go in? There's also users who use both in the same package. Except for those running just a free AV, this test will show that "You get what you pay for" doesn't apply to software.

    I'll be entering a clone of my stripped down, freeware and abandonware protected default system.
     
  4. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    PM's never seem to be enforced, it's just publicly posting the malicious links.

    Sure, makes sense.

    Yes, I think the majority of malware would be 0day programs and exploits but we may also handpick a few oldie but goodies to see how they work.

    Default deny may be a very powerful security method but it's definitely not the easiest to use.

    Yes, Java and Flash exploits will certainly be necessary. Browser exploits are all but useless for Chrome users and they'll be difficult to find for Firefox 4+ users.

    Makes sense.

    I'm not sure about combining free and paid. I see your point though, I'd like to hear more input.

    I'm not "in charge" or anything and as far as I'm concerned this is a community project. So if you guys want it one way, that's the way it'll be.

    Sounds good. I'll probably be entering a similar setup to my own. Just slightly different.

    Thanks for all of that input.

    I don't think we need to rush this project. We should take our time and make sure the tests are done properly.
     
  5. Sully

    Sully Registered Member

    Joined:
    Dec 23, 2005
    Posts:
    3,719
    If you plan on actually doing this "test", at least do it right.

    Malware or virii are not picky, the are equal opportunity exploiters :D As such, why should those who get involved in such a test compete against each other? Why should there be an "even" playing field? Paid or free, default deny or not, Admin or User, use what you have and see what breaks through. After all, that is how it works in real life.

    I would propose those interested gather on a specified 24 hour period, and everyone begins. Test a link with your chosen browser, test a .pdf file with your reader, etc etc etc for all the exploits being found to test with.

    In this fashion there is no winner, only "I was successful on all but exploit X and infected file Z". This way you don't get that "competitive spirit" vibe going on that ruffles feathers, and as each posts his/her results, you can have a concise "this is my results" portion and a "this is my config" portion. This would be of use rather than a trivial test for bragging rights, as others who don't want to participate could look at the results, see something they are intersted in "why" it passed/failed, and compare the "security" to others that had a similar or dissimilar reaction.

    This also dismisses the need for a judge. I don't see how you can ever possibly quantify a way to judge this sort of thing unless everything is exactly the same except for which AV you use, or which HIPS you use, etc. Some peoples services alone can be vastly different, among other things.

    The advantage of the proposed test is that most of us have been communicating with each other for a given time period, and I believe none of us are really trying to blow smoke to each other, but rather learn and share. As such, with a decent number of participants, the whole community can study the results and be somewhat sure that it isn't biased. If you insist on keeping it a competition, pitting my security against your security, then I for one won't give the results much credence as I am not sure I trust the results of the "winner", because even honest people can be very tempted to compromise thier values to win.

    Sul.
     
  6. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    The "judge" would be for the subjective tests such as "ease of use."

    The competition aspect is not set in stone. No part of this is. I just figured it would give a fun edge to it.

    This is what I was proposing. We all test the same links/ malicious files/ exploits. Obviously a pdf exploit would open in the pdf reader etc.

    Yup. That's why I was hoping for video proof.
     
  7. Page42

    Page42 Registered Member

    Joined:
    Jun 18, 2007
    Posts:
    6,941
    Location:
    USA
    You know, that Sully just keeps on making sense, doesn't he? :)
     
  8. Dark Shadow

    Dark Shadow Registered Member

    Joined:
    Oct 11, 2007
    Posts:
    4,553
    Location:
    USA
    I agree 100 percent,makes perfect sense indeed.
     
  9. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    By this I meant that all the participants should get the same samples, no matter what their setup or security policy. This way the results would have more meaning. I suspect that there are several members, each with very different setups, that will be completely unaffected by the samples, but not for the same reasons.

    I wasn't viewing this as a competition, but more of a testing of security policies and their implementation, and how well they hold up against the real thing. If we do this properly, the results should be useful to others, and should help them with comparing different security policies and seeing what is involved in correctly implementing them.

    Regarding the videos, if we're doing this, the details need to be covered. Example, one video for each malware or one video with all of them? What format? The closest thing I have is Wink, and I've never used it.
     
  10. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    We would need someone to get the proper samples.
     
  11. Page42

    Page42 Registered Member

    Joined:
    Jun 18, 2007
    Posts:
    6,941
    Location:
    USA
    Yes, some of us, if this endeavor does get off the ground, are not going to be able to participate, despite how much we may want to, due to logistics of the operation. I don't have a test machine... I have imaged backups I keep handy if needed. But I'm not set up to intentionally run malware on my computer. Then there is the recording of it. All new to me.

    This is why I said a couple of days ago, I will be limited by the time involved and the expertise needed. :(
     
  12. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    Totally understandable. We can also bring in other members from other forums, I'm sure users at malwaretips would enjoy it.
     
  13. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    18,278
    Location:
    New England
    Re: Should we do a security competition using live malware samples?

    I've been taking serious consideration of what is proposed in this thread since shortly after the thread was started. The staff has also discussed it some in private. Unfortunately, what I have to say is not going to be what most of you want to hear. I just don't see any way that this can go forward here, as proposed. I know many of you won't understand, so, I'll try to explain my reasoning.

    It's easy for those of you who are not potentially in the hot seat, to complain that we have "silly rules" about requests for malware sharing or regarding the posting of home-grown "malware testing" threads. You aren't likely to be concerned with the rules we must abide by.

    One, but not the only key rule we are bound to, prevents us from endorsing or even allowing malware trading to occur here on the forum. It is based on the following principle.

    Malware must not be shared with people outside the professional industry with which we are associated, i.e. the anti-virus/anti-malware industry. Paul Wilders worked in the industry for years, and long ago subscribed to this ethic. Therefore, we have our rule against the sharing of malware.

    Here's a test you can do in order to see if we're alone in taking this position. Go to any one of the industry insiders who are members here, (those with the color coded, expert and specialist titles), and see if they'll send you malware samples. I'm talking about those guys from the major AV or AM companies. Try to get the guys from Avira, BitDefender, Panda, Avast, Symantec, or the AM guys from MBAM, SAS, and TH, not to mention Eset, Prevx and Returnil, to send you a sample of live malware because you want to test with it. It's extremely doubtful you'd get anything but a polite "Sorry, but I can't" reply. Heck, it might even mean their jobs if they got caught send malware samples out to unknown persons.

    That same ethic is applied to us and that's the position we must take when dealing with this type of issue on the forum. Not to mention the legality question. One issue is with Webhosting contracts. Almost all professional hosting companies, certainly in The US, Canada, AU, most of the EU, and many others, have TOS and AUP on all their hosting agreements that disallow hosting or propagating malware, warez and other questionable objects. Our hosting company also has such a policy.

    Sure, there are people running forums that allow malware trading, live links to malware on public posts, and maybe even host malware and warez downloads on their servers. But, it is highly likely that they are doing it in violation to their hosting agreements, and simply haven't gotten caught. (There was one case a while back that some of you will recall, where a website was supposedly hosting warez and/or had large lists of warez based links in public posts, and, an AM company reported them to their hosting company which got them taken offline. I believe they ended up moving to some other hosting company, perhaps in another country, that didn't care whether such content was hosted. Now, maybe they no longer have such content, or, maybe it was all questionable and in doubt at the time, but, that's not the point here. They were thought to be in violation of their contracted hosting and were terminated because of it. These are real rules and they "have teeth.")

    So, while our rules may seem silly to some of you. And, you think, "Hey, everyone else allows this. Why doesn't Wilders Security?" It's because we try to follow the rules, whether it's likely we'd ever "get caught" for not doing so. Do we miss some requests like this though? Of course, we do.

    We try to remove all requests for malware samples, but, we don't catch them all. Sure, in some ongoing thread, a member makes a post like, "Hey, can you send me that link (or sample), and I'll check it for you?" And, guess what, we missed it. Later, when the thread has moved on, it seems pointless to remove it so long after the fact. But, we do the best we can. And, when we catch a fresh one, we do remove it. If you don't think we do, just ask the many members here who've gotten royally ticked off at us because their request for a PM sample got deleted. One particular member had this happen to him in just the last 24 hours. He was not happy, to say the least.

    Regarding this particular thread. Do you believe that I don't think there is some merit in the concept being proposed here? Like many of you, I think this could be useful, provided the points Sully made above were the focus - not a "competition for bragging rights" but, a helpful "what worked and what didn't" type thread where people learn from the "didn't work" and figured out how to "make it work." The problem is that this involves not just some missed malware request posting, but, involves a deliberate and scheduled malware sharing effort. At some particular point in time, some member who has found some malware, is going to either post something or PM all those involved with the link(s) to be used. This very aspect of it means that it'd have to be specifically allowed by forum mgmt, since it involves so big a coordination effort. And, that is a clear violation of the rules I mentioned above. It's simply not possible for me to allow that given what I must abide by operating this forum.

    So, what am I supposed to do? Pretend it's not happening and be shocked after the fact? I'm sorry, I can't work that way. If this thread's concept moves forward to actual action, clearly that point is eventually coming. I know it. You know it. So, it simple can not be done - well, at least not using malware samples.

    The only workable alternative I can see, which can be done, but, may make the testing less dramatic or exciting for people, would be to use some legal software product as the sample. A commercial keylogger, a testing simulator, or one of those suite testing toolkits some AM companies publish to show effectiveness of HIPS, firewalls and so forth. That would not break the rules involved here. Of course, as I said, maybe that makes it less exciting for those participating, weakening the purpose and making it not worth doing. Only you folks can decide if you'd want to participate in something like that, instead of live malware testing.
     
  14. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    Just to clarify: I don't think that malware-sharing being against the rules is silly.

    And thank you for the long and detailed response.
     
  15. dw426

    dw426 Registered Member

    Joined:
    Jan 3, 2007
    Posts:
    5,543
    It is what it is, I don't think a reasonable person would be upset after hearing your explanation. At least we now know that talk has happened behind the scenes. Thanks for taking time out to explain the position clearly :thumb:
     
  16. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    I'm definitely not upset. I figured it would likely not be allowed due to the distribution of malware/ malware lists/ sites hosting malware.
     
  17. Page42

    Page42 Registered Member

    Joined:
    Jun 18, 2007
    Posts:
    6,941
    Location:
    USA
    Yes, you can see that LWM (and staff) devoted considerable time to pondering this venture. Offering a detailed analysis as to why it won't be permitted was quite admirable.
     
  18. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    :D
    Seriously, can another location can be found.
     
  19. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    You would have to find a site that's ok with it. I don't really know what sites are and aren't ok.
     
  20. Sully

    Sully Registered Member

    Joined:
    Dec 23, 2005
    Posts:
    3,719
    Why don't you create a throw-away email account, create a document with links to all the tests/links/files etc, share that doc (like google docs or something), and have everyone do the test based off that material. You can still post the results back here in a proper thread.

    When you are done, don't use the account again. Or something along those lines, there are many options. You don't have to rely on the sensitive matter being hosted here, and if it all done through email/docs, where you are only giving links, there is nothing really to break TOS, as you aren't actually hosting/sharing the files, just links.

    After done with testing, as long as you don't post the links in the test, you can describe your tests to your hearts content.

    I would give a suggestion though, that you create a form for the results and a method for the test. Sort of like "test link A first, then link B, then while link B is open, press button X or start program Y", so that all tests are done in an order so when a failure comes, the cumulative results of all tests findings can help to pinpoint where flaws erupted from.

    Likewise, a form for submitting your results would be nice, so that everyones format is very similar, allowing onlookers (and other testers) to be able to view things in a concise manner rather than willy nilly for each report from each user. (nothing against anyones way of formatting such a thing, but it would make it easier ;) )

    As for the regulations imposed here, I think most everyone knew it was coming. Frankly, I was suprised no one mentioned earlier on to take the actual testing phase off-site and only posting results back.

    Sul.
     
  21. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    The Google Docs idea is very smart. We would still need some people to sort through which malware is viable for testing. No point giving people a bunch of duplicates and no point giving people something that's broken.

    Definitely.

    I'd assumed haha I just hadn't given much thought to the project in general.

    I'd like a mod/admin to sign off on this before we go any further. We do still need some people who are willing to volunteer to test malware and see that it's working/ not down.
     
  22. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    18,278
    Location:
    New England
    Unfortunately, that's not really better. While that "might" reduce the Hosting company issue somewhat, since the malware isn't passed directly through our server, it does nothing for the ethic mentioned as the main reason for the rule. This forum would still be the coordinating point for the sharing of malware samples.

    Doing a "once removed" type thing is simply a way of trying to side step the rule by pushing the data exchange one website away from here. Whether its email or a link over to "Bob's Forum" where the list of links would be posted. Everyone still gathers here. They coordinate the plans and timing from here. Someone still finds the malware and asks everyone here who is participating for an email address, which they'll later use to transfer malware links or samples to them.

    Yes, it's a wink and a nod degree of separation, but, without Wilders Security, the people could not have gathered, the coordinating could not have happened, the signing up to recieve the samples could not be done, the results could not have been posted and jointly discussed... In short, it would still be our responsibility of encouraging and allowing malware trading to occur from here.
     
  23. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    Eh, well that's that.
     
  24. Sully

    Sully Registered Member

    Joined:
    Dec 23, 2005
    Posts:
    3,719
    Let me see if I understand this, because I find it a bit ambiguous.

    It is OK to discuss how we mitigate threats or what happened when we were attacked by MalwareX.

    It is not OK to share those links or provide malware samples. (makes good sense)

    It is not OK for members here, to discuss a test, and go somewhere else entirely to do that? (huh ? )

    I don't see the relation to Wilders providing anything to the set of individuals doing the test, other than a meeting room to decide if they would like to do something, and to post the results.

    Don't get me wrong, rules are rules, and they are there for a reason and everyone should be held accountable. Thats the way I like it anyway. I do have to scratch my head though and wonder just where Wilders has any responsibility for what a set of given users does away from here. In a way, it makes me think that even discussing malware in detail would fall under the same umbrella.

    I admit that for some reason it really irritates me. To have a place like this, with so much promise for such a test as is being discussed, only to be told we can't do it, even though it needn't include Wilders at all except for discussing it. I guess it irritates me because it is like the doctor (the official testers of malware) telling the patient (us users) they cannot diagnose a headache because they aren't doctors, and they cannot administer aspirin for the said headache, because, they aren't doctors. I would have thought, following ethics, Wilders would not promote or endorse malware, not share and make available samples, but would be THE place to go when a set of focused users wanted to do a test like this. o_O

    This is just one of those things that doesn't make much sense to me, but then, the world according to sully might not be a good thing either ;)

    Sul.
     
  25. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    If it would be better we could simply host everything, including discussion, off site. But I'd like there to be a discussion of the results on wilders and I'd like it to be clear that Wilders members can take part in this test.

    We could see if malwaretips has the same rules (I would not be surprised.) Or perhaps an unrelated forum.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.