Criteria for making security claims

Discussion in 'other security issues & news' started by Mrkvonic, Dec 18, 2007.

Thread Status:
Not open for further replies.
  1. Mrkvonic

    Mrkvonic Linux Systems Expert

    May 9, 2005

    I would like people to help me with something that has been slightly bothering me recently.

    What kind of criteria (professional / personal / otherwise) do people (you) use to substantiate the logic of your threads and posts regarding security topics?

    I'm asking this because I have recently read several topics that do not feel quite right, yet seem to be taken in the most serious of lights. Personally, I think such ideas only fuel the fear and confusion rather than actually adding to the understanding and possible solution of the problem.

    Here are a few simple examples:

    1. The claim of the most secure browser

    - How can one validate his/her claims? By merely repeating the said? By using numbers? AHA! Here we have the classic problem. Numbers can be used any which way. Some like them in quantities, like Microsoft, who will tell you the more the merrier. Thus if a product X has had 10 vulnerabilities but the product Y has had 20 vulnerabilities, product X is safer.

    - Proving the negative ... how can we do that?

    2. Hacking

    - I have read that someone suggested a computer can be hacked even if it's turned off. Now, this is very interesting, because it suggests that 3-4kg of metal are actually someone alive beyond the electricity that powers them, assuming magical ghost IPs or what not. What kind of impact does this have on a newbie for the first time here, reading that his rig can be hacked even if it's turned off?

    3. Misconceptions - user vs. program

    - The fact many a man has fallen down the pit of misusing software to obtain other software for free when this should not be so does not make the tools bad - it makes the user bad. But for some reason, p2p and various contents on the web are blamed - instead of people. Which is pretty much like blaming guns or cars for the deaths they cause.

    - This has transgressed into an almost cult, where certain words / concept have become almost illegal / criminal / dangerous in their meaning.

    So ... after this lengthy intro:

    Could you please share your insight how you build your logic, what you base your claims on. I'd like to understand the mechanisms behind people's thoughts. What drives them - security wise. Some people I understand completely - and I guess they understand me equally well. But some ... it's a pure mystery to me.

    Should the claims be professional - 100% objective - and based on actual knowledge / skills with software?

    Should they also involve personal elements? While not written in code, they might give a good indication of what to expect - a sort of a rule of thumb.

    What is a good, reputable reference? Do we average over many people or stick with single claims?

    How to sift good from bad - or pure FUD from actual business?

    Where does fear begin and logic end?

    What level of responsibility should one assume when writing something?

    I hope this can bring about a fruitful discussion.

    Cheers all,
  2. beethoven

    beethoven Registered Member

    Dec 27, 2004
    Mrk, an interesting thread though I am not sure you will get a "real outcome".
    I think you will find a wide range of posters on this forum . The range of technical and/or practical experience will vary widely and consequently the quality of the posts or the advice provided may vary. I would consider myself interested in programs that help me with my daily work, that keep my pc free of problems and that's why I come here to learn and to gain insight. As I know that my knowledge is limited , I usually refrain from posting advice or answers, still over time I have developed some idea of which posts are factual, technical and helpful. Some other posts are more along the lines of my AV is better than yours but even these sometimes have interesting debates. Often even those posts that may not be totally correct or factual are useful as they either provide links to programs I was not aware of, provide first hand feedback on how some users experience a certain program or the support or bring out more experienced posters to correct or query obvious mistakes. After lurking on these boards for a while, you develop a feeling on how to take answers provided based on the contents and tone but often you also "know" the posters and have already formed a judgement. Not much different as in real life - you know your friends and who to ask for advice on money or who to ask for horse racing tip.

    Most members of this forum probably have opinions on the safest browser but at the same time would realise that the answer to this question is not as clear cut as a simple mathematical equation.
  3. BlueZannetti

    BlueZannetti Administrator

    Oct 19, 2003
    I typically approach it as I do any other technical or engineering problems - deconstruct the situation and break it down into manageable pieces. A lot of times that takes a large black box and makes it into a whole bunch of little black boxes - and that's a simple reality of this situation - we have end use applications in which details of the specific inner working are often held as proprietary and for the most part these inner details will remain obscure.

    However, black boxes can be probed in terms of functional traits via challenge-response experiments with the internal consistency of results from those tests constantly assessed and re-examined. The challenge-response can be anything from using the product to assess the user interface to directly challenging claimed functional traits.

    I think the best one can hope for is that claims will be based on actual experiences and observations. Others can assist in placing an interpretive wrapper around those experiences and observations based on more detailed technical knowledge or insight. Problems emerge when folks confuse observation with interpretation and start to mix the two.

    Community consensus can be a good tool to use, and it is often a more robust indicator, but it is only as good as community itself. The consensus of a group of paranoid individuals is unlikely to be rational.

    There are a few very simple steps anyone could use....
    • Try to base statements on personal knowledge. One's interpretation may be wrong - and one needs to be willing to accept that - but the observation should stand even in the face of a flawed interpretation.
    • Most of the outrageous statements I've seen here and elsewhere start with either a confusion between observation and interpretation (my system BSOD'ed after an installation of "X" vs. "X" is a trojan infested work of the devil which has placed firmware based malware on my machine which I can't remove) or a casual repeat of implausible statements with absolutely no referencing to their original source or critical assessment of that original source (i.e. being able to do something on one machine with substantial effort and physical access does not mean it can be accomplished remotely via some ill defined automated task on random hardware).
    • Look to personal experience. A personal pet peeve of mine is when firmware based malware gets tossed into the mix as a general and immediate issue. Technically, can it exist? Of course it can. Firmware is no different than any other program, it's just code and code can exist anyplace designed to hold it. On the other hand, I also know that if I do a flash update of my firmware with the wrong package from the right vendor, the result will likely be a mess. Yet embedding existing firmware with a malware segment or redirect can stealthily occur without disruption of functionality on a random piece of hardware. Right. Are there possibilities involving the EFI? Sure, but currently they are only vague possibilities. When immersed in a sea of real possibilities, one needs to get real about concern of the purely hypothetical.
    Everyone is completely responsible for what they write, and they should also understand what they write is persistent. It's not like a casual verbal conversation which has a fleeting existence, although the banter here often has that flavor. Based on a lot of what I read, the responsibility of the author is not a well understood concept.

  4. Peter2150

    Peter2150 Global Moderator

    Sep 20, 2003
    Some extremely valid questions, MRK.

    Good example, of what I try, and may not always succeed in is this thread on an approach Erik had settled on a while back.

    From that I will make the statement, that if you have more than one disk drive installed, you'd be very wise to protect all the drives, as malware can jump and install on all drives.

    Now the conclusion that malware can install on both drives, I can state as fact. I observed it happening with real malware.

    Then there is the question, if I rollback my system drive, can the malware some how be autostarted from the other drive. I don't think so, but don't know for sure. This is really nothing more than a guess.

    Finally if it can't run is it any big deal to leave it. One of my criteria when I tested was that if it did install on another drive, that it be gone when I used the tested software's clean/up feature. What was the criteria based on. Nothing more than that is what I wanted, based on I wanted my system clean. Reason... Plain and simple, I wanted that, no factual reason.

    Hopefully that came thru in that thread.

    Hopefully this answer makes some sense.

  5. Osaban

    Osaban Registered Member

    Apr 11, 2005
    I think what you're asking in a nutshell is for people to have some kind of integrity in their posts.

    When you say 'criteria' you are saying analyses which most people are neither qualified nor inclined into making an objective statement (I'm including myself in the lot to a certain extent).

    But then again this the internet, where people can claim to be scientists when they might be good amateurs at best or outright charlatans boosting their egos.

    "I heard it on the grapevine..." It is true not many people can really prove that one is safer than the other, but practical experience (an empirical approach) could lead people to draw some conclusions.

    Typical example of 'urban myth' which I agree it just fuels a state of paranoia among laymen, similar to the alarmist warnings about the avian flu.
    (there is nothing you can do about it, you'll die)

    It will depend on the category of the person. I'm not even a good amateur, as much as this field is concerned, it is totally opposite to what I deal professionally, and perhaps this is the reason I'm attracted to it. I learned a lot I think, in the sense that I can tell what is good information, and what is worthless. I try to be objective (we are all a bit biased), and base my statements exclusively from my own experiences. A lot of people confuse facts and opinions which leads to a lot of misinformation.

    I agree with BlueZannetti that people should be responsible for what they write as it's going to be stored and available for future reference. There are people who post to have some fun, to have a say on something, you can't blame them for being inaccurate, they are often well meaning. I think to even try to reply to your questions one would have to write an essay.
  6. ErikAlbert

    ErikAlbert Registered Member

    Jun 16, 2005
    You have to understand my situation first :
    My knowledge regarding internet, malware, anti-malware is very poor and that means that I don't understand what I'm doing or what they are telling me.
    I don't understand my router, my firewall and most security softwares are so vague that I don't even know against what I'm protected. So this is a big problem.

    Another problem is that I don't like to spend time on keeping my computer clean, because I consider this as a negative activity that results in NOTHING.
    "PC is clean - PC gets infected - I remove the infections - PC is clean again"
    In other words, I lost alot of time on something that gives me back, what I already had in the first place. This is more than absurd.

    Another problem was the incompleteness of security softwares, especially blacklist-based softwares.

    So I developped my own approach and that's what I did.

    First I separated my system from my data, because :
    1. It was technically possible without exceptions.
    2. I didn't want to store my precious data in the most attacked partition.
    3. My data would be always in the way for what I was planning to do in my system partition.
    4. System objects and data objects are totally different in too many ways.

    Then I created an off-line computer without internet, where I can do my work and hobbies without all the misery of internet, malware, anti-malware, etc.
    Nothing happens there and that's where I'm safe and above all restful

    Then I created my on-line computer, to "enjoy" the internet and that's where all the misery begins.
    I decided not to allow any change in the system partition, unless I do the change.
    So I replace my system partition during each reboot with a fresh installed partition and that cleans all the mess.

    Although I had the perfect removal tool, I still had to stop the installation and the execution of malware between two reboots.
    So I installed security softwares that try to do this job by :
    - not allowing execution
    - limiting their actions
    - isolating them.
    If these security software fail to do their job, I will at least remove the malware during reboot.
    Personally I trust my boot-to-restore alot more than all my security softwares.

    My last big problem was the Data Partition, which is nothing than a collection of folders and files and has no protection at all.
    I solved that problem by LOCKING my second harddisk, when I leave my desktop to visit internet.
    Locking = no reading, no writing, no stealing, no access and that was the perfect solution to protect my data.

    The idea was good, but the security software "PC Security" didn't do its job and Peter proved it.
    So my theory, that security softwares can't be trusted was proven again.

    Until now, I still don't have a safe, practical and convenient locking software. So my data partition is still a problem.
    I'm lucky that Sandboxie/DefenseWall seems to know how to lock a second harddisk, alot better than PC Security, but it's not a complete solution. I still want a locking software, but one that WORKS.

    The final result of all this is that I have now a computer that cures and cleans itself automatically during each reboot and this without using any scanner or any cleaning tool.
    I also saved myself alot of time, because I don't have to solve any problems, which I can use now for positive activities and that was my goal.

    I hope this satisfies Mrkvonic. :)
    Last edited: Dec 18, 2007
  7. gkweb

    gkweb Expert Firewall Tester

    Aug 29, 2003
    FRANCE, Rouen (76)

    I smiled while reading this passage :) On one side, it is absolutly true and I perfectly understand you and what you mean. However on the other side, there is people like me for who it is a hobby, securing for the sake of security. It is no more different than playing video games, it gives absolutly nothing but happiness. Then everyone finds it's happiness where he wants to.

    Just my two useless philosophical cents ;)

  8. herbalist

    herbalist Guest

    I'm not sure how to reply to the original questions. I'm not an expert on anything but I have spent over 4 years helping users with removing malware and setting up security packages. I try to limit posts to "how to plan/configure" type information on the apps I know, and theoretical "how it could be done" type discussions.
    Or the best firewall, HIPS, AV, etc.
    I try to avoid those threads. The exception to the browser statement would be their "as installed" security, for which IE6 is easily the worst. I also take issue with statements such as "just use firefox and you won't have a problem". No browser is exploit proof. I also have a problem with the comparing of firewalls based on leaktests and sites that misuse them to persuade (exploit) users to part with their money in order to get the "newest and best." I share your feelings about the misuse of numbers. How many potential or existing vulnerabilities one has compared to another can be twisted any way they want, but what matters is how their system and security package deal with the next one they contact.

    There's a fine line between fiction and reality. "It won't happen to me" doesn't always hold true. Your example about hacking a PC when the power is off is an extreme example, but users also need to understand that they and their data aren't always the target. Their PCs have value to those who run botnets. I've cleaned too many of these to ever tell a user that they're not likely to be a target. Unfortunately, that's all I can do with this kind of problem, warn users that it does happen and clean the zombie units one at a time when I find them.

    I share your position on P2P. What I find disgusting is that this "criminalizing" of P2P is primarily done by filthy rich corporations for the sole purpose of getting even richer. It's also sickening that they've used this wealth to buy and control lawmakers, basically making themselves the lawmakers at the expense of ordinary people. It might be the paranoid side of me showing thru, but I also wonder if the powers that be have another problem with P2P besides it's potential use for piracy. It does enable the common person to send and receive messages, files, etc directly to another person without it being routed thru a central point they can monitor. When combined with strong encryption, it's a very secure way to exchange data.
    My logic is pretty straight forward. Default-deny. As close to total control as I can get. I don't trust the big name companies and I definitely don't trust Microsoft. When their policies and software prove they don't trust the users, why should I? Paranoid? That depends on how you define it. I don't trust sandboxing, behavior blocking, and virtualization to detect and contain ever more sophisticated malware. IMO, that's the equivalent of trying to see how close to the edge of a cliff you can walk without causing it to give way.

    ErikAlbert's approach I understand, a clean system every time you reboot. I do have one question for him. What mechanism do you have in place that would prevent malware from infecting your system and stealing data between reboots? I realize that your setup prevents permanent infections, but a trojan or keylogger doesn't have to be permanent to steal data. Unless I've missed something, I'd view that as a vulnerability on my own system. Do you have any security measures in place that prevent unknown processes from executing?

    Blue mention firmware based malware. At present, that's more of a theoretical problem than a real one. That said, firmware is the next logical target for malware, including firmware for external devices like routers and hardware firewalls. Firmware based malware would be very difficult to detect and remove without specialized equipment or software. We have a theoretical vulnerability that is not yet being exploited, as far as we know. Should the users security policy and software take this into account as much as possible? At what point should it start, when it's found in the wild? With malware of this type, how widespread could it get before being discovered? I'd agree that throwing this at the average user would serve no useful purpose other than to make them paranoid, but for those who understand the implications and realize that there's no way of knowing where the next malware attack will come from or what it will target, shouldn't it be considered? Since firmware updates (or malicious firmware modification) are done by executables, a default-deny policy already addresses this theoretical problem. If it can't run, it can't infect.

    IMO, we're seeing a different reality than what we've seen in the past. Common sense is still useful but insufficient. It's one thing to tell a user not to visit crack or porn sites, but that doesn't help when sites a user trusts, sites that have always been clean get hacked. If educating users was going to work, it would have by now, but the rules are changing. Trusting will get you owned sooner or later. I fear that web 2.0 will create a monster that's beyond the average users ability to remain safe in. We're already seeing malware that security apps can't remove, malware that has to be removed manually. There's more ways to deliver it than there's ever been. We're seeing operating systems that treat the user (and kernel mode security apps) as hostile, which are nowhere near as secure as was claimed. SP1 for Vista supposedly fixes over 300 security issues already. What do you tell the average user?
  9. ErikAlbert

    ErikAlbert Registered Member

    Jun 16, 2005
    I perfectly understand you too. We only have a different attitude towards malware.
    My goal is to get rid of malware as soon as possible without even looking at it or wasting time on it.
    If I was working in the security sector, I wouldn't talk like that, but I'm an average user. :)
  10. ErikAlbert

    ErikAlbert Registered Member

    Jun 16, 2005
    Although I mentioned this in my first post, I didn't mention any software names, because that was not the intention of the thread.
    Besides a router and firewall, I have :
    1. Anti-Executable
    2. DefenseWall
    3. Sandboxie, which locks my data partition.
    I'm not saying that this is enough and I'm open for any other security software as long I understand how it works and what it does, except blacklist-based softwares, I don't want those. :)
  11. Ilya Rabinovich

    Ilya Rabinovich Developer

    Sep 13, 2005
    1. 100% objectiveness is impossible die to psychological issues. All the claims are subjective if made by people. :)

    2. Skills? Very mood point here. At I have following reviews for my DefenseWall HIPS: a) the program is great, all the bugs are quickly fixed by the author (5 stars review). b)the program is crap, I had met bugs with this software that caused inconvenience (1 start review). Both are based on user's skills. Which one is correct?
  12. Mrkvonic

    Mrkvonic Linux Systems Expert

    May 9, 2005

    I guess neither, because I don't know anything about those two people. How long have they tested the software, on what platforms, what did they do to get the either A or B. However, since I have read quite a few 5-star reviews and very few 1-star reviews and personally tested (and still occasionally do) test DefenseWall, I'd be inclined to believe that first guy more rather than the second.

    And here's the core of the problem I have mentioned earlier.

    Someone without any prior knowledge about software X has no way of knowing which one is true - 50:50 chance.

    This is why I think claims require something that should make them more than an opinion, a few lines of explanation, a proof, a counter-proof, a screenshot, a compilation of facts and ideas that should make the claim real.

    Erik, you asked me if your answer satisfies me? Well, partially. I understand your conflict and the solution you seek. But that does not answer me how you base your claims. For example, if someone asks you what you think about Firefox or Opera, what criteria do you use for your answer?

    Now, personally, what do I base my own claims?

    1. I know I'm quite a polar person when it comes to views and opinions and sometimes my answers can be perceived as a bit ... revolutionary.

    2. I'm doing this in order to shake the routine, to make people introspect and think, to get rid of the indoctrined fears and think globally.

    3. I believe that my claims are fair, having accumulated about 8+ years of (fast) online experience, 15+ years of programming and related experience - on various levels. In other words, no I'm not a smartass, but more of a rule of thumb follower:

    - If something worked well for 5 years or 10 years, there's no reason it shouldn't work well tomorrow. There's a chance it might not, but it's small.

    - Furthermore, I'm aided in my claims by a group of friends and relatives who share similar views and practices to my own - with similar results.

    That's my basic creed.

  13. Ilya Rabinovich

    Ilya Rabinovich Developer

    Sep 13, 2005
    So, you are talking here about some kind of "expert" ranks and technical background?
  14. Mrkvonic

    Mrkvonic Linux Systems Expert

    May 9, 2005

    Not really. I'm talking abut a more "precise" approach.

    Let's create a scenario:

    Someone says that program X sucks. Does this answer satisfy you? No.

    To take the claim seriously, I would expect:

    - A detailed explanation of what happened
    - Maybe a screenshot / a few words from a log entry showing the minutes of the problem
    - Steps that allow the problem to be reconstructed - otherwise it could be a one time glitch only
    - Explanation what the user did to induce the problem

    I don't expect the forum to become a field of analytics - not at all! I expect that when serious issues are raised, such as hacking, security, possible crashes and such, relevant info be provided to:

    - Help others understand the problem
    - Help yourself - someone might know the answer if you present your problem nicely
    - Prevent panic by people who know little about computers but can very much read words like identity theft, panic, uber-dooper-keyloggers etc.

    This can be difficult sometimes, especially when you're pissed off because your pc just crashed - I have sinned this sin once or twice as I remember, but ultimately, it will help people.

    No real expertise is needed. On the contrary, too much expertise can also hurt. Showing files in hexadecimal or timing the system interrupts or whatever is not very productive except for 0.01% of ultra-hyper-professionals.

    But general security ideas can be obtained, somewhere in the middle, between "sucks lol" and "00 01 03 04 05."

    Of course, technical knowledge can help, but it's not necessary. However, unsubstantiated technical claims by either too knowledgeable and too little knowledgeable can be very dangerous.

  15. ErikAlbert

    ErikAlbert Registered Member

    Jun 16, 2005
    In theory I consider each NEW object as a THREAT to my system or data partition.
    That's why I'm trying to learn how to verify new objects.
    My conclusion until now is that I will never be able to verify a new object as being good or bad, simply because I don't have the source code of new objects and even when I had the source code, I wouldn't be able to read it. It's hopeless.

    So when I installed Firefox as a new object, I had nothing else than the good reputation of Firefox to trust this new object. What else can I do ?
    I don't consider Opera safer than Firefox, both are just browsers to me and I like Firefox more than MSIE.
    Even when Opera is safer than Firefox, Opera will end the same way as Firefox : a fully patched Swiss Cheese.
    I run Firefox + NoScript under protection of Sandboxie and DefenseWall HIPS and that should be enough.
    When all four fail to do their job, my boot-to-restore will fix it.
    Last edited: Dec 19, 2007
  16. herbalist

    herbalist Guest

    I hadn't seen that you also ran real time protection. That answered my question.
    I don't know of any realistic way to evaluate how a new app will run on any given system. There's only so much you can do, even if you have and can read the source code. You can scan it for malicious code. You can run it thru an online sandbox. You might even be able to find a reasonably objective analysis from someone with a similar system. In the end, there's still some risk when installing something new. Other than making a system backup before installing (or using software that does the same thing), there's no way to completely eliminate the risk. When I install a new app, I make a full system backup first and monitor the install with Inctrl5. SSM and Kerio remain on. I don't have permanent rules for regedit, regsvr32, and other executables that are involved in the install process, so I'm prompted for each instance. SSM lets me see the command line parameters each time, so I can see what's being registered, etc. It's not a procedure the average user would tolerate, but I rarely install anything new on my primary PC so I don't have to do this very often.

    Those "which is better" questions are their own problem. You can count vulnerabilities over the time period of your choice, compare how long each stays unpatched, try to determine if and how much each vulnerability is being actively exploited, and basically make the numbers say anything you want them to. Example: Mac OS: More critical flaws than Windows in 2007.

  17. ccsito

    ccsito Registered Member

    Jul 27, 2006
    Nation's Capital
    Both are correct. :rolleyes: ;) Just as with car and movie reviews, there will always be opinions that range from five to one stars. :isay:
Thread Status:
Not open for further replies.