Archives scanning, pure waste of resources or the final wisdom to clean PC?

Discussion in 'other anti-virus software' started by Firefighter, Feb 1, 2003.

Thread Status:
Not open for further replies.
  1. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    I have found on these sites, that there are very few discussions about the purpose of archives scanning capability. I am not a virus expert, and I can’t make any programs either, since Pasic program or what was the name of it some 25 years ago, but one issue makes me very suspicious about the whole av-business. Why very few av-programs are capable to scan enough archives? Let’s take for example NOD32; Kaspersky is capable to scan about 2,5 x files from my PC than NOD32.

    When your are sending files from your PC to someone you want to be safe, and you are not sure about the security issues in that PC, there are about 60% of files that you don’t know anything about it, if you use for example NOD32. I know that Norton is also not so good archives scanner, but it is still number 1. sold program! As a pure amateur of making programs, this irritates me very much!

    Let’s take a theoretical example: You have an absolute infected PC, you have never used av-programs before in your PC and your have surfed all the time in web in all possible dangerous sites. After that in your PC’s “My downloads” folder there are all kinds of infected files amount of 1000. Then you take an av-program to your PC, which is an average scanner, and it can detect only 99% of viruses, but it is capable to scan 80% of all files from your PC. After that you are sending the whole folder to your best frend, so the result is that your best frend gets some 208 viruses from you. But if you have NOD32, the ultimate best scanner according to Virus Bulletin, your best frend gets 600 infected files from you. So where is your responsibility?

    After all that nasty things, I use Avast 4 Pro as my resident, DrWeb and RAV as my backups, and I have not been infected yet, but I know that the time will come some day in the future.

    I use Avast 4 in pure sentimental reasons, because Alwil’s product has something in it which is very deep in Finland’s history and it has some unique properties that I haven't found elsewhere. That’s why I have adopted it as my growing “prince”, but I understand that RAV and DrWeb are just now more reliable choices, so I must use the very best backups! :eek:

    “The truth is out there, but it hurts!”

    Best regards,
    Firefighter!
     
  2. root

    root Registered Member

    Joined:
    Feb 19, 2002
    Posts:
    1,723
    Location:
    Missouri, USA
    You really should turn your computer off and go get some fresh air. :)
     
  3. sig

    sig Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    716
    At some point, unless someone is just collecting and transmitting zip files with never opening them, eventually the zip file will be opened, yes? So shouldn't the test be, does the AV and or AT catch the malware when it is unzipped and/or attempts to execute?

    My understanding is that there are ways of packing files to defeat AV and AT scanners. While the malware is compressed (zipped) it does no harm, only when it is executed does it go into action. BOClean, for example, catches trojans when they attempt to run. (I recall reading one of Kevin M's newsgroup posts long ago on why in his view this method was more effective than on demand scanning, although apparently BOClean is working on an on demand scanning function due to popular demand for that feature.) In some cases evidently Trojan Hunter also may not alert on a zip but does when the file is unzipped and/or the program attempts to execute.

    So, the test is whether the AV/AT catches the malware and prevents infection. How and when they do it may differ. The end result is what counts, in my view.

    Your hypothecial scenario begs the question, why wouldn't you know if your computer is infected? Just having a zipped file with malware in it doesn't infect your computer. Running the executable would. If you have an AV and the malware is something covered in the sig definitions or covered by heuristics, then it would catch it. Ditto with an AT. Ditto for your friend's PC if he has AV/AT that would catch the malware.

    And if the AV and/or AT doesn't catch it at the unzipping or execution point, that means they can't catch it at all. So the test shouldn't be so much on the compressed, packed files (which in effect are harmless and malware can be disguised to avoid detection) but on the efficacy of the signatures and heuristics of the AV and AT to catch the malware when it does pose an imminent threat.

    Would I want programs with a better record of detecting and stopping infection at the point of real threat or programs that miss more bugs at that level? Which products, in the end, provide more effective protection?
     
  4. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To Root and Sig from Firefighter!

    I am surely not here to please anyone. My only purpose is to help people to think issues from different aspects.

    So the main question is: Is the whole web world safe, if you have an 100 % proof av-program which is capable to scan less than 50% of your files, when you are sending those files to someone else, whose secure you don't know anything about? That is what I am talking about debate! There is nothing about personal! Calm, calm and have a nice day! ;)


    “The truth is out there, but it hurts a lot”

    Best regards,
    Firefighter!
     
  5. msingle

    msingle Registered Member

    Joined:
    Jan 25, 2003
    Posts:
    82
    Firefighter asks about responsibility for this. And Sig makes a good point about not really being infected until the virus actually executes.

    Here's hypothetically how it could hurt a business or website:

    Let's say someone has a brand new PC. They don't understand about the need for an antivirus program. They put their tax, business, and personal information on the computer. Then one day someone says that they should really look into getting AV protection and that Wilders.org is a good place to get some free downloads. They can't find a good free antivirus software to download but they do find some of the other great files that are in zips available for download from the site.

    They unzip and install XXX program from the site and now they are infected and maybe all their data is erased IF that zip file they downloaded had a virus because the scanner being used at the site didn't effectively scan archives.

    What are the legal and financial implications if someone could prove they downloaded a virus from an infected zip? In court, it could be easily proven, that newbies may not be aware of the risks of viruses and the need for AV software.

    There's tons of people out there who don't have AV protection at all. Or they put more emphasis on pop up blockers and such.

    So, to me and it's only my opinion, archive scanning is important.
     
  6. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    17,874
    Location:
    New England
    I'm not completely sure I follow your hypothetical situation. Are you saying, just by way of example, someone looking for an AV doesn't find one, but, they do download from the same site that provides such AV products, some other program, which happens to contain a virus, that then infects the unprotected system?

    So, in this hypothetical case, it'd be the download site that let the malware into its server, right? If that is the case, I think this is a very different animal than just a simple client-side AV scanner... Server-side malware scanning is a far greater responsibility for sites that provide public downloads. But, malware scanning is only a piece of their total responsibility. They also need to verify that the files they are providing are legimate and are presented unaltered from whoever the original source is. They'd need to test the files to make sure they work as advertised, which they'd probably do on a system with good malware scanning capabilities. They should not just take a zip or other packed version of an installer from somewhere and post it without any testing at all.

    If I were providing files for download, I'd fully unzip / unpack and scan the files in their most basic form, before providing any for download. Any responsible person running a download site should do that.

    It is clear, given the number of threads in which this is now being discussed, that some people really believe that a key feature in a virus scanner is the ability to scan deep into archives. The arguments they have presented certainly make sense, and from that perspective, they are correct...

    But, it is just as clear that others here believe it is not necessary to scan into archives, or at least not to so deep a level of nested archives. These people believe that any malware in such files are safe, and they would be caught once they are unzipped. And these people are also right.

    Arguments obviously can be made on both sides of this issue, as both views have some merit, but, somehow I doubt either side is going to sway the other to their viewpoint. :doubt:
     
  7. msingle

    msingle Registered Member

    Joined:
    Jan 25, 2003
    Posts:
    82
    LowWaterMark you make some very good points.

    I guess for me the idea, whether it's harmful or not in its current state, that a virus is embedded in a zip file on my computer is a little troubling.

    Everyone here knows that there are two basic functions of a good AV scanner: on demand and on access.

    If the on access part is working correctly then all is well. However there could be a lot of examples where, for one reason or another, the on access isn't working - like your wife or kids turns it off not knowing any better or Windows gets corrupted and nothing is running right including the on access scan.

    If there is ever a chance that the on access scan doesn't or won't work correctly all may not be lost if the on demand scan thoroughly scans and cleans and if you ran the on demand scan often enough.

    It's a peace of mind thing. I have a friend who was told by their doctor that they had a type of cancer but since it wasn't spreading and since it wasn't hurting anything that they weren't going to take any action but were just going to wait and see. There was the option to remove the cancer but the doctor didn't want to because, in his mind, there was no need to remove it until it was a problem. My friend's life turned into a living h*ll worried that every headache or weird pain was the cancer "coming to life". Finally a doctor was found who did remove the cancer and all was well and my friend could live with a little more peace.

    Now I'm not comparing cancer to a computer virus because my family has had a string of bad luck with cancer and there is no comparison.

    But it's the same type thing. Why NOT scan the zip files thoroughly just in case something goes wrong at the on access level? Why rely on 1 level of protection when you could have 2?

    You're right if people's minds are made up a few posts here isn't going to change their mind. But why not provide optimal protection instead of minimal?

    The same thinking goes into questioning AT tools like BoClean that don't scan the disk because if a trojan is going to do anything it has to get into RAM somehow. Why wait and hope that it will catch it in RAM versus also scanning the hard disk for better protection?

    My two cents.
     
  8. *Ari*

    *Ari* Registered Member

    Joined:
    Feb 15, 2002
    Posts:
    431
    Location:
    Finland
    So true ! :D and this sentense for everyone at wilders ! hehehe Root ...darn...I laugh sh** out of me ....!

    ¨Ari ^


    [ I was laughing at myself, I do not do anything but sit and search for security holes/ softwares....chat occasionally with someone on messenger.........]
     
  9. sig

    sig Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    716
    Firefighter: I'm not at all excited and, frankly, don't give a fig what products you use. It's your machine. I was just trying to explain why some people (including some AV/AT vendors) don't put all the emphasis in the efficacy of scanning zips. As far as I know there is no such thing as 100% security. It's all about risk management, whether AV/AT/firewall, etc.

    FF and MS: the underlying assumption behind your scenarios involves people trading in zip files and evidently not running sufficient protection to protect them when they unzip and run a file so they get infected.

    Any responsible and knowledgable website I'd imagine would be insuring that they are clean prior to zipping the files they provide to others. Or, unzip files received from another source to ensure that the contents are what they claim to be and malware free before they make them available for download.

    As is often recommended good security practice is to be wary of downloads from not reputable or unknown sources or even your friend who may not have a clue. One of the well known hazards of peer to peer file sharing, for example, is that you really have no idea what the file may really contain. People who engage in such practices should really be armed with a good AV and AT at least. So there is no getting around that ultimately if users wants to protect their system it is up to them to take responsibility and run the best protection they can find.

    Again, would I prefer to run an AV and AT which ultimately provide among the highest detection rates, preventing malware from executing and infecting my computer? Or those that seem to catch more on zip scans but in the end has overall poorer detection rates when the malware is uncompressed and/or attempts to run? The first protects me as much as reasonably possible, given timely sig defs and good heuristics; the latter only lulls me into thinking it's protecting me better.
     
  10. xor

    xor Guest

    Cuz it is much more easier to scan RAW FLAT Binary Data direct in the memory.
    You can avoid filescan detection by using runtime packer / crypter on trojans for instance. If the trojan is executed, it unpacks in the memory. (he has to unpack to run)
    This is the moment where you can catch him easy. But this is the moment where the trojan is active and can shut down your scanner unless you have strong ring0 protection such a direct kernelcode injection to avaid terminate process / terminate task functions. This also not so easy, cuz you dont know the process id (which to protect) during the system boot. But you can use Spinlocks to comunicate from kernelmode to usermode for instance.
    On Win9x is this a big flaw because you have no access rights there, however with smart tricks you can handle this too.
    The trojan, worm or whatever can have during the file scan different size and different binary data. This depends on the "tools" with which the worm was "protected" this means you have to detect with which type of packer the worm is packed and with which version. Then you have to take care of recursive packed methods such es packed and after this crypted. This means you must first uncrypt; after this check this file again for the next packed/crypted method and process it. Here you must take care that you are not running in a "Death Loop".

    [-xor-]
     
  11. sig

    sig Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    716
    Mssingle: I can't answer your very reasonable question about why don't companies improve detection in zips. I'd like to know what's involved and the pros or cons. Or as Firefighter noted in his title, is it to some of them a waste of resources to increase emphasis on that? That resources are best put to increasing detection on the point of threat and keeping on top of latest malware being produced and distributed?

    Unfortunately, I can't reproduce Kevin's comments regarding his views and the issues presented by on demand scanning in reference to trojans. In general, if I recall correctly, it was that countermeasures taken by the bad guys could too easily render such detection ineffective and that catching the bug when it reveals itself and attempts to run is most effective.

    I'd certainly welcome someone from an AV or AT company or someone knowledgable in the field to expound on the archive issue, if they would.
     
  12. xor

    xor Guest

    Just image the following....
    You have a worm on a pc - yes i know not a good dream, but image :D

    Ok, and you try to copy this worm from your local hard disk C: to a network drive. Yes it will catch this worm. It catch's this worm cuz the File IO Manager rises a event up. "CreateFile" this, plus the path is transfered to the scanner to scan the file. Now guess what would be happen if you try only to copy a ZIP File with 64.768 Files inside to an other PC.... :D Yes.... normaly it "should" unpack this ZIP and scanning all files before the copy action starts. Nobody can work with this solution. At least not in real time.
    You can use "step for step" scanning in the ZIP (level depend scan) this means if you open this zip (for instance with windows commander) it scanns the part where you are in. NOT the subdirecties.

    So lets make it step for step....
    You open a zip file -> raises event "File is accessed"... -> ZIP File....
    You know now not direct executable file.
    If you click inside this zipfile on a exefile it raises a second event -> it will unpack this file.... This is the moment where you can catch the virus.

    I hope this helps a little bit
     
  13. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To everyone from Firefighter!

    All this archives scanning problem is more than personal issue. Using poor archives scanning programs, your PC is actually delayed-action-bomb and you think there is no problem ever.

    The whole web community may catch a serial virus from you by using iMesh, Kazaa or what ever. It seems to be so that when now about 50 % of av-users are using Norton or something like it, the security is atleast only an illusion.

    When I earlier on these sites wrote about priorities and my first priority was good archives scanning capability, the result will be that you have the whole problem (all viruses) in your hands which you can cure. You have not thrown anything important away. :cool:


    "The truth is out there, but it hurts!"

    Best regards,
    Firefighter!
     
  14. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To everyone from Firefighter again!

    About 98 % of all possible PC programs in Russia are illegal copies. The serial numbers and registration keys of those programs are downloaded from several crack sites, which are totally infected with every ways.

    Two of the absolute top av-programs, Kaspersky and DrWeb, both from Russian origin, are very good in archives scanning? Those programs are the very best against trojans too, DrWeb maybe only in the Wild level. So is it pure stupidity that they are wasting their resources and money to absolute nonsense, or is it so, that the PC:s to be scanned with those programs are really clean in all possible levels?

    I know several youngsters, who have a hobby to surf with iMesh and Kazaa downloading what so ever, almost every month they have to reload the whole Windows to their PC:s, just because they are using poor archive scanning av-programs with poor in the Wild and Zoo capability too! ;)

    “The truth is out there, but it hurts”

    Best regards,
    Firefighter!
     
  15. sig

    sig Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    716
    Firefighter: It's not a personal issue for me either....I don't use Kazaa or any of those P2P progrrams, so that's not an issue for me or anyone else.

    If I zip something to send to someone or myself at work, the uncompressed contents have already been scanned prior to zipping. If I receive a zip from someone else I scan it with an on demand scanner and it is scanned again by the resident scanner when it is unzipped. And scanned again if it's a program when it executes. So how do you figure anything I have on my computer is a "time bomb" when the files have been scanned in the uncompressed state before and after zipping?

    And if I did P2P file share, the procedure would be the same.

    You have files you've never unzipped and share those? The only way your concern makes any sense to me if you're sharing files you've never unzipped and run yourself. In which case, that's your responsibility. And if your security tools are not among the best at detection in the uncompressed state (in which case they wouldn't catch some things in archived format either), then you'd be more likely to be passing on infected files than would I. ;)
     
  16. sig

    sig Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    716
    Ah, Firefighter...just saw your last post. Well, that's a different case.

    Those people downloading cracked programs (notorious for including malware), if they're going to engage in such dangerous behavior, should be running the best AV and AT they can find...which would catch malware if not in archived format (again things can be disguised in archives so they are not caught by scanners) at least in the uncompressed form. However, likely many of those are not running any AV or AT programs (or perhaps only cracked versions, LOL, which may in themselves be suspect).

    Again, you don't see that it's not poor archive scanning, but poor detection in the uncompressed state and upon execution that is the issue. AV's (and AT's) that don't catch something in the uncompressed state will not catch it in the archived format either. If the AV or whatever they were running had the capacity to catch the malware, the infection would not occur when they ran the programs or files. That rather proves my point. (And that's IF they are using ANY AV or AT programs at all.)

    At any rate, best computing practices are not what those people are about or concerned with. Or perhaps they rely on an AV alone as I have seen and wind up with some interesting Trojan not covered by the AV and then complain about their AV when it is their own practices which get them into trouble. They are being irresponsible (and engaging in illegal activity) and their fate is what they worked for.

    AV's and AT's are mainly reactive products, despite heuristics, so if someone is going out of their way to engage in known dangerous behavior, they incur the risks. Again, AV's and AT's are not 100% all the time and people should not rely on those programs to save them from their own risky behavior. In those cases, the user bears responsibility for his own activities and security practices.
     
  17. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To Sig from Firefighter!

    I don't use P2P programs either, it's an other issue if my kids are doing so when I am at work or something. But I have at least such av-programs, which are very good in archives scanning (Avast 4 Pro), and I am doing double full backup scans once a week with RAV and DrWeb, so far there were no problems, but you can't never be sure! :D

    "The truth is out there, but it hurts"

    Best regards,
    Firefighter!




    thats capable to scan very gn
     
  18. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To everyone from Firefighter!

    There is so called world wide risk assesment tool as FMEA and it is a part of a worldwide standard too. The FMEA tells you the risk of certain bad things to happen with one number called Risk Priority Number (RPN).

    The RPN is actually a product of three different factors.

    1. Severity; it is ranked 1 to 10
    2. Occurence; it is ranked 1 to 10
    3. Detection; it is ranked 1 to 10

    When the risk is so high as possible, the RPN is S x O x D = 10 x 10 x 10 = 1000. So you must have as small risk ranking as possible.

    When we are ranking Severity of av-programs, the result should be with all av-programs 10, absolutely, because what ever could happen after infections. Let's think about a computer based airplane steering systems! :eek:

    In commercial av-tests only Occurence matters, because then we are talking about detection rates which are enough simple issues to an average consumer. But when we are evaluating the rankings in the Occurence category, the results varies about between 4 and 5 points with all common av-products according to FMEA tool. :rolleyes: :cool:

    The Detection category again is among all bad things to be able to find. When we are evaluating the rankings in the Detection catecory, we are actually ranking archives scanning capability. The results in Detection catecory varies about between 1 to 6-7 points with all common av-products according to FMEA tool. The 1 is at least Kaspersky, the 6-7 are for those very poor archives scanners. The rest of all are something between 1 and 6. :eek: o_O

    So the total RPN will vary from 40 to some 280 ... 350. All above RPN 100, may say be unacceptable in quality terms. The 40 is not that figure which is Symantec's, they have only that money what counts. :cool:

    According to that FMEA tool (a part of QS-9000 std) the Occurence category is not so important issue in this case, when we are looking the entirety within av-programs.

    The biggest differences are among archives scanning, because all the time the main goal have been to assure the effectiviness of av:s to the majority of quite simple people with those in the Wild tests or what ever. One million flies can't be wrong, the s... is good! :D :cool:

    When we are using those programs in car industry, Norton or McAfee have got some 2 or 3 times higher RPN:s than for excample RAV, which was the choice to Volvo (Volvo is a part of Ford Motor's), when it took that av-program to use. Ford was one of the most enthusiastics players with making QS-9000 standards worldwide.

    A Ford is always a Ford, but that's why there is GM, so that Ford does not have to be the worst car manufacturer ever in the world! :D :cool:

    Is the packed file dangerous when there is a virus? If you have a lot of files in your system already, and you have to add an other complex av-scanned program to your system to do something special with those original files, it may be so, that under a situation your never thought , the virus not scanned with a poor archives scanning av, will be activated during the program run in different uncommon areas, and the rest is history! o_O :rolleyes:

    We must always remember that those products that are going to an average consumer, are definitely not the same as those the professionals are using with.

    It is needless to care about which products we are handling on with all this issue!. ;)


    "The truth is out there, but it hurts!"

    Best Regards,
    Firefighter!
     
Loading...
Thread Status:
Not open for further replies.