MD5 based exclusion lists ... please, we need them

[LuckMan212]

I have posted before (and others have as well) about the various problems with NODs exclusion system. I propose that an a hash-based (MD5 or SHA1) exclusion system be implemented. Each file added to the exclusion list could be hashed and only the hash stored-- similar to the way that the ProcessGuard allow list worked. Off the top of my head, I can think of several obvious advantages over "path based" system currently used:
1) if the file is changed (new version, infected with virus, etc etc) the hash-based system would detect it and ask the user what to do-- the "path based" one would simply ignore it and the system could potentially be compromised

2) multiple copies of the same file but located in different places (network shares, external drives, usb keys etc) would only have to be added once, instead of adding TWO instances (long+short path) for each file. So If I have a copy of a file on my hard drive, a usb key and on 2 machines on my network, rather than have to add EIGHT entries, I can add just one.

3) moving an excluded file to another directory would not require you to re-add those entries to your exclusion list.

There is a great freeware author named Nir Sofer who has produced quite a great assortment of utilities, many of which have become indispensable to me for troubleshooting and performing general system upkeep on both my own system and systems I administer. The utilities are small and efficient, and require no installation. You can find Nir's utilities at the following site: http://www.nirsoft.net

Now then, among others, there is a particular utility called "Protected Storage Passview" (pspv.exe). This tool is classified as a threat by NOD32 and when it is found NOD throws up an alert. I can see how this tool has potential for abuse, but it has many legitimate uses as well. I have many (let's be nice and just call them "forgetful") users who constantly forget this password or that one. Rather than waste time resetting passwords on the Exchange server etc, I can quickly recover their original password for them. I carry all the Nirsoft utilities on my USB key, and I would like to exclude that file from NOD32, however since each computer assigns the USB drive a different letter, there is no effective way to exclude it globally.

Hopefully the developers may see this and consider adding this feature. Perhaps if people are really attached to the path-based method, the exclusion dialog could offer all three choices (exclude folder, exclude file by path, exclude file by hash). I suppose another idea is to limit the size of a file that is excluded using the MD5 method. Example, don't allow MD5-based exclusions on files > 25MB. (just an example). This way the system would not grind to a halt while NOD tries to hash a 1GB file. What do you think? Please! ::)

[WSFuser]

have u posted this in teh Future Changes to Nod32 (http://www.wilderssecurity.com/showthread.php?t=49674) thread?

[LuckMan212]

I have now. Thanks ;)

[FanJ]

-{ Quote: "I have posted before (and others have as well) about the various problems with NODs exclusion system. I propose that an a hash-based (MD5 or SHA1) exclusion system be implemented. Each file added to the exclusion list could be hashed and only the hash stored--

...snip...
" }-

Definitely an interesting thread !

I'm curious about the answer from Eset.

Well...eh... be cautious how safe those checksums are stored.
Years ago I posted a little bit about it in general ...... ;)

[MichelB]

I feel we also need this... like to hear we are going to get it ? ;-) hope so...

[webyourbusiness]

"on paper" it sounds like a superior solution - we'll see if the Eset development team prioritize it... I'm sure their list of enhancements is rather long... ;)

[RejZoR]

This should be optional. I have my Quarantine folder(here i prepare files for submission to AV vendors) excluded using folder exclusion. Test files in there get constantly changed,replaced etc.
I'd get hundreds of popups just because of this.

[LuckMan212]

-{ Quote: "This should be optional [...] I'd get hundreds of popups just because of this." }-RejZoR, I completely agree with you. I think if this was implemented it should be in addition to the current path/folder exclusions. That would give the user the flexibility to use either method, or a blend of both.

[LuckMan212]

just discovered yet another reason we really need this feature--

my backup software uses the Volume Shadow Copy service to backup my local drives. I am not quite sure exactly how this works but it appears to create a "snapshot" of a volume and mount it in such a way that it is sort of invisible to the user but can be accessed like a frozen point in time.

NOD32 always generates alerts when hitting that "pspv.exe" file on the VolumeShadowCopy volume but I have no way to exclude it because it is not accessible via the normal path.

So my backups are constantly failing with errors due to not being able to access this file.

[mrtwolman]

-{ Quote: "This should be optional. I have my Quarantine folder(here i prepare files for submission to AV vendors) excluded using folder exclusion. Test files in there get constantly changed,replaced etc.
I'd get hundreds of popups just because of this." }-

IMHO computing MD5 from a file of some size would cost some time...Would you like this feature e.g. in AMON ?

[RejZoR]

I was just talking about how many files i get changed and how many problems this MD5 would cause to me. I haven't even think further...
Yeah even simple CRC32 takes quiet some time (for bigger files) on AthlonXP 3200+, MD5 takes even longer.

[LuckMan212]

this is from my 1st post-{ Quote: " suppose another idea is to limit the size of a file that is excluded using the MD5 method. Example, don't allow MD5-based exclusions on files > 25MB. (just an example). This way the system would not grind to a halt while NOD tries to hash a 1GB file." }-
and this is from my 3rd post-{ Quote: "I completely agree with you. I think if this was implemented it should be in addition to the current path/folder exclusions. That would give the user the flexibility to use either method, or a blend of both." }-

[LuckMan212]

any official word from ESET moderators on whether this feature is being considered? I feel that the exclusion lists are one area that is quite in need of some dusting off. Having recently renewed my NOD32 license, it would make me quite happy to hear that this feature is on the drawing board..... :)

[webyourbusiness]

Eset moderators are not in the habit of releasing information on upcoming improvements - something which is very normal in the software industry... don't expect any "official" word from Eset or it's staff on this, and you'll not get disappointed.

[LuckMan212]

adding to this thread again... this problem really frustrates me!!
I started getting popup alerts for something I had added to my exclusion list already. I went to check on it and lo and behold, somehow the path:

I:\UTILITIES\NIRSOFT\PSPV\PSPV.EXE

had been mangled into:

\Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE

??? what is this ... please Eset I love NOD but this exclusion crap is just silly and needs some prompt attention. Thanks.

[Marcos]

You must be using a non-standard hard drive / partition configuration. Please clarify what disk I is. Is it on a RAID drive? Is it on a dynamic disk volume? What oper. system do you use?

[LuckMan212]

I:\ is a USB flash drive. I am running Windows XP. It is not part of a RAID set or a Dynamic disk. The flash drive is 1gb in size, formatted with FAT32. I always plug it into the same USB port and it always gets assigned the same drive letter.

[Marcos]

What about excluding the file "\Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE" ? Does it make a difference?

[NOD32 user]

I find the same thing with my removable devices on XP - the exclusions still seem to work the same anyhow, so I've never been bothered by it.

[LuckMan212]

-{ Quote: "What about excluding the file "\Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE" ? Does it make a difference?" }-
But can you explain what "\Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE" is? I am unfamiliar with that syntax. Anyway in my earlier post I did mention that NOD32 apparently changed my original entry into \Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE without my knowledge--And even still, I was getting alerts from that file so the exclusion was not working.

So Anyway this morning I also got an alert on that file, and when I checked my exclusion list it had changed from:

I:\UTILITIES\NIRSOFT\PSPV\PSPV.EXE into

J:\UTILITIES\NIRSOFT\PSPV\PSPV.EXE

Again, this happened without my knowledge and I do not know why it is doing this. The current exclusions model seems to be a real problem for me I hope NOD32 3.0 will address this and provide a more robust exclusion system that takes into account some of my suggestions above.

[alglove]

-{ Quote: "But can you explain what "\Device\Harddisk3\DP(1)0-0+9\UTILITIES\NIRSOFT\PSPV\PSPV.EXE" is? I am unfamiliar with that syntax." }-
This syntax is how Windows keeps track of the drives internally, instead of using drive letters. Here is an article in Microsoft's Knowledge Base on how to decode this syntax, with a little help from the Registry Editor.

http://support.microsoft.com/default.aspx?scid=kb;en-us;159865

[webyourbusiness]

-{ Quote: "So Anyway this morning I also got an alert on that file, and when I checked my exclusion list it had changed from:

I:\UTILITIES\NIRSOFT\PSPV\PSPV.EXE into

J:\UTILITIES\NIRSOFT\PSPV\PSPV.EXE

Again, this happened without my knowledge and I do not know why it is doing this." }-


set drive letters in the higher ranges if you have a number of transient drives... this prevents a lot of re-lettering of drives in my experience....

[LuckMan212]

-{ Quote: "set drive letters in the higher ranges if you have a number of transient drives... this prevents a lot of re-lettering of drives in my experience...." }-what I am trying to say is the drive did NOT re-letter itself, the drive was still I: but somehow NOD32 exclusion list changed to J: which I don't even have a J: on my system!! so I am saying this exclusion feature just seems clunky and buggy to me.

[Marcos]

NOD has nothing to do with the way the OS assigns an internal system name for the removable drive. Maybe you could use Winobj from Sysinternals to see how the drive's system name changes in time.

[Defenestration]

I have to say that I am extremely surprised that NOD's exclusion system does is not signature based (ie. stores hash of excluded file), as well as path based.

This is a big flaw IMO and needs to be fixed ASAP.

[LuckMan212]

-{ Quote: "NOD has nothing to do with the way the OS assigns an internal system name for the removable drive. Maybe you could use Winobj from Sysinternals to see how the drive's system name changes in time." }-
Marcos you did not read my post carefully. The drive's letter DID NOT CHANGE. It only changed in NOD's exclusion list. The drive is still I:. It has always been I:. It always will be I:. However NOD insists on scrambling this up for some reason in the various ways I have outlined above. This problem has become so annoying to me that I now just disable AMON whenever I am working with my USB drives. Obviously this is stupid and ridiculous and leaves me wide open to infection but that's what its come to because I cannot seem to get NOD's exclusion list to cooperate. All I am asking for is to hear someone from ESET say "yes we are overhauling this for 3.0 and it will work right, thank you for your patience" I hope this is not asking too much...... :-\

[alglove]

LuckMan212,

I think what Marcos is trying to say is that NOD32 stores the target drive in "\Device\Harddisk?" format in the registry. HKEY_LOCAL_MACHINE\SOFTWARE\Eset\Nod\CurrentVersion\Modules\AMON\Settings\Config000\Settings --> exc . The exc value will look like gibberish until you double-click it. Once you double-click it, however, you can make out what it says. From what Marcos is saying, NOD32 does not change this value, but it sounds like Windows is changing the "behind the scenes" definition of the I: drive.

To get an idea of what the "behind the scenes" value of the I: drive is, go to the Device Manager and look for your USB drive. In the General tab, look at the value given for "Location" and tell us what it is.

I suspect what is happening is that the "\Device\Harddisk3\...." value changes slightly when you unplug the USB drive and then plug it back in. Windows just assigns it the letter I: each time, since it is the next available letter. However, NOD32 stores the original "behind the scenes" value of the USB drive. Since it does not match anything currently connected to the computer (since your USB drive is now something else), it shows up as the "next" drive letter, which of course is J:.

I have no way of verifying this myself, since I do not have a USB drive. Maybe you or somebody else can. If we can somehow get a handle on what is going on here, then we may find a way to avoid this problem (either from the user end or the Eset end).

[LuckMan212]

so if what you say is true then in other words there is NO WAY to make this work and the only solution is to

a) disable NOD when working with my USB drives
b) do not use this particular program (which is actually not a virus but quite a useful program)

Neither of these solutions is very appealing.
Well anyway I have said what I needed to in the previous posts. Hopefully someone at ESET will have seen this and taken it into account for the 3.0 version.

[alglove]

I am not sure about NO WAY. Just some way I haven't thought of. There's gotta be a way.... :lurking:

What if you try webyourbusiness's suggestion of forcing the USB drive to a particular letter? Maybe that will force it to stay consistent. Also, try making it something different than I: or J:, just for the purpose of making sure it is something distinct and away from all the other drive letters. Something like U:, for example.

Once you have forced the USB drive to U:, try deleting the existing AMON exclusion and create a new one. I am interested to see if it stays as U:, changes to I: or J:, or does something totally different. Again, I would try this myself, but I do not have a USB drive.

For people who do not know how to force a drive letter assignment in WinXP, a good tutorial may be found here: http://www.nursing.upenn.edu/otis/helpdesk/how-to/map.asp

[LuckMan212]

bump... can we get any idea whether to expect this feature to be added to NOD v3.0 ?