NISFileCheck guidelines

I made these guidelines on the old forum.
Click on the words 'old forum' and you will go to that old thread.
I post them here again because they were maybe a bit too much burried in a long thread.

[hr]

NISFileCheck can be downloaded here:
http://www.capimonitor.nl/niscrc.htm

The program is made by Albert Janssen.

You do not need to have the firewalls NIS, NPF, AtGuard on your system to use NISFileCheck.

[hr]

I start NISFileCheck not automatically but by myself.
Generally spoken I run it once every day, or (when I (un)installed or upgraded a program) I do it after such an (un)installation.
With NISFileCheck you can check whether any file, which is in his database, has changed or is deleted, and you can check whether there is any new file on your system, based on pre-defined description of file-extensions.
That checking is taken place by using some kind of checksum algorithm.
You can choose between three different algorithms: Options > Hash.
You can let it automatically search for files, based on file-extensions, and/or you can add a file by yourself.
You can have only one database, or you can have two or more databases; it is up to you. For example you can have one database for the dll files, one for the exe files, and one for all the ocx vxd sys and bat files.
You can have one database for your partition C, and, in case you have also a D partition, one for your D partition.

Now take this example: you want to make a database for all the dll files on your C partition.

Start NISFileCheck.
Go to: Files > Add files on extension
Make sure that in the box 'Drives to check' is only C mentioned.
Make sure that in the box 'Add file with the extention' is only dll mentioned.
Now click Cancel (this is important in the case you already have a database for the C dll files, otherwise you will end up with every file twice mentioned in your database).
Go to: Files > Search for new files (based on 'Add files on extension').
Now NISFileCheck will search for every such dll file on your C partition.
The new files will be colored red, and the number of new files will be showed.
Go at the top to: Filter > Only new files.
Go to: Edit > Select all records
The new files (which you just selected) will be colored blue.
Go to: Validate > Validate all applications
You have to repeat this validating until all the records are validated (every time you hit that button, half of the records will be validated).
Go at the top to: Filter > Validated.
Go to: Check > Check if All the files are the same.
Again you have to repeat this until all the files are checked.
Go at the top to: Filter > Filter off
Go to: Edit > Unselect all records.
All the files are now colored white.
Go to: Files > Save database.

Now give your database a name; or, when you already have the desired database, click on his name, and you will be asked whether you want to change it; in that case, answer yes.
Now you close NISFileCheck.

Now, after a while (say the next day), you want to check whether any dll file on C is changed.

Start NISFileCheck.
In case a different database now is opened (when you have more than one databases), go to: Files > Load a database, and click on the name of the desired database.
Note: when you have more than one databases, make sure that you give them a logic name that you can easily remember, f.e. C_dll for your dll files on your C partition.
Now you want to check whether any dll file on C was changed.

So go to: Check > Check if all the files are the same.
In case any file has changed or was not found, these will be colored red, and the number will be shown.
Now some different things can happen:

Some files were deleted from your system:
Go at the top to: Filter > File not found
The files, which were not found, are colored red.
Go to: Edit > Select all records
The selected files are colored blue.
Go to: Edit > Delete selected records (there is no reason why these files would stay in your database).

Some files were changed:
Go at the top to: Filter > Only changed files
The files, which were changed, are colored red.
Go to: Edit > Select all records
The selected files are colored blue.
Go to: Validate > validate all applications
(maybe you have to repeat this validating)
Go at the top to: Filter > Validated
Go to: Check > Check if all the files are still the same
(maybe you have to repeat this).
Go at the top to: Filter > Filter off
Go to: Edit > Unselect all records
All files are now colored white.

Now you want to check whether there are any new dll files on your C partition.
Do the same thing as you did when you first made your database.

 

NISFileCheck

In the following postings will be given some more info.

I would like to thank Joseph for giving his permission to make use of his information !

 

NISFileCheck General Tips

Introduction

NIS File Check actually performs some very fundamental functions:

• 1. It allows you to establish a baseline of files that you wish to monitor using a hash algorithm of your choice (SHA1, Ripe MD 160 or Haval).
  This monitoring is not done in real time (resident).
  In case you want this monitoring done in real time, you need another utility. For Windows 2000, NT, XP there is for example its newer brother File Change Alarm that does this monitoring on these Operating Systems in real time.
• 2. It allows you to check these files to ascertain if they have been changed or deleted (not found).
• 3. It allows you to search for new files that meet the criteria of the baseline.
• 4. It allows you to validate any modified or new files that you may find as a consequence of the preceding operation.
• 5. It allows you to remove files, which were not found, from the baseline.
  (At the current time, the capability to physically delete a new, changed or not-found file or to physically restore a changed or deleted file is not provided by NIS File Check. These decisions must be accomplished using other tools.)
• 6. It allows you to add a file of your choice manually.

[hr]
What is a Baseline?

A Baseline is a complete specification of the files that you wish to have monitored; this monitoring will not be done in real time (resident).
In NIS File Check, this is the first step that you need to accomplish to use the application.
You tell it what files, based on extension(s) and drive(s), you want to monitor.
In NIS File Check, when you first identify the scope of the baseline, initial hash signatures are computed, based on the hash algorithm you have selected.
However, the initially computed hash algorithms do not, in and of themselves, constitute authentication of the baseline files currently present on your computer. This initial procedure simply calculates the hash values (for the selected algorithm) of files satisfying the specified baseline specification that are currently present on your computer. There is absolutely no connotation that these files were not previously corrupted, subverted, or replaced. This is a critical point that you should well understand.

[hr]
What is Checking?

Checking is the process of ascertaining if a file in the baseline has changed.
NIS File Check not only checks the selected hash value, but also the File-Size, File Date Last Modified, and File Version information for each file in the baseline set.
If any of these parameters have changed, NIS File Check will highlight the entry for the affected file in its display.
NIS File Check identifies a specific file by the combination of the Drive on which it resides, the fully qualified pathname to the file, the filename itself, and the file-extension.
If any of these parameters are modified, NIS File Check will identify a missing file.

If all of the information on a file in the baseline is unchanged (including its fully qualified path name), the Check option will so indicate by marking the status of the file as Unchanged.
If any of the items indicated above has changed, the Check option will so indicate by appropriately resetting the Status value for that entry in its display and highlighting the row for that file.

The Check option simply identifies files in the baseline that have been modified, deleted, added; it does not make any determination as to whether this change is legitimate.
It remains for the user to determine if there is a legitimate reason for the absence of a not-found file, the change of a changed file, the presence of a new file.

 

Hash Algorithms and File Authentication

Hash Algorithms and File Authentication

The following sections provide background information on what hash algorithms are, what they are intended to do, and the extent to which they can be compromised.
Hash algorithms provide a (more or less) unique fingerprint for a file on your computer or even a string of text or numbers. The most fundamental concept is that no one can read the hash signature and then reconstruct the file or string to which it applies. The second concept is that the hash signature is unique, i.e.: the hash signature for a file or string should not be possessed by any other file or string. This second concept is a goal, not a reality. A lot of effort has gone into breaking hash algorithms so that a different file or string would possess the same hash signature. The obvious goal (from a security perspective) of breaking a hash is to be able to replace a trusted application or string with something that cannot be trusted. Given enough time and computing resources, any hash algorithm can probably be broken. In fact, some of the older hash algorithms have been broken for quite some time. Therefore, these older hash algorithms are not necessarily desirable.
The hash of a file or text string is a more or less incomprehensible hexadecimal string. In and of itself, it means nothing. From a security perspective, all you really want to know is: Is the hash now different from what it was previously? If it is, the file (or string) has been changed.
Furthermore, there are at least three reasons why a hash may change:

• You may have intentionally changed the file or string involved, e.g., you may have installed an upgrade to an application, changed a password, or modified a data file.
• The file or string may have been corrupted in the normal course of using your operating system on your computer. (And, in Windows, this is hardly an extraordinary phenomenon.)
• The file or string may have been maliciously altered by another user, a virus, a trojan, or a worm that has somehow managed to access your computer (or a server on which you depend).

So, if you find that a hash algorithm for a file or string now is different from what is was previously, how do you know which of these three options is pertinent? Simple: you really don't. If you practice safe hex, you should remember if you intentionally altered or updated the file or string in question. Otherwise, the file or string has either been corrupted somehow, or a potentially malicious file or string has been implanted surreptitiously.
Reality, of course, is the difference. If you run an update on MS Internet Explorer, it can change files all over your directories. How do you know if a change is the result of the update or something else? Well, you need to check your hashes before the update and immediately after the update.

 

How are hashes calculated?

How are hashes calculated?

In simplest terms, the hash algorithm reads a file or string as a series of binary bytes, takes the hexadecimal value of each byte (or maybe a group of bytes) and performs a mathematical calculation using the resulting numbers. The important thing about the calculation that a hash algorithm produces, is that uniqueness is highly desirable in the result for different files or strings. This, of course, is what makes life fun for cryptographers, cryptanalysts and mathematicians interested in number theory.

Some links to general information about this topic (with thanks to luv2bsecure for giving me all these links; many thanks John!):

The following three links give all three of them very good, simple information, in particular the first link from Web Monkey.

http://www.webmonkey.com/programming/php/tutorials/tutorial1.html

http://www.ssh.com/support/cryptography/introduction/

http://world.std.com/~franl/crypto.html

Further:

http://www.cryptography.com/

http://www.crypto.com/

You also might find things of interest at:

http://www.iacr.org/~iacr/index.html

The above is the site of the International Association for Cryptologic Research. Anybody serious into the topic eventually lands here. It is run out of the Santa Rosa Administrative Center at the University of California in Santa Barbara. However, it is extremely INTERNATIONAL in nature, with conferences all over the world.
IACR also publishes the Journal of Cryptology.


Bruce Schneier's Applied Cryptography (2nd Ed.) is the "crypto Bible" for the professional engineer and interested layman. It is a good survey of the state of the art in crypto techniques and protocols. You can also subscribe to and read back issues of Crypto-Gram, Bruce Schneier's monthly newsletter about cryptography.
http://www.counterpane.com/crypto-gram.html

 

Hash Algorithms (1)

What are the hash algorithms that you are likely to see?

The following represent some of the hashes that you are likely to see from time to time:

• CRC-32 (32-bit hash)
• MD2 (128-bit hash)
• MD5 (128-bit hash)
• SHA1 (160-bit hash)
• RIPEMD160 (160 bit hash)
• HAVAL
• More Complex Hashes

This is hardly an exhaustive list. All of these hashes (with the possible exception of CRC-32) are based on documented standards that have undergone extensive scrutiny by the number-crunchers.
The more bits in the hash, the less likely that some other file or string will have the same hash (accidentally, that is). And, also, the more difficult it is to easily fudge another file or string to yield the same hash. However, the algorithm itself also plays a role in how easy it is to crack a hash. The MD2 hash, for example, is identical in size to the MD5 hash, but the former is much easier to crack than the latter.
Also, it should come as no surprise that a really good 160-bit hash takes more computing time than a 32-bit hash to calculate.

 

Hash Algorithms (2)

CRC-32
(not provided by NIS File Check, but mentioned here for historical reasons).

Back in the days of 486 CPUs running at 33 MHz on an 8/16 MHz bus, very few people had PCs with telecommunications; even fewer were likely to have Internet access. The primary threat was a virus morphing a file by attaching or inserting itself. The most likely source of such a virus was an infected diskette or (if you were on a LAN), you could pick it up from some other user with whom you shared a file.
With the lack of PC processing power at that time, it was not a trivial operation to crack a CRC-32 hash, either. Today you can crack a CRC-32 in a matter of minutes, if not seconds.
You will still see CRC-32 hashes, however. ZIP utilities, for example, often use a CRC-32 to provide some assurance that a file has not been damaged during the ZIP or transmission process.
You can find a very brief discussion of CRC-32 and CRC-16 at http://www.oreilly.com/reference/dictionary/terms/C/Cyclic_Redundancy_Code.htm


MD2
(not provided by NIS File Check, but mentioned here for historical reasons).

The 128-bit MDn hashes represented a rather significant increase in complexity (and difficulty of cracking) when they were introduced. MD2 has long since been replaced, first by MD4 and then by MD5.
It is unlikely that you will see an MD2 hash, but they are authentic. You can find technical descriptions of MD5, MD4, and MD2 from http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc1321.html or http://rfc1319.x42.com/


MD5
(not provided by NIS File Check).

The 128-bit MD5 hashes (like those used by Zone Alarm (ZA), Zone Alarm Pro (ZAP), and Tiny Personal Firewall (TPF)) represent the end of the first-generation of MDn hashes. They are fairly robust, but the first publication of their being crackable dates back to around 1996. This is by no means a trivial task. But you should be aware that it has been (and can be) done. For our purposes, the important thing that you need to know is that an MD5 hash is more robust, i.e., likely to be unique, than an MD2 hash or a CRC-32 -- and also far more difficult to crack.
You can find technical descriptions of MD5, MD4, and MD2 from http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc1321.html


SHA1
(provided by NIS File Check).

SHA1 was developed with extensive support from the U. S. Government. Indeed, the National Institute of Standards and Technology (NIST) is the official keeper of the SHA1 standard. The SHA1 hash is currently used by Norton Internet Security (NIS) and Norton Personal Firewall (NPF).
So, why did Symantec not use the MD5 hash, like ZoneLabs did for its products? Two reasons:

• First, the SHA1 hash is more robust than the MD5 hash. It is longer (and consequently takes more time to compute), but it is also much more difficult to crack.
• Second, at least on current Microsoft systems, SHA1 is an integral part of the OS' authentication that is already in place. Indeed, if you take a look in your Windows directory (maybe in your Windows\System directory), you will already find an obscure little subfolder called \catroot. In that folder (maybe down a level or two), you will find a number of *.cat files. If you double-click on one of these files, it will open and you will find a list of attributes for system files (at least) that have been installed on your system. One of these attributes is the SHA1 hash. To some extent, Microsoft already makes considerable use of this hash to authenticate certain executables on your machine.

A reference to SHA1 and possible follow-on hash algorithms may be found at http://csrc.nist.gov/cryptval/shs.html . The standard reference may be found at http://www.itl.nist.gov/fipspubs/fip180-1.htm


RIPEMD160
(provided by NIS File Check).

The RIPEMD160 hash is developed in Europe, another 160-bit hash.
An overview of RIPEMD160 and a general comparison to SHA1 may be found at http://www.esat.kuleuven.ac.be/~bosselae/ripemd160.html


HAVAL
(provided by NIS File Check).

HAVAL is a one-way hashing algorithm that supports 15 different levels of security. It was designed in 1992. Its code was last revised in April 1997. While many of its peers, including MD4 and MD5, have been fully or partially broken, no successful attack on HAVAL has been reported so far. Hence it can serve as a "drop-in" replacement of MD5.
(Thanks to luv2bsecure for giving me the following info about HAVAL).
HAVAL is a cryptographic hash created by Yuliang Zheng, Josef Pieprzyk, and Jennifer Seberry  at the University of Wollongong in New South Wales, Australia.

In the book, "Applied Cryptography: Protocols, Algorithms, and Source Code in C" by famed cryptographer and crypto-analyst Bruce Schneier, he gives a good introduction to Haval.
---Some quotes from that book---
"HAVAL is a variable-length one-way hash function. It is a modification of MD5. HAVAL processes messages in blocks of 1024 bits, twice those of MD5. It has eight 32-bit chaining variables, twice those of MD5. It has a variable number of rounds, from three to five (each of which has 16 steps), and it can produce a hash length of 128, 160, 192, 224, or 256 bits."

"HAVAL replaces MD5's simple nonlinear functions with highly nonlinear 7-variable functions, each of which satisfies the strict avalanche criterion. Each round uses a single function, but in every step a different permutation is applied to the inputs. It has a new message order and every step (except those in the first round) uses a different additive constant. The algorithm also has two rotations."
---End quote---

There has been talk about an attack against HAVAL, but the accusation is flawed. The theory is based on it being developed on MD5 as a foundation, but the modifications make it a VERY safe hash - with the 5 pass method.

Some links for more info about HAVAL:

http://citeseer.nj.nec.com/zheng93haval.html

http://www.sis.uncc.edu/~yzheng/src/


More complex hashes
(not provided by NIS File Check, but mentioned here for historical reasons).

These come in several varieties.
First there are RIPEMD256 and RIPEMD320, which are even longer, more computationally intensive, and (presumably) more difficult to crack than RIPEMD160. As PCs become more powerful, you can expect to see more of these larger hashes in the future. There are also other hashes like HAVAL which are not necessarily sanctioned by any regulatory body, but which also are standardized implementations.
Second, there are the compound hashes, in which a combination of hash algorithms is used. Finally, there are the trick hashes. Fundamentally, all hash algorithms work by applying a mathematical computation to a binary array representing the file. Obviously, it is equally simple to prefix or postfix any (or any combination) of the following information to the file itself:

• the filename (with or without the file extension),
• (or alternatively) the fully-qualified pathname of the file
• the File Data Last Modified (File Date Last Accessed or File Creation Date are pretty much useless)
• the File Version information (internally from the file itself, for executable files)
• the File Size (in bytes)

Now, these trick solutions are all very well and good for an individual user. There is just one problem: You cannot compare your hash with anyone else's in the event you start to get worried about what you are seeing. At that point, you want the File Date Last Modified, the File Version, and the File Size as explicit and additional parameters, not as an integral component of the hash itself.

 

(probably) more to come ...