Are advanced rootkits going to bring about fundamental changes in operating systems?

Hi,

I have been reading about the growing complexity and danger of rootkits. Recently, MS security people have been warning about advanced spyware and other malware using rootkits to hide themselves.

Now, ProcessGuard is certainly a good solution, for preventing rootkits from being installed in the first place. But not everyone will run PG, and since PG is software, designed to defeat other software...

it can, in turn, be defeated by software targeting it. Nothing is perfect, as the saying goes. And this statement brings into question the worthiness of creating a new, modern OS, written from the ground up to address security concerns... Maybe it's not really a worthy goal after all. Anyway....

So is it possible to develop an effective scanning routine, like that which exists with viruses and trojans, to detect rootkits? From the system that the rootkit has embedded itself? I know that remote scans from another system are the best bet for detection, at the current time.

As for rewriting major operating systems...well, MS and *nix variants are based upon ideas and architecture that are pretty old. They have new ideas slapped on top of the old architecture, true, but there are burgeoning problems (more so with MS, it seems) related to backwards compatibility, and growing intertwining of system functions. Just massive complexity resulting from trying to shoehorn new ideas to work with very old (in computer years) underlying architecture.

This complexity, and security issues that weren't relevant many years ago, are big factors in the huge advantage rootkits have over current detection methods. The sheer complexity of interactions within the OS allows for hiding things away, and the complexity allows for lots of security failures, buffer overflows, escalation attacks, etc.

I was wondering if the world will see a totally new architecture, designed from the ground up to address security concerns. One that tosses aside all the less functional ideas in *nix and Windows.

Rootkits and software design are thought provoking topics, at least. If I'm way off base, someone will come along and correct me But I hope to get some discussion started.

 

Any operating system which allows it's features to be extended by software on it will be prone to rootkit like software. In Windows, antivirus companies first started using technology which rootkits later used to hide themselves. If you allow a valid program to be able to say a file doesn't exist on the file system, then malware will use it. If you allow a valid program to deny execution requests, then malware will use it, etc.

The second area to be covered is patching the operating system to make it do what you want, which sort of ties in to the first area. Windows currently allows you to do this, though there is some speculation this functionality will be removed in Longhorn. A lot of software does patch the kernel so it is going to be interesting what happens in regards to that. Also if the operating system has to "patch" itself with updates, like you think any OS would be able to do, it means there is a vector of attack for malicious users to put things like rootkits in there too.