Ehdrv.sys causing STOP: F4 BSOD (? Buggy driver)

Cruachan · #1 September 11th, 2012, 09:43 AM

Hi Marcos,

Sorry, but it's me again

For some time now I've been experiencing intermittent STOP error BSOD crashes, but the info provided on the Blue Screen never mentions the driver involved. The STOP error code is F4.

WinDbg is not very helpful either.

Today I stumbled across a great little utility called 'BlueScreenView' and, finally, this has revealed the driver culprit and, I'm afraid to say, ESET NOD32 appears to be in the frame again.

Here is a link to a screen capture showing 3 Minidump crash files with quite a lot of useful info associated with each file.

https://backup.filesanywhere.com/fs/...8d616072beac6b

Operating System: Windows XP SP3

NOD32: current version 5.2.9.1

As ever, I would welcome your feedback. This is really quite annoying as it can happen anytime and quite unexpectedly.

Thank you.

Regards,

Mike

Marcos · #2 September 11th, 2012, 10:09 AM

Please upload the minidump itself instead of a screen shot. If possible, configure Windows to generate kernel memory dumps, reproduce BSOD and upload the kernel dump (in a compressed form).

Cruachan · #3 September 11th, 2012, 07:09 PM

Quote:

Originally Posted by Marcos

Please upload the minidump itself instead of a screen shot. If possible, configure Windows to generate kernel memory dumps, reproduce BSOD and upload the kernel dump (in a compressed form).

Hi Marcos,

Thanks for responding. I've PM'd the link to those minidump files as seen in the screenshot.

I've now configured Windows to generate kernal memory dumps, but it may be a while before the next BSOD as I can't reproduce them to order.

Meantime, perhaps something can be gleaned from those minidump files.

Regards,

Mike

siljaline · #4 September 12th, 2012, 09:00 PM

Was the BSOD encountered during an uninstall of your software ? If so, you may have a leftover driver. Remove it and post back your findings.

Cruachan · #5 September 13th, 2012, 08:12 AM

Quote:

Originally Posted by siljaline

Was the BSOD encountered during an uninstall of your software ? If so, you may have a leftover driver. Remove it and post back your findings.

Hi,

No, software uninstalling is not part of the equation.

It appears to happen at any time and, it would seem, quite unpredictably. I can be doing something as simple as browsing the Internet, Exploring a folder or working in Word and the first thing I notice is that the system seems to slow down and operations become sluggish. Thereafter, a lockup soon occurs and the system promptly crashes to a BSOD. At other times I can see a message for a short period in the Firefox title bar that it's not responding. It then goes away and I can resume operations. However, the system is sluggish at this point and a BSOD quickly follows. Right now I have been unable to identify a pattern and am unable to reproduce the crash on demand. Also, several days can go by between instances during which time the system appears quick and stable being up and running for many hours at a time. However, there have been rare occasions when I have returned to the computer after leaving it idling overnight to be met with a BSOD.

This may not be relevant, but I made a couple of discoveries last night:

1. On examining the entries under the Startup tab I came across a strange one pointing to a file 'PDBoot.exe' tucked away deep under Documents and Settings in an FSX (Flight Simulator X) sub folder. Now this file is used by PerfectDisk for boot time defrags and the only other instances appear in the System32 and remnant folders of previous installations of PerfectDisk (I have been upgrading this program for years). So, this file was a rogue and was easily removed. Also the entry in MSConfig was easily removed using CCleaner.

2. When I rebooted and confirmed that the PDBoot entry had gone I was alarmed to see that it had been replaced by vssadmin. The path this time pointed to vssadmin.exe, again deep under Documents and Settings, but this time in an FS9 (Flight Simulator 9) subfolder. When I browsed to this folder the folder appeared empty so I am assuming the file was hidden. I unchecked the entry in Msconfig but it was rechecked following the next boot. This behaviour persisted at each attempt to get rid of it. Furthermore, the entry did not appear in CCleaner!? In the end, I was able to see and delete the file while running in SAFE Mode. Thereafter the entry appeared in CCleaner and it was also deleted. I believe vssadmin.exe has something to do with Volume Shadow Copy. This is not running and is set to Manual under Services. The only other places it appears are the WINDOWS\Prefetch, WINDOWS\System32 and WINDOWS\System32\dllcache folders.

I deleted All the entries from the WINDOWS\Prefetch folder to give it a fresh start. Also I cleared out the relevant TEMP folders - Mine (under Local Settings) and WINDOWS.

Several reboots later and neither entry has reappeared in Msconfig. I scanned both files with ESET before deletion but nothing turned up. So, it would appear I had 2 unnecessary programs running at boot time and I do have to wonder whether these were malicious implantations.

One other thing noted over the past few weeks. From time to time the 'explorer.exe' process CPU resource usage jumps to 50% where it remains with a consequent fall in the System Idle Process percentage making the system less responsive. I'm now starting to suspect that this may in some way be related to the above discoveries. This can be resolved temporarily by ending the explorer.exe process in Task Manager and then restarting it.

It's difficult to understand where ESET fits into all of this and I suppose the ehdrv.sys involvement at the time of a crash may be purely coincidental and appears in the minidump simply because it was performing some operation at the time of the crash. An In-depth scan of my System partition C:\ plus Operating Memory and Boot Sector hasn't turned anything up. I'll do it again, just in case.

Today, the system appears fast, very responsive and stable. Explore.exe is currently sitting at 0% and after an hour or so of up time there has been no hint of trouble. I will continue to monitor.

Any clues yet from those minidump files?

Mike

Cruachan · #6 September 13th, 2012, 10:37 AM

Scan completed and nothing turned up:

I've PM'd the link to the log file and also the SysInspector export XML file to Marcos.

Mike

Cruachan · #7 September 19th, 2012, 04:09 PM

Hi Marcos,

Do you have any feedback yet regarding my various submissions?

In particular, I would be very interested in any theories ESET may have regarding the ehdrv.sys driver being flagged by 'BlueScreenView' on each of those occasions when my rig crashed and minidumps were generated.

On a positive note, since my last post there has been no recurrence of BSODs and the system remains responsive and stable. The explorer.exe process continues to behave normally without the disproportionate and persistent consumption of resources as was seen before.

Now all this may indicate that I have managed to resolve this issue by removing those two executables that were being called at system boot (PDBoot.exe, vssadmin.exe). However, I am at a loss for an explanation as to why they appear to have affected the stability of the system and, indeed, how both were implanted in msconfig and for what reason other than malicious intent.

Your thoughts on these matters would be most welcome if only to put my mind at rest regarding NOD32 which otherwise seems to be working well.

Regards,

Mike

Cruachan · #8 September 24th, 2012, 08:27 AM

I'm trying to be patient, but there is a strong feeling of déjà vu creeping in here.

It seems that when ESET doesn't know the answer the best policy is simply to ignore you rather than admit to anything. Tell me, how does that policy win the respect of your paying and loyal customers?

My renewal date for NOD32 is fast approaching. This will require some serious thought as I'm not sure I can trust this AV solution any more. I tried my best to be helpful. I asked for some clarification and reassurance and none has been offered, neither here nor via PM. Seems pointless now to persist with this topic.

You really should take a leaf out of Raxco's Technical Support team's book. My experience is that they will make every effort to help when assistance is requested and will persist until satisfied an issue has been resolved. Take, for example, the external USB drive connection/system crash issue. We were never really given a full explanation as to why ESET were pointing the finger at various 3rd Parties to take full responsibility rather than accepting that they should at least shoulder some of the blame for the protracted period of chaos and inconvenience caused. All they had to do was provide us with an effective workaround until everyone got their act together and sorted it out.

Very disappointed.

Mike

Marcos · #9 September 24th, 2012, 09:50 AM

The problem was that the respective ticket was initially assigned to a developer working on high priority task, hence the delay. If the issue was tackled via customer care, they would have received a notification about this and could act accordingly. Anyways, I've asked another developer to look into it. We assume that ehdrv.sys may not be necessarily responsible for crash; when a user-mode application crashed ehdrv is marked as the cuplrit due to the way it works. We either suspect a hard drive fault or snapman.sys which is an Acronis driver.
That said, I'd suggest the following:
1, rename c:\windows\system32\drivers\ehdrv.sys in safe mode and try to reproduce the crash
2, if renaming ehdrv fixes the issue, uninstall (tempoarily) the Acronis sw you have installed and see if the crash occurs in this scenario

In order to investigate the issue deeper, we'd need to get a complete memory dump from BSOD for analysis.

Cruachan · #10 September 24th, 2012, 11:47 AM

Hi Marcos,

Thank you for responding. However, I do wonder why I was not advised about the reasons for the delay before now as it would have avoided my having to post in the manner I did to get someone's attention.

Not another application! Now Acronis is in the frame, for many of us another vital product we have grown to rely on and trust. I Have Acronis True Image Home 2011, Update 3 (build 6942) and Disk Director Suite, version 10 (build 2160). Yes, these are not the latest versions, but have both proved stable and reliable with XP SP3.

It does seem unlikely that my HD is faulty as no further BSODS have occurred since the removal of those presumed malicious implantations of PDBoot.exe and vssadmin.ex (see post #5 above). Incidentally, I would still welcome your comments about this finding. My system seems stable, generally faster and more responsive since those files were removed. The originals remain in their proper folders. No further rogue entries have appeared under the 'Startup' tab of Msconfig.

Also why would ehdrv.sys be implicated while these executables were running unless, of course, it had something to do with a potential system security breach? Why did ESET not report this?

How could snapman.sys be conflicting with ESET when the Acronis software isn't running? In particular, I've verified that snapman.sys isn't running when Acronis True Image Home is started.

Sorry to keep firing all these questions at you, Marcos. I'm just trying to understand what has been happening.

Regards,

Mike

Marcos · #11 September 24th, 2012, 12:18 PM

Quote:

Originally Posted by Cruachan

Not another application! Now Acronis is in the frame, for many of us another vital product we have grown to rely on and trust. I Have Acronis True Image Home 2011, Update 3 (build 6942) and Disk Director Suite, version 10 (build 2160). Yes, these are not the latest versions, but have both proved stable and reliable with XP SP3.

Also why would ehdrv.sys be implicated while these executables were running unless, of course, it had something to do with a potential system security breach? Why did ESET not report this?

I didn't state that Acronis is the culprit but it could be involved in the crash as the minidump showed snapman.sys in the stack.
We will definitely need a complete memory dump from BSOD in order to figure out the cause of the crash. Minidumps generally show very little information about crashes and a kernel memory dump won't suffice either as it is a user-space process that is crashing in this case.

Cruachan · #12 September 24th, 2012, 06:14 PM

Hi Marcos,

Out of curiosity I ran the latest minidump file (created by the 'F4' BSOD) through WinDbg again and this is the output:

Code:

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\WINDOWS\Minidump\Mini090712-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows XP Kernel Version 2600 (Service Pack 3) MP (2 procs) Free x86 compatible
Product: WinNt
Built by: 2600.xpsp_sp3_gdr.120504-1619
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720
Debug session time: Fri Sep  7 01:07:25.203 2012 (GMT+1)
System Uptime: 0 days 0:04:48.957
Loading Kernel Symbols
..............................................................................................................................................................
Loading User Symbols
Loading unloaded module list
..........
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck F4, {3, 89a48020, 89a48194, 805d22aa}

Probably caused by : ntkrpamp.exe ( nt!PspCatchCriticalBreak+75 )

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 00000003, Process
Arg2: 89a48020, Terminating object
Arg3: 89a48194, Process image file name
Arg4: 805d22aa, Explanatory message (ascii)

Debugging Details:
------------------


PROCESS_OBJECT: 89a48020

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT

BUGCHECK_STR:  0xF4

EXCEPTION_RECORD:  b82879d8 -- (.exr 0xffffffffb82879d8)
ExceptionAddress: 75b7b421
   ExceptionCode: c0000006 (In-page I/O error)
  ExceptionFlags: 00000000
NumberParameters: 3
   Parameter[0]: 00000008
   Parameter[1]: 75b7b421
   Parameter[2]: c0000185
Inpage operation failed at 75b7b421, due to I/O error c0000185

LAST_CONTROL_TRANSFER:  from 805d13f3 to 804f9f5f

STACK_TEXT:  
b8287520 805d13f3 000000f4 00000003 89a48020 nt!KeBugCheckEx+0x1b
b8287544 805d2355 805d22aa 89a48020 89a48194 nt!PspCatchCriticalBreak+0x75
b8287574 8054168c 89a48268 c0000006 b82879b0 nt!NtTerminateProcess+0x7d
b8287574 8050117d 89a48268 c0000006 b82879b0 nt!KiFastCallEntry+0xfc
b82875f4 804fe832 ffffffff c0000006 b82879f8 nt!ZwTerminateProcess+0x11
b82879b0 805028f5 b82879d8 00000000 b8287d64 nt!KiDispatchException+0x3a0
b8287d34 80544f6f 00e6fbe8 00e6fc08 00000000 nt!KiRaiseException+0x175
b8287d50 8054168c 00e6fbe8 00e6fc08 00000000 nt!NtRaiseException+0x33
b8287d50 75b7b421 00e6fbe8 00e6fc08 00000000 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
00e6fff4 00000000 00000000 00000000 00000000 0x75b7b421


STACK_COMMAND:  kb

FOLLOWUP_IP: 
nt!PspCatchCriticalBreak+75
805d13f3 5e              pop     esi

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!PspCatchCriticalBreak+75

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrpamp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  4fa3cc43

FAILURE_BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 00000003, Process
Arg2: 89a48020, Terminating object
Arg3: 89a48194, Process image file name
Arg4: 805d22aa, Explanatory message (ascii)

Debugging Details:
------------------


PROCESS_OBJECT: 89a48020

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT

BUGCHECK_STR:  0xF4

EXCEPTION_RECORD:  b82879d8 -- (.exr 0xffffffffb82879d8)
ExceptionAddress: 75b7b421
   ExceptionCode: c0000006 (In-page I/O error)
  ExceptionFlags: 00000000
NumberParameters: 3
   Parameter[0]: 00000008
   Parameter[1]: 75b7b421
   Parameter[2]: c0000185
Inpage operation failed at 75b7b421, due to I/O error c0000185

LAST_CONTROL_TRANSFER:  from 805d13f3 to 804f9f5f

STACK_TEXT:  
b8287520 805d13f3 000000f4 00000003 89a48020 nt!KeBugCheckEx+0x1b
b8287544 805d2355 805d22aa 89a48020 89a48194 nt!PspCatchCriticalBreak+0x75
b8287574 8054168c 89a48268 c0000006 b82879b0 nt!NtTerminateProcess+0x7d
b8287574 8050117d 89a48268 c0000006 b82879b0 nt!KiFastCallEntry+0xfc
b82875f4 804fe832 ffffffff c0000006 b82879f8 nt!ZwTerminateProcess+0x11
b82879b0 805028f5 b82879d8 00000000 b8287d64 nt!KiDispatchException+0x3a0
b8287d34 80544f6f 00e6fbe8 00e6fc08 00000000 nt!KiRaiseException+0x175
b8287d50 8054168c 00e6fbe8 00e6fc08 00000000 nt!NtRaiseException+0x33
b8287d50 75b7b421 00e6fbe8 00e6fc08 00000000 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
00e6fff4 00000000 00000000 00000000 00000000 0x75b7b421


STACK_COMMAND:  kb

FOLLOWUP_IP: 
nt!PspCatchCriticalBreak+75
805d13f3 5e              pop     esi

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!PspCatchCriticalBreak+75

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrpamp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  4fa3cc43

FAILURE_BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 00000003, Process
Arg2: 89a48020, Terminating object
Arg3: 89a48194, Process image file name
Arg4: 805d22aa, Explanatory message (ascii)

Debugging Details:
------------------


PROCESS_OBJECT: 89a48020

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT

BUGCHECK_STR:  0xF4

EXCEPTION_RECORD:  b82879d8 -- (.exr 0xffffffffb82879d8)
ExceptionAddress: 75b7b421
   ExceptionCode: c0000006 (In-page I/O error)
  ExceptionFlags: 00000000
NumberParameters: 3
   Parameter[0]: 00000008
   Parameter[1]: 75b7b421
   Parameter[2]: c0000185
Inpage operation failed at 75b7b421, due to I/O error c0000185

LAST_CONTROL_TRANSFER:  from 805d13f3 to 804f9f5f

STACK_TEXT:  
b8287520 805d13f3 000000f4 00000003 89a48020 nt!KeBugCheckEx+0x1b
b8287544 805d2355 805d22aa 89a48020 89a48194 nt!PspCatchCriticalBreak+0x75
b8287574 8054168c 89a48268 c0000006 b82879b0 nt!NtTerminateProcess+0x7d
b8287574 8050117d 89a48268 c0000006 b82879b0 nt!KiFastCallEntry+0xfc
b82875f4 804fe832 ffffffff c0000006 b82879f8 nt!ZwTerminateProcess+0x11
b82879b0 805028f5 b82879d8 00000000 b8287d64 nt!KiDispatchException+0x3a0
b8287d34 80544f6f 00e6fbe8 00e6fc08 00000000 nt!KiRaiseException+0x175
b8287d50 8054168c 00e6fbe8 00e6fc08 00000000 nt!NtRaiseException+0x33
b8287d50 75b7b421 00e6fbe8 00e6fc08 00000000 nt!KiFastCallEntry+0xfc
WARNING: Frame IP not in any known module. Following frames may be wrong.
00e6fff4 00000000 00000000 00000000 00000000 0x75b7b421


STACK_COMMAND:  kb

FOLLOWUP_IP: 
nt!PspCatchCriticalBreak+75
805d13f3 5e              pop     esi

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!PspCatchCriticalBreak+75

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrpamp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  4fa3cc43

FAILURE_BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

BUCKET_ID:  0xF4_nt!PspCatchCriticalBreak+75

Followup: MachineOwner
---------

Now I don't pretend to understand what I'm seeing here, but there does not appear to be any mention of snapman.sys.

Also BlueScreenView only mentions 2 drivers found in the crash stack: ntkrnlpa.exe and ehdrv.sys. No snapman.sys.

It would seem that you are not prepared to speculate as to why the removal of the rogue PDboot.exe and vssadmin.exe files has had a positive effect on the stability of my system. Perhaps you don't know, so why not just say so? I won't think any less of you. Moreover, it does seem more likely that I am not going to see another STOP: F4 BSOD anytime soon and I am unable to recreate the situation responsible for causing it. So it would appear we have reached an impasse.

In the event that an 'F4' crash does happen again I have reconfigured XP to create a complete memory dump as you have suggested.

FWIW I changed my HD a few months back as part of my attempts to rid myself of these recurrent 'F4' crashes. However, it continued to happen intermittently until, that is, I removed the rogue implants PDboot.exe and vssadmin.exe from running at startup. Why? That is the question I would like answered.

Regards,

Mike

Marcos · #13 September 25th, 2012, 04:38 AM

You didn't list the content of the stack so you couldn't see snapman.sys being there.
As for PDboot.exe and vssadmin.exe, if you suspect them to be rogue applications that should be detected, please submit them to ESET as per the instructions here.

Cruachan · #14 September 25th, 2012, 03:37 PM

Quote:

Originally Posted by Marcos

You didn't list the content of the stack so you couldn't see snapman.sys being there.
As for PDboot.exe and vssadmin.exe, if you suspect them to be rogue applications that should be detected, please submit them to ESET as per the instructions here.

Hi Marcos,

I thought I had listed the content of the crash stack: ntkrnlpa.exe and ehdrv.sys

In fact I described PDBoot.exe and vssadmin.exe as rogue implants which were being called at startup (See 1. and 2. in post on 13 September). Regretably I failed to make copies of these files before deleting them so we have no way of determining whether these executables had been modified or whether these implants were copies of the original files.

It's clear now that this discussion is going nowhere as I am unable to comply with your requests. I accept it would be unfair to expect you to speculate without more concrete information so I suggest we now draw a line.

My system is running fine at present. The measures taken appear to have restored performance and stability, but the mystery remains as to why this is so. I suppose I should be happy, whereas in actual fact the absence of an explanation, even an educated guess, continues to frustrate me. My own fault, I suppose.

Thank you very much for your forbearance.

Kind regards,

Mike

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search