Severe PG 3 slowdown when Alerts tab fills up

Discussion in 'ProcessGuard' started by LuckMan212, Dec 3, 2004.

Thread Status:
Not open for further replies.
  1. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    Hi guys,
    I did some heavy batch-filing today and there were several hundred executions, perhaps spaced out by about 5-10 seconds. As the batch file ran, I noticed my machine slowing down exponentially. I checked Task Manager and Procguard.exe was eating 70-80% cpu. The GUI window was closed, but the tray icon was running.

    After the batch file completed, Procguard.exe continued to take 50-60% cpu, even though I was doing nothing. As soon as I cleared the "Alert" tab list, Cpu went back to 0%. So it seems that there may be some problem sorting or even maintaining in memory a large alert list.

    Perhaps this code can be optimized to handle this better? It is not uncommon for me to have several hundred executions in a session. Or could something else be causing this? It is indeed strange.
     
  2. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Hi LuckMan212, There may well be some optimisation possible to the alert log but for now you could try switching of Execution protection temporally whilst running your batch files.

    Jason will have to answer from the technical point of view.

    Thanks. Pilli
     
  3. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    perhaps there could be a commandline that could be run to temporarily disable PG from the batch file itself? it runs at night unattended so I am not there to manually disable PG usually.

    some sort of secure command line disabling would be nice (of course with option to disable it for users who dont want the risk)

    example:
    Code:
    >pgcontrol.exe -disable -password:xyzzy
    of course password set via the GUI somehow beforehand and stored encrypted in pgdata files.

    bad idea / good idea?
     
  4. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Jason will have to comment on your idea, using a command line sounds like an interesting idea.
    Another thought occurred to me about this, ProcessGuard is still protecting your PC if the procguard.exe (GUI) is not running, so exiting the GUI completely may also stop some or all of the logging - Worth a try :)

    Pilli
     
  5. Andreas1

    Andreas1 Security Expert

    Joined:
    Jan 29, 2003
    Posts:
    367
    Location:
    Mainz (Ger)
    To be more specific (at least as far as I understood it): The service does the logging to the logfiles, but the logwindow is the gui's business. So you would most certainly avoid problems with the log window (as you seem to be having insofar as clearing the log window seems to help), but you would still have all the information in the log files...

    Andreas
     
  6. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    Thanks for the ideas guys, but... even if closing the GUI was a solution it's not an ideal one for 2 reasons:

    1) program should be able to handle this gracefully without needing to close the GUI

    2) I am not around in the middle of the night to manually close the GUI or exit the tray icon anyway

    Hopefully Jason can comment on this :)
     
  7. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Agreed and I am sure Jason will comment on this

    Surely if you are not around it does not matter? As the GUI does not need to be running for protection to work, your logfile will be there when you next open the GUI, just close the GUI before you leave the machine. As we have suggested it is worth a try and I am sure Jason will look into the matter ASAP.

    Pilli
     
  8. Jason_DiamondCS

    Jason_DiamondCS Former DCS Moderator

    Joined:
    Nov 11, 2002
    Posts:
    1,046
    Location:
    Perth, Western Australia
    I would guess the issue is "sorting" the list with many thousands of items is causing the CPU usage issue. I might remove all sorting from the ALERTS list because it isn't as useful there and would resolve any CPU usage issue with it.
     
  9. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    sounds good to me! :D thanks!
     
  10. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    One caveat is that, if EXECUTION alerts cannot be disabled separately, it is a tedious job to browse hunderds or thousands of alerts looking for anything important.
     
  11. siliconman01

    siliconman01 Registered Member

    Joined:
    Mar 6, 2003
    Posts:
    780
    Location:
    West Virginia (USA)
    Maybe a user defined option that is "clear all alerts more than xx hours old" at a user defined time of day? Just a thought. :D
     
  12. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    I was thinking more of a case where a batch script runs all night, then trying to look through thousands of EXECUTION alerts the next morning. Not sure what's best to suggest. The log file is available, but Unicode makes it difficult to use with most programmer tools. Also (I think) the log file tends to omit the process name when a hook is blocked (but maybe that's getting fixed :) ).

    Perhaps new alerts could be appended to the window, and the window's contents "marked as dirty". When the API send you a message to update the window, you could re-sort at that time. That may be a fairly long sort though (as opposed to insertions). Looks like a lousy idea the more I think about it.
     
  13. gottadoit

    gottadoit Security Expert

    Joined:
    Jul 12, 2004
    Posts:
    601
    Location:
    Australia
    Maybe a little options box somewhere that specified how many of the alerts are to be kept in memory could resolve the issue by avoiding the problem of CPU and memory use....

    Something like a tickbox "limit alert messages in memory"
    and a numeric counter "number of messages in memory"
    for the full monty: a tickbox "disable alert sorting"
    and make the default to allow sorting with 50 messages in memory

    These options could go onto the new "options" tab when the "advanced" mode happens (ever hopeful) as one of the many little options
     
  14. Andreas1

    Andreas1 Security Expert

    Joined:
    Jan 29, 2003
    Posts:
    367
    Location:
    Mainz (Ger)
    As a sidenote:
    This is working on my comp.:
    type pglog_11*.txt | grep something

    (with grep and type from unxutils)
     
  15. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    I'm no longer struggling to search/process log files now. I discovered that my editor (Vim) handles it as easily as it handles ascii. I thought I'd already tried that and failed, but it must have been before I lost my mind. :)

    I do still have a concern about not having the program name in log files when PG blocks a program. It does appear (albeit, sometimes, erroneously) on the Alert Tab. Once that's fixed, I'd be fine with an abbreviated alert screen. Truth be told, I'm not too bothered either way. :)
     
  16. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    Andreas, your sidenote looks like something interesting, but I may be confused. I would have expected either a DOS-style "type" or a UNIX-style "cat" to pipe the input to grep. The "type" in UnxUtils.zip is different than what I'm used to (MKS toolkit) but neither "type" writes the contents of a file to stdout.

    I tried the UnxUtils' version of grep.exe, but could only match "EXECUTE" as "E.X.E.C.U.T.E" because of the multi-byte characters. Even though the second version matched, grep still complained, "Binary fiile (standard input) matches", and would not display the matching text. My grep didn't even cooperate that much.

    I didn't see a switch for grepping multi-byte characters, so I'm wondering whether you convert log files to ascii before grepping, or perhaps that conversion happens transparaently on your system. o_O

    In any case, thank you for the pointer to UnxUtils.zip. I saw some handy new grep switches, and I'm sure there'll be other useful things, too.

    Regards, Mike (sorry if OT)
     
  17. Andreas1

    Andreas1 Security Expert

    Joined:
    Jan 29, 2003
    Posts:
    367
    Location:
    Mainz (Ger)
    sorry off-topic still ;)
    hi earth,
    in fact, I've simply forgotten about 'cat'. I tried 'less' first, but it obviously couldn't handle unicode and produced garbage. Then I tried 'type' and it worked. But it seems that i was mistaken in thinking i was using the 'type.exe' that is in my unxutils folder. While that folder is in my search path, obviously what got called is w2k's cmd.exe's "internal" type. (I've now explicitly stated the path to the unxutils in calling and got a PG Execution protection request for type.exe - which i don't remember having gotten before. DOH!) Sorry for the confusion.

    Andreas

    And yes .. . . .. . vim rules!!
     
    Last edited: Dec 6, 2004
  18. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    my thread's been hijacked! :eek:
     
  19. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    Sorry Luckman, I got carried away trying to make sure I'd know what to do without my many thousands of sortable Alerts. :)
     
  20. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Hi earth1, Maybe you and Andreas should contact each other via the boards IM :D I still find the topic interesting though :)
     
  21. Andreas1

    Andreas1 Security Expert

    Joined:
    Jan 29, 2003
    Posts:
    367
    Location:
    Mainz (Ger)
    LuckMan,
    could you -- for the time being -- confirm whether or not closing the gui solves the CPU problem. I.a.w. does the CPU troubles also occur when only the service does the file logging?

    and sorry for the ... ehm ... off-topicity

    Andreas
     
  22. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    I will check that out and post back when I have the results... and no hard feelings for the hijack, I was just kidding. Interesting stuff.... :p
     
  23. gottadoit

    gottadoit Security Expert

    Joined:
    Jul 12, 2004
    Posts:
    601
    Location:
    Australia
    It would also be interesting to know if you have to click on any of the headings to do explicit sorting prior to the CPU usage happening, and also to know if that is done does the CPU usage get any worse ?

    ie: if you sort by the program name then it will need to do some work and move things around in the list box so it should be more CPU intensive
     
  24. earth1

    earth1 Registered Member

    Joined:
    Oct 17, 2004
    Posts:
    177
    Location:
    Kansas, USA
    I started poking around and got some surprising results. When I got past 2000 alerts things started slowing towards a crawl. At that point, it was pretty easy to start measuring CPU spikes with taskmgr on fast update. It took about half a second of procguard time to run a little do nothing program. The following is a rough estimation of what taskmgr attributed to procguard for CPU usage.

    Inserting one entry (running the program once more) tended to be consistent in its time requrement. Clicking on a column heading in procguard's Alert Tab varied from a low that equaled the insert time, to a high of about 50% greater. The average seemed to be about 15% greater than the insert time.

    I wonder if procguard may be re-sorting the entire list of alerts each time it adds one (as opposed to traditional insert logic) and/or sending one or more Windows messages for every entry when it adds one. The latter might best explain why there seems to be a point at which the system bogs down so completely.
     
  25. LuckMan212

    LuckMan212 Registered Member

    Joined:
    Aug 19, 2004
    Posts:
    252
    earth your results are consistent with mine and I appreciate your thorough testing.... I have not had a chance yet to confirm that number but it "sounds" right. I will post my results but hopefully Jason can chime in. Perhaps the sort routines can be optimized? I know that for example MS Excel can sort 5,000 rows in a basically unmeasurably short time (at least on my system) so perhaps there is some room for improvement in the sort algorithm. Or perhaps a piece of 3rd party sort code can be used as I know most programmers do not like to write such routines themselves. Reinventing the wheel and all that....
    :blink:
     
Thread Status:
Not open for further replies.