Problem with Trusted Zone in Firewall ESS 4 installed on WHS

I have installed ess 4 on my Windows Home Server (HP MediaSmart) primarily due to the fact that it is the only firewal + AV product that includes WHS in its spec sheet (and I have tried the small business editions of several products, they disable the firewall in the windows home server).

I realize that installing a firewall on a "headless" server is a bit of russian roulette, and also have heard folks say that the Windows Firewall is good enough (it isn't, trust me I am currently under attack! and had to completely stealth my entire media server at the router so that NOTHING can get in or out). Due to the circumstances, I will not feel comfortable until I have a firewall that inclusions intrusion detection, behavioral analysis, and root kit detection (none of which the Windows Firewall can do).

So I am willing to take my chances, but I just cannot override the firewall's insistance on blocking itself out of existence!!!!! See below.

Here is what I did first: (1) add my entire subnet to the Trusted Zone (192.168.1.0 with a netmask of 255.255.255.0), (2) selected Automatic Filtering with Exceptions. With that setup in place, I figured I was good to go - or at least I figured I was safe because I should always be able to remote in to my WHS box from a local area connection if all else is blocked. Right?

Wrong. As soon as I applied those changes and activated the firewall, I lost my remote desktop, and my WHS server disappeared!!!

I thought I was done for, but luckily there is an ever so slight security weakness in the ESS firewall - it takes a few minutes when rebooting before it applies filters to the network ports. After several frantic reboots and attempted remote logins, I finally got into the WHS just in time to disable the firewall before it filtered itself out of existence. Whew.

I fired up the configurations for the ESS firewall, and added a new ALLOW rule (remember, my entire subnet is already in the Trusted Zone) as follows: Added a new ALLOW rule so that ALL traffic on ANY PORT from any SOURCE to any DESTINATION is allowed IFF BOTH the SOURCE and DEST are in the Trusted Zone. That should do it, I fooled myself into thinking. Turn the firewall back on and ... you guessed it, the WHS server disappears!

Several reboots later I manage to disable the firewall again remotely (FYI, don't try this at home! I noticed that it becomes harder and harder to get remote access to the system during what appears to be an ever decreasing length of time before the filters go live).

This time, I pulled up the firewall log to see what might be happening. See below, and note that 192.168.1.79 is my WHS media server; 192.168.1.17 and 192.168.1.20 is my laptop (which is running with very capable Norton 360 software, so it is very unlikely that my laptop is infected or compromised). .23 is another machine that is running firewall protection from Bitdefender (yes, I know I should upgrade this) so also pretty unlikely to be infected.

What is going on? From the logs it looks like, even though these machines are in the trusted zone, ESS has decided they are security threats and summarily shut them down. I doubt they are infected, so it seems like a detection error - but a hugely costly one if someone is ultimately locked out of their firewalled headless server!!!

More importantly, (1) how do I tell the firewall to allow AT LEAST the remote desktop connection from a specified machine and have THAT rule ONLY override the behaviorual block that appears to be causing the problem? I could find no way to prioritize rules or prioritize A RULE over the automatic actions. And, (2) in general, how does ESS prioritize the rules? I notice that there is no way to alter the priority of a user entered rule versus the "preset" rules.

Or is there something else I am missing?


Here is the relevant log portion:

Code:

9/19/2010 11:19:22 AM	Packet blocked by active defense (IDS)	192.168.1.79:139	192.168.1.23:58610	TCP			
9/19/2010 11:19:20 AM	Packet blocked by active defense (IDS)	192.168.1.79:139	192.168.1.23:58610	TCP			
9/19/2010 11:19:18 AM	Packet blocked by active defense (IDS)	192.168.1.79:139	192.168.1.23:58610	TCP			
9/19/2010 11:19:16 AM	Packet blocked by active defense (IDS)	192.168.1.79:139	192.168.1.23:58610	TCP			
9/19/2010 11:19:14 AM	Packet blocked by active defense (IDS)	192.168.1.79:139	192.168.1.23:58610	TCP			
9/19/2010 11:17:54 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:54 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:54 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:43 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:43 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:40 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:38 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:35 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:34 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:33 AM	Packet blocked by active defense (IDS)	192.168.1.23:58612	192.168.1.79:1138	TCP			
9/19/2010 11:17:29 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:29 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:27 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:27 AM	Packet blocked by active defense (IDS)	192.168.1.79:1138	192.168.1.23:58612	TCP			
9/19/2010 11:17:26 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:25 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:24 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:17:24 AM	Packet blocked by active defense (IDS)	192.168.1.23:58610	192.168.1.79:139	TCP			
9/19/2010 11:03:49 AM	Detected Port Scanning attack	192.168.1.23:58589	192.168.1.79:80	TCP			
9/19/2010 10:58:52 AM	Detected Port Scanning attack	192.168.1.20:3208	192.168.1.79:80	TCP			
9/19/2010 10:48:29 AM	Detected Port Scanning attack	192.168.1.23:58423	192.168.1.79:445	TCP			
9/19/2010 10:38:00 AM	Detected Port Scanning attack	192.168.1.20:2956	192.168.1.79:1138	TCP			

 

Hello,

Generally speaking, ESET Smart Security is not recommended for use on Microsoft Windows Home Server's because of the issue with administering ESET Personal Firewall on a headless system. Specific details are found in:

ESET Knowledgebase #2212, "How do I remotely install ESET NOD32 Antivirus (4.x) on Windows Home Server?"

Since ESET Smart Security is already installed, though, what I would suggest is that you try configuring the firewall in Learning Mode and then running it that way for several days until you have built a ruleset for RDP and other tasks you perform remotely against the server. Information on enabling learning mode can be found in ESET Knowledgebase #2236, "Why did ESET Smart Security notify me that the usage period for Learning mode has elapsed and then block all communication? (4.x)."

Despite the specific-sounding title, the article is more of an overview about the firewall's Learning Mode and how to use it.

Regards,

Aryeh Goretsky

 

Thanks, but my firewall rules are fine. I have a lot of experience in firewall setup, and I know how to do this. Adding my entire LAN subnet to the trusted zone, AND adding an extra ALLOW rule to allow all traffic on any port, any direction, within the Trusted Zone, is more than enough as far as rules go.

The problem is that (a) eset is falsely detecting port scanning and some other kind of "attack" when none exists, (b) eset (understandably) elevates is detection of portscanning/hacking above my rules, thus creating a lockout, and (c) there is no way (that I can find) to "correct" the mistaken identification of hacking from my own secure computer or to override this detection and lockout without disabling the feature entirely. This is a limitation in ess that should be corrected (or if there is a way to correct I would very much like to know).

Learning mode would only complicate matters. Even after enabling learning mode and developing a full set of rules, the problem will still be there because the issue is NOT with Windows Home Server itself or my rules, but instead is due to a mistaken "hack"/port scanning detection and the inability of the ess firewall to prioritize a specific allow rule over the behavioral rules that are falsely detecting the hacking attempt.

I have been running with full auto mode + my exception rules above without any issue for 2 days now... what I did was disable the specific behavioral rules that were triggered in the logs - detection of port scanning - which is clearly a defect in the firewall. I also disabled the option to lock out IP addresses that are determined to be hackers and instead just logged hacking attempts. That seems to have done the trick. Anyone who is planning to run the latest and greatest ess firewall should not be afraid to do so provided they disable the IP lockout and portscan detection, and add the LAN subnet to the Trusted Zone.

It is too bad that there is no way to elevate a rule to prioritize over the behavioral algorithms though. Disabling portscan detection and IP lockout severely weakens the firewall (though its still probably better than Windows Firewall). It would be really great if eset could look into this.

 

Hello,

Since you are certain the firewall rules are fine, it is probably best to open a ticket with support in order to see exactly what's going on, network traffic-wise.

You can use the form at this page on ESET's web site to initiate the request.

Since this particular issue is with the firewall, you'll want to generate a packet capture, which you can do by following these instructions.

Regards,

Aryeh Goretsky

 

Thank you... I am just evaluating the product. If I do decide to purchase, I will definitely follow through. I did already do a packet capture to go along with the log though I don't want to repeat the experience of being locked out of my headless server so unfortunately I can't do much in the way of troubleshooting!

Someone mentioned in a PM that there was an option for a list of IP addresses that would NOT be subject to IDS. I could not find that option, it would be exactly what I was hoping to find. If such an option exists, could someone point me to it?