perftcp/perfudp

[alex_s]

Does anybody have old config for these two tests ? I'd like to test the latest OA with them but those old tests do not run with the new tests set (where these two tests are removed. I get:

D:\Pub\LeakTests\Matousec\bin\Level 1>perfudpsrv.exe
Security Software Testing Suite - PerfUDPsrv
Copyright by Matousec - Transparent security
http://www.matousec.com/

Configuration for this test was not found in file "ssts.conf".

:(

[alex_s]

Thanks good people for providing me with the old perf tests, for new tests do not work here whatever I do claiming wrong config file.

Using OA v3.5.0.2 (the latest beta), I've got the following interesting results:

perfudp (which was the most "weak" part for the most firewalls.

pure system:

1.) 22.978 sec
2.) 23.431 sec
3.) 23.103 sec

windows firewall on

1.) 25.038 sec
2.) 25.147 sec
3.) 25.225 sec

OA

1.) 22.621 sec
2.) 22.634 sec
3.) 22.241 sec

I'd say the results are confusing. For one Windows Firewall which I thought should be the fastest is actually not that fast and produces ~2 sec degradation which makes ~86% performance. For two it seems that with OA it works faster than with just clean system. But is this possible ?

Everybody is welcomed to join the tests. I can email them on demand. I'd also like that anybody else tested the latest OA and shared his results, for I do not dare to believe my own :)

[alex_s]

OA v3.5.0.2, perftcp

pure system

1.) 22.861 sec
2.) 22.863 sec
3.) 22.896 sec

Windows Firewall

1.) 22.746 sec
2.) 22.842 sec
3.) 22.872 sec

OA

1.) 22.850 sec
2.) 22.841 sec
3.) 22.966 sec

Perftcp seems to show no visible affect. In all the three configs the result is inside allowed statistical mistake, which makes ~100% for any tested config.

[alex_s]

A bit different results I have got on XP SP3. This test is interesting because here were the three firewalls tested under the very same conditions.

Clean system

PerfTCP: 28.171, 28.156, 28.162
PerfUDP: 20.406, 20.468, 20.431

Windows XP SP3 Native Windows Firewall

PerfTCP: 29.515, 29.125, 29.187
PerfUDP: 21.515, 21.437, 21.390

OA, latest public RC (3.5.0.6), default setup, after learning mode

PerfTCP: 28.562, 28.375, 28.515
PerfUDP: 21.703, 21.678, 21.656

Comodo, latest version, default setup, clean PC, no AV

PerfTCP: 28.984, 29.046, 29.078
PerfUDP: 26.046, 26.296, 26.093

[Pedro]

A couple of comments/questions Alex, if i may.

How does that tool work, how reliable.

And how would you account for stateful firewalls vs. stateless ones.
Extending that to firewalls with "pseudo stateful inspection" for UDP, and those without. That would affect differences in performance of course, one has to do extra work to track connections and filter accordingly.

[alex_s]

-{ Quote: "A couple of comments/questions Alex, if i may.

How does that tool work, how reliable.

And how would you account for stateful firewalls vs. stateless ones.
Extending that to firewalls with "pseudo stateful inspection" for UDP, and those without. That would affect differences in performance of course, one has to do extra work to track connections and filter accordingly." }-

Can you clarify what is "true stateful inspection" for UDP ? As far as I know UDP is connectionless protocol, so stateful inspections for UDP should be very very simple. As for the tool, you can take its sources and judge yourself how much it is reliable. I think it is the same reliable as any other network utility. But just in case I run every test tree times and results fall inside reasonable statistical range.

[Pedro]

-{ Quote: "Can you clarify what is "true stateful inspection" for UDP ?" }-
-{ Quote: ""pseudo stateful inspection" for UDP" }-
As in, not true. :)

[alex_s]

-{ Quote: "As in, not true. :)" }-

Can you explain the technical difference ? I understand some vendors say that this is they who make true inspection while others do just "pseudo". But you, personally, can explain the difference ? Because some vendors just exploit the fact that most users are not technically educated and say them the things that do not correspond with reality and then the users repeat this BS.

I'd like to turn our talk to exclusively technical way, as much away from marketing as possible. So if you state your question technically, I'll be happy to answer. If not, then we will go to a blind alley.

[Pedro]

Sure, just don't ask me for really technical details, as i'm not that good at it, nor have the time or inclination to delve into it.

As i understand it, pseudo stateful inspection is a term used by some referring to tracking connections with stateless protocols - as close as you can to stateful inspection, for a stateless protocol like UDP.

I believe it's about keeping a table in memory about outgoing UDP connections, to allow subsequent reply, and no more.
For instance, DNS request to port 53 on your DNS servers, firewall keeps that information, then the server replies and the fw allows it since it's the same IP, within allowed timeframe. Same IP, wrong timeframe, blocked; the right time, wrong IP, blocked etc.

This is opposed to allowing everything IN/OUT remote port 53 and remote IP's so and so.

[Sully]

Oh, a tech thread. I would love to participate. However, I am unfamiliar with perftcp/perfudp. Is it a performance counter or something? How can I also test and add to the data?

Sul.

[alex_s]

-{ Quote: "Sure, just don't ask me for really technical details, as i'm not that good at it, nor have the time or inclination to delve into it.

As i understand it, pseudo stateful inspection is a term used by some referring to tracking connections with stateless protocols - as close as you can to stateful inspection, for a stateless protocol like UDP.

I believe it's about keeping a table in memory about outgoing UDP connections, to allow subsequent reply, and no more.
For instance, DNS request to port 53 on your DNS servers, firewall keeps that information, then the server replies and the fw allows it since it's the same IP, within allowed timeframe. Same IP, wrong timeframe, blocked; the right time, wrong IP, blocked etc.

This is opposed to allowing everything IN/OUT remote port 53 and remote IP's so and so." }-

This is what I believe is normal statefuls UDP inspection and this is what every normal firewall does. It also should be added that request can have broadcast address so response can arrive from the different addresses. What else can be done ? Really a lot, anyone can duplicate the whole tcp/ip stack and do the same the stack does. Does it make much sense ? I believe not, because in ideal model the tasks should not be duplicated.