Discussion in 'privacy technology' started by mirimir, Jan 9, 2012.
It's working now
Double check your DNS servers on both pfSense VMs. Use the -grc.com/dns check from workstation VM connected to each one. Most importantly, make sure that workstation2 VM is using DNS servers that you want (either those provided by VPN2, or third party).
just a quick update on this, in your pfvpn box you will need to add a floating rule to prevent the machines that are supposed to only go thru the vpn from being switched over to the pfvpn wan if the vpn drops
make an alias with your "protected" internal machines ips
add a floating rule in the pfvpn box:
reject or deny (i used reject so the machines are't left firing away syn's until it gives up)
Apply immediately on match
source = your alias
log -> yes
your machines will NOT bind to the vpn interface if the connection drops, pfsense will try to route it to the unsecured WAN which is your regular lan -> isp router, it re-routes surprisingly fast too, under a minute
for added measure i also added rules on that router to deny any lan traffic from the secured boxes that are supposed to be behind the pfvpn box. these rules will never actually get hit in theory but the added measure doesnt hurt. that has the added benefit of blocking if by some chance your pfvpn box goes down entirely so i recommend it.
you can bind to a client mac but not to the router interface apparently... which i verified thru traffic logs
this is a "feature"
easy fix none the less. just make sure you do it otherwise you may spill the beans as they say.
In the configuration that I've described, there is no outbound NAT rule for LAN to WAN. There's just a manual outbound NAT rule for LAN to OpenVPN. I don't see how pfSense could route LAN to WAN if the VPN drops.
I've experienced service interruptions with two providers. In each case, Internet connectivity was entirely lost.
Under what circumstances have you seen LAN outbound routing switch from VPN to WAN?
i see what you're saying, by default NAT from lan to wan is enabled on a new pfs install. ive disabled that and will go back and test again.
Yes. In fresh pfSense installs, "Automatic outbound NAT rule generation" is enabled. If you leave that enabled, you must create and properly configure an explicit interface for each VPN. If the VPN goes down, LAN clients will default back to WAN. But, in most enterprise environments, that's a good thing.
Alternatively, you can just disable "Automatic outbound NAT rule generation", and edit the outbound NAT rule from WAN to OpenVPN. If the VPN goes down, there's no outbound connectivity, unless you manually edit the rule. For us, that's a good thing. For someone managing typical enterprise VPNs, it's a pain.
actually i was still able to get it to route to "a" WAN while nat on pfvpn was disabled.
for a min i thought the secured machines were being assigned a new gateway from the lan pfs router but they're not. even though they're on the same subnet as the regular lan, the route should in theory fail @ the pfvpn box w nat non existent to the WAN as its the gateway for the secured boxes
it appears the requests are sent to the gateway (pfvpn) fail there and then seek an alternate path via LAN and hit the regular lan router where they are then passed out.
thats the trade off it seems if you choose to keep the secured machines on the same subnet.
VPN services are clever. They say no we don't save the IP or we only save this part of it or so and so but what they don't tell you is that they get your MAC addy and save it forever. Thats how they detect customers using free services creating more accounts. I use a VM and change my MAC every couple of days with this
Its got 100 of MAC addresses you can use I just use the set random MAC.
I'll take your word for that
That's why I don't put both LANs on the same subnet! (emphasis added)
I'm still surprised by that, but routers like to route, I guess.
It's a bad trade off, in my opinion. I never put things on the same subnet that I want to keep separate. I have three physical subnets here (work, house and play).
I disabled vpn in pfsense and tested a few urls and direct ip-addresses on Ubuntu VM. I sniffed packets on my host adapter with wireshark. My connection was dead, no connection to sites. However, DNS was routed through my real adapter. This is a problem.
How about you mirimir?
Yes, that's a problem. If I disable or misconfigure an OpenVPN client in pfSense, I also see some DNS leakage. Queries to DNS servers that I've specified in pfSense's DHCP server on LAN hit the host's WAN adaptor. And a few (two of 32 during my test) hit the perimeter router's WAN adaptor. Actually, I don't understand why it's just two (one per DNS server IP address). Why don't all 32 make it, if any do?
In any case, it's easy to close the leak. Just add firewall rules in pfSense to block traffic from LAN to those DNS servers. I'll add detailed instructions for doing that shortly.
Thank you for catching that, Asus125
THIS IS AN IMPORTANT REVISION TO BLOCK DNS LEAKAGE WHEN THE VPN IS DOWN
In your management VM (LiveCD ISOs are nice for this) browse to the pfSense VM's webConfigurator. Select "Firewall | Rules", and then the "LAN" tab. Click the "+" button at the lower right to "add new rule". Select these parameters: "block" for "Action"; "LAN" for "Interface"; "any" for "Protocol"; and "any" for "Source". Under "Destination" select "Single host or alias" for "Type", and specify the "Address" of the DNS server that you don't want to hit when the VPN is down or disabled. Then click "Save".
If you need to block another DNS server, click the "+" button at the right of the rule that you just created to "add a new rule based on this one". Specify the other address. Repeat as needed.
Now select the rules that you've just added, using the check boxes on the left. Then click the arrowhead ("move selected rules before this rule") to the right of the "Default allow LAN to any rule" rule.
When you're done, click the "Apply changes" button at the top.
I need to make two corrections for this.
1. I have to say that addi6584 was right. I deeply sniffed my packets again with wireshark when I had vpn disabled. TCP-connections was made and they used my real adapter - however, Ubuntu VM couldn't access those urls (IPs). Maybe first test failed because pfsense didn't route those packet so fast; I did that only a short period time because, I guessed those packets wouldn't route anywhere. They do!
2. I managed to block dns queries with that simple firewall rule. There is a minor correction for that: when you have done the rule, default lan rule, which allow all lan traffic to anywhere, must be placed to bottom.
I'll check that out. I'm using private (10.x.y.z) DNS servers, so they don't work outside the VPN connection.
Thanks. Reading the pfSense firewall page, I see that advice. I've done that, and it seems to work. Maybe I didn't test long enough before. Only 2/32 DNS queries got out even without the block rules, after all.
yes on default lan rule @ bottom. pfsense guys had the bright idea to invert how pf rules are processed vs regular pf on openbsd. why!? probably so linux kiddies wouldn't get confused coming from iptables/netfilter but its royal pita for us obsd fellas who use the "real" pf
2nd, dont just stop w only blocking the dns servers @ the host for a cpl reasons
a: someday you may want to/need to change dns servers esp if you are doing load balancing or have to change vpn providers or servers if one of their links goes down. also im noticing TONS of dns hijacks w some of the defualt dns servers listed by these vpn guys. another reason why you may switch dns servers (or run your own somewhere)
b: off chance your box starts rerouting everything thru the wan for some reason. pfs guys have documented that svs such at vpns are supposed to auto reroute for uptime per the link i posted above... this "feature" also doesnt exist in obsd pf
2a: block EVERYTHING @ your lan router/firewall coming off those secured boxed not just the dns servers as insurance like I mentioned in an earlier post.
i saw all this crap rerouting just like you but i did it through tcpdump vs wireshark @ the lan router which is why i posted the above info
this is also another reason why i do not like the idea of using vmw or vbox to nat/bind/whatever it does to the host machines nic. ESXi does NOT provide this ability by default and instead makes you config networking on every machine as if it were a real machine w its on real nics etc. it puts you in control instead of some behind the scenes voodoo you cant see (the bugs if any)
thats why im rebuilding that tor gateway img for you guys, to get rid of the virtulization networking requirement.
its good you double checked everything w wireshark, you have to every time you try to secure something like this before doing anything thru the connection, is verify it actually works the way its supposed to. ive got the traffic logs parsed thru a deamon that will hit my blackberry if any such leaks occur
Thank you, addi6584 and Asus125!
I acknowledge you for finding the pfSense leaks. In retrospect, it was foolish of me to think that pfSense wouldn't leak when VPNs go down, unless firewalled properly. I should know by now that mainstream VPN management favors connectivity
I need to check for leaks in nested VPN configurations with possible failure scenarios, and fix whatever needs fixing. I should have an update by Monday.
While I posted the instructions primarily as a contribution, I was also looking for peer review. And you've provided it. I apologize if my errors have put anyone at risk. Once we finish this conversation, I intend to rewrite the instructions as a single document. If it's OK with Wilders, I'll leave it here.
Is this setup below pretty much the objective of the post mirimir, what you're most comfortable with for a simple setup?
Sure. But, as with VPN connections generally, you need firewall rules to block leaks when the VPN goes down. If you have suggestions about that, please share. I'm doing exhaustive leak testing for various configurations.
I could of sworn the way pfSense is running it protected against a VPN drop unless that was just the redirect-gateway def1 as you mentioned before...
But the thing is, even at this point in time, if one of the layers dropped, we still have a few others, so I'm not seeing much of a problem here unless someone is overly paranoid and I admit I can be that way, LOL...
So what leaks are we talking about, just a dropped connection?
What I see is this (thanks to Asus125 and addi6584). For the test, I have a pfSense VM set up for a VPN service, with redirect-gateway def1, and DNS servers at private IPs specified in the LAN DHCP server. I capture packets on the VirtualBox host's WAN using Wireshark, and on my ISP WAN connection (pfSense router). I test using -grc.com/dns.
With the VPN connected, all I see on VirtualBox host WAN and ISP WAN is UDP traffic between the host and the VPN entry node. If I misconfigure the VPN, by changing the server to a name that doesn't exist, about 30 of the 1.5K-2K DNS queries to the VPN's DNS servers (which are at private IP addresses) that the GRC test generates make it to the VirtualBox host's WAN. About two of them (maybe it's one per IP address) make it to the ISP WAN connection. I don't know whether they get out to the ISP, and I'm not going to ask them
According to GRC's explanation, this test is very rigorous, and may crash routers. There are three routers in the setup: the pfSense VM, the VirtualBox internal network router, and my perimeter pfSense router. The pfSense VM is getting hammered the hardest, I think.
I take it you're talking about the GRC DNS test?
Still, what I was saying, it doesn't matter what is dropped where in the chain, as long as something is still up at least in the Host you're protected, how do you think you can't be? It's impossible.
I know for the technical side of things it's proper and more geek to explain in the technical sense, but I'll be honest, even as a geek I just like to talk English and put the geek on the table, because sometimes the geek takes over and gets in the way and you are blinded, if you get what I mean, LOL...
So let me run it down like this and I'd like someone to explain how they think there's going to be a problem because I don't see it, because I know it's not there.
1. Host system is running OpenVPN and in the network adapters is set up using a primary and secondary DNS, like from Comodo or OpenDNS.
2. pfSense is running OpenVPN (Now the host is going through both VPN connections, still on the same DNS from the Host adapter settings)
3. Start Guest VM on 'internal' network for pfSense, now the Guest is picking up OpenVPN from pfSense and the Host and if you wanted to, not sure it's needed, you can also add in the primary and secondary DNS on the Guest adapters for Comodo or OpenDNS. Also on the Guest VM, you can also run OpenVPN which is what I do to add another VPN layer.
4. Now looking at this chain, where does it all start? As long as the Host isn't going down I don't see how loosing the VPN on pfsense or the guest is the problem, because you are still sitting behind the Host running the VPN and the DNS through it's adapters... But even if the Host goes down, then the chain should be broken and you should not be getting back online at this point either, since the starting point of the tunnel has been broke...
This is just the classic DNS leak that's been discussed for years on Wilders. If the VPN goes down, queries to currently specified DNS servers will get out -- unless (1) the VPN config has changed the client's routing to prevent that, or (2) there are firewall rules that prevent that (or both). So, no matter where the VPN is running, on pfSense or a workstation, you need the firewall rules as insurance. Right?
If you have just one VPN, and it goes down, DNS queries may get out to the Internet. My "outer" VPNs all use DNS servers with non-routable IPs (such as 10.x.x.x) and so I'm not sure whether queries to them get past my perimeter router. But, as I've noted, I'm not going to ask my ISP. What do you think? To be safe, I'm assuming that they do. So I've added firewall rules to prevent that. I also need to add firewall rules to block other non VPN traffic, but that's not a pressing issue, because there's no DNS resolution.
If you have two nested VPNs, and the "inner" one goes down, DNS queries may get out to the outer VPN. I'm less concerned about that, because I use public DNS servers, and the queries will get out through the outer VPN. I do plan to add firewall rules to block that, however.
I'm not sure what happens if you have two nested VPNs, and the outer one goes down. The inner one obviously goes down too. But I don't know what happens to DNS queries from the inner one. So I need to test (after I finish some other work that's more interesting).
The problem is when you use the VPN's DNS, I don't, as I mentioned I use Comodo or OpenDNS in the adapters and then when a VPN goes down you're not going to expose your ISP DNS, you're still behind these DNS.
So yes, if your adapters are using the DNS from the ISP, yes when the VPN goes down you can expose these, but put in a primary and secondary dns, don't use the ISP DNS and you're good to go...
It's true that you're not exposing -- or querying -- your ISP's DNS server(s). But you may be (let's assume "are") sending queries to some DNS server. Let's say that it's OpenDNS. So, when the outer VPN goes down, unless you have firewall rules that block, you're sending queries to OpenDNS for something. If you're testing, it might be grc.com or whatever. That's not a problem. But it might be the name of your inner VPN server, and you don't want that to be associated with your real IP address. Right?
What you just said makes no sense...
1. We are hard coding the DNS of choice into our adapters for a primary and secondary DNS, examples I gave, Comodo, or OpenDNS.
2. If you are hard coding the DNS in, then it's not going to matter if the VPN is up or down the only requests are going to these DNS, nothing else.
3. So what is this 'sending queries to some DNS server'? We are only sending queries to the ones we've hardcoded in and nothing else, that's the point.
4. As long as you've hardcoded the DNS in, at what I'd call the beginning of the chain, meaning the host, then all the Guest are going to be using the same DNS, but as an extra addesd safety precaution, I'll still hard code those on the Guests as well, it's not going to hurt anything, plus you could also run those through other DNS, I guess creating a layer for that as well, hehe...
5. So I also don't get what this 'inner vpn' is suppose to be, but there's not going to be any associations to your IP, unless you were able to send requests over your IP with the VPN down because it's not properly configured. But as you stated before; "redirect-gateway def1" will not allow you to get back online, so how do you think you're going to expose an IP...
I thought we were talking about not leaking DNS, not exposing IP addresses?
If your VPN is down and you get back online through the hardcoded DNS, well of course it's your own IP and I know you know this, but it's through the hard coded DNS, so I'm really missing the point here, again I thought we were discussing only DNS leaks...
Let's consider a basic VPN setup: one machine, running OpenVPN, connecting to your VPN provider's access server "gw.foo-bar.baz.net". Let's say that you're using OpenDNS for your network DNS server, and baz.net's default DNS server(s) for the VPN. Normally, only DNS queries to OpenDNS for "gw.foo-bar.baz.net" go out through your ISP. But let's say that "gw.foo-bar.baz.net" goes down, and DNS queries are leaking. Now you also have DNS queries to baz.net's DNS server(s) going out through your ISP. Even if they're not routable (like "10.x.y.z") your ISP can see them, and can see in the packet data what site(s) you're trying to reach through the VPN.