Block browser fingerprinting at system level (hosts file)

Hi everyone,

In my last post back in 2012, I was talking about BlueCava which was a company that conducted browser fingerprinting for their customers.

Since then, I've had their domains blocked in my hosts file. Recently, a whitepaper was released researching various other companies in-depth that are also engaging in browser fingerprinting for some of the top sites on the web.

I compiled a list from this PDF of the most prominent offenders which may be useful for some of you to block their javascript serving domains at system level.

Here it is:

Code:

##
# block browser fingerprinting
##

0.0.0.0 bluecava.com
0.0.0.0 ds.bluecava.com
0.0.0.0 clients.bluecava.com
0.0.0.0 lookup.bluecava.com
0.0.0.0 device.maxmind.com
0.0.0.0 inside-graph.com
0.0.0.0 cdn.inside-graph.com
0.0.0.0 live.inside-graph.com
0.0.0.0 mshare.net
0.0.0.0 tags.master-perf-tools.com
0.0.0.0 h.online-metrix.net
0.0.0.0 gmyze.com
0.0.0.0 cdn-net.com
0.0.0.0 analyticsengine.s3.amazonaws.com
0.0.0.0 img.alipay.com
0.0.0.0 mp.pianomedia.eu
0.0.0.0 go.eu.bbelements.com
0.0.0.0 b.siftscience.com

 

On one hand it's good to know and spread the word on the organizations that perform browser fingerprinting upon surfers/customers.

But there are many other organizations that perform and record browser fingerprinting activities unknowingly or in the name of law/security. Best example is our ISP servers, which users are obligated by their whatever terms and conditions.

 

Really? Here's some homework

Code:

2o7.net
adcatch.net
addthis.com
adfootprints.com
adform.net
adition.com
adnxs.com
advertising.com
adzoe.de
atdmt.com
beemway.com
bkrtx.com
bluecava.com
bluekai.com
chartbeat.com
crwdcntrl.net
de.com
doubleclick.com
doubleclick.net
e24.com
effectivemeasure.net
emediate.eu
eu-survey.com
facebook.com
facebook.net
fagms.net
feedsportal.com
followistic.com
google-analytics.com
googleadservices.com
googlesyndication.com
googletagservices.com
gotraffic.net
ilius.net
imrworldwide.com
interclick.com
iomigo.com
ivwbox.de
kaufda.de
krxd.net
lijit.com
linkpulse.com
llnwd.net
moatads.com
npario-inc.net
npario.com
nuggad.net
omtrdc.net
plista.com
quality-channel.de
quantcast.com
quantserve.com
revsci.net
scorecardresearch.com
serving-sys.com
smartadserver.com
visualwebsiteoptimizer.com
webtrekk.net
yieldmanager.com
zanox.com
sitestat.com
chango.com
adexprt.com

 

how are those companies who are browser fingerprinting? those are mostly ad companies (corporate domain -- not even their adserving domains), along with some analytics providers. the ones i posted are solely just domains that conduct browser fingerprinting (the domains the javascript is served from)


if you look over the pdf, you see that bluecava is the top offender according to their research. you included bluecava.com in your list, however blocking bluecava.com in your hosts file isn't going to do anything for you. you need to block the subdomains that are serving the javascript, unless you are routing your dns requests through a dns proxy on your machine and blocking wildcard subdomains.

 

Do add-ons like Blender ("Blend in the crowd by faking to be the most common Firefox browser version/OS/etcetera.") block browser fingerprinting?

 

And here begins the ancient game of cat and mouse...

 

From looking over the extension, it seems as if it just changes your user agent. Browser fingerprinting is done with javascript. You still have to worry about them being able to read your system fonts, along with utilizing flash to get operation system specific details (that's if you don't already use something like flashblock, or click to play)

 

So basically exactly what TPLs/ABP do currently with an anti-tracking list.

 

I've never understood why there couldn't just be a plugin that blocks the data that is sent for use with browser finger printing. Or where you could pick and choose what data is sent. Or send totally fake information.

Blender doesn't really seem to do much. In fact, on the EFF's Panopticlick site it makes my browser fingerprint more unique. I think having a Windows user agent, with some plugins and font names that only exist in Linux probably makes the sum total of information more unique, since it doesn't even make any sense.

 

would noscript stop them from tracking you?

 

I think if you block all javascript entirely, it might. Disabling javascript for EFF's Panopticlick site seems to cause it not to work.

But if you want to do that then you don't really need NoScript, you can just disable javascript in your browser. Of course, this will totally break most websites. Hence the whole point of NoScript's existence, to selectively allow some javascript on some sites. And which point, you can no longer be sure that you're not being fingerprinted.

So I don't think NoScript can just selectively block browser fingerprinting. If the fingerprinting is being run by a domain that you want to have NoScript allow, then you will also end up allowing running the fingerprinting.

I guess you probably do block some of it, since any third party domain that NoScript blocks, would not be able to execute javascript that fingerprints you.

 

Technically speaking, Javascript isn't required to perform fingerprinting. It is simply used to acquire more datapoints for the analysis. Practically speaking, it would depend on how they implemented pages and fingerprinting. If they *chose* to make things reliant on Javascript being enabled, then it would be.

FWIW, Panopticlick seems to work OK, for me, with and without Javascript enabled. My browser appears far more unique when it is enabled because of those additional datapoints that are collected. I would initiate each test via the Test Me button at https://panopticlick.eff.org/. I think the results are illustrative rather than representative of what is possible through aggressive fingerprinting techniques.

 

interesting point.

 

I've had some interesting results testing at Panopticlick. I've been experimenting with my non-Tor browser, Palemoon and a browser package I use exclusively with Tor. The Tor "browser package" I'm using consists of SeaMonkey>Proxomitron>SocksCap>Tor with specific firewall rules to prevent leaks and bypassing. When I test Palemoon, the results are consistent and it appears quite unique, over 1 in 3.5 million. By comparison, the results with the SeaMonkey "package" appear much less unique, and strangely enough, somewhat variable. The results vary between 1 in 300K to 1 in 500K, or between 7 and 12 times less unique. By far, the biggest identifying factor seems to be my screen resolution, a detail I'll be addressing in my Proxomitron filters. What really interests me is the variable results. I haven't looked into it to any degree yet and haven't ruled out whether the exit node being used is affecting the results. If my Proxomitron filters are the cause of this variability and if so, can they be constructed to interfere with different fingerprinting tactics.

 

Yeah, I also get variable results, both with and without javascript enabled. I guess at least some of this can be accounted for by the fact that other people out there are running the test too, so the background you're compared against is constantly changing. But given that I can see variation in tests that I run within seconds of each other, I really wonder if that fully or even mostly explains it.

 

FWIW, if I run several tests from the same IP Address in a very quick back to back fashion while cookies for panopticlick.eff.org are disabled, I see the N value in:

Within our dataset of several million visitors, only one in N browsers have the same fingerprint as yours.

decrease by 6-8 each time. Which may be due to the way they try to estimate how many browsers they've actually seen. If I enable cookies for panopticlick.eff.org the N value is reasonably stable.

 

I'm starting to question the value of the results, and some of the accuracy. The results are giving me 2 different color depths. With Palemoon which doesn't run through Proxomitron, it inaccurately reads my display as 24bit. With SeaMonkey and the Proxomitron filters, it displays 32bit, which is correct. For my resolution, it shows 32bit to be more unique than 24 bit, 1 in 118.21 vs 81.06 respectively. Out of curiosity, I switched the display to 16bit and retested both. Both browser tests indicated 16bit and were very unique, over 4 million. I switched back to 32 bit and retested. The results were back to 24 and 32 bit again. It appears that Palemoon misreports 32 bit color depth.

Another item that makes no sense there is the HTTP_ACCEPT headers. Both browsers report the same thing. The only difference is a space in the line.
From SeaMonkey, uniqueness 46.88
text/html, */* ISO-8859-1,utf-8;q=0.7,*;q=0.7 gzip, deflate en-us,en;q=0.5

From Palemoon, uniqueness 11.5
text/html, */* ISO-8859-1,utf-8;q=0.7,*;q=0.7 gzip,deflate en-us,en;q=0.5
On second thought,I wonder if little changes like these are deliberate and hard coded into browsers just to make them easy to identify. Something this minor could slip right past those looking at the code but would be all that's required for an attacker to know what to use. Does anyone else think a separate thread that compares data reported by different browsers/versions would be useful?

edit.
I've had cookies from domain enabled for both browsers throughout the testing. Tested with and without clearing them. The tests also use ETags.

 

I haven't been to the Panoptclick website in ages. For got all about it...

Any way I have two versions of Opera running...kind of strange the results.

First time I clicked on the test button, Opera shutdown.

 

The user agent, HTTP_ACCEPT headers, and browser plugin details are the most identifying factors, at least on their site. Looking at your results, Opera makes you appear very unique. Both of my browsers are set to not report a user agent. It appears that this is a fairly common practice, at least with their site. Apparently the number of identifying bits available is one of the factors that carries the most weight. If not, the results in the images below make no sense. The results are too similar overall and the details show SeaMonkey to be more unique but the overall results say the opposite.

 

Each individual value for SeaMonkey was as unique or more unique than the corresponding value for Pale Moon. However, it is the combination of values that would form your fingerprint.

Pants value = Pleated wool dress pants (common)
Shoes value = Sneakers (common)
Fingerprint = Not common because people rarely wear that combination of pants and shoes.

Perhaps there is just something very unusual about that particular combination of values seen with your Pale Moon?

PS and for reference: I recently saw "Your browser fingerprint appears to be unique among the 3,514,965 tested so far".

 

Except for the missing space in the HTTP_ACCEPT headers and the incorrect color depth, the results on the last test were identical.

Both should be extremely unique. I'm using Win 98. Those particular browsers only work on 98 if it's equipped with KernelEx. There can't be that many out there running this combination.

 

So I would assume one or both are involved in making your Palemoon fingerprint very unique. Perhaps Panopticlick has only seen that Palemoon HTTP_ACCEPT string and Screen details string together a few times. Perhaps it has only seen those strings along with a missing User-Agent string together a few times.

I don't see a way to search its database and try to zero in on what might be causing such fingerprint uniqueness. Which leaves you guessing and/or experimenting. If you are setup to proxy the HTTPS requests you can override any outbound data you want. You could also use a request generating tool to duplicate what your browser would send but with a modification like that space being taken out. If you considered it worth pursuing that is.

In those two tests you posted results for, I don't think Panopticlick knew what OS you were running or which specific browsers/builds you were running. You had them configured to not send User-Agent, and I don't think the Javascript-based sniffing code that POSTs additional information to the server would have communicated such information due to its design and your setup.

 

You can see in the images of Panopticlick that Tarnak posts above that his OS is reported as part of his user agent, "Windows NT 5.1." He's running Opera. For me, with Firefox, the user agent also includes my OS.

That aside, for me it's the browser plugins and system fonts that clearly add the most bits of information. That stuff is blocked with javascript off, but with it on that seems to be what makes one most identifiable. Obviously the more plugins one uses, the more one's particular combination will make you uniquely identifiable. This also goes for fonts. In addition, using Linux makes you more unique (one area in which I guess it's less secure than Windows). And the uniqueness of Linux is compounded by the fact that its fonts have a lot of different (i.e. less typical) names from what one usually sees in Windows.

I am now strangely noticing that suddenly my browser is a lot less unique. It's gone from 1 in 3 million to about 1 in 850 thousand (since yesterday). Strangely, if I blank out my useragent, it decreases the bits of information contributed by the user agent from about 11 to about 5, but overall my fingerprint becomes more unique (goes about up to about 1 in 3.5 million). I'm wondering if the mere fact that I've tested my browser several times since this thread began has made it appear less unique to panopticlick (how many people use the site anyway?). It seems like there are some pretty rough edges on panopticlick's ability to meaningfully convey the uniqueness of one's browser fingerprint. It's not based, after all, on a random sample of browsers in the world. It's based on a self selecting group of users, some of whom, like me, might repeatedly use it, but appear to be multiple different users.

 

According to their site, the dataset is made from several million visitors. It doesn't specify if that's the number of tests performed, the number of unique IPs seen, or if it's based on something else entirely. Several million also represents less than one percent of the number of browsers, PCs, internet devices in use. The results were obtained from a place where most of the visitors are privacy and security conscious, and may have already implemented anonymity or privacy measures to some degree. I seriously doubt that their results represent what is seen at less privacy oriented sites. As mentioned by TheWindBringeth, they weren't checking browser and OS version info obtainable by javascript. That means they also weren't looking for uniqueness due to mismatches in the javascript data and the user agent. Failing to make them agree could make you appear very unique, or at the very least make it obvious that you're spoofing the user agent.

 

FWIW, I rewrote some earlier reply to noone_particular to clarify that I was partly talking to his configuration/results. Sorry for any confusion.

I think the bottom line (for visitors) is that you want to block all requests to fingerprinting servers where you can identify them (thanks OP). Assume you will miss some and possibly many. Any server, including the one you are purposely visiting, may make use of fingerprinting techniques. When using HTTP, intermediaries will see basic information. If an intermediary knows the site you are visiting collects more info via HTTP requests they could try to collect that additional info as well. Use HTTPS where you can in order to reduce the threat from intermediary fingerprinting. You want the general/default scenario to be a well locked down browser that exposes as little information as possible and doesn't appear very unique. Selectively "lower your guard" for those specific sites/contexts where you think it necessary and appropriate. Assume that any unique identifier will be abused [for fingerprinting purposes]. Which includes your public IP Address, so make sure it is changing at least periodically. If your ISP provided equipment doesn't allow for that, get different equipment. If your ISP policy doesn't allow for that, get a different ISP. Edit: Lookup your IP Addresses in public databases to see what geographic location they correspond with. That location information can be used for fingerprinting purposes as well. Don't use WiFi based geolocation services while on your own network as that can make location information for your public IP Address more precise and the nearby AP information is yet more data that might be used for fingerprinting.