How do ISPs keep track of where we visit?

In the US, ISPs are required to keep logs of all activity, but how and where is this info gathered?

Let's say an unproxied connection with a static IP, no DHCP, and known IPs for the ISP's DNS servers, Windows XP Pro.
On boot up, Windows Automatic Update tells svchost to get the ip for MS update from the dns server on port 53.
Whenever you use a command line util like ICMP Ping or nslookup, those also connect to the DNS server.
And when your browser wants to go to a page, svchost gets the ip from the dns server.
So then aren't all the sites, individual pages (and access dates and times) visited by your ip address kept track of in the ISP's DNS server logs?

What about when the browser connects to port 80 of the visited site. How is that data logged (or would it even need to be if the DNS logs are available)?

What happens if you set your Preferred DNS server (in TCP/IP settings) to a different DNS Server than your ISP?
Is your ISP still able to monitor the connection through port 53 even though it is not going to their DNS server?

If you connect through a proxy, like JAP, TOR, Anonymizer Total Internet Shield, etc., then the DNS is done by the last server in the chain. But Windows Auto update, Ping, Port Explorer, etc. still need an IP in the Preferred DNS Server to work correctly for DNS.
Is this where a SOCKS proxy is used?

Any info in this area or a point in the right direction would be appreciated.

 

Well it's been over a week on this thread, so I thought I'd ask.

Are ISPs are able to keep a log of your DNS requests even if you are not using their DNS server (in the preferred dns server in TCP/IP properties) by using a packet sniffer on any outgoing port 53?

 

We built a Virtual ISP for a client a few years ago, so I can help you with the answers here I think.

Firstly, an ISP "owns" the connection for administrative purposes, so in simple terms "anything is possible". That is - in the same way that you have control of your computer, the ISP has control over their network. They can do pretty much whatever they want with it because you connect through it.

Take Squid, for example. Its a proxy server - but it has a transparent proxy mode. This means it will proxy all port 80 traffic without requiring proxy server setup in your browser.

We use that here in our office in this mode, and it is capable of tracking where each IP goes, what time - block undesirable sites, and a host of other interesting features.

This would probably be the easiest way of logging where you visit. When you authenticate to their servers, the radius server will hand you an IP address, and it will create a record that contains some data. For example, we noted the time, the IP, the username.

This particular ISP did not use any proxy servers (because they are a virtual ISP, they in fact were piggybacking on someone elses network) - but it would be possible with Squid and these logs to track exactly where you went (assuming, that you weren't using an anonymising proxy of some kind, in which case only this proxy would show up in the logs).

This would be enough - we would not need to necessarily track DNS requests, since once the address is resolved it is requested through squid. Although, I am sure that it could easily be done if it became necessary.

Just for example, here are our proxy logs for the last few minutes. It's possible to see that I have been looking at our forums, and Wilders, and everyone else is hard at work not surfing

Here are our proxy logs for the last few minutes:

Total number of websites matching selected criteria for September 29: 359
Older Newer
Time Source IP Website
10:43:12 192.168.51.101 https://www.wilderssecurity.com/newreply.php?
10:42:41 192.168.51.101 https://www.wilderssecurity.com/showthread.php?
10:42:40 192.168.51.101 https://www.wilderssecurity.com/showthread.php?
10:42:23 192.168.51.101 https://www.wilderssecurity.com/index.php?
10:42:19 192.168.51.101 http://www.tallemu.com/favicon.ico
10:42:19 192.168.51.101 http://www.tallemu.com/forum/index.php
10:42:16 192.168.51.101 http://www.tallemu.com/favicon.ico
10:42:16 192.168.51.101 http://www.tallemu.com/forum/viewforum.php?
10:42:13 192.168.51.101 http://www.tallemu.com/favicon.ico
10:42:13 192.168.51.101 http://www.tallemu.com/forum/posting.php
10:41:17 192.168.51.101 http://www.tallemu.com/favicon.ico
10:41:17 192.168.51.101 http://www.tallemu.com/forum/posting.php?
10:41:15 192.168.51.101 http://www.tallemu.com/forum/posting.php?
10:40:59 192.168.51.101 http://www.tallemu.com/favicon.ico

 

this is a subject that confuses me no end.

so are we saying that even if we are going through a proxy server our isp knows which websites we go to? do they also have a record of what data has been viewed and what messages have been exchanged? (e.g - this message)

and is this also true if using https? e.g megaproxy uses https

(my current understanding is the website + all data will be noted for http but no website or message data will be logged if using https - just a record that you are logged into a proxy server)

is there any such thing as a truly 100% anonymous connection?

 

i believe some proxy servers (paid for) offer url encryption which hides the url from your ISP? - is url encryption - https? perhaps a network expert could comment on this?

 

Hi Top,

Going through a Proxy only makes you invisible to the other end, the site you want to visit. Your ISP sees all your connections etc this way, even if you use so called Anonymous Proxys.

On the other hand, if you use an Anonymous SSL etc encrypted service, the TOR/Privoxy network or a VPN channel, then that's a different story altogether.

In these cases whatever site you visit, HTTP, HTTPS etc will be invisible to your ISP. All they know is that you connected to one of the above sites, not where you went through them, or what you Uploaded/Downloaded etc.


StevieO

 

thanks StevieO

 

Mike,

Thank you for the informative and helpful answer. It is appreciated!

 

Thank you StevieO!

 

Devinco,

Look here for more information.

-- Tom

 

Tom,

Many thanks!!!

 

Hi, Its sounds all too complicated to me. Yes we would all like privacy and not the 'big brother', watching over us, but why would we go to so much trouble to hide our activities, if we remain legal.??
As a footnote,about 2 weeks ago I saw in the news here in the UK that a European wide Paedophile ring was tracked down by interpol even though they were using [the news stated] encrypted methods to visit their sites. Which I undertook to mean encrypted proxies. Nothing is 100% anonymous.

 

"Hi it's Mike calling from InternetMarketingWeasels. We're affiliated with <yourISP>, yes, that's right... Well, I'm calling today because you've been reading a lot recently about premature ejaculation, impotence... and, we notice you've been clicking on those "get an extra inch emails...."

"What's that? Oh, you're Gordon's brother... ummm... Would Gordon be there, or I should I call back later?"

There are probably some things about you that you would not want anyone/everyone in the world to know, regardless of their legality

Each to their own though - personally, I take few, if any steps to "cover my tracks" online.


Mike

 

There are a few reasons why I don't like to be tracked around the net, even though I'm not doing anything wrong (like porn etc..) or illegal (p2p, crack sites etc...) in any way. It should be our right to have privacy, if we want it, when we go on the net.

I don't want some damn nosy government official, or anyone else, looking over my shoulder at everywhere I go on the net, and possibly keeping some kind of record of the sites I choose to visit. Why, you may ask if your not doing anything wrong? Because it's none of their goddamn business, that's why!

Remember absolute power absolutely corrupts. And who knows if that information could be used against you in some way, down the road. If you for example, visit some site that the government considers controversial in some way, you could end up on some kind of list, even if you went there by accident! This is just one reason I don't like being monitored.

I don't feel ANYONE should be able to monitor where we go without a warrant, including our ISP's. That's the way it should be IMO. We should have the right to our privacy. I'll use everyway available to me, and make it as difficult as I can for others to track me, for as long as I can, as just another way to say, "up yours!" & "screw you!" to any nosy lowlifes who feel they need to spy on me for no real reason.

 

My views exactly.

 

My views too, I wholeheartedly agree about privacy as stated, I personally dont bother with proxies.
PS I dont need any extra inches thanks.

 

Just saw this fly past on the full disclosure mailing list and seems relevant to the current discussion on this thread. One more reason to keep your credit cards safe while online.


http://news.independent.co.uk/uk/legal/article316391.ece


From the article:

Despite accepting the news in a "steady fashion", the commodore was
dead the next day. His brother Rupert told the inquest that the news
of his removal had caused his "mental collapse", and that he was in "a
state of catatonic shock".

The head of the Royal Navy, the First Sea Lord, Admiral Sir Alan West,
expressed his "deep regret" over Commodore White's death yesterday,
after the inquest recorded an open verdict.

The coroner, Charles Pitto, said there was insufficient evidence to
conclude whether the commodore's death was accidental or suicide. If
it was suicide, it would have taken to 34 the total number of people
who have killed themselves after being identified as suspects by
Operation Ore, Britain's biggest child-sex probe. The nationwide
police investigation was launched three years ago after a list of
7,200 British suspects was handed to British police by US authorities.
The men on the list are accused of using credit cards to pay for child
porn through Landslide, a sex website that operated in Texas from
1996-99.

The results have seemed impressive. Nearly 4,000 people have been
arrested, some 1,600 have been charged and 1,200 convicted. But the
operation has placed some apparently innocent individuals under
suspicion. In one case at Hull Crown Court last year, a distinguished
hospital consultant was acquitted after it emerged that hackers had
used his credit card on Landslide. The judge dismissed some police
evidence as "utter nonsense".

 

Yes agreed. The majority and innocents always suffer at the hands of an unlawful minority.
In an ideal world we would all enjoy 100% anonymity, where Mr Smith can visit Wilders or sex.com without worry. But with equal anonymity Bin Laden could be sat in an internet cafe in Kabul sending out instructions to his henchmen in New York to hijack a few more planes.
Where do we draw the line
By the way this thread has digressed from How to why.

 

Mike Nash's post above is largely spot on in terms of what information ISPs will likely retain - people should note that their ISP can see all data sent and received from their system so could (if they had the storage) take this further by monitoring cookies, any text you sent to a site and browser headers also (those who have Proxomitron installed can simply open its Log Window and load a webpage to see what can be viewed by their ISP).

With HTTPS, the data is encrypted but the ISP can still see the URL of the website you access. Some anonymity services do offer a proxy service via HTTPS which would conceal your activities from your ISP (they would see you accessing that service but the details would be encrypted) but the anonymity service itself could log your (unencrypted) traffic in a similar fashion.

Anonymity networks (Tor being the best example) route traffic via multiple systems so even if someone was logging traffic, they would not know whose it was. These offer the best security but having traffic routed via multiple systems with several layers of encryption can greatly slow web access.

With regard to Operation Ore, the UK magazine PC Pro did an investigation (PC Pro reveals false claims of child porn investigation) which indicated that misleading evidence led to people being accussed of abuse, even though they had not visited that specific illegal website. While indicative of over-zealousness (and a certain degree of incompetence) by the police, this should be a separate concern from the all-pervasive network monitoring that governments are trying to impose.

The following threads do cover specifics on Internet anonymity in more depth and are worth reviewing by those seeking more information:

Could use some advice re: anonymous surfing
Encrypting internet traffic
How do I prevent being "sniffed"?
tor and internet security

and finally that monster thread...

Don't Fear Internet Anonymity Tools.

 

Paranoid2000,

Thank you for the help.

 

Peace Fire

http://www.peacefire.org

 

this articles a little on the old side but may be of interest....

http://www.theregister.co.uk/2001/11/14/doityourself_internet_anonymity

 

MPC

MPC Anon Offshore Tunneler

an open question - would you consider using a paid for service if it could demonstrate it's superiority to free services?

# High Speed Anonymous HTTP Internet Proxy
# High Speed Anonymous SOCKS Internet Proxy
# Ultra High Anonymity, Slower Speed TOR HTTP Internet Proxy
# Ultra High Anonymity, Slower Speed TOR SOCKS Internet Proxy
# Ultra High Anonymity I2P Network HTTP Proxy
# Single hop High-Speed Proxy Anonymity
# TOR Network Onion Routed Internet Access

it offers a connection to TOR for superior anonimity which i find confusing cos the TOR website states "Tor is an important piece of building more safety, privacy, and anonymity online, but it is not a complete solution. And remember that this is development code - it's not a good idea to rely on the current Tor network if you really need strong anonymity"

 

toploader,

Thanks for the register link, it is useful and interesting.

The whole topic of censorship (including PeaceFire), the ways it is imposed, and the methods to get around such censorship, truly deserves at least a new thread (maybe a whole forum!).

As for the open question, I say yes. Privacy is neither cheap nor easy, but worth it.

 

Re: MPC

Thanks for the link - it certainly brought a smile to my face! From their website:

Chaining their service onto Tor won't provide "ultra high" anonymity at all since they have created a weak link in the chain - specifically they have the ability to monitor your traffic before it enters Tor. This makes it less secure than running Tor on its own, where relays have no way of identifying users.

A domain lookup reveals:

Registrant:
Axcess Financial Mrkt
7809 Southtown Center # 312
Bloomington, MN 55431
US

Domain Name: MYPRIVACYCLUB.COM

Administrative Contact, Technical Contact, Zone Contact:
Axcess Financial Mrkt
Marketing Director
7809 Southtown Center # 312
Bloomington, MN 55431
US
612-677-3190
612-677-3190 [fax]
http://www.emailaddressprotection.com

Domain created on 31-Jul-2002
Domain expires on 31-Jul-2006
Last updated on 02-Aug-2005

Doing a [url=http://network-tools.com/default.asp?prog=trace&Netnic=whois.arin.net&host=myprivacyclub.com]traceroute[/url] to their website gives:

1 0 0 0 66.98.244.1 gphou-66-98-244-1.ev1.net
2 0 0 0 66.98.240.119 gphou-66-98-240-119.ev1.net
3 0 0 0 67.15.105.190 ev1s-67-15-105-190.ev1servers.net

So both the company and their website are US-based. It may be that they use anonymising relays located elsewhere, but that doesn't really help if the company itself is the subject of a subpoena from a US court.[quote]Who developed and operates the Anon Offshore Servers?

They are die-hard believers in civil liberties, the Bill of Rights, and in entrepreneurship.

[b]They are a group of experienced, dedicated privacy fanatics.[/b][/quote]Given the lack of names supplied, the "privacy fanatic" part is likely true - but then this makes the subsequent sentence "Members of their team have been responsible for some of the biggest innovations in online privacy." unprovable and quite likely just empty marketing talk.[quote]If you're really serious about Internet security, only use Open Source solutions.

The Anon Offshore Tunneler is the only online privacy service that will actually let you download the code to our products. The real, actual, honest-to-goodness source code that creates every one of our programs....Once you're an Anon Offshore Tunneler customer, just ask us for the code and we'll email it right to you.[/quote]Um, do these people know what "open source" means? The source code should be available to [i]everyone[/i], not just those who've already signed up.

Finally, best of all:[quote]Unlike the "newcomers", those behind the Anon Offshore Tunnel have been discretely providing privacy services since the earliest days of the Internet.[/quote]Go to the bottom of the page and there we have [b]Copyright 2005 MY PRIVACY CLUB All Rights Reserved[/b] (not to mention their domain name having been created in 2002 according to the registration above).

On the good side, they offer the option of paying by e-gold which should provide greater anonymity than a credit card, but with all the inconsistencies shown above, I'd suggest steering clear of this.