HTTP Switchboard for Chrome/Chromium:

No... It's not "merely choosing": The extension wouldn't be able to properly enable javascript if user blocked it using Chrome settings. I wouldn't blame users to quit using such a useless extension that would show "javascript whitelisted" while it's still blocked.

I think this is what is happening. Well, I did try.

 

í still find this confusing:

in the Settings page:
Process behind-the-scene HTTP requests.

i assume when the box is checked that HTTPSW is blocking this stuff, as per:

it is not obvious from reading the Help, at least not to me, what checking the box does.

maybe you should just add: "Nerds that want more privacy and security stuff please check this box".
----
after reading the changelog, it shed some lights on the whole topic:

0.7.5.0

* New feature: Ability to use the matrix on behind-the-scene requests (issue #122).
* Since the matrix reflects the network traffic of a web page, in order to access the matrix for behind-the-scene requests I had to pick a web page to allow for this.
* The web page picked is the Statistics page. So if you go to the Statistics page, HTTPSB's popup menu will show the matrix populated according to behind-the-scene requests.
(very nice that feature, one has to keep track of the changelog carefully to spot these new features though)
*A default scope called http://chromium-behind-the-scene is created by default for behind-the-scene rules.
* Important read to prevent any misunderstanding: Behind-the-scene requests
* Remember, whitelist/blacklist rules in the matrix for behind-the-scene requests will apply only if the option "Process behind-the-scene HTTP requests" is enabled on the Settings page.

https://github.com/gorhill/httpswitchboard/wiki/Change-log

 

That's really a great new feature. Raymond, you're amazing

EDIT: I've been playing with that new feature for a while. It allows for a detailed control of what the browser and the extensions are doing. If you enable that feature it's obligatory that you first open the HTTPSB statistics page and then click the HTTPSB button to open the corresponding matrix. You will see that many of your extensions need XHR requests allowed in order work properly. Examples: Adblock (in order to update its subscriptions), Autopager, Lastpass etc. Once you save the needed rules everything is okay. But you have to do it - otherweise you'll run into problems. You should leave the statistics page open for a while or, at least, open it repeatedly at the beginning in order to catch and save all necessary behind-the-scene rules.

 

@gorhill: Why does the behind-the-scene statistics show XHRs and (sometimes) cookies and other for normal websites (e.g. https://www.wilderssecurity.com/, https://www.github.com/) which do not seem to be related to extensions?

 

From Changelog: "Important read to prevent any misunderstanding: Behind-the-scene requests":

"Behind-the-scene requests may or may not be actually related to one of the URL addresses appearing in one of the opened tabs."​

I explain the reason of why is this in the above wiki.

 

Thanks - understood

 

Amazing feature, could you mind if you check what I posted here is correct? (Based on my understanding in how Behind-the-scene works. Thanks. https://www.wilderssecurity.com/showpost.php?p=2326051&postcount=723

 

Looks to me like you have the right conclusion.

If AdGuard was requesting through HTTPS and using POST that would take care of the privacy issue (though I personally wouldn't accept that my whole browsing history goes to one third-party.)

 

Thanks for the confirmation.
BTW is it possible if you could add option to remove the right click context menu? Maybe you could add it into you low priority list. Thanks.

 

Actually, thinking about it, I could get rid of this setting if I create a behind-the-scene scope with "all" whitelisted. So the default would be to allow all behind-the-scene requests. Then if the user wants to be more restrictive, he would just use the matrix, rather than a checkbox then the matrix.

I really wish I could get rid of this button now... Let's see:

If I removed it for someone for which it is unchecked, then I add a whitelist-all rule in the matrix, except if there are rules in the matrix which suggests the user has been tampering with the matrix for behind-the-scene requests.

If I removed it for someone for which it is checked, then I add a blacklist-all rule in the matrix, except if there are rules in the matrix which suggests the user has been tampering with the matrix for behind-the-scene requests.

Removing this checkbox would simplify code in a couple of places, and unify more the logic. I dislike special cases in code.

 

Here: https://github.com/gorhill/httpswitchboard/issues/130

 

I agree. IMO, you can remove that button.

 

I'd like to see the rule manager be a full regex parser. So that you could have something like s[0-9].youtube.com so that a rule would a play to anything with s, the set 0-9, and youtube.com. This would replace adding a rule to *.youtube.com, restricting it to a subdomain that matches the expression.


P.S. nbc.com, no video ads, playback works fine for me.

 

Then HTTPSB would lose one of its main advantage which is speed and relative simplicity and this would also complicate things with probably cases we wouldn't know what to do.

Currently rules are used as a key in a map. Very fast. The core evaluation code may need to look up many rules before reaching the proper result (subdomains, domains, types, etc. )

Now if this was regex based, this means that all regex rules would need to be evaluated, for each level of evaluation. This is scary. And on top of that the complexity of having to deal with ambiguities like when two or more regexes satisfy contradictory conditions. And how would the matrix render regexes rules?

I rather not go that way. I am actually pondering longer term how to make the UI simpler without sacrificing what we already have.

 

I've used this extension on Opera 18 but in some instances the matrix is grayed out or shows a white box when the matrix is clicked. In one instance of this occurrence is on Youtube. I'm was using an extension on Opera which can download and utilize Chrome extensions from the Chrome Store. If feasible, when will we see HTTP-SB on Opera's Store? Thanks and appreciate this fantastic program gorhill.

 

I've resubmitted the extension to Opera crew, it's currently under review. My previous submission was refused because I had used Chromium screenshots, and I sort of gave up in the short term. I re-submitted yesterday after a user gratefully provided me a screenshot (a couple more would be great).

Usually a blank popup means no net traffic was seen for a given tab. For example, this happens when the extension is reloaded after a web page was already loaded. A force reload of the web page will enable the popup to be populated with intercepted net traffic. If it's not the case, then there might be some particular issues with Opera which I would have to investigate.

 

Tried it on Chrome portable. Nice extension, but being the lazy guy that I am, ScriptSafe seems to have better blacklisting since it syncs from multiple sources. Certainly this is better for whitelisting, but I won't be using it regularly for now.

 

ScriptSafe is unreliable (as repeatedly discussed in this forum) and it no longer seems to be actively maintained (which includes the integrated hosts files). Regarding blacklisting: Have you looked into the settings page of HTTPSB? It also includes various big hosts files (I think that's what you're talking about). I'm not sure why you consider its blacklisting capabilities inferior compared to ScriptSafe. Besides, everything is reliably blocked in HTTPSB (regardless if contained in the hosts files or not) - I wouldn't rely on that for ScriptSafe.

 

I don't think it would add any complexity, it's just a regex parser. It's kinda built in that everything 'makes sense' as a basic grammar. If you had a contradiction it would be the same as if you had one now, I would imagine.

I hadn't realized you were using keys. That's interesting, though it explains a lot.

To me, regex would simplify things in a way, because you would have a standard language for your ruleset. From the point of the code, perhaps not.

 

Well that sure is a misinformed statement. I'm really curious how you reached this conclusion.

It doesn't sync, just as with HTTPSB, the lists are local. Actually, ScriptSafe author is making his life more difficult to update the list because he concatenates all the lists together in one file locally. For HTTPSB I just created a parser which allows me to just copy the file locally as is, and this allows me to update or add new lists with minimal work.

Now regarding "better blacklisting"...

ScriptSafe:

pgl.yoyo.org/adservers/serverlist.php?hostformat=plain&showintro=0 (~2,762)
www .malwaredomainlist.com/hostslist/hosts.txt (~2,198 )
winhelp2002.mvps.org/hosts.txt (~14,728 )
hosts-file.net (~10,629)
mirror1.malwaredomains.com/files/immortal_domains.txt (~1,516)​

HTTPSB (all updated in January 2014):

pgl.yoyo.org/as/serverlist (2,522)
www .malwaredomainlist.com/hostslist/hosts.txt (1,803)

hosts-file.net/ad-servers (14,458 )
mirror1.malwaredomains.com/files/immortal_domains.txt (2,132)
mirror1.malwaredomains.com/files/justdomains (16,542)
someonewhocares.org/hosts/hosts (10,050)
hosts-file.net/hosts.txt (over 350,000, not enabled by default)
httpsb-blacklist.txt (43)​

Now I wasn't aware of "winhelp2002.mvps.org/hosts.txt", I will see to include it if the terms of use allows it.

EDIT: Also notable, ScriptSafe add a significant overhead to each request, as it very inefficiently evaluate the block/white status of a request. When you add the overhead of all non-blocked requests on a page, I expect it to add seconds to a page load, not milliseconds. Actually I had tested a while ago on an old laptop. I just checked and on my desktop computer the added overhead less dramatic. Still, it's questionable to read from `localStorage` for every single request. I currently roughly get over 2ms per request (was 30ms on laptop). Compare with Disconnect at 0.27ms/request, and HTTPSB at 0.10ms/request. Ghostery is worst though, 3ms/request.

 

Keep in mind it's all javascript code.

Now:

while (evaluation level)
... if white.rules[hostname] return allow;
... if black.rules[hostname] return block;
... Load next evaluation level
​

Regex-based:

while (evaluation level)
... for all white.rules
... ... if white.rules.test(hostname) return allow;
... for all black.rules
... ... if black.rules.test(hostname) return block;
... Load next evaluation level
​

Evaluation level follows the logic of testing narrowest to broadest, i.e. specific subdomain/specific type, ancestor subdomain/specific type, domain/any type, subdomain/any type, ancestor subdomain/any type, domain/any type, any domain/specific type, any domain/any type, and more tests required to enforce strict blocking.

With regex-based rules I would have to test each of the above for each ruleset until a match is found.

That's for the internal, now for the matrix, what if a user click a cell for which a regex was entered. What if he clicks again? Is the regex put back? Maybe it's not what he wants? It's just a click, how to interpret it?

I am geeky and I never used ABE in NoScript. If the extension becomes too complicated, less people will be compelled to use it. If I put in features which requires a lot of explanations, or which side-effects require a lot of explanations even to users not using the features, they will just give up using it. I consider I am already stretching it.

But the good thing is the project GPL, so whoever can turn it into whatever without asking permission.

 

That's entirely fair. I'm all for code simplicity.

 

Sorry, it was my inexperience that resulted in this misunderstanding. Somehow I've overlooked that setting whilst halfheartedly playing around with it.

Now that I look over closely, this extension is indeed superior to ScriptSafe in every way except synced settings. That feature isn't really necessary, but if you may add MVPS HOSTS, that would be great. Thank you.

Oh, and I'm trying out "Auto whitelist page domain". Hopefully it will be enough for my laziness. *Nope, got tired of making rules especially for StumbleUpon, so I allowed everything by default.

 

That site is attempting to load a lot of blacklisted content. You may need to allow some of it for desired viewing.

 

That's the one issue you will always face as a user of these types of blockers. These extensions/programs try to put the power back in your hands but, as you can see with some websites like JL is dealing with, sometimes the only choice you have is to see what they demand you see or get off of their website. GorHill and others do a lot of much needed good for others, but they don't control content providers and can only do so much.