One of the first things I do on installing a new browser, is to install the Ad block extension, for a cleaner, stress free browsing experience. It is also the primary reason I prefer Firefox on my mobile devices (Chrome still does not support extensions on mobile). One of the most popular and most efficient extension in uBlock. More than ads, they can also be configured to block other web annoyances like social buttons (which have been proven to be a significant drain on your bandwidth, and slow down your browser), and sites that are known to inject malware. Multiple lists are being maintained for known ad/malware servers and they can be chosen in the add-on setup screen.
But if you want to block these annoyances on all devices in your home (unsupported browsers, Apple devices etc), it is now possible to have this functionality in the home gateway (this is usually the wireless router, connected to your DSL/Cable modem) itself, with a little bit of hacking. This has unintended (but welcome) consequence of blocking interstitial ads in games and YouTube videos too! For this we will need a hacker-friendly routers than can run open-source firmwares like tomato, openWRT, or DD-WRT. Even though the steps needed to accomplish this might be different among these distributions, the basic mechanism is the same - the gateway advertises itself as the DNS server to all the clients in the home network, and for every DNS query made by the clients, it matches them against its block-list and returns an invalid reply if there is a match. So whenever a web page is loaded on the client, all requests to load ads, malware or social buttons simply fails.
The only downside to this approach is that blocking is universal among everyone in the network, and sometimes even legitimate sites get shot down due to false positives or over aggressive rules. We'll see couple of ways of bypassing the gateway blocking. The browser add-ons can be configured to be disabled on certain domains, or even just few select pages, but with gateway method, it is an all or nothing proposition.
Now let us see the steps to accomplish this in detail:
The first step in this process would be to enable DNS server in the gateway. On Tomato this is done by enabling internal server under DHCP/DNS, and on dd-wrt by enabling dnsmasq under 'Services' tab. You should also configure your clients to get their DNS config from the gateway (which is usually the default, unless otherwise changed) We'll see that this also helps in bypassing the gateway adblock if need be (like when some legitimate pages fails to load).
Multiple lists are being maintained for ad and malware servers. You can checkout some of them at AdAway site. You can choose one or multiples of them to your router, remove duplicates and convert it to a format dnsmasq understands (one host per line, with destination IP as itself). You should strike a balance because having too big a hosts file will make the router unstable, depending on how much memory it has. Then you have to edit the dnsmasq configuration file to pint to your blocklist. You might have to restart the service for new configuration to be read.
But if you want to block these annoyances on all devices in your home (unsupported browsers, Apple devices etc), it is now possible to have this functionality in the home gateway (this is usually the wireless router, connected to your DSL/Cable modem) itself, with a little bit of hacking. This has unintended (but welcome) consequence of blocking interstitial ads in games and YouTube videos too! For this we will need a hacker-friendly routers than can run open-source firmwares like tomato, openWRT, or DD-WRT. Even though the steps needed to accomplish this might be different among these distributions, the basic mechanism is the same - the gateway advertises itself as the DNS server to all the clients in the home network, and for every DNS query made by the clients, it matches them against its block-list and returns an invalid reply if there is a match. So whenever a web page is loaded on the client, all requests to load ads, malware or social buttons simply fails.
The only downside to this approach is that blocking is universal among everyone in the network, and sometimes even legitimate sites get shot down due to false positives or over aggressive rules. We'll see couple of ways of bypassing the gateway blocking. The browser add-ons can be configured to be disabled on certain domains, or even just few select pages, but with gateway method, it is an all or nothing proposition.
Now let us see the steps to accomplish this in detail:
DNS Configuration
The first step in this process would be to enable DNS server in the gateway. On Tomato this is done by enabling internal server under DHCP/DNS, and on dd-wrt by enabling dnsmasq under 'Services' tab. You should also configure your clients to get their DNS config from the gateway (which is usually the default, unless otherwise changed) We'll see that this also helps in bypassing the gateway adblock if need be (like when some legitimate pages fails to load).
Block List
Multiple lists are being maintained for ad and malware servers. You can checkout some of them at AdAway site. You can choose one or multiples of them to your router, remove duplicates and convert it to a format dnsmasq understands (one host per line, with destination IP as itself). You should strike a balance because having too big a hosts file will make the router unstable, depending on how much memory it has. Then you have to edit the dnsmasq configuration file to pint to your blocklist. You might have to restart the service for new configuration to be read.
Since they are based on squashfs they do not have any persistent filesystem that spans reboots. So even though all of this can be on the telnet/ssh console of the routers, all of this is lost when gateway restarts. So it is a good idea to package the all the above steps in to a shell script, and type it out in the web admin page (Tomato: under admin->scripts->Wan up, DD-WRT: admin->commands). Anything saved in the web admin is saved into NVRAM of the gateway which is persistent.
Here is one such script. It is customized for dd-wrt but with little changes it will work fine in tomato too.
#!/bin/sh
logger WAN up script executing
logger Downloading http://www.mvps.org/winhelp2002/hosts.txt
wget -O - http://www.mvps.org/winhelp2002/hosts.txt |
grep 0.0.0.0 | tr -s ' ' | tr ' ' '\t' | tr -d '\015' |
sed 's/[[:space:]]*#.*$//g;' > /tmp/hosts
wget -O - http://www.malwaredomainlist.com/hostslist/hosts.txt
| grep 127.0.0.1 | grep -v localhost | tr -s ' ' |
tr ' ' '\t' | tr -d '\015' |
sed 's/127.0.0.1/0.0.0.0/g; s/[[:space:]]*#.*$//g;' >> /tmp/hosts
if test -s /tmp/blocklist
then
rm /tmp/blocklist
fi
uniq -u /tmp/hosts /tmp/blocklist
rm /tmp/hosts
logger Restarting dnsmasq
killall dnsmasq
grep addn-hosts /tmp/dnsmasq.conf ||
echo "addn-hosts=/tmp/blocklist" >> /tmp/dnsmasq.conf
dnsmasq --conf-file=/tmp/dnsmasq.conf
DNS Target
You can spoof the target of the blocked hosts in multiple ways:
- It can be the address of the gateway itself, in which case client tries to fetch the 'ad' from the gateway, and just times out. This is not favoured because it slows down the browser waiting for requests to timeout
- It can be 0.0.0.0, in which case the borwser (at least Firefox) times out immediately
- In both cases above, the block containing the 'ad' will have the standard error screen of the browser, making the page ugly. A better alternative would be to host a super simple webserver on the gateway, which responds with a single pixel image for any request it receives. PixelServ is just one such example.
Regular Updates
The blocklists are regularly updated to keep up with new ad and malware servers by the maintainers. So it is imperative to keep the gateway also updated with the latest blocklist. So you can make this download/convert script a regularly scheduled cron job. Both Tomato and DD-WRT have provisions for this in their web admin.
Gateway Bypassing
If for any reason you would like to avoid this gateway ad-blocking, you could do that in couple of ways:
- Since this works because the gateways injects itself in the middle as the DNS server, the first method would be to manually reconfigure the client to bypass the gateway and direct all its DNS queries to the ISP or better yet, Google's servers.
- Almost all routers have a button on the outside for WDS connections. Almost all third-party firmwares allow us to reconfigure this button to execute any command. We can make it to stop or start the dnsmasq service. This method affects all the hosts in the network, not just one client, as in the previous method.