August 26th, 2016

linux  |   server  |   network

Having a web filtering proxy available could really improve the experience of web browsing by filtering out some certain unwanted content and thus speed up the browsing.

The filtering is especially important when the web is browsed on (older) mobile devices, when there is no fast broadband connection available or when usage allowance is seriously capped. Many websites do not have alternative mobile views and irresponsibly combine all kinds of Javascript libraries and plugins that enable useless bells and whistles (well, code re-use can be complicated - instead of a banana you may get a gorilla holding the banana). Images may have huge resolutions and add their part to the excessive bandwidth usage. Some sites may even follow the nasty new trend of playing an advertisement video with the loud sound almost instantly upon a visit. Being web design at its worst, this is a privacy-invading experience for a user who would then be forced to relentlessly start to scroll the page up and down to find the culprit.

Setting up a filtering proxy is quite simple because there are so many cloud server providers available and there is no need to manage the hardware by yourself. For just a few dollars per month, you can take your pick, spin up a virtual Linux instance and off you go! In this tutorial, it is assumed that a minimal instance of Ubuntu has been set up. Here are the programs needed for the filtering proxy:

  • Privoxy
  • Squid
  • Apache2-Utils


Privoxy is the filtering proxy and the other two are needed to secure it by authentication. The package of Privoxy available on the 14.04 version of Ubuntu does not have authentication capabilities on its own. While there is a patch available, it has some issues. Squid takes care of authentication and passes requests on to Privoxy and as a bonus, it acts as a caching proxy. Apache2-Utils is needed to generate a valid htpasswd file needed to save access credentials for the authentication process. NCSA authentication mechanism is secure for this purpose as only password hashes are transmitted and Squid does support many other authentication means by the use of plugins.


1. Let’s install the needed packages:
sudo apt-get install privoxy  squid apache2-utils


​2. Start Privoxy
sudo service privoxy start


3. Generate the passwd file, using the proxyuser as the username:
sudo htpasswd -c /etc/squid3/passwd proxyuser
chmod o+r /etc/squid3/passwd


4. Find the NCSA auth helper:
dpkg -L squid3 | grep basic_ncsa_auth

The response will be like the following:
/usr/share/man/man8/basic_ncsa_auth.8.gz
/usr/lib/squid3/basic_ncsa_auth

The location is The location is /usr/lib/squid3/ . This will be needed in the configuration file.


5. Change the configuration for Squid at /etc/squid3/squid.conf:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
 
# use some non-standard port
http_port 12345

# remove proxy headers
# https://scheller-it-consulting.com/linux/remove-proxy-headers-squid/
via off
forwarded_for off
header_access From deny all
header_access Server deny all
header_access WWW-Authenticate deny all
header_access Link deny all
header_access Cache-Control deny all
header_access Proxy-Connection deny all
header_access X-Cache deny all
header_access X-Cache-Lookup deny all
header_access Via deny all
header_access Forwarded-For deny all
header_access X-Forwarded-For deny all
header_access Pragma deny all
header_access Keep-Alive deny all

# define Privoxy as parent proxy
cache_peer localhost parent 8118 7 no-digest no-query

# no one else should access
never_direct allow all

# using the NCSA auth helper
# https://www.cyberciti.biz/tips/linux-unix-squid-proxy-server-authentication.html
# http://www.linuxjournal.com/magazine/paranoid-penguin-building-secure-squid-web-proxy-part-iii?page=0,1
auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid3/passwd #the helper used for authentication
auth_param basic children 1 #the number of concurrent authentication processes
auth_param basic realm Squid proxy-caching web server #greeting text on the dialog
auth_param basiccredentialsttl 24 hours #the user will need to re-authenticate after 24 hours
auth_param basiccasesensitive off #user names are not case sensitive

# enabling the ncsa authentication
acl ncsa_users proxy_auth REQUIRED

# enabling the access list
http_access allow ncsa_users


6. Start Squid
sudo service squid3 start


7. Adding Privoxy to the startup/shutdown sequence. Squid should already start during boot, you can check with sudo initctl show-config squid3
sudo update-rc.d privoxy defaults


Now everything can be tested. Insert your server’s public IP address along with port number, username and password into the proxy settings on your mobile device and/or download a proxy add-on for your desktop browser (for example FoxyProxy).

Some ideas for further thought:

  • bypassing certain URLs, showing all content for these
  • bypassing certain IP addresses
  • replacing Squid with some lightweight proxy
  • starting to use a cron script for Privoxy to automatically updates block lists of well-known malicious servers that are churning out ad videos and images