Hi all! It’s seems to be difficult to keep up with the EasyList expression lists, and to make matters more confusing, the conversion from one EasyList expression list to SquidGuard can be cumbersome. There are a few article out on Google that people have posted their own SED files to convert from EasyList to squidGuard, that will manipulate the EasyList expression list and convert it to be compatible with SquidGuard. However, most are outout of date and will cause squidGuard to fail to initialize the expression list when issuing a
>squidGuard -C all
I found a site whose owner (I believe) has developed a new SED file that will parse through the latest easylist.txt expression list file and convert it successfully. The site appears to be in Russian, and Google Translate is not able to translate it. Therefore, the purpose of this article is to increase the authors visibility of their SED file.
NOTICE: I am not the owner of the SED file below, no do I take any responsibility for it’s use and distribution.
From https://sites.google.com/site/uttools/squid-adblocker
s/r//g;
/Adblock/d;
/.*$.*/d;
/n/d;
/.*#.*/d;
/@@.*/d;
/^!.*/d;
/^[.*]$/d;
s#http://#||#g;
s////||/g
s,[+.?&/|],\&,g;
s/[/\[/g
s/]/\]/g
s#*#.*#g;
s,$.*$,,g;
s/\|\|(.*)^(.*)/.1\/2/g;
s/\|\|(.*)/.1/g;
/^.*$/d;
/^$/d;
Download the latest easylist.txt expression list.
https://easylist-downloads.adblockplus.org/easylist.txt
Commands:
cd /tmp
vi pattern.sed
–Paste in the SED pattern from above.
wget –no-check-certificate https://easylist-downloads.adblockplus.org/easylist.txt
cat easylist.txt | sed -f pattern.sed > expressions
Now just append or replace your current SquidGuard expression list, and don’t forget to issue a
>squidGuard -C all
to re-initializes your block and expression lists.
Sources:
the wget command should be changed. add –no-check-certificate option to the command line in order to enable the httpS download .
Thank you for the good catch!
not working 🙁
Hmm. Could you elaborate on which part is not working? The sed file? The blocklist?
how to make this work on pfsense commands?
What proxy server does pfsense use? Can you install SquidGuard?
sed output looks broken…
&ad_box_
&ad_channel=
&ad_classid=
&ad_height=
will create
&aamp;d_box_
&aamp;d_chaamp;nnel=
&aamp;d_claamp;ssid=
&aamp;d_height=
reply to enable notifications…
This pattern appears to still work but it has a couple of issues.
1. This website is converting the ampersand and greater than symbols to the HTML entity values. Running the pattern and command directly from copy and paste via bash will not work. Someone else pointed out the sed output is wrong, that’s because its not meant to be the HTML entity in the pattern file!
2. The pattern is missing some semi colons on a couple of line endings.
There are lots of pattern rules around, however most are old and actually cause squidGuard to crash due to segmentation fault. Correcting the above pattern file, appears to work OK in 2015!
Why don’t you paste the corrected pattern file?
I’m actually working on a small GitHub project that automatics the conversion and building of the lists. I actually used a different script from somewhere else.
I’m currently modifying it to work better, and will publish on GitHub soon.
You can find my script here:
https://github.com/jamesmacwhite/squidguard-adblock
Uses the latest working sed pattern