Results 2006
Our 2006 benchmark involved 110 parents and teachers speaking nine
different languages. In addition, we built a lab with its own web servers,
mail servers, file servers and chat servers to test all 30 tools in
exactly the same circumstances. To test the effectiveness of the tools
against a variety of content, we compiled an impressive 5,000 test cases
and classified them using criteria that simulate the concerns of an
average European parent.
The 2006 edition of the benchmark shows that today's filtering tools
are capable of filtering potentially harmful content without seriously
degrading the Internet experience of the youngsters. Yet, the industry
should aim at filtering not only the obvious harmful content but take the
fine-grained considerations of European parents and teachers into account.
Indeed, our tests show that filtering content on-the-fly in a consumer
context remains a challenge.
The benchmark indeed shows that tools performed well in filtering
content from sites with millions of hits per day, containing obvious
content (read: porn) expressed in a common language (read: English). In
fact, we identified 12 products which made a wrong filtering decision in
less than half the cases and there was one tool that was wrong in only 1
out of 16. However, when trying to filter less obvious but equally harmful
content, expressed in a non-English language on for instance private
sites, we found that none of the tools are capable today of adequately
filtering this. In fact all products, without exception, got it wrong in
more than one quarter of the test cases.
 |
- Download the SIP-Bench Synthesis Report, to obtain an overview of the scores of 30 filtering solutions and to learn about the general observations:
- Download the SIP-Bench Detailed Report, to obtain a detailed evaluation of each of the 30 filtering solutions in the benchmark:
- Download SIP-Bench Test and Scoring Methodology, to view the methodology we applied to obtain the test results and scores:
- Download SIP-Bench Overview Results, to view the Summary Results Table:
|
Results 2007 - Results 2008
|
