Skip to main content


Showing posts from March 25, 2015 - a search engine bringing the Dark Web into the light

by Mark Stockley

The Dark Web is reflecting a little more light these days.

On Monday I wrote about Memex, DARPA's Deep Web search engine. Memex is a sophisticated tool set that has been in the hands of a few select law enforcement agencies for a year now, but it isn't available to regular users like you and me.

There is another search engine that is though.

Just a few days before I wrote that article, on 11 February, user Virgil Griffith went onto the Tor-talk mailing list and announced Onion City, a Dark Web search engine for the rest of us.

The search engine delves into the anonymous Tor network, finds .onion sites and makes them available to regular users on the ordinary World Wide Web.

Up to now the best way to search for .onion sites has been to get on the Tor network using something like the Tor browser, but Onion City effectively does that bit for you so you can search from the comfort of your favourite, insecure web browser.

The site can do this because it's a Tor2web p…

Memex - DARPA's search engine for the Dark Web

by Mark Stockley

Anyone who used the World Wide Web in the nineties will know that web search has come a long way. Sure, it was easy to get more search results than you knew what to do with in 1999 but it was really hard to get good ones.

What Google did better than Alta Vista, HotBot, Yahoo and the others at the dawn of the millennium was to figure out which search results were the most relevant and respected.

And so it's been ever since - search engines have become fast, simple interfaces that compete based on relevance and earn money from advertising.

Meanwhile, the methods for finding things to put in the search results have remained largely the same - you either tell the search engines your site exists or they find it by following a link on somebody else's website.

That business model has worked extremely well but there's one thing that it does not excel at - depth.

If you don't declare your site's existence and nobody links to it, it doesn't exist - in search e…