The Development of Web Directories
News September 11th, 2017Web directories have been around since before the Internet became a part of everyone’s lives. In fact, the earliest web directory is the World Wide Web Virtual Directory (vlib.org), which was created Tim Berners-Lee. Ever since, directories have become a tool for many different purposes. Most prevalent are those that simply store information and online resources in the form of “listings” in structured categories that make them easier to find. These will often include descriptions of the resources, and other bits of information like load times, security, and the last time the site was edited. Web directories can vary widely in scope and purpose from listing information about local businesses to human edited lists of resources, which imply a certain editorial discretion.
One of the oldest and most popular web directories was Yahoo Directory, which started as a simple listing of links, provided by its creators Jerry Yang and David Filo, and evolved into the largest search engine on the web before Google took over. The directory served the same purpose as any of its kind: a simple means to store web links before search engines provided instant, constantly-changing web results. Web-crawling came about around the turn of the 21st century, and the days of hand-typed directories were numbered. Fortunately, Yahoo Directory made the transition to the new technology, launching Yahoo Search in 2002, and they’ve been (unsuccessfully) competing with Google for web search supremacy ever since. Despite its antiquated underpinnings, Yahoo Directory was maintained by the company up until 2014, when it was abandoned in favor of localized directories for each country.
Before 2000, Yahoo Directory’s main competitor among the popular directories was a The Open Directory Project, also known as DMOZ. DMOZ was launched in 1998 by a couple of programmers at Sun Microsystems, the company that created the Java programming language. It is one of the early open-content platforms on the web–meaning volunteer editors could edit its content–making it a foundation for later open formats like Wikipedia. Shortly after its creation, DMOZ was acquired by AOL along with Netscape. AOL maintained DMOZ up until March 2017, ultimately shutting it down because they no longer wanted to maintain it, according to the press release. During its height DMOZ had over 93,000 active users, so it should be no surprise that there have been constant talks about bring it back from the archives sometime in the near future.
Current web directories are following much of the same trends as the old, only with the more robust technology behind them that make these sites accessible by search engines rather than competing entities. Jasmine Directory, a web directory that combines the minimalist interface of DMOZ with the searchable navigation of Yahoo directory, shows that search engines have not rendered the web directory obsolete. In the current Internet culture that is obsessed with search engine optimization, there are many such web directories that operate with a nefarious purpose: to proliferate the web with untrustworthy backlinks from unverified sources. Jasmine Directory seems to take its cue from the integrity of the previously mentioned sites. Their foundational listings are already extensive, with thousands of sites already present and categorized in all topics, from finance to sports to cooking. Navigation seems to follow the three-click rule, which states that you should be able to find your way from point A to anywhere on a website with less than three clicks, and the breadcrumb trail provided on each page makes it impossible to get lost. There’s also a handy search engine at the top that can pinpoint keywords within blocks of text (not just in the titles) and find the page you’re looking for if the categories aren’t your thing.
The web directory industry is both a thing of the past and a thing of the present. As long as listings are useful for a web directory’s users, there will be a place for them on the Internet. It can be difficult to screen through the many imposters to find a legitimate web directory, however. You should avoid the sites that try to get you to download software, place advertisements that pose as main content, and have a high percentage of links that lead to broken pages. Also, it doesn’t hurt to find directories that are verified by trusted services like the Better Business Bureau or Best of the Web trust seal.