Smarter Search for South Park

By David F. Carr Print this article Print

MTV Networks moves to Solr-powered search technology for the websites behind many of its popular shows.

See our slideshow on Solr, and one on a related project, Hadoop.

The open source Solr search engine is emerging as the standard for new and revamped websites launched by MTV Networks and is already powering the search on key properties such as TheDailyShow.com, ColbertNation.com, and SouthParkStudios.com.

"We have about 15 sites on Solr and more coming later this year," says Mark Cohen, vice president of the data platform group at MTV.

The flagship MTV.com website still bases its search function on the Google Search Appliance and is unlikely to be converted anytime soon, he said. That main website is so sprawling and more complex that the Google approach to crawling and indexing web content is the most practical approach. But on the more focused websites MTV Networks is building around specific shows or to appeal to appeal to specific audiences, Solr is becoming the default choice because of the way it can take advantage of categories or "facets" within search results, Cohen said.

Solr is a branch of the Apache Lucene project, which has also spawned other powerful spin-offs such as the Hadoop distributed computing system. Lucene itself is a Java library used as a core component for multiple search systems. Solr was originally developed by Yonik Seeley while at CNET Networks, where it is used on the CNET Reviews site to help visitors drill down through search results by category. Seeley has since joined Lucid Imagination, a firm formed to provide commercial distributions, training, and support for Lucene and Solr.

MTV is talking with Lucid (which provided us with an introduction) but has yet to sign up for its services, says Warren Habib, Senior Vice President of Digital Platform Development at MTV Networks. MTV likes to take advantage of open source products wherever practical, Habib says. "We're fundamentally a media company, not a technology company," he says, and open source software not only helps keep costs down but tends to be backed by a large pool of skilled developers. But he says MTV is also "open" to taking advantage of commercial support options to make its use of these technologies more effective.

This article was originally published on 2009-09-17
David F. Carr David F. Carr is the Technology Editor for Baseline Magazine, a Ziff Davis publication focused on information technology and its management, with an emphasis on measurable, bottom-line results. He wrote two of Baseline's cover stories focused on the role of technology in disaster recovery, one focused on the response to the tsunami in Indonesia and another on the City of New Orleans after Hurricane Katrina.David has been the author or co-author of many Baseline Case Dissections on corporate technology successes and failures (such as the role of Kmart's inept supply chain implementation in its decline versus Wal-Mart or the successful use of technology to create new market opportunities for office furniture maker Herman Miller). He has also written about the FAA's halting attempts to modernize air traffic control, and in 2003 he traveled to Sierra Leone and Liberia to report on the role of technology in United Nations peacekeeping.David joined Baseline prior to the launch of the magazine in 2001 and helped define popular elements of the magazine such as Gotcha!, which offers cautionary tales about technology pitfalls and how to avoid them.
eWeek eWeek

Have the latest technology news and resources emailed to you everyday.