OPINION
Can you ethically cloak your Web content?
12-02-2003
by Bernie Goldbach
A Web site can use "cloaking" to show search engines different information than it shows to a real visitor. But is it ethical?
When the Web content manager of a leading Irish hotel announced to a search engine seminar that the hotel used "cloaking" to enhance its placement, several Web developers challenged the ethics of that practice. As the presenters at an Irish Internet Association event were careful to explain, ethical cloaking is the only method of cloaking worth considering. Ethical cloaking will cause dynamic pages to appear as static content. That helps enhance search engine placement. However, even this couched explanation proved unacceptable for dozens of content developers.
By "cloaking" a Web page, a Web developer serves the search engine spiders a different optimised page than the page shown to an actual Web site visitor. To cloak, you redirect visitors when they arrive at your Web site.
Successful cloakers actively monitor the electronic identities of every search engine spider. Identities are known as "User-Agent strings." Spiders are electronic robots (or bots) crawling the Internet looking for content. Spiders often change their Internet addresses, so anyone who cloaks needs to stay ahead of highly volatile identification features of the bots.
"So if the Google spider comes along to index your page, it will serve it a page specially designed for Google," said Bernard Tyers from the National Centre for Sensor Research. "If the Altavista bot comes along, it will get an Altavista optimised page."
Developers can manually develop this optimised content. Or they can use cloaking software to serve the spider bot the optimised content-oriented page for harvesting.
"Because cloaking gives spiders different pages than viewers, many search engines regard it as cheating," Tyers said. That's because cloaking is often meant to trick or mislead a search engine about the content on a Web site.
"I have seen optimisation go to the point were the sites in question were of absolutely no use to any human being, despite being apparently brilliant as far as the engines go," said Jon Hanna, a Web developer with Spin Solutions in Dublin.
Discussion on the Open mailing list revealed various cloaking techniques. One way of doing it can be just to load a specific page that is heavily coded with meta-tagged information and content optimised for keyword frequency. Another method uses lots of meta tags and no content. A less-frequent technique involves using a "doorway" page, just for search engine submissions. These doorways often redirect visitors into adult sites.
"We should just write a page with the correct content on it and focus on having the headlines, titles and hyperlinks present a consistent message," said Feargal McKay, Webmaster at Baltimore Technologies.
"The reason I don't like cloaking is that it's an easy way for a competitor to nark on you and for you to be purged from a search engine," said Ruth Maher, Internet co-ordinator from South Tipperary County Council. "If a search engine administrator thinks you present one perspective of content to his spider and another thing to a human, you're on thin ice." Most people cloak to optimise content placement or to hijack phraseology from competition. All the search engines can impose a "Web site death penalty" where the offending site is booted out of the search engine's index.
"Cloaking at its worst is one of the main factors behind many search engines dumping legitimate sites from their databases," said Michele Neylon of Blacknight Solutions. "Cloaking does little to aid the quality of submissions to search engines and directories."
"You are always going to get people trying to subvert search engine results for their own ends," said John McCormac from Hackwatch. "The whole idea of ethical cloaking only means that these guys have not been caught yet."
McCormac thinks search engine operators do not respect the concept of ethical cloaking and that they could easily implement methods of detecting cloaking at work. "It is as simple as presenting two different User-Agent strings and checking the difference in the pages," McCormac said.
"Some spiders will visit a site twice," McCormac said. "The first run includes an initial page check and the second run is the full page check. The first spider may masquerade as an ordinary browser while the second presents the User-Agent of a spider."
Marketers and Web developers can avail of a host of good resources to construct meaningful content for search engine placement. If you need to remain on the visible Web, you should hone your copywriting skills first before trying to cloak your content.
Creating successful email surveys: Denise Cox of email specialist Newsweaver argues that you can tap into your readers' likes or dislikes by surveying them.
