Open Source Network Blog

Ray Bogman

Ray Bogman is a former security expert at KPN, is co-founder of Jira ICT, gives Magento and Joomla! consults and trainings. Authors profile

In the world of search engines, duplicate content is always a big problem. Search engines need to determine which websites are useful to users and which are not, and one of the ways to accomplish this is to check for duplicate content.

A site with a lot of duplicate content is considered to be spamming the visitor, but sometimes duplicate content is created without the webmaster knowing.

An example with Joomla!: If you create an article and this article is visited through a menu-item within the main menu, Joomla! will create an unique URL with not only the MySQL identifier for the article itself but also an unique identifier of the menu-item (the Itemid):

index.php?option=com_content&view=article&id=4&Itemid=5

When a second menu-item is created pointing towards the same article, a new URL is constructed, so duplicate content is created.

index.php?option=com_content&view=article&id=4&Itemid=6

Even worse, menu-items are something you can control, but sometimes new URLs are constructed by Joomla! without you knowing. For instance the Latest News-module is notorious for this behaviour.

Though the search engines try to be intelligent enough to detect whether duplicate content is the work of some evil spammer or the work of the CMS, it would be useful if we could help the search engine a bit with this.

Search engines try to find the solution

Because webmasters are facing this challenge and are sometimes unable to solve this problem, search engines like Google Search or MSN Search try to find alternative ways of preventing the indexation of duplicate content.

I found this article on yoast.com which explains that Google, Yahoo and Microsoft came up with a solution that allows webmasters to insert a "Canonical" metatag in the header of an HTML document.

<link rel="canonical" href="/ index.php?option=com_content&view=article&id=4&Itemid=5" />

The author of the blog already mentions plugins for WordPress and Drupal, but what about Joomla!?

The canonical metatag and Joomla!

The tools for WordPress and Drupal allow you to easily add a "Canonical URL" to a blogpost or article. Such an extension does not (yet) exist for Joomla!? But what's the use of such a tool? It will require you to manually set the correct URL for each article, but is this the real fix for the problem?

In fact, Joomla! 1.5 solves many of the problems. When the standard Joomla! SEF is combined with handy tools like the HP Router Plugin (which removes the Itemid from SEF URLs), duplicate URLs can be prevented. It requires some fiddling, true, but in the end you'll get there.

Canonifying non-article content

With some work you can prevent duplicate article-URLs. But what about other Joomla! components? Take for instance VirtueMart with which you can built an entire webshop within Joomla!. VirtueMart offers a different kind of content (products) which are displayed on various pages.

On a product-overview page you can reorder all the shown products, for instance order them by name. As soon as you reorder the items, the URL will change:

index.php?orderby=product_name&DescOrderBy=DESC&Itemid=55&option=com_virtuemart&page=shop.browse&category_id=1&manufacturer_id=0&keyword=&keyword1=&keyword2=

But in fact the URL that should be indexed by search engines could be much simpler:

index.php?Itemid=55&option=com_virtuemart&page=shop.browse&category_id=1&lang=en-GB

The logic here is that (in a perfect world) the VirtueMart component should add a metatag "canonical" to its own pages, making it easier for search engines to identify and index the right content:

<link rel="canonical" href="/ index.php?Itemid=55&option=com_virtuemart&page=shop.browse&category_id=1&lang=en-GB" />

Not in the core, but in the extension

Ofcourse the same counts for all Joomla! extensions out there: If the extension uses multiple URLs which in fact use the same duplicate content, the extension should fix this problem by adding the proper metatags. It's up to the extension builder to prevent duplicate content.