When we set out to build AutoBlogged, the blogging climate at the time was already tense. Autoblogging in general had already received a reputation as a blackhat SEO method and was strongly associated with spamming. In fact, there was even a cute name for spamming autobloggers: sploggers.
In our minds, autoblogging was not about stealing content or spamming, it was about taking a world of content and adding value to it. But we knew to change the negative perception of autoblogging tools we had to make it much easier to play nice with those original content creators.
Over the last few years we have added a number of features to respect content ownership and also to meet the quality guidelines of search engines. Of course, anyone can bypass these features and abuse the power of AutoBlogged all they want, but we have found that most users realize the value in implementing these features.
This is how we play nice:
AutoBlogged has always created small excerpts of the content by default, even if the RSS feed contains full articles. AutoBlogged allows you to create excerpts based on a number of paragraphs, sentences, or words. While the exact definition of fair use depends on a number of factors, many would consider a few sentences to be an acceptable excerpt that does not violate the copyrights of others.
With images, AutoBlogged will by default create smaller thumbnails and gives the user the option to copy images to host them locally or to link to the original images URLs. Strangely, some content owners will get angry if you hotlink to their images, and other content owners will get angry if you make copies. AutoBlogged let’s you choose the most appropriate method.
It is important to give credit to the content owners. The default post template creates a link after the excerpt that points to the original content. The title and anchor text are the article title and the link is followed by the search engines. What most content owners don’t realize is that this is an extremely valuable link because it is usually coming from a page that is already packed with keywords and content closely related to the subject. Anyone with link building experience knows that it is easy to build generic backlinks to a site’s homepage, but much more difficult to build deep links coming from pages with relevant content and keywords. Where possible, AutoBlogged will include the original author’s targeted categories and tags when adding the post.
AutoBlogged also has other ways to supplement attribution to the original site. It extracts the source name and description, the name of the author, links to logos and favicons, and even copyrights if they are available. All of this can be added to the post template using our post template syntax.
HTTP Retrieval Customization
AutoBlogged allows you to customize several HTTP options to respect other sites. You can set both the HTTP referrer and the user agent. This allows you to identify your site URL and, if you want, add a message to the user-agent identifying it as an aggregator.
Furthermore, you can set RSS cache times to avoid having to visit a feed URL more than necessary.
URL and Keyword Exclusion
Occasionally when running autoblogs in the past, we have been contacted by other bloggers who have kindly asked that we not include their content on our site. Wanting to accommodate these authors, we added blacklists that let you exclude articles with certain keywords, URLs, or entire domains. AutoBlogged also saves the original source as custom fields in WordPress which allows us to quickly select all the content from a particular source so we can remove it.
Publisher Meta Tags
AutoBlogged also has an option to communicate to search engines the original content URL via publisher metadata. AutoBlogged can use the syndication-source tag to indicate where the content originated. This allows search engines to better identify which sources are the original authors so they are favored search results.
Now some of you may think that doing this will hurt your own search rankings but it is important to note that autoblogs never rank well on individual content pages. After all, the article is normally just an excerpt and already links to the original content. The good will both with content owners and search engines will go much farther than the search rank of any one page.
Respecting Robots Restrictions
Depending on your configuration, AutoBlogged will normally visit the original URL to analyze the original content. When it does this, you can configure it to also take a look at any robots meta tag restrictions. If AutoBlogged sees any instructions to not index the content, it will skip that particular post.
AutoBlogged will look for meta tags directed at googlebot, msnbot, bingbot, or slurp and will respect any of these. It will also respect the noindex, noarchive, nosnippet, and none tags directed at robots.
Furthermore, we have even added our own meta tags that allows a content owner to specifically prevent copying by AutoBlogged (and we hope that other autoblogging tools will adopt this). This can be accomplished by adding one of these two meta tags:
<meta name="autoblogs" content="noindex, nosnippet, noarchive">
<meta name="robots" content="noautobloggers">
These meta tags will be ignored by search engines but will be respected by AutoBlogged if the user has it configured to do so.
What You Can Do
We have had quite a bit of experience running autoblogs over the years and have found a number of techniques that have helped us quite a bit when it comes to working with others. First, we found that it is important that you make your web site look professional and focused on a particular topic. A blog using the default WordPress template with unrelated content will quickly get your site flagged as spam or your hosting provider hit with a DMCA takedown notice.
It is also important to provide a contact form so that users can communicate with you directly rather than having to go to your hosting company. We have found that providing a contact form always leads to polite and open exchanges with content owners.
It also helps to identify your site as an automated content aggregator, such as in a footer or an about page. There is little benefit in trying to pretend that your site is not automated and this gives you the opportunity to explain that others can communicate with you to have their site excluded.
Most of all, please do not abuse AutoBlogged’s power to create massive spam networks. This benefits no one and hurts everyone. If you are going to autoblog, get serious about it and do it right. And most of all, play nice.