About a year ago, the power and speed of a caffeinated Google was supposed to help improve long tail search query results. In this YouTube video, Matt Cutts from Google says of the 2010 update:

“It’s an algorithmic change that changes how we assess which sites are the best match for long tail queries.”

And as Vanessa Fox describes, he recommends that site owners affected by this algo:

• Evaluate the quality of the site and if the it’s really is the most relevant match for the impacted queries
• Think about what “great content” could be added
• Determine if the the site is considered an “authority”
• Ensure that the page does more than simply match the keywords in the query

Let’s keep that criteria in mind as, one year later, we take a peek at how they’re fairing. Here is a long tail query related to a very specific jewelry item: yellow gold stud labret 4mm

Here is a slightly marked-up look at the top 100 results at Google.com, as searched from Canada (so international index, which I have discussed previously over at NVI), starting with the top 10. The endemic problem appears to be with the top level domains .tk and .co.cc, both of which are offered for free to register in bulk. The dotTK registrar even advertises that not only will they give you a free domain name, but they’ll auto-link it with other similar .tk domains. Wonderful.

SERP for results 1-10:

• SERP for results 11-30
• SERP for results 31-50
• SERP for results 51-70
• SERP for results 71-90
• SERP for results 91-100

Here is the breakdown of the top 100 results returned by Google for that query:

Sure, this is a bit of an odd test because we’re looking at a product that is being pushed out to feed partners. But that is a fair test, this is content that is commonly duplicated around the web and Google is doing a downright MISERABLE job of organizing it. Isn’t that their mission statement?

Two things are made crystal clear from this:

• Google needs to crack down on full-domain-spam for TLDs that are free to bulk-register. They are being abused heavily, but only for long-tail, so it’s not getting much attention.
• Online commerce folk need to PROTECT YOUR PRODUCT DESCRIPTIONS!

We had to have this client write entirely different descriptions for their distribution feed and website, the only way to ensure the website contained unique content – this is a long, expensive process, but as it stands right now, you can’t trust Google to get it right on its own.

The Google Long Tail Fail Whale is a post from:

The Discovery

Following an overnight traffic drop due to the “Panda update” we began investigating the potential issues of why the site was targeted.  The main observations we found on WebmasterWorld from colleagues was that low quality, commonly duplicated content would be affected.  At first we dismissed this as a potential cause because Freshtrends had been manually creating unique product descriptions for all products for years.  Aside from Google Analytics, we looked for more clues of where our rankings had been lost and started grabbing chunks of text to use as queries – This is when we noticed that thousands of long tail phrases (resembling product title pages) such as:

“3mm Blue Zircon 14kt Yellow Gold Prong Set Labret”

Would bring pages upon pages of spammy search results instead of the actual original Freshtrends page containing that information.  We were literally seeing the majority of the first 100 results comprised of nothing other than spammy domains within the .tk and .co.cc extentions.  Here are a few examples and screenshots or SERPs:

Screenshots of first 2 pages of SERPsattached here and here.

The perpetrator was actually a member of the Freshtrends affiliate program via Commission Junction. Clicking through from the SERP links would get you a nice elegant Javascript redirect to the actual product page on Freshtrends.com (complete with Affiliate cookie).  Sneaky right?  This affiliate essentially hijacked the previous ranking for Freshtrends, sent the traffic anyways, and made a cut on the sales.

How did this affiliate fool Google?

1. Using .tk and co.cc domains he was able to mass-generate free and unique domain names.  This meant that he could practically add a domain name per product offered on the site and create domain names that closely resembled all the words within a product title (including sizing and product variations).

2. The Affiliate then scraped all unique content from the original domain and added fake comments using a random twitter feed.  Sometimes random Youtube videos were used. This fooled Google bots into thinking that it is an active page with comments and unique content.  See Google Cache for one of the SERPs.

3. The Affiliate then used Javascript redirection (Cloaking) to send users to the site with the original content.

We contacted Commission Junction about this user, and they promptly had him removed.  Since then all the domains we look at are returning 403 errors and are being purged from the SERPs…  But as of this writing there are still copies in Google’s cache complete with the JS scripts that are used to cloak.

We also contacted Google about this.  It seemed impossible to us that the first 100 search results for a product title could actually be spammy URLs with a .tk extention.  I mean we’re talking about Google here… in 2011 – these look like prototypical spam sites from years ago and they’re dominating the SERPs.

Throughout the course of this Bing hasn’t shown any problems. In fact Freshtrends.com is on the top result for the query and no other .tk domains are on the 1st page. Perhaps Google’s caffeinated indexing has gotten so fast it can’t keep up with itself, and hasn’t got the time to account for quality… even after an update proclaimed to deal with just that.

Everything in this blog post was printed with the permission of Freshtrends.com, as they would like to see as much light shed on these shady practices as us. Has anyone else seen any recent egregious examples they’d like to share?

Hijacking Google Rankings Through Content Scraping and Cloaking in 2011 is a post from:

I think Mr Hammerbacher is being oversimplistic.  This was probably taken out of context to “explain” the reason he is leaving Facebook, which is likely to be more complex than just this comment.  I doubt he only sees platforms such as Facebook as only concentrating on Ad revenue ; these platforms have allowed whole countries to have peaceful revolutions.   Facebook, Twitter and even Google, aside from being profitable marketing machines, have created tools which inform and allow people to organize themselves through grassroot movements.

It is the same as saying that because Fox news misinforms the population and advertises, journalists in general are only interested in ratings. I think a blanket statement like this is untrue. Many gray areas exist. Think about the implications of what internet tools have made possible in the last 10 years: Democratization of political thought, open source programming and advancement and the sharing of scientific knowledge regardless of distance.

Furthermore, and to be “Facebook specific” they did not begin advertising until the platform was already functioning as a powerful communication tool. Yes they are monetizing through ads, but they are less intrusive than what we are normally used to through TV, print and any other form of media. The trick is to keep a balance between advertising and the usefulness of a platform. I think we can all agree that Facebook ads are not particularly aggressive in nature and are clearly distinguishable from the content you go to Facebook for.

The fact that Web Analysts like me use “user behavior” to analyze improvements in a system is not necessarily bad. It isn’t solely done to increase “ad clicks” as much as to make improvements on the user experience: Google for example constantly improves its search algorithm, Facebook constantly uses data to improve user interaction and experience, and twitter as well.

These improvements help us shape the future of technology and communication tools by defining how users interact with them. The fact that it is currently motivated by an advertising platform does not necessarily mean it is bad as long as that delicate balance of Ads/usefulness of the application is maintained.   If advertising is abused, users will abandon a platform (ie. Myspace & Digg).  On the other hand too little advertising and a platform can quickly become too expensive to maintain.   What happens then? Platforms who abuse advertising die off… those who find and maintain that delicate balance are the ones who survive.

What do you think?

Are The Best Minds of Our Generation Being Wasted On Optimizing Ad Revenue? is a post from:

But this blog is not about hating Omniture and praising any other tool. This blog is about Adapting. The real world of Web Marketing does not accept excuses. Either you get your skills working in your company or you Die. Its that simple. Here is Part 2 of “The Good The Bad and The Useless.”

GA rocks, but there is a particular report that really bugs me…

The Visits to Purchase Report:

This report shows how many visitors purchased on their first visit.

So out of all the purchases made, approximately 70% made their purchase on their first visit. That means that there were 30% that needed some reassurance before coming back and making the purchase. Maybe they needed to consult with a spouse or friend for reassurance, ask their parents for a credit card when they got home from work, etc.

But this information is not that useful if we don’t segment it. So let’s segment for referral traffic only. You can see details of this in Avinash’s Web presentation.

Great, now we can see that referral traffic converts only at approximately 56% on the first visit? That’s a big difference from the site average! What’s going on here? At this point my logic would be to see this information and compare sources against each other. Let’s see what happens when we try to compare Direct traffic, Organic Search referrals and referring domains:

Do you see the problem with this report? When we saw only one segment, the referral report contained 100% of referral traffic of which 56% converted on the first visit and approximately 16% on the second visit. But all of a sudden when I add the other segments to compare against each other in the same report, the segments are showing the percentage of the overall site traffic.

This report is telling us that out of ALL transactions, 1.46% of them converted on a second referral traffic visit. So what?!! This brings everything out of context. With this report, I can no longer compare referral traffic as performing worse on the first visit than “non-paid” traffic.

I now have to go into excel and do my number crunching because of the way existing data is being shown. I get something like this:

Now that we have some context I can clearly see that referral purchases are converting less on the first visit than the site average, or any other traffic source in this report. Obviously there is standard deviation, because we have more data on some of the sources than others, but a larger data set should give you more accurate results.

If we are going to segment properly, we need to keep the important metrics in context, otherwise we can make important errors and costly marketing mistakes.

Why is this report useful?

Think of all the 1st time visitors that could have converted and thought “I’ll come back tomorrow” but forget, and never do so at all. If you notice a particular segment’s traffic sucks, you can create incentives for these particular visitors. Through landing pages or using IP Delivery wisely.

An idea: Offer a catfish ad or Ajax bar on the top with a call to action such as: “10% off Coupon Code: jhtus987 only valid for next 3 hours.” Would it work? Maybe, Maybe not. But shouldn’t you test it out?

What reports would you improve in Google analytics? How would you improve them?

Google Analytics: Do You Have a Segmentation Problem? is a post from:

Omniture… my old friend, your old argument that I don’t quite understand your capabilities will not work. As a professional web analyst, and having consulted on some of the biggest North American websites, Oggy is calling you out on your bullshit.

Top 10 Reasons your Web Analytics packages are useless:

10. The company’s IT department has to get involved and “brainstorm” how they are going to implement that “new company metric” instead of copy/pasting a code provided by a marketing geek on any page/link/action on the site.
9. “Visits” are called “Searches” when they come from Search Engines, “Instances” when they come from Referral Sites and “Visits” when they are Direct Traffic. Oh no, “instances” are for Direct Traffic, no wait!…. sigh
8. You can only segment data if marketing contacted their Omniture “account manager” and informed him what they want to segment. This has to be at least 3 weeks in advance so the “installation” of that “customization” can be applied. *Additional fees not included.
7. Getting around the dashboards and navigation is difficult because the “analyst” in the Marketing Department has just come back from two a week vacation and the “new” 14th Beta release came out. “Did we mention that your code no longer works properly? *cough cough*, trust us it will do soooo much more when you update it.” (see #1)
6. The Company’s CEO tells Marketing that he wants a new report, and when they say it’s not possible, he reminds them that they pay top dollar for their Web Analytics solution. If Marketing can’t produce it, it must be due to the department’s incompetence.
5. The Company’s CEO is still telling the Marketing Department what the reports should look like?! Tsk, tsk….
4. Marketing cannot access reports because an update to the Omniture servers is being done. Additionally, for the past week, the application has taken so long to load that Marketing pretended the printer was broken. Consequently, the IT guy wasted 2 hours to notice the printer was just fine.
3. The company invested in 2 employees’ flight to Utah with a 2 week stay for an “installation course” (one guy from IT and one from Marketing). When they came back they knew what Evars and Sprops were, but forgot the company site’s KPIs.
2. The tool costs more than the 2 full time employees that are trying to install that new, “complicated” metric the company wants, yet still cannot provide one insightful, business changing actionable item.

And the Number 1 reason Why Omniture is useless…..

1. Reports that look like speedometers:

Conclusion:

@Omniturecare Why is it, that whenever I have convinced any of these companies to install Google Analytics, every one of these 10 problems ceased to exist? IMHA (In My Humble Analysis), they were simply looking at reports and dashboards that kept them too busy making changes to improve their site….

What a concept. Analytics for insights?!!!

What are your thoughts on Omniture? If you like it or not, please provide a a reason and/or example. Now let’s bring it on

Web Analytics Tools: The Good, the bad and the Useless. Part 1 (The Omniture Effect) is a post from:

Digg.com announced today that the Diggbar was going to have some changes here. Should this appease publishers and Webmasters?

1. New treatment to the behavior of Digg short URLs. All anonymous users, on or off Digg will be taken directly to the publishers content via a permanent redirect (301), no toolbar, straight to the site. Logged in users that have not opted out will continue to see the DiggBar (200). These changes ensure that content providers receive full search engine ‘juice’ or credit for all links on and off Digg. They also ensure that Digg short URLs won’t appear in the indexes of any major search engines.

Since a 301 redirect is done through a server header response and for unlogged users, Google will follow the 301 redirects. Any links pointing to the Digg short URL should theoretically get lnkjuice “if” we believe that 301s transfer 100% of the linkjuice of a page over.

So far the change has not happened, but I will perform a Http Header check when they do this and update the post.

2. Because we want to ensure the best user experience, the DiggBar will soon only be shown to you when you are logged into Digg. While the vast majority of Digg users find the DiggBar valuable (only a very small number of users have disabled the feature or hit close with any frequency) we understand that many folks were confused when opting out. We want you to be able to have the option to permanently disable the DiggBar with ease. For registered Digg users receiving the bar, we are also making a few changes to make the process more obvious.

Transparency is good.

This new method still allows Digg to collect all the data mentioned in my previous post about the Diggbar when logged in users do not opt-out.

It’s a slippery slope to try and get consensus between publishers and Webmasters while trying to falsely inflate your pageviews with Quantcast . We’ll see what happens next!

EDIT (April 21st, 2009):

I performed a header server response on the digg short URLs. Header check is returning a proper 301 redirection. Additionally when not logged in, the links were direct (which is wat Search engine robots will see). Glad to see that Digg.com listened to Publisher’s and SEO’s “Valuable Feedback.”

Diggbar Modified due to “Valuable Feedback” From Publishers and the SEO Community is a post from:

Finally, someone has the balls to analyze the code in detail. Why Has Digg added this feature? Why does it benefit Digg? Does it pose a threat to Associated Press and other News publishers?

Welcome to the world of Adapt or Die Marketing, only here, will you will find the unmasking of the code behind the Digg toolbar.

To analyze this properly, we need to click on a link on any article submitted to Digg.com and we see the source code. Since the actual content on the page is in an i-frame, we will see Digg.com’s code where we are still under their watchful eye. All the html in the page still belongs to Digg: Meta Tags, body, and any tracking tags they choose to use. For our study, we will be using the following page: http://digg.com/d1oIkJ

The first thing we need to look at about the source code is the Meta information that Search Engines use to categorize the page in their index:

Meta Robots tag:

<meta name="robots" content="noindex"/>

The Meta Robots tag with a “noindex” description in the content attribute basically tells search engines that they should not include this page in the index. This is great for the publishers, as it will not allow the page to compete with the original in the Search engines results. Digg went to a lot of trouble to make sure they got this right with Google (as stated in their blog post), as they wanted to make as little a ruckus as possible. But today, with the Associated Press constantly attacking Google, Digg figures no one will pay attention to them.

Canonical Tag:

<link rel="canonical" href="http://www.baltimoresun.com/news/health/sns-health-breaking-habits,0,4763864.story"/>

This is Digg’s way of telling search engines “Hey Mr. Search engine Bot, the original source of the story contained in the link: http://digg.com/d1oIkJ belongs to this page: http://www.baltimoresun.com/news/health/sns-health-breaking-habits,0,4763864.story”.

The canonical tag was introduced a couple of months ago by Google and announced by Matt Cutts to allow webmasters control over duplicate pages their CMS may produce. This tag, tells search engine bots that the page is a duplicate of another page and supposedly gives the “link juice” of any links pointing to it towards the URL contained in the tag. The only issue here is that this only works within a domain, and not through cross-domains (See explanation here), so essentially were it not for the sole link on the nav bar pointing to the originating source, Digg.com could be considered a Blackhole website.

User Tracking

If you are logged in, once you are on that page, Digg will track your login username with this code:

<script type="text/javascript" charset="utf-8" src="http://digg.com/dynjs/loader/DynJsc9c5738b9b1e73c2a8a4b3887e6bdc6c">
</script>   <script type="text/javascript" charset="utf-8"> /* <![CDATA[ */
        var permaLink_url = '/health/Studies_Show_Bad_Habits_Can_Be_Broken_Not_Forgotten';
        aHash = "ac8392587bc94e8e71f6cc68d8b7f0b4";
        D.meta.page.type = "11";
        D.meta.user.loggedIn = "YOUR USER NAME HERE";
        D.analytics = new Object(); D.analytics.enabled = "1";
        // ]]>
    </script>

So that they can now track how many links you visit on the external site (notice how you can navigate the external site and the short Digg URL remais the same) while attributing that behavior directly to your user name. Why is this important? If you are a user that is more prone to click on “science news” then the Advertising presented to you on Digg could be targeted more specifically to you.

Digg is gathering information about your behavior on other sites to better target personalized ads by profiling your username.

Quantcast Tag

Quantcast is the internet advertising equivalent of the Television Nielsen Ratings. The more pageviews and visits your site has, the more your “channel” or website is appraised for advertising purposes. By adding this tag to all loaded pageviews that were previously external to Digg.com’s site, they now appear to have a lot more pageviews.

Conclusion: overnight, Digg.com will appear to have a significant amount of additional pageviews per visitor without having really done anything other than adding a toolbar.

When we consider that Digg’s popularity has dropped within the past 12 months:

We see that this quantcast tag could be their savior by artificially boosting their pageviews:

<!-- Start Quantcast tag -->
    <script type="text/javascript" src="http://edge.quantserve.com/quant.js"></script>
    <script type="text/javascript">_qacct="p-34Iv4_7kv_i-6";quantserve();</script>
    <noscript>
    <a href="http://www.quantcast.com/p-34Iv4_7kv_i-6" target="_blank"><img src="http://pixel.quantserve.com/pixel/p-34Iv4_7kv_i-6.gif" style="display: none" border="0" height="1" width="1" alt="Quantcast"/></a>
    </noscript>
    <!-- End Quantcast tag -->

Internal code for Digg use

Here we find a whole bunch of code that is the bread and butter for users of Digg. You can see comments, bury, share, digg etc without clicking the back button of your browser to vote the story up after having read it.

Now, and for the first time, Digg can actually measure if you were on the page long enough to read it before digging it up.

Can this help with their algo? Of course. Users digging a story after having been on the page for a mere 2 seconds may not count as a vote to get to the home page as much as a vote that saw the page for 1 minute and 32 seconds. This way Digg assures itself that the vote was more of an honest push than one that may have been solicited by a friend and is not really read by the voter.

Digg advertisement on toolbar?

While currently there is no advertisement on the toolbar, there is part of the code that looks like this:

<div id="sourceWhoop" class="whoop">
    <div class="t-contents">
        <div class="ad">advertisement2</div>

This has absolutely nothing to do with the publisher source code, and is included outside the publisher’s iframe. Does Digg.com have the intention of Advertising on the toolbar? This would be crucial in understanding Digg’s intent with the toolbar, seeing how they are blatantly using a third party website’s content, taking up valuable fold-over space on the publisher’s page while benefiting from advertising revenue? Sketchy indeed.

Their own tracking: Omniture code:

Digg’s Omniture code. Nothing is left behind, Digg is tracking all of your behavior on the external site.

What Else is There?
Other than the code, we should take into consideration that a large percentage of web users which want to share their favorite articles with friends will no longer link directly to the publisher’s links. Instead, they will be linking to the new short Digg URL, and seeing how the canonical tag does not work cross-domain, it will be denying publishers from well deserved link-love which would help the publisher sites appear higher in the Search Engine results for their own content.

Digg’s Intent with all this:

1. Instantly increase the internal traffic and Pageviews on your site which equals more Advertising Inventory.
2. Profile your users through their login username to better target ads.
3. Improve Digg algorithm by measuring time on external site (did the voter actually read the article?)
4. Keeping link love towards itself which increases the website’s authority in all the search engines.

What have you found in the Digg Toolbar code?

The Digg Toolbar Exposed; What’s in the code? is a post from: