Web Analytics Tools: The Good, the bad and the Useless. Part 1 (The Omniture Effect)
Ok, I’ve had it. Too many Multi-National companies that I have consulted for use Omniture. That “extremely expensive enterprise edition” of Web Analytics that only Fortune 500 companies can afford to pay for… and ironically, also afford to be held back by.
Omniture… my old friend, your old argument that I don’t quite understand your capabilities will not work. As a professional web analyst, and having consulted on some of the biggest North American websites, Oggy is calling you out on your bullshit.
Top 10 Reasons your Web Analytics packages are useless:
- The company’s IT department has to get involved and “brainstorm” how they are going to implement that “new company metric” instead of copy/pasting a code provided by a marketing geek on any page/link/action on the site.
- “Visits” are called “Searches” when they come from Search Engines, “Instances” when they come from Referral Sites and “Visits” when they are Direct Traffic. Oh no, “instances” are for Direct Traffic, no wait!…. sigh
- You can only segment data if marketing contacted their Omniture “account manager” and informed him what they want to segment. This has to be at least 3 weeks in advance so the “installation” of that “customization” can be applied. *Additional fees not included.
- Getting around the dashboards and navigation is difficult because the “analyst” in the Marketing Department has just come back from two a week vacation and the “new” 14th Beta release came out. “Did we mention that your code no longer works properly? *cough cough*, trust us it will do soooo much more when you update it.” (see #1)
- The Company’s CEO tells Marketing that he wants a new report, and when they say it’s not possible, he reminds them that they pay top dollar for their Web Analytics solution. If Marketing can’t produce it, it must be due to the department’s incompetence.
- The Company’s CEO is still telling the Marketing Department what the reports should look like?! Tsk, tsk….
- Marketing cannot access reports because an update to the Omniture servers is being done. Additionally, for the past week, the application has taken so long to load that Marketing pretended the printer was broken. Consequently, the IT guy wasted 2 hours to notice the printer was just fine.
- The company invested in 2 employees’ flight to Utah with a 2 week stay for an “installation course” (one guy from IT and one from Marketing). When they came back they knew what Evars and Sprops were, but forgot the company site’s KPIs.
- The tool costs more than the 2 full time employees that are trying to install that new, “complicated” metric the company wants, yet still cannot provide one insightful, business changing actionable item.
- Reports that look like speedometers:
And the Number 1 reason Why Omniture is useless…..
Conclusion:
@Omniturecare Why is it, that whenever I have convinced any of these companies to install Google Analytics, every one of these 10 problems ceased to exist? IMHA (In My Humble Analysis), they were simply looking at reports and dashboards that kept them too busy making changes to improve their site….
What a concept. Analytics for insights?!!!
What are your thoughts on Omniture? If you like it or not, please provide a a reason and/or example. Now let’s bring it on 
{ 40 comments... read them below or add one }
Leave a comment
Previous post: « Diggbar Modified due to “Valuable Feedback” From Publishers and the SEO Community
Next post: Google Analytics: Do You Have a Segmentation Problem? »
I’ve got to call BS on most of this post. Comparing SiteCatalyst to Google Analytics is like comparing Photoshop to Microsoft Paint. I don’t know about you, but the elementary capabilities of Google Analytics are in no way powerful enough to extract the complex visitor behavior information that is required to improve my sites.
@VaBeachKevin what complex visitor behavior would that be? Please comment on an example. I’d love to see.
Try segmenting engagement metrics such as: Time on site or average pageviews by referrering sites or referring keywords in Omniture… maybe you can show it to me “speedometer style”.
Omniture may not be perfect but the arguement that GA does it all or does enough is silly. Not every company needs to import multiple data sources or segment on custom variables but many do and many more should. However every site should be getting reports sent on their timeline and not what google defines as daily. By the same token variables should be customizable and that means ditching odometers or basic chart options as the user sees fit.
ga gets a good response because it’s simple and simple can be good but there’s a big differance between having the right reports without clutter and having just too few reports to get clutter from.
Omnitures testing tools are pretty complex too but I don’t think anyone says optimizer is better. Cheaper, sure. Easier too. Maybe even quicker. But not better.
Like I said there’s no doubt there’s flaws but that doesn’t mean ga is the solution.
@Ted S I never said GA was perfect. There is a lot which can be improved. (filtering statistical significant data for example, I will touch that in a later post). However, its “out-of-the-box” features are superior to 90% of Omniture installations out there.
GWO has more to play around with than 99.9% of websites out there need, and they can actually get to testing things because it’s a no-brainer. The concept of Adapt or Die applies here: While your one test is taking too long to implement with Omniture your competitor using GA has already done three.
Some of these are, IMO, untrue, and many of them imply that the web analysts and others working with the tool are not quite up to the job:
10) If your “marketing geek” is a bright guy and knows Javascript and can edit page code, s/he can in fact do what you said.
9) I agree with you on this one. Sometimes I think they are almost deliberately confusing in how they go about naming their terms.
7) Yes, SiteCatalyst sometimes updates their page code. So does Google Analytics (ga.js vs urchin.js) and pretty much every other WA vendor… and of course that is actually a good thing.
6) If Marketing has SiteCatalyst and DataWarehouse and still can’t produce the report, then most likely it is, in fact, because of their incompetence. If the report is literally impossible to create because e.g. the query itself is logically inconsistent, then it’s Marketing’s job to better educate the CEO.
5) Has nothing to do with the WA vendor… the CEO wants things how he wants them.
4) The 2nd item on this list I agree with… they need to work on reducing downtime
3) First, I think you mean “two days” not “two weeks.” (Actually, it’s 3 days for the IT guy though.) But secondly and most importantly, if your employees learn about s.props and evars but aren’t able to mentally make the leap that they are tools to collect the raw data which will eventually become KPIs (and figure out how to implement them as such) then the problem is not that Omniture is bad, it’s that your employees are dumb.
2) Again, if 2 full time employees are working on the tool and “cannot provide one insightful, business changing actionable item” I’d replace them with an employee or consultant who’s a bit more clued in… we may be hard to find but we’re still out there.
1) Glad I could find a third item on this list I agree with. I find spedometers pretty cheesy. (But sometimes they impress the CEO and keep him placated, so IU can’t complain too much.)
By the way… I would agree with you that GA is much more useful “out of the box” than is SiteCatalyst. That’s because SiteCatalyst is meant to be configured and thought out during it’s deployment.
If a company lacks the intelligence or motivation to configure and deploy SiteCatalyst properly, then certainly they would be better off with GA.
@Chris Thanks for your comment! I agree that some of these points are more due to the employees incompetence. Its just that from experience I have seen that people using a tool like GA learn so much quicker and hit the ground running, while people exposed to Omniture have more of a difficult time learning how to go about getting insights and usually miss the point.
@Oggy
Yeah, I would agree with you on that.
To me, it’s like people who buy a fancy SLR camera, but don’t learn how to use it and only ever use the “Auto” mode- and then complain about the camera when their pictures don’t look as good as the “pro’s.” They’d be much better off buying a point-and-shoot.
I thought this list was a joke at first. All this proves is that you’re not fully utilizing Omniture correctly. There’s no way Google Analytics is superior, but it is easier to use as it’s meant for the masses. My advice would be to take some time to read the documentation or hire someone with Omniture experience.
LOL! Funny post. I can’t really comment because I am not using Omniture but another “paid” solution – which is giving us satisfaction even if far from being perfect. GA is a great tool, very simple and covering a lot of needs. Still it does not offer (yet) many advanced features that we (and other large entreprises) are using today from our ‘paid’ solution.
But the day may come…
Cheers,
Michael
@Jon Your attitude is the main reason why the Web Analytics community does not stand up enough against these types of tools. The old “If you think Omniture is bad then you simply don’t understand it” intimidation tactic no longer works with me. I have been using it for over 5 years and I know it in and out….
Show me just ONE report or dashboard from Omniture that will knock my socks off… so far no one has given me any reason to change my mind.
99% of all the sites I work on are classified advertising sites. At a high level they include a home page, search results pages, and listing pages. I need a tool that can tell me for visitors that are located in Texas, how many made it to the site from a Yahoo paid campaign, where the keyword was greater than $1.50, where it was at least their 3rd visit, and they did an advanced search for items that were over $150k on the site. And then did they completed conversions 1 or 2, but not 3 or 4. And out of those completed conversions, how many of those listings had 9 or more photos, how many included a matching print ad, and how many were the bronze, silver or gold package, and how many were from one of 6 different dealers.
This is an actual business question I have had, not just something I made up to throw at you. I get 5-6 questions like this every day, with all different variables. The 4 custom variables you get with Google Analytics is no way enough for me. I have even tapped out the 50+50 evars and props on some sites.
The sites I deal with get 10 million+ page views per day, so I have the volume that can handle that very fine granularity. Now don’t get me wrong, I’m not a Google Analytics hater. I actually use them on personal sites. Google Analytics has come a long way and has become a really impressive tool. Its just not for everyone and it can not handle the questions I would throw at it for the corporate sites I manage. The extra features like the SAINT classifications, data insertion API and Excel client just make my day that much easier.
@VABeachKevin You make an excellent point. However installing these segments probably took you as long as a complete redesign of these large sites. Advance Segments in Google Analytics can accomplish over half of what you mention Out-of-the-Box with almost 0 time spent on technical installations.
GA’s ability to drill down (Which Omniture can only give you through Excel Client as it would be too demanding to their servers if done through the platform) gives you insights you may have never even thought of looking for. Omniture reports look 2 dimensional and do not give you any flexibility unless you thought “beforehand” what it was you may want to segment later (see point 8).
I’m not saying Omniture offers 0 insights, but 90% of Web Analysts need to drill down on a whim and can’t wait 3-4 weeks installation time before they can access their data… Meanwhile their competition is way ahead.
Oggy, your final point nearly cost me a new keyboard, I wasn’t sure I’d be able to get all the coffee out from between the keys. But that is my on big complaint about Omniture, the quality of the dashboards. Everything else you listed have pretty much been non-issues for us, especially when viewed in light of the segmentation by custom variables, custom success measurements, and for us, we use actually utilize the “real time” reporting on occasion.
It really seems that the majority of your complaints are rooted in the incompetence of wa staff (like others have pointed out already) and in a severe lack of planning/research on the part of executives/marketing/IT whoever is involved with selecting the wa tool and getting it implemented. Omniture is not for everyone, and probably a large segment of their client list really has no need to use Omniture over GA
Site Catalyst is incredibly complex (arguably too complex) but that complexity comes from abilities to customize and segment (granted using Discover) not available with GA. GA’s elegant simplicity (and I do thing GA is elegant in its simplicity) is itself an advantage over Omniture, as long as you don’t need have robust customization.
I do agree with assessment of learning curves. I wouldn’t wish Omniture on anyone just learning about wa except for very select personalities or learning styles. And that’s not to say that GA is basically training wheels, GA is a much better teacher of wa than Site Catalyst is, and there is a lot of power that can be tapped. But once one knows what’s going on, Omniture does provide a lot more horse power.
And our implementation of Omniture took far less time that our current site redesign is taking.
@Tim This post started off with a Rant, but I am happy to see how many intelligent comments have been shared. I think its very important to “analyze” the Web Analytics solutions very closely according to the capacity and qualification of the staff in a company. Many companies which existed before the web still don’t understand it. They have difficulty asking their IT staff to add Title Tags and Meta Descriptions… And we’re not talking about mom and pops shops here. We’re talking about the “Coca-Cola” type brands everyone knows about.
My BIGGEST issue with Omniture is the fact that large companies use it simply because its marketed as a Powerful tool, and expensive. That changes the perception of a product. In reality it can only be as powerful as the people setting it up (which they do not hire). And 99.9% of the time SiteCatalyst platform is not worth the money, time and effort for the final insights you will get.
@Oggy you may be right, but is that Omniture’s fault? It’s like me buying a set of Ping clubs, I have a hard time playing a scratch round on the local junior course (and that’s from the red tees). I have no business spending that kind of money on clubs, unless I’m going to invest enough in my game to make it worth it.
Or might that actually be a good thing for the industry? Similar to Automakers and race/sports cars; the technology developed for high performance vehicles that few every use to their potential (but boy it’s nice to know you could pull 1.5 g’s through a corner and accelerate out of it still with all that torque) eventually filters down pragmatically to “consumer level” vehicles. Features people with too much money went hunting for 15 years ago are now standard on a Kia, and we’re all better for it. But if those high performance vehicles weren’t marketed so that everyone wanted one, we could still be driving chevettes and no wiser that a better alternative existed.
But I can certainly empathize with your frustration looking over the landscape of frustrated analysts and executives, and I do appreciate your candor.
As I expected, there are more people on the defensive than there are otherwise, at least publicly. This is normal because so many people’s livelihoods depend on Large companies using Omniture. I thought it interesting to post and show how much secrecy needs to be involved whenever we do not agree with a product’s design/features. Here is an Email I received (printed with permission from the author as long as I promised anonymity):
I can’t comment publicly on your Omniture post, but off-the-record I want to say
Amen.
There may be issues with incompetent web analytics staff, but I would say that
in many cases, a company’s resources are better spent on a good analyst with GA
than a mediocre analyst with SC.
.
Agree on all points. Although I’m disappointed you didn’t go into the whole “Omniture Suite” vs “Core Competency” issues users face daily because Omniture WEB ANALYTICS is trying to be all things to everyone. I’ve been sitting here looking at a spinning circle while my report is building for an hour now.
Don’t even get me started on SearchCenter.
BRAVO OGGY!
Bravo!
There you will find a link and a spinoff around the GA/Omniture tensions.
Entertaining post. I think that you and TIm are spot-on in your discussion. From time to time, I get involved in projects where I (and my clients) would really benefit from the more complex feature set that SiteCatalyst provides. Of course, they would also benefit from having other web analysis resources that go along with a SiteCatalyst implementation (staff, training, etc.). To be successful with a tool like SiteCatalyst, the organization needs to be fully invested in the tool and committed to drawing value from it.
But that isn’t why I commented… the gauge chart bit hits way too close to home for me. I spent years working as a lab assistant in a Human-Computer interaction lab in college and one of the bigger projects that I worked on was for data visualization, and in particular, gauge charts. Then (over 10 years ago) and now, all empirical evidence points to the ineffectiveness of gauge charts at delivering information. So why are they still so commonplace? I blame marketers (to a lesser extent web analysts) for continuing to use them in reports. I wrote a little bit on them here:
http://www.tomsanalytics.com/2008/07/phase-out-gauge-charts-for-clarity/
while not necessarily calling BS on this post, I think its unfair to position them against much less feature rich freeware like google analytics. anyone who as ridden the albatross that is webtrends knows how much slicker and easier omniture is to use, and to leverage for clients.
we as an agency license omniture, then resell to our clients. This actually is quite nice because the individual clients can be small and still get a lot of value out of omniture, while the big clients get what they need as well. I actually think Omniture is a little better as an agency’s tool than as something sold direct to a single client.
as an agency we’ve been extremely happy with Omniture. We do run GA simultaneously for some clients as an audit tool, but definitely get more mileage out of Omniture.
And sadly, CEOs do NEED their speedometers.
This post actually rings so TRUE to me. After struggling w/ a dev team attempting to implement Omniture SiteCatalyst for what should be a very basic beginning implementation I’m about to go crazy. It took hours and hours of conf calls w/ our portal partner and the OM implementation consultant to just get everyone talking the same language. I sat through 2 days of training and still ended up confused by the terminology Omniture uses to try and describe data and metrics.
Yes, GA is more basic than SiteCatalyst and I’m thinking that we’d be better off ditching SiteCatalyst for GA. When our annual renewal comes around and we’re staring at a hefty invoice, i think our sr execs may consider it.
And I have found Omniture difficult to do business with. Several phone calls to the acct mgr and hand off to the ‘partner’ that we originally signed through, 3 weeks later, and I was still having problems purchasing Omniture business consulting hours. And their hours are EXPENSIVE. So far high priced consultant hours haven’t impressed me at all.
Excellent post. As a web development manager I can’t stand Omniture. While GA is sometimes overly simplistic, Omniture is the complete opposite. We have had to sit in meeting after meeting about how to setup some simple metric. I’ve even been in meetings where the omniture rep themselves are confused if something should be stored in an “eVar or sProp”. And whenever you have an issue they act as if you are an idiot who implemented it wrong, and not their product that no one can figure out. The whole business is built in an archaic software model of charging for everything, over complicating the tool, and then charging for training, support, etc…
@Bill
I’ve even been in meetings where the Omniture rep themselves are confused if something should be stored in an “eVar or sProp”.
Nuff said!
It seems as though the web analytics space is quickly following the same old recipe:
- big guy comes in, owns the space, and charges unreasonable prices (see Omniture)
- new guy comes in, offers nearly identical functionality, gives it away for free (see Google)
- upstarts come in, give everything that free guy doesn’t include and big guy doesn’t feel the need to (see us!)
Companies need actionable value from their web data, and they need it now. Segmentations. High-value content. Accurate recommendations.
The value needs to be uncovered from within the data. Just logging the data doesn’t add value. In fact, it’s valueless. Thanks Google Analytics!
Here is another comment I recieved by email. Another person who wished to remain anonymous and agreed to publish their views as long as I edited anything which would give his/her identity away.
Read your post and the comments. Pretty interesting. Although I do not necessarily agree with everything here is my opinion as a user of both platforms
1- Omniture is obviously the fully equipped BMW series 7. The one with the computer which is so complicated you need almost a full time person dedicated to it. Some companies probably need to have Site Catalyst to track their client programs. I know Canoë is using it with satisfaction… but my understanding it that it took a lot of effort to come to that.
2- Some organizations do not want to use GA. We even had one client who specifically mentioned that for one of our infosites (his issue was using a third party doing reporting). It also disqualified Omniture. We had to stick to WebTrends.
3- As a two member team I do not have the luxury to be working full time on Omniture. That’s where I believe Omniture falls short and where GA has value.
4- Omniture is not necessarily an option for me. It’s the solution adopted by My Company even though I have basically three needs:
a .Member recruitment
b. Member behavior on the site
c.Program reporting (which is the most complex and is the reason I’m going on Omniture… Although I believe Google could do it in an easier way)
5- Honestly I would stick to Google (and I’ll stick to it even after having the full Omniture working).
I’m loving the debate on this topic. Web Analytics is such a small part of the complete customer experience and has become such a huge issue. These 5 cent anonymous interactions end up costing companies millions of dollars annually because of the resources it takes to maintain such complex systems and because of the (much) more expensive interactions they create down the line. 5 cent web visit becomes a 50 cent IVR call becomes a $5-15 live agent call (multiplied by the number of repeats).
When does part 2 come out? Can’t wait to read it. Great stuff!
I think with Google Analytics, and Yahoo Web Analytics both provided free of charge, you have all you need. The new currency is called data, and if you aren’t willing to pay that (Some Fortune 500′s are afraid of Google in terms of data) – then you have to pay money. Loads of them. And IMHO Omniture has sit back and watched the world go by…
And the number one reason not to use Google Analytics?
Because then Google has your data, and your customer’s information is stored in US databases. For EU companies with tough legal departments this is a problem.
@Louise I don’t buy that argument. Google already has more information on every website out there. How do they gather data?
1. Google Toolbar
2. Something crazy like 80% of web traffic goes through Google (not exact % please correct me if you know the exact estimate). But they know how much traffic they refer to you.
3. Other 3rd party vendors like Omniture also collect data, its not in your servers. Omniture is definitely not a solution for this problem.
If this is an issue though, there are internal tools like Urchin which keep all data within your company… I think the licences are peanuts compared to what
Omniture costs.
GA can’t be a real pro tool. No access to data; no way to valudate results,
Switch to webtrends software version and read Stephen Few
I’m half joking; can’t believe we’re having a tool debate again..
To be fair though, GA easiness when it comes to cross-tab stuff on the fly seriously challenges the paid solutions. And GA provides very basic info, such as bounce rate, all over, and one wonders why we still have to manipulate those expensive products to get it (!).
Love it! Yes you can do more custom variables with SiteCatalyst. Big whoop, most clients I have worked with (and big ones I might add), are barely starting to think that maybe they should consider something beyond page views as a metric.
IMHO, Omniture is fantastic for sophistcated companies with dedicated resources and highly segmented markets (aka, 0.1% of all companies), for the rest of them, there’s GA.
For me, the real pro of GA is that it’s easy to setup. Want to try a new segmentation? Takes… about 1 minute? While Omniture is more powerful, GA is so easy to use and to customise without having to employ a consultant and wait weeks, that it encourages expirementation. I’ve come up with a few take to the bank actionable insights through GA simply by playing around with segmentation and seeing what happens. With infinite resources AND highly skilled competent people (and let’s face it, that’s a VERY rare combination) then Omniture is better.
A lot of companies just go straight to the biggest and most expensive tool without evaluating their actual needs, and level human resources required to support it. They ended up getting less value from using a more expensive tool.
Interesting, i agree with some points that’s why i use google analytics, but some clients do have the staff resources and training to make full use of it. Ive spoken in depth with Omniture on the issue, but they advise clients that they are not entry level web analytics solution for mum&pop shops.
They have some very advanced features and google analytics still has some limitations, but it is a the case that you cant do stuff on the cheap with Omniture, but what ever you need they should be able to track/test/report.
Oggy – I’ve been in Web Analytics for half of my 20′s and all of my 30′s and honestly the items you listed just don’t seem very hard to overcome for me. I’ve been able to overcome all of these “challenges” fairly easily including the dreaded “IT Involved” nightmare that you obviously loose sleep over.
Like this one… if you don’t like speedometer style reports… don’t use one!
I’m not an Omniture defender nor a GA hater and I have have been responsible for Web Analtyics programs that have utilized a number of the tools out there (SiteCatalyst, HBX (old school), GA & the analytics tool formally know as Visual Sciences). There are pros and cons with all of these tools but if I had to point to the greatest weakness with Omniture it wouldn’t be so much with the tools capabilities or limitations but rather with how all consulting from them is “exta”. But then again if the company that I’m working with today had been “set up” so well my Omniture when they first signed up; I probably wouldn’t be here managing their web analytics program.
I agree with a lot of these statements. But when it comes to GA I think Omniture uses the “Fox in the Hen House” analogy. Why let Google have data about your site? Sure they say they will not use it to modify SEM bids based on success on your site, but if revenues are suffering what’s to stop them from doing it? The “Don’t be Evil” motto?
Check out my own post about my experience at Omniture. I loved working there, but I was ready to move on.
Six months since I left Omniture
[...] Useless? Maybe not. But with Google Analytics constantly releasing new features Omniture is doomed to stay isolated as a niche player only appropriate to companies with full dedicated web analytics teams composed of both analysts and programmers. [...]
After 5 full months with Omniture I have to say I agree with you. It feels like a cross between an 800 hundred pound gorilla and an 800 pounds sloth. It’s uber powerful but moves so slowly, it’s maddening.
I wanted to participate in my own way to the debate so I wrote a little story titled: Tales from the Click: Bounce Rate in Omniture SiteCatalyst