June 15th 2009 | Andrew Edwards

Advertising agencies—and especially digital agencies—are among the most fascinating companies in the world. They are staffed (most of the time) by whip-smart, sometimes arrogant, fun, creative, young, attractive people.

Doesn't it always seem like the good-looking smarty-pants you knew in high school ended up working at an agency? Sometimes, maybe when you're slogging through yet another column of analytics numbers or trying to figure out how to make your analytics tool recognize a measure it wasn't born to measure—admit it—you get a little jealous. Those agency people are probably skateboarding down the hall right now, coming up with a new way to make oven-cleaner cooler than Lady Gaga.

Agencies are soaking in the suds of what everybody now calls "spin." You know—lemons become lemonade. Pigs can fly. And ho-hum conversion numbers can be made to seem either "not that relevant" or hidden behind a pitch for new creative. Agencies are amazingly good at taking things that are not so wonderful and making them seem like just the thing you wanted.

All of which is precisely why you can't let your agency perform your analytics work.

What do you want most out of your hard numbers? You know—like the ones you get from the bank? Or the ones you get from your sales database? If you're like me, there are a couple of things: accuracy of course, but also you need to trust that they are not getting "spun" for you by somebody who's got a vested interest in making them look better than they have a right.

You can see where this is heading.

Let's get to the "accuracy" part first, because that one is easier to handle. Like I said before, agencies and their digital counterparts are creative shops. Only in recent years have they attempted to "do" technology—mostly creating web sites. Which, as any real geek knows, is definitely technology-lite compared with crunching application code or running a massive network in real time—or getting analytics to work the way it's supposed to.

Agencies have done a great job building sites. Nearly all the major sites today have been built by agencies. And agencies are chock-a-block with experts in Flash, Photoshop, HTML, XML and all the other "developer" skills that go into building award-winning sites.

What they lack is the really deep technology DNA that drives careful, accurate measurement. More and more agencies these days have talented web analytics people on staff, but they are often talkers and aggregators, not coders and configuration experts. And with few exceptions, agencies are certainly not devoting as much effort on ensuring accuracy in the numbers as they are on the creative for that next banner campaign.

Agencies will tell you, quite justifiably, that you go to them for "best of breed" expertise when it comes to creating awareness for your brand. But too often, those that attempt to bundle analytics with their creative projects are giving you "best of breed" creative with lukewarm analytics skills. Now I'm not saying agencies are hopeless when it comes to analytics skills. But best-of-breed? I think not. And if you're in the market for the best (as you should be), the technology alone should send you looking in other directions than your agency for analytics.

The other problem comes down to the mind-set of agencies. You know, that whole "spin" thing. Here's where I go back to my title: fox in the henhouse. Here's how I recommend looking at it: if you knew your bank had a vested interest in showing you that you had more money in the bank than you really had, would you give them your business? Of course not.

Agencies often—and I emphasize often rather than always—sublimate the facts generated by analytics to the message that the account team wants to portray to the customer (that's you). They will create some very pretty charts and graphs—prettier, probably, than your analytics tool makes on its own and prettier than a web analytics consultant might make—but in many cases the charts and graphs are built to tell a story that fails to be objective. And they can't really help it.

Can most agencies really afford to tell you that the site you just spent a million dollars on is tanking? That the old, clunky one was better at conversions than the new, Flash-based one that takes forever to load and where everybody gets lost because the clicks don't quite get to the server in time before they click again and cause havoc? The answer is no, in most cases, they cannot.

What's the remedy? Let the agencies keep doing what they're great at: building your brand online.

But think hard before you let them do the measurement too. Find an objective, third-party source for measurement—and the interpretation of the measurement. Then, schedule a meeting with your agency and either reward them for creating an amazingly effective site; or chew them out for creating that bloated, Flash-site where everybody gets lost.

In either case, you'll know you've got accuracy and objectivity on your side.