If I were to create a statistic called “Talk of the Town”, and with it try to gauge which restaurants were in the news, I think most folks would have a good instinctive idea what such a statistic would measure. It shouldn’t be limited to a particular forum. It shouldn’t be limited to a particular group. It should have some sense of who is talking about what, and how wide spread that conversation actually extends.
Urbanspoon provides such a statistic, and it’s called Talk of the Town. In general, they weight contributions by mainstream media, alternative media, and to some extent, their own top 10 bloggers in creating this list. I’m in the top 5 bloggers as ranked by Urban Spoon, and I noted one day that if I reviewed a restaurant, and another top 5 blogger reviewed a restaurant within a few days, that restaurant would end up in the bottom half of the top 10 for a week or so. It was something I noted. It didn’t really disturb me. The metric, as it is currently implemented, is however notably imperfect.
There is an ‘elite’ status that Urbanspoon can confer to an active user of the system, and they are called Primes. Some primes are well known bloggers. Marie Let’s Eat, Foodie Buddha, Chow Down Atlanta and your truly all are Primes. Some Primes, and some of the best Primes, have no personal blog and yet wield, in my mind, considerable influence on the Atlanta food community. Barney (who also has a handle on 285 Foodies) comes to mind.
Over the past months, there has been a push within Urbanspoon, by a certain set of Primes, to decouple US’s Talk of the Town stat from any meaningful connection with any influence outside of Urban Spoon and make it dependent purely on what Primes think. This is one proposal. Another is to make it dependent on what “Top Contributors” think. Understand that US is a web and phone application whose parent company is small, with few employees. They depend heavily on Prime contributions to do dirty work for them and keep their database fresh. If a particularly important subset of Primes push hard enough, and no one speaks up, things will change, and not necessarily for the better.
The subset pushing for decoupling is noted for another couple peculiarities. They are envious of the mainstream media, and they are jealous of bloggers. Every time they open their mouths, bloggers like me are depicted as giving a less than sincere or “genuine” contribution to Urban Spoon. I’ve seen a ton of blogger versus Prime debates, and this group of Primes disappoint me routinely with their myopia. The notion that a major fundamental difference between Yelp and Urbanspoon might be US’s investment in its blogger community does not occur to them. The notion that Yelp might be more read than Urbanspoon – much less TV or print media – never occurs to them either. And it’s exactly this subset leading the charge to remove any “external” influence from the Talk of the Town stat.
Metrics at times don’t entirely encapsulate the influence of certain people on the food scene. Take Mr Jones of Eat Buford Highway as an example. Take Sean, of Take Thou Food, as another example. These folks have, on occasion, turned the whole of the food community’s opinion on the reputation of various eateries, and yet are not to be found on Urbanspoon’s Prime list, nor among their top 10 Atlanta bloggers (Sean, though, is still the top ranked Athens GA blogger). That said, thinking about how to measure their contributions leads to the abstract notion of reputation.
So what is a reputation? How do you measure it? Why are we concerned at all? It’s because if we’re in the shoes of a small web company with limited resources, we really can’t measure a Talk of the Town. We don’t have the resources to, say, interview millions of people continually. We, at best, have to estimate it. And if we can estimate reputation, and find a subset of folks who have plenty of it, then the “Talk Score” for a restaurant becomes, in pseudocode:
Score = Sum(Mention(i)*Reputation(i)/(Age_Of_Mention + 1 ))
So, who should be counted? How do we measure their influence? But simply put, about the worst solution I can think of is to total a few select members of the US community. Questions that come to my mind are: why should Primes or Top Contributors have any more say than a regular US member, and if the answer is no, don’t we already have the results of what they seek in the regular Urbanspoon restaurant ratings? If not in the rating itself, just look at new restaurants with high approval rankings (< 3 months old, circa 20 votes, over 90% positive).
What’s wrong with a US statistic that is what it claims to be? Call it Prime Picks, or Top Contributors Recommend? That’s the appropriate forum for the “elite” users. It’s otherwise a travesty to call such a stat Talk of the Town, because the notion that I’m paying as much attention to my local Top Contributors* as much as mainstream media in town, is asinine.
Now, much of my points of view come from someone living in a city with a large number of Urbanspoon users, a large blogger community, a large and active collection of media talking about food. In communities served by one or two newspapers with an indifferent attitude towards food (I’m thinking about Shreveport LA, near where my dad lives), an extended definition of “Talk of the Town” may create a metric that serves the community better than the current implementation. But a *cough* Talk of the Town *cough* metric that basically counts only what Primes think is in my mind, exclusive, degenerate, too reminiscent of the top 100 list already, and in many way, overly empowers the vote of those Primes.
* One problem is that it’s too easy to become a Top Contributor. I can become one by making 10 reviews, taking 40 photos of every restaurant I review, and uploading each photo into Urban Spoon. This could be done in the span of a week, and suddenly, I’m a “Top Contributor”.
Update: fixed the formula to add an age component. Older mentions should eventually disappear.
September 16, 2012 at 8:19 pm
That thread got stranger and stupider once I resurrected the fool thing, didn’t it? But I’m a big boy, and should have learned by now that speaking about that weird hiccup / loophole that I found was going to result in a lot of finger-pointing. Nobody to blame but myself, really, but it gave me an idea for a truly amusing prank that I’ll have to write about.
For fear of sending additional unkind vibes to that community, I should probably leave it at that. Except to say that Sean’s ironclad hold on the # 1 spot in Athens has been an irritation of epic proportions for a really long time, and that Marie and I should definitely go to town again and do something about it.
September 16, 2012 at 9:21 pm
There is no prank that’s going to solve the enmity unleashed in that thread. It may make you feel better but I don’t think anyone else cares as of the moment.
There are a fair amount of politics on Urban Spoon. A lot of it is pretty banal and trivial. ToTT is probably in that category, but I’d hate to see it ruined. And I really don’t know if Stinky is going to stop throwing hate bombs long enough for me to emotionally disengage.
I feel pretty beat up by the “Primes rule, media and bloggers drool” faction. Normally, Stinky is a pretty nice guy. In this thread he’s way over the top.
FnS.
September 18, 2012 at 1:33 am
From what I gather it’s certain Primes that want to “see” their influence. They have their knickers in a knot because their “say” can’t be “seen” on the Talk of the Town. It’s just critics and bloggers.
(It’s the same Primes that want to the reviews at the top of the page instead of critc and blog posts. This is a whole other can of worms…)
If US is to include the Primes in ToTT then they would have to include ALL users of US. Prime or not. Otherwise, in those small areas, with no/few bloggers and critics, the same problem that they are whinging about now will continue. The only difference is it will be the local Prime that controls the ToTT. And you know there won’t be a thread boo-hooing a ToTT that is controlled by a few Primes 😉
September 18, 2012 at 12:28 pm
Thing is, US already has a stat that includes everybody. It’s their most important and best stat, and that’s the local top 100 list. Every Prime, every user, every Top Contributor gets one vote, and its the best, most honest, most statistically reliable stat that comes out of US. Duh.
For some, though, it’s become an emotional issue. We’re Primes, we work so hard, how can you deny us? But though they are crucial for making the content better, and deserve every commendation for doing that, fixing the restaurant entries of 400 restaurants doesn’t automagically make you a go-to food critic.
If you want a “Primes Picks” that’s cool. Barnes and Noble lets their staff recommend books, and often those books are better than the current NY Times Top 100 (because, in part, they’re picking from centuries worth of books). They’ll know things the Average Joe won’t, the cool little spot not quite in the spotlight. So let them provide that.
Deal is, something called Talk of the Town should be that, Talk of the Town, not Talk of a Couple Primes and Their Buddies.
Having said all that, in some neighborhoods, where the media are dead and the bloggers nonexistent, perhaps ToTT should simply not be shown. Another approach is the one you suggest: The ToTT algorithm needs to degrade gracefully in the absence of media and blogger info, and if under those circumstances, the fallback is recent activity on US itself, I can see that as a solution in relatively unpopulated areas. But weight everybody, not just Primes.
In big cities, however, media should dominate. Sorry, that’s just the easiest way to get an inexpensive, easy to calculate take on what people are actually talking about.
FnS.