There’s been a lot of noise surrounding the 50/50, and we know the Spend Matters client services team have received a number of enquiries from companies who felt they were entitled to make the list, or make the “to know” list, but didn’t, based on their customer status, and even Jason, the founder of the list, has gone on record as to why he hates the list in “3 reasons i hate the spend matters 5050”. Peter (in “spend matters 50 to know and 50 to watch questions and answers”) and Taras (in “why good things come in threes”) have chimed in too. Now the doctor is chiming in.
Let’s begin by reiterating the title. Generally speaking, the doctor despises lists. First of all, most lists come from the analyst firms that release the tragic quadrant procurement grave reports which, as we all know, change the requirements for consideration, and grading, every few years. (This often has the benefit of increasing the rank of some companies and decreasing the rank of others in an often arbitrary fashion. This is why the doctor is working with the maverick and the prophet to define standard requirements for different solution types so that vendors can be graded equally and fairly against a common, consistent, benchmark.)
Secondly, most are subjective lists that tend to represent the views of just one or two analysts, often heavily influenced by a small number of vendors that they spend the majority of their external interaction time with. While this doesn’t mean that they will be anymore biased to these vendors as opposed to others when doing their rankings, their view of what a product should, and should not, do are heavily influenced by these vendors and, thus, the rankings of these vendors are always good.
Third, the lists are usually limited to sourcing, or procurement, or SRM and not broad enough to identify related, emerging, complementary technology that can prove just as useful to an innovative firm. There always comes a point where the same-old, same-old fails to add value.
Fourth, any lists that stops at Vendor #X does not include Vendor #X+1, which may be just as valuable to a (slightly) smaller group of potential clients and, more importantly, may actually do more to warrant watching in the months that follow than Vendor #X, that might become complacent given their recent ranking.
Fifth, as pointed out by Jason, there are always going to be accusations by vendors not included, third parties with their own agendas, and even by vendors included (but not ranked where they feel they should be).
Sixth, and not least, no list is perfect. At any given time there are vendors the analysts are unaware of that might deserve a spot on the list, there are vendors on the list that might not be keeping up the innovation, and many of the services oriented vendors have to be subjectively ranked on limited customer interviews.
But that doesn’t mean that a properly constructed list cannot be useful. A well constructed list, that is objective as possible, can open one’s mind to options one might not have known of but should consider. A well constructed list can help vendors realize how well they are known and what they are known for and where they need to spend more effort on education and marketing. And it gives vendors something to strive for, so long as that list is created equally each time it is created.
While the list is not perfect, this is the first list the doctor is aware of that, while subjective, was created in a fashion that was as objective as a subjective list can be. It was debated over by seven analysts across three continents — the revolutionary, the civil crusader, the money, the public defender, the maverick, the doctor, and the prophet — who covered a variety of areas including, but not limited to, S2C, P2P, SRM, Analytics, Services, Risk, and Finance, and who had very strong opinions on who should and should not be considered. And client status played no part whatsoever. As the prophet said, about half were Spend Matters present or past clients, half were not. With respect to the recommendations for the list from the doctor, the split was about the same.
Basically, when all was said and done, to make the to know list, at least three of the analysts had to agree (and more than that to be guaranteed a spot), and to make the to watch list, at least two (and three to make the list with certainty), and they had to persuade the profit that their choice was better than another choice that had the support of two analysts. While each of us can point to a handful of vendors and say we would have liked to seen them on a list, the fact of the matter is that if only one analyst sees a vendor as worthy, that’s a singular subjective view point. When at least three people agree, especially when they cover different sub-sectors, in different parts of the globe, that’s a much stronger statement than just analyst X likes vendor Y (especially when, as pointed out in our recent post on 30K a Day and You Haven’t Even Seen the Solution, any company that did not demo at least one analyst was not considered). Is it perfect? No. But is it better than everything else? Yes. And it’s going to get better still. As more types of applications and services pop-up, the prophet is going to add more experts to the pool. the doctor suspects next year’s list will be argued over by nine analysts, which will make the results stronger still.