Transcript for UCStrategies Experts Discuss Nemertes PilotHouse Awards 2012

Michael Finneran: Good day, everyone. This is Michael Finneran. I'm here with some of the members of the UCStrategies team and the topic for our podcast today is the Nemertes PilotHouse Awards.

Nemertes does an interesting award system based totally on user surveys. I do similar surveys with Information Week and believe me, it is tougher than it appears. Quite a piece of work by Nemertes, given the phenomenal number of categories in which they surveyed. With the UC experts today, what we would like to do is give some of our views and things that we agree with, things that we were surprised by. Of course, there are only two here really that impacted me, Mobile Device Management, where they did not have a top provider, but the second market challenger was AirWatch, who indeed, is one of the better suppliers. Of course, there are quite a few others out there including MobileIron, Good Technology, Fiberlink, even Blackberry. My own surprise was that they could not find a market leader.

In wireless LANs, they came up with quite a surprise. Hewlett-Packard was the market leader and the challenger was NetGear. They are not big players in the wireless LAN market. Certainly not to the scale of Cisco, Aruba, Motorola or some of the smaller, more nimble players like Meraki or Aerohive. But again, that’s what they heard from their end users, and you have to go with that.

I know Dave Michels, you had some feelings on this as well.

Dave Michels (1:27): Thanks, Michael, yes. I think it's nice when the analyst firms publish reports like this and put a stake in the ground on their opinions. Of course, it’s always controversial because people have differing opinions. But I would like to applaud Nemertes for putting this out. I don’t have access to their detailed report so I am only looking at the press release and just a few things jump out at me.

One is I think it is interesting that the PilotHouse awards are 100 percent based on customer surveys. Gartner's Magic Quadrant, for example, is 100 percent based on Gartner's analysis. I do not even know if they could consider customer feedback. Ultimately, the customer is always right over the analyst. But, the limitation of concern that I have is customers do not necessarily have basis to compare like the analysts do. We meet with all the vendors and really compare their products side by side.

I think of how content I was with United Airlines until I started flying Southwest. I feel that it’s sometimes misleading to ask customers who don’t really have a choice; they made a long-term investment and all they could really evaluate is what they have.

Along those lines, a couple other things jumped out at me. They awarded quite a few categories. I’m not even sure how many. It looks like close to 20. The ones that I’m concerned about or know about are IP, Contact Centers, IP Telephony, Managed IP Telephony (kind of surprised me), Unified Communications and Video Conferencing. Looking at those, I wondered what the difference was between IP telephony and unified communications in their criteria. For example, in Unified Communications they did not include ShoreTel, Siemens Enterprise Communications, or NEC. Their logic for not including these companies is that they didn’t have – based on their surveys – they did not have enough feedback on those companies to include them. But then, for example, ShoreTel is included in IP Telephony, and I know ShoreTel considers themselves a UC company. So I am wondering how they draw these lines... and it’s not answered in the press release, put it that way.

It’s interesting also because – I think it was last year in the UC – Siemens was one of the companies that was recognized and this year they were not even included. I think ShoreTel has won the IP Telephony category in the past and they weren’t recognized as either a leader or a challenger. No explanation as to what could have made that impact or change; at least, not in the press release. Maybe it’s in the detailed report.

The other thing that stood out to me was Radvision was on here recognized as a video conferencing market challenger. They’re not treating Avaya and Radvision as the same company, which they now are. I am wondering if that changes anything in terms of this. That’s about all I have on it right now. I am curious to hear what some of the other experts have to say.

Michael Finneran (4:19): Thanks, Dave. It could also be a matter of timing that when they did the survey Radvision was still an independent firm.

Dave Michels: Yes.

Michael Finneran: Phil Edholm, you had some close looks at this.

Phil Edholm (4:29): Yes. Absolutely. I think that Nemertes is to be commended. And a few comments, though, first, a bit about methodology and then some comments about some specific areas.

One of the things that Nemertes has done here is they have taken these surveys and gone out to users and surveyed users and then validated from those surveys which ones they believe are, in fact, a valid representation. I think, out of 4,500 surveys they used about 1,500, actually to generate this overall report. That’s basically one in three. So, two out of three are being rejected for one or another reason. Either the person didn’t have the right information, was not the right person, et cetera.

Then what they are doing is, basically, in a category, they are surveying for companies. They are breaking those companies down into essentially three representations within the category. The first is a market leader as defined by market share. To be a market leader in this report you have to have relatively large market share. To be a challenger, then, you are a smaller share but above the threshold. Below that threshold, you’re not included. That may explain why, for example, if you went and asked, who do you use for UC? And asked “n” number of enterprises, even though ShoreTel positions itself, for example, as a UC vendor; if only one of those companies came back and said, “we use ShoreTel,” they would fall below the statistical inclusion as a challenger in the methodology of the report.

So I think one of the things you need to do when you look at this is realize this is not a representation of the market but rather the view of customers and what they are using and currently deployed in the market. That’s the first point that I think is really important.

The second point is that it’s very important if you are a potential acquirer of any of these technologies, to look beyond the award numbers. In many ways, the awards here and what has been listed, to my mind, almost can hide the real underlying meaning. For example, in one area – in the contact center space where I did happen to see the data – the variance, for example, between the top, between Avaya and Cisco is about .03 points on a scale from one to four. Within the methodology, it’s stated that there is a three percent validity rating in the statistics of the overall survey. So what that says is, it’s three percent. It doesn’t specify plus or minus three percent. It just says three percent. So that may be plus 1.5 percent.

We’re all familiar, obviously, being presidential election time, with the polls that comes out and say candidate “A” is ahead of candidate “B” by five points with a poll validity of plus or minus X percent, plus or minus three percent. So, if you say it's plus or minus three percent and there is only two percent difference in the candidates, what they will say is statistically you cannot call and say that is statistically relevant. What’s interesting is that when you look under the covers here you actually see that the variation in the customer appreciation of Cisco and Avaya is probably not statistically relevant or statistically different. You cannot really say that Cisco customers or Avaya customers are more satisfied.

What it does tell you though is overall, with a rating that is well over four points – it’s up around 4.1 overall – both Cisco and Avaya customers are satisfied with their purchase and their vendor. That’s really an important piece of information. Because I think one of the things you want to do if you decide to use the Nemertes data is to make sure that when you do it you get the data and look below it and use it not as a ranking as much as a validation of, are the customers that use this vendor satisfied with their conclusions?

In the contact center space, for example, there is one vendor and I won’t mention them by name, that is dramatically lower in customer evaluation than all the others. If you are considering an acquisition and considering that vendor, I would consider that a red flag.

I think what this really points out to me when you look here across the piece is that while it’s interesting to have winners and losers and we all love awards categories and maybe there should be a red carpet for these awards. When you look under the covers, what you really realize is that the key in this, is to pick the product that meets the use case needs in your business. For example, I did a recent small assignment working for a client who needed to make a decision: do they buy the Cisco Jabber based UCIM environment, or do they buy Lync from Microsoft? This was a customer; the end user actually used Cisco for VoIP and data, and used Microsoft for PCs, desktops, productivity applications, as well as a SharePoint back end. They were in the process of making some decisions about some other acquisitions that would impact this.

What really became clear in the discussion about how this new technology would implement and integrate was you had to consider the complete environment. I think that's the key that really this points out to me in the Nemertes research is that we've gotten to the point in this industry where the technologies cannot be segregated based on easy-to-pick categories and easy-to-pick numbers.

It really is about how those technologies fit into your business, how they fit into your existing infrastructure and environment. Most importantly, how do they impact your users and your business processes to give you value either in cost savings in running your business or increased revenue? So, again, I really compliment Nemertes for the numbers. It's great to have awards. I would encourage everyone to look at these as not just awards but as again, a representation of users' perceptions of the value of what they bought and the ease of the vendor to deal with.

Michael Finneran (10:54:): Great, Phil, some great insights. Art Rosenberg, I know you had something to say on this topic.

Art Rosenberg: I just wanted to amplify a little bit of what Phil brought up. My impression is they are evaluating the technology they have gotten from these providers based on the ways that, for example in the contact centers, have been used in the past which is mostly telephony and call centers and call processing and so on. They’re not evaluating them yet because they have not done it or these things are just premature...but they are not evaluating where the contact center of the future is going. It’s in the process of change. It is not just a question of being premise-based or even cloud-based.  But it is starting to move into more of a multi-modal kind of situation. It is bringing into focus self-service that’s not just the old IVR, which was very limited.

But it’s going into online applications which we will now call mobile apps that can now be multi-modal because people are using smartphones and tablets and some things they can use with speech and they can click to connect for assistance. That’s a whole different world than what people used to do in the call center. I think what they are evaluating, to a large extent, is the old way of doing things. It doesn’t say how they are going to be doing it or maybe they started doing it and are they prepared for that? I think that is going to be a key evaluation for who do you go with. How do you get from here to there, and who do you trust, and who is going to make it easy to do that? There are some, and I will not name names either who are looking at it exactly that way. That’s my perspective.

Michael Finneran (12:48): Thanks very much, Art. Do any of the other UCStrategies experts have anything to weigh in on here?

Don Van Doren: I think Phil's comments are right on track as Art just mentioned. Clearly, looking at use cases, looking at strategic partnerships within your organization, looking at the environment, those are all the keys. Especially for what I’ll call generic UC applications. And I don’t mean to demean that in any sense but I am speaking about the way many organizations are currently deploying unified communications capabilities.

What starts to get interesting is that what I think the Nemertes study shows is that many vendors are getting, let me say, above the bar. They have adequate, good enough features in many different categories to meet a fairly broad spectrum of needs. Where we’re going to see, I think, the differentiation coming in the future more and more is going to be very specialized applications that are really uniquely tied into, for example, specific business processes aimed at a particular vertical market, for example.

We’re going to start to see, I think, companies start to develop these things. Probably not so much by the suppliers themselves but rather by their distributors, the systems integrators and the value added resellers that take these products and have specific capabilities within, for example, a particular vertical industry where they are really using some of the APIs and they are growing techniques that the vendors are now building into these products to uniquely tailor them and integrate them into other business software applications.

I think that is really the next plateau that we are going to see moving beyond what I will call sort of these generic applications into these more specialized and unique ones. That, I believe, is going to continue to really drive forward the benefits that organizations will get from UC deployment. Back to you, Michael.

Michael Finneran (14:55): Thank you, Don. Well, I think it’s safe to say our hats are off to Nemertes, and the contribution they have made on such a wide front. But as Phil and Don and Art have pointed out, there are other differentiating factors that do have to be considered. This is still a tough business but I think Nemertes' work has given us some good insights and certainly, our conversation this afternoon has helped to put a little more flesh on the bones. With that, I will thank my UCStrategies experts who contributed today and wish you all a good day.