Methodology for Category Reports:
Given the prevalence of quadrant-based market ratings throughout IT, many will consume our research without a deeper investigation into our methodology at Cloud Ratings.
Cloud Ratings Category Report process combines private vendor responses to our Request For Information (RFI) with publicly visible measures of market adoption and customer satisfaction.
All vendors are presented with the opportunity to complete the RFI.
The Cloud Ratings RFI focuses on measures of:
- Customer Satisfaction: Greatest weight is given to internal Net Promoter Score (NPS) data. Why? Certain software vendors are skilled at driving (and often incentivizing) happy customers to submit reviews to popular software directories. This can lead to situations where publicly observable vendor reviews are more favorable than the “silent majority” of customers that do not submit to software review platforms. As such, Cloud Ratings’ use of private NPS data provides a more holistic view of customer experiences. To measure adoption and implementation risk, significant consideration is also given to vendor responses regarding severely negative customer outcomes and retention rates.
- Market Adoption: The RFI covers a range of traction metrics, including but not limited to revenue, revenue growth, and the current number of customers. Why? Vendors can appear to have an artificially high market share through aggressive marketing spend, public relations, and skillful solicitation of reviews to software directories. Accordingly, Cloud Ratings’ use of RFI-derived information helps provide software buyers with a more realistic view of market adoption.
This graphic – which compares the number of customer ratings provided via vendor NPS data against that vendor’s aggregated reviews on leading software directories – highlights the enhanced market coverage provided by Cloud Ratings RFI:
Definitions:
Shaded Areas of Category Chart: Each category of software has a unique likelihood of generating business value and positive experiences for customers. Accordingly, Cloud Ratings customizes the shaded surface area for each category as a visual representation of the zones of positive customer outcomes. A category with higher risk or lower satisfaction ratings (like ERP or HR software) will have smaller Zones of Excellence and Zones of Solutions than a safer, higher satisfaction category. For Direct Mail Automation Software, a normalized customer rating score of 4.60 (on a 5.00 scale) was used in setting the Zones’ surface area.
Zone of Excellence (Light Green): This represents a category’s strongest vendors as reflected by their combination of market share and high customer satisfaction scores.
Zone of Solutions (Light Yellow): Encompasses vendors with both meaningful marketplace adoption and sufficiently high absolute customer satisfaction ratings.
Notable Vendor: This distinction highlights other vendors identified in the research process as worthy of buyer consideration. Common reasons for Notable Vendors to not be featured in the Category Chart include differing levels of product focus (a secondary feature within a broader suite) or comparatively lower market adoption metrics, especially for earlier-stage companies.
Momentum: Derived from sources like RFI-sourced vendor growth metrics and buyer search interest, the Momentum indicator compares a category’s growth rate with overall information technology (IT) spending growth.
Risk: Influenced by RFI-sourced vendor NPS and adverse outcomes data, the Risk indicator compares the risk profile of a software category to software applications generally. The distribution of strongly negative public user reviews also factors into the Risk indicator level.
Cloud Ratings Disclaimer:
Cloud Ratings research analysis and publications represent opinions – expressed at a specific moment in time – and should not be viewed as statements of fact.
Cloud Ratings is not responsible for any incorrect information supplied by vendors, customers of vendors, or derived from publicly accessible information.
Cloud Ratings assumes no liability for damages resulting from the application or usage of information, content, or research.
Cloud Ratings disclaims all warranties as to the commercial success or outcome of any resulting business activities, vendor selections, or investment decisions from the information provided.
Cloud Ratings does not endorse any solution or service named in its research.