Data Loading...
Case Study: Gamification at SAP Community Network Flipbook PDF
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibite
272 Views
66 Downloads
FLIP PDF 309.89KB
Case Study: Gamification at SAP Community Network Gamification drives engagement at global community with more than 2 million members Reference Code: IT015-001879 Publication Date: 23 Jul 2013 Author: Carter Lusher
SUMMARY Catalyst In the early 2000s SAP was a global enterprise applications giant that was locked in competition with legacy vendors including Oracle and PeopleSoft as well as upstarts including salesforce.com and NetSuite. A key success factor for market leadership was not only having a large developer and customer base, but also having engaged developers and customers loyal to SAP’s products and technologies. Launching an online community to provide a platform for customers, independent developers, partners, and SAP employees to interact was a good step. However, having an online community did not ensure that customers and developers would engage with SAP. To create engagement, the SAP Community Network (SCN) needed to incorporate functionality that would encourage, recognize, and reward participation in the community.
Key messages Even simple game elements such as points and levels can drive important results. Gamification cannot be static, but has to evolve over time to remain fresh and interesting. Involving active community members in major changes as well as fine-tuning the game is key to continual improvement efforts. Communicating with community members about changes to the game elements is critical to long-term success. Managers and staff using gamification need to be open to the unexpected, and must be ready to evolve as new approaches, ideas, and technologies appear. Rewards do not have to be tangible. Intangible and intrinsic rewards are preferred. Integration with existing parts of the install software infrastructure can be a key decision point for gamification procurement projects.
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 1
Ovum view The SAP Community Network (SCN) is an interesting case study because it was using game elements and game mechanics long before gamification became a buzzword and before there were commercial gamification software tools. As a consequence, SCN’s executives and staff have developed a deep understanding via trial and error of what works and what doesn’t. This hard-won experience was especially valuable when the team was preparing to take SCN’s gamification to the next level by shifting from in-house-developed software to a commercial SaaS-based product. Because of their long experience and deep understanding, the SCN team was able to devise a phase 2 plan and vendorselection process that built on their gamification success while also providing critical new capabilities. SCN’s use of gamification is relevant and interesting to enterprises and public sector organizations at all stages of their gamification journey, including learning about the technique, going through the initial planning and tool procurement process, ongoing usage, and the decisions needed for taking gamification to the next level.
CASE STUDY Introduction to SAP Community Network SCN was launched in 2003 as the SAP Developer Network. Over the next few years the membership grew, new communities, including SAP BPX Community for business process experts, were launched, and new functions such as wikis were added. In 2007, SAP Community Network was launched to bring together all the various groups under one umbrella brand. Subsequently, SCN has added new communities, including BusinessObjects and University Alliances, as well as new capabilities, including EcoHub App Store and Code Exchange. Membership growth has been steady with 100,000 members in 2004 and more than 2 million today. Because SAP was an early adopter of online communities to promote tighter ties with customers, it was forced to develop what was in essence a social network platform. This in-house-developed platform formed the basis for its community technology until 2012.
Business issue triggered interest in game elements Soon after the launch of the developer community, SAP ran into the typical problems for social networks, including how to get members to really participate. What was needed was a way to encourage the sharing of knowledge and collaboration between community members. Not only would increased participation reduce the burden on SAP staff to provide content, but also it would dramatically increase the stickiness of the network by giving members a reason to return on a regular basis.
The solution: simple points initially and simple levels later SCN is atypical because it has been using game elements for a nearly a decade (see Figure 1 for a timeline of SCN’s gamification journey).
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 2
Figure 1: Evolution of gamification on SCN
Source: SAP
SCN’s use of game elements started in 2004 when it awarded points for participating in forums, helping other members with technical questions, posting content, sharing knowledge, and so on. The awarding of points, while not tied to a particular reward or status, had the effect of stimulating participation. In 2006, the SCN team used point totals as the sole criteria for the judging for the “Top Contributors Annual Contest”. This use of the points increased the visibility of the point system and the desirability of contributing to the SCN in order to accumulate points, while spurring on the sharing of knowledge. It even enabled the identification of individual topic experts. In 2008, points were incorporated into the selection process for the new SAP Mentor Program. SAP Mentors are the top community influencers of the SAP ecosystem, with most mentors working for customers or partners of SAP and having hands-on expertise of SAP products or services. Being selected as a SAP Mentor is a significant career plus in part because of the elite nature of the program (currently there are only 120 SAP Mentors). While SCN points are not the sole determinant of who is selected for the program, the fact that points figure into the decision mix again increases the points’ visibility and desirability. In 2009, levels were added to provide additional acknowledgement of achievement in the community. The levels used (bronze, silver, gold, platinum) are very recognizable from sporting events, and are easy for members to understand, giving members intermediate goals which can encourage more participation than a simple count of points. Because the number of long-time members accumulating a large sum of points was growing, there was a danger this could be demotivating to members who joined recently, because the new members would rightly perceive that they could never catch up. So in 2009, the SCN team announced that point totals
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 3
would be reset each year so that everybody could start on an equal basis. This announcement caused a negative pushback from the members who saw their points levels begin to drop as previous contributions reached their 12-month life, and felt that their long-time contributions to the community were not valued. The SCN team saw the problem the change was causing and modified the program. Lifetime point totals would continue to be displayed (although less prominently) in addition to the annual point totals. While the reaction to the initial change was a short-term negative, it was useful for the SCN team because it showed how much the members valued something as simple as recognition based on community participation. It is also important to note that the SCN team discovered that some rewards are not really necessary. In 2005 and 2006 the SCN team sent actual physical items such as t-shirts to members when they achieved a certain level of points. This practice turned into a major logistical challenge so the team decided to discontinue the practice. The SCN management team sought the community members’ advice on how to redirect the t-shirt budget, and were encouraged to make a donation to the United Nations World Food Program based on cumulative points on behalf of the entire community rather than rewarding individual achievement with t-shirts. Unlike the annual points reset which caused pushback, the members were fine with dropping the tangible, or extrinsic, reward. It is the social and reputation recognition, the intrinsic, that turns out to be the major reward for members. In 2011, the SCN team modified the game elements to have the community itself award points for quality, replacing the early system, which required moderators to evaluate and reward points. In the new system, a small number of points are awarded automatically by simply posting content, but in order to earn really significant numbers of points for a post, the content must be well received by the community. Contributors are awarded points whenever other members rate their content with 4 or 5 stars or “like” the content in the system. This helps offset the behavior of some people to “game” the system by flooding the community with low-value content just to get points.
Unexpected outcome: building professional reputations Like other IT-based business initiatives, gamification can have unexpected outcomes in addition to or instead of the planned outcomes. In the case of SCN, the desired outcome of the gamification effort was to increase the engagement of members with the community. The strong growth in unique visitors, discussions, blogs, and number of members who have contributed content cannot all be attributed to gamification, but there is ample evidence that the SCN team’s use of points and levels did help drive participation. One of the unexpected outcomes, and proof point on the impact of gamification, is how some members have used their SCN status (bronze, silver, gold and platinum) as part of their professional credentials. The crowd-sourced aspect of the quality assessment (members awarding points based on their assessment of usefulness by rating and liking content) gives SCN status validity. What is interesting is that SCN status is not only useful for SAP and within the community, but also other organizations. Employers looking to hire IT professionals with SAP product expertise are asking “what is your SCN status?” Therefore, SCN members, whether IT professionals or third-party consultants, are now adding their SCN status to resumes, LinkedIn profiles, brochures, and other documents. This is a powerful validation of the points and levels gamification.
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 4
Another unexpected outcome is that the point system also helped SAP to quickly identify collaborative and engaged thought leaders and technical experts outside and inside SAP. This gives SAP the ability to tap these individuals for their insights and advice as well as invite them to participate in programs such as SAP Mentors or to be recognized as a Member of the Month.
“Leveling up” gamification sophistication and flexibility In 2011 the SCN team decided to migrate the network off the in-house platform to commercial products. The in-house code was cumbersome, requiring IT to make even minor changes to the social network and gamified functions, and it was limiting the ability of the team to use new game concepts. Because of engagement decay that occurs naturally with game elements such as points and levels, the SCN team needed access to more modern gamification functionality and the ability to implement new initiatives quickly without requiring IT support. After researching the marketplace and current literature, the team decided it wanted to exploit functions, such as: Badges based on missions (for example, a mission can have the goal to explore, engage, create, or influence). Additional levels at the low end so new members have an easier way to get going, and at the high end to recognize some superstar members. More robust reputation dashboards. Flexible leaderboards. The team also wanted a platform that permitted team staff to create and launch missions on their own quickly without requiring IT support. The migration of the infrastructure took part in two steps. First, the home-grown social network platform was replaced with Jive Software, and after the decision to also use Jive for the enterprise social network was made, the team turned to modernization of the gamification platform.
Why the gamification solution was selected The SCN team started the gamification platform step in April 2012. The team reviewed the nascent gamification marketplace and quickly narrowed the vendor shortlist to Badgeville and Bunchball. These two vendors have taken a broad, platform approach rather than a narrow, role-centric approach such as a leaderboard only for sales or contact center talent management. In addition, Badgeville and Bunchball were the most established vendors, with many other gamification vendors still early startups. The SCN team thought the two finalists were very competitive. Badgeville was a bit less technically mature, but had interesting concepts around behavior modification, and pitched its consulting services as a key differentiator. Bunchball excelled in its thinking, presentation, and the working prototype it created. A critical decision criterion was Bunchball’s integration with the Jive Software platform because that had already been chosen and was in the process of being deployed. At the time of the selection, Badgeville did not yet have a formal partnership with Jive. In the end, the SCN team picked Bunchball in strong part due to its established partnership with Jive Software and working integration with the Jive product.
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 5
Deployment The contract with Bunchball was completed in August 2012, with the deployment project starting soon after. The deployment project for gamification was “relatively” smooth, in part because the team had learned a lot during the Jive deployment. In addition, the SCN team’s experience of gamification had been built up over the previous eight years so it did not have the learning curve and stumbles that are typical for beginners first exposed to real-world basic game elements and mechanics. One key learning point from the Jive project was the importance of keeping the community up to date on the transition, and soliciting the members’ input. Members want feel invested in the community and will push back if changes do not meet their approval. The SCN team therefore worked to keep the communication flowing to members throughout the project.
Benefits of gamification (Phase 1) The SCN team has diligently collected statistics about the growth of the SAP Community Network. The numbers are impressive. Examples include: More than 2 million unique individual visitors per month in 2012. 38,000 contributors in the past 12 months, and 200,000 over the life of the SCN community. About 1,170 discussions per day with 17,000 likes, 7,000 comments, and 6,000 ratings. It is not possible to attribute the exact impact of gamification on SCN’s statistics and success due to so many variables. However, it is clear from indirect indicators, such as the use of SCN status as a measure of professional achievement, that the gamified functions do have credibility and drive engagement.
Benefits of gamification (early Phase 2) The upgraded gamification features (powered by Bunchball on the Jive platform) were put into production April 28, 2013. As of June 30, 2013, the results for the first two months have been impressive. The metrics indicated as “baseline” were taken the month prior to the April 28 rollout. Activity: content creation, comments and feedback up by 1,113% o
From 107,000 actions baseline to 1,299,000 actions
Community feedback: up by 250% o
From 2,900 feedback elements baseline to 10,100
Points: up 147% o
25,000 points baseline to 62,000 points
Missions with badges earned: 79,500+ earned o
53,000 badges awarded the first month, 79,500 awarded by the second month, 50% month-over-month increase
The activity, community feedback, and points are very solid hard numbers. Remember that the SCN has been awarding points since 2008 and collecting other data since before that so these increases are in comparison to an extensive database. Badges and missions are new features that were added with the
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 6
April update, so percentage changes are not relevant. That said, the sheer number of badges earned and missions completed illustrate the acceptance of the new features by SCN community member. Example missions include "Ready, Set, Go!" (series of three missions) for on-boarding and content missions such as "I Blogged!" and "In a Blogging Mood." The business value implications of changes such as the 1,113% increase in activity (content creation, comments, and feedback) are significant. For example, one of SCN’s roles is to enable members to help one another with technical issues. By increasing activity, there are more questions that are being answered by the community, alleviating the need for SAP to provide customer service. At the same time, the knowledgebase of searchable answers is growing, which means future user conundrums can be answered without the need to even submit a question. Not only will this result in more satisfied customers, but also a reduction in direct customer support costs. Less easy to quantify, but important nevertheless, is that SAP product managers will have more data points for making product feature and investment decisions.
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 7
Figure 2: SCN Gamification Phase 2 metrics
Source: SAP
Lessons learned The SCN team shared a number of lessons learned about leveraging gamification in a social network, both for the initial deployment and ongoing use. Tie your vision to business goals. Know your audiences and what they value. Involve community members in the process. Leverage “design thinking” by taking an audience-in approach. Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 8
Give your project team time and space to experiment and evolve. Have an open mind to changing the status quo. Inject playfulness. Ovum also observed a number of key learnings. Even simple game elements such as points can have important results with measurable impact. Game elements and mechanics used cannot be static, but have to evolve over time in order to retain the members’ attention and to counter bad behavior (exploiting the system). Organizations have to be open to the “players” feeling ownership of the “game” and that community members will expect to be part of the planning. Unexpected benefits can sometimes outweigh the original intentions of the gamification efforts.
APPENDIX Further reading 2013 Trends to Watch: Customer Relationship Management, EI015-000005 (October 2012) The Customer-Adaptive Future, IT015-001822 (December 2012) On the Radar: Bunchball, IT015-001778 (October 2012)
Author Carter Lusher, Research Fellow and Chief Analyst: Enterprise Applications Ecosystem, Enterprise Solutions [email protected]
Disclaimer All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of the publisher, Ovum (an Informa business). The facts of this report are believed to be correct at the time of publication but cannot be guaranteed. Please note that the findings, conclusions, and recommendations that Ovum delivers will be based on information gathered in good faith from both primary and secondary sources, whose accuracy we are not always in a position to guarantee. As such Ovum can accept no liability whatever for actions taken based on any information that may subsequently prove to be incorrect.
Case Study: Gamification at SAP Community Network (IT015-001879) 23 Jul 2013 © Ovum. Unauthorized reproduction prohibited
Page 9