Papers

Gaming Leadership Development with Multi-Rater Assessment

Zain Ali & Anita Knox

Abstract

Game development companies are founded based on a passion and love for gaming. Founders have spent a lifetime playing games, which has motivated education in a given development area like design or programming. The disruptive technology experienced in the gaming industry requires constant updating of these game development skills, which consumes the focus of founders, leaving skill gaps in running the business. To remain sustainable gaming leadership, require assessment of skills, to identify a targeted plan for continuing development.

Leadership development is a highly researched area in academia as well as a multi-billion-dollar industry annually. One of the most effective assessments used for leadership development is multi-rater assessments. Research and successful case studies indicate that when multi-rater-based leadership development is implemented properly with aligned competencies, it leads to individual behavioral change and collectively organizational change. Multi-rater assessments in the gaming industry appear to be a new concept due to the emerging nature of this industry.

Our research first focuses on identification of leadership competencies in the gaming industry. We shift focus to mutli-rater evaluation and exemplify key feedback that fosters permanent behavioral changes in gaming leaders, as well as process steps to support organizational change.

Keywords: Multi-rater Assessment, 360-Assessment, Leadership Development, Capacity Development, Competency Development, Video game leadership skills, video game industry

Introduction

Gaming has reached new heights in the entertainment industry. The console video game market reached $38.4 billion in 2010 (Broekhuizen et al., 2013). Starting in 2005 console manufacturers released digital storefronts, accounting for $1.9 billion by 2010. Social media and device app stores digitally distributed $20 billion worth of games in 2010 (Broekhuizen et al., 2013). The gaming industry is expected to reach over $300 billion by 2025 (Koksal, 2019). Top game development markets are in the US, Japan, Canada, South Korea, and the United Kingdom, with an audience of over 2.5 billion gamers globally (Koksal, 2019).

This explosive growth of the industry shadows the challenges game development studios have in sustainability. Gaming studios are founded by lifelong gamers who are passionate about game development (Lysova & Khapova, 2019). They maintain the creative and technical skills required to develop games; however, they lack in the skills needed to run the company (Lysova & Khapova, 2019; Landoni et al., 2020; Broekhuizen et al., 2013). Skills directly related to game development must be continuously practiced and improved upon to innovate and adapt to changing technology requirements (Cabras et al., 2017). Although the highly technical competencies needed to develop the product are maintained, founders lack the competencies required to generate economic return (Broekhuizen et al., 2013).

The phenomenon investigated in the current article is the emergence of leadership in the gaming industry. The gaming industry is unique in that it is experiencing extreme revenue generation during its growth stages. Aside from this rapid growth, individual gaming studios have similar characteristics of other creative and software industries, each with their own sustainability challenges. Identifying the leadership competencies required for this unique sector will provide a basis for sustaining its growth while also introducing patterns of leadership that could be modeled for other game-based organizations.

Leadership is one of the most widely researched topics in the world and has the highest share of training and development budgets (Ardichvili, 2008). Based on the derivation from the findings of several reports (Bersin, ATD, SHRM, Forbes), the global market spending for development and training was estimated to be approximately $130 billion (USD) with $68.5 billion global market spending for leadership development between the years of 2013 to 2017. In the United States alone, the estimated spending for leadership development was $15.5 billion in 2013.

A survey by the Center of Creative Leadership (CCL, 2019), identified the number one concern facing executives is finding and retaining the talent they need. Only two out of ten leaders believe that they have the leadership bench strength that they need for succession and only 50% of the leaders believe that they have the high-quality leader’s development program required to build a stronger team. The general consensus from the survey is that the investment made into leadership development is not yielding the desired results. The DDI Global Leadership Forecast conducted in 2021, compromising of 15,787 leaders and 2,102 HR professionals across 1,742 global organizations, identified that the bench strength of available leaders is at an all-time low, with only 11% to fill leadership roles in their respective organizations (DDI, 2021).

This paper will first discuss the methodology for identifying literature, next we will review the competencies associated with the gaming industry. These include competencies related to game development, and those needed to run the business. Once the competencies have been identified, we will focus on multi-rater assessments with the review of assessment results that support behavioral change based on research.

Method

Database searches for gaming competencies were performed in Science Direct, EBSCO host and JSTOR. These databases were selected because they cover disciplines related to the gaming industry (e.g., computer science, software engineering, arts & sciences, and business). Searches included peer-reviewed articles and book chapters between the years of 2010-2021. Article selection began with the review of abstracts to identify the use of keyword combinations.

The initial search criteria included topics in leadership and management in the gaming industry. Review of the literature identified leadership as hands-on, with leaders working along-side the rest of the team. This influenced expansion of the criteria to include day-to-day activities. Keywords included: leadership, management, manage, career, skills, talent management, game industry, gaming industry, video game industry, digital game industry, game sector. The ‘gaming industry’ keyword resulted in a larger focus on gambling, the search was updated to be more specific by the use of ‘video game industry’ and later ‘digital game industry.’

The first search for multi-rater assessments was performed by using two databases, EBSCO host and JSTOR, with the following keywords, multi-rater feedback OR 360 AND leadership”. Our initial search yielded 255 and 41,040 results initially from the two respective databases with no dates identified. By refining the search with limiting the years to be between 2011 to 2021, English language only and modifying the key words to “multi-rater feedback OR 360 and leadership development”, we were able to reduce the search results to be 29 and 360 in EBSCO Host and JSTOR respectively. We were able to further refine the JSTOR search to 274 by focusing on journals that were related to Business, Education, Communication, Management & Organization Behavior, Science & Technology and Technology. We then started reviewing the 303 papers and eliminated all the articles that were in a classroom setting to finally look at titles and then abstracts to come to our final list of articles to be used for our research on multi-rater assessments.

We performed two independent searches one with the focus on leadership development in gaming industry and second on leadership development and multi-rater or 360 assessments. In hindsight, we should have also combined the keywords for these two independent searches together to find targeted relevant publications.

None of the identified research articles identified an approximate budget for corporations to spend on leadership development so we started looking at trade articles and web publications of reputable industry organizations, such as American Training and Development (ATD), Society of Human Resource Management (SHRM), Forbes Magazine and Bernstein report, to identify the global and U.S. spending on leadership development.

A review of the articles from the search results covered several important topics such as the effectiveness and importance of competencies in multi-rater assessments, the recommendation of the processes to administer multi-rater assessments, the importance of rater education, collection of feedback, as well as the approach for coaching interventions used in the administration of multi-rater assessments. However, we did not find any research articles that explored the output of a multi-rater assessments that would support the research findings mentioned earlier. We saw this as a gap in research and went on a quest to identify multi-rater system outputs to see how they would support the research findings of effective leadership development with a multi-rater assessment.

We then performed google searches to find commercially available “360 or multi-rater assessment tools” as keywords and came up with 538,000 search results and narrowing down the search was also not helpful. Our academic research articles had identified a number of research papers that were by the publishing company Wiley so we altered our approach to see if there as a 360 assessment offered by Wiley and found a multi-rater assessment “363 for Leaders”. We then looked for commercially available instruments comparable to Wiley’s 363 for Leaders and identified four multi-rater instruments: Profile Incorporated Checkpoint 360, Wiley Leadership 363, Skyline C4X and SVI World 36Dollar360. Website and Google reviews and sample output reports from these four leadership multi-rater tools were reviewed for the current research study.

The combination academic journal articles, case studies, trade journals and applied multi-rater instruments provided different views for our research on the topic of multi-rater assessments for identifying strengths and weakness for leadership development.

Literature Review

Video Game Industry

Game development studios are started based on a creative calling for the founder (Lysova & Khapova, 2019). The founder contributes to design or technical aspects of game development depending on individual interests or education (Lysova & Khapova, 2019). Since gaming founders have more interest in the game development aspects of the business, they lack the managerial skills and experience required to actually run the business (Landoni et al., 2020). Founders have been gamers for most of their lives and are attempting to monetize a lifelong passion (Lysova & Khapova, 2019). As entrepreneurs’ founders work alongside team members contributing a given specialty. Lysova & Khapova (2019) interviewed game founders to identify a common recognition that pursuing passion has left gaps in the business aspects of game development (i.e., market needs, publisher partnerships, internal team mentoring and relationship building). Broekhuizen et al., (2013) identified that founders have and maintain specialized skills required to develop the game, but lacked a broader skillset required to sustain the business.

The argument could be made that game studio founders could hire managers with the necessary competencies. Although gaming is a high revenue generating industry, game development studios operate as a project-based business, which imposes constraints on temporality of the team and organizational finance (Hodgson & Briand, 2013; Parmentier & Picq, 2016). The team of game developers, designers, programmers, artists, animators, audio designers, are made of a highly specialized network of freelancers, that commonly work on a contract basis (Hodgson & Briand, 2013; Sholz, 2012). This temporariness of the development team influences high turnover in all aspects of the business (Sholz, 2012). It can take time for new game releases to gain a following, with some never taking off. For example, digital distribution encounters ‘app clutter’ by the magnitude of games available (Broekhuizen et al., 2013). New and previous knowledge, methodologies and technologies of founders are capitalized on to recoup costs from past innovations (Parmentier & Picq, 2016). Some game founders are forced to take on project work to sustain the business, leaving a lack of consistent revenue to maintain additional salaries (Lysova & Khapova, 2019).

Competencies in Literature: A competency can be viewed as a skill, ability or knowledge that is used or applied (Shet et al., 2017). It is possible competencies are acquired through learning or natural talents based on personal characteristics of an individual (Aisha et al., 2019). Regardless of how they are acquired, competencies are applied in a productive manner, and have the ability to influence other individuals. Individual competencies must align with the competencies required to fulfill a job role. It is possible to observe and measure competencies to determine alignment and identify gaps that can be filled with development (Aisha et al., 2019).

There is a gap in gaming industry competency literature, since this industry is not commonly studied (Hodgson & Briand, 2013). The literature that is available focuses directly on competencies related to game development which include creative, technical, innovation, and delivery. Although the body of literature offers some discussion about competencies to sustain the business they are suggested and indirectly addressed. After a review of literature eight competencies were directly or indirectly identified: creative, innovation, technology, relationships, communication, visualization, delivery, diversity.

Competencies Directly Discussed: Directly discussed competencies include creative, technology, innovation, and delivery. Game development is a balance between creative and technical industries influenced by knowledge and innovation with elements of the entertainment industry (Mollick, 2012). The technical aspect is commonly associated with the software development industry (Mollick, 2012; Hodgson & Briand, 2013; Aleem, 2016). The ability to keep up with technology requirements can influence the competitive edge, and ability to sustain (deVaan, 2015). For example, technical specifications for game consoles vary between generations. Games produced for today’s console are significantly different than prior generations (deVaan, 2015). Technology skills must be renewed on a regular basis to keep up with these changes (Parmentier & Picq, 2016). There is a dynamic exchange between creativity and technology (Panourgias et al., 2014). Creativity could reference artistic work as well as approaches and processes (Landoni et al., 2020). Regardless, the environment must foster creativity since it is not possible to directly manage (Parmentier & Picq, 2016; Landoni et al., 2020). Creativity can be influential to innovation however Panourgias et al. (2014) discusses the mere surprise that comes with the output from the creative process. This makes alignment with organizational goals difficult and requires more of an emergent plan that can iteratively adapt (Panourgias et al., 2014).

Innovation is the exploitation of new ideas, and successfully bringing those ideas to the market (Aleem, 2016). There are various types of innovation that could incorporate technical, creative and even operational skills (Parmentier & Picq, 2016). Technology innovation refers to disruptive approaches in addressing both accommodation of emerging technology and the development possibilities (Parmentier & Picq, 2016). Design innovation includes a framework that incorporates new technology, innovates artistry and expresses the overall vision of the game (Mollick, 2012).

Delivery refers to version output of a game, and where that game is published. Publishing can be direct to market, or through a publishing partner. Competencies would vary based on how delivery is done. Traditionally publishing partners were used to handle game delivery tasks. The start of online game channels created an opportunity for self-publishing. A lower dependence on publishing partners can keep more revenue in the gaming studio, however self-publishing opens the need for gaming leadership to address marketing, organizational reputation, and relationship management with proprietary distributors (Broekhuizen et al., 2013). Broekhuizen et al. (2013) studied a smaller game studio and identified no marketing budget in place, nor any individuals that were skilled to handle that task. Founders focused on specialized skills related to game development and lacked skills needed to market the game directly to consumers. In this example an intern was used to write press releases and perform social marketing. These efforts were not successful in effectively selling the game. Potential skills that need to be addressed for self-publishing include marketing directly to customers (advertising, promotion, social presence, press release), organizational reputation (trustworthy, offer a good product, external reviewers, reputation in industry), and relationship management with distributors (gatekeeper for online resources). In addition, publishers help with the valuation of products to determine where they belong in the marketplace. They develop relationships with external reviewers, who are instrumental in positioning games to the public, and they handle digitizing games for the variety for format requirements (Broekhuizen et al., 2013).

Competencies Indirectly Discussed: Literature indirectly discusses competencies for vision, relationships, communication, and diversity. Vision is discussed throughout literature as an implied competency. The mere act of starting a game design studio requires vision, however the manner in which vision is carried through the process is not discussed. The founder’s vision must be conveyed to the team of artists, designers, and programmers to develop a game. As the initial vision is carried out and improved the market will dictate new directions. The constant change of the industry disrupts the initial vision, gaming studios must quickly shift to address rapid change, or they will not survive (Aleem et al., 2016). There must be balance struck between adhering to organizational vision and maintaining the freedoms required to create, innovate and elevate the vision (Parmentier & Picq, 2016).

Relationships and communication influence the inner workings of the game organization, as well as success in the market. Although vital to sustainability there is little information on these competencies. Artists, designers, animators, and programmers intermingle throughout the development process, with each role dependent on each other (Sholz, 2012). The ongoing workflow requires co-development relationships (Parmentier & Picq, 2016). Iterative communication is necessary between every member of the game development team (Hodgson & Briand, 2013). In addition to team relationships gaming studios need to develop relationships with customers (Aleem et al., 2016). A trustworthy reputation must be projected to consumers (Broekhuizen et al., 2013). Social communities are composed of individuals that are not directly associated to a gaming studio; however, they directly affect the success or failure of a game release. Relationships developed between gaming studios and social communities are so important they have become part of the industry business model (Burger-Helmchen & Cohendet, 2011).

Diversity can refer to the culture of an organization as it relates to global culture or job role. Companies recruit top talent from a global workforce. Cultural diversity can create some challenges, it also promotes creativity within the organization (Sholz, 2012); (Parmentier & Picq, 2016). In addition, diversity can relate to the consumer since gaming produces cultural goods (Sholz, 2012). The creative assembly of symbols through imagery and audio can take on different meanings to different cultural groups. An understanding and respect of the symbolism is important to maintaining a positive reputation.

Competency Occurrence in Literature: The competency used the most frequently in literature was creative, which appeared 784 times. The second was innovation, which was found 416 times. Relationships was fourth with 182 occurrences, and technical next with 97 occurrences. Least found competencies were diversity 73, communication 31, delivery 13, and visualize with 12 occurrences. Detailed in the table below.

Competency	Keyword	Occurrence	Reference
Creative	Creative	522	(Aleem et al., 2016); (Burger-Helmchen & Cohendet, 2011); (Broekhuizen et al., 2013); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (Mollick, 2012); (Panourgias et al., 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
Creativity	262	(Burger-Helmchen & Cohendet, 2011); (Broekhuizen et al., 2013); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (Mollick, 2012); (Panourgias et al., 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
	784
Technical	Technical	97	(Aleem et al., 2016);(Burger-Helmchen & Cohendet, 2011); (Broekhuizen et al., 2013); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Mollick, 2012); (Panourgias et al., 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
Innovation	Innovation(s)	316	(Aleem et al., 2016); (Burger-Helmchen & Cohendet, 2011); (Broekhuizen et al., 2013); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (de Vaan, 2014); (Mollick, 2012) (Panourgias et al., 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
Innovative	76	(Aleem et al., 2016); (Burger-Helmchen & Cohendet, 2011); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (Mollick, 2012); (Parmentier & Picq, 2016); (Scholz, 2012)
Innovate	17	(Burger-Helmchen & Cohendet, 2011); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (Mollick, 2012); (Parmentier & Picq, 2016); (Scholz, 2012)
Innovating	3	(Landoni et al., 2020); (Scholz, 2012)
Innovator(s)	2	(Broekhuizen et al., 2013)
Innovativeness	2	(Chiambaretto et al., 2019)
	416
Relationships	Relationship(s)	182	(Aleem et al., 2016); (Burger-Helmchen & Cohendet, 2011) (Broekhuizen et al., 2013); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Landoni et al., 2020); (Lysova & Khapova, 2019); (de Vaan, 2014); (Mollick, 2012); (Panourgias et al., 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
Communication	Communication	31	(Aleem et al., 2016); (Burger-Helmchen & Cohendet, 2011); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Lysova & Khapova, 2019); (de Vaan, 2014); (Mollick, 2012);(Scholz, 2012)
Communicate	4	(Lysova & Khapova, 2019); (Parmentier & Picq, 2016)
	35
Delivery	Delivery	4	(Aleem et al., 2016)
Deliver(s)	7	(Broekhuizen et al., 2013); (Hodgson & Briand, 2013); (Lysova & Khapova, 2019)
Delivered	2	(Hodgson & Briand, 2013)
	13
Diversity	Diversity	73	(Burger-Helmchen & Cohendet, 2011); (Broekhuizen et al., 2013); (Cabras et al., 2017); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Lysova & Khapova, 2019); (de Vaan, 2014); (Parmentier & Picq, 2016); (Scholz, 2012)
Visualize	Vision	12	(Burger-Helmchen & Cohendet, 2011); (Chiambaretto et al., 2019); (Hodgson & Briand, 2013); (Lysova & Khapova, 2019); (Mollick, 2012); (Parmentier & Picq, 2016)

Competencies directly related to game development had the highest level of keywords found in literature; this combination of creative, technology, innovation and delivery totaled 1310 occurrences. Competencies indirectly discussed largely related to business and operational functions which include relationships, communication, vision, and diversity which totaled 302 occurrences. This indicates literature prioritizes leadership competencies required for game development. Competencies related to running the business are a significantly lower in priority in literature. This highlights a gap in development for actually running the business.

Discrepancies in Competency Identification: The competency of technical was expected to occur more than it was identified. Technical describes a variety of tasks that could include software development tasks like programming, working in specific software as in audio and animation, or relate to a perspective. It is believed searching more specific tasks will generate a higher number of occurrences, however other competencies were not detailed by specific task so technical followed suit.

Multi-rater Assessments

Popularity of Multi-rater Assessments: Multi-rater assessments have gained global popularity with reports suggesting that multi-rater assessments are used by 90% of Fortune 500 companies and 85% of Australia’s top 500 corporations leading to significant interest in the topic from academia since the 1990’s. Most of the research provides a favorable view of multi-rater assessments stating that the process leads to favorable outcomes for the individual as well as the leader. While the empirical evidence on multi-rater assessments provides mixed results especially in a one and done approach of multi-rater assessment (Dai, 2010).

Multi-rater assessments, also known as 360-assessment, typically refers to a process of collecting feedback from the self and any combination of raters from one or more categories such as manager, peers, direct reports, customers, suppliers and others to provide feedback to an individual being rated typically a leader. The feedback is focused on the performance of the rated leader with the intent of providing guidance on the perceived strengths and weaknesses by the raters for learning and development. These multi-rater assessments are most effective when it meets the following four criteria (Basu, 2019).

Includes both quantitative and qualitative feedback
When the raters providing feedback are credible
Feedback is constructively phrased to encourage development
Collection of rating is paired with mentoring and follow-up

A survey conducted by the Institute for Corporate Productivity (i4cp) of 610 representatives from several industries to identify what instruments are used by the industry for leadership development identified multi-rater assessment to be the mostly utilized (77%) followed by Myers Briggs (MBTI) (68%), and DiSC (61%). About 54% of the surveyed organizations had a formal leadership development plan and 75% of the organizations were using more than one tool such as a multi-rater assessment in combination with MBTI, or DiSC or Hogan or Lominger (Brotherton, 2012). This led to the conclusion that a majority of the companies are using two instruments for leadership development.

Leadership Development = multi-rater Assessment + Personality profile

Where the multi-rater assessment (36Dollar360 or Checkpoint 360 or Leadership 363 or Coaching for Leaders or others) + Personality Profile (MBTI o DiSC or Hogan or Lominger or others) is used by most organizations.

Effectiveness of multi-rater Assessments: Leadership development is one of the most desired form of training by managers and executives (Corporate Training and Development, 2006) and a comprehensive favorable study (Bracken, 2011) on individual behavior and organizational change states that multi-rater assessments have great promise but only if the multi-rater instrument and the implementation process meet the four conditions:

Relevant Content: Organizationally aligned leadership capacities
Credible Data: Unbiased data collected from multiple raters
Accountability: Aligned with company goals and values
Census participation: All leaders in the organization are participating

Turner (2018) categorizes leadership capacities into 31 sub-categories of skills, traits and behaviors of leaders as part of leadership development techniques using a typology of development that can be used as a foundation for relevant content. This is even more important since one of the complications for leadership development is choosing the correct competencies since there are numerous theories of leadership development without one clear model or framework for competency selection (Gentry, 2007). It is critical that in the design process, the correct leadership capacities are identified to ensure that the content for evaluation in the multi-rater instrument is aligned with the behavioral change that the organization is seeking to bring in the leaders as well as in the organization.

On the other hand, the software application of the existing multi-rater instruments is rigid with little flexibility to modify what is available in the application. When leadership capacities are designed properly, then it sets the foundation for behavioral change as shown in the figure below:

Figure 1- A Systems view of multi-rater Feedback (adapted from Bracken and Timmereck 2001a)

Challenges of multi-rater Assessments: One would assume that if leaders are getting feedback on relevant leadership competencies than the performance of the leader would improve, especially when compared to leaders that receive no feedback, but that does not appear to be the case according to a study conducted on feedback effectiveness of multi-rater assessment appraisals (DiNisi, 2000). In his research, DiNisi identifies that in many organizations multi-rater assessments are administrated only once making it very difficult for leaders to see their performance over time. Another reason cited for low effectiveness of multi-rater assessments is the lack of inclusion of goal setting as part of the multi-rater-evaluation process, which could be incorporated easily. The summary of DiNisi’s (2000) findings on the current state of multi-rater assessment implementation is listed in table 1 below.

	Condition	Status	Multi-rater Appraisal Status	Impact on Feedback effectiveness
1	Use of comparative or normative data	Yes	In most cases, comparing each source with self-rating	Focuses attention on self which makes the feedback effects more problematic and performance decline more likely.
2	Consequences for evaluations	Sometimes	For about half the cases	Increase anxiety which is likely to result in performance decline.
3	Goal setting program included	Sometimes	Only about half of the time	Goal setting with feedback increases the feedback of the assessment.
4	Repeated feedback with information about improvement	No	In most cases these appraisals are done only once	Frequent feedback with messages about improvement increases feedback effectiveness
5	Complex Tasks	Yes	Typically used for managerial jobs	Feedback is more likely to interfere with performance on complex jobs
6	Information about correct solutions	No	Not clear which source of feedback is the correct one	Feedback that provides information about correct solutions is more likely to be effective, although not always.
7	Multiple Sources	Yes	Always present	Potential effects not clear but more likely to focus attention on ought self especially when messages are inconsistent.
8	Coach	Sometimes	Not a formal part of most systems.	Likely to help employees deal effectively with feedback, and especially to help formulate accurate hypothesis on how to improve performance.

Table 1 – Feedback characteristics related to multi-rater-degree appraisal systems

It has also been identified that the cost of implementing multi-rater assessments are also high. This includes the cost of performing the multi-rater-instrument assessment ($365 per assessment), the investment of time taken by all raters to provide feedback and the cost of an internal or external coach to facilitate assessment (ranging from $200 to $3,500 per hour). The coaching model is antiquated, expensive and reactive allowing companies to use coaching only for a selected few privileged executive in 35% of the organizations (Crush, 2009).

One of the four components for multi-rater assessment is the qualitative data collected via free comments and in a review of 11,483 rater forms that included 4,777 forms with qualitative data concluded that in the current form the qualitative data provided very little value towards the leader’s personal development (Vivekananda-Schmidt, 2013). The key recommendations from this study included educating the raters on the purpose of the assessment, as well as recommendations on how to provide feedback to the raters.

In a study of 294 leaders who participated in a 5-day leadership development program about goal setting and leadership behavioral change, it was identified that leaders might have different individual goals from the same development program. The results were consistent with belief that there is a relationship between goals and perceived behavior change. (Johnson, 2012). The multi-rater assessment should lead to personalization of goal setting based on the quantitative and qualitative feedback received on the multi-rater assessment.

Successful case studies: Case studies from Comcast and Suntrust demonstrate that leadership development programs that leverage multi-rater assessments can help the organizations become able to reinvent the organizations from inside-out. At Comcast, the leadership program for 700 middle managers focused on creating professional leaders that are self-aware of their strengths and weaknesses, as well as improving emotional self-awareness with the use of a multi-rater instrument and by sharing the aggregate results with everyone and linking leaders’ behaviors to outcomes. These self-aware leaders were able to see more job opportunities and more project opportunities then those that were not self-aware (Gallagher, 2012). Suntrust bank managed to achieve transformational results, 38% increase in mortgages, 48% increase in investment sales and 59% increase in business bankers by focusing on their leadership talent. They invested in leadership development by taking 3,500 of their employees through slightly different variations of three leadership development programs that focused on longitudinal duration, multi-rater-degree assessment, coaching and on the job assignments where leadership capacities were tied to Suntrust’s 3 guiding principles of operating as one team- putting our clients first and focusing on profitable growth.

The literature review identifies that multi-rater assessments can be a very effective tool in the development of leaders by identifying the strengths and weaknesses of the leader, as well as creating positive change in the leader and organization. The academic research and case studies support the idea that when the leadership development processes meet the conditions:

        •       selecting the correct leadership capacities,
        •       aligning the leader’s goals and objectives with the organization,
        •       collecting appropriate quantitative data,
        •       collecting adequate qualitative data,
        •       using qualified coaches or facilitators to complete the process successfully.

Discussion

The literature discussed in this paper has identified competencies necessary for gaming studio leadership to manage a sustainable business. More specifically related literature has identified the value of competencies for creative, technical, innovation, relationships, communication, delivery, diversity and visualize. Targeted professional development would be beneficial to running the business and being more effective within the industry marketplace.

A multi-rater assessment can foster development of a more tailored plan to address needed competencies, without spending valuable time on unnecessary training. Based on the requirements identified in the multi-rater literature, we identified twelve conditions that should be met for a successful multi-rater-based leadership development program. We reviewed the identified multi-rater instruments to see if these instruments met the conditions and at times have identified as to what components should be made available for leaders for personal development.

The twelve conditions for successful multi-rater assessment-based leadership development are:

Leadership competencies are aligned to support company goals and values
Relevant data is collected to support the leadership competencies
Questions are complex and pertaining to the role
Quantitative data is captured
Qualitative data is captured
Feedback is constructively phrased
Raters are credible
Multi-rater assessment used with personality profile assessment
Goal setting is done as part of the assessment process
All members of the organization participate in the data collection
Collection of data is paired with mentorship
There are consequences associated with goal setting objectives

Identified the eight leadership competencies: creative, innovation, communication, relationship, delivery, visualization, and technical. Each one of the leadership capacities identified by the organization should be clearly defined such that the leadership capacity development is aligned to the overall objectives of the organization. In this case we have identified the eight leadership competencies for gaming studio leadership development to be Creative, Innovation, Communication, Relationship, Delivery, Diversity, Visualization and Technical.

#	Leadership Capacity	Description
1	Creative	Demonstrates artistic work in game design as well as in processes approaches.
2	Innovation	The exploitation of new ideas, and successfully bringing those ideas to the market.
3	Communication	Promote iterative interaction between each development role.
4	Relationship	Ability to develop and maintain relations between gaming studios and social communities; as well as the co-development relationships required to produce a game.
5	Delivery	Refers to version output of a game, and where that game is published; either through direct to market publishing, or through a publishing partner
6	Diversity	The culture of an organization as it relates to global culture or job role.
7	Visualization	The ability to adhere to the organizational vison while fostering productive adaptations.
8	Technical	Includes hardware and software development tasks like programming, working in specific software as in audio and animation, or relate to a perspective.

Table 2- Gaming Industry Leadership Competencies

Leadership competencies are aligned to support company goals and values:

In addressing targeted development, instruments must be adaptable to varying organizational needs. The capability of identifying unique competencies for each organization is missing from three of the instruments that were under review. 36Dollar360 website mentions that the assessment can be customized where ten validated competencies are available and organizations with unique needs can enter their custom competency models and questions. C4X website mentions that 28 competencies are available in the system. There are four different assessments identified on the website for C4X where X = Leadership, Management, High Potentials and Women. Checkpoint 360 focuses on eight universal management competencies for leadership and management development. Leadership 363 focus on eight leadership approaches for leadership development. Based on the review of websites, only 36dollor360 allows users can customize the solution to include their unique competency models. All the multi-rater assessments have a radar chart similar to the shown in figure 2 below.

Figure 2: Radar chart for gaming leadership competency development.

Review of the four reports indicates that Checkpoint 360 and C4X allow for alignment of the competencies as prioritized competencies where the Manager and Mentee output highlights alignment and misalignment areas for both the manager and mentee to review. They can then identify the competencies necessary for development. This begins the conversation between the two parties.

	Leadership Competency	Self	Manager	Aligned
1	Creative	Yes		No
2	Innovation
3	Communications	Yes	Yes	Yes
4	Relationships
5	Delivery		Yes	No
6	Diversity
7	Visualize	Yes		No
8	Technical		Yes	No

Table 3- Competency prioritization

Relevant data is collected to support the leadership competencies: For the multi-rater assessments under review this would be true whenever the competencies identified in the assessment would match the competencies that are identified as competencies required for the organization. Only 36Dollar360 provides the capability to incorporate the unique competencies of the organization in addition to the ten provided by the system. C4X provides twenty-eight competencies and all multi-rater assessments show eight competencies for leadership development. C4X provides four different assessments for leadership, management, high potential and women. If the competencies are not aligned between the organization and the multi-rater assessment than it is difficult to conclude that all data collected to support the leadership competency is relevant except in the case of 36Dollor360.

Questions are complex and pertaining to the role: The questions reviewed from the sample reports are complex. It is difficult to judge by looking at the question to identify if they are pertaining to the role without a proper analysis of the role.

Quantitative and Qualitative Data is captured: All the multi-rater assessments have feedback broken down by all raters and represent the quantitative feedback on a Likert scale between 5 and 7. The quantitative data is also captured in the form of comments and shown anonymously in the multi-rater assessments. In Checkpoint 360 the manager comment is not shown anonymously. A sample visual view of how the raters provided both quantitative and qualitative feedback on the leadership capacity is shown below:

Communications Feedback:

Figure 3: Quantitative and Qualitative feedback at competency level.

Feedback is constructively phrased All the multi-rater assessments provide good mechanisms to collect feedback. Leadership 363 provides the capability to select predefined text as comments which are not negatively phrased. In the other systems it is difficult to identify if the system has capability to check for constructively phrased feedback.

Raters are credible All the multi-rater assessments provide good mechanisms to segregate raters into categories like manager, peer, direct report, customer and others. It is difficult to identify if the system has any capability to check for rater data feedback to be credible.

A rater view is available in Checkpoint 360 that highlights how the raters provided the feedback for each competency as shown below.

Figure 4- Sample rater group comparison

Multi-rater assessment to be used with personality profile assessment: Wiley’s Leadership 363 multi-rater assessment incorporates Wiley’s DiSC personality profile assessment to provide a unique combined multi-rater assessment. This capability does not apply to the other three multi-rater assessments under review.

Goal setting is done as part of the assessment process is where Goal setting is identified as an important part of multi-rater assessment outcome on the website of all the multi-rater instruments under review. Goal setting to be documented as part of the assessment is not available in any of the assessments under review.

Figure 5: Personal development planning and personal development plan.

All members of the organization participate in the data collection are not part of the multi-rater system review. Literature review identified the cost of multi-rater assessments limits the use to senior leaders of the organization.

Collection of data is paired with mentorship is applicable for Checkpoint 360, Leadership 363 and C4X since the go to market for these multi-rater systems are through a coaching network implying there is a correlation between data collected and mentorship for these three multi-rater assessments.

There are consequences associated with goal setting objectives. This is not part of any of the multi-rater assessments that were reviewed.

Ranked Strength did not come up in the literature, yet it was one common feature in all the multi-rater assessments that were reviewed. The ranked rating for all the leadership capacities with comments would be a good input for personal development planning. Checkpoint 360 shows ranked strength with the priority identified for both aligned and not aligned leadership capacities.

Figure 6- Sample Highest to Lowest Rating Report

Future Studies

The identification of gaming industry competencies highlighted an opportunity for future research. Broader competencies were identified in literature, however more detailed related tasks could be studied. For example, as discussed technology was not identified as much as anticipated, however technology encompasses a wide range of tasks that would likely be addressed more often. Although gaming studio competencies align with a variety of industries, additional competencies could be identified by providing more research related to the gaming industry. This in turn can be realigned with the instruments discussed to show versatility in multi-rater assessments.

The design of a new generation of multi-rater assessment solutions should consider the twelve conditions for successful leadership development programs using multi-rater assessments. It should allow for customized leadership capacity development, incorporate the major personality profiles, manage personal development planning online, assist in constructive feedback and increase awareness on the credibility of raters.

Conclusion:

This literature review identified competencies for gaming industry, those that were directly identified in the literature and those that were indirectly identified. It was found that gaming industry founders have specific and unique skills related to game development, however, these skills are different from those required to run and operate a start-up. Gaming development founders focus on a specific skills related to game development and lack the skills needed to run a sustainable business. Targeted professional development can prepare game studio leaders to manage a more sustainable business. This requires a reliable instrument for analyzing competencies that need to be addressed.

Based on our research, multi-rater assessments can be a very useful instrument for this targeted sector, gaming industry. To implement any leadership development program for this or any sector, it is essential to identify requisite leadership capacities and assure that these are in alignment with the organization’s goals and values. Multi-rater-implementation requires a significant role in awareness from leaders and peers, before they can recognize their true strengths and weaknesses. This feedback provides leaders with the potential of changing their behaviors in a positive manner. The executive coach plays a key role in the success of the program, yet all implementations do not use an executive coach, which could play a role in the mixed success rates of leadership success rate. Our expectation from the new generation of multi-rater assessment system is to provide more control to the learner while the role of the coach becomes that of a facilitator. The multi-rater discoveries exemplified by gaming industry competencies will allow for leadership development in the gaming industry to benefit in a similar manner as other more established industries.

Bibliography

EBSCO Sources:

Archer, J. (2013). Feedback: it’s all in the CHAT. Medical Education, 47(11), 1059–1061. https://doi-org.libproxy.library.unt.edu/10.1111/medu.12308

Basu, M. (2020). Multi-source feedback: A tool for assessment. Medical Journal of Dr.

D.Y. Patil University, 13(4), 300–301. https://doi-org.libproxy.library.unt.edu/10.4103/mjdrdypu.mjdrdypu_143_19

Brotherton, P. (2012). 360 Instruments Are the Most Popular Way to Assess

Leadership. T+D, 66(8), 18.

Bywater, J., Stirling, E., & Bunyard, C. (2017). Predicting leadership derailment through

alternative means. Assessment & Development Matters, 9(1), 2–7.

Chrobot-Mason, D., & Leslie, J. B. (2012). The Role of Multicultural Competence and

Emotional Intelligence in Managing Diversity. Psychologist-Manager Journal (Taylor & Francis Ltd), 15(4), 219–236. https://doi-org.libproxy.library.unt.edu/10.1080/10887156.2012.730442

Connelly, B. S., Warren, R. A., Kim, H., & Di Domenico, S. I. (2016). Development and

Validation of Research Scales for the Leadership Multi-rater Assessment of Personality (LMAP). International Journal of Selection & Assessment, 24(4), 362–367. https://doi-org.libproxy.library.unt.edu/10.1111/ijsa.12154

Correia, M. C., dos Santos, N. R., & Passmore, J. (2016). Understanding the Coach-

Coachee-Client relationship: A conceptual framework for executive coaching. International Coaching Psychology Review, 11(1), 6–23.

Downs, L. J. (2012). INTEGRATED TALENT MANAGEMENT: Building a Strategy

One Block at a Time. T+D, 66(8), 42–47.

Fisher, Z. (2019). Everyone Gets a Vote: 360 Assessments and the Human Factors

System. Air & Space Power Journal, 33(1), 62–69.

Gallagher, D., Costal, J., & Ford, L. (2012). Validating a Leadership Model Pinpointed

Self-Awareness as Key to Success. T+D, 66(11), 50–54.

Gorman, C. A., Meriac, J. P., Roch, S. G., Ray, J. L., & Gamble, J. S. (2017). An

exploratory study of current performance management practices: Human resource executives’ perspectives. International Journal of Selection & Assessment, 25(2), 193–202. https://doi-org.libproxy.library.unt.edu/10.1111/ijsa.12172

Gorun, M., Kayar, İ., & Varol, B. (2018). 360-Degree Performance Appraisal

and Feedback System: A Study with Heads of Departments in Çanakkale Onsekiz Mart University. Gaziantep University Journal of Social Sciences, 17(4), 1425–1437. https://doi-org.libproxy.library.unt.edu/10.21547/jss.449154

Hafford-Letchfield, T., & Bourn, D. (2011). ‘How Am I Doing?’: Advancing Management Skills Through the Use of a Multi-source Feedback Tool to Enhance Work-based Learning on a Post-qualifying Post-graduate Leadership and Management Programme. Social Work Education, 30(5), 497–511. https://doi-org.libproxy.library.unt.edu/10.1080/02615479.2010.505263

Haigh, C. A. (2016). The 360° Performance Evaluation Tool. Fire Engineering, 169(7),

45–52.

Parmentier, G., & Picq, T. (2016) Managing creative teams in small ambidextrous organizations: The case of video games. International Journal of Arts Management, 19 (1), 16–30.

Phillips, J. J., Phillips, P. P., & Ray, R. (2015) Derive hard numbers from

soft skills. TD: Talent Development, 69(9), 54–59.

Schoepp, K., & Skuba, A. (2014). Effective Leadership Assessment: A 360-Degree

Process. (Cover story). Assessment Update, 26(2), 1–16.

Slaughter, M. (2011). Success at Suntrust begins and ends with talent . T+D, 65(11), 38–42.

Scholz, T. M. (2012). Talent Management in the video game industry: The role of cultural diversity and cultural intelligence. Thunderbird International Business Review, 54(6), 845–858, https://doi.org/10.1002/tie.21507.

Turner, J. R., Baker, R., Schroeder, J., Johnson, K. R. & Chung, C. (2018) Leadership development techniques: Mapping leadership development techniques with leadership capacities using a typology of development. European Journal of Training and Development; Limerick, 42(9), 538-557. DOI:10.1108/EJTD-03-2018-0022

Turner, J.R. & Baker, R. (2018) A review of leadership theories: identifying a lack of

growth in the HRD leadership domain. European Journal of Training and Development, 42(7/8), 470-498. https://doi.org/10.1108/EJTD-06-2018-0054

Vivekananda, S. P., MacKillop, L., Crossley, J., & Wade, W. (2013) Do assessor

comments on a multi-source feedback instrument provide learner-centred feedback? Medical Education, 47(11), 1080–1088. https://doi-org.libproxy.library.unt.edu/10.1111/medu.12249

Wachter, A., & Lion, R. W. (2016). Multi-Rater Feedback: Improving Students’

Readiness For the or. Techniques: Connecting Education & Careers, 91(3), 44–49.

JSTOR Database

Abrell, C., Rowold, J., Weibler, J., & Moenninghoff, M. (2011) Evaluation of a Long-

term Transformational Leadership Development Program. Zeitschrift Für Personalforschung / German Journal of Research in Human Resource Management, 25(3), 205-224. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/23279289 (article not found)

Baughman, J. A., Brumm, T. J., & Mickelson, S. K. (2012) Student Professional

Development: Competency-Based Learning and Assessment. Journal of Technology Studies, 38(2), 115–127. https://doi-org.libproxy.library.unt.edu/10.21061/jots.v38i2.a.6

Beehr, T., Ivanitskaya, L., Hansen, C., Erofeev, D., & Gudanowski, D. (2001) Evaluation of

360 Degree Feedback Ratings: Relationships with Each Other and with Performance and Selection Predictors. Journal of Organizational Behavior, 22(7), 775-788. Retrieved March 6, 2021, from http://www.jstor.org/stable/3649566

Berkovich, I. (2014) Between person and person: Dialogical pedagogy in authentic

leadership development. Academy of Management Learning & Education, 13(2), 245–264. https://doi-org.libproxy.library.unt.edu/10.5465/amle.2012.0367

Bracken, D., & Rose, D. (2011) When does 360-Degree feedback create behavior change?

And how would we know it when it does? Journal of Business and Psychology, 26(2), 183-192. Retrieved March 6, 2021, from http://www.jstor.org/stable/41474867

Caroline Bailey, & Clive Fletcher. (2002) The impact of multiple source feedback on

management development: Findings from a longitudinal study. Journal of Organizational Behavior, 23(7), 853-867. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/4093637

Dai, G., De Meuse, K., & Peterson, C. (2010) Impact of multi-source feedback on

leadership competency development: A longitudinal field study. Journal of Managerial Issues, 22(2), 197-219. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/20798905

DeNisi, A., & Kluger, A. (2000) Feedback effectiveness: Can 360-Degree appraisals be

improved? The Academy of Management Executive (1993-2005), 14(1), 129-139. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/4165614

Feldon, D., Maher, M., Hurst, M., & Timmerman, B. (2015). Faculty mentors’, graduate

students’, and performance-based assessments of students’ research skill development. American Educational Research Journal, 52(2), 334-370. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/24546757

Hodgson, D., & Briand, L. (2013). Controlling the uncontrollable: ‘Agile’ teams and illusions of autonomy in creative work. Work, Employment & Society, 27(2), 308-325.

Johnson, S. K., Garrison, L. L., Hernez-Broome, G., Fleenor, J. W., &

Steed, J. L. (2012). Go for the goal(s): Relationship between goal setting and transfer of training following leadership development. Academy of Management Learning & Education, 11(4), 555–569. https://doi-org.libproxy.library.unt.edu/10.5465/amle.2010.0149

Manz, C., & Sims, H. (1987) Leading workers to lead themselves: The external

leadership of self- managing work teams. Administrative Science Quarterly, 32(1), 106-129. doi:10.2307/2392745

Miller, J., & Cardy, R. (2000) Self-Monitoring and performance appraisal: Rating

outcomes in project teams. Journal of Organizational Behavior, 21(6), 609-626. Retrieved March 6, 2021, from http://www.jstor.org.libproxy.library.unt.edu/stable/3100392

Mollick, E. (2012) People and process, suits and innovators: The role of individuals in firm performance. Strategic Management Journal, 33(9), 1001-1015, https://doi.org/10.2307/23261314.

Toegel, G., & Conger, J. A. (2003) 360-Degree assessment: Time for

reinvention. Academy of Management Learning & Education, 2(3), 297–311. https://doi-org.libproxy.library.unt.edu/10.5465/AMLE.2003.10932156

Waldman, D., Atwater, L., & David Antonioni. (1998) Has 360 degree feedback gone

amok? The Academy of Management Executive (1993-2005), 12(2), 86-94. Retrieved March 6, 2021, from http://www.jstor.org/stable/4165460

Science Direct

Aisha, A. N., Sudirman, I., Siswanto, J., & Andriani, M. (2019) A competency model for SMEs in the creative economy. International Journal of Business, 24(4), 369-392.

Aleem, S., Capretz, L. F. & Ahmed, F. (2016) Empirical investigation of key business factors for digital game performance. Entertainment Computing, 13, 25-36, ISSN 1875-9521, https://doi.org/10.1016/j.entcom.2015.09.001.

Broekhuizen, T. L. J., Lampel, J. & Rietveld, J. (2013) New horizons or a strategic mirage? Artist-led-distribution versus alliance strategy in the video game industry. Research Policy, 42(4) 954-964, ISSN 0048-7333, https://doi.org/10.1016/j.respol.2012.12.007.

Burger-Helmchen. T, & Cohendet, P. (2011) User communities and social software in the video game industry. Long Range Planning, 44(5-6), 317-343, ISSN 0024-6301, https://doi.org/10.1016/j.lrp.2011.09.003.

Cabras, I., Goumagias, N. D., Fernandes, K., Cowling, P., Li, F., Kudenko, D., Devlin, S. & Nucciarelli, A. (2017) Exploring survival rates of companies in the UK video-games industry: An empirical study. Technological Forecasting and Social Change, 117, 305-314, ISSN 0040-1625, https://doi.org/10.1016/j.techfore.2016.10.073

Chiambaretto, P., Massé, D. & Mirc, N., (2019) “All for One and One for All?” – Knowledge broker roles in managing tensions of internal coopetition: The Ubisoft case. Research Policy, 48(3), 584-600, ISSN 0048-7333, https://doi.org/10.1016/j.respol.2018.10.009.

de Vaan, M. (2014) Interfirm networks in periods of technological turbulence and stability. Research Policy, 43(10), 1666-1680, ISSN 0048-7333, https://doi.org/10.1016/j.respol.2014.07.007.

Landoni, P., Dell’era, C., Frattini, F., Messeni Petruzzelli, A., Verganti, R. & Manelli, L. (2020) Business model innovation in cultural and creative industries: Insights from three leading mobile gaming firms. Technovation, 92–93, 102084, ISSN 0166-4972, https://doi.org/10.1016/j.technovation.2019.102084.

Lysova, E. I. & Khapova, S. N. (2019) Enacting creative calling when established career structures are not in place: The case of the Dutch video game industry. Journal of Vocational Behavior, 114, 31-43, ISSN 0001-8791, https://doi.org/10.1016/j.jvb.2018.06.004.

Panourgias,N. S., Nandhakumar, J., Scarbrough, H. (2014) Entanglements of creative agency and digital technology: A sociomaterial study of computer game development. Technological Forecasting and Social Change, 83, 111-126, ISSN 0040-1625, https://doi.org/10.1016/j.techfore.2013.03.010.

Shet, S.V., Patil, S.V. & Chandawarkar, M.R. (2019) Competency based superior performance and organizational effectiveness. International Journal of Productivity and Performance Management, 68(4), 753-773.

Other Sources

36Dollor360. (n.d.). Https://Www.Sviworld.Com/New-Page. Retrieved April 21, 2021, from https://www.sviworld.com/new-page

(n.d.). Retrieved March 29, 2021, from https://www.ddiworld.com/contact-us/2401/605521ba-94cb-45f0-9758-fdd2cf2ac23c

Best 360 degree feedback Software 2021: Reviews of the most popular tools & systems. (n.d.). Retrieved March 29, 2021, from https://www.capterra.com/360-degree-feedback-software/

Checkpoint360. (n.d.). Https://Www.Profilesincorporated.Com/Assessment/Checkpoint-360/. Retrieved April 21, 2021, from https://www.profilesincorporated.com/assessment/checkpoint-360/

Coaching For Excellence. (n.d.). Retrieved March 29, 2021, from https://www.c4x.com/

Everything+DiSC®+363+for+Leaders™ (John Wiley & Sons, Inc. – EverythingDiSC). (n.d.). Retrieved March 29, 2021, from https://www.trainingsolutions.com/Training-Product.asp?product_id=b363&title=Everything%2BDiSC%26reg%3B%2B363%2Bfor%2BLeaders%26%23153

Freifeld, L. (2020, December 17) 2019 training industry report. Retrieved March 29, 2021, from https://trainingmag.com/2019-training-industry-report/

Gilbert, B. (2020, December 23) Video-game industry revenues grew so much during the pandemic that they reportedly exceeded sports and film combined, Business Insider, Retrieved from https://www.businessinsider.com/video-game-industry-revenues-exceed-sports-and-film-combined-idc-2020-12

Koksal, I. (2019, November 8) Video game industry & its revenue shift, Forbes. Retrieved from https://www.forbes.com/sites/ilkerkoksal/2019/11/08/video-gaming-industry–its-revenue-shift/?sh=3045e439663e

L., E., H, K., P, J., & S, C. (2017, June 7). How big is the global “leadership development” market? Retrieved March 29, 2021, from https://askwonder.com/research/big-global-leadership-development-market-5r1furaib

Pages: 1 2 3

Archives

Meta