Abstract
Orientation: Organisations face productivity and efficiency challenges brought on by global pressure. To cope with the challenges, they seek to develop and enhance their human capital as a source of sustainable competitive advantage. Evidence suggests that less than 10% of what is learned on training courses is applied effectively to enhance performance and business results.
Research purpose: This abstract research critically examined existing training evaluation models to propose a new model.
Motivation for the study: Smart investment in scarce and critical skills development by means of training is expected to enhance human capital; however, the challenge lies with the uncertainty in whether the return on these investments are measured and whether training risks are managed.
Research design, approach and method: Theoretical, abstract research was conducted to understand existing measurement and evaluation models of training with regard to costs, benefits and risks.
Main findings: This conceptual paper resulted in a new business model to measure training return on investment and risks. The proposed model adapted and built on the Kirkpatrick-–Phillips training evaluation model, adding a sixth, risk evaluation step and specifying measurement factors for each step.
Practical and managerial implications: Training and line managers must note that although the evaluation of trainee’s satisfaction, learning, application, impact and financial return is imperative and must be measured, ignoring the measurement of risk factors such as learning barriers and challenges may jeopardise the ability of leaders and managers to predict how investments in human capital development will impact business results.
Introduction
Although management practices have evolved since before the industrial revolution, now more than ever, the roles a manager plays receive great emphasis. Drucker (1986:119) states that managers should spend 90% of their time working with and through people so that each individual is able to employ their own strengths to the collective benefit of the organisation. Taylor (1911:9, 2004) elaborates about overcoming inefficiency by addressing among others, the misfit between a worker’s capabilities and his or her job requirements. Taylor’s work is still appreciated by the 21st century managers as it is evident that management has the ability to increase organisational efficiency through their employees.
Middle managers are the managers who report directly to the top management team, and the quality of the interaction between these managers and their employees is central to successful strategy formulation and implementation (Raes et al. 2011:102). Due to their potential as top-level managers of the future, more research is required on their current leadership performance, whether it could be enhanced through management training and whether that training translates into business results.
Human capital accumulates through experience and education, improves productivity and is more relevant in the higher echelons of organisations, such as middle and top management (Kai Ming Au, Altman & Roussel 2008:19). More recently, business executives admitted that global competition is intensifying and is linked to the capabilities of competitors to enhance their knowledge, expertise and talent (Alexandros & Bouris 2008:63). There is in fact no shortage or literature on the enhancement of productivity and efficiency through the enhancement of talent, skills and knowledge. However, the rate of return on the investment in training is not always measured, and empirical evidence related to the topic has produced mixed outcomes (Kai Ming Au et al. 2008:19).
The relationship between training and work performance is both supported and criticised by researchers. Bernthal and Wellins (2006:32) found that training impacts return on investment (ROI) and return on equity (Meyer 2007b). Training also had an impact on non-financial indicators such as customer satisfaction and reduced employee turnover (Lawler, Mohrman & Ledford 1998). Different corporate users have different perspectives and motives and therefore different perceptions of the value added by training (Hipwell 2005:37). Although training may have positive effects on individual and organisational performance, the effect may be quite different in different sectors. Some researchers used subjective research methods, others experienced low response rates and therefore results may be methodologically flawed possibly resulting in a gap in the body of knowledge on this particular topic (Babbie & Mouton 2011; Nguyen, Truong & Buyens 2010:41).
While a high-calibre workforce is a driving force in the attainment of an organisation’s objectives, the employee’s job satisfaction, motivation and commitment to learning are central to skills and competency enhancement (Alexandros & Bouris 2008:64). Organisations often fail to explain the personal benefits from training that may befall the individual employee, mostly because it is assumed that employees know it already (Latif 2012:213). It was found that training practices have enhanced employee motivation and commitment to their organisation. Many employees show appreciation for the organisation’s investment in their development, indicating that they do perceive value from the training and in turn become more motivated, committed and productive (Alexandros & Bouris 2008:64).
Although it appears that organisations are aware of the internal and external pressure to continuously improve the skills of their employees to remain competitive, keep their employees motivated, or align with changing circumstances, what is not clear, is whether managers and leaders are measuring the ROI and the risks involved with that improvement. The objective of this conceptual paper is to present a business model for managers and leaders to measure ROI and risk in training. The contribution of this paper is the theoretical and practical value added. On a theoretical level, the paper adds to the body of knowledge on training evaluation, especially since the training measurement model presented expands on the Kirkpatrick–Phillips training evaluation model. On a practical level, the paper offers business managers and leaders proper measurement factors that can be analysed at each of the six steps of the model.
Literature review
Training through the ages
Among the challenges brought about by the industrial revolution was a cultural revolution in the lives of workers and entrepreneurs who faced the complexities of the market system, assembly lines and the onset of technology. The entrepreneurial factory owners are needed to find both capital and capable managers who could manage the growing workforce and other production factors (Wren 2005:66). Taylor (1911:9, 2004) held that management should have the maximum prosperity for the organisation and its employees as their main object. As a theorist of Scientific Management, Taylor’s emphasis was on the employers, whereas Hugo Munsterberg, a behavioural scientist, emphasised the psychological conditions that impacted productivity and how best employees could be shaped to fit the organisation’s needs (Shafritz, Ott & Jang 2005:152). During the development of the behavioural management theory, Elton Mayo proved through experimentation that a concerted effort by management to increase human motivation and satisfaction would have a simultaneous increase on productivity. By making employees feel like they had a valued role in business operations, Mayo and his group of colleagues observed that psychological and social factors play a bigger role in the success of the workplace than physical factors such as working conditions (Bateman & Snell 2011:40).
Therefore, if leaders, managers and employees are to successfully meet the evolving needs of the organisation, their personal development plan (PDP) must form an integral part of the strategy of the organisation. Employee training and development are integrated into management development. Job content training is needed for all levels of management although more training in leadership behaviour is needed as managers progress through their careers. A different set of development methods may also be employed for different levels of employees. Training may be formal or informal, on the job or off the job (Erasmus et al. 2011:229).
By training their employees, organisations attempt to modify their knowledge, skill and behaviour with the intention to improve their performance and ability to contribute to the organisation achieving its goals. Training includes learning activities often dictated by the tasks the employee needs to perform (Erasmus et al. 2011:2). The knowledge and skills mastered during education and work experience build human capital, which, particularly in higher organisational echelons, significantly impacts productivity (Kai Ming Au et al. 2008:19).
Training and development consist of activities that prepare leaders, managers and employees to manage subordinates, increase performance and implement strategies towards business goal achievement. It requires interrelated elements such as vocational and task-specific training, self-development and enterprise support (Erasmus et al. 2011:234). Research conducted in China found that investment in middle management training programmes, in lieu of formal degree programmes, yields the most appropriate ROI. Lower skilled employees are easily replaceable and are seen as unworthy of such investment. The intermediate-level employees and junior managers are expected to advance to senior levels and, if properly trained, they will be more equipped to contribute to organisational productivity and to deal with complex business issues of the future (Kai Ming Au et al. 2008:26).
The extent of learning through training is also positively related to organisational performance (Alexandros & Bouris 2008:66), making the extent to which human capability is enhanced during training interventions particularly important (Choi & Dickson 2010:104). Erasmus et al. (2011:104) stated that learning is planned by the learner. By implication, the learning experience is a personal one, making the rate of knowledge processing, the choice of what to learn and the value placed on it a private journey. Therefore, while the individuals may value learning through training as a means of increasing their personal effectiveness, career mobility, market value and job satisfaction (Latif 2012:212), organisations value learning through training by the creation of high-quality human capital that will improve productivity and competitive capability (Alexandros & Bouris 2008:68). In fact, in the absence of a precise measure of ROI, other indicators of value become more important. These include a manager’s own opinion of his growth, efficient use of resources and the extent to which new knowledge is applied to job situations (Goldstein, Gorman & Smith 1973:11; Langer, Bapna & Gopal 2014; Meyer 2007a, 2007b).
Human capital development evaluation models and techniques
Noe (2002) said that developing and training the human capital of an organisation are essential for its progress. A training infrastructure as a means of developing people and giving them opportunities for growth thereby enhancing organisational capacity is an important predictor of an organisation’s success (Jacobsen, Rubin & Colemen-Selden 2002:486). Noe (2010) further argued that the results of the training intervention are no good if it is not evaluated upon completion. As trained employees become in greater demand and they search for better opportunities, the employers’ ability to retain these employees becomes vulnerable and so is its ability to meet strategic objectives. Once a certain skill level is mastered, the job holder may experience a decline in job satisfaction, which strengthens the case for sustained developmental activities to retain trained employees (Shepherd, McKay & Bowie 2010:369).
Therefore, continued human capital development and skills enhancement are necessary actions that contribute to organisational success; yet, unabated spending on it is not sustainable and organisations cannot afford to pay for such training without evaluating the contribution made to organisational results (Farjad 2012:2838). Therefore, for both employer and employee, the knowledge that a training intervention achieved its objectives in the sense of changed behaviour, knowledge imparted widely and impact on workplace objectives is vital (Kennedy et al. 2013:2). This knowledge can only be collected through effective training evaluation.
In South Africa, there is much emphasis for the human resource function in organisations to drive training and development (a national priority), although the profession is criticised as having too little business insight to contribute meaningfully to organisational direction setting (Erasmus et al. 2011:280). Traditional bureaucracies in South African organisations have been challenged to transform from a paradigm of exclusivity to inclusivity and towards greater participation, delegation, trust and transparency for all South Africans (Francis 1996:117). Yet, the empowerment paradigm can only be successful in the face of strong support from the top, commitment from all involved, and a shared corporate vision (Francis 1996:93).
Whichever means are used to conduct the training, whether in-service, off the job or organisational development intervention, accurate and relevant evaluation techniques and measuring systems are needed to obtain information, control quality and manage transfer barriers (Erasmus et al. 2011:216; Welman, Kruger & Mitchell 2010; Welman, Mitchell & Kruger 2005). Training evaluation is needed to ascertain whether the individual benefitted from it, whether he or she could impart his or her newly acquired knowledge to his or her team members and whether the organisation’s money was well spent (Erasmus et al. 2011:215).
Yet, human capital development efforts frequently fail to achieve their intended objectives. Spitzer (2007:21) suggested that training in organisations is often characterised by a lack of innovation, coupled by assumptions that the duration and location of the training intervention have a bearing on its success. According to Erasmus et al. (2011:223), training fails for reasons ranging from insufficient rewards for changed behaviour to insufficient management and financial support for the training programme. While Spitzer (2005b:55) referred to a crisis in the profession of human resources (HR) as HR practitioners have failed to demonstrate material business results flowing from costly investments in training, Mattson (2000:466) stated that the credibility of HR practitioners is at stake and that training budgets may be cut if the efficiency of training programmes is not evaluated and verified.
This evidence points to the fact that the enhancement of the human factor through training and development, more than any other organisational resource, is a vital component of business effectiveness and efficiency. While it appears that there is an awareness of the external pressure to continuously improve the skills of their employees, it remains to be seen whether organisations become competitive or align themselves with changing circumstances. What is not clear, as well, is whether leaders and managers are measuring the improvement in performance resulting from training and development or have any means to measure and manage the risks associated with training transfer barriers after training has occurred.
The Kirkpatrick–Phillips training evaluation model
Many top performing American and British organisations use the Kirkpatrick–Phillips five-level training evaluation model to measure the effectiveness of their training programmes on organisational outcomes (Medina et al. 2015:126). Initially, a four-level model was developed by Kirkpatrick as part of his doctoral dissertation in 1959; this model was later improved upon by Jack Phillips who added a fifth level (Kennedy et al. 2013:2). The levels and the description of the levels of the Kirkpatrick–Phillips training evaluation model are provided in Table 1.
| TABLE 1: The Kirkpatrick–Phillips training evaluation model. |
According to Phillips (2013:2), the Kirkpatrick–Phillips training evaluation model is widely used and can be coupled with an online forecasting decision tool even before the investment in a training intervention is made. This is a good indication of the benefits side of the ROI calculation (Langer et al. 2014; Meyer, Opperman & Dyrbye 2003). This model measures training outcomes at five levels, presented in Table 1 as follows: Level 1: satisfaction with training; Level 2: learning from training; Level 3: application of training; Level 4: impact of training; and Level 5: return on investment or ROI.
The various stakeholders in an organisation’s training initiatives, such as the participants, human resources practitioners, managers and leaders, may have different expectations about the purpose and anticipated outcomes of a training programme. To measure whether their expectations were met, an objective set of criteria must be used. However, it is not clear if all organisations measure the impact of training, although they may have the desire to do so (Bernthal 2012:4).
Mattson (2000:465) agreed and stated that although training evaluation is critical in a time of limited resources, competing projects and pressure to spend budgets prudently in a globally competitive trading arena, organisations do not always automatically evaluate the effectiveness of all projects embarked upon. Reasons cited are the complexity of the training evaluation or difficulty to prove a causal relationship between money spent on training and actual improvement of business results or a real impact on the bottom line (Spitzer 2005a:55). The most obvious reason for evaluating training is to measure its value or worth (Kennedy et al. 2013:3). Organisations investing valuable resources in improvement projects, whether for employee efficiency or skills enhancement, are now insisting on the evaluation of the impact of such programmes (Russ-Eft & Preskill 2005:71).
Research conducted at NASA (North American Space Agency) tested various aspects of value created by training. Although no precise measure was found to quantitatively assess ROI, aspects such as own opinion of changed performance, the type of job, a supportive line manager and familiarity with the instructor were considered by participants as aspects that maximised the course value (Goldstein et al. 1973:13).
The Kirkpatrick–Phillips model, although widely used, is not free from criticism. To this end, although it is widely accepted that learning impacts performance and ultimately business objectives, even when the Kirkpatrick–Phillips model is applied, it is only the lower levels that are reported on (Spitzer 2005b:32). It is estimated that close to 90% of training interventions are evaluated in terms of the reactions of the participants, their satisfaction and learning (ASTD 2011) and barely in terms of business value. Mattson (2005:118) reported that the limited research on the topic of evaluation has found that management is more interested in financial results than employee feedback. Furthermore, according to Brinkerhoff (2006:304), training alone cannot be responsible for its success (or failure for that matter), and evaluation is really based on the performance management system in the organisation, not the training programme only. Finally, not all training benefits can be translated into financial outcomes, and managers focusing only on the financial results may overlook other (difficult to quantify) benefits (Russ-Eft & Preskill 2005:72).
It is, however, possible and feasible to attempt to calculate ROI, according to Hipwell (2005:39). ROI is defined and perceived differently by different stakeholders; however, it is possible to monetise the benefits from training, and a simple calculation could quantify the value. Hipwell (2005:40) also suggested the Kirkpatrick–Phillips model for evaluating training. This model can be used to forecast the potential pay-off before the investment is made (Bailey 2002:11). ROI or the benefits–cost ratios use the costs of the training intervention as a factor of the benefits. Although the cost element of the training programme is easy to arrive at, it is the benefit, value and risk that are subjective and difficult to quantify (Phillips 2013:9).
Two critical steps suggested for calculating such benefits are data collection and data analysis. During data collection, methods such as observation and focus group interviews involving the training participants and their line managers should be used. Data collection should take place during and after the intervention, in the form of follow-up surveys and questionnaires (Bailey 2002:6; Hipwell 2005:38). During data analysis, estimation, isolation and adjustment are suggested as three further steps to make a clear distinction between the impact of the training intervention and other influences on changed performance-related behaviour. Firstly, estimates of changes in performance must take into consideration the influence of people, processes, markets, technology and the training itself. Secondly, by isolating the estimated improvement in performance through only training, the training recipient is asked to (subjectively) estimate how much the training contributed to his or her performance, relative to other factors. Thirdly, to counter the trainee participant’s subjectivity, a third party adjusts the ROI calculation (Hipwell 2005:39).
Using this model, a number of organisations have reported mixed feedback on investment in training, including a 285% return on management training (Bailey 2002:11). Perhaps there are limitations to this model that can be modified to add more value to the measurement of training effectiveness as follows:
- Identify the relevant measurement factors and analytics that can be gathered at each level.
- Identify and quantify the risks that occur before, during and after training that could prevent the effective transfer of training into individual and organisational performance improvement.
Critique: Comparing human capital development evaluation models
Zinovieff and Rotem (2008:12) also wrote about the Kirkpatrick training evaluation model and described the four levels of training evaluation: (1) reaction, (2) learning, (3) behaviour and (4) organisational results. It was found, however, that less than 15% of organisations measured the effect of training on organisational results, including business and financial results (Mattson 2000:466). Furthermore, less than 10% of what is learned during off-the-job training is applied back at work (Latif 2012:215). If success can be measured by the extent to which desired outcomes are met, management should have a set of quantifiable desired outcomes for any training arranged or training provider selected (Dawe 2003). These outcomes need to be negotiated and contracted with the external training provider. The trainer’s ability to achieve efficacy is vital and ranges from the ability to communicate to the extent to which he or she facilitates the transfer of knowledge, right down to his or her contribution to the organisation’s achievement of desired success from the training (Spitzer 2007). Failure to communicate these expected outcomes may result in the training provider concentrating more on favourable participant ratings, and too little on the other aspects of learning, change and organisational impact (Zinovieff & Rotem 2008:14). Although training is primarily offered with the aim of improving skills, an increase in job satisfaction (and employee retention) may be an unintended side effect (Latif 2012:215) and may increase the impact of the training intervention. Spitzer (2005a:57) agreed that the emphasis should not be on justifying the learning from training, but rather on the impact on all the organisation’s stakeholders.
In the Jack Phillips Return on Investment (ROI) Model (Phillips & Phillips 2002), Phillips drew attention to the weakness of the Kirkpatrick model and its inability to calculate the costs and benefits (ROI) of the training and introduced this on a fifth level. The critique of this contribution by Phillips is that evaluation needs to draw on level 4 (overall business and financial results) and express these results in monetary terms before comparing it to the cost of the training. If these results are flawed, the ROI calculation is flawed (Zinovieff & Rotem 2008:14).
Other studies conducted found significant correlation between satisfaction with the training session, training content, the trainer and the rate of transfer of learning (Latif 2012:218; Meyer et al. 2003). Learning as a measure of training effectiveness is in fact the Level 2 outcome of the Kirkpatrick–Phillips model. However, given Harrison’s (2002) theory that learning triggered by training contributes to organisational success, adopting a training activity may not always solve poor organisational performance challenges (Alexandros & Bouris 2008:66; Meyer 2007b).
There are many other goal-based and system-based models written on, tested and used; however, the Kirkpatrick model remains the most popular and widely used. Perhaps a more recent approach, Contribution Analysis, is worth mentioning here. This evaluation technique has recently been applied in Australia and uses outcome-based monitoring and evaluation. However, Contribution Analysis is complex and is best used in conjunction with other models (Zinovieff & Rotem 2008:24).
From the above discussion, it is apparent that it is not uncommon for organisations to invest in employee development and training with the aim of improving their overall performance and ability to compete successfully. What is uncommon, however, is a set of indicators that enable the investing entity to measure the value and the risk of the training transfer. Literature abounds about the need to train managers and employees; yet, many projects initiated by South African organisations to develop leadership, management and employee capabilities have failed for a number of reasons (Naidoo & Xollie 2011:2). Public institutions are criticised for reasons ranging from bureaucratic time lags, lack of accountability (Rehman, Khan & Khan 2011:40) to ineffective leadership and governance (Naidoo & Xollie 2011:2). The fact that the value created by the training interventions is difficult for training practitioners to measure further exacerbates the criticism that current training models and frameworks are ineffective (Erasmus et al. 2011:280).
The aforementioned research has produced measuring instruments that may have been effective to achieve the objectives of training research in the last three to four decades; however, a customised, modified training evaluation model is required to not only benefit the unique South African work situation but also enhance the working relationship between the trainee and the organisation, training provider, leaders, managers and peers. Furthermore, if employees and managers are to benefit from training and development interventions on a personal and professional level, the risks, challenges and barriers associated with effective training transfer must be measured together with the measurement of satisfaction, application, costs and benefits.
Training return on investment and risk measurement model
The aim of this paper is to propose a new, expanded training evaluation model to incorporate actual training measurement steps and factors that calculate more than just the cost–benefit analysis, levels of trainee satisfaction and training effectiveness than the Kirkpatrick–Phillips model (Bailey 2002; Meyer 2007b) does. The proposed model is called the Training ROI and Risk Measurement Model. The Training ROI and Risk Measurement Model is designed to be a strategic tool for leaders and managers, especially training managers, to effectively and efficiently employ metrics when managing training.
This model consists of six steps to measure specific evaluation criteria in the form of questions posed at each step. The model evaluates and measures these factors: trainee performance gaps and intrinsic motivation (Step 1); trainee learning prior to and after training (Step 2); trainee performance prior to and after training (Step 3); organisational growth capabilities prior to and after training (Step 4); training costs and benefits (Step 5); and risk factors that prevent effective training transfer into performance and organisational improvement (Step 6). The steps, measures and evaluation criteria for the six-step Training ROI and Risk Measurement Model are presented below. This is followed by a description of the two key features of the conceptualised model: the measurement factors for each step and the risk feature of Step 6.
Step 1, the Personal Development Plan (PDP) and Motivation step, evaluates whether the need for training was established at performance appraisal phase due to a performance gap and that the training was agreed upon by both an employee and a manager. This step immediately ensures that the training intervention is selected to improve job performance. Furthermore, with buy-in from the employee, he or she is stimulated and motivated not only to attend training but also to benefit from it so that he or she improves his or her performance. The aim of this step is to measure whether employees are intrinsically motivated to develop enhanced skills despite the many external or extrinsic forces that may impact successful learning and performance improvement.
Step 2, the Learned Behaviour step, evaluates whether the trainee employee improves his or her knowledge, skills, values and attitude after training compared to prior behaviour before training. If the need for training was established correctly due to a performance gap and the correct training intervention was selected to close this gap in Step 1, then the behaviour of the employee must change after training.
Step 3, the Applied Behaviour step, evaluates whether the learned behaviour in Step 2 is effective and allows the employee to apply the learning to improve his or her performance. Measuring the applied behaviour after training and comparing it to the applied behaviour before training allows employees, managers and trainers to evaluate whether the training intervention was relevant and effective in plugging the performance gap.
Step 4, the Business Impact step, evaluates whether the employee performance improvement measured in Step 3 contributes to achieving the business strategic goals. The employee, manager, organisation and training provider benefit if the answer to the evaluation question in this step is positive. However, if the answer is negative, the possible and probable causes may be identified by reviewing and reflecting on Steps 1–3.
Step 5, the ROI Calculation step, evaluates whether benefits of training exceed the costs of training. If the answer to this evaluation criteria question is positive, then all stakeholders have benefitted from the training intervention. If the answer is no and the ROI calculation statistic is negative, then Steps 1–4 must be reviewed to identify and eradicate the cause. This step of the model allows all stakeholders to authentically evaluate the evidence and determine whether the training intervention resulted in improved performance, capability and ROI.
Step 6, the Risk Management step, evaluates whether any specific barriers prevent trainees or employees from improving their performance after training and hence hinder the achievement of business strategy and growth. This step highlights the fact that training stakeholders (employees, managers, organisations, training providers, sectors, economy and communities) are impacted by and impact the results of training interventions. Identifying risks that could present challenges to a positive ROI involves close scrutiny of the factors in Steps 1–5 of this model. All identified risks should be quantified, managed and minimised or eliminated. This step allows employees, managers and trainers to be aware of and take appropriate action to control risks that may occur as unforeseen incidents and events that prevent positive results in Steps 1–5.
Table 2 presents the Training ROI and Risk Measurement Model as proposed by the authors.
| TABLE 2: Training return on investment and Risk Measurement Model. |
A key feature of the Training ROI and Risk Measurement Model is the measurement factors listed for each of the six steps in the model. Step 1 measures factors such as whether the need for training was established due to a performance gap and whether the employee is motivated to achieve his or her PDP using training and development interventions. Furthermore, Step 1 measures whether the external factors, such as SETA (Sector Education and Training Authority), grants for training for an unaccredited training provider, positively or negatively impact the training intervention.
Step 2 measures whether the competencies, behaviour and engagement factors improved after the training event. Step 3 measures the factors related to whether the employee applied the learned behaviour to improve his or her work performance. Furthermore, Step 3 measures factors related to whether the employee displays improved responsibility and accountability in his or her job, team and projects. Step 4 measures the factors related to whether the employee performance improvement translates into the achievement of the business strategic goals. Furthermore, Step 4 measures factors linked to whether organisational capabilities improve or deteriorate after the training intervention.
Step 5 measures the cost and benefit factors in training, that is, whether the tangible and intangible benefits or results exceed the soft (time away from production or operations) and hard (registration fees) costs associated with the training intervention. The difference between hard (tangible) and soft (intangible) costs is that hard costs are easily accounted for, whereas soft costs are more hidden. Step 6 measures whether the risk factors identified before, during and after training, especially those associated with effective training transfer, do in fact present barriers to the successful transfer of learning into improved performance and organisational achievements.
Another key feature of the Training ROI and Risk Measurement Model is the risk element itself. According to Schoeman and Vlok (2014:98), risk is the result of the likelihood of an event and its consequences. Risk results from uncertainty and it may lead to unmet objectives (International Standards Organization [ISO] 2011). Globally, organisations are facing more and greater risks, yet research proves that most do not have a formal enterprise risk management programme (ERM). Regulators and stakeholders alike are demanding a proactive risk management in the shape of a formal ERM programme (Controller 2015:12). Proactively managing enterprise risk may contribute to organisational growth when risk management becomes a factor before entering into business transactions (Quirin 2015:7).
In the context of training, the constant need for innovation introduces the risk of employees that lack the appropriate skills and competencies to think innovatively or embrace changes in technology (Walker 2005). A further risk is that the skills of older workers may become obsolete, resulting in lower levels of productivity as technological changes demand new skills from them (Dickerson & Green 2004). A need, therefore, arises to improve employees’ skills to reduce the risk of loss of productivity. Training employees is a controllable organisational activity with which managers can reduce enterprise risk (Quirin 2015:12).
Therefore, in the context of this research paper, risk in training translates into the risk of unmet objectives; knowledge learned on training courses reaching and benefiting the organisation at large; and knowledge translated into individual and organisational performance improvement.
This section of the paper introduced the proposed Training ROI and Risk Measurement Model by outlining the measures and evaluation criteria for Steps 1–6 and highlighting the two key features of the model, the measurement factors for each step and the risk element of the sixth step. The next section presents a discussion of the implications for leaders and managers, especially ETD (education, training and development) and HRD (human resource development) managers, when implementing this model.
Implications for leaders and managers
The use of the Training ROI and Risk Measurement Model empowers leaders and managers to employ metrics and analytics to ensure that training delivers and impacts business results positively in the short, medium and long terms. Besides accuracy in training need assessment, managers must ensure that employees learn and apply new skills to improve performance and contribute to organisational growth. Furthermore, leaders and managers must calculate the ROI of each and every training intervention so that investments in employee development pay off. Moreover, leaders and managers who use predictive analytics to determine what risks can be managed to mitigate effective skills transfer and positive ROI are better able to achieve business objectives and strategy.
According to Purser (2004:542–546) and Sumner (2000:320), ROI is achieved through risk mitigation and must be included as a factor in the ROI calculation. Tactical, operational and strategic initiatives must be implemented to counteridentified risks in the short, medium and long term after training to ensure that training barriers are eliminated and that trainee performance does improve (Meyer 2007a, 2007b). Employees who are risk tolerant benefit more from training than less risk- tolerant employees. Furthermore, employees who prefer workplace autonomy are more risk tolerant, more innovative and benefit more from training in the short term (Fairlie & Holleran 2012:370).
Ineffective risk identification and management can jeopardise strong business models, processes, structure and leadership strategy (Drew, Kelley & Kendrick 2006:131). There is an urgent need for the leaders and managers of today to manage potential, unforeseen and unintended surprises that could cause effective training to fail by undertaking responsibility for risk intelligence (Apgar 2013; Aven & Krohn 2014:8).
The implications for leaders and managers is that the Training ROI and Risk Measurement Model forces a paradigm rethink, and a complete reengineering and reinvention of how they previously or currently measure and evaluate the effectiveness of training interventions. There is a dire need for managers to shift their training strategy to move beyond the evaluation of trainee satisfaction, learning, application and skills transfer. In fact, they have to think outside the box even about their ROI calculation and its impact on business results. Furthermore, if training risks are ignored or even downplayed, this could have negative implications for effective training transfer, and training barriers could be overlooked to the detriment of positive ROI and business strategic goal achievement. Leaders and managers of both the training provider and the customer organisations are urged to measure individual skills gaps, performance improvement, ROI and risks before, during and after training events. In this way, trainees and employees are treated and managed as human capital or assets that must require sustainable protection, evaluation and validation.
Labour mobility into, within and out of South Africa (SA) (Dainty 2008; Davids & Esau 2012) creates a war for talent. As a result, SA leaders and managers are forced to create flexible, multiskilled talent pools and invest heavily in talent engagement and retention strategies. Talent pools assist to mitigate against recruitment and development costs, especially for scarce (short supply) and critical (core to business) skills (Meyer 2007b; Schutte & Barkhuizen 2016). Employing the Training ROI and Risk Measurement Model to manage performance, engage and retain talented employees benefits the business, sector, country, continent and global economy through effective and efficient measurement and evaluation practices. Furthermore, talent retention can become a planned, phased, successful business process using the proposed model. Managers and employees should authentically, fairly and reliably evaluate their own training needs, learned behaviour, applied behaviour, training costs versus benefits and potential risks to effective performance transfer. In this way, they would feel encouraged and validated before, during and after training events. This engagement allows talented employees to create their own employee value proposition, and commitment to the growth of the organisation.
Finally, executive management, business unit managers and financial managers could prevent major downturns in business strategy just by measuring the factors of the six steps in the proposed Training ROI and Risk Measurement Model. Leaders and managers should alert and encourage trainers, facilitators, developers and educators to use this training measurement and evaluation tool, especially if they are responsible and accountable for training strategy, planning, implementation, budget and ROI.
Conclusion
This conceptual paper addresses the issue of whether businesses employ the relevant measurement model to assess the ROI and risk in performance improvement after training. A detailed review of the literature on training through the ages, training in the SA context and measuring training ROI was presented. Although the five-level Kirkpatrick–Phillips training evaluation model measures trainee’s satisfaction, learning, application, business impact and ROI, and was relevant in the past few decades, it has to be modified if current businesses are to measure ROI and risk factors that impact on the successful, effective transfer of training into individual and business performance results.
The African context requires a unique training measurement model to mitigate the risks of a lack of training application, insufficient manager support and insufficient time for applying new behaviours after training interventions (Meyer 2007b; Schutte & Barkhuizen 2016). This paper proposes the Training ROI and Risk Measurement Model as a solution for businesses to maximise their ROI and minimise their risks on training and development initiatives. The implications for business leaders and managers are that if they identify the risks, quantify them, manage them well and eliminate them, then the ROI on training will be high; however, if risks are ignored, ROI will be low or non-existent, and talent and business performance will stagnate. This could lead to failed business, training and personal development strategies. Measuring ROI without measuring, managing and monitoring risk is tantamount to expecting strategic goals to be achieved without capacity or resources.
Recommendations for further research include empirically testing the proposed Training ROI and Risk Measurement Model by making it available to key stakeholders in the public and private sectors of the economy. Use of the model in daily practices to measure the effectiveness of training interventions will force training providers, managers and leaders to ensure that training impacts positively on performance improvement, as well as on training and business results.
Acknowledgements
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
Both authors contributed equally to the study.
References
Alexandros, G. & Bouris, J., 2008, ‘Employee perceived training effectiveness relationship to employee attitudes’, Journal of European Industrial Training 32(1), 63–76, viewed 24 April 2015, from https://doi.org/10.1108/03090590810846575
Apgar, D., 2013, Risk intelligence: Learning to manage what we don’t know, Harvard Business Press, Boston, MA.
ASTD, 2011, ‘Overcome the three toughest challenges to an ROI study’, in ASTD International Conference and Exposition, viewed 15 May 2016, from www.astdconference.org
Aven, T. & Krohn, B.S., 2014, ‘A new perspective on how to understand, assess and manage risk and the unforeseen’, Reliability Engineering & System Safety 121, 1–10. https://doi.org/10.1016/j.ress.2013.07.005
Babbie, E. & Mouton, J., 2011, The practice of social research, Oxford University Press, Cape Town.
Bailey, A., 2002, The Kirkpatrick/Phillips Model for Evaluating Human Resource Development and Training, viewed 15 May 2016, from www.businessGurus.ca
Bateman, T.S. & Snell, S., 2011, Management: Leading & collaborating in a competitive world, McGraw-Hill/Irwin, New York.
Bernthal, P., 2012, Measuring the impact of training and development: A white paper, Development Dimensions International. Inc, Washington, USA.
Bernthal, P. & Wellins, R., 2006, ‘Trends in leader development and succession’, Human Resource Planning 29(2), 31–40.
Brinkerhoff, R.O., 2006, ‘Increasing impact of training investments: An evaluation strategy for building organizational learning capability’, Industrial and Commercial Training 38, 302–307. https://doi.org/10.1108/00197850610685824
Choi, Y. & Dickson, D.R., 2010, ‘A case study into the benefits of management training programmes: Impacts on hotel employee turnover and satisfaction level’, Journal of Human Resources in Hospitality & Tourism 9, 103–116. https://doi.org/10.1080/15332840903336499
Controller, 2015, Homepage, viewed from www.controller.iofm.com
Dainty, P., 2008, ‘Attracting and retaining staff in an era of skill shortages’, The Melbourne Review 4(1), 26–41.
Davids, B. & Esau, M., 2012, ‘The prevailing skills crisis in South Africa: An exploration of the skills development strategy of the city of Cape Town’, Journal of Developing Societies 28(1), 81, viewed 8 June 2012, from http://jds.sagepub.com/content/28/1/81
Dawe, S., 2003, Determinants of successful training practices in large Australian firms, National Centre for Vocational Education Research, Adelaide.
Dickerson, A. & Green, F., 2004, ‘The growth and valuation of computing and other generic skills’, Oxford Economic Papers 56(3), 371–406. https://doi.org/10.1093/oep/gpf049
Drew, S.A., Kelley, P.C. & Kendrick, T., 2006, ‘Five elements of corporate governance to manage strategic risk’, Business Horizons 49(2), 127–138. https://doi.org/10.1016/j.bushor.2005.07.001
Drucker, P.F., 1986, The frontiers of management, Mackays of Chatham Ltd., London, UK.
Erasmus, B.J., Loedolff, P.V.Z., Mda, T.V. & Nel, P.S., 2011, Managing training and development, 5th edn., Oxford University Press, Cape Town, South Africa.
Fairlie, R.W. & Holleran, W., 2012, ‘Entrepreneurship training, risk aversion and other personality traits: Evidence from a random experiment’, Journal of Economic Psychology 33, 366–378. https://doi.org/10.1016/j.joep.2011.02.001
Farjad, S., 2012, ‘The evaluation effectiveness of training courses in University by Kirkpatrick Model (case study: Islamshahr University)’, Procedia – Social and Behavioral Sciences 46, 2837–2841. https://doi.org/10.1016/j.sbspro.2012.05.573
Francis, C.V., 1996, ‘The importance of training in the hotel industry’, unpublished master’s thesis, Oklahoma State University, Stillwater, Oklahoma, USA.
Goldstein, S., Gorman, J. & Smit, B.B., 1973, ‘A partnership in evaluation’, Training and Development Journal 27(4), 606–637.
Harrison, J.K., 2002, ‘The induction of newly qualified teachers in secondary schools’, Journal of In-Service Education 28(2), 255–276. https://doi.org/10.1080/13674580200200206
Hipwell, W., 2005, Value creation and measurement in learning environments, viewed 15 May 2016, from www.clomedia.com
International Standards Organization (ISO), 2011, ISO 31000, principles and guidelines, Geneva, viewed 15 May 2016, from http://www.iso.org/iso/home/standards/iso31000.htm
Jacobsen, W., Rubin, E.V. & Colemen-Selden, S., 2002, ‘Examining training in large municipalities: Linking individual and organisational training needs’, Public Personnel Management 31(4), 485–506. https://doi.org/10.1177/009102600203100405
Kai Ming Au, A., Altman, Y. & Roussel, J. 2008, ‘Employee training needs and perceived value of training in the Pearl River Delta of China: A human capital development approach’, Journal of European Industrial Training 32(1), 19–31. https://doi.org/10.1108/03090590810846548
Kennedy, P.E., Chyung, S.Y., Winiecki, D.J. & Brinkerhoff, R.O., 2013, ‘Training professionals’ usage and understanding of Kirkpatrick’s level 3 and level 4 evaluations’, International Journal of Training and Development 18, 1–21. https://doi.org/10.1111/ijtd.12023
Langer, A., Bapna, R. & Gopal, R., 2014. ‘Estimating returns to training in the knowledge economy’, MIS Quarterly 38(3), 757–771.
Latif, K.F., 2012, ‘An integrated model of training effectiveness and satisfaction with employee development interventions’, Industrial and Commercial Training 44(4), 211–222. https://doi.org/10.1108/00197851211231487
Lawler, E.E., Mohrman, S.A. & Ledford, G.E., 1998, Strategies for high performance organisation – The CEO report, Jossey-Bass Publishers, San Francisco, CA.
Mattson, B.W., 2000, ‘Development and validation of the critical outcome technique’, Human Resource Development International 3(4), 465–487. https://doi.org/10.1080/713767868
Mattson, B.W., 2005, ‘Using the critical outcome technique to demonstrate financial and organisation performance results’, Advances in Developing Human Resources 7(1), 102–120. https://doi.org/10.1177/1523422304272173
Medina, L., Acosta-Perez, E., Velez, C., Martınez G., Rivera, M., Sardinas, L. et al., 2015, ‘Training and capacity building evaluation: Maximizing resources and results with success case method’, Evaluation and Program Planning 52, 126–132. https://doi.org/10.1016/j.evalprogplan.2015.03.008
Meyer, M., 2007a, Measuring ROI in mentoring and coaching, Knowres Publishing, Johannesburg, South Africa.
Meyer, M., 2007b, ‘The skills revolution: South African companies meet international training standards’, Management Today 23(1), 57–58.
Meyer, M., Opperman, R. & Dyrbye, C., 2003, Measuring return on investment in training: Practical guidelines for implementation, Knowres Publishing, Johannesburg, South Africa.
Naidoo, G. & Xollie, T., 2011, ‘The critical need for an integrated leadership approach to improve services delivery by the SA public service’, Journal of US-China Public Administration 8, 1–15.
Nguyen, T.N., Truong, Q. & Buyens, D., 2010, ‘The relationship between training and firm performance: A literature review’, Research & Practice in Human Resource Management 18(1), 36–45.
Noe, R.A., 2002, Employee training & development, 2nd edn., McGraw-Hill/Irwin, Boston, MA.
Noe, R.A., 2010, Employee training and development, McGraw-Hill/Irwin, Boston, MA.
Phillips, P., 2013, The bottom line on ROI. How to measure the ROI in learning and development, performance improvement, and human resources, viewed 15 May 2016, from www.roiintitute.net
Phillips, P. & Phillips, J.J., 2002, The bottom line on ROI. Basics, benefits, & barriers to measuring training & performance improvement, CEP Press, Atlanta, GA.
Purser, S.A., 2004, ‘Improving the ROI of the security management process’, Computers & Security 23(7), 542–546. https://doi.org/10.1016/j.cose.2004.09.004
Quirin, R., 2015, Manage risk – Don’t react to it, Controller’s Report, viewed from www.iofm.com
Raes, A.M.L., Heijltjes, M.G., Glunk, U. & Roe, R.A., 2011, ‘The interface of the top management team and middle managers: A process model’, Academy of Management Review 36(1), 102–126. https://doi.org/10.5465/amr.2009.0088
Rehman, A.U., Khan, A.M. & Khan, A., 2011, ‘Measuring training effectiveness: A case study of public sector project management in Pakistan’, Journal of Diversity Management – First Quarter 2011 6(1), 39–48. https://doi.org/10.19030/jdm.v6i1.4170
Russ-Eft, D. & Preskill, H., 2005, ‘In search of the Holy Grail: Return on investment evaluation in human resource development’, Advances in Developing Human Resources 7(1), 71–85. https://doi.org/10.1177/1523422304272169
Schoeman, J.S. & Vlok, P.J., 2014, ‘The possible influence of risk management, forecasting and personnel training in physical asset management’, South African Journal of Industrial Engineering 25(2), 96–104. https://doi.org/10.7166/25-2-740
Schutte, N. & Barkhuizen, N., 2016, ‘The reciprocal relationship between human resource management professionalism and a diverse South African workplace context’, Journal of Applied Business Research 32, 2. https://doi.org/10.19030/jabr.v32i2.9591
Shafritz, J.M., Ott, J.S. & Jang, Y.S., 2005, Classics of organisational theory, 6th edition, Wadsworth Publishing, Boston, MA.
Shepherd, A., McKay, J. & Bowie, P., 2010, ‘Training the trainer in general practice: The perceived value of independent peer feedback for learning activities’, Education for Primary Care 21, 368–375. https://doi.org/10.1080/14739879.2010.11493942
Spitzer, D.R., 2005a, Verifiable value: The learning effectiveness model, MediaTec Publishing, viewed from www.clomedia.com/Chief learning officer.
Spitzer, D.R., 2005b, ‘Learning effectiveness measurement: A new approach for measuring and managing learning to achieve business results’, Advances in Developing Human Resources 7(1), 55–70. https://doi.org/10.1177/1523422304272167
Spitzer, D.R., 2007, Transforming performance measurement: Rethinking the way we measure and drive organisational success, American Management Association, Washington, DC.
Sumner, M., 2000, ‘Risk factors in enterprise-wide/ERP projects’, Journal of Information Technology 15(4), 317–327. https://doi.org/10.1080/02683960010009079
Taylor, F.W., 2004, Scientific management, Routledge, London.
Taylor, F.W., 1911, The principles of scientific management, Harper and Row, New York.
Walker, B.H., 2005, ‘A theoretical integration of user satisfaction and technology acceptance’, Information Systems Research 16(1), 85–102. https://doi.org/10.1287/isre.1050.0042
Welman, C., Kruger, F. & Mitchell, B., 2010, Research methodology, Oxford University Press, Cape Town.
Welman, C., Mitchell, B. & Kruger, F., 2005, Research methodology, 3rd edn., Oxford University Press, Cape Town, South Africa.
Wren, D.A., 2005, The history of management thought, Leyh Publishing, Austin, TX.
Zinovieff, M.A. & Rotem, A., 2008, Impact evaluation methods, and proposed measures to support a United Nations System Fellowships Evaluation Framework, UN Task Force on Impact Assessments of Fellowships, Geneva, Switzerland.
|
Email this article 
Hide
Show all
