Data Loading...
Understanding the return on investment from TVET Flipbook PDF
Understanding the return on investment from TVET
118 Views
88 Downloads
FLIP PDF 525.16KB
Understanding the return on investment from TVET A practical guide
Published in 2020 by the United Nations Educational, Scientific and Cultural Organization 7, place de Fontenoy 75352 Paris 07 SP France and National Centre for Vocational Education Research (NCVER) Level 5, 60 Light Square, Adelaide 5000, South Australia © UNESCO and NCVER 2020 UNESCO ISBN: 978-92-3-100375-2 EAN: 9789231003752 NCVER ISBN: 978-92-3-100285-4
This publication is available in Open Access under the Attribution-ShareAlike 3.0 IGO (CC-BY-SA 3.0 IGO) license (http://creativecommons.org/ licenses/by-sa/3.0/igo/). By using the content of this publication, the users accept to be bound by the terms of use of the UNESCO Open Access Repository (http://en.unesco.org/open-access/ terms-use-ccbysa-en).
The designations employed and the presentation of material throughout this publication do not imply the expression of any opinion whatsoever on the part of UNESCO and NCVER concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The ideas and opinions expressed in this publication are those of the authors; they are not necessarily those of UNESCO and do not commit the Organization. This Guide was written by Jane Schueler and Phil Loveder of the NCVER, Australia. The project was coordinated by the UNESCO-UNEVOC International Centre for Technical and Vocational Education and Training. Design Christiane Marwecki Print Printed in Germany
Understanding the return on investment from TVET A practical guide
Acknowledgements The authors would like to acknowledge the following people who helped guide the research. Ms Kenneth Barrientos from the UNESCO-UNEVOC International Centre, Germany, for co-managing the project and Mr Wouter de Regt for his support in preparing the report. Professor Andrew Dickerson, University of Sheffield (United Kingdom), Mr Mohammed Naim bin Yaakub, Ministry of Higher Education (Malaysia), Mr Jin Park, KRIVET (Korea), Dr Jaeho Chung, KRIVET (Korea), Ms Ursula Mendoza, TESDA (Philippines), Ms Chandra Malkanthi Jayawardana, Ministry of Education (Sri Lanka), Ms Miki Nozawa and Ms Pooja Gianchandani, UNESCO-UNEVOC International Centre, and Mr Rahmhari Lamachhane, Colombo Plan Staff College for Technical Education, Philippines. In addition, the authors wish to recognize the very significant contribution and foresight of Dr Borhene Chakroun, Director for Policies and Lifelong Learning Systems at UNESCO Paris, Dr Shyamal Majumdar, former Head of UNESCO-UNEVOC International Centre, and Dr Craig Fowler, former Managing Director of the National Centre for Vocational Education Research, Adelaide, in establishing this cooperative research initiative.
2
Understanding the return on investment from TVET – a practical guide
Table of contents Understanding the return on investment from TVET – a practical guide
6
5. Planning to calculate the return on investment from TVET
39
Aim Overview of the Guide Using the Guide
6 6 6
1. Introduction
7
What is return on investment? Why is return on investment important? Who can use a return on investment analysis?
7 8 8
Step 1: Define the TVET environment Step 2: Adopt guiding principles Step 3: Establish objectives Step 4: Collect data Step 5: Identify costs and benefits Step 6: Analyse the data Step 7: Calculate return on investment Enablers and challenges to measuring the return on investment Key guiding questions
39 39 40 40 42 43 43 44
6. Implementing frameworks to measure the return on investment (Case studies)
45
Case study 1: Workplace literacy Case study 2: Apprenticeship training Case study 3: Social benefits of TVET for individuals Case study 4: Cost-benefit ratio for construction craft training Case study 5: Economy wide costs and benefits
46 49 51
Conclusions
60
References
71
2. Evaluating the return on investment 10 from TVET
Context, scope and purpose Guiding principles, models and indicators Costs and benefits Data issues
11 11 17 22
3. Establishing a framework to measure the return on investment
23
Background Stakeholder groups Economic and social dimensions TVET economic and social objectives Key return on investment indicators Return on investment framework
23 23 23 23 24 25
4. Mapping the return on investment indicators
29
29 30 33 34
Identify costs Identify benefits Define data sources Isolate factors affecting the return on investment Convert data Identify intangible benefits Calculate return on investment
44
54 56
34 35 36
Understanding the return on investment from TVET – a practical guide
3
Tables and figures TABLES 1 Types of users and return on investment activities 2 Return on investment models 3 Return on investment indicators by stakeholder group 4 Training costs by stakeholder group 5 Benefits of TVET by stakeholder group 6 Benefits of training for employees and employer 7 Short- and long-term benefits of training by stakeholder group 8 Factors that impact the return on investment results 9 How TVET contributes to economic and social outcomes by stakeholder group 10 Training costs and description 11 Return on investment indicators by stakeholder group and examples of measures 12 Data sources, associated costs and benefit measures 13 Example costs and benefits for return on investment calculation 14 Benefit-cost ratio and return on investment calculations for training programmes 15 Costs and benefit measures by category for workplace literacy, Australia 16 Training costs, benefits and estimated return on investment, Australia 17 Training costs and benefits post-training, Australia 18 Average gross costs and returns per year per apprentice by category, Germany 19 Gross costs, returns and net cost by company attributes per apprentice per year, Germany 20 Personnel recruitment costs per new skilled worker by cost categories, Germany 21 Social indicators, measures and data source 22 Education, individual and system variables by classifications 23 Summary of results for model in levels base and controls, for all ECHP countries by social outcomes 24 CII model plant baseline cost rates, North America 25 Average minimum and maximum expected cost savings, benefits and returns from craft training, North America 26 Benefit-cost ratios using consolidated company data for the CII model plant estimate, North America 27 Total funding of baseline levels and net increase for policy scenarios, 2013 to 2017 ($ billion), Australia 28 Cost and benefits of additional TVET funding ($ billion), Australia
4
Understanding the return on investment from TVET – a practical guide
9 13 15 17 18 19 19 21 24 29 31 33 36 37 45 46 48 49 50 51 51 52 53 54 55 55 57 58
FIGURES 1 2 3
Return on investment evaluation framework elements Steps to measuring the return on investment and stakeholder questions Return on investment framework
10 12 25
PRO FORMA RESOURCES AND ATTACHMENTS Summary of return on investment indicators by stakeholder group
61
Return on investment Data Collection Pro Forma Guide
62
Return on Investment Data Collection Summary List
63
Pro forma resource 1: Return on investment job-related indicators for individuals Pro forma resource 2: Return on investment non-job-related indicators for individuals Pro forma resource 3: Return on investment market indicators for business Pro forma resource 4: Return on investment non-market indicators for business Pro forma resource 5: Return on investment economic indicators for the economy Pro forma resource 6: Return on investment social indicators for society Pro forma resource 7: Return on investment financial data and costs to stakeholder groups
64 65 66 67 68 69 70
Understanding the return on investment from TVET – a practical guide
5
Understanding the return on investment from TVET – a practical guide AIM
OVERVIEW OF THE GUIDE
USING THE GUIDE
This Guide is designed to help improve our understanding of the return on investment (ROI) from technical and vocational education and training (TVET ). The primary aims are:
To better understand the ROI from TVET requires an overarching view. Hence, this Guide presents an analytic framework that summarizes some of the main elements and issues that need to be considered in measuring ROI. This includes establishing the scope, context and purpose for measuring ROI, adopting guiding principles, identifying costs, benefits and factors that impact on ROI, and approaches to data collection and analysis. The Guide then introduces a framework that looks at the ROI equation from a range of perspectives – including economic and social dimensions – and for different stakeholders, including individuals, businesses and societies. The Guide presents relevant ROI indicators and measures drawn from the existing international research. Finally, it provides guidelines to planning and collecting ROI data, along with a set of practical pro forma resources and case studies to assist the reader.
In this Guide, it is important to note that the ROI framework is based on a set of guiding principles. There are no right or wrong answers to establishing a ROI process. It is necessary to use appropriate judgement and decisions that best fit the organizational needs and priorities to achieve outcomes that add value to the specific TVET context.
• to identify the main issues •
•
and key elements to measure the ROI to present a framework that encompasses different stakeholder groups and identifies key ROI indicators to provide fundamental guidelines and tools for ROI planning and data collection
The Guide also emphasizes the importance of ensuring that the foundation of ROI should be closely aligned to the objectives of the TVET system of a country.
6
Understanding the return on investment from TVET – a practical guide
The guiding framework endeavours to identify the main issues to better understand ROI, present key ROI indicators and demonstrate how to prepare a plan and implement a data collection process. However, the maturity of data systems in TVET will determine the most appropriate sections to focus on. For those new to ROI or intending to integrate ROI indicators into a new system, reading the entire Guide with particular reference to the main issues that impact ROI would be of most benefit. If the intention is to survey stakeholders to collect data, consider the planning section and steps to the implementation of a data collection process. Those with experience in ROI may wish to use the Guide as a reference tool and implement the key ROI indicators.
1. Introduction Strengthening technical and vocational education and training (TVET ) is an important strategy to contributing to equitable, inclusive and sustainable economies and societies. The United Nations (2015) lists ensuring ‘inclusive and equitable quality education and promote lifelong learning opportunities for all’ as one of its Sustainable Development Goals (SDG 4). However, in a world of competing demands, this comes with challenges for the funding and financing of TVET systems internationally, and for providing evidence of the return on investment (ROI) from TVET. Measuring the ROI in a given country has its challenges, but the diversity of TVET systems and the differing contexts pose considerable difficulties for comparative analysis of the ROI across multiple countries. One global challenge is to develop measures of the ROI that can be utilized broadly across countries, while another is having appropriate data to enable the measurement of the ROI. However, since only some data is available that compares multiple TVET systems using standardized measures and definitions (for example the OECD At-a-Glance), it may be more useful to establish the challenge on this basis and address the potential for measurement within this scope. There is likely to be considerable variation between countries in terms of priorities regarding the objectives, costs and benefits of TVET. There will almost certainly be differences in terms of the data that is available to measure the ROI from TVET. Our first step is to discover whether there is existing data that relates to the ROI. If so, it will almost certainly come in multiple forms from quantitative and qualitative sources with differing contextual underpinnings. The quality of data in terms of validity and reliability will vary significantly and the data completeness and accessibility to ROI information will further reveal the disparity between stakeholders. Furthermore, there may be variation between data definitions and the extent of data gaps available to measure the ROI.
Selecting a ROI model that is fit-for-purpose and suits the type of data available is imperative in order to produce useful and practical information. The decision to include social or economic returns will influence your choice, along with adopting an evaluative (actual return) or forecasting (potential return) perspective. For example, the Cost-Benefit Analysis assigns monetary value to training costs to determine the cost-benefit ratio, the Social Return on Investment approach has a stakeholder-driven evaluation with a strong focus on social impact, while the Net Present Value model compares the value of money now to the value in the future. Ultimately, a ROI model must add value and measure factors that matter to the stakeholder. Thinking through what is involved in the ROI process can give a better understanding as to what type of information and data is required to calculate the measure.
WHAT IS RETURN ON INVESTMENT? Firstly, a working definition of ROI as it relates to TVET enables the many layers and dimensions of ROI to be represented in the right context. For the purposes of this Guide, we use the following definitions: Return on investment refers to a measure of the benefit of an investment relative to the cost of that investment. Therefore, in the TVET context, the ROI is the benefit derived by individuals, firms and nations from investing in training (VET Glossary 2017). Returns to education refer to the individual gain from investing in more education, especially focussed on the relationship between education attainment and earnings. However, for consistency and simplicity, the Guide tends to use ROI. Technical and vocational education and training comprises education, training and skills development for a wide range of occupations. It can take place in secondary schools and at tertiary education, and includes work-based learning and continuing education and training.
Understanding the return on investment from TVET – a practical guide
7
Having said this, in this Guide, we mainly refer to TVET that is offered as a post-compulsory or postsecondary education option by a tertiary training provider (or school), rather than informal TVET. The main reason for this is that systems of formal TVET are in the main more likely to have data to support the conduct of an ROI exercise.
WHY IS RETURN ON INVESTMENT IMPORTANT? Accountability for public spending in education is more important than ever before. The international spotlight is now clearly fixed on the need to measure the ROI and promote the financing of TVET. As evidence of this trend, UNESCO adopted ‘Accountability in education’ as the central theme in the 2017 issue of the Organization’s Global Education Monitoring Report. The report reinforces the urgency and relevance of investing in TVET and for stakeholders to take the lead in measuring the returns of professional skills training. Although the report is premised on the shared accountability of public and private sectors only in limited aspects, such as regulatory frameworks of qualifications and social partnership dialogues, the case for cost-shared responsibilities is imperative and building accountability mechanisms to strengthen engagement in training provision crucial. Providing information on the ROI from TVET is important as it provides governments and funders analytical information or direct evidence on the performance of the system, and further provides justification for the expenditure on TVET. Information on the ROI is also useful at the enterprise and individual levels and the role it can play in the evaluation of public policies related to TVET.
WHO CAN USE A RETURN ON INVESTMENT ANALYSIS? This Guide has been prepared for stakeholders that want to better understand the social and economic value being generated by their training activities, or by the programmes or activities they are funding. Given the multi-layered contexts of ROI, numerous stakeholders in TVET could benefit from utilizing this framework. A ROI analysis is versatile in its application. It is useful for public policy-makers to gather knowledge to understand economic and social value of TVET investments and evaluate policies, and for publicfunded institutions to collect evidence of benefits to establish compelling business cases to invest in training. A ROI analysis is a practical tool for managers to use to improve performance, and for businesses to assess the productivity monetary returns from investing in human capital. It also provides funders with a decision-making tool for identifying, assessing and measuring investments, and provides guidance to individuals on career choices. Table 1 gives an overview of the potential usability of a ROI analysis for different types of users and activities. This Guide also provides a general resource for organizations at varying stages of readiness to use a ROI analysis and enables the information provided to be implemented in a practical way to meet the needs and priorities of different stakeholders.
A ROI analysis can fulfil a range of additional purposes. It can be used as a tool for strategic planning and for promoting business improvement and efficiency, it can advise funding agreements, or it can help make investment decisions. It can help guide choices, communicate impact, and assist in attracting investment and inform public policy. Most importantly, it can measure individual, organizational and economic/societal performance - from small business to large enterprises, from nonprofit organizations to entrepreneurial ventures, from employees to the economy.
8
Understanding the return on investment from TVET – a practical guide
TABLE 1 Types of users and return on investment activities USERS
ROI ACTIVITY
EXAMPLE
Public policymakers
Evaluative tool for understanding the economic and social value of TVET investments, and for reviewing the effectiveness of TVET policies and programmes.
Establishing funding and financing strategies that recognize the shared responsibility for skills training between public and private sectors or setting incentives for businesses engaged in training.
Funding agencies
Decision-making tool for deciding where to invest and for assessing performance and measure progress.
Supporting the investment process in strategic planning, valuation, performance assessment, monitoring and evaluation, and contract management.
Enterprises and employers
‘Evidence-of-benefit’ generation tool for making the case for industries and enterprises to invest in training.
Companies directly investing in TVET as part of their overall workforce development strategy.
Management tool to improve performance, inform expenditure and highlight added value.
Implementing a performance management system that records revenues and costs, identifies key drivers of organizational value, and administers benchmarking processes.
Guidance tool for making career choices and life decisions based on sound information on the economic and employability returns from TVET courses.
Using information related to wage factors, time spent opportunity costs and increasing competency.
Individuals
Understanding the return on investment from TVET – a practical guide
9
2. Evaluating the return of investment from TVET Measuring the ROI from TVET is very complex. Firstly, there are different dimensions and factors that limit our ability to generalize the outcomes. At the macro-level, these include the level of economic development, expressed social equity objectives, and the maturity and status of TVET to name a few. The variation in the level of emphasis of these aspects in different countries will naturally influence which measures of a return on investment analysis they should focus on. Context is also important. The political, economic and education system of a country, and the stakeholders and social partners involved, all have an influence on which aspects of the return on investment from TVET are important. This means that what is measured in one country in terms of the return on investment might not necessarily be the same as in another country, although there may be some common ground.
Another complexity is the availability of data to measure the return on investment from TVET. The maturity of administrative and other statistical data systems in countries varies. In some cases, they may not be in a form that is useful for the measurement of ROI or they may not exist at all. Given the above considerations – and to better understand the return on investment from TVET – we propose an evaluative framework that explains the key elements and explores the main issues that have an impact on the return on investment. The key elements of the evaluation framework are shown in Figure 1. The various quadrants in the framework are discussed in this section.
FIGURE 1 Return on investment evaluation framework elements
Guiding principles ROI model Indicators
Context Scope Purpose
RETURN ON INVESTMENT Costs Benefits Impact factors
Data collection Data validation Data conversion ROI calculation
Source: Schueler et al. (2017)
10
Understanding the return on investment from TVET – a practical guide
CONTEXT, SCOPE AND PURPOSE The context, scope and purpose form the foundation of the framework. Developing a ROI evaluative framework requires an understanding of the TVET context. The evaluative framework is challenged by variations in TVET systems and ROI methodology. It is context specific and impacts the definition and calculation of TVET costs and benefits. Hence, the outcome from any ROI analysis tends to be relative and restricted to a specific environment. In terms of scope, there are multiple levels of stakeholders including individuals, enterprises and economies. There are multiple dimensions of ROI in each of these levels. These include the economic, social and environmental measures of the ROI. Defining a scope keeps this measurement practical and focused. Clarifying the purpose is integral to the evaluative framework. This provides the core driver to perform a ROI analysis and helps to specify the parameters.
GUIDING PRINCIPLES, MODELS AND INDICATORS Guiding principles The guiding principles ensure that a consistent and standard frame of judgement is applied to the ROI evaluation. The guiding principles consider the following: •
•
The implementation of the ROI model is practical and will provide information that meets the needs of stakeholders. The model also needs to cater to a range of measures and data sources, a variety of types of training, and its application before, during, and after training.
•
The development of the methodology and data collection instruments and processes can be customized to particular contexts while being specific enough for the data that is required. In addition, they should place minimal load on the stakeholders that need to administer them.
•
The compilation of credible evidence about the impact of training must satisfy a number of requirements. The data has to be of sufficient quality and the ensuing analysis needs to be scientifically valid and address the possibility that training may not be the only factor that explains changes in performance or outcomes.
Approaches to measuring the return on investment The rest of the information, divided into quadrants in the evaluative framework, can be represented by a flowchart or a decision tree. Figure 2 presents steps towards measuring the ROI within the evaluative context, and questions that may need to be considered by stakeholders. Some components of the flowchart are referenced in this section and discussed further in the Guide.
The ROI model or method adopted must be customized, fit-for-purpose and add value. It requires an overarching clarity of purpose and measures factors that are specific and relevant to the context.
Understanding the return on investment from TVET – a practical guide
11
FIGURE 2 Steps to measuring the return on investment and stakeholder questions
Evaluative or forecast ROI model
Collect data sources
Identify training costs
Identify benefits to stakeholders
Isolate the net benefit of training
Convert costs/benefits to hard data monetize where possible/required
What type of ROI model is fit-for-purpose? What type of ROI model is appropriate – evaluative or forecasting? What ROI measures are most important? What is the scope?
What existing data sources can be used to measure the ROI? What is the data quality and completeness? Is the information available/accessible? Are there data limitations? Are there data gaps? What is the data context?
What are the direct costs? What are the indirect costs? Who pays for the training? Do the costs differ by industry? Over what period are the costs calculated? Are intangible costs measurable? How can we measure intangible costs?
What are the tangible benefits? What are the intangible benefits? Are intangible benefits measurable? How can we measure intangible benefits? What are the short-, medium- and long-term benefits of training? What are the most important data collection points?
What factors impact results? How do we define and calculate key variables? How should key data variables be aggregated? How can we control variables that impact results? What statistical techniques can be used to isolate the effect of training?
Should intangible costs/benefits be monetized/quantified? How can costs and benefits be converted to monetary/ quantifiable values? What is an appropriate conversion method/process?
Validate data
Is the data valid? Does it measure what it is supposed to measure? Is the data reliable? Is the data consistent and reproducible?
Determine level of data aggregation
What is the degree of data aggregation? Is the data comparable?
Conduct statistical analysis
What type of analysis fits the ROI model? Does each indicator require a different or specific analysis?
Record contextual underpinnings
What are the contextual underpinnings of the data? Source: Schueler et al. (2017)
12
Understanding the return on investment from TVET – a practical guide
Return on investment models TVET research studies use various models to determine the ROI. These models include measuring economic and social impact. At the economic level, the ROI from TVET falls into two broad categories: i.
determining the ROI for spending that has occurred (evaluative) ii. investigating the potential return should spending/funding be altered (forecasting)
Different models apply to different situations and may suit specific types of data. For example, CostBenefit Analysis assigns monetary value to costs of the training programme, Internal Rate of Return refers to the rate of interest, and Social Return on Investment is a stakeholder-driven cost-benefit analysis with a strong focus on social impact. Ultimately, the return of investment model of choice must be fit-for-purpose.
The decision to include economic and social returns will influence the selection of the model used to determine the ROI, along with the choice of an evaluative or forecasting perspective. The bestfit model enables customization, adds value, and measures factors that matter and are specific. Some examples of ROI models are shown in Table 2.
TABLE 2 Return on investment models MODEL
DESCRIPTION
Cost-Benefit Analysis
Assigns monetary value to costs of the training programme to determine the cost-benefit ratio
Internal Rate of Return
Rate of interest that equals the returns from an investment to the cost of the investment
Kirkpatrick/Phillips Evaluation Model
4 levels of evaluation – Reaction, Learning, Behaviour, Results plus Level 5 ROI that converts 4th level to monetary value
Net Present Value Return on Expectation
Compares the value of money now with the value in the future
Social Return on Investment
Stakeholder-driven evaluation with cost-benefit analysis with a strong focus on social impact
Estimates returns to training relative to stakeholder expectations. Uses surveys and interviews
Source: Schueler et al. (2017)
Understanding the return on investment from TVET – a practical guide
13
SNAPSHOT Workplace literacy and the Kirkpatrick/Philips Evaluation Model Understanding ROI is important for employers who invest in literacy, language and numeracy training for their workers. A Kirkpatrick/Philips Evaluation Model was applied to the context of workplace literacy in a range of organizations to measure the ROI with a focus on personnel costs, productivity gains, operational costs, human resources and other financial benefits. The results from the study (Brown et al., 2015) indicate significant results: •
A large building and construction company conducted a ROI analysis to measure workforce engagement and worker advancement. Their systems were large, complex and multifaceted and made data collection difficult. The ROI was difficult to quantify across the entire programme, however the sub-programme identified $192,600 in savings.
•
A medium-sized manufacturing facility measured workplace literacy and supporting workplace health and safety learning. Their multi-layered programme produced a measurable sub-set of data. Their analysis indicated a 163% return from error reduction and saved supervision time.
•
A small aged care facility focussed on improving documentation to enhance organizational culture. They were able to monetize changes in supervisor time across all data collection points. The outcome was 117.5% savings through reduced documentation errors.
SNAPSHOT Implementing a Social Return on Investment Model for employment creating social enterprises To measure value these employment creating social enterprises in Australia implemented a Social Return on Investment Model. The organizations applied both evaluative and forecasting approaches. They undertook the analysis as part of funding agreements. Although the outcomes varied, they were relative to the needs of the enterprise. All organizations were able to validate the social impact they created or intend to create (IIP, 2012):
14
•
People Power Cleaning provides employment to refugees and migrants at risk of mental health issues who face employment barriers. They aimed to understand the costs associated with providing support to marginalized employees. They identified that the main support costs were training on-the-job supervision. The Social Return on Investment Model provided a credible and transparent justification for funding an additional employee.
•
Livingin Construction constructs affordable buildings/landscapes for families in need while employing people excluded from the labour market. They aimed to understand the additional costs incurred and value created from employment. The Social Return on Investment Model indicated that the key driver of social value was through increased employee social interaction. The company also gained clarity on outcomes, values and costs. The analysis informed what data to collect and evaluate in the future and how to reduce costs.
•
Tasty Fresh Community Catering provides support, training and job opportunities for underprivileged women. The Social Return on Investment Model identified the real costs associated with achieving the social mission, many of which had been underestimated and not accurately factored into forward projections.
Understanding the return on investment from TVET – a practical guide
Return on investment dimensions There are many layers and dimensions to measuring the ROI from TVET. They are different for each stakeholder. There are economic and social aspects. It is easier to measure economic impact, but it is the social impact that completes the whole ROI picture. It is important to understand the social implications of training as they provide the true value of training that is often neglected in TVET research (due to difficulty in measuring). Table 3 shows a sample of ROI indicators for individuals, employers and the economy. For individuals, the two main influences of training can be measured by improved employability and increased productivity. Having said this, there are also gains unrelated to job performance that result from training, including self-confidence, health and engagement. Measuring the ROI is also influenced by the individual’s ambitions, such as promotion, higher education pathways and personal development. Organizations and employer’s training outcomes are commonly analysed by productivity gains and efficiency. In addition, there are nonproductivity returns through employee wellbeing, work practices and organizational culture. The reasons for committing to TVET also goes beyond productivity to legislative and licensing
requirements, introducing new technologies and business improvements. In society, in addition to productivity gains, education and training has also been shown to bring other non-financial returns of social cohesion and inclusion. Data collection There are two main steps in the data collection process. The first is preparing information to guide the process. This includes defining what type of analysis is most suitable and what type(s) of data will be required to support it. The second step is establishing existing data sources. This includes identifying types of quantitative and qualitative datasets such as national data collections, administrative datasets, longitudinal studies, surveys, interviews and case studies. Of further importance is documenting the completeness of the data, its quality, gaps in the data and contextual underpinnings. Once the data has been collected, it may need to be converted into monetary terms. Not all the data will be in monetary form, particularly the less tangible costs and benefits. Therefore, consideration needs to be given as to how the data can be converted to monetary or quantifiable values (if possible or required) using an appropriate conversion method.
TABLE 3 Return on investment indicators by stakeholder group INDIVIDUALS
EMPLOYERS
WIDER COMMUNITY
Job related Employability Productivity – skill gains Earning capacity Foundational skills – literacy Training pathways – vocational/higher education
Market Productivity Efficiency Employee workplace literacy Employee skill gains Business innovation
Economic Labour market participation Labour force productivity Increasing the tax base Gross domestic product
Non market Organizational culture Motivated workforce Employee well-being Employee work practices
Social Social cohesion Social inclusion Health and well-being Crime reduction
Non job-related Well-being Engagement Satisfaction Self-esteem/confidence
Source: Schueler et al. (2017)
Understanding the return on investment from TVET – a practical guide
15
SNAPSHOT Economic returns for individuals, business and the economy To establish the economic outcomes for individuals, business or the economy requires a quantitative approach. These studies use different ROI models: •
To assess the relationship between training and the individual, Cedefop (2011) used a variety of comparable datasets from across the European Union. Multi-regression analysis examined the effect of vocational education and training on wages and employment status. The results indicated returns for an extra year of tertiary education are equal to the return of one extra year of initial TVET. The effect of education on wages and employment are about 7% for both males and females and adjusted returns to account for the short duration of the training (17 weeks on average) gives yearly returns for TVET of 10% for men and 7% for women.
•
To examine the returns to training and productivity, Bernier and Cousineau (2010) based their analysis on the longitudinal Statistics Canada Workplace and Employee Survey (WES), using a Cobb-Douglas function within a distributed lag estimation framework. They found positive effects of training on productivity spread out over a three-year period. They also found interactions between investments in capital and investments in training. A 10% increase in investments accompanied by expenditure in structured training per employee resulted in an average increase of 0.6% in corporate productivity the following year compared to a company that did not invest in capital complementary to training.
•
To investigate the ROI from training to the economy, Independent Economics (2013) used a cost-benefit analysis to estimate the return of increased funding in TVET. The committed 5.6% increase in funding was predicted to result in an 18% internal rate of return to the economy.
SNAPSHOT Conversion of intangible costs and benefits Intangible costs and benefits can also be grouped into three conversion categories – monetized, quantified (but not monetized), and qualitative (not quantified nor monetized). Using employee engagement as an example, we need to define how (or if ) we can measure the indicator, what variables can be used to calculate a value and develop a formula to quantify the result if possible. Here are measures of employee engagement: CATEGORY
MEASURE
CONVERSION VALUE
Monetized Quantified Qualitative
Employee turnover costs Absenteeism (annual) Job satisfaction
Recruitment costs ($)/Number of new employees Number of lost days per year/Number of employees Scales from interview, observation or self-rated survey
It is important to note that some measures can be considered tangible or intangible, dependent on the organization. For example, in some organizations employee retention data is readily available with costs of recruitment (e.g. advertising, human resources time, induction training, etc.) for new staff and therefore can be converted to financial figures and the ROI calculated. However, smaller organizations may not have this information and so employee retention would be reported separately from the ROI figure.
16
Understanding the return on investment from TVET – a practical guide
The data will also require validation before statistical analysis of the ROI is conducted. That is, the data being used to calculate the ROI should measure what it is supposed to be measuring. The data also needs to be reliable in terms of being reproducible, consistent and accurate. A ROI analysis can then take place using an appropriate statistical approach. For example, regression or multivariate analysis has been applied to ROI data to control for variables to ensure that the ROI outcome is a direct result of training.
COSTS AND BENEFITS Determining the costs and corresponding benefits of TVET are critical to the ROI analysis. Related to this is a consideration of how these can be impacted by other factors. Training costs Training costs and total investment are generally underestimated. There are two categories of costs —direct and indirect. These costs differ by stakeholder type and attributes of the specific training programme. TVET costs are paid by students, businesses, industry, training providers and the community, for which data may be difficult to collect. Table 4 provides examples of individual and employer costs.
Direct costs are easier to measure. For individuals these expenses can vary. Financial costs may differ between courses, providers and with concession where subsidies and student vouchers provide external financial support. Industry requirements and the field of study may call for additional equipment, materials and for some protective clothing to be purchased. As an employee, the tuition costs are paid by the employer. However, accessing cost data at the business level may be difficult. Indirect costs are not as clearly quantifiable or easily captured. For example, older students may need to pay for childcare or forgo employment for a period and absorb loss of income. Employers also bear the costs of not having adequately skilled employees that are not fully proficient at their job, lost time while employees are in training and increased workloads in their absence. Intangible costs may also be difficult to convert into monetary terms. The point in time when training costs data are collected also impacts on the ROI calculation. Costs can be measured over different periods— before, during or at the end of training. The point of time costs are captured is an important factor of the evaluative or forecasting ROI analysis. Training benefits There are two categories of benefits — economic and social. For example, in the workplace these refer to job-related and non-job-related outcomes.
TABLE 4 Training costs by stakeholder group INDIVIDUAL
BUSINESS/EMPLOYERS
GOVERNMENT
Direct costs Tuition and special fees (library) Books, materials and equipment Childcare Travel/parking
Direct costs Course costs for employee Salary of staff while on training Course design and development Off-site travel, lodging and meals
Direct costs In addition to public expenditure
Indirect costs Opportunity costs Foregone or reduced earnings while studying Non-completion costs
Indirect costs Loss of productivity while training Induction costs Trainee replacement costs while training Higher wastage rates until fully proficient Missed opportunity costs
Indirect costs Payroll tax rebates Workforce development programs Completion bonuses of employers of apprentices
Source: Schueler et al. (2017)
Understanding the return on investment from TVET – a practical guide
17
TABLE 5 Benefits of TVET by stakeholder group INDIVIDUALS
EMPLOYER
ECONOMY
Job-related Employment/higher employability Higher salaries Higher savings levels Improved working conditions Professional mobility Productivity (highly skilled)
Market Productivity Sales and profitability Customer service and satisfaction Occupational health and safety Quality product and services Saving on material and capital costs
Economic benefits Higher employability Increased participation in the workforce Decrease in unemployment levels Productivity gains Higher skilled workforce
Non-job-related Higher education pathway Pathway to further study Improved self-esteem Communication skills Engagement, social inclusion Improved problem solving Improved health and wellbeing Improved economic standards of living Life satisfaction
Non-market Motivated workforce Improved organizational climate and culture. Increased literacy in workplace Employee skill gains Employee well-being Employee workplace practices
Societal benefits Improved health Improved environment Reduced national crime Increased social cohesion Increased social inclusion Strengthened social capital Active citizenship Technological change adaptation
Source: Schueler et al. (2017)
Table 5 illustrates some of these benefits for individuals, organizations and the economy. Economic benefits are directly measurable and relative to the stakeholder group. The main benefit of TVET that influences individual’s earnings is improved employability, for the economy it is increased participation in the workforce, while productivity is the key indicator of business performance. Social benefits are not so easily quantifiable. Employee social and well-being aspects or business workplace literacy, safety and workforce flexibility
18
are more difficult to measure. Outcome measures tend to have an economic focus, excluding community and personal outcomes that are more difficult to quantify. A model that takes into account economic and social benefits is important. Benefits also vary depending on the stakeholder’s perspective. Table 6 illustrates tangible and intangible benefits of training to an employee and employer. The table shows that the individual (employee) benefits cover improved earnings, skills and work practices, while the employer benefits are concerned with productivity, compliance, safety and quality.
Understanding the return on investment from TVET – a practical guide
TABLE 6 Benefits of training for employees and employer BENEFITS
EMPLOYER
EMPLOYEE
Tangible
Improved employee pay Improved language and literacy Improved technical skills Increased use of new technologies Improved workplace practices and procedures
Increased productivity and efficiency Increased sales and profitability Improved product quality and services Improved customer service and satisfaction levels Improved occupational health and safety
Intangible
Improved self-confidence and morale Reduced stress Improved motivation Improved work ethic Improved physical and mental health Job satisfaction
Better management and employee workplace relations More cooperation among employees Reduced internal conflicts Developing a learning culture Supporting social cohesion and inclusion
Source: Brown et al. (2015)
The training benefits may arise at different points and may extend well beyond the completion of the training. Estimating the ROI can relate to a time-period during a training programme, at its completion or long after the event.
Table 7 shows a comparison of the short- and long-term benefits of training to the individual, organizations and the economy. Medium- to long-term benefits such as mobility or the capacity to upgrade skills later in life are more difficult to quantify.
TABLE 7 Short- and long-term benefits of training by stakeholder group BENEFITS
INDIVIDUAL
EMPLOYER
ECONOMY
Short-term
Employment opportunities Increased earnings levels Work satisfaction
Higher productivity from trained workforce Saved costs from recruiting external skilled workers Improved quality of products and services Improved customer satisfaction levels
Reduced reliance on welfare Social cohesion
Long-term
Greater employee flexibility and mobility Lifelong learning
Reduced employee turnover Improved safety record Better workplace relations
Productivity gain from educated workforce Increase in tax income from higher earnings
Source: Hoeckel, K. (2008)
Understanding the return on investment from TVET – a practical guide
19
SNAPSHOT Cost and benefits of investing in a dual training system A dual training system provides two learning environments – the company workplace and training at vocational schools. There is shared responsibility for the apprentice. The company provides learning in the workplace and the government delivers vocational training off-site. Identifying accurate training costs and benefits are crucial to establishing the ROI value. Germany dual training system These figures show the proportion of costs across categories and include both short- and long-term benefits. They demonstrate a good example of well-defined categories and include intangible benefits that are not directly measurable but add value (Jansen et al., 2015; Rechmann, 2017). Gross Costs: • Apprentices’ benefits (62%) = Training allowance, social benefits and fringe benefits • Salaries of trainers (23%) = Full-time, part-time and external trainers • Equipment (5%) = Workplace (tools, machinery and materials), training workshops, incompany lessons • Other costs (10%) = Training documents/media, protective gear, administration, external training, chamber/examination fees Benefits: • Short-term benefits during apprenticeship and productivity contributions during apprenticeship: 50% simple tasks (savings for unskilled or semi-skilled staff ), 47% skilled tasks (savings for skilled staff ), 1.5% production in-training workshops, and 1.5% public subsidies • Long-term benefits after and during training period and not measurable: custom-fit training, staff selection, avoiding skilled-labour shortages, better employee loyalty and image/ corporate social responsibility Philippines dual training system A cost-benefit analysis of the dual training system in the Philippines (TESDA, 2016) shows that the overall training cost per trainee per month is Php 12,800 (US $272). In comparison to Germany, the percentage of total training cost related to the costs to trainees (44%, Php 5,588) and the costs of trainers (18%, Php 2,365) are less. Having said this, the physical costs (30%, Php 3,828) are proportionally significantly higher. The results also show that overall the short-term benefits of productivity contribution and seasonal benefit is about Php 5,487 ($US 117) per trainee per month, representing approximately 43% of the average cost of training. The long-term benefits if the trainee stays in the firm flows on to savings in recruitment costs (advertising, human resources, orientation materials and training) plus a productivity difference of 16% (Php 2,032) between a worker from the dual training system and an externally hired employee. The analysis further indicates that the costs and benefits for firms per trainee per month differ by firm size, sector and region, and duration of training. It is noted that 33% of trainees that completed the programme are hired by the firm. The average costs for firms are lower and the short-term benefits are higher if trainees are retained by the firm after they complete their training. The cost-benefit analysis shows that the benefits are far greater than the costs of investing in the dual training system.
20
Understanding the return on investment from TVET – a practical guide
Other factors impacting the return on investment
education and skills — then we need to isolate and control these variables to ensure that the ROI outcome is a direct result of the training.
There are a number of other factors that can influence the ROI apart from costs and benefits. Table 8 shows a summary of the determinants which range from individual age characteristics and level of qualification, to the size of an organization and to the quality of the trainer. For example, if training works better in the workplace than the classroom, in collaboration rather than self-directed, associated with a specific application (for example, new technologies) or for those already possessing sound foundation
In particular, there are specific factors that impact the ROI for each stakeholder group. These are educational background and qualification level of the training programme for individuals; industry type, organization size and profit/not-for profit sector type for enterprises; and the profile characteristics of the population for the economy. Of particular note for all stakeholder groups is the type of training programme and the field of education or industry and the impact on ROI.
TABLE 8 Factors that impact the return on investment results FACTOR
DESCRIPTION
Stakeholder characteristics
Individual’s demographics – age, educational attainment, level of schooling (often different calculations) and apprentice Employers/business – organization size, industry, private/public and type of training
Training status
Qualification completers or module completers Part-time or full-time status Reskilling or upskilling
Training Programmes/ Course
Qualification level Non accredited versus accredited training Training level – for example, foundational – literacy and learning Industry/field of study Types of training – leadership, management, innovation and apprenticeship Initial TVET or on-going training (for example apprentices/trainees) Highly specific or general training (more transferable)
Training context
TVET in schools (teacher quality, student engagement, employer relationships (relevant/effective)) – training pathways, material resources Workplace training versus classroom training Training in partnership versus self-directed
Training provider
Private versus public Quality of trainer Quality of resource materials
Labour market
Demands for skills, labour market regulations and trade union influences Source: Adapted from Brown et al. (2015).
Understanding the return on investment from TVET – a practical guide
21
SNAPSHOT Returns to training in the construction industry A craft training survey and existing datasets in the United States construction industry provided the basis for a quantitative analysis of training benefits measuring productivity, turnover, absenteeism, safety and rework (Wang, 2008). Using the data, a benefit-to-cost ratio was calculated for a hypothetical industrial project (the CII model plant). The results show the average cost savings and average improvement from the baseline data. The study found the benefits of training do not occur at once. Some of the training payoffs occur immediately such as improved safety, reduced absenteeism and reduced turnover. However, increased productivity and reduced rework will take more time to allow an increase in craft skill. Cost savings and average improvement from baseline data: • • • • •
Productivity improvement Decrease of turnover Decrease of absenteeism Decrease of injury Decrease in rework
Total benefits Total training costs Benefit-cost ratio (average)
$322,257 $32,149 $74,870 $66,940 $25,377
(10.6%) (13.9%) (14.5%) (25.5%) (23.2%)
$521,594 $230,296 2.26
(1% of labour costs) (range 1.52 to 3.01)
The benefit-to-cost ratio shows that the average for every dollar that the company invests in craft training, $2.26 benefits will be generated (with a lower and upper range of $1.52 to $3.01). The study was based on industrial construction and the benefit-to-cost ratio based on the CII model plant.
DATA ISSUES Any ROI model must be supported by useful and practical indicators. However, there are various considerations regarding data for ROI measures. This is in addition to establishing the availability of data as discussed earlier. One consideration is the integrity and credibility of the data. This refers to the level of data accuracy, validity (is it measuring what it is supposed to measure?) and reliability (is it consistent and reproducible?). There is also a need to recognize that data validity and reliability may vary with the level of data aggregation. Another issue is data comparability. In order to compare data, there needs to be consistent definitions across datasets and a basis for data comparisons. This includes establishing what data is and is not directly comparable.
22
The data also needs to be transparent. This means that its conceptual underpinnings are clearly explained and a description of data quality and completeness is necessary along with any gaps, anomalies or caveats. In summary, measuring the ROI from TVET requires context, scope and purpose with guiding principles that form a standard frame of judgement. It requires a practical ROI model that is customized, fit-forpurpose and that is able to measure what matters. It must be able to identify relevant costs and benefits of TVET and how these can be impacted by other factors. It supports useful indicators, practical measures and quality data, and then applies scientifically valid techniques that seek to address the influence of other variables. Implementing a framework that supports these elements and guides the process provides the foundation to create an effective practical tool to measure the ROI from TVET.
Understanding the return on investment from TVET – a practical guide
3. Establishing a framework to measure the return on investment This section outlines the main elements of the ROI framework. It presents a framework to understand the ROI from TVET from different stakeholder perspectives across economic and social dimensions.
Although the framework separates these three perspectives, they are not independent of each other. The dynamic relationship between stakeholder groups can have flow-on effects.
BACKGROUND
ECONOMIC AND SOCIAL DIMENSIONS
The ROI framework featured in this section is based on extensive analysis of existing research approaches. It is clear that the ROI is context-specific to the stakeholder and relative to the environment. Furthermore, the differences between areas of focus, demographics and methodology add to the variable outcomes. Although measuring the ROI is diverse and complex, this framework provides an overarching structure that identifies key indicators to support a practical approach with broad application. The following section aims to provide insights into the perspectives, dimensions and objectives that underlie the framework, along with the highlevel indicators that direct attention to capture measurable outcomes.
Economic and social dimensions feature in the framework to provide a complete ROI picture. Historically, there has been a stronger focus on economic returns to stakeholders. These include financial returns to individuals in the labour market, productivity and profitability to businesses, and economic growth. However, social outcomes of TVET – which are not always as easy to measure as economic markers – are of great value. Indicators on the health and well-being of individuals, employees and society create a more accurate estimate of the ROI of investing in TVET.
It is important to note that although the framework separates stakeholder perspectives into groupings, these groupings are not independent of each other. There are flow-on effects and understanding the interaction between economic and social dimensions helps us to assess the true value of TVET.
STAKEHOLDER GROUPS The ROI framework uses three main stakeholder groups – individuals, business and the economy: •
•
•
Many research studies focus on returns to the individual, with a particular focus on the relationship between qualifications levels and employment and wages. The business group includes employees, employers, individual businesses and industry. In this sector, training is often highly specific for target groups. The ROI maps widely across many areas including operations, profitability, product/ services and human resources. At the macro-level, the impact of TVET on the economy has been studied to measure economic growth. Moreover, analyses of varying population profiles provide insights into social returns.
For the purpose of this framework, economic indicators relate to job-related outcomes for individuals, market indicators for business and economic growth for the economy. Social indicators refer to non-job-related outcomes for individuals, non-market indicators for businesses and health and social outcomes for society. Most importantly, understanding the interaction between the economic and social benefits is vital to assessing the true and full value of TVET. The dynamics of social returns to directly impact economic drivers can be significant. Environmental impact is also of primary importance across stakeholder groups. In this ROI framework, this measure has been included in the business sector tier relating to environmental sustainability and work practices.
TVET ECONOMIC AND SOCIAL OBJECTIVES The foundation of the ROI framework is based on ROI objectives for each stakeholder group. Table 9 summarizes how TVET contributes to the economic and social outcomes of stakeholder groups that are part of this framework.
Understanding the return on investment from TVET – a practical guide
23
TABLE 9 How TVET contributes to economic and social outcomes by stakeholder group STAKEHOLDER
ECONOMIC
SOCIAL
Individual
TVET provides the skills required to participate in the labour market
TVET contributes to improved social outcomes for individuals
Business
TVET meets the needs of business/ industry outcomes
TVET contributes to a healthy, safe and sustainable workplace
Economy
TVET contributes to improved economic outcomes in the economy
TVET contributes to improved social outcomes in society Source: Schueler et al. (2017)
KEY RETURN ON INVESTMENT INDICATORS The following part presents an overview of the ROI indicators separately by stakeholder group, although they can influence and interact across domains. The indicators that follow have been selected based on their usefulness, practicality and capacity to add value, along with the ability to apply to different types of training and contexts. Individuals The key types of economic ROI indicators for individuals from TVET are categorized as job-related measures. This primarily focuses on the labour market, employment and productivity in the form of financial returns or higher wages. Attainment of employability skills and improved labour force status are also highly valued job-related returns. Importantly, an entrepreneurship indicator is essential to capture the skills and knowledge gained from participating in entrepreneurial education. Non-job-related indicators focus on social ROI. This covers individual well-being such as self-esteem, confidence and life satisfaction, foundation skill gains (such as literacy and numeracy), social inclusion and improved socio-economic status. Business The business stakeholder group includes employers, employees, individual businesses and the industry sector. Key economic measures in the form of market indicators focus on productivity, profitability, quality improvement of products and services and business innovation by for example introducing new technologies and work practices.
24
Businesses operate similar to small communities, and as such, generate social and environmental benefits. In particular, employee well-being, employee engagement (which reduces absenteeism and staff turnover), a safe workplace and environmental sustainability practices (such as increased recycling and reduced waste) are key non-market indicators of business returns. It is important to note that non-market returns can impact market indicators and deliver economic payoffs. For example, a high level of employee engagement may translate into lower absenteeism and increase business productivity. Economy and society The key ROI indicator related to the economy is economic growth as measured by gross domestic product. Other related key measures include the level of labour market participation, reduced unemployment rates and having a more skilled workforce. Entrepreneurial activity is yet another measure as it brings value, innovation and employment growth to the economy. The ROI from TVET also brings other benefits to society. Key reliable indicators of social returns measure improved health, increased social cohesion (increased democratization and human rights), improved social equity through increased access, the participation of disadvantaged groups in TVET and strengthened social capital through participation in networks.
Understanding the return on investment from TVET – a practical guide
RETURN ON INVESTMENT FRAMEWORK Figure 3 – derived from Schueler et al. (2017) – presents a diagram of the ROI framework for each stakeholder group and the economic and social indicators that relate to them. The ROI objectives are also featured. Following the ROI framework is a detailed description of the key ROI indicators and measures that are shown in the following tables:
• • •
ROI indicators and measures for individuals ROI indicators and measures for businesses ROI indicators and measures for the economy/ society
The figure also provides a list of measures for each stakeholder group and related economic and social indicators.
FIGURE 3 Return on investment framework INDIVIDUAL Qualification level and educational background impact the level of ROI JOB-RELATED TVET provides the skills required to participate in the labour market
NON-JOB-RELATED TVET contributes to improved social outcomes for individuals
1. Employability skills 2. Employment 3. Improved employment status 4. Wages/earnings 5. Entrepreneurship
1. 2. 3. 4.
Health and well-being Foundation skill gains Socio economic status Social inclusion
BUSINESS Industry type, organization size and sector impact the level of ROI MARKET TVET meets the needs of business/industry outcomes
NON-MARKET TVET contributes to the health, safety and environmental needs in the workplace
1. Increased productivity 2. Profitability 3. Quality product/service 4. Business innovation
1. 2. 3. 4.
Employee well-being Employee engagement Workplace safety Environmental sustainability
ECONOMY/SOCIETY The profile of the population of interest impacts the level of ROI ECONOMIC TVET contributes to improved economic outcomes
SOCIAL TVET contributes to improved social outcomes in society
1. 2. 3. 4. 5.
1. Health 2. Social cohesion 3. Social equity 4. Social capital
Economic growth Labour market participation Unemployment rates Skilled workforce Entrepreneurial activity
Understanding the return on investment from TVET – a practical guide
25
Return on investment indicators and measures for individuals Indicator
Measure
Example of measures
Attainment of employability skills for those who have completed training
Attainment of non-technical skills and knowledge required to get a job and participate effectively in the workforce. These include communication, selfmanagement, planning, teamwork, decision-making and problem-solving skills
2. Employment
Employment rate of those not employed before training
Proportion of TVET graduates who are employed at the end of their training
3. Improved employment status
Improved employment status of those employed before training who have completed training
Proportion of TVET graduates who report improved employment circumstances. For example, casual to permanent status, parttime to full-time status or promoted to a higher level of employment
4. Wages/Earnings
Income of full-time workers after training
Earnings of those employed full-time after training. Measured by gross net earnings, gross monthly earnings, weekly earnings, pre-tax hourly wages or annual earnings
5. Entrepreneurship
Attainment of entrepreneurial skills and knowledge
Attainment of skills, knowledge and attitudes that aim to build an entrepreneurial mindset and skillset required to transform ideas into action
1. Well-being
Students have an improved sense of well-being after training
Improved self-esteem Improved confidence Life/work satisfaction Self-rated health Satisfaction with financial situation
2.
Foundation skill gains
TVET graduates/completers have improved foundation skills after completing training
Attainment of language, literacy and numeracy skills and financial literacy
3.
Socio-economic status
Improved socio-economic status of those completing TVET programmes
Proportion of TVET graduates who report positive changes to employment status, household income, remoteness or living standards
4.
Social inclusion
Participation in social groups or communities
Membership of a club or organization, social network group, volunteering, sporting associations or other communities
JOB-RELATED 1.
Employability skills
NON-JOB-RELATED
26
Understanding the return on investment from TVET – a practical guide
Return on investment indicators and measures for businesses Indicator
Measure
Example of measures
Increase in the productivity of the organization
Value added per hour of labour, value of sales per labour hour, items sold per hour of labour Management processes/work practices
2. Profitability
Increase the profitability of the organization
Reduced costs Increased sales Reduced supervision time Reduced scrap/wastage Reduced induction costs
3. Quality product/ service
Improve the quality of products or services
Customer service satisfaction Reduction in error/defects rate
4.
Contribute to organizational innovation and business practices
Introduction of new technologies Best/new business practices Increased efficiency and use of resources Leadership/management practices/culture
MARKET 1.
Increased productivity
Business innovation
NON-MARKET 1.
Employee well-being
Employees have an improved sense of well-being
Improved self-confidence Improved motivation Improved morale Job/working conditions satisfaction Reduced employee stress
2.
Employee engagement
Employees are more engaged in the workplace
Skill gains – workplace language and literacy skills Engaged in further study Reduced absenteeism Reduced staff turnover Increased retention rates
3.
Workplace safety
Employees experience a safer workplace
Reduced injuries Decreased accidents Improved safety records Meet compliance regulations Meet licensing requirements
Workplace practices contribute to environmental sustainability
Improved energy and fuel management Efficient waste management Increased recycling of materials Reduced energy and water usage
4. Environmental sustainability
Understanding the return on investment from TVET – a practical guide
27
Return on investment indicators and measures for the economy/society Indicator
Measure
Example of measures
ECONOMY 1.
Economic growth
Increase the capacity of the economy to produce goods and services
Gross domestic product Real gross domestic product (labour productivity)
2.
Labour market participation
Increase the labour market participation of TVET graduates/ completers
Labour market participation of TVET graduates
3.
Unemployment rates
Decrease the rate of unemployment for TVET graduates/completers
Unemployment rate of TVET graduates
4.
Skilled workforce
Level of educational attainment of TVET graduates/completers
Qualification levels of TVET graduates who complete training by industry group Productivity level. Higher earnings Skill types by industry groups
5.
Entrepreneurial activity Level of entrepreneurial activity of TVET graduates/completers
Business start-up rates Job creation/employment growth Social enterprise/community development Technological innovation/commercialization Enterprise size/growth rate
SOCIETY
28
1. Health
Improve community health and foster a longer and better life
Self-rated health Reduction of chronic health conditions, body mass index Mortality/death rates
2.
Social cohesion
Improve the well-being, social inclusion and values that support cooperation within or among groups
Reducing disparity and avoiding marginalization Crime reduction Civic unrest status Freedom status (political rights, civic liberties)
3.
Social equity
Increase access and participation of disadvantaged groups in TVET
Participation rate of disadvantaged groups in TVET including those from low socioeconomic backgrounds, disability, location remoteness and cultural/language barriers
4.
Social capital
Participation in networks that strengthen social capital
Number and types of social networks Active citizenship, civic engagement, volunteering or member of social network. For example participation/member of clubs, neighbourhood groups, organizations, political parties
Understanding the return on investment from TVET – a practical guide
4. Mapping the return on investment indicators So far, this Guide has identified why ROI is important and identified key issues and indicators to address in the evaluative framework. This section focuses on how to map the indicators to costs and benefits. It also features examples of data sources and conversion methods to calculate the return on investment. The aim is to provide this material as a reference guide for the implementation of the ROI framework and enable stakeholders to select what is relevant to their specific needs and priorities. Although there are many ROI models – including Cost-Benefit Analysis, Social Return on Investment and economic modelling – for the purposes of this Guide the focus will be on defining ROI indicators and identifying training costs and benefits that may feed into these models. To this end, this section draws on case studies presented at the end of the Guide.
IDENTIFY COSTS In order to calculate the ROI, we need to identify training costs associated with delivering the training programme. To estimate changes in individual, business or economic performance over the course of TVET, costs can be classified into five main categories. Programme costs are related to tuition fees, course design and development and course purchase. Personnel costs cover trainer wages, support staff, labour costs and lost opportunity. Further expenses are incurred for facilities and equipment hiring, training materials and costs borne by individuals ranging from student fees to foregone earnings. A more detailed list is presented in Table 10.
TABLE 10 Training costs and description TRAINING COSTS
DESCRIPTION
Programme
Tuition fees and government funding Course needs analysis/surveys and/or programme design and development costs Purchase of the course
Personnel
Wages/fees for full-time, part-time or external trainers. This may include additional fees for travel, lodging, overnight stay and off-site costs Additional training support staff and wages for human resources, administration and management staff Gross training wages for apprentices (plus subsidies) Staff labour costs for employer. For example, staff paid to attend training and lost time/productivity while participants in training (lost salaries plus benefits) Staff replacement costs (including recruitment costs)
Facility
Training room/venue hire and other facilities hiring costs Equipment and hardware On-site computer costs/hire
Training materials
Teaching and learning materials Instructional resources Printing, photocopying and administration
Individual
Student fees and charges Equipment, tools and protective clothing Transport, travel fees and parking costs Membership fees for student association, licence fees, computers, text books and materials, printing costs and other consumable materials Foregone earnings (loss of income during study)
Understanding the return on investment from TVET – a practical guide
29
Costs can also be reduced. The total investment needs to take into account book or tool allowances, tax deductions that offset costs, part-time work for full-time students, government payments or subsidies and the training paid by an employer. It is also important to note that employer costs may be underestimated due to commercial sensitivities.
committed funding scenario for 218,000 students over a five-year period, and $11.3 billion based on an expanded funding for 528,000 additional individuals.
Accounting for direct and indirect costs will reflect a more accurate and credible ROI analysis. Direct costs are easier to capture and represent objective measures. These include, for example, training provider fees, cost of training materials and travel or accommodation. In contrast, indirect costs must be derived from other variables. For example, indirect costs for employers may include participant wages (including on-site costs, tax, and superannuation and workers compensation), cost of temporarily replacing staff (recruitment and induction costs) or productivity loss while being trained. Brown et al. (2015) calculated that the costs to a manufacturing enterprise for staff paid to attend workplace literacy training totalled $73,000. These figures are derived from hours in training and linked to hourly rates to calculate labour costs.
Table 11 presents a mapping of the ROI indicators by stakeholder group and examples of how to measure each item. Each group has a slightly different focus:
For individuals, foregone earnings constitute a large proportion of costs for students. A cost of foregone earnings is based on the time spent studying and the wage that the student would have earned had he/she not been studying. This figure is an indirect cost and can be derived from the difference between the annual net income of persons who have completed a specific level of schooling and are not studying full-time for a qualification, and persons who have completed the same level of schooling and are studying full-time for the qualification (for persons aged 15 to 64) (Long and Shah, 2008).
IDENTIFY BENEFITS
1. ROI indicators for individuals target student numbers related to employment, skill attainment, improved status or active participation, increased earnings and satisfaction ratings 2. ROI indicators for employers focus on measures related to time, quality and cost savings for market benefits, while non-market benefits are associated with employee satisfaction ratings, absenteeism/retention and injury rates and environmental cost savings 3. ROI indicators for the economy/society capture direct GDP figures, number of students by qualification and social networks, self-rated health scales, participation rates and level of business activity Table 11 also indicates some of the measures that may convert to monetary values. They are marked by the dollar sign symbol ($).
Alternatively, foregone earnings can be calculated based on the proportion of study time derived from the contact hours for each level, plus an allowance for study outside of class hours (Independent Economics, 2013). During a normal working week, an average student is estimated to spend 37% of their time studying for a Diploma or Advanced Diploma, 30% of their time studying for a Certificate III/IV and 25% of their time studying for Certificate I/II. The loss from the reduction in time that the student has available to work is calculated. For individuals who are likely to have been employed if they had not entered TVET, foregone earnings are estimated using the wage and occupation that they would have had. For those not employed before entering TVET, there are no foregone earnings. Independent Economics (2013) estimated foregone earnings of $4.7 billion under an additional
30
Understanding the return on investment from TVET – a practical guide
TABLE 11 Return on investment indicators by stakeholder group and examples of measures ROI INDICATORS
EXAMPLE OF MEASURES
Individuals Employability skills
Number of students attaining non-technical skills and knowledge required to get a job. Skills include communication, self-management, planning, teamwork, decision making and problem solving
Employment
Number of students employed after training (full-time/part-time work). If the occupation is known this can be linked to a wage and converted to a monetary value
Improved employment status
Number of TVET graduates who improve their employment situation. Categories include casual to permanent, part-time to full-time or promotion to a higher level position. A monetary value can be estimated if the occupation is reported
Wages/earnings
Increased gross earnings after tax, weekly earnings, income earnings per hour, pretax hourly wages or annual earnings
Entrepreneurship
Number of students attaining skills, knowledge and mindset to transform ideas into action. Includes business planning, resource allocation, innovation, creativity, project management, financing, business skills, customers, marketing for social enterprises and commercial ventures. Number of start-ups, social enterprises or community projects
Well-being
Satisfaction with job/work or financial situation using rating scales of one to five – ‘not satisfied’ to ‘very satisfied’ – or derived from interviews
Foundation skill gains
Number of students attaining language, literacy and numeracy skills (reading, writing, oral communication, numeracy and learning skills) and financial literacy. The impact of literacy skills can be measured by an improvement in performance (e.g. reduced errors in the workplace) and this can be converted to hours saved and monetized by hourly wage
Socio-economic status
Number of TVET graduates reporting improved employment (part-time to full-time, self-employment), increased household income, ability to make ends meet (survey question) , positive change to remoteness or living standards
Social inclusion
Number of students who are a member of a club, organization or social network. Civic participation includes volunteering, sporting bodies and other social communities
Business Increased productivity
Reduced management, supervision or trainee/employee time to complete tasks can be converted to hours and associated hourly rates ($). Other examples include increased response time to service orders ($), reduced downtime (hours, $), reduced shutdowns (hours, $)
Profitability
Cost savings from reduced absenteeism and recruitment cost savings from reduced turnover. Other examples include increased sales ($), reduced management/ supervision/employee labour cost savings and time ($) and waste reduction measured by removal costs ($)
Quality product/ service
Customer satisfaction/service measured on rating scales of one to five, or ‘unsatisfactory’ to ‘very satisfied. Can be measured indirectly by, for example,
Understanding the return on investment from TVET – a practical guide
31
increase in revenue, reduced number of complaints and time handling complaints ($), reduction in rework costs (can be measured as % of labour costs), and decreased errors or defects and cost per unit ($) Business innovation
Number of new products/services, number of new competencies attained related to innovation, percentage of employees/management time spent on strategic innovation practices ($), and percentage of employees trained on new tools and technologies
Employee wellbeing
Satisfaction with job or financial situation using self-reported rating scale one to five or ‘not satisfied’ to ‘fully’ satisfied, and improved self-confidence, morale or reduced stress as measured by rating scales or qualitative measures from interviews or case studies. A measure of stress reduction can be linked, for example, to absenteeism or retention ($)
Employee engagement
Decrease in absenteeism measured by number of workers x absenteeism rate x unit of absenteeism ($), turnover decrease measured by number of workers x turnover rate x unit of turnover ($), and increase retention and reduction in advertising and induction costs.
Workplace safety
Injury rate measured by reduced workplace premium insurance ($), reduced number of injuries, decreased number of incidents reported, and Lost Time Injury (LTI) indicator.
Environmental sustainability
Waste reduction measured by reduction in waste removal costs ($) or energy savings indicated by reduction in energy costs ($)
Economy/society
32
Economic growth
Gross domestic product (GDP), real GDP (labour productivity)
Labour market participation
Increase in the TVET graduate labour market participation
Unemployment rate
Decrease in the TVET graduate unemployment rate
Skilled workforce
Number of TVET qualifications by level for graduates who complete training by industry group, occupation or trade, or level of higher earnings by qualification and productivity level as measured by wages ($)
Entrepreneurial activity
Number of business start-ups, number of jobs created and number of patents/ inventions filed. Other examples include increase in enterprise size or growth, number of social enterprises or community development projects and value creation indicators
Health
Self-rated health based on rating scales one to five or ‘unsatisfied’ to ‘fully satisfied’. Other examples include Body Mass Index (BMI) derived from individual weight and height and chronic health condition indicator from survey questions
Social cohesion
Indicator of crime reduction, civic unrest status, freedom status (in terms of political rights and civic liberties)
Social equity
Positive change in the participation rate of disadvantaged groups in TVET in relation to low socio-economic status, disability, location remoteness and cultural/language barriers
Social capital
Number and type of social networks. Others include indication of civic engagement through volunteering, active citizenship, member of clubs, organizations and social groups
Understanding the return on investment from TVET – a practical guide
DEFINE DATA SOURCES The data collection process begins with identifying appropriate data sources. These can range from qualitative research such as case studies, interviews, surveys or questionnaires, to sophisticated database systems that support many years of data capture. A practical way to demonstrate valid data sources is through examples of the datasets used by real studies. Table 12 presents examples of the data sources used to calculate the ROI based on the case studies featured in Section 6, as well as the associated cost and benefit measures. It shows a sample of the varied TVET themes and the costs and benefits that are specific to the stakeholder needs and objectives. It provides an understanding of how costs and benefits are selected from relevant data sources that match the aims of each study to determine a ROI.
Brown et al. (2015) used case studies to derive outcomes from Australian enterprises implementing workplace literacy programmes. Jansen et al. (2015) analysed the data from a BIBB cost-benefit survey consisting of a sample of German companies surveyed on apprenticeship training and recruitment. Cedefop (2011) extracted social data from the European Community Household Panel survey covering fifteen European countries. Wang (2008) applied a blend of craft training survey with industry data and estimates from experts in the construction industry to establish a cost-benefit ratio. Finally, at the economy level, Independent Economics (2013) used three national datasets. It is also important to note that the timeframe for these studies ranges significantly, from six months to many years.
TABLE 12 Data sources, associated costs and benefit measures THEME
DATA SOURCE
COSTS AND BENEFITS
Workplace literacy (Brown et al., 2015)
Data collection template for workplace literacy provided to a sample of Australian enterprises to estimate costs and productivity gains from the Workplace English Language and Literacy programme. Data was collected at three collection points – at or near commencement, directly after training and six months after completion.
Enterprise A: Lean manufacturing, worker engagement and productivity. Total costs consisted of employer contribution to the Workplace English Language and Literacy programme, staff labour costs to attend training and lean manufacturing programme costs. Cost savings related to injury rates, energy, labour and recycling waste.
BIBB Cost-benefit survey 2012/13 dataset consisting of a sample of 3,032 German companies providing training and 913 nonteaching companies surveyed on apprenticeship training and recruitment.
Personnel costs cover wages and expenses for apprentices and trainers, procurement costs for apprentice tools and equipment, training workshops/in-company teaching, training materials and administration. Benefits are referred to as returns and include workplace wage savings for unskilled and skilled tasks, production in training workshops and grants received to support programmes.
Apprenticeship training (Jansen et al., 2015)
Enterprise B: Improving documentation and productivity in health and community services. Training costs covered employer contribution to the programme, additional staff to support training, training materials and programme development and customization. Post-training benefits were identified as supervisor labour cost savings and trainee labour cost savings to complete documentation.
Understanding the return on investment from TVET – a practical guide
33
Social benefits of VET for individuals (Cedefop, 2011)
European Community Household Panel survey consisting of economic and social indicators from 1994 to 2001 across fifteen European countries.
Health, well-being and civic participation were selected to measure social outcomes. Health is measured by self-rated health, chronic health conditions and Body Mass Index (BMI), well-being by work and financial satisfaction levels and civic participation by membership of an organization.
Cost-benefit ratio for construction craft training (Wang, 2008)
Craft training survey of ninetythree North American construction companies, existing construction industry data and case studies of project data from two construction companies.
Baseline labour costs were derived from estimations from experienced contractors and training costs were estimated at one per cent of labour costs. Cost-savings (benefits) were determined by the productivity improvement and decrease in turnover, absenteeism, injury and rework.
Economy wide costs and benefits (Independent Economics, 2013)
Included National Centre for Vocational Education Research (NCVER) financial, student and course datasets, Australian Bureau of Statistics Education and Training Experience Survey and the Bureau’s 2011 National Census.
The forecasting economic model examined three policy scenarios over five years to estimate the number of TVET students that can be funded from additional spending and the flow on costs and benefits. Costs covered direct tuition costs consisting of government contribution, student fees and charges. A cost of foregone earnings was also calculated. The study estimated economic benefits of employability and productivity (wages).
ISOLATE FACTORS AFFECTING THE RETURN ON INVESTMENT Performance is affected by many factors. Although training may influence performance outcomes, it is difficult to separate the impact of training from the influence of other variables. Therefore, it is important to validate the results from descriptive analysis and isolate mediating factors. A variety of multivariate statistical techniques can be used to isolate and control variables from basic regression to more complex equations. For example, to test whether the results of descriptive analysis of apprentice training costs and benefits in Germany were valid, Jansen et al. (2015) used a multivariate three OLS (ordinary least squares) regressions. The dependent variables were gross costs, returns and net costs. The company attributes formed the control variables. They were categorized as company size, domain of training, region, duration of training and existence of company training workshop and number of apprentices for a training occupation within the company.
34
To estimate the association of TVET and social outcomes across fifteen European countries, Cedefop (2011) used two forms of linear regression. The study focussed on social benefits of health, well-being and civic participation. A linear regression was used to establish the average difference in levels of social outcomes while controlling for year, country effects, initial educational qualifications and gender. A fixed effect linear regression was used to estimate the average change in social outcomes over time while controlling for year effects only.
CONVERT DATA To calculate the ROI, measures must be converted into monetary benefit. There are a number of techniques that can be used to achieve a monetary value. Measures of indicators based on time, quality or cost savings are easier to value and monetize. Wages or earnings are already monetized. Firstly, to calculate a monetary value, identify the unit of improvement and establish the value of each unit. Then determine the unit performance change
Understanding the return on investment from TVET – a practical guide
and the annual (or relative timeframe) performance change to finally calculate the monetary value (Phillips, 2015). For example, if we were to use turnover as a measure of employee engagement, then a unit of turnover is one occurrence. The value of each unit is $2,000 per occurrence. The unit change is an average turnover rate of 15.41% of total workers. The total monetary value is calculated as total workers by turnover rate by unit cost of turnover. This translates to 751 workers x 15.41% of total workers x $2,000 per occurrence = $231,458 (Wang, 2008). Some social indicators can also be converted to monetary value. The indicator can be monetized by associating with a related objective measure from a tangible benefit. For example, reduced stress as a measure of employee well-being can be linked to absenteeism or retention. These figures can be valued as cost savings based on individual hours and monetized using average wages. There are several methods that can be used to determine the value of a measure (Philips, 2015): 1. Standard values are available to derive measurement, particularly in terms of turnover, productivity and quality. Productivity improvement is directly converted to cost savings. Wang (2008) used 4.4% of labour costs to derive baseline rework figures and 1% of labour costs to establish total training costs in the construction industry. 2. The value of time can be converted into monetary value. Based on individual hours, the associated wages can be linked and monetized. In measuring the impact of workplace literacy training and productivity, a manufacturing enterprise collected data on the change in trainee hours to complete documentation tasks and associated supervision time (Brown et al., 2015). Trainee labour cost savings were calculated based on hourly wage rates, i.e. 10.5 hours by $33.10 per hour, a cost saving of $348 per trainee. 3. Internal/external experts can estimate units of measure. In construction craft training (Wang, 2008) study, baseline labour costs were derived from the average time estimated by experienced contractors/owners for each major construction activity. The hours were reassigned into trades and the average hourly cost of each trade was identified.
4. External databases can be used to calculate the value of a unit of measure. This includes research, industry and government datasets. For example, injury rates can be a measure of workplace safety with a baseline average rate of 6.4 per 100 full-time workers per year, as derived from labour statistics (Wang, 2008). 5. Qualitative measures may be linked to other variables that are easier to measure and convert to monetary value. Connect the intangible measure to a tangible one. For example, customer satisfaction may be linked to revenue or sales and job satisfaction to employee turnover. Foundation skill gains and literacy may be measured by reducing supervision time and labour costs, as well as trainee time and labour costs associated with time to complete tasks (Brown et al., 2015).
IDENTIFY INTANGIBLE BENEFITS Intangible benefits are defined as measures that are not converted to monetary values. They are measures that cannot easily be monetized or purposely not converted because the outcomes are not meaningful. Some social indicators, such as health and wellbeing, may be quantified but not in terms of economic value. Cedefop (2011) measured health on a five-point self-rated scale from ‘very bad’ to ’very good’ and derived Body Mass Index (BMI) from individual weight and height. Satisfaction with the job or financial situation measured wellbeing using self-reported scales of ‘not satisfied’ to ‘fully satisfied’. The Brown et al. (2015) study also collected qualitative measures that focused on the level of satisfaction with training, gains related to new knowledge and skills and behavioural changes. These findings were reported alongside the ROI. Some intangibles can be counted. For example, the number of individuals employed after training and the number of business start-ups from entrepreneurial activity is quantifiable. Other benefits can be presented as percentages as in the case of labour force participation and unemployment rates. ROI indicators for businesses are primarily monetized, however those based on attitudes or perceptions illustrated by financial or job satisfaction rating scales to measure employee well-being can be considered qualitative measures. If the data cannot be converted to a monetary value, then it can be left as an intangible benefit.
Understanding the return on investment from TVET – a practical guide
35
CALCULATE THE RETURN ON INVESTMENT
Return on investment
After totalling all costs and benefits, the ROI is used to calculate the benefit training will receive in relation to its investment cost. The higher the ROI the greater the benefit earned. The ROI is calculated using the monetized training benefits and training costs. This section will briefly consider the benefitcost ratio and the ROI calculation for the costs and trainings benefits that have been monetized.
ROI is the net training benefits divided by the training costs. The net benefits are the training benefits minus training costs. The result is expressed as a percentage of the costs.
A worked example is also presented to guide the process. Benefit-cost ratio The benefit-cost ratio is calculated as the training benefits divided by the training cost. The result is expressed as a ratio. Benefit-cost ratio = Training Benefits Training Costs When the benefit-cost ratio is greater than one, the benefits outweigh the costs. When the benefit-cost ratio is less than one, then the costs exceed the benefits.
ROI (%) = (Training Benefits – Training Costs) x 100 Training Costs = Net Training Benefits x 100 Training Costs
Below is an example of a ROI calculation for a training programme designed to improve workplace literacy levels for thirty-one employees in a manufacturing environment (Brown et al., 2015). As can be seen by the costs and benefits featured in Table 13, the enterprise measured productivity through cost savings from reduced injury, energy, labour costs and waste reduction. The complete case study is featured in Section 6.
TABLE 13 Example costs and benefits for return on investment calculation MEASURE
CALCULATION
Injury rate reduction
Insurance premium reduced from 5% to 3.7%, saving approx. $1,300 per week
$67,860
Energy savings
Projected annual savings based on initial monthly savings
$42,000
Labour savings
Efficient practices save 316 hours per year @ $26 per hour
$8,216
Recycling waste
Waste collection costs reduced from $5,500 to $4,000, saving $1,500 per month Total training benefits
$18,000 $136,076
Literacy training programme Staff labour costs
Employer contribution
$10,000
Staff paid to attend training
$73,000
Lean Manufacturing Programme
Cost $85,000, less $64,800 Government rebate
$20,200
Total training costs
36
ANNUAL ($AUD)
Understanding the return on investment from TVET – a practical guide
$103,200
Using these figures from Table 13, a worked example of the ROI calculation is presented:
Worked example of ROI Calculation Benefit-cost ratio = $136,076 = 1.31:1 $103,200 For every $1 dollar invested there is a $1.31 training benefit Net training benefits $136,076 - $103,200 = $32,876 ROI ROI (%) = $136,076 - $103,200 x 100 = 31.8% $103,200 For every $1 dollar invested, $0.31 is returned after the investment is deducted
Based on this example formula, ROI figures vary with the stakeholder context. If we consider the costs and benefits from the case studies presented in Section 6, the results in Table 14 give a range of cost benefit ratios and ROI values. In particular, note
that apprenticeship training is focussed on reducing the net costs. However, these figures are calculated as a benefit-cost ratio and ROI (as a percentage) to provide a consistent frame of reference and comparison to the other financial metrics.
TABLE 14 Benefit-cost ratio and return on investment calculations for training programmes TRAINING PROGRAMMES/THEME
Workplace Literacy (Brown et al., 2015) (AUD$) Lean manufacturing and worker engagement Improving documentation and productivity Apprenticeship training (Jansen et al., 2015 / EUR) Average per year (net costs) Construction craft training (Wang, 2008) (US$) Expected from craft training - Average Expected from craft training - Minimum Expected from craft training - Maximum Economy wide increased funding (IE, 2013/ AUD$) Committed funding scenario over 5years Expanded funding scenario over 5 years
BENEFITS
COSTS
NET BENEFITS
$136,076
$103,200
$32,876
1.31
31.8%
$11,756
$10,005
$1,751
1.17
17.5%
€12,535
€17,933
€5,398
0.69
- 30.1%
$521,594 $349,252 $693,937
$230,296 $230,296 $230,296
$291,298 $118,956 $463,641
2.26 1.52 3.01
126.5% 51.6% 201.3%
$20.4 billion $7.0 billion $13.4 billion $49.5 billion $17.0 billion $32.5 billion
2.91 2.91
191.4% 191.1%
BENEFITCOST RATIO
ROI (%)
Understanding the return on investment from TVET – a practical guide
37
Report return on investment ROI is by design a simple indicator of benefits relative to costs. However, it is important to note that not all data can be monetized. This is particularly the case for some social indicators. Intangible benefits are just as important as financial figures, most notably in the context of social returns. To produce credible evidence and a more complete picture of returns, it is important to report intangible benefits alongside ROI. It is necessary to also acknowledge that ROI calculations need to be kept in perspective and recognized as estimates.
38
Understanding the return on investment from TVET – a practical guide
5. Planning to calculate the return on investment from TVET Planning to implement a ROI calculation requires a number of steps, with the aim being to set up and implement a practical, coherent and transparent methodology; one that measures the ROI from TVET on the basis of selective and useful variables, and one that produces the most compelling evidence possible based on what data is available.
To develop the scope:
This section maps out a planning strategy to address these key aspects of the ROI framework. It presents a list of helpful steps to guide the planning process and form the basis of an ROI implementation plan, designed to suit the needs and priorities of the specific TVET environment. It also refers to more specific information on mapping indicators and the section concludes by highlighting the enablers and challenges to measuring ROI.
•
STEP 1: DEFINE THE TVET ENVIRONMENT Describe the TVET context ROI is context specific to the stakeholder and relative to the environment. Hence, developing an implementation plan requires an understanding of the TVET context. Therefore, a definition and description of the TVET environment is the first step to identify the context upon which to base the findings of the ROI evaluation. To define the ROI context: • Describe the TVET system • Define the national and economic structures in which TVET are embedded • Describe the nature of training • Adopt a working definition for ROI from TVET Define the scope There are multiple levels of stakeholders including individuals, enterprises and economies. In each of these groups, there are multiple dimensions of ROI. These include economic, social and environmental perspectives. Information about the stakeholder profiles provides the scope to measure the ROI in TVET.
• • • •
Identify who the stakeholders are Identify stakeholder aims, objectives, needs and priorities Define the nature of the training Develop a specific statement that helps to focus the ROI analysis Determine how to involve stakeholders (if required)
Establish the purpose A ROI analysis can fulfil a range of purposes. For example, this can range from funding agreements to improving business efficiency. Understanding the ROI intent of the stakeholder provides focus and drives better decisions to measure performance. To understand the purpose of measuring ROI from the stakeholders’ perspective: • • •
Clarify the reasons why the stakeholder is measuring the ROI Identify specific motivations driving the work Establish how the ROI information will be used
STEP 2: ADOPT GUIDING PRINCIPLES Adopting a set of guiding principles provides a foundation to focus and plan to measure the ROI, and is an underlying core for a consistent and standard frame of reference to achieve better outcomes. Consideration of these four aspects is important in the planning stage: 1. The ROI approach adopted is fit-for-purpose, adds value, measures indicators that matter and are specific to stakeholders’ needs and priorities 2. Implementation is practical, useful, feasible and agreed upon by all stakeholders 3. The development of the methodology and data collection instruments/processes place minimal administrative load on the stakeholders, can be customized to particular contexts, and are sufficiently specific regarding the data elements required 4. A compilation of credible evidence about the impact of training must satisfy a number of requirements – the data must be of sufficient quality, analytical techniques applied must
Understanding the return on investment from TVET – a practical guide
39
be scientifically valid and the analysis should address the possibility that training may not be the only factor explaining changes in performance
STEP 3: ESTABLISH OBJECTIVES Determine return on investment indicators to measure There are many layers and dimensions to measure the ROI from TVET. They are different for each stakeholder. As illustrated in the framework presented in this Guide, there are economic and social aspects and within each of these there are high-level ROI indicators that measure different areas of performance. These are varied across individuals, business and the economy. It is crucial to measure the ROI indicators that meet stakeholder needs and objectives. To identify the most important ROI measures to stakeholders: • •
• • • •
Determine the level of impact – economic and/ or social – that is relevant to the stakeholder Select ROI indicators based on usefulness, practicality and capacity to add value, along with the ability to apply to different types of training and contexts Define the ROI measures that are most appropriate for stakeholders Identify the range of activities to target and analyse Describe what you intend to measure Establish the timeframe or data collection points
Determine the return on investment model Various models are used to determine the ROI. These models include measuring economic and social impact along with the choice of evaluative or forecasting measurement tools to calculate returns. Different models may suit specific types of data, which range from case studies to full-scale data systems and includes Social Return on Investment to Cost-Benefit Analysis. Furthermore, a ROI model lends itself to different types of measurement. The decision to include specific economic and social returns and the stakeholder expectations will influence the selection of the most appropriate ROI model to deliver the best outcomes. For the purposes of this Guide, we have focussed on the ROI indicators and measures that can apply to different models.
40
However, to identify a fit-for-purpose ROI model: •
•
•
Select what type of ROI model is most appropriate – evaluative (determining the ROI spending that has occurred) or forecasting (investigating potential return should spending/funding be altered) Define a ROI model that suits stakeholder outcomes and economic or social indicators. The Cost-Benefit Analysis is an example of an evaluative model and economic modelling fits into the forecasting approach Determine if the ROI model can be customized to fit the TVET context
Determine the resources required One of the key considerations to measuring the ROI is resource requirements and commitment to undertake this activity. Before embarking on the measurement process, planning the logistics of this ROI exercise is necessary. To understand the level of resources required to measure the ROI: • • • • •
Determine the resources required, including human resources and financial requirements Determine the availability of resources and who will carry out the work Establish who is involved with the decisionmaking process Determine a timeframe for the ROI calculation activity Establish the vision and goals
STEP 4: COLLECT DATA There are three main stages to the data collection process – preparing information, collecting data sources and establishing the quality of data. Listed are the guidelines to support an initial ROI data collection process. Establish data collection principles As a first step to collecting data based on the ROI framework and indicators, there are a number of underlying principles to guide the process: 1. The target stakeholders and the purpose for measuring ROI are well defined 2. The key ROI indicators are designed so that they are sufficiently specific but general enough to apply across various training contexts and environments
Understanding the return on investment from TVET – a practical guide
3. The focus is on identifying existing data sources that relate to the ROI indicators 4. The approach is to adopt transparency and clearly identify factors that may affect the type and quality of data and acknowledge the potential impact 5. The definition of the contextual underpinnings of the data and stakeholder groups is fundamental to understanding the ROI landscape Prepare data collection pro forma samples
To prepare information to guide the data collection process, pro forma samples for each stakeholder group by ROI indicators are developed. The ROI pro forma resources in this guide includes a summary of both economic and social ROI indicators. The data collection sheets provided include a ROI Data Collection Pro forma Guide and ROI Data Collection Summary List of ROI indicators and includes a ROI financial data and costs data sheet categorized by stakeholders groups. The following pro forma resources are provided: ROI job-related indicators for individuals ROI non-job-related indicators for individuals ROI market indicators for business ROI non-market indicators for business ROI economic indicators for the economy ROI social indicators for society ROI financial data and costs to stakeholder groups
Steps to establish existing data sources that can be used to measure the ROI indicators and the data availability include:
• •
• • •
Document a description of the data, including the year and source Define the scope of the data or profile of the population (e.g. graduates, employees, etc.) Identify the key data elements — for example stakeholder demographics, training programme and field of education Record factors that affect the result and describe the potential outcome
The ROI Cost-Benefit Analysis ratio can be an estimate of the impact of training. The degree to which it is an estimate of return depends on the quality of the data that are used to calculate the ratio. Issues of data availability, completeness and usefulness will also affect the integrity of the data. Keeping in mind the importance of data quality at all stages of data collection, analysis and interpretation will enable a more true and accurate measure of performance. In order to establish the quality of data and interpret the data accurately a number of steps need to be considered. Record the information on the pro forma sheets: • • • • •
Identify the existing data sources that relate to the key ROI indicators and stakeholder groups, including financial training cost data Determine the availability of data and level of accessibility Define the type of data source —both quantitative and qualitative. Examples include administrative datasets, international collection,
Determine the completeness of the dataset Identify any data limitations Record the information or data gaps by stakeholder group and ROI indicators Explain any anomalies in the data Define the contextual underpinnings of the data
When planning for the collection, analysis and interpretation of data, further consideration is required to: •
Collect data sources
•
•
Establish data quality
The data collection process requires preparing information guidelines, data templates, establishing existing data sources and determining the completeness of the data and highlighting any information gaps and anomalies. This includes customizing data collection instruments, supporting documentation that fits the context and defining the ROI indicators to be measured.
1. 2. 3. 4. 5. 6. 7.
case studies or longitudinal surveys
•
•
Establish the integrity and credibility of the data. This refers to data validity (is it measuring what it is supposed to measure) and reliability (is the data consistent and reproducible) Establish the basis for data comparisons (if required). In order to compare data, there needs to be consistent definitions across datasets to determine which data is and is not directly comparable Ensure that the data is transparent in order to establish a more accurate interpretation of ROI results. This requires explaining the contextual underpinnings
Understanding the return on investment from TVET – a practical guide
41
STEP 5: IDENTIFY COSTS AND BENEFITS
Identify training benefits
Identifying accurate costs and benefits to TVET are critical to determine the ROI. However, costs and total investment are generally underestimated and indirect costs are more difficult to measure. The relationship between costs and benefits in the ROI analysis is also important to defining a more accurate ROI calculation. Costs are known upfront – before training – but benefits may accrue over time, and it may be difficult to determine when to assess the impact. Furthermore, direct training costs are more likely to be known and expressed in monetary terms, but the benefits may be subjective and more difficult to quantify for monetary conversion.
First, establish the timeframe or data collection points for training benefits. For specific stakeholder groups and ROI indicators, identify data that measure change in terms of cost savings, time, number of people, participation rates or rating scales associated with:
During the planning stage, consideration needs to be given when to collect data on the costs and benefits. The collection points may target before training (baseline), after completing training or six months post-training. An annual timeframe or over several years can be used for forecasting models or when using longitudinal survey data. Identify training costs Training costs primarily fall into five main categories – identify data according to these classifications. Refer to the Mapping ROI Indicators for a detailed list. • •
• • •
Programme costs related to tuition fees, needs analysis, course design and programme development and purchase costs Personnel costs for internal and external trainers, additional training support staff, apprentice wages, labour costs for staff attending training and expenses for staff replacement Facilities and hiring costs of venues, equipment and hardware Teaching and learning materials, administration and development of instructional resources Individual costs related to student fees, equipment and tools, protective clothing, transport, books and materials, membership fees and licences, other consumable materials and foregone earnings
Derive the indirect costs. For example, foregone earnings are a significant indirect cost that must be derived from other variables. Refer to the Section 4 (Mapping the ROI Indicators) to define how to calculate this cost. Staff replacement costs (recruitment plus induction costs) for employees during training and opportunity costs in terms of productivity loss also need to be derived.
42
For individuals: • Job-related benefits from employability skills, employment, improved employment status, wages/earnings and entrepreneurship • Non-job related benefits from well-being, foundation skill gains, socio-economic status and social inclusion For businesses/employers: • Market benefits from increased productivity, profitability, quality product/services and business innovation • Non-market benefits from employee well-being, employee engagement, workplace safety and environmental sustainability For the economy: • Economy benefits from economic growth, labour market participation, unemployment rates, skilled workforce and entrepreneurial activity • Societal benefits from health, social cohesion, social equity and social capital Isolate the impact of other factors There can be many influences on the outcomes of the ROI analysis. Understanding the factors that affect the results are as important as costs and benefits to have an accurate picture of the ROI from TVET. The multitude of influences that impact on training benefits include characteristics of stakeholders, training programmes, enterprises, training providers and the quality of the data: •
•
• •
Select an appropriate statistical technique that isolates the effect of variables. For example, a multivariate statistical method using regression analysis to validate the impact of training Determine the control variables. For example, for employers this can relate to company size, region or domain of training and for society benefits in terms of health and well-being, variables such as gender, country effects or initial educational qualifications may be appropriate Analyse the data to determine the influence on ROI estimates Determine the reasons for any variations in the ROI estimates when interpreting the results
Understanding the return on investment from TVET – a practical guide
STEP 6: ANALYSE DATA
STEP 7: CALCULATE THE RETURN ON INVESTMENT
Convert data to monetary values To calculate the ROI, measures must be converted into monetary values. First, to calculate a monetary value, identify the unit of improvement and establish the value of each unit. Then determine the unit performance change and the annual (or relative timeframe) performance change to finally calculate the monetary value.
The ROI is used to measure the benefit training will receive in relation to its investment cost. The ROI from TVET is calculated using the monetized training benefits and training costs: • •
To determine the value of a measure, several conversion methods can be implemented: • • • • •
Use standard values to derive measurement, particularly in terms of turnover, productivity and quality measures Convert time into a monetary value based on individual hours and associated wages Use internal/external experts to estimate units of measures Access external databases – for example research, industry and government datasets – to calculate the value of a unit of measure Link qualitative measures to other variables that are easier to measure and convert to monetary value. For example, customer satisfaction can be linked to revenue
Indirect cost variables, such as foregone earnings, must be calculated from other variables. Refer to Section 4 (Mapping the ROI Indicators) for examples of this definition and conversion of this data to a monetary value along with other conversion methods.
To determine the ROI, total the training costs and training benefits Calculate the Benefit-Cost Ratio based on the training benefits divided by the training cost. The result is expressed as a ratio:
•
Benefit Cost Ratio = Training Benefits Training Costs
Calculate the ROI as the net training benefits divided by the training costs. The net benefits are the training benefits minus training costs. The result is expressed as a percentage:
ROI (%) = (Training Benefits – Training Costs) Training Costs x 100
= Net Training Benefits x 100 Training Costs
Identify intangible benefits It is important to note that not all the data collected will be in monetary form. Measures not converted to monetary value are referred to as intangible benefits. If the data cannot be converted to a monetary value, then categorize the measure as an intangible benefit and quantify or qualify the result. For example, job satisfaction with a rating scale, and health and well-being measures based on scaled subjective responses are qualitative data, while the number of students attaining a job is a quantitative outcome.
Not all data can be converted to monetary value. The ROI can be supported by other qualitative and quantitative data. Report the ROI with an analysis of the intangible benefits.
Understanding the return on investment from TVET – a practical guide
43
ENABLERS AND CHALLENGES TO MEASURING THE RETURN ON INVESTMENT Several key factors can enable the effective planning and implementation of a ROI calculation. These include clarifying the purpose to drive and focus the ROI calculation with full engagement of stakeholders, and adopting a set of principles that can help guide the implementation process. It is also important to ensure that the ROI measures are useful, add value, and meet the needs and priorities of the stakeholder. The ROI calculation needs to be practical and needs to minimize the administrative burden for those involved in the process. The data collection needs to support a variety of data sources types and apply to different contexts. Finally, to obtain a more accurate understanding of the ROI outcomes, the contextual underpinnings of the data need to be well explained and a clear picture of the TVET environment needs to be documented in which to base the findings. There are also challenges to effectively calculating the ROI from TVET. These include a lack of vision and buy-in to the decision-making process and limited stakeholder involvement in the ROI activity. Selecting an inappropriate model to calculate the ROI can produce variable results, as can inadequately identifying costs and benefits, and inappropriately converting programme results to monetary or quantifiable values. Difficulties in isolating or controlling for variables that may affect training can also pose an issue, along with the quality of the data.
KEY GUIDING QUESTIONS Context • What are the contextual underpinnings of TVET specific to your environment? • What is a working definition for ROI in TVET? Scope • Who are the stakeholders? • What level of impact should be considered – economic, social and/or environmental? • What is the nature of the training? Purpose • Why are stakeholders measuring the ROI from TVET? • How will the ROI information be used?
44
Return on investment measures • What measures are most useful and valuable to stakeholders? • What is the data timeframe? Return on investment model • What type of ROI model is appropriate evaluative or forecasting? • Do we measure economic and social impact? Which model is fit for purpose? • Can we customize the ROI model? Does it add value? Resources • What resources are required and available? • Who will carry out the implementation? Data sources • What existing data sources are available? • What is the data quality and completeness? • What are information gaps? Training costs • What are the training costs? • Over what period are the costs calculated? • How can we best measure indirect costs? Training benefits • What are the tangible benefits? • What are the intangible benefits? • When are the most important data collection points? Impact on training • What are the control variables? • What statistical technique can be used to isolate the effect of training? Data conversion • What is the best method to convert data? • What data can be converted to monetary value? • How can we best calculate indirect costs? Return on investment • What are the total costs and benefits? • What is the Benefit-Cost Ratio? • What is the ROI? • What other data is important to report?
Understanding the return on investment from TVET – a practical guide
6. Implementing frameworks to measure the return on investment (Case studies) This section presents five different case studies to illustrate how the ROI is measured and calculated in differing contexts. These studies have been selected to present real examples of deriving ROI. They include returns to individuals, enterprises and the economy, encompass social and economic
indicators, and use varied statistical methods and datasets across different industries and countries. Overall, the cases illustrate the diversity of approaches and measures that fit the context and needs of the stakeholders.
TABLE 15 Costs and benefit measures by category for workplace literacy, Australia COSTS AND BENEFITS WELL project costs
MEASURE
DESCRIPTION
WELL and employer contribution, additional costs
Additional costs include facility hire, training support staff, new facilities, training materials and programme development
Personnel costs
WELL trainee wages, supervisor wages
Average hourly minimum and maximum wage + on-costs (superannuation, long service leave, workers compensation, payroll tax)
Operational costs
Estimated annual incidents costs
Lower insurance premiums, penalties, fewer worker compensation, reduced productivity, accidents or injury For example materials consumables, equipment, etc. For example lost productivity, resetting machinery and lost revenue
Estimated waste costs Estimated shutdown costs Human resources costs
Turnover rates (organization) Turnover rates (WELL trainee group) Recruitment/replacement costs Other operational costs
Productivity gains
Hours supervisor(s) work directly with trainees (average per week)
Hours trainees take to complete key tasks (average per week) Other financial benefits
Other financial benefits
Number of employee departures in one quarter/ average number of active employees during the same period Number of employee departures from trainee group in one quarter/average number of remaining group members WELL trainee level Other areas with measurable operational cost over time Changes in supervisor time checking and correcting work (for example notes, documentation, measurements, calculations), explaining job requirements, ensuring standards are met Completion of timesheets, documentation, entry of notes into an IT system, technical/ specialist tasks Product costs per unit, output (per hour, per shift, per machine) and reduced absenteeism
Understanding the return on investment from TVET – a practical guide
45
CASE STUDY 1: WORKPLACE LITERACY A study by Brown et al. (2015) aimed to assist Australian employers who invested in language, literacy and numeracy training to estimate the ROI. Based on the Phillips/Kirkpatrick Evaluation Model, they developed a data collection instrument for workplace literacy to estimate costs and productivity gains from the Workplace English Language and Literacy (WELL) training programme – a then Australian government workplace-based vocational adult literacy programme. The training consisted of core learning, reading, writing and oral communication skills. Table 15 shows the training costs and benefits categories. Training costs included project, personnel, and operational and human resource costs. Productivity gains in supervision time and WELL trainee hours to undertake key tasks were measured across three collection points – at or near commencement, directly after training and six months after completion; these were monetized by hourly wages. Qualitative measures were also included and focused on the level of satisfaction with training, gains related to new knowledge and skills and behavioural changes. The following two enterprises implemented elements of this data collection to measure and calculate the ROI based on their specific needs.
ENTERPRISE A: Supporting lean manufacturing and enhancing worker engagement A manufacturing enterprise in New South Wales (a State in Australia) has been growing, packing marketing and distributing products to supermarkets, food processors and independent grocers in Australia and overseas for more than twenty years. The company has a workforce with significant English proficiency issues and the company aimed to implement the WELL programme to support its lean manufacturing programme and enhance worker engagement and productivity. The WELL and lean manufacturing programme training operated concurrently over one year with systems and process rollouts, including a waste reduction programme. Thirtyone employees attended training during the day for two-hour periods on a bi-weekly basis, with the alternate week attending the lean manufacturing programme. Cost and benefits As detailed in Table 16, costs included the lean manufacturing programme (less a government rebate), the employer contribution to the WELL programme and staff labour costs to attend training totalling $103,200.
TABLE 16 Training costs, benefits and estimated return on investment, Australia PROGRAM COSTS AND BENEFITS
CATEGORY
WELL programme Staff labour costs (employer) Lean manufacturing programme
Employer contribution Staff paid to attend training $85,000 minus $64,800 Government rebate
ANNUAL AUD$ $10,000 $73,000 $20,200 $103,200
Total programme cost Injury rates Energy savings Labour savings Recycling waste
Workcover premium reduced from 5% to 3.7%, saving approximately $1,300 per week Projected annual savings based on initial monthly savings Efficient practices saved 316 hours per year, at $26 per hour per employee Waste collection costs reduced from $5,500 to $4,000, saving $1,500 per month Total programme benefits
Return on investment
Programme benefits/costs x 100
$67,860 $42,000 $8,216 $18,000 $136,076 131.8%
Source: Brown et al. (2015)
46
Understanding the return on investment from TVET – a practical guide
Programme benefits were measured as cost savings to the enterprise over a year. The company identified four main areas of savings totalling $136,076: 1. Injury rates were measured in terms of Workcover1 premiums reducing from 5% to 3.7% and saving $67,860. The WELL programme increased proficiency by employees completing incident reports, increasing their confidence to communicate work health and safety issues in the workplace 2. Energy Savings: As a result of training, energy usage improved (e.g. install sensor lights to save power). Based on the savings over the initial months, the company calculated a $42,000 benefit 3. Labour savings: Downtime of 316 hours per year were identified due to poor communication and loss of production time when forklifts were not available to move raw materials, causing late deliveries and labour costs waiting for products to be moved. The training improved the systems and communication between work areas, with cost savings of $8,216 based on a $26 hourly wage 4. Recycling waste: A major focus of the company is fresh produce and a range of recycling bins were installed. Effective communication and reading skills from training led to a more efficient process and waste collection costs were reduced by $1,500 per month, with total annual savings of $18,000 Return on investment The ROI was calculated as a percentage of the programme benefits divided by the programme costs. This enterprise estimated $136,076 of programme benefits to $103,200 costs, with a resulting ROI of 131.8%. The company indicated this figure to be understated as it excluded flowon effects to productivity gains not reported due to commercial sensitivities. The training improved worker confidence and communication. The WELL and lean manufacturing programmes were delivered concurrently with the same trainer and Registered Training Organization.
1 Workcover is a government agency responsible for overseeing and regulating the workers’ compensation and injury management scheme operating in Australian states and territories.
It was difficult to isolate benefits of each programme, but running the lean manufacturing programme independently had previously been unsuccessful. ENTERPRISE B: Improving documentation and productivity An organization in health and community services in residential aged care located in South Australia provides high and low care services and independent living units. Enterprise B implemented the WELL programme to improve documentation to enhance organizational culture and measure productivity. The training operated for one year over 238 programme hours. Forty employees attended the programme with a focus on supervision and responsibilities, standards of service delivery, workplace culture, open communication and worker performance. Costs and benefits The quantified costs and benefits of the WELL training programme is itemized in Table 17. Training expenditure included employer contribution and additional costs to the employer, staff, materials and programme development totalling $10,005. Productivity benefits were measured by the level of documentation errors and labour cost savings. Data was recorded at three collection points – before, directly after and six months post-training. Management time to read and correct notes of six carers reduced from twenty to five minutes and the average time taken per carer reduced from five to two minutes. The change in total supervision time and time taken for trainees to complete key tasks was calculated and the hours monetized by wages (per hour). Annual cost savings for six months after the WELL training totalled $11,756. Return on Investment The ROI was calculated as an annual percentage of training benefits divided by employer costs. With $11,756 of cost savings and $10,005 employer contribution, the annualized ROI is 117.5%. The training reduced the number of document errors, increased data accuracy and data entry speed and work health and safety reporting, resulting in less time required by senior staff to correct communications. The improved documentation skills of carers produced a feeling of confidence and fewer errors among higher-level staff to correct and rework, saving thirty minutes of management time each day.
Understanding the return on investment from TVET – a practical guide
47
TABLE 17 Training costs and benefits post-training, Australia CATEGORY
LABOUR COST SAVINGS
Before or near start of training
Supervisor hours
Hours per group
9.3
2.8
2.8
6.5
6.5
Supervisor wages ($) Labour costs (supervisor) – hours x wage Number of groups
$50.60 $471
$50.60 $142
$50.60 $142
$329
$329
4
4
4
4
4
Labour costs (supervisor x groups)
$1,882
$567
$567
$1,316
$1,316
17.5
10.5
7
7
10.5
$33.10 $579
$33.10 $348
$33.10 $232
$232
$348
WELL trainees
Total benefits
Six months after training
Directly after training
Change Change six directly after months after training training
Hours to complete documentation Trainee wage ($) Labour costs (trainees) – hours x wage Number of trainees* Labour costs (worker) x number of trainees
27
27
27
27
30
$15,640
$9,384
$6,256
$6,256
$10,440
Annual cost savings
$17,522
$9,951
$6,823
$7,572
$11,756
WELL programme Employer contribution ($35,576 – $27,676 Commonwealth funding) Cost of additional staff to support training Expenditure on training materials (for example computers, printing) Expenditure on programme development/ customization Other (for example photocopying) Total costs Total employer contribution Return on (Annual) Benefits/costs x investment 100 = $11,756/$10,005
$7,900
$1,000 $80
$1,000
$25
$10,005
117.5%
Source: Brown et al. (2015) *trainee numbers adapted to annual cost savings
48
Understanding the return on investment from TVET – a practical guide
CASE STUDY 2: APPRENTICESHIP TRAINING A study by Jansen et al. (2015) aimed to measure the cost and benefit ratio of apprenticeship training in Germany. The analysis used data from the BIBB Cost-Benefit Survey 2012/13 containing a sample of 3,032 companies providing training and 913 nonteaching companies surveyed on apprenticeship training and recruitment of skilled workers. The study also explored additional benefits of training on personnel recruitment costs. Costs and benefits The personnel and non-personnel costs that a company incurs for apprenticeship training constitutes the gross costs. As shown in Table 18, the personnel costs cover wages and expenses for apprentices and trainers. Non-personnel and premise costs include apprentice tools and equipment and teaching consumables, while other costs cover administration, external courses and training materials. Benefits are referred to as returns, and apprentices make productive contributions during their training that generates these returns. In the workplace, the
returns are calculated on the equivalent principle; that is, costs the company incurs if the apprentice activities were carried out by normal employees and valued in terms of wages of the employee group. Some enterprises also receive grants from support programmes run by the German Federal Government, the German federal states, European Social Fund, Federal Employment Agency, occupational or sectoral associations. Return on investment The cost-benefit ratio is derived from gross costs less the returns to give a net cost. With gross costs of EUR 17,933 deducted from returns of EUR 12,535, a net cost of EUR 5,398 was calculated. However, the study indicated all variables presented a high variance as only the average cost per apprentice per year was analysed. To explain the variance, the costs and returns were analysed by company attributes. Table 19 shows the results of the analysis. The gross costs and returns increased with company size, with the highest net costs of EUR 7,354 for more than 500 employees. The public sector domain (EUR 8,032), technical occupation group (EUR 8,939), training workshop (EUR 12,942), three and half year duration (EUR 9,093) and
TABLE 18 Average gross costs and returns per year per apprentice by category, Germany ANNUAL (EUR)
COSTS/BENEFITS
DESCRIPTION
Personnel costs: Apprentices
Gross training wages of apprentices, voluntary and statutory social benefits Wage costs for full-time, part-time (included if it reduces productivity in actual work tasks) and external trainers (fees, travel and overnight stay) Procurement costs for apprentice tools and equipment, costs of training workshops or in-company teaching, materials for teaching Includes chamber fees, teaching and learning materials, external courses, in-company training administration
Personnel costs: Trainers Premises/non-personnel costs Other costs
Training workshop Grants
€ 925 € 1,866
€ 6,210 € 5,875 € 209 € 241
Unskilled activities (wage) Skilled worker activities (wage) Production in training workshop Support programmes
€ 12,535
Total Returns Return on investment
€ 4,125
€ 17,933
Total Gross Costs Workplace
€ 11,018
Gross costs – Returns = Net costs (17,933 less 12,535)
€ 5,398
Source: Jansen et al. (2015)
Understanding the return on investment from TVET – a practical guide
49
TABLE 19 Gross costs, returns and net costs by company attributes per apprentice per year, Germany
COMPANY ATTRIBUTE
CLASSIFICATIONS
GROSS COSTS
RETURNS
NET COSTS (EUR)
Region
Eastern Germany Western Germany
€ 15,726 € 18,309
€ 9,412 € 13,067
€ 6,314 € 5,242
Company size
Up to 9 employees 10 to 49 employees 50 to 499 employees 500+ employees
€ 15,911 € 16,452 € 18,111 € 21,757
€ 10,807 € 12,199 € 12,720 € 14,403
€ 5,104 € 4,254 € 5,391 € 7,354
Training domain
Industry and trade Skilled crafts Agriculture Free professions Public service Home economics
€ 19,535 € 15,187 € 14,043 € 16,474 € 19,801 € 15,329
€ 13,389 € 10,798 € 12,750 € 12,769 € 11,768 € 8,945
€ 6,146 € 4,390 € 1,293 € 3,705 € 8,032 € 6,385
Occupational group
Commercial occupations Industrial occupations Technical occupations
€ 18,206 € 16.116 € 19,092
€ 14,684 € 11,859 € 10,153
€ 3,522 € 4,257 € 8,939
Training workshop
No Training Workshop Training workshop
€ 16,889 € 21,869
€ 13,492 € 8,928
€ 3,396 € 12,942
Duration of training
Two-year Three-year Three-and-a-half year
€ 16,970 € 17,666 € 18,636
€ 11,652 € 13,853 € 9,543
€ 5.318 € 3,814 € 9,093
€ 17,933
€ 12,535
€ 5,398
Total
Source: Jansen et al. (2015)
Eastern Germany (EUR 6,314) indicated the highest net costs.
as do companies providing training in technical occupations. The regression analysis showed that the gross and net costs correlated negatively with To test whether the results of the descriptive analysis the number of apprentices; that is, with every are valid when relevant variables are controlled, additional apprentice the net costs fell by an the study used a multivariate analysis of three average of EUR 47. OLS regressions. The dependent variables were gross costs, returns and net costs. The company Additional benefits attributes formed the control variables, categorized as company size, training domain, region, training The study also analysed data from 1,605 training duration and existence of company training and 406 non-training companies that recruited skilled workers from the external labour market in workshop and number of apprentices for that the last three years. Table 20 identifies the costs training occupation within the company. incurred by employers in the recruitment process, The analysis indicated that the company size had which cover the application process (EUR 928), a significant influence on gross costs and returns, continuing TVET (EUR 723) and induction costs but not so in terms of net costs. Regional variables (EUR 7,063). Hiring qualified apprentices created had no influence on net costs and training duration recruitment cost savings of EUR 8,715 per showed lesser influence. However, the influence apprentice and provided additional benefits to of the training workshop on the cost-benefit lower net costs. ratio indicated higher net costs (over EUR 6,000)
50
Understanding the return on investment from TVET – a practical guide
TABLE 20 Personnel recruitment costs per new skilled worker by cost categories, Germany AMOUNT (EUR)
RECRUITMENT CATEGORY
RECRUITMENT COSTS
Application process
Advertising costs Personnel costs for application process Costs of external consultants
€ 275 € 413 € 240
Continuing TVET during induction
Continuing vocational training costs (fees and subscriptions) Costs of absence from work for continuing vocational training
€ 349 € 374
Induction costs
Personnel costs of induction Reduced performance during induction (recruit skilled workers)
€ 4,097 € 2,966
Total recruitment costs per new skilled worker
€ 8,715 Source: Jansen et al. (2015)
CASE STUDY 3: SOCIAL BENEFITS OF TVET FOR INDIVIDUALS
Social indicators
Cedefop (2011) examined the social benefits of vocational education for individuals in Europe. They explored the association between an episode of vocational education or training with a change in health and social outcomes based on a dataset from the longitudinal European Community Household Panel (ECHP) survey. The ECHP survey consists of economic and social indicators from 1994 to 2001 across fifteen European countries.
To measure social outcomes, health, well-being and civic participation were selected on the basis of the importance to individuals and society, comparability across countries over time and variation of the indicators over time. Table 21 shows the key social indicators and their data source. Health is measured by self-rated health, chronic health conditions and Body Mass Index, well-being by work and financial satisfaction levels, and civic participation by membership of an organization. These social indicators are quantified, but not in terms of economic values.
TABLE 21 Social Indicators, measures and data source SOCIAL INDICATORS
MEASURE
ECHP SURVEY SOURCE
Health
Self-rated health
Question “How is your health in general?”; five ratings from ‘very bad’ to ‘very good’
Chronic health conditions
Question “Are you hampered in your daily activities by chronic physical or mental health problem, illness or disability?”
Body Mass Index
Calculated using individual’s weight and height
Civic participation
Membership of civic organizations
Member of club, neighbourhood/local group or a political party
Well-being
Satisfaction with work/ current main activity
Self-reported scale of one (not satisfied) to six (fully satisfied)
Satisfaction with current Self-reported scale of one (not satisfied) financial situation to six (fully satisfied)
Understanding the return on investment from TVET – a practical guide
51
TABLE 22 Education, individual and system variables by classifications VARIABLES
CLASSIFICATION
Highest qualification achieved
Recognized third level education, second or first stage of secondary education or less than secondary education
Episodes of education/ training
(a) General education (GE) / GE and language; (b) GE and TVET / GE, TVET and language; (c) TVET / TVET and language; (d) Language; (e) No education / training
Life course stages IVET (a) CVET (b- e)
(a) Youth: < 25; (b) Early career: 26-45 + work experience; (c) Mid-end career: 46-60 + work experience; (d) Retired: > 60 + work experience; (e) Never worked: > 25
CVET formal (a-c) and non-formal (d)
(a) 3rd level qualification (tech college); Specific TVET in (b) vocational school; (c) system with work experience/instruction elsewhere; (d) workplace; (e) Other
CVET duration
Short duration < 2 weeks or 2 to 9 weeks and long duration > 9 weeks
CVET funding source
For individuals whose employers paid for the education or training
Demographics
Gender, Cohabitation (Married/cohabitation/single)
Income
Gross hourly wage based on gross monthly earnings in main/additional jobs obtained in national currency and converted to EUR Purchasing Power Parity (PPP)
Household income ability
Household ability to make ends meet with monthly income using 6 point scale from ‘great difficulty’ to ‘very easily’
IVET system 1 to 5
1: Apprenticeship-based - Denmark, Germany, Luxembourg, Austria 2: Continental school-based - Belgium, France, Netherlands 3: Market-led - Ireland, UK 4: General Education - Greece, Spain, Italy, Portugal 5: Egalitarian school-based - Finland, Sweden
Education, individual and system variables were also used in the study as shown in Table 22, along with their classifications. The highest educational level achieved and episodes of education and training were used to isolate individuals who had taken TVET. To differentiate initial vocational education and training (IVET ) from continuing vocational education and training (CVET ), age and work experience status were reclassified into five life course groups, with young people 25 years or younger used in IVET estimates and all other ages in CVET. Formal/non-formal CVET, duration and funding source were also included in the analysis. Individual variables consisted of gender, cohabitation and measures of income, based on average hourly wage, and a subjective measure of
52
household’s ability to meet needs. Fifteen countries were reclassified into five IVET system groups. Features such as the differences in IVET system regulations and labour market models were taken as criteria for grouping the countries. Method: Estimation strategy To estimate the association between TVET and social outcomes, the study used two models: (1) ‘Model in levels’ to estimate the average differences in levels of social outcomes using linear regression and controlling for year, country effects, initial educational qualifications and gender
Understanding the return on investment from TVET – a practical guide
(2) ‘Model in changes’ to estimate average change n social outcomes over time using fixed effect linear regression and controlling for year effects only
(b) Base model in changes: average differences in changes of social outcomes associated with changes in CVET episode over time over the life course for individuals
The study used individuals aged 25 years or younger to explore:
(a) Base model in levels: average differences in levels of social outcomes for individuals who participated in IVET courses and those who did not (b) Base model in changes: average differences in changes of social outcomes associated with changes in IVET episodes over time and
For (a) and (b) the Base Model plus income, Base Model by systems and Base Model by systems plus income were used
Individuals over 25 and work experience status were used to explore:
For (a) and (b) the Base Model plus income was used. Model of changes was also used to analyse formal/informal CVET, duration and funding source
Social Returns Table 23 shows the results from the regression analysis of the models in levels. In summary, the study found positive social outcomes associated with TVET in terms of civic participation and improved health. Workers in their early career particularly showed strong positive returns from participating in TVET. The social outcomes are influenced by the institutional settings, as the nature of the social benefits will change depending on the type of system and country
(a) Base model in levels: average differences in levels of social outcomes for individuals in CVET courses and those who did not over the life course
TABLE 23 Summary of results for model in levels base and controls, for all ECHP countries by social outcomes MODEL IN LEVELS (Base) (Control)
HEALTH Self-rated health
Lack of chronic health problems
Body Mass Index
CIVIC PARTICIPATION
WELL-BEING
Civic Self-rated Self-rated participation satisfaction satisfaction main activity finances
IVET Systems Europe 1: Apprenticeship-based 2: Continental school-based 3: Market led 4: General education 5: Egalitarian school-based
(+) (+) (+) (+) (+) (+) n.s. (-) (+) (+) (+) (+)
CVET life stages Early career Mid- to end-career Retired Never worked
(+) (+) (+) (+) (+) (+) (+) (+) (+) (+) (+) ( -) (+) (+) (+) (+) n.s. n.s (+) (+) (+) n.s (+) n.s. (+) (+) (+) (+) n.s. n.s (+) (+) (+) (+) n.s. n.s. (+) (+) n.s n.s n.s. n.s (+) (+) (-) (-) n.s. (-)
(+) (+) (+) n.s n.s. n.s. n.s. (-) (+) (+) (+) (+)
(+) (+) (+) (+) (+) n.s n.s. n.s (+) (+) (+) (+)
(+) (+) (+) (+) (+) (+) (+) (+) (+) (+) (+) (+)
(+) (+) (+) (+) (+) (+) n.s. n.s. (+) (+) (+) (+)
n.s. (-) n.s. (-) n.s. n.s. (+) n.s. n.s. n.s. n.s. n.s.
Source: Cedefop (2011) Note: (+) or (-) association between TVET and improvement/decrease in the social outcome, n.s. not statistically significant.
Understanding the return on investment from TVET – a practical guide
53
CASE STUDY 4: COST-BENEFIT RATIO FOR CONSTRUCTION CRAFT TRAINING Wang (2008) developed a cost-benefit ratio for the construction industry. The analysis was based on a craft training survey of ninety-three North American companies and existing industry and project data. The study: (a) estimated baseline costs to build a CII model plant in terms of labour, turnover, absenteeism, injury and rework (b) implemented a hypothetical craft training programme in the CII model plant to measure performance (c) calculated the cost savings from the craft training to derive a return on investment Baseline costs The study estimated the costs of building a CII model plant under the no craft training condition to establish labour, turnover, absenteeism, injury and rework figures. The CII model plant project, a hypothetical petro-chemical processing facility,
provided a standardized physical facility for productivity measurement. Construction duration was 78 weeks (390 days) with 9 construction areas. Table 24 shows the baseline and unit cost rates. Baseline labour costs were derived from the average time estimated by experienced contractors/owners for each major construction activity. The hours were reassigned into trades and the average hourly cost of each trade was identified. Labour costs include hourly wages, fringe benefits and workers’ compensation insurance. Baseline turnover and absenteeism rates were derived from the craft training survey. The minimum average monthly turnover rate of 15.41% and average absenteeism rate of 7.10% were calculated for workers without training, with a turnover cost of $2,000 per occurrence and absenteeism of $110 per absence. Injury rate was based on the industry average of 6.4 per 100 full-time workers per year (US Bureau of Labor Statistics). Rework rate was calculated at 4.4% of labour costs.
TABLE 24 CII model plant baseline cost rates, North America INDICATORS
BASELINE RATES
UNIT COST RATE
Labour
527,457 hours
Varied by trade and included fringes and workers compensation insurance
Turnover
15.41% of total project work force Unit cost rate: $2,000 per hire
Number of workers x turnover rate x unit turnover cost per hire
$233,261
Absenteeism
7.10% of daily work force Unit cost rate $110 per absence
Average workers on site x absenteeism rate x unit cost of absenteeism
$522,182
Injury
6.4 per 100 full-time workers per year
Varied by types of incidents Total number of workers hired x duration
$262,100
Rework
4.4% of labour cost
Estimated construction labour costs
$807,030
COST $US
$22,020,934
Source: Wang (2008)
54
Understanding the return on investment from TVET – a practical guide
TABLE 25 Average minimum and maximum expected cost savings, benefits and returns from craft training, North America EXPECTED BENEFIT Average benefit
Lower bound
Upper bound
COST SAVINGS US$ Average Cost savings Cost savings cost savings Lower bound Upper bound
Productivity improvement Turnover decrease Absenteeism decrease Injury decrease Rework decrease
10.6% 13.9% 14.5% 25.5% 23.2%
Total benefits Training cost (1% of labour cost) Benefit-cost ratio
$521,594 $349,252 $693,937 $230,296 $230,296 $230,296 2.26 1.52 3.01
6.8% 10.3% 10.0% 18.1% 17.2%
14.4% $322,257 $207,564 $436,951 17.5% $32,149 $23,790 $40,509 19.1% $74,871 $51,592 $98,150 33.0% $66,940 $47,452 $86,428 29.1% $25,377 $18,854 $31,899
Source: Wang (2008)
TABLE 26 Benefit-cost ratios using consolidated company data for the CII model plant estimate, North America PERCENT CERTIFIED
100% Certified
80% Certified
50% Certified
30% Certified
Productivity improvements Turnover reduction Absenteeism reduction
$664,364 $5,749 $25,185
$531,491 $4,600 $20,148
$332,182 $2,875 $12,592
$211,061 $1,725 $7,555
Total benefits ($) – cost savings
$695,299
$556,239
$347,649
$220,341
Total workers hired Total workers trained to be certified Total Hours Basic training costs (10 cents per hour) Advanced Cost 1 (15 cents per worker during contract) Advanced Cost 2 ($75 per worker for 10 week session)
751 751 527,457 $52,746 $81,108 $101,385
751 601 527,457 $52,746 $81,108 $81,108
751 376 $527,457 $52,746 $81,108 $50,692
751 225 $527,457 $52,746 $81,108 $30,415
Total training cost ($)
$235,239
$214,962
$184,546
$164,269
2.96
2.59
1.88
1.34
Benefit-cost ratio
Source: Wang (2008)
Understanding the return on investment from TVET – a practical guide
55
Cost savings from craft training Estimates of the average expected percentage of increase or decrease of productivity improvement, turnover, absenteeism, injury and rework of crafts undergoing or completed training was based on the craft training survey. These figures were monetized to derive the cost savings from craft training as indicated in Table 25, with an average of $521,594. Training costs included 1% of labour cost at $230,296. Return on investment As indicated in Table 25, the benefit-cost ratio indicates a positive benefit. The ROI from craft training based on CII model plant ranges from 1.5 to 3.01. Benefits will increase with craft workers’ duration in the training programme. Benefits do not occur at once even though some are immediate, for example employee safety, increased productivity and reduced rework. Construction company return on investment To validate the expected benefits of craft training, a benefit-cost ratio for two North American industrial construction companies was calculated under different percentages of craft workers completing training to certified plus levels. Actual company data was averaged over four projects to derive turnover and absenteeism rates. The results are presented in Table 26. Company costs and cost savings The costs for 751 workers over 527,457 hours included basic on-site training costs at $0.10 per worker hour, as well as advanced off-site certified training covering employer costs of $0.15 per hour per worker (Cost 1) and $75 per quarter per 10 weeks (Cost 2) for the CII Model Plan. These cost figures are presented in Table 26 by percentages of certified workers and show a total cost of $235,239 for 100% certification. The benefits are calculated as cost savings. Productivity improvements total $664,364 for 100% certification. A turnover rate of 6.5% without training indicates 49 of 751 workers are lost, with a cost of $2,000 per incident totalling $97,780.
56
A reduction of 5.88% in turnover costs after 100% of workers are trained calculates a cost saving of $5,749. The 7.3% absenteeism rate without training constitutes a cost of $110 per occurrence of 169 average workers on site (based on hours, project duration/hours per week) totalling $527,984. A reduction of 4.77% in absenteeism costs after 100% workers are certified indicates savings of $25,185. The benefits differ by the percentage of certified plus workers. Return on investment The benefit-cost ratio indicated positive benefits. For 100% certified workers, cost savings totalled $695,299 and training costs constituted $235,239, with a return of 2.96. Returns ranged from 1.34 for 30% of workers certified to 2.59 for 80% certified plus workers.
CASE STUDY 5: ECONOMY WIDE COSTS AND BENEFITS Independent Economics (2013) aimed to estimate the economy wide costs and benefits of increased TVET funding in Australia. This forecasting model compared net benefits of higher TVET investment policy scenarios against a baseline over a five-year period. The analysis included data from the National Centre for Vocational Education Research (NCVER) financial and students and courses datasets, the Australian Bureau of Statistics (ABS) Education and Training Experience survey, and the Bureau’s 2011 National Census. The study included graduates, up-skillers, module completers and re-skillers in the TVET system. Costs The study examined three policy scenarios over five years between 2013 and 2017. The baseline scenario is the estimated level of spending where TVET funding remains at 2011 policy settings. The committed funding scenario represents the impact of higher TVET expenditure as committed in policy announcements since 2011. The further expansion scenario includes higher TVET funding with an investment larger than the current policy commitment.
Understanding the return on investment from TVET – a practical guide
TABLE 27 Total funding of baseline levels and net increase for policy scenarios, 2013 to 2017 (AUD$ billion), Australia FUNDING SOURCE
2011
BASELINE
Actual
Total
COMMITTED FUNDING Total
Change
FURTHER EXPANSION Total
Change
Student fees and charges
1.4 8.6 9.0 0.5 9.7 1.2
State expenditure excluding Victoria
2.8 17.4 19.6 2.2 21.9 4.5
Victoria State expenditure
1.2 7.6 6.6 -1.0 6.6 -1.0
Commonwealth funding national agreements
1.3 7.8 9.6 1.75 11.3 3.5
Commonwealth excluding national agreements
1.2
Total
8.0
7.4
48.8
6.7 -0.75 5.9 -1.5
51.6
2.7
55.5
6.7
Source: Independent Economics (2013)
The direct tuition costs include government contributions, student fees and charges as shown in Table 27. The baseline scenario represents the Australian economy where there is no change in TVET policy in place in 2011. It assumes that all TVET funding sources grow at the same rate as nominal GDP, projected to be 5% in 2012 and 5.25% per year over the remaining years. The table also shows baseline funding estimations over five years totalling $49 billion with spending following its 2011 pattern. The total expenditure for the committed funding scenario is $51.6 billion over five years, an increase of $2.7 billion or 5.6%. Further expansion in total TVET funding of $55.5 billion includes $6.7 billion or a 13.7% increase compared to the baseline. A cost of foregone earnings, as shown in Table 28, is based on the time spent studying and the wage that the students would have earned had they not been studying. The proportion of study time is derived from the contact hours for each level, plus an allowance for study outside of class hours. During a normal working week, an average student is estimated to spend 37% of time studying a Diploma or Advanced Diploma, 30% of time studying a Certificate III/IV and 25% of time studying Certificate I/II. The loss from the reduction in time that the student has available to work is calculated. For individuals who are likely to
have been employed if they had not entered TVET, foregone earnings are estimated using the wage and occupation that they would have had. There is no foregone earnings for those not employed before entering TVET. Benefits The study estimated economic benefits of employability and productivity. Employability is defined as the probability that an individual participates in the labour force, and decreases the probability that they are unemployed. The modelling uses the full time equivalent employment rates for people aged 15 to 64 by gender and level of education to estimate the employability from TVET education. The study used wages as an indicator for productivity. The employability and productivity benefits from additional investment is introduced to the economy wide model as changes to the labour market supply of different occupations.
Understanding the return on investment from TVET – a practical guide
57
Method: Economic model The Independent Education Model is used to estimate the number of TVET students that can be funded from additional spending and the flow on costs and benefits. It consists of three interrelated models: 1. The TVET Expenditure Model (costs) estimates the number of students that can be supported by higher TVET funding and the timing for graduation, module completion or drop-outs. It takes into account the direct costs of each student, level of study and full-time or parttime status. 2. The Education and Labour Market Model (benefits) consists of two parts. Given the number of students, the model estimates the proportion of TVET level qualifications dependant on age, gender and previous qualification. It identifies up-skillers (study at a higher level) and re-skillers (study at the same or less than previous level). These educational outcomes are used in the Labour Market Model to estimate employment by forty-three different occupations for labour supply.
3. The Independent Computable General Equilibrium Model uses the inputs from the Expenditure and Education and Labour Market Models to estimate economy-wide costs and benefits of higher TVET funding of additional students. Given the increased spending from the TVET Expenditure Model, the effect on tax rates and the economy is modelled. Given the potential labour supply by forty-three occupations from the Education and Labour Market Model, the effect on the labour market and economy is modelled across 120 industries. The costs are incurred upfront during five years of the policy, whilst the benefits continue to accrue each year over the working life of the additional students. Costs and benefits are converted into the annual amount received indefinitely i.e. “perpetuity equivalent”. Return on investment The committed funding total costs are $7.0 billion, as shown in Table 28. This consists of additional expenditure of $2.7 billion between 2013 and 2017 that is converted to Net Present Value (NPV) of tuition fees of $2.3 billion in 2013 terms, and
TABLE 28 Costs and benefits of additional TVET funding (AUD$ billion), Australia Costs and benefits
Category
Additional funds
Additional funds
Number of students
Total additional student numbers Includes Certificate III/graduates
Costs
Benefits
Net benefits
Committed funding scenario
Expanded funding scenario
$2.7
$6.7
218,000 72,000
528,000 174,000
Tuition fees (2013 present value) Foregone earning opportunities
$2.3 $4.7
$5.7 $11.3
Total Costs
$7.0
$17.0
Employability Productivity- wages
$18.4 $2.0
$44.7 $4.8
Total benefits
$20.4
$49.5
Benefits –costs
$13.4
$32.5
Source: Independent Economics (2013)
58
Understanding the return on investment from TVET – a practical guide
foregone earnings of $4.7 billion. Total benefits are $20.4 billion, mainly from employment benefits of $18.4 billion and $2.0 billion of productivity wages, yielding a net benefit of $13.4 billion based on 218,000 additional students. The further expansion scenario with a larger five year TVET boost of 13.7% and initial $6.7 billion in additional funds converted to NPV of $5.7 billion indicates a higher net benefit boost of $32.5 billion with 528,000 additional students.
The study also estimated the Internal Rate of Return for additional investment in TVET Certificate III students2 and above at 18% and includes up-skillers, graduates, module completers and re-skillers.
2 Research indicates this qualification level and above exhibit observable economic and labour market benefits.
Understanding the return on investment from TVET – a practical guide
59
Conclusions
Information on the return on investment in TVET is valuable as it helps to argue the case for the funding (or additional funding) of training systems and programmes. Examining international research provides some evidence of the positive outcomes from investing in TVET, which ranges from labour market or employability benefits, to social and environmental sustainability perspectives. This Guide shows the importance of ensuring that any measurement of the ROI should be closely aligned to the objectives of the country’s TVET system and stakeholders. The ROI measures related to the economic outcomes of TVET are critically important for all countries and most stakeholders, while social aspects are also important.
60
However, measuring the ROI from TVET in a given country has its challenges. The diversity of TVET systems and the differing country contexts pose considerable difficulties for cross-country comparisons of the ROI from TVET. Indeed, this Guide has highlighted that one of the major challenges relates to developing ROI measures that can be broadly compared across countries, while another relates to having appropriate data to enable the measurement of the ROI. This Guide bridges this gap and paves the way to providing a framework to achieve this outcome.
Understanding the return on investment from TVET – a practical guide
Pro forma resources and attachments Summary of the return on investment indicators by stakeholder group
INDIVIDUAL Job-related indicators Employability skills Employment Improved employment status Wages/earnings Entrepreneurship Non-job-related indicators Well-being Foundation skill gains Socio-economic status Social inclusion
Attainment of employability skills for those who have completed training Employment rate of those not employed before training Improved employment status of those employed before training who have completed training Income of full-time workers after training Attainment of entrepreneurial skills and knowledge Students have an improved sense of well-being after training TVET graduates/completers have improved foundation skills after completing training Improved socio-economic status of those completing TVET programmes Participation in social groups or communities
BUSSINES Market indicators Increased productivity Profitability Quality product/service Business innovation
Increase in the productivity of the organization Increase in the profitability of the organization Improve the quality of products or services Contribute to organizational innovation and business practices
Non-market indicators Employee well-being Employee engagement Workplace safety Environmental sustainability
Employees have an improved sense of well-being Employees are more engaged in the workplace Employees experience a safer workplace Workplace practices contribute to environmental sustainability
ECONOMY/SOCIETY Economic indicators Economic growth Labour market participation Unemployment rate Skilled workforce Entrepreneurial activity Social indicators Health Social cohesion Social equity Social capital
Increase the capacity of the economy to produce goods and services Increase the labour market participation of TVET graduates/completers Decrease the rate of unemployment for TVET graduates/completers Level of educational attainment of TVET graduates/completers Level of entrepreneurial activity of TVET graduates/completers Improve community health and foster a longer and better life Improve the well-being, social inclusion and values that support cooperation within or among groups Increase access and participation of disadvantaged groups in TVET Participation in networks that strengthen social capital
Understanding the return on investment from TVET – a practical guide
61
Return on Investment Data Collection Pro forma Guide
Data collection elements
Guiding Questions
Instructions
ROI measures
Is there existing data that measures the ROI indicators?
YES/NO If YES, indicate the ROI measure used for each indicator in the pro forma sheet and how the measure is defined (calculated or quantified) If NO, indicate “No data”
Data source and description
What is the source of the data?
Indicate the organization, reference year and name of the dataset
What is the scope of the data?
What is the sample/population size? What is the country of origin? What elements are excluded from the data?
What methods have been used to collect the data?
Is the data quantitative? If YES, define the dataset type. For example questionnaires, surveys or existing dataset Is the data qualitative? If YES, define the information source (e.g. in-depth interviews, case studies)
Data Quality and completeness
Data availability and data gaps
62
Is the dataset well defined?
YES/NO If YES, indicate (or attach separately) the key data elements and data definitions
Are there factors that may affect the ROI?
YES/NO If YES, indicate the data element and reason
Is the data complete?
YES/NO If NO, what key data records or data elements are missing?
Is the data accurate?
YES/NO If NO, indicate the data discrepancies
Is the data available?
YES/NO If NO, indicate the non-availability reasons
Is the data accessible?
YES/NO If NO, indicate the access issues
Are there any data limitations?
YES/NO If YES, describe the limitations of the data
Are there data gaps?
YES/NO If YES, indicate the information gaps
Understanding the return on investment from TVET – a practical guide
Return on Investment Data Collection Summary List
Stakeholder
Individuals
Category Indicators
Job-related
ROI measures (Yes/No)
Data Data Data Data Data Data Data source description complete available accessible limitations gaps (Yes/No) (Yes/No) (Yes/No) (Yes/No) (Yes/No) (Yes/No) (Yes/No)
Employability skills Employment Improved employment status Wages/earnings Entrepreneurship
Non-job-
Well-being
related
Foundation skill gains Socio-economic status Social inclusion
Business
Market
Increased productivity Profitability Quality product/services Business innovation
Non-market Employee well-being Employee engagement Workplace safety Environmental sustainability Economy/
Economic
Society
Economic growth Labour market participation Unemployment rate Skilled workforce Entrepreneurial activity
Social
Health Social cohesion Social equity Social capital
Individuals
Finance
Direct costs Indirect costs
Business
Finance
Direct costs Indirect costs
Economy
Finance
Direct costs Indirect costs
Understanding the return on investment from TVET – a practical guide
63
PRO FORMA RESOURCE 1: Return on investment job-related indicators for individuals
Indicators
ROI measure
Data source Data quality/ and description completeness
1. Employability skills Attainment of employability skills for those who have completed training 2. Employment Employment rate of those not employed before training 3. Improved employment status Improved employment status of those employed before training who have completed training 4. Wages/earnings Income of full-time workers after training 5. Entrepreneurship Attainment of entrepreneurial skills and knowledge
64
Understanding the return on investment from TVET – a practical guide
Data availability/ data gaps
PRO FORMA RESOURCE 2: Return on investment non-job-related indicators for individuals
Indicators
ROI measure
Data source Data quality/ and description completeness
Data availability/ data gaps
1. Well-being Students have an improved sense of well-being after training 2. Foundation skill gains TVET graduates/ completers have improved foundation skills after completing training 3. Socio-economic status Improved socioeconomic status of those completing TVET programmes 4. Social inclusion Participation in social groups or communities
Understanding the return on investment from TVET – a practical guide
65
PRO FORMA RESOURCE 3: Return on investment market indicators for businesses
Indicators
ROI measure
Data source Data quality/ and description completeness
1. Increased productivity Increase in the productivity of the organization 2. Profitability Increase in the productivity of the organization 3. Quality product/ service Increase the profitability of the organization 4. Business innovation Contribute to organizational innovation and business practices
66
Understanding the return on investment from TVET – a practical guide
Data availability/ data gaps
PRO FORMA RESOURCE 4: Return on investment non-market indicators for businesses
Indicators
ROI measure
Data source Data quality/ and description completeness
Data availability/ data gaps
1. Employee well-being Employees have an improved sense of well-being 2. Employee engagement Employees are more engaged in the workplace 3. Workplace safety Employees experience a safer workplace 4. Environmental sustainability Workplace practices contribute to environmental sustainability
Understanding the return on investment from TVET – a practical guide
67
PRO FORMA RESOURCE 5: Return on investment economic indicators for the economy
Indicators
ROI measure
Data source Data quality/ and description completeness
1. Economic growth Increase the capacity of the economy to produce goods and services 2. Labour market participation Increase the labour market participation of TVET graduates/ completers 3. Unemployment rates Decrease the rate of unemployment for TVET graduates/ completers 4. Skilled workforce Level of educational attainment of TVET graduates/completers 5. Entrepreneurial activity Level of entrepreneurial activity of TVET graduates/completers
68
Understanding the return on investment from TVET – a practical guide
Data availability/ data gaps
PRO FORMA RESOURCE 6: Return on investment social indicators for society
Indicators
ROI measure
Data source Data quality/ and description completeness
Data availability/ data gaps
1. Health Improve community health and foster a longer and better life 2. Social cohesion Improve the well-being, social inclusion and values that support cooperation within or among groups 3. Social equity Increase access and participation of disadvantaged groups in TVET 4. Social capital Participation in networks that strengthen social capital
Understanding the return on investment from TVET – a practical guide
69
PRO FORMA RESOURCE 7: Return on investment financial data and costs to stakeholder groups
Financial data
ROI measure
Data source Data quality/ and description completeness
Costs to individuals
Costs to business, employers, industry
Costs to the economy or society
70
Understanding the return on investment from TVET – a practical guide
Data availability/ data gaps
References
Bernier, A. and Cousnieau, J.M. 2010. The impact of training on productivity in Canadian firms: estimating distributed lags from the WES 19952005. International Journal of Interdisciplinary Social Sciences, 5(7). Brown, J., Taylor, M., McKenzie, P. and Perkins, K. 2015. Investing in Workforce Literacy Pays: Building. Employer Commitment to Workplace Language, Literacy and Numeracy Programs. Melbourne, ACER. Cedefop [European Centre for the Development of Vocational Training]. 2011. The economic benefits of VET for individuals. Cedefop research paper no. 11. Luxembourg, Publications Office of the European Union. https://www.cedefop.europa. eu/files/5511_en.pdf. Cedefop. 2011. Vocational education and training is good for you: The social benefits of VET for individuals. Cedefop research paper no. 17. Luxembourg, Publications Office of the European Union. https://www.cedefop.europa. eu/files/5517_en.pdf Hoeckel, K. 2008. Costs and benefits in vocational education and training. Paris, OECD. http://www. oecd.org/education/country-studies/41538706. pdf Independent Economics. 2013. Cost-benefit analysis and returns from additional investment in vocational education and training. Kingston, Independent Economics. http://www.tda.edu. au/cb_pages/files/Econometric%20Final%20 Report.pdf Investing in Partnerships. 2012. Social Return on Investment: Lesson Learned in Australia. Prepared for: Investing in Impact Partnership. https:// socialventures.com.au/assets/SROI-Lessonslearned-in-Australia.pdf Jansen, A., Pfeifer, H., Schönfeld, G. and Wenzelmann, F. 2015. Apprenticeship training in Germany remains investment-focused - results of BIBB CostBenefit-Survey 2012/13. Bonn, Federal Institute for Vocational Education and Training. https://www.bibb.de/en/25852.php
Long, M. and Shah, S. 2008. Private returns to vocational education and training qualifications. Adelaide, National Centre for Vocational Education Research.. Phillips, J. and Phillips, P. 2015. Handbook of Training Evaluation and Measurement Methods (Fourth Edition). Abingdon, Routledge. Rechmann, P. 2017. Webinar: What is the Business Case of Investing in Vocational Education Training. https://www.dcdualvet.org/wpcontent/uploads/2017_GOVET-WebinarBusinessCase-dualVET.pdf (Viewed 1 December 2017) Schueler, J. 2016. Evaluation framework measuring Return on Investment (ROI) in TVET. Adelaide, NCVER. Schueler, J. Stanwick, J. and Loveder, P. 2017. A framework to better measure the return on investment from TVET, A report to the UNESCOUNEVOC International Centre. Adelaide, NCVER. TESDA, 2016. A cost-benefit study on dual training system Philippines. Taguig City, TESDA. https:// www.bibb.de/dokumente/pdf/TESDA_ costbenefit-tesda_oct_7_2016_interactive.pdf United Nations. 2015. Transforming our World: the 2030 Agenda for Sustainable Development. New York, United Nations. UNESCO. 2017. Global Education Monitoring Report 2017. Paris, UNESCO Publishing. https://unesdoc. unesco.org/ark:/48223/pf0000259338 VET Glossary, 2017, accessed on 8 December 2017 at http://www.voced.edu.au/content/glossaryterm-tvet Wang, Y. 2008. A Quantitative Analysis of Training Outcomes and Strategies in the Construction Industry. University of Kentucky Doctoral Dissertation. https://uknowledge.uky.edu/ gradschool_diss/600
Understanding the return on investment from TVET – a practical guide
71
Strengthening technical and vocational education and training (TVET ) is an important strategy to contributing to equitable, inclusive and sustainable economies and societies. Indeed, one of the Sustainable Development Goals focuses on ensuring ‘inclusive and equitable quality education and promote lifelong learning opportunities for all’ (SDG 4). However, in a world of competing demands, realizing this goal puts pressure on funding and financing TVET systems. More than ever before, there are calls for providing more evidence of the return on investment (ROI) from TVET. Measuring the ROI in a given country comes with its own challenges, and the diversity of TVET systems and the differing contexts makes it difficult to generate comparable data. There are considerable variations between countries in terms of priorities regarding the objectives, costs and benefits of TVET. Moreover, there are differences in terms of the data that is available to measure the ROI from TVET. The quality of data in terms of validity and reliability will vary significantly, and the data completeness and accessibility further reveals disparities between stakeholders.
There is therefore a need to develop ROI measures that can be utilized broadly across countries. Selecting a ROI model that is fit-for-purpose and one that suits the type of data available is imperative in order to produce useful and practical information. The model chosen will be influenced by whether or not to include social or economic returns, as well as whether to adopt an evaluative (actual return) or forecasting (potential return) perspective. This Guide presents an analytic framework that summarizes some of the main elements and issues that need to be considered in measuring ROI. This includes establishing the scope, context and purpose for measuring ROI, adopting guiding principles, identifying costs, benefits and factors that impact on ROI, and approaches to data collection and analysis. The Guide introduces a framework that looks at the ROI equation from a range of perspectives – including economic and social dimensions – and for different stakeholders, including individuals, businesses and societies. The Guide presents relevant ROI indicators and measures drawn from the existing international research. Finally, it provides guidelines to planning and collecting ROI data, along with a set of practical pro forma resources and case studies to assist the reader.
9 789231 003752
Sustainable Development Goals