The Event ROI Methodology

Proving the value of meetings and events

Most events could probably deliver twice the value on the same budget. Event planners typically know their planning and execution processes of event logistics inside out but it still challenges them to start at the beginning, and figure out the role meetings and events play in their company’s strategy.

Event professionals today should establish methods to make their event contributions much more strategic. Events are most frequently viewed as an expense rather than an investment to achieve a unified business goal. Event planners may change this perception by providing measurement of value.

How do you know if your event was a success? Because registrations were up compared to last year, the number of leads increased by 15% or the participants were very satisfied? Are these measures of value which made the investment worthwhile?

The ultimate purpose of any business event is to create a profit, or more precisely, to add value to shareholders. Unless the shareholders have given very precise instructions to the contrary, all corporate managers have this singular ultimate goal, and everything the organisation does, including its events, must contribute to it.

Outside the corporate world, organisations have a variety of missions, and typically not including the creation of monetary wealth. The mission of the cancer society is to reduce suffering from cancer, a trade union exists to improve the working conditions of its members, a political party exists to influence the ways in which we run our society, and so on. Setting relevant and detailed (monetary and non-monetary) objec-tives and measuring results against those same objectives will lead to higher event effectiveness and success.

Studies indicate that more than $500 billion are spent on events worldwide every year. Hosting, attending and exhibiting at events makes up the largest proportion of corporate marketing budgets at 21%. With companies spending tens or even hundreds of thousands of dollars on events, many organizations look at their business value . They measure the success of meetings and events but most of them only use some basic metrics such as:

➔ Direct Marketing and Social Media Results
➔ Revenue from registrations and/or sponsorship
➔ Participant numbers and satisfaction
➔ Number of new leads

Most event planners have a reasonable sense of how their events were received and a few metrics to measure success. But very few results provide data relevant to business impact and ROI or isolate the impact of the event from other activities.

This whitepaper explains a scalable and systematic approach to proving the monetary and non-monetary value of events – the ROI Methodology – a practical solution:

➔ Aligning events with business objectives and marketing strategies
➔ Reducing costs and increasing the value created by the event
➔ Demonstrating value, including ROI, to all stakeholders
➔ Measuring ROI
➔ Winning business or secure funding

The ROI Methodology is an industry standard for measuring the effectiveness of meetings and events.

The fundamental principle of value creation

Meetings and events create value to stakeholders by influencing the behaviour of the participants. This is the only way. If the event does not make participants do something they would otherwise not have done, there is no value. Thinking and feeling does not count, only what their feelings and thoughts make them do. This simple and universal fact is a fundamental principle of the ROI Methodology.

ROI Methodology Background

The ROI Methodology has been developed by Dr. Jack Phillips at the ROI institute Inc. www.roiinstitute.net since the late 1970‘s. His initial work was based on the Kirkpatrick Model, developed by Donald Kirkpatrick already in the late 1950’s. Kirkpatricks model had four levels; Satisfaction, Learning, Behaviour and Results. Phillips added ROI as the fifth level and later the target audience (or more generally ‚input variables‘) as the first level. Phillips also developed a number of guidelines and generally made the model more practical and consistent in its application.

Both the Kirkpatrick and Phillips models were initially aimed at measuring the results of training. The Phillips model in particular has since migrated to a number of other disciplines. More than 35.000 practitioners have been trained by Jack Phillips and the ROI Institute and their partners in more than 50 countries in the application of the methodology and more than 20 disciplines, including meetings and events.

The Methodology is explained and applied to different industries and professions in more than 60 books, most of them written by Jack and Patti Phillips and various co-authors.

Planning and Measurement

The most important application of the ROI Methodology is for planning meetings and events to deliver their best possible outcome. Failing to do so means that the event does not deliver its potential value.

The ROI Methodology was first thought of as an evaluation tool only, but it is easy to understand how it became a planning model at the same time. In order to measure results, there must be clear and measurable objectives, otherwise measurement is meaningless. How do you know if results are good or bad if you don’t have objectives?

The rest is just human nature; when we have clear and measurable objectives, we will always do our best to try to achieve them. Most events achieve below their potential because it was not clear what they were trying to achieve.

The context sets the scene

Events don’t create value on their own. How is the event supported by other strategies and programmes, and how do other initiatives support the event? The strategic context of the event aligns its outcome with other activities towards a common goal. The strategic context analysis also determines why we need the event at all, what role it should play to support marketing, sales, HR or other areas of activity.

A teambuilding event, for example, needs to be part of an organisational development programme. The event itself will not compensate for bad management, a faulty reward system, or anything else which could keep the organisation from performing well. A sales incentive event needs to be part of a sales training programme, a custo-mer hospitality event needs to be aligned with a customer care programme, a product launch could be the core of a launch programme, but without the support of other channels, it will not achieve its potential contribution to sales. Only when the context is clear, can we start setting objectives for the event.

Stakeholder value

The essential objective is to provide value, bottom line profit and a contribution to shareholder value for corporate events, achieving the mission of a not-for-profit organisation. There is usually a main stakeholder or meeting owner, but this is not the only one. Sponsors and exhibitors are typical stakeholders, as are participants. venues, speakers, and others. In fact, anyone who would say: “It is important to me that this event is successful and meets my objectives” is a stakeholder. The main stakeholder or meeting owner, the budget holder, will be responsible for reconciling the sometimes conflicting objectives of different stakeholders.

Six levels of objectives and evaluation

The ROI Methodology sets objectives and measures results at six levels, from Level 0 to Level 5. Objectives are first set for Level 5, the desired ROI or profit from the event, its contribution to shareholder value in the business world, or its contribution to the mission of a non-profit organisation.

The evaluation sequence works in the reverse order, starting at Level 0 and working step by step towards the top. There is a close interconnection between all levels with a knock-on effect if one is not carefully designed to serve the next level objective.

Step 1 Setting Objectives

Objective setting starts with determining the context of the event, what role should it play in a higher level strategy or campaign. This exercise often determines the target audience (level 0) and the kind of end result impact (level 4) of the event.

The essential question at this stage is; how may the behaviour of the target audience change in order to deliver the desired impact as a result. This determines the behaviour objectives (level 3). When behaviour objectives are known, the question becomes; what cognitive change (i.e. learning experience) will lead to the desired change in behaviour? This leads to the learning objectives (level 2). Finally, different learning objectives will influence the choise of learning environment, the format and content of the meeting (level 1).

Objectives on each of the 6 levels become measurable by setting clear success criteria = KPIs (key performance indicators) at the initial stages of planning. The KPI’s determine which data needs to be collected in order to measure the achievement of objectives at each level.

Level 5 Return on investment

ROI is another way of expressing the contribution to profit made by an event in the corporate world. The profit is the net value created by the event minus the event costs. ROI is the profit expressed as a percentage of the cost of the event.

ROI, or profit, is the first planning objective, how much do we expect the event to contribute to the bottom line or the mission of our organisation. For association and government events, profit is usually not an objective, the ultimate value is the mission or political purpose for which the organisation exists, this is the mission that replaces ROI.
➔ A medical congress may exist for the purpose of reducing a type of disease,
➔ The mission of a professional association may be to develop the skills of its members.

Association meetings and also corporate events often have sponsors and exhibitors and for these as stakeholders, financial ROI is usually a relevant measure.

Level 4 Impact Objectives

The Impact, or Business Impact, is the ultimate value contribution of the event to its stakeholders. The impact is used in the profit and ROI calculations. For a customer event, the Impact is usually the increase of sales. For an internal event it is typically organisational effectiveness

Example B2B Exhibition

Level 3 Behaviour Objectives

What do the guests or participants at meetings and events need to do differently, during and after the event, in order to create value for the stakeholders? The answers may well be different for different categories of participants. Some actions may be significant (e.g. buy the product) whereas others only make a small contribution to value, maybe increasing the probability of a purchase (e.g. ask for more information, share knowledge with colleagues, investigate alternative solutions, etc.).

Example Workshop non-profit

Level 2 Learning Objectives

What cognitive change (i.e. learning) is required for the participants to change their behaviour. All behaviour change is pre-empted by cognitive change. The cognitive change might be subconscious, but something always has to change in the mind before behaviour changes.

At an exhibition for example you want to increase the requests for appointments with sales post-event to drive sales for a specific product. So what should participants learn to request more appointments? Where ist the gap? Here event planners can set benchmarks as objectives by doing a pre-event survey with a broad sample of attendees to identify needs and concerns on multiple levels. Content can afterwards be focused on those with the greatest learning gap. A CRM system might also be a good source of information.

Example Sales Training

Level 1 Learning Environment Objectives

How can we design a learning environment which will make cognitive change most effective? Learning is influenced by the state of mind of the learner as well as ambient factors (e.g. room temperature and air quality), instructional design, speaker quality, presentation tools etc.

Example Conference

Level 0 Target Audience Objectives

Finally, how can we ensure that the right people are attending. Do they have oppor-tunities to apply what they learn to the benefit of the stakeholders? Are they learning something new, which will change their behaviour? The target delegates are those with the greatest learning and behavioural gap in the population of potential partici-pants. There is no need to ‘speak to the converted’, similarly, there is no need for par-ticipants to learn something for which they have no application.

This focus avoids defining a target audience too broad and as a result, limiting your ability to generate more quality. Again a pre-event survey may help to focus efforts on those with the greatest behavioral and learning gap.

Example Seminar

This sequence of questions is our model for planning any kind of meeting or event. By setting clear objectives for each step and planning for their achievement, we should get the greatest possible value in return for the investment.

Step 2 – Measuring Results

Results are measured in a strict sequence from Level 0 to Level 5 by using different data collection methods. Practicality and available measurement tools influence the appropriate method at each level.

Measuring Level 0 – Target Audience

The target audience is therefore defined by a method of deduction from desired behaviour (Level3) and required learning (Level 2). The results may be collected from the delegates themselves, for example by questions such as “To what extent is the topic of this session relevant to your job?”, “How much of what was covered in this session did you know already?”

On top of this, characteristics of the target audience may be translated into demographic data. For example, if previous research shows that ‘young physicians in private practice’ lack the knowledge being communicated – it is useful data for selection of Target Audiences at a medical congress, but finally the post event evaluation survey should determine more precisely whether there was a good match between meeting format and content and the needs of the delegates present.

Measuring Level 1 – Delegate Satisfaction and Learning Environment

At level 1 we measure delegate satisfaction (i.e. through online surveys) with hospitality, form and content. Did they learn something new which they can use in their jobs, did they feel welcomed, relaxed and confident, would they recommend the event to colleagues and peers, etc. These are key elements of participant satisfaction.

But participant satisfaction is essentially a proxy variable for the quality of the learning environment, which is what really matters if learning, in one form or other, is what must happen in order to change delegate behaviour and provide value to stakehol-ders. The educational quality of session formats, including the ability of speakers to communicate effectively, should preferably be assessed by adult education experts.

Measuring Level 2 – Learning

Everything that happens during a meeting or event is a series of learning. We categorise learning into Information, Skills, Attitudes and Relationship learning.

Traditional information and skills learning is typically evaluated by means of tests, preferably both before and after the learning activity in order to define the size of the knowledge or skill gap and the extent to which it was filled by the learning exercise.

But tests are rarely practical in the traditional setting of a meeting or event. We use therefore the second best, and very much quicker and easier method of ‘self reporting’. It is for example perfectly reasonable (even though not as accurate as a test) and practical to ask the delegate to indicate his level of knowledge or skill both before and after the session.

Attitude learning, such as changes in brand perception, is traditionally and scientifically measured by self-reporting by the use of so-called Likert scales where the respondent expresses his or her degree of agreement or disagreement with an attitude statement. For example, one congress sponsor may know that their target population will average 3,7 on the 5-point Likert scale in response to the statement; “I think of Company X as the technology leader in my field.” If this average score increases over the meeting, a change in brand perception may well have taken place.

Relationship learning refers to the building of affinity between people, getting to know others, trust and liking. All forms of peer learning benefit from the strength of personal relationships, it is the foundation for subsequent information, skills and atti-tude learning in the relationship. Relationship learning may be measured in much the same manner as other forms of learning. At the most detailed level, individual relati-onships of trust and liking may be scored on a scale from very low to very high, or more general reports of relationship learning may be collected.

The method of self-reporting makes it quick and easy to measure all forms of learning. It may not be as accurate as tests or other methods, but the greater degree of ac-curacy rarely justifies the extra time and complication of measurement.

Measuring Level 3 – Behaviour

Behaviour is the application of learning. Learning without application is without value. You may give the participant a certain product experience, but if he or she has no opportunity to apply the newly acquired knowledge or skill, the learning was wasted.

“What exactly do you know that participants should do differently after the event?”

The behavioural change may involve to stop doing something, doing something diffe-rently, or taking some new actions as a result of attending the event (i.e. ask for more info, buy a new product). Behaviour is often best measured by observation, such as the use of a ‘mystery shopper’. But this can be time consuming and expensive. Again, self-reporting is a quicker and cheaper method, if maybe not as accurate.

If, for example, a delegate at a medical congress learns a new diagnostic procedure, a skill which he claims to understand and remember well enough to put into practice (Learning measured by self reporting), then it should be quite straight forward to ask some time later if he has been using the new procedure in situations where it has been appropriate.

Example press Roundtable

Planned Actions

It is often useful to start by measuring intended behaviour immediately after learning. ‘How do you plan to use what you have just learned?’ By suggesting possible actions and asking delegates to consider whether they are likely to happen or not, we are also communicating ideas for behaviour and to some extent, perhaps, strengthening the commitment to do what we want them to do. We call this the measurement of planned actions. Planned actions are nearly always best case forecasts of the application of learning which will take place.

Measuring Level 4 – Impact

The business impact is the ultimate purpose for which the event was designed, and it is the result of what participants do differently because of the event.

➔ Customer events will typically increase sales as a desired business impact, perhaps selling a wider range of products to existing customers, increasing customer penetration by becoming a dominant or single source supplier, or perhaps the business impact is not to sell more, but to protect existing levels of sales through greater customer loyalty.

➔ Internal events, such as teambuilding, are likely to reduce costs as their business impact. If a group of people work better and more efficiently together as a result of the teambuilding, the same work can be done in less time which is a cost saving for the company.

The business impact data may be obtained from company records, such as sales or cost accounting. More ‘soft’ data may be obtained by asking participants to estimate the business impact of certain changes in behaviour. For example, what could be the impact in terms of employee turnover if managers were trained to look better after the needs of employees. The effect could be captured by employee satisfaction surveys and by analysing the reasons for leaving in exit interviews.

Isolating the effects of the meeting

Whenever we measure business impact, we must always use at least one method to isolate the effect of the meeting. If sales went up after the customer event, how do you know that it was because of the event and not the advertising campaign which was launched at the same time?

There are several methods for isolating the effect of the meeting and in general it can always be done. The best method is to do a control group experiment, comparing the results from one group which attended the meeting with another group that did not. But for this method to be reliable, the control group which did not attend, must be carefully matched with those that did. If the groups are carefully matched, they will respond in the same way to other influences and the difference in results will be due to the event.

There are many examples of control group experiments, but the method can be difficult and costly, in particular the careful selection of the control group. Furthermore, a control group experiment is often inconvenient, you want to invite all your customers to the event and all your employees to the annual sales kickoff or the teambuilding event. For these reasons, the control group experiment is not the most commonly used method of isolation.

An alternative method is to estimate the effect of the event compared to other factors. Such estimations are not as accurate, but they are quick and easy to perform. Already available data collected within an organization can sometimes also be used. CRM systems or other databases may have a systematic collection of information helping to isolate the effect of an event.

Converting Impacts to monetary values

Some business impacts are monetary, such as sales, but others need to be converted into money values for the ROI calculation. A team building or management training seminar, for example, may lead to reduced staff turnover or absenteeism, less time spent on resolving conflicts between departments, etc.

Impacts which lead to time savings can be measured by assigning a standard value to an hour of time, it could be an intra-company billing rate, salary cost or a standard or estimated rate reflecting the full cost or opportunity value of staff time. Many other impact values have standard monetary values, such as the cost of staff recruitment saved by reducing staff turnover.

Intangible values

It may not be practical or economical to attempt to convert all business impacts into monetary values. Those which are not converted are referred to as ‘intangibles’. For example, an exhibition may lead to new customer prospects and sales, but at the same time motivate and improve relations between staff. We may follow up leads and eventually apply a sales value to them as this is the most important impact from the exhibition, but we may leave the motivational effect as an intangible which is not converted into money for the ROI calculation.

Measuring level 5 – Calculating ROI

When impact values are expressed in monetary terms and the total costs of the event are deducted, we have a profit or loss for the event. That profit or loss value as a percentage of the same costs, is the ROI percentage figure. The Return is the sum of Impact values and the Investment is the total cost. Cost of the event also include the opportunity cost of own staff planning and attending the event including the event team or costs of an account manager associated with the time he/she stays away from doing customer visits in his/her territory. In return cost saving of being able to meet with many customers in a short period of time can be offset.

Example Data Collection Methods

The ROI on ROI

Irrespective of how far you go up the levels of the pyramid in terms of measurement, the benefits of applying the ROI Methodology will always outweigh the costs. The methodology forces you to identify the precise event objectives at different levels.

Armed with these objectives you set about planning the event, and because your objectives are detailed and clear and measurable, the event programme will be more focused on achieving them. Because you decide to measure, the event gets better

Conclusion

Meetings and events are one of the last major items of controllable expenditure still avoiding management accountability. Many meeting planners are already finding that procurement and finance take a greater interest in how they spend the money. This is a great opportunity, not a threat, because procurement and finance are familiar with ROI measurements in many other areas of management decision. They will probably be very interested to join your event ROI measurement project.

About Event ROI Institute

The Event ROI Institute was founded in 2005 with the mission to help companies improve the value of meetings and events through the application of the ROI Methodo-logy. More than one thousand professionals have been involved in training courses and seminars, including representatives from Novartis, Nike, SAP, Astra Zeneca, Schlumberger, IKEA, Radisson, MCI, Toshiba, Vok Dams, Lego, Samsung, Deutch Bank, Barclays, and many others. Event ROI Institute is a strategic partner of Jack Phillips and the ROI Institute, Inc. (www.roiinstitute.net) for the meeting and events industry in Europe.

Contact

Dr. Elling Hamso, Managing Partner, Event ROI Institute
Phone +47 90 12 24 18, elling.hamso@eventroi.org

Ilka Dzeik, Senior Partner, Event ROI Institute
Phone +49 175 2939126, ilka.dzeik@eventroi.org

www.eventroi.org
[purchase_link id=”3388″ text=”Download” style=”button” color=”red”]