This post may contain affiliate links. If you click on an affiliate link and make a purchase, I may earn a small commission at no additional cost to you. For more information, see our Privacy Policy.
Evaluation and assessment are both important tools for measuring a learning experience for a student. They both measure the impact of the learning experience on the student as an individual and their work. If employed, evaluation and assessment can also inform trainers and educators on the impact of the given training on a company’s service users.
But what are the differences between assessment and evaluation?
In short, assessment focuses more on the learning, teaching and outcomes of a curriculum.
On the other hand, evaluation is more concerned with grades and what has been learned.
Despite these differences, both assessment and evaluation have the overall purpose of using set criteria, measures and evidence to reach their outcomes.
Evaluation can include assessment, but ultimately, evaluation is a broader process than assessment alone.
Kirkpatrick’s Four-Level Training Evaluation Model
There are a range of methods that can be implemented as training evaluation tools.
Among these methods, that of Kirkpatrick’s Four-Level Training Evaluation Model helps to analyse the effectiveness of learning in the following 4 stages.
Stage 1 – Reaction
In this first stage, as the teacher, you want your students to feel valued. You also want to be able to measure how your students were engaged with a learning activity or course as a whole.
Measuring students’ reactions and experiences is important for you to understand how well a learning activity or course was received and enjoyed by your students. You will also be aiming to measure reactions based on students’ differing learning styles. You will be able to do this by looking at whether or not the practice-theory gap has been bridged and if your students understand how to apply the knowledge that you have taught them in their everyday lives. Your students might not have completely bridged this gap yet, and might still be learning to apply their new knowledge in context.
Stage 2 – Learning
In this second stage, you are measuring what your students have and haven’t learnt. Such activities will usually link to specific learning objectives.
Kirkpatrick advocated the benefits of measuring learning before and after an activity/course.
This enables the student and teacher to see what has been learnt, and how skills, knowledge, attitudes and confidence have increased. Such activity reinforces the accountability of your students to engage with their own training and education.
Stage 3 – Behaviour
This third stage considers how your students apply what they have learnt.
As a teacher, this stage helps you to unpick where extra support may be required.
For some teachers, measuring behaviour may be difficult, depending on their place of work. However, this stage is concerned with how students put their learning to use. It’s therefore quite important for you as a teacher to know that what you have taught has been understood and taken on board by your students.
Depending on your teaching role, you may be limited in how you gain this information. This is particularly true of an online course where you might never find out how your students use what they learned in your course after its completion.
Therefore, if you are running an online course, it is important to gain feedback through evaluation forms from your students. You could even follow up with an automated email a month or so after they’ve finished your course to see how useful it has been to them. This way, you will still be able to gain that all important information that your students have been able to bridge their practice-theory gap and apply their knowledge in real life.
If you can follow up with your students in person, you may be able to gain this type of information more holistically through observation and interviews.
If you can’t see your students face-to-face after their course has finished, consider how you could check if students could hypothetically use what they have learned in practice. Hypothetical questions that you may be asked include in an evaluation form may include: ‘If this situation arose in practice, what would you do, keeping in mind what you have just learnt?’
Stage 4 – Results
This fourth stage of Kirkpatrick’s Four-Level Training Evaluation Model is concerned with analysing the results of your training.
Engaging well with this stage can be time-consuming, and even costly when you take into account the time taken to analyse the information you’ve gained. But the results you will get from analysing your results are well worth the effort.
Analysing results gained from evaluation and assessment can:
- Increase student productivity
- Heighten student morale
- Heighten quality ratings
- Increase student satisfaction
This is because you have looked at what has worked, what hasn’t worked, and used this information to improve your course. Once you make a change to your course based on assessment and evaluation, your next round of assessment and evaluation results will tell you how well your changes have impacted your student experiences and learning.
The Phillips Return of Investment (ROI) Model
A similar method of evaluation to Kirkpatrick’s Four-Level Training Evaluation Model is The Phillips ROI Model.
The Phillips ROI Model follows the same 4 stages detailed in the Kirkpatrick’s Four-Level Training Evaluation Model, but adds a fifth stage which looks at return on investment.
Ultimately, this fifth stage measures the difference between training cost and training results.
Such an approach can be useful for ensuring that training and education activities stay within a set budget whilst still meeting required learning outcomes.
Like the fourth stage of analysing results of assessment and evaluation, this fifth stage may be time consuming. But doing it will ensure that you do not spend money unnecessarily when making changes to your online course.
For example, if there is an easy, fixable solution for why training results might not be matching up to the invested cost, this will become apparent on your ROI analysis.
For example, if you are implementing an online course in a workplace, you might find that training satisfaction and knowledge are being met when the course is evaluated. However, you might find that students are not being enabled to use their new knowledge and skills in practice.
In this case, the ROI of the company training their staff is quite poor. Money is being spent on training but improvements are not being seen as new skills aren’t being implemented. A way to improve ROI would be to ensure that a sufficient mentoring programme was put in place to enable course students to implement their new skills in their place of work.
Involving students
When collecting feedback to evaluate learning, a common problem is that students see this process as a tick-box activity.
Therefore, evaluation forms might not always be as truthful or as complete as they could be. This is particularly true if students must return feedback in order to complete a course and be issued with a certificate.
For our online learning courses, usually, a part of the requirement to complete the course and gain a certificate, a student must first fill out a course evaluation form.
Whilst these forms tend to gain teachers very valuable feedback, they can sometimes appear rushed as the student wants to complete the course and gain their certificate.
A way to motivate students to fill out the evaluation form as fully as possible is to honestly tell your students why their feedback is so important.
For example, at the top of each evaluation form include a couple of sentences explaining that feedback and course evaluations are vital for you to improve your course and student experience.
In face-to-face training, a similar message can be given verbally.
The theory behind such a message is that engaging students in how they can influence training and education for their own future learning and that of their peers may motivate them to become more involved in their evaluations and feedback.
Continuous evaluation of learning
Evaluations do not always have to just take place at the end of a course or module.
To this end, evaluation can be formal or informal.
Involving students in course evaluations may be more specific. For example, evaluating a new method of training delivery, asking what was easy or difficult about a task, or how the student felt an assessment went.
In an online course, such information could be gained via a virtual meeting, an online forum, or a mini-evaluation form strategically placed mid-course.
Applying principles of assessment and evaluation throughout an online course
Principles of evaluation can be applied to the start, middle and end of an online course or activity in order to gauge effectiveness and relevance of your students’ learning.
Applying principles at the start of programme
Some teachers like to engage in the final stage of Kirkpatrick’s Four-Level Training Evaluation Model at the beginning of their programme.
This might seem like a bit of an odd concept, but there is sound logic here.
By starting out with applying the final stage first, you can improve course outcomes before your even start. For example, when establishing learning outcomes, you can also set goals that you want to meet within your results.
This is a tactic that I employ regularly when planning any online course which I create. When teaching, one of the results which is always kept in my mind is whether at the end the student will be able to practice in a safe and effective manner when applying theory in practice.
Having a background of working in healthcare, this type of evaluation is based on manager feedback, service user feedback (which is gained through wider, organisational surveys), and observation in practice from myself, my team and departmental staff who feedback to me.
Additionally, when applying principles of assessment and evaluation at the beginning of a programme, learning styles must be considered.
Whilst it is not always feasible to adapt course curriculums to every learning style, consideration can be given to ensure that a variety of activities are available which lend themselves to different styles of learning.
Applying principles at the mid-programme
During the programme, you can carry out student evaluations from student reactions, learning and behaviour. In an online course, this information might be gained from engagement and responses put in student forums or through assessment scores.
If you are teaching through blended learning or hosting online sessions via video software such as Zoom, you can observe the verbal and non-verbal cues from students.
Some behaviours that you might pick up on include students who are actively participating, students who are avoiding group work, informal questioning and observation, or formal mid-way assessment.
At this stage of a teaching course, ongoing evaluation is useful, as gaps in your students’ knowledge and understanding can be identified early and corrected prior to the end of the programme or course.
Applying principles at the end of programme
End of programme evaluation takes the form of formalised feedback surveys. For formalised and accredited courses, internal and external quality assurance activity also contributes to the assessment and evaluation of a course.
There is also the evaluation of whether or not assessment results have been met and if the course offers a positive return on investment.
Applying principles as a basis of team or organisation review and quality assurance processes
When applying the principles of evaluating learning to the evaluation of learning programmes, internal and external requirements must be considered and adhered to. Internal requirements include Internal Quality Assurance (IQA) processes and adhering to organisational policies and procedures. External requirements include adhering to the requirements of Ofqual and our awarding bodies. IQA also ensures that assessment decisions made are reliable and valid. IQA feedback will be given to the relevant assessor about maintaining or raising standards.
These two processes complement each other. IQA activities are regularly undertaken within organisations by way of regular IQA activity and standardisation meetings. These activities ensure adherence to organisational policies and processes, as well as curriculum standards.
External Quality Assurance (EQA) happens less often and is undertaken by the awarding body to ensure compliance with their standards, and those of a regulatory body (if relevant) (Gravells, n.d.b). The EQA process ensures that students receive quality services and that IQA and assessment decisions are reliable and valid. EQA feedback will be provided to the centre about maintaining or raising standards.
What do you think? Did I miss anything? Let me know in the comments!
