How to work out if your E-Learning is a success and what to do if it isn’t

Image saying containing an illustration of a businessman flying into the sky in a rocket with a briefcase on which has been written the word SUCCESS. The title of this image is 'How to work out if your E-Learning is a success and what to do if it isn't'

In a bid to rush to the next project, many E-Learning professionals carelessly skip over the evaluation process. Because we believe it is vital to learn from experience, our mantra of the moment is:

 “Sometimes you win, sometimes you learn”.

Therefore, today we want to take the opportunity to unpack best practice in E-Learning evaluation.

Evaluation is often overlooked because people lack the time, patience or clarity on the right approach or tool to use. Often, they are unsure of what data to collect or how to use it once they have collected it.

The first step in evaluating an E-Learning campaign lies in identifying the success metrics or performance parameters. These are formed by aligning or comparing organisational aims and goals with the results of the learning campaign.

Following this crucial first step, there are a number of ideas and approach methods to evaluation. One of the foremost ideas being propounded currently is Kirkpatrick’s taxonomy.

Kirkpatrick identified four levels or criteria for successful training evaluation:

Level 1: Learner reaction

Level 2: Learner knowledge

Level 3: Learner behaviour

Level 4: Learning results or impact

Sadly, many learning and development departments move no farther in their evaluations than level one or two.

They listen to how a selection of learners feel about the learning campaign and measure how much information they have retained but fail to research how the newfound information is being applied or what impact it is having on the organisation.

So what are the best tools to use to evaluate success?

We recommend a combination of the following:

  • Knowledge assessments both before and after completing training to assess learning progress;
  • Conducting a pilot to spot flaws early on;
  • Using big data to track learner engagement as well as weak points within the E-Learning content;
  • On the job assessments by management to gauge behavioural changes;
  • A forum (such as Survey Monkey) to receive feedback from learners about their personal experience;
  • Encouraging learners and stakeholders to give feedback in the long term; and
  • Brinkerhoff’s Success Case Method suggests that we must look at success at an organisational or system level rather than just the individual learning programme. Brinkerhoff also asserts that it is useful to look at the “outliers” i.e. those who have been particularly successful or unsuccessful. By looking at these outliers, it may be possible to identify the differentiators between E-Learning success and failure on an individual level.

Now what?

After using this toolkit to evaluate your E-Learning success, you may find that learners are happy, clued in, applying their knowledge and your organisation is flourishing.

Hooray for you!

If you have a different result and find issues with your E-Learning, then take a deep breath, it’s going to be okay!

There are a number of reasons why your E-Learning campaign might not have hit the target this time.

It could be as a result of lack of support from management, the learning campaign being too theoretical or inaccuracy in identifying knowledge gaps pre-training. Luckily, your evaluation should point you in the right direction.

An important nugget of knowledge to keep in mind is not to throw the baby out with the bath water. In the majority of situations, change can be made in small, incremental amounts.

We recommend asking the people who have given you feedback and identified issues to become part of the solution by suggesting what they would have needed or preferred to better their learning experience.

There is always room for improvement and evaluation is one of the most consistently helpful tools we have at our disposal as learning and development professionals. Just remember our mantra, “Sometimes you win, sometimes you learn”.


Please let us know your comments or share with others who you think may benefit from this. We’d love to hear about your biggest E-Learning successes and failures. What is your strategy for recovering and fixing your mistakes? Follow us on twitter @aurionlearning for our latest blog articles and updates.


Top 5 Features of Articulate Storyline

Here at Aurion HQ, Articulate Storyline is our E-Learning authoring tool of choice. We like it because it produces fantastic learning content that hits the target both in terms of visual appeal and learning outcomes. Find out five reasons we think Articulate Storyline is fantastic:

Infographic displaying the information contained in the blog below.

This infographic is available to download here.


1. Ease of use

The PowerPoint style interface will put new users at ease instantly. Storyline is designed as simplistically as possible with the option to build your own slide or use one of the Articulate templates as a starting point. Storyline is great for instructional designers and learning professionals at any level of technical expertise.


2. Tracking and Reporting

Storyline content is Tin Can, SCORM and AICC compliant so any data tracking and reporting is an integrated part of the content development process. Learner progress and outcomes can easily be recorded and measured using your learning management system (LMS) or using Articulate Online.


3.    Interactivity

Great E-Learning contains high levels of interactivity. Storyline allows you to quickly create:

  1. Screen Recordings
  2. Quizzes and Assessments
  3. Slide Layers
  4. Triggers
  5. States


4.    Characters and Templates

Storyline contains a number of templates and character packs which mean you don’t have to spend hours building your E-Learning design from scratch. For those developing or following brand guidelines, it is possible to create a template and share it with colleagues to ensure there is consistency in the E-Learning that you produce.


5.    HTML 5, Flash and Mobile devices

E-Learning created on Storyline is publishable in HTML5, Flash and the Articulate Mobile Player, meaning it is available to learners on a wide range of devices. Users can tap into learning content whenever, wherever with whatever device they prefer.

Follow us on twitter @aurionlearning for our latest blog articles and updates.

4 Types of “Problem” E-Learners and how to deal with them

In an increasingly tech-savvy and budget driven age, the potential for E-Learning is limitless. The key to a successful E-Learning strategy is buy-in by people at all levels of an organisation. Sadly, anyone who has ever delivered an E-Learning programme will know that there will always be people who find transition difficult. We’ve identified four types of E-Learners to watch out for and how to deal with them when you spot them:

Infographic representation of the blog

This infographic is available for download here.

1. The Technophobe

Who? Often, but not always, this person is of an older generation and has limited experience with technology. They are timid as a mouse around the computer and if they had their way, they would print out the whole E-Learning course and complete it manually.

How to bring the best out in them: The greatest tool at your disposal for a technophobe is buckets full of encouragement. Every time they complete a task, make them feel like a champion. User interfaces should be simple and designed intuitively, with the learner in mind. If your organisation has a high concentration of technophobes, consider introducing blended learning or providing equipment so that these E-Learners can meet in groups and help each other out with the technological side of things.


2. The One with Great Intentions (but not much else)

Who? This particular character is enthusiastic and excited about all of the new facts and lessons they can learn with all of the beautiful online content you have created. However, when it comes to actually completing online modules, they fail at even the first hurdle of logging in.

How to bring out the best in them: Expect that you will have to sell your learning content to them. An enthusiastic person like this one will have many different interests competing for their time. When you release a new course, create a short description that tells the learner how much of their time they will need to spend and give them two or three returns they will gain from investing their time. If you can engage this learner and get them to take part in and enjoy your E-Learning, you’ll find that their enthusiasm is infectious.


3. The Shortcutter

Who? This type of E-Learner is their own worst enemy. They would like all of the benefits of hard work without actually having to do any. In their constant rush to the next achievement to add to their trophy cabinet, they have little more than superficial knowledge in any particular area.

How to bring out the best in them: There are three keys to slowing down this student long enough for them to learn something. Firstly, you can make it mandatory to spend time on each page of learning. Secondly, consider integrating assessments into the learning content to identify knowledge gaps and diagnose additional learning material to fill those knowledge gaps. Last but not least, try to create spectacular E-Learning content that distracts them from racing on.


4. The Spoon-fed Student

Who? This individual has always consumed superior quality, highly personalised learning content, which is certainly no bad thing in itself. However, because they have always been looked after so well, they tend to lack the initiative to further explore external learning content that would supplement and improve their learning experience.

How to bring out the best in them: The first step to creating a great E-Learning experience for this student is to communicate expectations. If he/she knows from the beginning that it will be beneficial to read supplementary materials, then that will hopefully plant a seed in his/her mind. We recommend that you include regular prompts throughout your E-Learning course that refer learners to relevant websites or repositories of valuable information. Try creating a social platform for learners to communicate and grapple with difficult questions. To kick-off conversations in these online forums, you could set a project which requires students to interact. If you succeed with this type of student, you won’t only have taught them something new in your subject area, but also the value of taking initiative.


We’d love to hear about difficulties you have experienced when delivering E-Learning and successful techniques you have used to overcome them. Join the conversation on twitter using the hashtag #elearningwin. Follow us on twitter @aurionlearning for our latest blog articles and updates.