Using your learning edge
Part 9 – Evaluating your learning
Jeff Mitchell – Community Sport Advisor – Sport Auckland
A learning plan is designed to help increase your knowledge, leading to an increase in your effectiveness as a coach. Rather than being a static document, your learning plan should be regularly reviewed and updated. In this issue we will look at the role of evaluation and how to review your learning plan.
Purpose of evaluation
Evaluation is an important stage in the process of using of your learning plan. A learning plan should be an ongoing document that helps you to keep improving your coaching. It is important that you regularly evaluate the plan to make sure that it is effective and to improve your processes in implementing it.
You want to ensure that the plan is effective so that you know that you are improving. This means evaluating the plan itself and determining the effectiveness of the learning activities within the plan. You want to know if your learning objectives are being met. Implementing the plan will use up a precious resource: your time. As such, you also want to determine if the activities and resources identified in the plan are being used correctly.
Along with evaluating the plan, you also want to review it and renew it on a regular basis. This will involve looking at its strengths and weaknesses and using your evaluation information to improve your learning processes as well as your decision making.
Evaluation should be performed throughout your implementation of the learning plan, as you will want to track your short, medium and long-term progress. It should occur prior to a learning episode to make sure the learning objectives and actions are well written and relevant. It should occur during a learning episode to confirm that the learning activities are well conducted. And evaluation should also occur after a learning episode to check that the outcomes have been achieved and that there has been a real impact on your coaching.
Planning your evaluation
You need to plan your evaluation as you are writing the plan. Make sure you are clear on the purposes of your learning plan and also of its evaluation. Planning for evaluation involves determining some evaluation questions. What is it that you want to know? For the evaluation of a learning plan you could use some of the following evaluation questions:
To evaluate your learning plan you need to decide how you will answer your evaluation questions. This requires you to determine what data you need, how you will collect it, and what the timeline is for collection. Part of this will involve understanding what success looks like – what are the success indicators for the objectives in your plan? To help you with this you could work through the following questions:
As part of your planning phase you need to think about collecting baseline data. If you are looking for an improvement in a certain area (for instance the number of errors that you are able to correct) then you need to know what your starting point is. What is the level that you are trying to improve on? Decide how you will collect this data and when you will collect it again to look for an improvement. This comparison is important as it is how you determine if you are making progress.
The data you collect will depend on your evaluation questions. Some examples of data you might collect are detailed below.
It is important that your learning plan is designed to allow you to evaluate it successfully. You should design learning activities that lend themselves to being evaluated; if there is no way of knowing if the activity has had any impact, how do you know if it is working? For each objective identify the criteria by which you will assess its completion. Be clear on how you will measure their success. Is it to be able to do something, or do it a number of times? Is it a percentage? Will you test yourself or will it be measured when you are coaching? Make sure your plan clearly identifies how you will measure each objective and what is required for the objective to be considered achieved.
What to evaluate
What you choose to evaluate will depend on the evaluation questions that you have formulated. It will also depend on when you are evaluating – prior to a learning episode, during the use of your plan, or after the plan has been implemented. Some specific areas you might look to evaluate could include your objectives, the impact on your learning, and the learning activities that you have completed.
You might choose to evaluate the objectives that you set. Ideally this will be done prior to implementing the plan, although they could be part of a final evaluation. Evaluating your objectives before you implement them allows you to improve them before taking action on your plan. Ask yourself if they are appropriate and if they are likely to lead to the learning you are after. Are they SMART? Do the objectives make it clear how they are to be achieved (i.e. do you know how to measure them)? If you cannot answer “Yes” to each of these questions then your objectives need rewording. Following your implementation of the plan you could look at whether your objectives were achieved and why or why not.
Impact on learning
The purpose of your learning plan is to improve your coaching. An important consideration, therefore, is to determine the impact of the plan on your coaching. Your focus here could be to evaluate the change to your coaching that has led to improved outcomes for your athletes. This will require you to be clear on what you want your athletes to achieve. This means you need to understand how your learning objectives should impact on your athletes and how you will determine the achievement of your objectives. Identify what will be used as evidence of learning or the achievement of your objectives. Collect the necessary data and identify the visible markers of progress. This could be done by being observed by a mentor.
The most important component of the plan is the activities that you select that will help you to learn and to improve your coaching. One option is to look at your perception of the effectiveness of the learning activities that you participated in. How effective do you feel they were? What did you say about them when you reviewed your sessions or reflected on your learning? You could then examine the activities in greater detail by reflecting on the following questions:
By evaluating your learning activities you will get a better idea of which are useful and have led to greater learning. You will also develop a greater awareness of how to design and then implement learning activities more effectively in the future.
How to evaluate
The actual evaluation process is where you assess your plan and actions and then try and answer your evaluation questions. It will involve taking the data that you have collected (what has happened) and comparing it to your objectives (what you wanted to have happen). From this analysis you can determine if the plan has been successful, based on how you answer your evaluation questions.
Your evaluation will lead into your review, where you look to improve on the learning plan. To do this, your evaluation should tell you:
- If you achieved your objectives
- What led to the achievement of your objectives
- How effective your learning activities were
- What change has occurred in your coaching as a result of implementing the learning plan.
Once you have completed your evaluation and can answer these questions, along with your evaluation questions, you are ready to review your learning plan.
Reviewing your learning plan
The purpose of reviewing your learning plan is to help you to create a new, improved plan. This is done by taking the information from your evaluation and then reviewing each section of your plan. In reviewing each section you will decide what has worked well, what needs to change and what you could add. Your evaluation information should help you to understand what has worked and what did not.
Your review should look at the following elements of your plan:
- Your learning objectives
- Your learning activities
- Your implementation of your learning plan, and
- What you have learned.
The best way to conduct your review is to identify for each component some Start, Stop and Continue actions. This approach involves looking at your plan and deciding:
- What you could Start doing that would improve the plan and your learning
- What you should Stop doing as it has been ineffective or not worth the time spent on it, and
- What you should Continue doing as it has positively contributed to your learning.
Coach Gary has performed an evaluation of his learning plan that he worked through this year while coaching the Blockhouse Bay Buccaneers, a lacrosse team in Auckland. Following his evaluation he then conducted a review of his learning plan. Let’s look at how you would review your learning plan using a Start, Stop and Continue approach. For each area we will discuss what you could look at in your review and provide some example Start, Stop and Continue actions from Coach Gary’s review.
Hopefully you evaluated your objectives after you wrote them (and before you started working towards them). Your review could look at how difficult your objectives were, what areas they were focused on, your use of the SMART methodology or how many objectives you set.
Reviewing learning activities
To review your learning activities you will look at each activity in the plan and decide the following:
- Which activities to continue or repeat
- Which activities to drop
- What alternative activities could be more effective
- What types of activities you have responded best to (e.g. maybe you found that research activities provided you with the most success)
- Which activities you have enjoyed, and
- Which activities helped you to learn more and achieve your objectives.
Think over each of your learning activities and decide how effective they have been. This part of the review has two purposes: one, to help you to identify some activities for your updated plan, and two, to raise your awareness of how you learn best and what activities work best for you.
Reviewing your implementation of your learning plan
You can write a brilliant learning plan, however, it won’t help you to learn if you don’t implement it. Part of your review should involve looking at how well you implemented the plan. You might want to consider if you actioned each of your learning activities, how well you reflected on them to draw out the available learning, how much effort you put into following the plan, how you used it to guide your coach development, and what you could do to implement it better in the future. This part of your review is about identifying how you can make better use of the learning plans that you write.
Reviewing what you have learned
A good way to determine the effectiveness of your learning plan is to review what you have learned. This will help to solidify this learning and also tell you how effective the plan has been. Start off by listing what you have learned based on each of your learning objectives. Can you explain each of them in detail, or is your knowledge still lacking in one or more areas? Where you find that you do not yet know enough on a topic, you can dedicate some time in your next learning plan to further explore this area.
After you have worked through this review process, have a think about what support you need to implement your next plan and how you will get it. The outcome of your review should be that you are able to write a new, improved learning plan that keeps the effective elements from the previous plan and replaces the parts that proved ineffective.
Evaluation is an important part of the development and implementation of a learning plan. Rather than being left till after the plan has been implemented, evaluation should be planned from the start and performed throughout the period of learning. Use your evaluation information to review and update your plan, and to improve your learning in general. In our final issue we will explore some general tips that can help you to write and then implement your learning plan successfully.