This week we talked about evaluation. There are usually two types of evaluation: formative evaluation and summative evaluation. We have learnt them and their purposes in IDE 631. While in most cases, a summative evaluation is conducted to see whether learners achieve predecided goals and objectives, people tend to neglect formative evaluation because of time or budget constraint. Sometimes, if the training or instruction is only a one time thing which is so well-defined that you can hardly generalize it, we do not necessarily need to conduct a formative evaluation. I still remember in IDE 631, the project I worked on was to design a 2-hour workshop for 24 students who were required to do image editing work in a given task. In this context, everything was clearly defined; from students characteristics to learning content. Part of my evaluation plan was methods & instrument of conducting a formative evaluation. I did it without thinking much. After I presented my work, Tiffany just asked me one question, “If you won’t do it twice, why do you design the formative evaluation?” Then I realized we can’t regard ADDIE model or any other theories as the bible. Instead we should modify them to fit the current situation.
We also discussed the importance of involving both internal evaluator and external evaluator during the development and implementation. Through this mixed approaches, we not only increase the objectivity and independency of the evaluation, but ensure that experts who know the context well are included.
Kirkpatrick’s four-level of evaluation is a useful model that people could examine to what degree their evaluation achieves. For level 1, Reaction, interview, (usually attitude) questionnaire and observation can be utilized to see people’s reaction towards a specific item. Exam can be conducted at the Learning level. Evaluations on the third level, Performing, which is about learning transfer, are faced with some key issues such as when learning transfer occurs and how much can be transferred. It is not uncommon to see that learners did a good job in Level 2 but failed to perform well in their work. Reasons of it vary from inadequate time to a lack of motivation. The highest level, Results, is almost beyond consideration as its complexity. It is concerned with the long-term impact on the whole organization.