Evaluation of the development and application of multimedia computer assisted learning in Higher Education.

Newton, Robert

Abstract

This thesis deals with approaches to the evaluation of multimedia computer assisted learning in higher education. The thesis is presented in two parts. The first part consists mainly of a literature based review of the rationale and methods employed in the development of multimedia CAL systems focusing on the ability of such systems to deliver a variety of pedagogic aims and objectives which the literature on the subject generally attributes to them. This was done in order to identify and examine the important features which should be incorporated in the effective evaluation of such systems. 1) the pedagogical basis of multimedia learning environments with particular reference to the mechanism by which they claim to encourage an approach to learning which facilitates 'deep' rather than 'shallow' learning' (Chapters 3 and 4); 2) the basis on which multimedia CAL systems claim to provide interactive learning environments which allow the teaching materials to be tailored by learners to accommodate their own individual preferences for adopting particular learning strategies. In particular this focused on the importance of individual learning styles and learners' degree of computer confidence (Chapter 5); 3) the institutional/delivery factors which must be understood to explain fully the context in which evaluations are carried out and which may have important effects on the outcomes of evaluation (Chapter 6). This literature review, together with a practical survey of a range of existing CAL courseware and an e-mail survey of CAL developers provides the basis for presenting an approach to evaluation which differentiates systems on the basis of the pedagogic approach they adopt and the context in which they are implemented. Finally, a critical review of existing evaluation methods was undertaken and important elements within these methods were incorporated into a new framework for evaluation. The framework provides a tool for determining an evaluation strategy that encompasses all stages of development, formative and summative evaluation of CAL courseware. Evaluation is based on the explicit aims and objectives of the courseware being provided and is moderated by contextual factors that define the pedagogical approach being taken, any individual learner differences that must be taken into account, and the institutional/delivery context within which the courseware is used. An analysis of the implications of the framework when formulating an evaluation strategy demonstrates weaknesses in the assessment instruments currently being used in evaluation studies - particularly for providing reliable measures of 'learning effect' as part of summative evaluation and also with respect to accurate quantification of costs associated with development and use of CAL courseware. The second part of the thesis tests the framework. The approach taken was to develop and formatively and summatively assess a multimedia CAL system used to teach parts of a course on bibliographic classification to students at the Robert Gordon University in Aberdeen. Qualitative and quantitative tests to accomplish this are described and the result of statistical analyses of learner performance when using the system are presented. This empirical study provides further insights into the practical problems involved in developing and evaluating a multimedia CAL system and in particular highlights: 1) the influence which individual learning style (as measured by the Gregorc Style Delineator) has on student performance in a context in which postgraduate students were required to use the CAL courseware rather than attend lectures - results indicate that CAL does not serve all learners equally; 2) the importance of the delivery context in a study in which undergraduate students were provided with CAL materials to supplement the delivery of their course. The evaluation framework was found to be a robust framework for developing and testing didactic teaching packages which were developed in the context of improving the quality of the teaching and learning of bibliographic classification to both undergraduate and postgraduate students. Recommendations are provided for future research based on using the framework to explore other contexts in which courseware is developed and implemented.

Citation

NEWTON, R. 2001. Evaluation of the development and application of multimedia computer assisted learning in Higher Education. Robert Gordon University, PhD thesis.

• Thesis Type: Thesis
• Deposit Date: Aug 4, 2015
• Publicly Available Date: Aug 4, 2015
• Public URL: http://hdl.handle.net/10059/1257
• Award Date: Mar 31, 2001