Utility of extended matching questions as an assessment tool in homoeopathic education

INTRODUCTION

According to Bloom’s Taxonomy, the cognitive domain has been the primary focus of most traditional education and is frequently used to structure curriculum learning objectives and assessments and activities.^[1] All innovations in teaching/learning serve little purpose if not followed by complementary assessment methods. A defining quality of a good assessment is that it should be valid, reliable and continuous or at least spread evenly over the whole course. A similar assessment taken at different times ought to yield similar results. It should help provide evidence of student learning. It must promote an environment where continuous improvement is well understood and ingrained in the institutional culture.^[2]

Various methods are available for student assessment, including global faculty ratings, structured oral examinations, standardised patient simulations, patient management problems, computer-based simulations and various forms of multiple-choice questions.^[3] Each method has inherent strengths and weaknesses associated with its replicability, validity and utility.^[3] These are a combination of what the student knows, partially knows, can guess, or is cunning enough to surmise from cues in the questions.

Extended matching questions (EMQ) is a written assessment tool widely used in Western medical colleges mainly to assess diagnostic thinking and clinical reasoning.^[3] Previously, EMQs were known as R-type MCQs.^[4] The central idea is to extend the learning acquired in classroom teaching about a clinical concept to other similar or different clinical scenarios within the same system. This method enables medical students to develop the ability analyse the clinical data given and synthesise their clinical diagnosis. According to a study conducted in Belgium, the correlation between the DTI score and the EMQ examination score indicated that the EMQ measures an aspect of student achievement that is related to clinical reasoning.^[5] Another study conducted in Belgium also proved the utility of EMQ in clinical reasoning.^[6] A paper presented at the Eastern Educational Research Association stated that EMQs helped to develop diagnostic pattern recognition.^[7] For instance, the concept of evolving diagnostic thinking is taught in the classroom on the topic of pneumonia while teaching respiratory system disorders. The concepts used to arrive at clinical diagnosis are signs and symptoms, clinical history with examination findings and investigations available in the case. This is demonstrated and discussed in the lecture. To assess how much the student has grasped this, EMQs are used post class. As a part of written assessment post class, caselets are given to students which provide information about patient symptoms with their onset, duration, and progress, examination findings (signs) and the available investigations. Most caselets are based on actual cases from practice. The student is expected to analyse the given data, synthesise it, apply the concept of reasoning taught in the classroom and arrive at a diagnostic conclusion while solving his assessment test. Thus, the learning is extended.

EMQ is a written assessment format similar to multiple choice questions (MCQ); the major difference is that they test knowledge in a far more applied, in-depth sense. MCQs encourage rote learning, focusing on recall and memorisation as the student must recall factual data from textbooks and use that to choose the correct option.^[8] A study conducted at a Dutch University revealed that EMQ helped to detect excellent students in the class who had an annual average grade equal to or more than 8, whereas the MCQ examination was 3 times better in identifying the poor students in the class who had an annual grade <6. This helped to design the study advice methods for the students in the propaedeutic phase of medicine and biomedical science study.^[9]

EMQ are reasonable alternatives to MCQS or free-response questions. EMQs allows one to ask questions, where any number of answers from a large provided list may be correct or incorrect. Examinees may also rank from a list of possibilities, those that are more correct. The same list of possible answers can pertain to any number of independent test items. There is no restriction on the number of times a given answer may be correct. Relative to other multiple-choice formats, there is less cuing and less chance of examinees guessing the correct response, both because there are more options and the list contains all relevant responses.^[3]

The structure of EMQ has three key elements: ^[10]

• 1) Answer option list: Sources suggest using a minimum of eight answer options to a ratio of 5 scenarios (also called vignettes) to ensure that the probability of getting the correct answer by chance remains reasonably low. The exact number of answer options should be dictated by the logical number of realistic options. This ensures that the test item has authenticity and validity.^[10]

• 2) Lead in question: This should be as specific as possible and on reading the lead in question it should be understood exactly what the student needs to do without needing to look at the answer options.

• 3) Two or more scenarios or vignettes: There should be at least 2 vignettes since the item allows for an in-depth test of knowledge; each of the scenarios should be related to one another by a theme that summarises the question overall. Each has one best answer from amongst the series of answer options given.

A study conducted at the Royal Veterinary College, United Kingdom, reports on the identification and implementation of a valid and reliable method to assess clinical reasoning using clinical scenario-based EMQs in the final examination. It demonstrated the positive student response to the new examination format using EMQ.^[11]

EMQs have been used effectively in the areas of clinical reasoning for the evaluation and diagnosis of musculoskeletal injuries and treatment techniques in athletic training. EMQs are used in pharmacology regarding drug adverse effects or drug pharmacodynamics (mechanism of action). Schilling reported on the use of EMQs in anatomy training. They stated that using scenarios instead of facts, the questions provide an authentic approach that assesses the application of knowledge instead of simple factual recall.^[12]

In homoeopathic philosophy, a miasm is considered the fundamental root cause for the initiation of any disease process. It is known to influence the course of disease pathogenesis. Thus, miasms have their reflection in the Onset and Progress of symptoms, the pace of the disease phenomenon, the potential complications of the disease process and the sequel of the disease process. Miasm thus is an important core concept of Homoeopathic philosophy.^[13]

In the current study, the platform of Advanced Teaching of Fundamentals of Homoeopathy sessions, which uses a case-based teaching methodology, was used. These sessions integrate Homoeopathic Philosophy, Repertory and Materia Medica with clinical medicine under one roof with case data without revealing patient identity.

To assess whether a student can apply what they have understood after a discussion about dominant miasm, EMQ could be the best method by giving other similar scenes from the practice experiences of teachers, converting them into questions (items) to be solved. Hence, it may be worthwhile to explore the use of the EMQ assessment method, especially at the postgraduate level, to understand the extent to which basic concepts are grasped. It may be applied to assess bedside training as well.

MATERIAL AND METHODS

RESULTS

Some students were found to be irregular in solving the pre- and post-test. Some could not attend the classroom discussion sessions on the pre-circulated case for the planned integrated session. Those students who were absent for the post-test were given a zero score. Hence, the conversion of the difference in the score between pre- and post-test showed negative results.

The marks obtained in the pre-test versus the post-test of each miasm were compared and the findings were subjected to statistical analysis using SPSS software (IBM Corp. SPSS Statistics for Windows, Version 24.0. Armonk, NY). Paired sample t-tests were used [Table 1].

The results indicated a significant change in the Tubercular miasm EMQ test, where the mean pre- versus post-test mark difference was 3.00 (significance level (P > 0.05) and validity of results.

DISCUSSION

In this study, EMQ was used as an assessment tool for the 1^st time with Homoeopathic postgraduate students. This can be considered a pilot study to understand the utility of this assessment tool. The existing review of literature so far shows the significant utility of EMQ as an assessment tool in the subject of Clinical Medicine. A study published in the Advances in Health Sciences Education Journal aimed to determine why students perform differently across different cases, it was based on the hypothesis that the variability of the performance links to variable levels of relevant knowledge. EMQs were found to be the ideal format to test this hypothesis. EMQs were designed to test the structure of knowledge and reasoning strategy. This experiment elicited a particular pattern recognition while solving the items on the same topic.^[14]

Homoeopathic postgraduates are expected to develop logical reasoning and the ability to perform fine analysis and synthesis and apply the concepts in practice. Homoeopathic educators can use them to assess how much a student exposed to certain core concepts of Homoeopathic Philosophy during classroom coaching can use them in other similar scenarios.

This tool of EMQ will prove very useful in the core subjects of Homoeopathic philosophy, Materia Medica and Repertory which needs to be assessed using the abilities of reasoning, analysis and synthesis and repeat application to similar situations.

In this study, the attendance of the student for the classroom discussion session and the knowledge gained showed definite influence on the application and correlation done by the student to extended scenarios. There was a definite positive change in the students’ scores, which proved that they could think laterally and thus could extend their learning to other case scenarios.

A study by Case and Swanson reveals that students answering this test have a greater chance of answering incorrectly if they cannot synthesise and apply their knowledge.^[15] It is one of the advantages of EMQ as there is less chance of getting the right answer by guessing. These observations were confirmed in the present study. In an experiment done in Belgium, on final-year medicine students, it was found that increasing familiarity with the question format showed measurable influence on the students’ scores.^[16]

As mentioned in the observations, this was not done for the participants in this study. They were not familiar with the format; they did not know how their performance will be graded and how it will reflect on their grades. Moreover, the pre- and post-tests were not taken on the same day at times as classroom integrated sessions used to happen fortnightly. This left a long gap between pre- and post-tests for Sycotic and Syphilitic miasms. The difference in scoring in pre- and post-test was noticed and a significant P value was found in the Tubercular miasm experiment.

These lacunae were left in implementation. They need to be corrected when this assessment tool will be used next time. Participants should know the consequential relevance of doing this exercise and should know that the performance will reflect in gradation. They should be familiar with the question formats.

The participants undergoing this experiment must be constant, committed and sincere and should be made to understand the importance of the application to EMQ. Considerable hard work on the teacher’s part is needed for implementing this methodology, but the advantage for students is significant as analysis, synthesis, application and correlation ability will increase. The pre- and post-test should be filled on the same day as the actual classroom discussion and demonstration sessions. If this care is taken, then the efficacy of this assessment tool in homoeopathic education will be established correctly.

Further work and experiments on this need to be planned properly to judge the efficacy of this assessment tool in application-based teaching of homoeopathic core subjects such as Homoeopathic philosophy, Materia Medica and Repertory along with clinical integration.

CONCLUSION

EMQ is an assessment tool that was found useful in homoeopathic postgraduate students. Concepts, reasoning and replication are the key features of EMQ. Therefore, EMQ as an assessment tool can be utilised in homoeopathic philosophy core subjects like miasms. EMQs are a more effective tool of assessment when implemented in a structured and planned way and it tests the higher cognitive domain of the learner.