Assessing learners' understanding through node-link analysis - Adidah binti Lajis (Kulliyyah of Information and Communication Technology, International Islamic University Malaysia, 2011)
Assessing learners' answer is a time consuming task that makes teachers or educators cut down the time they can devote to other important tasks. In fact, many academic examinations make heavy use of questions that require learners to write one or two sentences and which are worth for some amount of marks. These short answer questions are highly valued and integral to the examinations but have been absent from computer-based assessment due to limitations in computerized marking technology. This thesis presents an assessment technique to assess learners' understanding for short free-text answer based on a hybrid approach with a combination of natural language processing, node-link representation and excess entropy. A textual answer of learner and expert are tagged according to its part of speech and only noun, verb, adverb, and adjective terms will be extracted. Each expert term is assigned to synonym terms before any comparison of a learner's term is performed. Later, the knowledge model for both expert and learners' are generated by converting the terms into a node link representation to uncover the hidden knowledge structure. By applying an information theory approach that is excess entropy, the generated knowledge models of an expert and learner are analyzed and results are in bits unit. Testing materials are obtained from 4 different domains that is computer science, medical, engineering, and science and also being categorized into 2 categories that are school and tertiary. Bloom taxonomy is used in categorizing the competency level test for each testing material. Pearson correlation, exact agreement and exactplus- adjacent agreement are used as an evaluation method. Results show computer science, and medical domain of tertiary level has achieved an excellent agreement of between 81% and 87%. And science domain of school level has achieved a fair to good agreement of between 48% and 66%. Experiment also shows that NL Scoring able to assess knowledge, understanding, and evaluation competency test for both tertiary and school level. Benchmarking NL Scoring with other baseline method such as vector space model, and latent semantic analysis has showed that NL Scoring performed better than them. Research hypotheses have indicated that proposed approaches is domain-independent, workable for sentence length of at least 3 words for both tertiary and school level, and has achieved a comparable result to other existing technique concerning short answer text assessment.