Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities

Rating scales are useful tools that can mitigate the validity issues of objective writing tests. In addition, they can reduce the reliability problem of essay tests that are scored impressionistically. The existing instruments are not appropriate for use in Malaysian universities. They are either ge...

Full description

Saved in:

Bibliographic Details
Main Author:	Nimehchisalem, Vahid
Format:	Thesis
Language:	English English
Published:	2010
Subjects:	College students' writings Report writing - Case studies English language - Rhetoric
Online Access:	http://psasir.upm.edu.my/id/eprint/19551/1/FPP_2010_13_F.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-upm-ir.19551
record_format	uketd_dc
spelling	my-upm-ir.195512013-04-04T04:33:44Z Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities 2010-10 Nimehchisalem, Vahid Rating scales are useful tools that can mitigate the validity issues of objective writing tests. In addition, they can reduce the reliability problem of essay tests that are scored impressionistically. The existing instruments are not appropriate for use in Malaysian universities. They are either generic or their grading system is not suitable for this context. The only genre-specific scale developed in Malaysia (Wong, 1989) is suitable for the narrative mode. The lack of adequate scales leads the lecturers to score their learners‟ written pieces impressionistically (Mukundan & Ahour, 2009). This developmental study included design, operationalization, trial and validation of an analytic scale of argumentative writing. ESL writing lecturers‟ views on the importance, wording and inclusiveness of the criteria were investigated (through qualitative and quantitative methods) with the help of a checklist. Parallel with these studies, a number of argumentative samples (n =20) were also analyzed. Thes samples had been selected from a batch of 167 argumentative essays written by students from six different faculties of Economy and Management, Health and Medicine, Design, Communication, Agriculture and Ecology in Universiti Putra Malaysia (UPM). Additionally, a focus group study helped the researcher further refine the checklist based on its respondents‟ views. They included four female senior lecturers from a Malaysian public university. As a result, a prototype scale was developed. Focus group participants reconvened to further refine the prototype, the clarity of its rubrics and their weights, the anchor papers and extended scoring guide. The result was the Analytic Scale of Argumentative Writing (ASAW). It followed an equal-weight scheme with five subscales of „content‟, „organization‟, „language conventions‟, „vocabulary‟ and „overall effectiveness‟ and could diagnose five levels of performance. The scale was tested for its inter/intra-rater and internal reliability, which resulted in moderate/high coefficients. Concurrent validity tests showed high and significant relationships between the students‟ MUET bands and the scores assigned to their written samples using ASAW. In addition, the samples were scored using four other well-established writing scales. Correlation tests indicated moderate and significant correlations between these scores and those assigned using ASAW. As for consequential validity, the raters‟ responses to a questionnaire on usefulness of ASAW indicated their moderate-very high levels of satisfaction with it. The scale can be used as a tool to benefit three main contexts in language instruction “(a) research, that is, theory building; (b) policy development; and (c) professional practice, that is, classroom or curriculum development” (Kiely, 2009:101). College students' writings Report writing - Case studies English language - Rhetoric 2010-10 Thesis http://psasir.upm.edu.my/id/eprint/19551/ http://psasir.upm.edu.my/id/eprint/19551/1/FPP_2010_13_F.pdf application/pdf en public phd doctoral Universiti Putra Malaysia College students' writings Report writing - Case studies English language - Rhetoric Faculty of Educational Studies English
institution	Universiti Putra Malaysia
collection	PSAS Institutional Repository
language	English English
topic	College students' writings Report writing - Case studies English language - Rhetoric
spellingShingle	College students' writings Report writing - Case studies English language - Rhetoric Nimehchisalem, Vahid Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
description	Rating scales are useful tools that can mitigate the validity issues of objective writing tests. In addition, they can reduce the reliability problem of essay tests that are scored impressionistically. The existing instruments are not appropriate for use in Malaysian universities. They are either generic or their grading system is not suitable for this context. The only genre-specific scale developed in Malaysia (Wong, 1989) is suitable for the narrative mode. The lack of adequate scales leads the lecturers to score their learners‟ written pieces impressionistically (Mukundan & Ahour, 2009). This developmental study included design, operationalization, trial and validation of an analytic scale of argumentative writing. ESL writing lecturers‟ views on the importance, wording and inclusiveness of the criteria were investigated (through qualitative and quantitative methods) with the help of a checklist. Parallel with these studies, a number of argumentative samples (n =20) were also analyzed. Thes samples had been selected from a batch of 167 argumentative essays written by students from six different faculties of Economy and Management, Health and Medicine, Design, Communication, Agriculture and Ecology in Universiti Putra Malaysia (UPM). Additionally, a focus group study helped the researcher further refine the checklist based on its respondents‟ views. They included four female senior lecturers from a Malaysian public university. As a result, a prototype scale was developed. Focus group participants reconvened to further refine the prototype, the clarity of its rubrics and their weights, the anchor papers and extended scoring guide. The result was the Analytic Scale of Argumentative Writing (ASAW). It followed an equal-weight scheme with five subscales of „content‟, „organization‟, „language conventions‟, „vocabulary‟ and „overall effectiveness‟ and could diagnose five levels of performance. The scale was tested for its inter/intra-rater and internal reliability, which resulted in moderate/high coefficients. Concurrent validity tests showed high and significant relationships between the students‟ MUET bands and the scores assigned to their written samples using ASAW. In addition, the samples were scored using four other well-established writing scales. Correlation tests indicated moderate and significant correlations between these scores and those assigned using ASAW. As for consequential validity, the raters‟ responses to a questionnaire on usefulness of ASAW indicated their moderate-very high levels of satisfaction with it. The scale can be used as a tool to benefit three main contexts in language instruction “(a) research, that is, theory building; (b) policy development; and (c) professional practice, that is, classroom or curriculum development” (Kiely, 2009:101).
format	Thesis
qualification_name	Doctor of Philosophy (PhD.)
qualification_level	Doctorate
author	Nimehchisalem, Vahid
author_facet	Nimehchisalem, Vahid
author_sort	Nimehchisalem, Vahid
title	Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
title_short	Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
title_full	Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
title_fullStr	Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
title_full_unstemmed	Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities
title_sort	developing an analytic scale for evaluating argumentative writing of students in a malaysian public universities
granting_institution	Universiti Putra Malaysia
granting_department	Faculty of Educational Studies
publishDate	2010
url	http://psasir.upm.edu.my/id/eprint/19551/1/FPP_2010_13_F.pdf
_version_	1747811409368973312

Developing an Analytic Scale for Evaluating Argumentative Writing of Students in a Malaysian Public Universities

Similar Items