Objectives of peer assessment

 

Across centuries, a myriad of studies on the importance of peer assessment have been conducted. Most authorities conclude that there are several major objectives of peer assessment. The ultimate goal of this peer assessment proposal is to achieve these objectives, and it is summarized as follows:

 

• To enhance learning motivation: active rather than passive role (Rosalind Duhs 2010)  

• To encourage students to take responsibility (Sally Brown and Angela Glasnser 1999)[1]

• To develop critical thinking skills(Sally Brown and Angela Glasnser 1999)

 

Methodology

 

In order to justify in what extend we achieve the above mentioned objectives, both formative and summative peer assessment are adapted and outlined specifically.  

 

Formative Assessment

A popular definition of formative assessment can be referred to assessments that “provide information to students and teachers that is used to improve teaching and learning” (Atkin, Black & Coffey, 2001, p25). To simplify, formative assessment is the “assessment for learning” (Gipps & Stobart, 1997; Stiggins, 2002). [2]Besides, most scholars conclude that peer feedback is the indispensable element of a successful formative assessment (Sadler, 1989)[3]. Therefore, peer comments would take a key role in our assessment.

 

• Free comments from peers

Students are encouraged to leave comments, recommendations etc. to their peers. This serves as a means to receive valuable feedback from colleagues.

 

• One compulsory appreciatory comment

Numerous researches reveal that positive feedback show affirmative impact on motivation (Handbook of Motivation at School, 2009; Motivation and Self-Regulated Learning: Theory, Research, and Applications: Dale H. Schunk, Barry J. Zimmerman). Therefore, unlike usual practice, one appreciatory commend is made compulsory in each assessment.

 

Summative Assessment

The modern studies refer summative assessment to “assessments of learning” (Black & Wiliam, 2003; Broadfoot, 2008) [4] . Summative assessment should be distinct from formative assessment in the sense that it mainly centres on the achievement of individual in different aspects (National Research Council [NRC], 2001; Shavelson, 2006). Since the achievement of individual is inevitably scaled according to a measurable standard. Therefore, a rating system is adopted as a means of measurement.

 

• Four domains in summative assessment

Students will be assessed in four domains, within which the assessment criteria are set out to achieve our primary objectives. Each domain shares equal weighting:

1. Contribution to meetings (25%)

• Participation in meetings

• Enriching discussion content with related research and knowledge

• Leading and developing the discussion

 

2. Commitment to common goals (25%)

• The quantity of time and effort been put in the work

• Awareness to improvement

• Playing a role as a team member in the project but not an individual

 

3. Hard and soft skills input (25%)

• Ability to deliver his ideas wholly and smoothly during discussion and deliverables

• Ability to perform in-depth and related research

• Ability to deliver specific knowledge and perspective from the Real Estate & Construction Profession

 

4. Responsibility to the group and tasks (25%)

• Persistent quality control on works delivered

• Overall participation and involvement

• Ability to work proactively and individually

 

• Seven-point rating system

Students will be assessed based on the criteria listed above according to a seven-point rating system (Excellent: 7; Good: 5-6; Average: 4; Bad: 3-2; Worse: 1). There are two major justifications.

 

To preserve a neutral perspective as an available option and reduce distortion of survey (Marketing Bulletin, 1991)[5]

A rating scale with mid-point is adapted. The performance of individual shall not be strictly evaluated as good or bad, excellent or worse. There should be a neutral choice representing an idea of “average”.  Otherwise, the result of survey will be subjected to a distortion of “either good or bad”. The reliability of result is then decreased.

 

To distinct between excellent and good, worse and bad in a better way

Unlike other mid-point systems (eg. five-point system or nine-point system), seven-point rating system is adapted because it efficiently differentiates the significance between excellent and good, worse and bad, in terms of range of magnitude.

 

​ To allow objective and reflective grading

Different scale system or grade system is not applied here because only by choosing an appropriate scale system can the true value of the evaluated person be reflected in the grade. Enough time and concrete measurements should be provided if a larger or more detailed scaling is to be used, otherwise, the evaluator will evaluate subjectively or the mark given will not serve its original purpose, i.e. reflect the true value of the effort of the evaluated.

Therefore, a seven-point rating system is a reasonable instrument to rate the peer’s performance.

 

Data Analysis

 

• Method to deal with extreme values

There is a concern that some students may corrupt and cheat in peer assessment so as to get a higher final mark. There would be some extreme values derivated largely from the median/mean of the overall mark. The present of these values may due to error, biases or corruption of some individuals. In order to ensure the fairness of the marks, the followings are some method to deal with outliners.

The simplest ways of finding outliers in one-dimensional data is to mark as a potential outlier any point that is more than two standard deviations, say, from the mean, referring to sample means and standard deviations, but the presence of outliers is likely to have a strong effect on the mean and the standard deviation, making this technique unreliable.

Another method is the trimmed mean, which is calculated by discarding a certain percentage of the lowest and the highest scores and then computing the mean of the remaining scores. Similar principle is also applied to the inter quartile range(Q3-Q1) or inter decile range(D9-D1) method, whereby the outliners are eliminated from the data set. However, the database of our peer assessment is small, using this method could result in great lost of the reasonable data and hence the result will not be accurate.

The method we chosen is the median absolute deviation (MAD). It is the median of the set comprising the absolute values of the differences between the median and each data point. We will set the cut-off MAD as 1.5, any points. For every criteria of the summative peer assessment, the final mark of that criterion will not simply take average on all the points (x) given by different group-mates, but eliminate the extreme value that fall outside MAD 1.5 i.e. x[which((abs(x - median(x)) / mad(x)) > 2)]. In other words, some data having a significant standard derivation will not be counted.

 

• Median

To convert the final mark to Grade and ensure our marks in peer assessment is comparable to other groups. The final mark of peer will be converted to a grade by making a scale table. To identify what grade that this score, i.e. 5.5/7 is referred to by compare to our overall group’s grade in the learning contract, delivery and the exhibition. In theory, if, in average, individual group-mate performs well in their daily works, it will lead to a good final grade of learning contract, delivery and the exhibition. So the grading of peer assessment should be correlated with the final grade of the rest in the project. The student whose score is the median among all in peer assessment should get the same grade with the grade of learning contract, delivery and the exhibition. But in this stage, we can first assume as follow.

 

Point 6-7 A-range

Point 5-5.99 B-range

Point 4-4.99 C-range

Point 2-3.99 D-range

Point 1-1.99 F-range

 

 

Implementation details of peer assessment

In order to effectively execute the proposed peer assessment, rules and guild-lines are outlined as follows:

 

• Biweekly assessment

The above assessments would be conducted biweekly, there are two fundamental reasons:

 

• A comprehensible and straightforward assessment rubric is set up

A crystal clear and user-friendly assessment rubric is constructed to establish proper criteria and standards for the assessment piece[6].  This shows students what standards they must observe with respect to different rating. (Please refer to appendix I for the assessment rubric)

 

• Practice opportunity

Students are given chances to trial the proposed peer assessment in the first two weeks.  Reasonable doubts about the assessment details can be raised in this transition period. This aims to engage students in a new system so as to smoothen the following assessment process.

 

Evaluation of peer assessment

Students will be given a final score according to the summative assessment. This final score is equivalent to the average of points scored by individual during the project.

 

 

 

[1] http://link.springer.com/article/10.1023%2FA%3A1003794000365

 

 

[2] http://www.gwaea.org/iowacorecurriculum/docs/AssessmentForLearning_LitReviewFinal.pdf

 

 

[3] Sadler, D.R. (1989) Formative assessment and the design of instructional systems

 

[4] http://www.gwaea.org/iowacorecurriculum/docs/AssessmentForLearning_LitReviewFinal.pdf

 

 

[5] http://marketing-bulletin.massey.ac.nz/V2/MB_V2_N3_Garland.pdf

 

[6] http://www.deakin.edu.au/__data/assets/pdf_file/0020/53462/peer-and-self-assessment.pdf