The reality of assessing ‘authentic’ electronic portfolios: Can electronic portfolios serve as a form of standardized assessment to measure literacy and self-regulated learning at the elementary level?
This study explores electronic portfolios and their potential to assess student literacy and self-regulated learning in elementary-aged children. Assessment tools were developed and include a holistic rubric that assigns a mark from 1 to 5 to self-regulated learning (SRL) and a mark to literacy, and an analytical rubric measuring multiple sub-scales of SRL and literacy. Participants in grades 4, 5 and 6 across two years created electronic portfolios, with n=369 volunteers. Some classes were excluded from statistical analyses in the first year due to low implementation and some individuals were excluded in both years, leaving n=251 included in analyses. All portfolios were coded by two coders, and the inter-rater reliability explored. During the first year Cohen’s kappa ranged from 0.70 to 0.79 for literacy and SRL overall, but some sub-scales were unacceptably weak. The second year showed improvement in Cohen’s kappa overall and especially for the sub-scales, reflecting improved implementation of the portfolios and use of the assessment tools. Validity was explored by comparing the relationship of portfolio scores to other measures, including the government scores on the open-response literacy questions for the Canadian Achievement Tests (version 4), the scores we assigned to the CAT-4s using our assessment tools, and scores on the Student Learning Strategies Questionnaire (SLSQ) measuring SRL. The portfolio literacy scores correlated (p<0.01) to scores we assigned the CAT-4s using our assessment tools, and to government pre-CAT-4 scores, but the self-regulatory learning scores did not correlate to our measure of student’s self-regulation. The results suggest that electronic portfolio assessment is time-consuming and difficult due to the range of varying evidence within even a single individual’s portfolio and that it may not be fair to do across diverse classrooms unless there are shared guidelines or tasks.
Canadian Journal of Learning and Technology
