Scalasupporting competency assessment through learning analytics

Resumen

The introduction of Information and Telecommunication Technologies requires skills to take advantage of them. These are the 21st century’s capabilities, characterized by abstract skills such as teamwork, time management, creativity, negotiation, etc. Accordingly, universities have emphasized competencies as central elements for students’ development, evolving from a content-based towards a competency-based educational model. The diversity of these new literacies demands new forms of assessment, able to measure richer and more complex learning tasks. However, the assessment of these competencies is difficult; there are four main reasons that make it a challenging issue: 1) the lack of scalability; 2) the subjective nature of interpreting the development of students’ competencies; 3) the difficulty to find latent skills behind activities; and, 4) the lack of assessment activities to measure students’ behavior to arrive to his or her answer. In this digital era, when a high proportion of interactions in education are computer-mediated, an unprecedented amount of data is becoming available. As learning tools and resources are increasingly moving into the cloud, the challenge is how to aggregate and integrate raw data from multiple and heterogeneous sources to create a useful educational dataset that reflect the distributed activities of the learner. Learning Analytics, understood as the measurement, collection, analysis and reporting of data about learners and their contexts, could support learning and competency-assessment in that way. We have designed and developed SCALA (Supporting Competency-Assessment through Learning Analytics), a gathering and processing system that helps to analyze educational data from technologyrich environments to support competency-assessment through the enrichment of rubrics. This system implements the whole lifecycle of a learning analytics process: selection, capture, aggregation, process, use and refine. It consists of four components: 1) Data Sources, which implement the sensor layer to collect data from students and teachers, and their activities; 2) Data Integration, which implements the normalization and aggregation layer to integrate data from different sources, according to the SCALAmodel; 3) Database, to allow extensibility and openness, offering datasets for further explorations out of the SCALA system; and 4) Dashboard, which implements the high-order indicator layer to display knowledge to teachers. To model and represent social and usage data for assessment purposes we have used IMS Caliper learning analytics interoperability framework. In order to generate standardized Enriched Rubrics, at the representational level, the Learning Events are stored in the form of a data triple Learning Context - Action - Activity Context. This storage format allows performing more human reasoning processes to discover interesting patterns and insights about students. To evaluate SCALA, two approaches have been followed: technical one (considering the normalization and data quality metrics) and user one (including a methodology to assess whether teachers consider the tool as useful as it has been proposed for competencyassessment purposes). The goal is not only to visualize learning activities’ metrics in a dashboard (SCALA-Dashboard), but mine them using analytics techniques for the discovery of students’ patterns and metric relations in technology-rich learning environments. Results obtained show that it is possible to gather, process and visualize heterogeneous learning data in an efficient way, and that teachers find SCALA suitable and useful for tasks related to competency-assessment. This work should be considered as a first step towards future research on supporting competency-assessment through the massive analysis of educational data.