Abstract
Peer feedback can be a powerful tool as it presents learning opportunities for both the learner receiving feedback as well as the learner providing feedback. Despite its utility, it can be difficult to implement effectively, particularly for younger learners, who are often novices at providing feedback. It can be difficult for students to learn what constitutes "good"feedback - particularly in open-ended problem-solving contexts. To address this gap, we investigate both classical natural language processing techniques and large language models, specifically ChatGPT, as potential approaches to devise an automated detector of feedback quality (including both student progress towards goals and next steps needed). Our findings indicate that the classical detectors are highly accurate and, through feature analysis, we elucidate the pivotal elements influencing its decision process. We find that ChatGPT is less accurate than classical NLP but illustrate the potential of ChatGPT in evaluating feedback, by generating explanations for ratings, along with scores. We discuss how the detector can be used for automated feedback evaluation and to better scaffold peer feedback for younger learners.
Original language | English (US) |
---|---|
Title of host publication | LAK 2024 Conference Proceedings - 14th International Conference on Learning Analytics and Knowledge |
Publisher | Association for Computing Machinery |
Pages | 55-65 |
Number of pages | 11 |
ISBN (Electronic) | 9798400716188 |
DOIs | |
State | Published - Mar 18 2024 |
Event | 14th International Conference on Learning Analytics and Knowledge, LAK 2024 - Kyoto, Japan Duration: Mar 18 2024 → Mar 22 2024 |
Publication series
Name | ACM International Conference Proceeding Series |
---|
Conference
Conference | 14th International Conference on Learning Analytics and Knowledge, LAK 2024 |
---|---|
Country/Territory | Japan |
City | Kyoto |
Period | 3/18/24 → 3/22/24 |
Bibliographical note
Publisher Copyright:© 2024 Owner/Author.
Keywords
- Generative AI
- Language Analytics
- Large Language Models
- Natural Language Processing
- Peer Feedback