Use expert knowledge instead of data - generating hints for Hour of Code assignments

Abstract

Within the field of on-line tutoring systems for learning programming, such as Code.org's Hour of code, there is a trend to use previous student data to give hints. This paper shows that it is better to use expert knowledge to provide hints in environments such as Code.org's Hour of code. We present a heuristic-based approach to generating next-step hints. We use pattern matching algorithms to identify heuristics and apply each identified heuristic to an input program. We generate a next-step hint by selecting the highest scoring heuristic using a scoring function. We compare our results with results of a previous experiment on assignments 4 and 18 of the Hour of code. For assignment 4 we show that a heuristics-based approach to providing hints gives results that are impossible to further improve. For assignment 18, we show that our heuristic-based approach outperforms all models of previous experiments on the Hour of code. These basic heuristics are sufficient to efficiently mimic experts' next-step hints.