Interesting hint system, but the part of the paper discussing how effective it was does not (to me) provide good evidence. Yes, there was an ~19% drop in attempts, but without controlling for (so many) other reasons for the drop, it’s impossible to say that, say, the Fall 2016 cohort had knowledge from the Fall 2015 cohort’s completion of the course. A randomized controlled study (some students getting useful hints, others getting unhelpful hints, and others getting no hints at all) would have provided better evidence.
Edit: This is not to say that the hints don’t provide benefit, they may, but there’s no evidence here to provide solid guidance to others.
Interesting hint system, but the part of the paper discussing how effective it was does not (to me) provide good evidence. Yes, there was an ~19% drop in attempts, but without controlling for (so many) other reasons for the drop, it’s impossible to say that, say, the Fall 2016 cohort had knowledge from the Fall 2015 cohort’s completion of the course. A randomized controlled study (some students getting useful hints, others getting unhelpful hints, and others getting no hints at all) would have provided better evidence.
Edit: This is not to say that the hints don’t provide benefit, they may, but there’s no evidence here to provide solid guidance to others.