I agree with OP on these specific examples, but I think writing code by hand on paper is great way for really understanding things. I had few courses where final exam (it took 70% of the grade) was writting code with pen and paper and answering what is the end result of few snippets of code. And that was really good way to learn C, It was tricky to solve all those snippets with nested for loops and bitwise operators, masking and pointers, but in the end I learned a lot. Is it the most efficient way? Probably not, but I learned things, without deceiving myself about understanding them.
The nice thing about pen and paper is that a reasonable grader won’t care if you wrote func("string", 34, a -> a ^2) or funcc(lambda a: a**2, 34, string)
(Of course, perfectionists got caught up in those things, even knowing the grader wouldn’t care, and missed points because of it)
That second example brings back memories to one course at uni, where the exams were full of these kind of questions. Getting though that course was pretty much a game of chance.
Not only did I (and everyone else) not learn anything through this kind of examination, it left us very frustrated.
In school, we learned programming via C++. There was usually a section on an exam of C++ semantics questions involving pointers and references that many people disliked. These types of questions would recur for about a year or two across courses, as well. The expectation was that you had the essential lingual forms nailed down in your head.
I don’t think everyone liked this sort of thing, but there’s something to be said for setting the bar high.
Hey, I took that class! (Maybe.) We definitely had an exam which involved decoding various c++ statements and identifying whether it’d be copy constructor or assignment or what happening.
In hindsight, I think part of the point was just to test if we’d been paying attention, but also to make the point that things are complex and subtle and if you don’t know what’s going on (or at the very least, the range of possible things going on), you’re going to have some sad times in the next course.
Huh, I don’t think I ever had a written exam in a programming-oriented course at university. (Though it’s not surprising universities do things very differently, since CS pedagogy isn’t that standardized.)
Within the CS dept, we usually only had written final exams in the more theoretical courses. For example in an algorithms course you might have an exam full of word problems setting up a scenario and asking you to sketch pseudocode for an O(n^2) algorithm for it, and prove it’s O(n^2). Or giving you some pseudocode and asking you what the running time is (and why). In the more programming-heavy courses, like C++/datastructures, compilers, etc., the grade was typically a mixture of weekly homework assignments, plus a final course project, but no final exam.
Well, pointers are key to understanding any non-trivial code, so I wouldn’t object to questions about that (although cdecl is a thing). But checking one’s knowledge about specifics in the java documentation doesn’t (imho) accomplish anything.
I’ve been thinking a lot recently about how much coding is done without thinking. Just throw stuff at the compiler until it seems to work. In Pragamtic Programmer, they call this programming by coincidence. I find myself doing this out of laziness, and it’s not good. It’s been a while since I’ve written anything by hand, but I’m pretty if I tried it would result in better code. I’d be wary of anybody saying they can’t program by hand.
That’s funny, because at the end of your first sentence, I was agreeing with you, on how all the real programming I do is on an intuitionistic level, below critical thinking.
There is a project on NYU’s Secure System’s Lab that tries to identify the programming constructs that lead to these ambiguities/misunderstandings. I may be biased, but I think it’s a really interesting project
One school of thought, Wirth’s, avoids this problem by keeping the language so simple the student could implement it themselves. Oberons are a good example with Modula-3 (Cardelli et al) and Component Pascal being more practical but harder to implement. Others along lines of Ada try to make the language very clear on what’s going on in about any situation to systematically remove errors. Steep learning curve and extra verbosity usually is a side effect.
That said, if one is concerned about corner cases, then they should eliminate them as the author suggests by using static analysis tools. If those are to be used, then a test like this might be pointless unless the specific behaviors are worth remembering. Another angle is instructor might teach them to use static analysis tools with expectation they’ll be applied to every assignment before submission. Then, a question like the OP’s on the students’ own code could be illustrative of why the behavior should’ve been avoided. Good answer with a fix might get less of a penalty than “I didn’t pay attention.”
I agree with OP on these specific examples, but I think writing code by hand on paper is great way for really understanding things. I had few courses where final exam (it took 70% of the grade) was writting code with pen and paper and answering what is the end result of few snippets of code. And that was really good way to learn C, It was tricky to solve all those snippets with nested for loops and bitwise operators, masking and pointers, but in the end I learned a lot. Is it the most efficient way? Probably not, but I learned things, without deceiving myself about understanding them.
Edit: typos.
The nice thing about pen and paper is that a reasonable grader won’t care if you wrote
func("string", 34, a -> a ^2)orfuncc(lambda a: a**2, 34, string)(Of course, perfectionists got caught up in those things, even knowing the grader wouldn’t care, and missed points because of it)
That second example brings back memories to one course at uni, where the exams were full of these kind of questions. Getting though that course was pretty much a game of chance.
Not only did I (and everyone else) not learn anything through this kind of examination, it left us very frustrated.
In school, we learned programming via C++. There was usually a section on an exam of C++ semantics questions involving pointers and references that many people disliked. These types of questions would recur for about a year or two across courses, as well. The expectation was that you had the essential lingual forms nailed down in your head.
I don’t think everyone liked this sort of thing, but there’s something to be said for setting the bar high.
Hey, I took that class! (Maybe.) We definitely had an exam which involved decoding various c++ statements and identifying whether it’d be copy constructor or assignment or what happening.
In hindsight, I think part of the point was just to test if we’d been paying attention, but also to make the point that things are complex and subtle and if you don’t know what’s going on (or at the very least, the range of possible things going on), you’re going to have some sad times in the next course.
Huh, I don’t think I ever had a written exam in a programming-oriented course at university. (Though it’s not surprising universities do things very differently, since CS pedagogy isn’t that standardized.)
Within the CS dept, we usually only had written final exams in the more theoretical courses. For example in an algorithms course you might have an exam full of word problems setting up a scenario and asking you to sketch pseudocode for an O(n^2) algorithm for it, and prove it’s O(n^2). Or giving you some pseudocode and asking you what the running time is (and why). In the more programming-heavy courses, like C++/datastructures, compilers, etc., the grade was typically a mixture of weekly homework assignments, plus a final course project, but no final exam.
Well, pointers are key to understanding any non-trivial code, so I wouldn’t object to questions about that (although
cdeclis a thing). But checking one’s knowledge about specifics in the java documentation doesn’t (imho) accomplish anything.I’ve been thinking a lot recently about how much coding is done without thinking. Just throw stuff at the compiler until it seems to work. In Pragamtic Programmer, they call this programming by coincidence. I find myself doing this out of laziness, and it’s not good. It’s been a while since I’ve written anything by hand, but I’m pretty if I tried it would result in better code. I’d be wary of anybody saying they can’t program by hand.
That’s funny, because at the end of your first sentence, I was agreeing with you, on how all the real programming I do is on an intuitionistic level, below critical thinking.
Yes I am! Anyone can be anything in Zootopia!
There is a project on NYU’s Secure System’s Lab that tries to identify the programming constructs that lead to these ambiguities/misunderstandings. I may be biased, but I think it’s a really interesting project
One school of thought, Wirth’s, avoids this problem by keeping the language so simple the student could implement it themselves. Oberons are a good example with Modula-3 (Cardelli et al) and Component Pascal being more practical but harder to implement. Others along lines of Ada try to make the language very clear on what’s going on in about any situation to systematically remove errors. Steep learning curve and extra verbosity usually is a side effect.
That said, if one is concerned about corner cases, then they should eliminate them as the author suggests by using static analysis tools. If those are to be used, then a test like this might be pointless unless the specific behaviors are worth remembering. Another angle is instructor might teach them to use static analysis tools with expectation they’ll be applied to every assignment before submission. Then, a question like the OP’s on the students’ own code could be illustrative of why the behavior should’ve been avoided. Good answer with a fix might get less of a penalty than “I didn’t pay attention.”