The problem is that basically nothing behaves like this in practice. We are always dealing with networks of facts and relations, with a fact within a context.
If a knowledgeable domain expert asserts that a => b, we may find that it’s actually a && p => b where the expert is used to assuming p, but we have to carefully check that p is actually the case.
In other cases, a => b actually does mean that ~a => ~b, because a => b is just a sloppy way of stating a && p <=> b && q, where p and q are always satisfied in practice.
Practically minded people are wary of abstraction, because abstractions are often too anemic to survive the confrontation with practice. Basic logical rules are one of the ultimate abstractions.
I’m not aware I’m saying it is. I’m saying that such a question, amidst a regular interview, is such a switch from practice to austere abstraction that you can’t blame people for misunderstanding the question.
That’s ridiculous. You don’t have to map domain expert knowledge to logic directly like that and expect the world to become suddenly consistent, of course; but logic still has a lot of uses in computer science and contrapositive is a basic tenant of classical logic. If someone uses boolean operators like &&||! daily, they should know how boolean logic works.
To me what you say is like saying “in practice computers use floating point arithmetic so it’s wrong to say that + on real numbers is associative”. Floats are messier than reals, but it doesn’t mean that reals don’t exist, or even that reals are irrelevant when reasoning about floats.
What is it that you think I’m saying that is ridiculous?
I think your analogy is very apt. The question whether the reals are associative is misplaced in a CS interview, because it is likely to be misinterpreted for a more complex question about floats.
IMO this is the sort of question that gives software engineering interviews a bad rap. I respect that formal logic is part of core CS fundamentals, but acting like it’s necessary to do your day job comes off as didactic.
As the author wrote, the conclusion and interpretation is for the reader.
IMHO, even if this is not the only aspect to check, such knowledge helps to make clean code. How to simplify complex code if you cannot ensure the equivalence of conditions?
I often see complex statements because of such lacks of basics (as foundation of house).
We cannot be expert in all aspects, but the team needs at least one member with such knowledge
The logical equivalence of the contrapositive is one of the very first things people ever learn about logic..along with the syllogism (a => b, b => c) => (a => c). Besides simplifying code, these are basic tools of reasoning in teenage (or younger) mathematics (and even in life). They apply really anywhere arguments take place, which is to say almost everywhere academic or even politics, etc.
Meanwhile, MVC (and many other design patterns) are more buzzwords for certain separations of concern, code/activity factoring. I would say you could be a great software engineer never having heard of MVC or design patterns. (Proof: There were great software engineers before it was coined! Convince me otherwise!) I am really not so sure you could be a great software engineer or programmer having forgotten basic logical equivalences, however abstract. As the author & Isammoc both write conclusions are for the reader, but this “study” was for some Master’s in SE program. So, besides filtering out weak candidates he was probably also trying to suss out curriculum issues in his program.
One non-objective thing here is that we have to trust that he asked his logic question in not too tricky ways and we only have his word on that, not transcripts or recordings. The equivalence is pretty obvious if you draw a Venn diagram, for example with an A bubble fully inside a B bubble in a set theory analogue. Maybe that gives it away too much or maybe that’s just like expanding the MVC acronym?
My experience interviewing and giving interviews for industry has led me to be wary of exactly the kind of questions in this article.
The quicksort question requires the candidate to have familiarity with O/Theta/Omega notation and knowledge of quicksort is off the top of their head.
The interviewer could get a better idea of the candidate’s understanding of complexity by presenting the candidate an implementation of quicksort and asking how they would expect it to perform on small, large, and maliciously constructed inputs. This selects for people with fundamental understanding of performance concerns, selects out people who memorized “quicksort is nlogn or n^2 in bad cases” without that understanding, and provides an easy avenue for the candidate to talk about tuning pivot selection based on the expected input domain.
Of course that requires the interviewer to do their share of preparation work as well.
Nearly every candidate could reason out the logic basis problem given time and comfort. Comfort isn’t a thing all interviewees have however; many are uncomfortable in general, but this is massively compounded by personal comfort with the interviewer. Stereotype threat means candidates that cannot mask their differences will, on average, be less comfortable. These types of questions produce a department that trends towards looking and acting superficially like the interview team. Moreover, they do so by impacting interviewee performance, and so cannot be compensated for by interviewers “acknowledging and reducing personal biases”
Since formal notation isn’t something that is likely to be top of mind for many candidates, this could be made more approachable by rephrasing it in concrete examples. e.g. “The undergraduate course CSC202 has a prerequisite of CSC101. Assume the department does not allow testing out of prerequisites, so that taking CSC202 implies a student must have taken CSC101. What can you say for sure if a student hasn’t taken CSC202? If they haven’t taken CSC101? If they haven’t taken either?”.
It’s clear after rephrasing that this question is barely a test of reasoning skills. That’s because it never was, and should probably be either expanded upon or dropped from the interview for time in favor of a more interesting prompt.
Do note however, that he is referring to interviews of CS graduates. That’s a different candidate pool than we typically see when interviewing for industry. I interview (and hire) many people who do not come from a formal CS education.
But when hiring specifically from a CS program, it’s not unreasonable to ask questions about basics that would indicate they at least attended their freshman year classes.
I was fortunate to take a formal logic course years ago as a philosophy student. Even though it was some time ago, the effort it required left an impression and an appreciation for the skills and work involved.
Almost 20 years later, I’ve found myself once again manipulating logical formulae – this time with an eye toward solving problems with SAT/SMT solvers. Problems, I should note, I would not know how to approach otherwise.
Without having had that exposure to formal logic back then, I seriously doubt I would have had the ambition to tackle the topic now from scratch.
Here’s what I think I read in Meyer’s piece: students can really only be expected to be familiar with concepts they’ve encountered in their coursework (at best). Presumably, folks know about MVC because they’ve been exposed to the concept in some form or another somewhere along the road.
“Over the past decade I have seen many computer science departments, including in top institutions, remove the logic course from the curriculum. And MVC is a great concept. But still.”
I read this more as a lament that formal logic is not given the place it deserves in the curriculum and less as a lament about the students’ knowledge per se. None of this strikes me as particularly controversial. Unless I misunderstand and there are folks suggesting that logic is unimportant for a computer science education. Is that the case?
“Few persons care to study logic, because everybody conceives himself to be proficient enough in the art of reasoning already. But I observe that this satisfaction is limited to one’s own ratiocination, and does not extend to that of other men.”
read this more as a lament that formal logic is not given the place it deserves in the curriculum
In any curriculum! A solid understanding of formal logic benefits any kind of structured thought. It is as important to the humanities as it is to computer science.
What kind of model answer do you think he’s expecting here?
Something like “A->B is equivalent to !A v B, so in the case of !A it doesn’t matter what B is, and in the case of !B we’d better have !A”? Or is he expecting someone to mention Modus Ponens and Modus Tollens?
I agree this is likely bread and butter for anyone with a theoretical CS / logic background, but I think if you haven’t ever come across/been taught the definition of ‘implies’ it’s really counter-intuitive at first.
Yes, I think so. Honestly though, with logic rules like this, if I’ve not seen a particular variant before I’m never 100% sure that I’ve got them right if I try and work them out in my head. I usually end up writing down a small truth table to make sure I’ve got the outcomes correct.
The contrapositive rule, modus tollens. I remember it. I can’t say I’ve actually used it recently. But if I’d gone through this interview I’d have been judged by it. This is the kind of question that un-diversifies your teams by selecting for access to advanced education. Understanding of low level theory is not nearly as important for every teammate to have, compared to qualities like empathy for your customers and users, owning your mistakes, or persistent curiosity. Good candidates can bring those qualities from any number of backgrounds.
Those details from my logic course come in handy in rearranging Boolean expressions to be more readable, notably the one where you distribute a negation and flip and and/or. It’s good to have someone who knows that on the team, who can share that someday at the whiteboard or in code review. But that question doesn’t belong in an interview.
The problem is that basically nothing behaves like this in practice. We are always dealing with networks of facts and relations, with a fact within a context.
If a knowledgeable domain expert asserts that a => b, we may find that it’s actually a && p => b where the expert is used to assuming p, but we have to carefully check that p is actually the case.
In other cases, a => b actually does mean that ~a => ~b, because a => b is just a sloppy way of stating a && p <=> b && q, where p and q are always satisfied in practice.
Practically minded people are wary of abstraction, because abstractions are often too anemic to survive the confrontation with practice. Basic logical rules are one of the ultimate abstractions.
That’s not a reason to stop teaching the basic rules at all.
I’m not aware I’m saying it is. I’m saying that such a question, amidst a regular interview, is such a switch from practice to austere abstraction that you can’t blame people for misunderstanding the question.
That’s ridiculous. You don’t have to map domain expert knowledge to logic directly like that and expect the world to become suddenly consistent, of course; but logic still has a lot of uses in computer science and contrapositive is a basic tenant of classical logic. If someone uses boolean operators like
&&||!daily, they should know how boolean logic works.To me what you say is like saying “in practice computers use floating point arithmetic so it’s wrong to say that
+on real numbers is associative”. Floats are messier than reals, but it doesn’t mean that reals don’t exist, or even that reals are irrelevant when reasoning about floats.What is it that you think I’m saying that is ridiculous?
I think your analogy is very apt. The question whether the reals are associative is misplaced in a CS interview, because it is likely to be misinterpreted for a more complex question about floats.
IMO this is the sort of question that gives software engineering interviews a bad rap. I respect that formal logic is part of core CS fundamentals, but acting like it’s necessary to do your day job comes off as didactic.
As the author wrote, the conclusion and interpretation is for the reader.
IMHO, even if this is not the only aspect to check, such knowledge helps to make clean code. How to simplify complex code if you cannot ensure the equivalence of conditions? I often see complex statements because of such lacks of basics (as foundation of house). We cannot be expert in all aspects, but the team needs at least one member with such knowledge
The logical equivalence of the contrapositive is one of the very first things people ever learn about logic..along with the syllogism (a => b, b => c) => (a => c). Besides simplifying code, these are basic tools of reasoning in teenage (or younger) mathematics (and even in life). They apply really anywhere arguments take place, which is to say almost everywhere academic or even politics, etc.
Meanwhile, MVC (and many other design patterns) are more buzzwords for certain separations of concern, code/activity factoring. I would say you could be a great software engineer never having heard of MVC or design patterns. (Proof: There were great software engineers before it was coined! Convince me otherwise!) I am really not so sure you could be a great software engineer or programmer having forgotten basic logical equivalences, however abstract. As the author & Isammoc both write conclusions are for the reader, but this “study” was for some Master’s in SE program. So, besides filtering out weak candidates he was probably also trying to suss out curriculum issues in his program.
One non-objective thing here is that we have to trust that he asked his logic question in not too tricky ways and we only have his word on that, not transcripts or recordings. The equivalence is pretty obvious if you draw a Venn diagram, for example with an A bubble fully inside a B bubble in a set theory analogue. Maybe that gives it away too much or maybe that’s just like expanding the MVC acronym?
I have seen quite expensive bugs in business logic that result from exactly this kind of fallacy.
Oo interesting, are you allowed to divulge more about those bugs?
I remember a bug that followed exactly this pattern:
So the customer was treated as if A were true (it wasn’t, and resulted in customer service complaints.)
My experience interviewing and giving interviews for industry has led me to be wary of exactly the kind of questions in this article.
The quicksort question requires the candidate to have familiarity with O/Theta/Omega notation and knowledge of quicksort is off the top of their head. The interviewer could get a better idea of the candidate’s understanding of complexity by presenting the candidate an implementation of quicksort and asking how they would expect it to perform on small, large, and maliciously constructed inputs. This selects for people with fundamental understanding of performance concerns, selects out people who memorized “quicksort is nlogn or n^2 in bad cases” without that understanding, and provides an easy avenue for the candidate to talk about tuning pivot selection based on the expected input domain.
Of course that requires the interviewer to do their share of preparation work as well.
Nearly every candidate could reason out the logic basis problem given time and comfort. Comfort isn’t a thing all interviewees have however; many are uncomfortable in general, but this is massively compounded by personal comfort with the interviewer. Stereotype threat means candidates that cannot mask their differences will, on average, be less comfortable. These types of questions produce a department that trends towards looking and acting superficially like the interview team. Moreover, they do so by impacting interviewee performance, and so cannot be compensated for by interviewers “acknowledging and reducing personal biases”
Since formal notation isn’t something that is likely to be top of mind for many candidates, this could be made more approachable by rephrasing it in concrete examples. e.g. “The undergraduate course CSC202 has a prerequisite of CSC101. Assume the department does not allow testing out of prerequisites, so that taking CSC202 implies a student must have taken CSC101. What can you say for sure if a student hasn’t taken CSC202? If they haven’t taken CSC101? If they haven’t taken either?”.
It’s clear after rephrasing that this question is barely a test of reasoning skills. That’s because it never was, and should probably be either expanded upon or dropped from the interview for time in favor of a more interesting prompt.
Do note however, that he is referring to interviews of CS graduates. That’s a different candidate pool than we typically see when interviewing for industry. I interview (and hire) many people who do not come from a formal CS education.
But when hiring specifically from a CS program, it’s not unreasonable to ask questions about basics that would indicate they at least attended their freshman year classes.
I was fortunate to take a formal logic course years ago as a philosophy student. Even though it was some time ago, the effort it required left an impression and an appreciation for the skills and work involved.
Almost 20 years later, I’ve found myself once again manipulating logical formulae – this time with an eye toward solving problems with SAT/SMT solvers. Problems, I should note, I would not know how to approach otherwise.
Without having had that exposure to formal logic back then, I seriously doubt I would have had the ambition to tackle the topic now from scratch.
Here’s what I think I read in Meyer’s piece: students can really only be expected to be familiar with concepts they’ve encountered in their coursework (at best). Presumably, folks know about MVC because they’ve been exposed to the concept in some form or another somewhere along the road.
“Over the past decade I have seen many computer science departments, including in top institutions, remove the logic course from the curriculum. And MVC is a great concept. But still.”
I read this more as a lament that formal logic is not given the place it deserves in the curriculum and less as a lament about the students’ knowledge per se. None of this strikes me as particularly controversial. Unless I misunderstand and there are folks suggesting that logic is unimportant for a computer science education. Is that the case?
In the opening lines of The Fixation of Belief (1877), C.S. Peirce writes :
“Few persons care to study logic, because everybody conceives himself to be proficient enough in the art of reasoning already. But I observe that this satisfaction is limited to one’s own ratiocination, and does not extend to that of other men.”
In any curriculum! A solid understanding of formal logic benefits any kind of structured thought. It is as important to the humanities as it is to computer science.
Cheers! Consider the choir preached to :)
What kind of model answer do you think he’s expecting here?
Something like “A->B is equivalent to !A v B, so in the case of !A it doesn’t matter what B is, and in the case of !B we’d better have !A”? Or is he expecting someone to mention Modus Ponens and Modus Tollens? I agree this is likely bread and butter for anyone with a theoretical CS / logic background, but I think if you haven’t ever come across/been taught the definition of ‘implies’ it’s really counter-intuitive at first.
I’m guessing he wanted
¬B → ¬Ahere.Oh, guess I misread it as “what can we say about !A, and what can we say about !B”, rather than “what can we say about (!A and !B)” :-)
Yes, I think so. Honestly though, with logic rules like this, if I’ve not seen a particular variant before I’m never 100% sure that I’ve got them right if I try and work them out in my head. I usually end up writing down a small truth table to make sure I’ve got the outcomes correct.
The contrapositive rule, modus tollens. I remember it. I can’t say I’ve actually used it recently. But if I’d gone through this interview I’d have been judged by it. This is the kind of question that un-diversifies your teams by selecting for access to advanced education. Understanding of low level theory is not nearly as important for every teammate to have, compared to qualities like empathy for your customers and users, owning your mistakes, or persistent curiosity. Good candidates can bring those qualities from any number of backgrounds.
Those details from my logic course come in handy in rearranging Boolean expressions to be more readable, notably the one where you distribute a negation and flip and and/or. It’s good to have someone who knows that on the team, who can share that someday at the whiteboard or in code review. But that question doesn’t belong in an interview.
He’s interviewing students for a masters program, not a software team. Questions about CS fundamentals are applicable here.
That is a fine point that I managed to overlook, thanks.