These questions really feel like that XKCD about communicating badly and acting smug about it. (I got most “correct” so I feel entitled to complain)
For example the sanitize question doesn’t mention that the requirement is to sanitize the input. We can guess it may be that and that // may be also something to handle. But it may as well be “upgradeFormatToV2”. The answers seem like they could punish you either way: “think of the other ways to sanitize”, or “don’t make assumptions, who told you it’s sanitizing?”
The enum answer is extremely language specific, because the enum “equivalent” in my mind is something ADT-like that allows the same benefits that the “correct” answer suggests, but is better. (There’s nothing on the page that I could find that tells you this is Java or restricts you to it)
The hash is “which of those insignificant tweaks do you think is better” while leaving out “this is a terrible function which is being too clever, leaks internal information, returns equality as 0 which is confusing unless you’re sorting, and implies constant time comparisons where they don’t actually happen - yeet the whole thing, give the hash its own type if your language allows it, and use equality comparison like a normal person”. None of the tweaks are better, don’t ask me which scent I prefer on a fresh poo.
You know those viral Facebook posts with math equations that use a notation that makes the operation priority ambiguous and claim “most people won’t know the answer” for engagement from the arguing in the comments? Yeah, that’s what this feels like, but for programming.
An old professor during my degree once told me: “If only a few of my students cannot solve what I ask for, that’s fine, and it’s probably on them; now, if the entire class cannot get what I’m asking for, that’s on me.”
I stopped at the first question and analysis. The analysis of the first question assumes certain user requirements that were not made explicit. It had the tone of “gotcha” that in combination with these missing requirements suggested the remaining quiz would be similarly unhelpful.
My interpretation of the comment above is that the quiz is heavily flawed.
I think you’re trying to suggest sharing questions most people don’t get is some kind of moral failing. It might be in a classroom context, but not in others.
I once took an algorithms course with Manuel Blum. On two occasions, he sneaked an unsolved famous problem onto the tests. Naturally, no-one got them.
Assuming you are interested in improving software development/educating software developers the main point to take away would be that you have advertised your solutions as “objectively correct”. This statement would be more defensible if you add clear requirements/constraints to the questions. Even then, development is an art.
As one gets older one thinks “I can’t tell good code, but I sure can tell bad code.” Even this is not universally applicable because some business situations need bad (hastily written, slightly buggy, throwaway) code.
Software is not written to be correct or fast or elegant. It is written to solve a problem for a certain amount of time.
PS: Giving people hard problems is completely fine. It does not teach people much however. These problems are for that famous demographic mentioned by some greek or the other in thier quote about the efficacy of teachers.
The questions in the quiz are not hard problems. They are poorly defined ones.
As one gets older one thinks “I can’t tell good code, but I sure can tell bad code.” Even this is not universally applicable because some business situations need bad (hastily written, slightly buggy, throwaway) code.
And sometimes your perspective may even change, “objectively” bad code can sometimes actually be really really good code written by experts who know what they’re doing. Like for example, intentionally breaching an abstraction to get a performance improvement that cannot be gotten any other way. In other places, this could be a terrible idea (think, e.g. using inline assembly or using inline SQL instead of your framework’s query builder), but judicious use of such abstraction breaches can mean the difference between an application that crawls to a halt and one that runs acceptably fast.
I hear what you’re saying, and know your position makes perfect sense given where you are.
A really funny thing I’ll share:
So I’ve gone through all the comments here twice now, and collected a document of feedback.
A lot of people made sweeping claims about something being gotcha or unclear, but I have the list of all the actual specific comments made.
And the number of specific comments about a question being unclear is actually very small.
And so, actually reading the contents of what people are saying and not the summaries…
….the claim that the questions are “poorly defined” is not really supported by what people are saying.
Or you could say that many are being poorly-defined when they call it “poorly-defined.” The predicted #1 source of communication failures on my list is something that no-one here said, but I guessed that some people were reading it that way by reading between the lines here.
My biggest problem with this quiz was that “use a language that gives you better tools for abstraction than Java” always struck me as the best code improvement to make for every question. Most of the questions had a clause about there being other improvements not named, and the possible improvements I thought of seemed to dwarf the improvements that this quiz wanted to make in importance.
I didn’t like question 3 - the ostensibly correct answer “The best refactored version contains the substring “(140, 200, 200)” or some equivalent. “ strikes me as pretty vague. It was difficult to guess what precisely the authors of the quiz meant by that. I note that the literal substring “(140, 200, 200)” does not appear in the improved code block, and that the change that it does have is using the closest construction Java has to an enumeration type. I’d say the text of the explanation is more consistent with C rather than B being the correct answer, in fact!
I agree that creating an enumeration type that describes all possible tuples of the actual integer values for the states is a good change, but I disagree with the reasoning that this is good because it removes a conditional. It doesn’t actually remove a conditional, it just moves that conditional to somewhere else in the program, the part that decides whether to call setProductDisplayType with one or another variant of ProductDisplayType.
I didn’t like question 4 either - I don’t actually think it’s obvious that “there useful subsets of the program which contain posts but not comments”. This may or may not be true depending on exactly what sort of program you’re writing that has a notion of posts, which we can’t tell from this code snippet. Refactoring the Post class to avoid having a list of Comments seems like a decision with tradeoffs that is exactly equivalent to the three proposed superior design solutions.
In any case, I dislike principles of software engineering that reference classes in a specific way - there’s no law that says your programming language needs to have the concept of a class at all. “(C) is incorrect because not copying the comments list leads to bugs from concurrent modification” also struck me as a bad explanation. This might well be true in Java, but it’s not true in Rust, the language that I currently use at my day job. The compiler would prevent you from accidentally introducing bugs related to concurrent modification, and force you to pick one of several strategies for sharing the comments list, which might or might not involve copying them.
I note that the literal substring “(140, 200, 200)” does not appear in the improved code block
When I tell you that this made me want to commit murder, that is not an exaggeration.
I thought this was referring to somehow storing the different options in arrays or something like that, which has the disadvantage of being much less clear. They literally argued against such approach in the rectangle example.
I was very surprised to see people commenting on this, as the quiz says “contains the substring ‘(140, 200, 200)’ or equivalent.” The code given contains an ordered triple of those three numbers, which is quite equivalent.
It was difficult to guess what precisely the authors of the quiz meant by that.
Hmmm. That’s not where we want people to go.
It’s intended that the vagueness of the answers to Question 3, where you are being asked properties of the solution instead of being offered possible solutions, would force people to first solve the question on their own, and then select the matching answer. It’s sorta like the AIME math competition, which asks for multiple choice answers between 0 and 999, but for questions about more complicated mathematical structures. So the questions tend to look like “The equation describing this system is of the form Ax^2+Bx+C. What is A+B+C?”
The question already reads “Try to come up with the best refactoring. We will not show you possible solutions, but below we test whether you thought of the best one.” So, if it’s getting the reaction that people are still trying to backsolve from the answers instead of forward-solving the question, then I’m at a loss for how to make it any better without entirely leaving the multiple-choice format.
You’ve motivated me to write a more thorough explanation of this answer, by showing me gaps in people’s understanding of the solution. Java has enums, and using them is the wrong choice. And I’ll include a link to this newsletter, which explains how this kind of change actually does eliminate a conditional: http://us16.campaign-archive.com/?u=8b565c97b838125f69e75fb7f&id=f9c7f0e02f
In any case, I dislike principles of software engineering that reference classes in a specific way
Which principle are you interpreting as having something to do with classes?
I’m guessing from context you’re suggesting that the Parnas Subset Criteria discussed in Question 4 has something to do with classes. But the principle was codified in 1979! Classes existed back then, but were not very common.
My biggest problem with this quiz was that “use a language that gives you better tools for abstraction than Java” always struck me as the best code improvement to make for every question.
Heh.
Usually when people say “The biggest improvement is to not use Java,” it’s meant as a joke. But it’s also true, as you point out.
I’ve considered writing a Typescript version of the quiz. It would be exactly the same, but perhaps you’d have a different reaction just because it’s a newer language.
Do you have any example improvements other than “Rewrite the whole thing in Rust?”
In type theory, an enum is any type of the form 1+1+….+1. C enums, Java enums, Typescript enums — all of them are like this. The current input to the function is S("compact")+S("detailed"), where S(x) denotes the singleton type consisting of all values equal to x. That means that changing it to an enum does not actually change the structure of the code.
Java also has a weird construct where enums can be extended to also contain constants, so that you can define, say a MonthWithNumDays type isomorphic to S("January")*S(31)+S("February")*S(28)+...S("December")*S(31) . It is not mentioned in the solution, precisely because this is not a Java quiz.
You’re limiting the scope of possible enums. Some other languages call their sum types enums, so if you wanted to be precise and not Java-specific, that should be unit types.
Very recent phenomenon. (Okay, 10 years; makes me feel old.) Rust and Swift are the only ones I know of that do this. (And I’m old enough to call those newfangled languages.) Enum had the same meaning for 40 years, but they decided it would be easier to change what “enum” means than to teach programmers a new word.
Do you have any example improvements other than “Rewrite the whole thing in Rust?”
Rewriting the whole thing in Scala or Kotlin is potentially tractable for a Java project, and would get you a compact algebraic data type you could stick the integer values of the display type parameters into. Rust isn’t the only good programming language that exists, it just brought a lot of good innovations into the mainstream, that other programming languages can and should adopt as well.
I do think that making a typescript version of this quiz would be helpful for you as the quiz designer to get a sense of what questions you’re asking are overly-grounded in the ways Java specifically encourages you to write software, vs more general software development concerns.
“Incorrect” correct answer: Cache the area/perimeter, because they cannot change after construction and unlucky developers will keep calling and paying for the calculations, burning watts and killing polar bears. You don’t want to kill polar bears, do you?
“Better way to replace single quotes?”
“Correct” answer: “Option 1 because Option 2 is just plain incorrect, even though it works.” Every option is misleading, and the real option of “Option 1 because it’s convenient and maintainable to handle data munging as applications of opaque data transformations” wasn’t given. Needlessly ‘gotcha’.
“Best factored code”
“Incorrect” correct answer: Use a switch instead of an if. Why not something more grandiose? We all know you’re out of the codebase in a couple of years, you have other features that need to ship, and honestly making it painfully obvious for an intern that here is the switch statement they just need to add a clause to–already in source control, already tracked via CI/CD, already able to trivially be debugged via a stack trace–is a win.
Claiming that a better version uses a particular substring is absurd, since maybe the real refactor is referencing site design constants by name or using CSS. In the meantime, just grow the switch statement and go back to reading Lobsters.
“Blog post maintainability”
Correct answer: God helped me we agreed on this, which is that Comments shouldn’t be in a post. Comments are a first-class entity, and hiding them inside a post makes analytics, metrics, cleaning, and migrations just a huge pain in the ass. Enforcing a single Author though is still a data-model mistake.
“XOR”
“Correct” answer: “Move the XORs into two new functions!” Why? If we’re already doing bit twiddling, adding a do_xor function isn’t really going to help maintainability–best keep the mess localized where it’s easier to spot bugs. To wit…
…if I’m not mistaken, the function doesn’t work as advertised–because XOR is commutative, any ordering of concatenated XORs will eventually come to the same answer…and that means that two differently ordered lists with the same values (which is to say: two different lists!) here will have the same output if fed to this function, which is not I think the intent.
I’m but a humble working stiff; I start my standups with a six-pack of beer and put on my programming socks one toe at a time just like anybody else–but I’m telling you, I’m really not sold that this sort of advice is correct for software as it is practiced.
Thank you for sharing this, but I really don’t think these gotcha questions are an indicator of design prowess.
I agree with your comments here. Other than the question of whether blog posts should hold comments, this was really all about purely surface issues, one level up from arguing about tabs vs spaces.
None of this matters at all (except the blog example, and even then it’ll probably work out fine).
Thanks for the feedback. Question for how to interpret it: Is this meant to be your initial reaction upon viewing the questions, or your actual response after viewing the question and reading the discussion. Asking because many of the statements in this comment are already addressed in the discussions. E.g.: Comparing the cost of a cache miss vs. a multiplication, it is far from obvious that caching the area and perimeter saves cycles. E.g.: “do_xor” is not the proposed way to factor out the digest and comparison operations into a function.
But I can say you’ve motivated me to write a stronger defense of the answer to question 3, now that “Everyone who sees the correct solution agrees it’s the best” is no longer true, and that you seem to be misinterpreting “Eliminates the conditional” as the only reason for its superiority. I would also like to translate the question to not be about anything graphical to eliminate the CSS question.
The discussion of performance in the rectangle question feels a lot like empty masturbation about performance of some code that no one has fucking profiled.
Unless you have specific knowledge of the specific context (stack, architecture , compiler, environment, input data) with which some performance critical code will run, most assertions about its performance are completely baseless, and only correct by chance.
Correct! Exactly as explained in the existing discussion of this answer.
As a total tangent, I do believe that, for things this small, it is quite possible to make useful predictions about performance without profiling. But this is a semi-specialized skill, and requires knowing a lot about CPU architecture. Also, your predictions may have caveats like “It will behave this way unless another hardware-level thread on the same core is hogging the multipliers.”
for things this small, it is quite possible to make useful predictions about performance without profiling
Wouldn’t go as far as to say it is impossible, but think it’s a bad habit. Kinda like using go to, there might be valid use cases, but they’re quite rare, so you’re better of avoiding it entirely.
I found myself much more likely to miss a question due to the confusing way an answer was phrased rather than a fundamental disagreement about the reasoning. My reaction was usually “oh, that’s what you were trying to get at with that answer” much more than “huh, I wouldn’t have thought that” or even “I disagree”.
Outside of that issue, I said “should not be copied” for Posts/Comments mostly out of being brain-addled by Rust and interpreting that choice through the lens of ownership semantics and clicked too quickly.
I see you’re a fellow Cantabrigian. (Perhaps we’ve run into each other at Boston Haskell?) The last question is actually inspired by an example shared by a third Cantabrigian.
Once you figure out that the key to correctly answering the questions, it’s easy to get a good score. What’s the key? Pick the option where you add more code, more complexity, more (unnecessary) abstraction, more indirection, and more obstacles for readers to truly understand how a program works.
Yow, the xor code is nonsense. You can’t fix it by refactoring the internals or parameters of the code, because the function doesn’t have the right return type. It should return a boolean: whether the target hash matches the combined input hashes. If it had the correct return type, the next refactoring would be to replace the int hashes with an opaque hash type, like jgit’s, allowing a hash of a size that one would actually use (unless one is using this to manipulate the hashes from Java’s hashCode function, in which case, one probably shouldn’t have this function at all).
One simply never wants the Hamming distance between hashes, because the whole point of hashes is that that distance will be around k/2 unless the inputs are exactly equal. If this were about some other flavor of hash, like a locality-sensitive hash, then the proposed hash-combining code would be buggy.
Agree with all of your points, but IMO the more fundamental issue with the code is that XOR (as used) doesn’t produce digests/hashes that are comparable.
I don’t know what you mean by “comparable”. It’s a common technique for combining hashes for hashCode, but otherwise not usually a good technique for combining hashes. But to solve that problem, you don’t add another layer of abstraction. You just solve the problem.
If your “fundamental” issue is that it’s a bad way to spell ==, I agree that that is bad. But compareHashes() is also bad.
These sorts of “gotcha” setups and answers geared to a specific theoretical teaching perspective are at best unhelpful. Additionally, they’re quite specific to the implementation language, being “whiteboard Java”.
The “fake real world” examples being used here are only to make things seem relevant, and you’re supposed to be commenting on the class implementation itself, but they also reflect on a poorly-designed ambient environment in which they are imagined to live, which is what the older more jaded of us will waste our time thinking about when seeing them.
And Rectangle should not contain logic about where it is. That belongs in a base class if you’re insisting on doing 1990s textbook Java, or part of the display list.
You seem to be misinterpreting the purpose of my comments. I’m not trying to convince you of anything in such detail, particularly given the opening text about how objectively correct you are and anybody who disagrees is clearly wrong because they haven’t spent their time explaining themselves to you.
For example take the checksum question; you ask the reader to chose between a few meaningless local changes as if one of them is clearly better, when everything about the function is wrong. And no, I will not iterate specific reasons it’s wrong. The literature on what makes a good or bad checksum is decades old.
You made several specific statements that I didn’t understand, and I asked you about them. You responded by changing the goalpost and switching to personal attacks. I can understand you being upset by someone having the audacity to claim they have objective answers, but that in no way justifies this behavior.
You guys know that C++ and Java are bad, right? I worry that you’re missing the forest for the trees, by encouraging folks to think about imperative details of an abstract machine, rather than talking about how the computer transforms data or how the computer’s internal state machines are actually constructed.
What a dumb quiz. This is obviously fishing for a certain kind of result, while pretending it has universal applicability.
On question #1:
the rectangle class is immutable, so you can only change a rectangle by constructing a new one… there is no mention of how you do this, so the design is extremely incomplete and it’s impossible to judge the ergonomics… it also makes the answer about caching confusing
the requirement that points must be passed in as (X,Y) and size as (width,height) means you have to construct a new tuple. Given the explicitly specified use case (GUIs) it is actually extremely likely you will calculate each of these values independently in an unvectorized form, so the packing is unnecessary overhead
I stopped there.
Have these people ever built their own UI toolkit?
I don’t have broad experience with different UI frameworks but I’ve worked plenty with PDFs and websites. x and y are passed in as two integers. I haven’t seen a point class outside a college textbook or one case of Ruby code written by a Java “expert” who was clearly “programmatically challenged”.
Im hoping this is subtle commentary on the egotistical developer archetype (AKA, I’m right and everyone else is an idiot) and not just another example of.
There was some elimination / guess work involved, but I did not hate it too much. Because every day where I read someone else code (or my past own one), I need to guess a lot about the intentions of the author and even the codebase itself based on imcomplete information and attempt to make the best judgement possible. So this was a good exercise imho.
I wrote you an email about Q3 (but going to share again for everyone) where I had trouble differentiating between B and C, because they are not mutually exclusive in certain languages, including the one you have chosen for the quiz.
In Java, one could have an enum with parameters, e.g.
public enum ProductDisplayType {
COMPACT(140, 200, 200), // or introduce the dimension type
DETAILED(500, 800, 600),
;
public final int maxDescriptionLength;
public final int width;
public final int height;
public ProductDisplayType(int maxDescriptionLength, int width, int height) {
// ... assignments
}
}
and reap both the benefits of avoiding conditionals when accessing the parameter, AND pattern matching, exhausting switch statements, etc (= compile time checks, yay) for other use cases.
I think the question is missing an explanation why using an enum with parameters would be worse over a plain class, or needs an improvement to disambiguate B and C.
The solution you shared is quite equivalent to the official solution. Note that it does not “contain an enum {COMPACT, DETAILED} or equivalent” but does contain the exact substring “(140, 200, 200)”. If you write it out using sum, product, and singleton types, you’ll find the solution you shared is quite close to the official solution except for being less open, and not at all isomorphic to the enum. See https://lobste.rs/s/ukoj9o/software_design_quiz#c_an0m2z for some discussion.
I wanted to avoid such Java-isms in the explanation, but there’s room for it in the extra discussion, and it will be added in the next version of the quiz.
Following is code you might find in a distributed system. A digest in this context is a hash of information relating to the cluster. This function is used to simultaneously compare a list of objects (files, messages, etc) against a checksum. It first combines the digests of each object by taking their XOR, and then compares the result to the target checksum, also using XOR.
Honestly curious — where did you get the idea that you can use XOR to produce checksums/digests like this?
From an example submitted by a student of our Advanced Software Design Course, to the question “Give an example of two functions with identical implementations but separate specs.” He works for a company that produces cloud-based engineering and manufacturing software.
Here’s the original:
Nice example taken from actual code (lightly tweaked). Note the implementations of the two functions are identical.
/* returns a ClusterDigest object containing the differences (i.e. bitwise xor)
* between two digests for the same cluster segment */
compare(other: ClusterDigest): ClusterDigest {
return new ClusterDigest(this.digest ^ other.digest);
}
/* Merge two digests for different cluster segments (using bitwise xor) to produce a
* new digest for the union of the two cluster segments */
merge(other: ClusterDigest): ClusterDigest {
return new ClusterDigest(this.digest ^ other.digest);
}
It’s possible that the bitwise XOR of two digests can represent “the difference” between them, for reasonable definitions of “difference”. But I hope it’s clear that the bitwise XOR of two digests doesn’t represent a “merge” or “union” of them, in any useful sense, as it’s trivial to produce collisions.
To get a collision in the sets, you would need to find a collision or near-collision in the hashes of the underlying objects…
…unless you interpret the lists to be ordered, which at least one other commenter did
If I keep the question in its current form, then the word “list” will be replaced with “unordered set.” (The alternatives: either write a different function that uses both the compare() and merge() operations above, or pick something else from our list of student-submitted examples of “Two pieces of identical code that should not be merged.” But this XOR one is my favorite of the examples.)
To get a collision in the sets, you would need to find a collision or near-collision in the hashes of the underlying objects
Would you? XOR loses so much information, I’m not sure. For example given discrete hashes
a = 0b0000_1001
b = 0b0000_0110
c = 0b0000_0101
d = 0b0000_1010
then the set {a, b} would have the same hash as the set {c, d}, as a^b == b^a == c^d == d^c; and the set {b, c} would have the same hash as the set {a, d}, as b^c == c^b == a^d == d^a. That can’t possibly be right, right?
I could be missing something, but I would definitely expect that hash({a, b}) != hash({c, d}). This property is achieved via e.g. sha256, for example.
I think you’ve mostly shown that finding collisions in 4-bit numbers is easy, which it is.
But for different reasons, I think it actually is the case that finding collisions is easy.
Not because XOR loses information.
But because finding a subset of a list of bitmaps whose XOR equals a target hash can be reduced to solving a set of linear equations mod 2.
Looking back at the original, it appears to be a non-adversarial context. That’s a piece of missing context that affects whether the code is reasonable, although it doesn’t affect the answer. I believe collisions are still as unlikely to happen accidentally as they are with sha256.
I have more feelings about this, but I’ll focus on 1, from the xor question: extracting 1 line of code into a function, and more, THE SAME line of code into 2 DIFFERENT functions, needs a much better justification. I feel like I can die on this hill.
You’re trying to conquer my hill by going down it, though. My contention is that extracting functions purely for semantic meaning is a dangerous path. If the complexity of the new function and the code around is not taking into account, you end with infinite indirection.
In the particular example in the quizz, I see no justification to reduce the original function any further.
Absolutely not trolling or a mistake. The quiz says “contains the substring ‘(140, 200, 200)’ or equivalent.” “(140, dimension(200, 200))” is equivalent to “(140, 200, 200)”.
My contention is that extracting functions purely for semantic meaning is a dangerous path. If the complexity of the new function and the code around is not taking into account, you end with infinite indirection.
Oh. I thought you were saying that it should be 1 function rather than two, not that it should be 0.
I think you’re thinking of a different kind of extraction. This is a case where it’s quite justified: there is two abstract operations with certain properties, whose implementations involve some technical math which is quite far from the knowledge needed for the code in the rest of the function, whose uses are less obvious than the rest of the code, and which may change. Perfect candidate.
The kind of wanton code extraction that Bob Martin and Martin Fowler do is a different beast, and quite dangerous.
I’ll quote from my Software Design Glossary, entry on “indirection.”
Many criticisms of “abstraction” are based on confusing abstraction with indirection. As the definition of abstraction shows, indirection is not abstraction.
Indirection is a degenerate a form of abstraction. For one function to abstract another, it must export a subset of the functionality, with information hidden. With indirection, the meaning of one function is defined in terms of another, which may yet be defined in terms of a third. However, the specs of each are interderivable. No details are hidden from either caller or callee.
For example, suppose some program P uses a function A, which is defined as a single call to B, which is defined as a single call to C. This is pure indirection if the spec of A must change upon any change to the spec of C (lack of existential modularity of C), and if a change in the demands placed upon A will also result in a change to C (lack of universal modularity of C).
Chains of method calls are also often used in code to express one function as a special case of a more complicated one. For example,: “computeFoo(x)” may be an alias for “computeFooGeneral(x, DEFAULT_STRATEGY, depth = DEFAULT_DEPTH)”. This is distinct from the indirection discussed here because computeFoo actually does have a simpler description. The spec of computeFoo will be a refinement of that of computeFooGeneral, given by instantiating variables in its spec to concrete values.
whose implementations involve some technical math which is quite far from the knowledge needed for the code in the rest of the function, whose uses are less obvious than the rest of the code, and which may change
I don’t think the question description makes that case we’ll enough. Without extra context that makes this actually necessary, like for instance, a requirement that the xor operation might configurable, I’d still reject this change in a code review.
The kind of wanton code extraction that Bob Martin and Martin Fowler do is a different beast
Not that I am an expert or anything, but I don’t think this is a fair to Martin Fowler, I find his advices/insights to be often useful.
I’ve only read Patterns of Enterprise Architecture and some of his blog posts, but for the most part I enjoy them. There are some specific pieces of advice that I disagree with in strong terms. In particular, the glossary entry I cited quotes the RevenueRecognition example shown in the Domain Model section of that book, with a screenshot of Figure 2.2 that I can’t post here. It’s a terrible example of indirection that does not create useful abstraction.
Terrible quiz. I got 3/5, with one answer “accidentally correct” and one “accidentally incorrect”:
On the first one, I thought the idea of “two arguments to the constructor” was to leave out x, y so that the same rectangle can be rendered at different places on the screen. So I got it correct accidentally. But reading the answer makes sense and I agree that it’d be better. If the answers had better wording I’d have chosen that (but the fact I didn’t come up with it myself is telling). However, an even better answer might be “use keyword arguments”. Even with Point and Dims classes, you can still mix up the horizontal and vertical directions to the constructors of those classes.
On the second question, I understood the problem at a deep level (having spent a lot of time pondering security), but none of the answers seemed to map to the thought that perhaps you’d want to have the flexibility to change the sanitation to also escape backslashes. “Option 1 because Option 2 is just plain incorrect, even though it works” was like the last way I’d word something like that. OTOH, if you worded it in a way that would be more clearly right, the answer would be given away.
On the third question, I went with the enum because the “correct” answer depends a lot on context. If that’s the only place the sizes are plugged in, it makes more sense to leave the sizes inline and introduce an enum so that at least the caller can’t accidentally pass in the wrong type. It also allows other kinds of dispatching elsewhere in the code, like what details to actually render depending on the display type, which might be a lot more code that would benefit from the enum improvement than merely setting the dimensions. The proposed option doesn’t allow that. It depends heavily on the rest of the code which of these choices is better.
The “posts” question was just weird. I chose the “correct” answer because the other answers made even less sense. Although that last answer (a post should not be required to have an author) could be correct in a system where a user can deregister.
The digests question was “obviously good design” if you were going by a textbook and answering a teacher’s questions on an exam, but changing the return type to boolean would be the one obviously best change in practice (which isn’t offered). Also, extracting the xor operations into other methods is IMO an unnecessary abstraction in this particular case. There are only two xors in there, and moving them to helpers won’t improve the maintainability all that much, and is also likely to introduce a performance hit.
All in all, pretty bad quiz design which is aggravated by all the smugness about it. The whole “less than 99% can get these” should be changed to say “less than 99% can find exactly the one issue the authors had in mind”.
This quiz lays bare the author’s cognitive biases with regards to code organization, and uses a set of questions who’s answers map to the biases instead of some universal “truth” that the author wants to point to.
The ideas behind the questions and the answers are great. In that sense, I got all 5 right. In reality, I got only 3 right because in one of the remaining ones I couldn’t figure out how to map my answer to an option (although the answer I came up with matched the answer given by the explanation) and in another one the answer was so poorly worded it just didn’t seem like it could ever possibly be the right answer except once you saw the explanation.
I liked the spirit of the quiz but I think the questions and/or answers were too vague. E.g. the one where one of the answers was “a change like (140, 200, 200)”, I think it would have been better to be explicit about what the code changes would be. In other words, “using a container object instead of discriminating on an input string to choose the relevant values.” When the questions were vague, it diminished the significance of the results of the quiz.
Possible source of false negatives: I agreed completely on their “right solution” to question three and indeed would have suggested the same but disagreed with their characterisation of it in the button text, so I ended up clicking the wrong button.
Fairly good quiz, and I’m always a fan of trying to capture and explicate what it is experts do. It would be interesting to try the quiz on people I know at various skill levels to see how strong an indicator it is.
I’m a little skeptical because I (only 8 years into my career) got them all right with no real difficulty. I’m wondering to what extent they’re really difficult and to what extent they pretend they are difficult to go viral when people feel special for doing well at it!
Some of these questions I’ve tried on a lot of people.
I don’t know the real overall numbers, as in different environments I see wildly different success rates.
E.g.: I first asked Question 2 at my Strange Loop talk, to a room of about 200 people. (I just asked Option 1 vs. Option 2, not broken down by reason.) Over 90% picked Option 1.
More recently, I asked the same question to a room of about 20 people at a workshop at a well-known cloud computing company. This time, it was a 50-50 split.
Question 3 we ask as a pre-test before our course (as a free response question, not multiple choice). Between 20% and 60% get it, varying each month.
Asked it on Reddit. Out of around 90 commenters, only 2 suggested the right answer, with many saying “Why does it need to be refactored at all?”
Hello everyone! Thanks for giving the quiz a good honest try. I know I ruffled some feathers, but take pleasure in hearing that many of you found the explanations quite educational, as well as from private commenters who were more praising than the average here. I found many of the comments here earnest, thoughtful, and well-written, and a few made quite successful attempts at humor.
I’ve now gone through all the comments to get a list of specific feedbacks. Like anyone building a thing for lots of people, I’ll be triaging them to decide how best to improve the next version. Thank you for helping me improve the world of software engineering!
Hi Lobste.rs! Excited to share the software engineering quiz we’ve been working on.
I’ve been writing and teaching about expert topics in software design for a long time; you might have seen some of them here. This is my attempt to condense many of these ideas into a small interactive format that can produce a sense of “Wow, there’s a lot of deep ideas I don’t know!”
The quiz is very short, but we’ve put a lot of work into getting a broad range of ideas into just 5 questions, and also making the correct answers ironclad in spite of the minimal context, and also trying to preemptively answer every objection that comes up, including (and especially) the idea that there are no objective answers in software design.
Is your reaction “Wow, this is interesting” or “Gawd, these guys are such know-it-alls”? Excited for your feedback!
I think the quiz is frustrating because it’s being deliberately obtuse. The “correct” choice is often worded in some manner that is tangential to the problem and only makes sense if the test taker can read your mind.
Take the very first question: “What is a design issue with this class?” The correct answer is “A) The constructor should take two parameters, not four” which is a very, very weird way to say “You should use the type system to make it harder to pass the parameters incorrectly.” The issue is not that there are four parameters, that is completely beside the point. It really feels like you’re going for a “gotcha!” moment the way the questions are worded.
I figured that this was the answer by process of elimination, but I agree that the wording is weird.
What I found annoying about this particular question is that it says “two instead of four”. Why limit ourselves to only two extra types (/s)? If a programmer could mess up the order of x, y, width, and height, they could also mess up the order of x and y when creating a point or width and height when creating a dimension! In that case, maybe what we really want is to create the types HorizontalPosition, VerticalPosition, Width, and Height. We could use those to create a position and dimension, but now that all components have different types, maybe our four-argument Rectangle constructor isn’t so bad since it’s impossible to mess up the order.
Similarly, the methods area and perimeter both return an int even though they are different kinds of measures. Surely if a programmer can mess up the order of the parameters of a rectangle, they can most certainly use a perimeter where an area is needed, so we should protect them by introducing more types, e.g., Perimeter, Area, Volume, etc.
I’m being intentionally cheeky because I don’t believe in universal answers to programming design problems. If the users of the Rectangle class are used to creating rectangles with x, y, width, height, then having a pos, dim constructor would create unnecessary friction for them. If the system is to deal with a huge number of rectangles, the idea of storing rectangles in arrays (either row-ordered or column-ordered) could be a better design than a Rectangle class. Let’s not automatically create extra abstractions because some high-level principle says we should.
Take the very first question: “What is a design issue with this class?” The correct answer is “A) The constructor should take two parameters, not four” which is a very, very weird way to say “You should use the type system to make it harder to pass the parameters incorrectly.” The issue is not that there are four parameters, that is completely beside the point. It really feels like you’re going for a “gotcha!” moment the way the questions are worded.
Hi Relax,
Would it help if the wording of the question was changed to “Which of the following changes would fix the design issue in this class?”
If the answer was phrased “Use the type system to make it harder to pass the parameters incorrectly,” that would give away the answer. Writing plausible distractors is tough, and writing ones that look as plausibly correct as the “Use the type system…” is toughest.
Originally I wanted to make the quiz open-ended where it would ask for a free response, but, implementation cost aside, that would make it much more cognitively demanding on the quiz-taker.
The problem is that there isn’t any single “design issue” with the class that needs to be fixed, in the general case. There are lots of issues with the code, all with different impact and relevance depending on details of context which aren’t part of the question as stated. For example, there are plenty of situations where not caching the area/perimeter calculations represents a far larger design issue than the parameters accepted by the constructor.
It might help to reframe the question as “Identify a footgun for the end user of this class”, because it may be easier to write distractors for that without tipping your hand entirely.
Q: Identify the most significant footgun for the end user of this class.
A. The arguments must be passed in in a certain order.
B. The getArea and getPerimeter functions recompute it each time
C. Oh dear, I’m having trouble coming up with more distractors thanks to the narrower scope
I see your point, and it is a tricky one pedagogically. I think the issue we all bump into here is that you’re asking folks to skip the step of identifying the problem and to jump straight to evaluating the best solution–and as you’ve seen here, a lot of people disagree with the diagnosis of the problem.
One classic technique is to take the correct answer and create a couple variants that are just a bit wrong: for example here, offering to have three arguments (anchor, width, height presumably) or having a single argument that’s just a map, or named arguments ala Python.
I’m not really sure that’s much of an improvement. It might be better to ask the taker, “What is the biggest problem you see with this class?” and adjust the answers accordingly.
Multiple choice tests are tough to do right – in avoiding giving away the answer you’ve gone completely the other direction. If that’s what you’re going for, great, but just realize it makes for a frustrating experience for the taker.
I think the quiz is frustrating because it’s being deliberately obtuse. The “correct” choice is often worded in some manner that is tangential to the problem and only makes sense if the test taker can read your mind.
I’ve re-read the quiz with this lens in mind.
I can see that being a fair criticism of questions 3 and 4. (I have some ideas for how to improve question 3; less so for question 4.) Are you intending the criticism to apply to any of the others?
I know you cited question 1 as an example, but I don’t find that a fair cop, because the discussion explains why, yes, it really should take two parameters even if passing parameters incorrectly was not an issue. For example, if you see a C function that takes a char* and an integer for its length, hopefully we’d be agreed it would be better off taking a proper string type, even though argument order is not an issue.
This might be an argument in favor of making the quiz longer, trying to find questions that only rely on one point, as part of the lesson seems to have been lost here.
I’d say it applies to question 2 as well, although there the answer is somewhat “obvious” through the process of elimination.
Definitely question 5 too – it feels like a simplified version of something more interesting but we’re supposed to know how the more real version should be refactored.
The answer to 3 is kind of ironic considering it’s avoiding the type system solution style of question 1.
Anywho, I know this took a lot of work and is probably hard to receive criticism, so I really respect that you’re listening and trying to make it better all around.
Saying these all have objective answers, and then justifying these objective answers with one person’s courses and strangeloop talks, makes it hard for me to accept these are objective.
Just looking at the first question, I’ve lived through situations where all the answers are legitimate concerns, with the constructor being the least of them. So to say that the answer is objective is, frankly, objectively wrong.
Can you clarify about the other situations? Are you saying that you’ve lived through situations where e.g.: caching
or changing an int to a double was required, or are you specifically saying you have an example situation which is closer to the question? If so, can you elaborate?
Here are the choices presented to me for the question on a Rectange class.
The constructor should take two parameters, not four
The application should pass around arrays of integers instead of the Rectangle type to avoid the overhead of classes
Rectangle should be made to be an implementation of an IRectangle interface
The class should cache the area and perimeter instead of computing them anew each time
The “correct” answer, as described in the quiz, is 1. This is a reasonable thing to do, but not objectively better than the other three because there simply isn’t enough information to make an informed choice.
First, there is no information at all about an IRectangle interface in the question. Does this make it a bad design decision? As the reader, I have no idea. For the sake of argument, let’s discount it entirely as an answer because of the lack of information. But I have been in situations where, say, the framework I’m working in requires the use of an interface, or perhaps coding standards do. Is the framework or coding standard a good design? It doesn’t matter because by not following it or attempting to subvert it in the name of supposed objective truth you are creating either massive churn or political upheaval. If such a situation applies, then making sure it follows the interface requirement is more important than the constructor.
How about passing around arrays of integers? This is absolutely more important than the number of arguments to the constructor if performance really matters. Game programming is filled with these kinds of data-oriented design decisions because accessing flat arrays of raw data is considerably faster than chasing pointers. And then, maybe you want a four argument constructor (or even more!) because regularly computing the values uses too many cycles compared just looking it up. This also covers the other case about caching the area and perimeter: you may even pre-compute them in the constructor. I have applied this approach many times for highly utilized data structures.
At best, this question could be said to be “objectively correct” for the incredibly narrow use case that is presented. But to generalize and imply that other decisions are incorrect is itself incorrect. There is plenty of experience out there to refute it. Would I typically use a two argument constructor for a rectangle class? Very likely. Will I always? Don’t count on it. The same can be said for the rest of the quiz. I can’t regard this quiz as anything useful and would not recommend it to anyone studying software engineering.
You may have noticed this question has been changed since last time you took it. The IRectangle answer replaces the “change int to double answer” (partially for consistency with the upcoming Typescript version, where int and double types don’t exist). The question now only asks “Which of the following is mostly likely to be a design improvement to this class,” and it sounds like you’re in agreement.
OK, after doing the whole thing I feel more positive. I mostly agree with the principles. Things that irk me a bit are the way that the quiz presents itself as being the ultimate source of truth. If you change the tone of the quiz to be more about “thinking about code architecture” instead of “applying design principles” my opinion would be much more favourable.
Feedback, a bit more structured:
The first question feels contrived. There is no context, and in my opinion a design is good when it works well in the context that it’s in. Without context, this is a simple implementation, which is good. Complicating it with introducing classes/structs for the dimensions and position seems bad to me.
The whole premise of being objective is flawed, in my opinion. Different “deep principles” are often at odds. For example, in the first question, the principle “keep it simple” is directly at odds with the “English language test” (which I have never heard of). I like using a single design principle which is called “use common sense”.
The answers are very verbose, because you are essentially convincing someone your opinion is right.
The whole “99% of software engineers get this wrong” thing… The kindest way to put it is “I wouldn’t do that”.
That said, the questions were very confusing and lacked necessary context. It felt like a game of “guessing what the teacher wants”. Sometimes I could guess. When I missed it was always a tossup between the correct answer and some other answer that would be correct in another context.
Two hallmarks of good code by any standard are clear communication and avoiding ambiguity, which this quiz does not achieve. Indeed, I think every single question had some amount of ambiguity:
Question 1: It’s a GUI environment – perhaps one where millions of rectangles are being drawn. How do I know the lack of caching isn’t relevant?
Question 2: This one was easier to guess right, but still the phrase “is just plain incorrect, even though it always works” threw me off.
Question 3: Correctly guessed that we needed some version of a “type” for the config, and just assumed that’s what you were getting at with “enum”. The phrase “contains the substring “(140, 200, 200)” or some equivalent” just seemed odd and I don’t think you can reasonably assume people will connect that with “a phrase you must use when constructing the type you need”.
Question 4: How do I know that the copying of what could be thousands of comments on thousands of posts at scale won’t be a problem? How do I know that concurrent modification will be an issue? Maybe the comments are read only? Again, the intended answer is a perfectly reasonable point – my problem is that there are other reasonable points here as well.
Question 5: Often inlining is the best solution. Especially when the logic is only used in one place. How are we to know it’s not? Making functions for everything, while (yes) providing the encapsulation you mention in the explanation, can also make a code base much harder to read. Depending on the situation, I might think the two function solution was ideal, and I might think it was a clear mistake.
Hey! I liked that the questions do touch on concepts I think are fundamental to software design, and force you to think about all of them in concert (and decide which supersede which on a case-by-case basis). I’ve learned from the explanations, as well as validated my own understanding.
With that said, and even keeping in mind your defense about objectivity both in the quiz and in this comment, I’d like to comment: please reconsider the use of the word “objective”. A subjective question (one based on personal experience) does not change its subjective nature regardless of how well-founded an answer is, or how unanimous people are in answering the question. One doesn’t necessarily need to take away merit from an argument just because it’s subjective, if that helps relieve the pressure of using “objective”.
Subjective questions are typically operationalized, that is, an operational definition is given which can be used as an objective question. E.g. the subjective “what soda brand is best” could be given an operational definition of “what soda brand will be voted the most when a population is asked ‘what soda brand is best’” – the answer is literally the count of votes, regardless of how people reasoned out their vote. If we think the operational definition is well-posed, we can try to form some answer to the original subjective question, but it doesn’t mean the original question is now objective – we simply have a proxy we can objectively answer.
I feel the same about “which code is best”-style questions (although the entire body of software design is so large that I’ll admit I can’t – and wouldn’t dare to – prove it to you from first principles that it is subjective). Are we sure we’re answering “which code is best”? Or are we answering an objective proxy, such as “which code patterns are employed by successful software teams, where success is measured as X” for some operational definition of success (which btw I don’t even think the quiz attempts to do, except possibly in a very implicit way)? Answers to the latter can be valuable I think, of course, but they don’t change the nature of the original question.
This absolutely isn’t meant to take away merit from the principles behind the quiz. I think there are deep principles behind it, and I think software designers should take the time to digest this instead of reflexively and defensively disagreeing if they get it wrong. Cynical takes would be “this is not objective, therefore this is wrong” or “this is subjective, so I’ll only listen to it if I agree”, both of which I wouldn’t condone. But I think the world needs more precision and nuance around language, not less – there’s already enough confusion between fact and opinion, subjective and objective.
IMO don’t just normalize “this is objective, therefore listen to me”; also normalize “subjective questions are okay; listen to this subjective answer because the principles are well-constructed”.
Then again, this entire piece (as you can probably tell from my constant “I think” hedging) is itself subjective, so take that as you will :)
These answers and the way they’re framed show very little regard for the subjectivity of context or much justification for the “objectively” correct answer other than software design principles that are just some other guys’ opinions: in other words, for different ways of knowing.
For example, you claim in the XOR explanation that the correct answer is the one in which the programmer does not have to think as much. Now, I agree with reducing mental load where possible and practical, but what even constitutes mental load is contextual, and so is the applicability of the principle. Sometimes you want the gory details spelled out in front of you, and sometimes abstracting a one-liner out to a function (let alone two) is just silly.
Or for the rectangle one, I disagree that having more constructs (Point, Dimension) is universally better. That means when I want to construct a Rectangle I first have to construct two other Things first, which could be less desirable for performance reasons, and also just strikes me as gross.
So maybe this makes me the first person to dispute that your answers are better, let alone objectively correct. Somehow, I doubt it.
Calling any of this “objective” left a foul taste in my mouth. def new_rectangle(x, y, w, h) has been fine for the past 300,000 years of human existence.
These questions really feel like that XKCD about communicating badly and acting smug about it. (I got most “correct” so I feel entitled to complain)
For example the sanitize question doesn’t mention that the requirement is to sanitize the input. We can guess it may be that and that // may be also something to handle. But it may as well be “upgradeFormatToV2”. The answers seem like they could punish you either way: “think of the other ways to sanitize”, or “don’t make assumptions, who told you it’s sanitizing?”
The enum answer is extremely language specific, because the enum “equivalent” in my mind is something ADT-like that allows the same benefits that the “correct” answer suggests, but is better. (There’s nothing on the page that I could find that tells you this is Java or restricts you to it)
The hash is “which of those insignificant tweaks do you think is better” while leaving out “this is a terrible function which is being too clever, leaks internal information, returns equality as 0 which is confusing unless you’re sorting, and implies constant time comparisons where they don’t actually happen - yeet the whole thing, give the hash its own type if your language allows it, and use equality comparison like a normal person”. None of the tweaks are better, don’t ask me which scent I prefer on a fresh poo.
You know those viral Facebook posts with math equations that use a notation that makes the operation priority ambiguous and claim “most people won’t know the answer” for engagement from the arguing in the comments? Yeah, that’s what this feels like, but for programming.
An old professor during my degree once told me: “If only a few of my students cannot solve what I ask for, that’s fine, and it’s probably on them; now, if the entire class cannot get what I’m asking for, that’s on me.”
A variation of, if all the players are playing badly, we change the coach.
How is that relevant here?
I stopped at the first question and analysis. The analysis of the first question assumes certain user requirements that were not made explicit. It had the tone of “gotcha” that in combination with these missing requirements suggested the remaining quiz would be similarly unhelpful.
My interpretation of the comment above is that the quiz is heavily flawed.
I think you’re trying to suggest sharing questions most people don’t get is some kind of moral failing. It might be in a classroom context, but not in others.
I once took an algorithms course with Manuel Blum. On two occasions, he sneaked an unsolved famous problem onto the tests. Naturally, no-one got them.
I see you are the author of this article. I think the current top comment (https://lobste.rs/s/ukoj9o/software_design_quiz#c_5zb0wx) is direct but accurate.
Assuming you are interested in improving software development/educating software developers the main point to take away would be that you have advertised your solutions as “objectively correct”. This statement would be more defensible if you add clear requirements/constraints to the questions. Even then, development is an art.
As one gets older one thinks “I can’t tell good code, but I sure can tell bad code.” Even this is not universally applicable because some business situations need bad (hastily written, slightly buggy, throwaway) code.
Software is not written to be correct or fast or elegant. It is written to solve a problem for a certain amount of time.
PS: Giving people hard problems is completely fine. It does not teach people much however. These problems are for that famous demographic mentioned by some greek or the other in thier quote about the efficacy of teachers.
The questions in the quiz are not hard problems. They are poorly defined ones.
And sometimes your perspective may even change, “objectively” bad code can sometimes actually be really really good code written by experts who know what they’re doing. Like for example, intentionally breaching an abstraction to get a performance improvement that cannot be gotten any other way. In other places, this could be a terrible idea (think, e.g. using inline assembly or using inline SQL instead of your framework’s query builder), but judicious use of such abstraction breaches can mean the difference between an application that crawls to a halt and one that runs acceptably fast.
Hi kghose!
I hear what you’re saying, and know your position makes perfect sense given where you are.
A really funny thing I’ll share:
So I’ve gone through all the comments here twice now, and collected a document of feedback.
A lot of people made sweeping claims about something being gotcha or unclear, but I have the list of all the actual specific comments made.
And the number of specific comments about a question being unclear is actually very small.
And so, actually reading the contents of what people are saying and not the summaries…
….the claim that the questions are “poorly defined” is not really supported by what people are saying.
Or you could say that many are being poorly-defined when they call it “poorly-defined.” The predicted #1 source of communication failures on my list is something that no-one here said, but I guessed that some people were reading it that way by reading between the lines here.
My biggest problem with this quiz was that “use a language that gives you better tools for abstraction than Java” always struck me as the best code improvement to make for every question. Most of the questions had a clause about there being other improvements not named, and the possible improvements I thought of seemed to dwarf the improvements that this quiz wanted to make in importance.
I didn’t like question 3 - the ostensibly correct answer “The best refactored version contains the substring “(140, 200, 200)” or some equivalent. “ strikes me as pretty vague. It was difficult to guess what precisely the authors of the quiz meant by that. I note that the literal substring “(140, 200, 200)” does not appear in the improved code block, and that the change that it does have is using the closest construction Java has to an enumeration type. I’d say the text of the explanation is more consistent with C rather than B being the correct answer, in fact!
I agree that creating an enumeration type that describes all possible tuples of the actual integer values for the states is a good change, but I disagree with the reasoning that this is good because it removes a conditional. It doesn’t actually remove a conditional, it just moves that conditional to somewhere else in the program, the part that decides whether to call
setProductDisplayTypewith one or another variant ofProductDisplayType.I didn’t like question 4 either - I don’t actually think it’s obvious that “there useful subsets of the program which contain posts but not comments”. This may or may not be true depending on exactly what sort of program you’re writing that has a notion of posts, which we can’t tell from this code snippet. Refactoring the
Postclass to avoid having a list ofComments seems like a decision with tradeoffs that is exactly equivalent to the three proposed superior design solutions.In any case, I dislike principles of software engineering that reference classes in a specific way - there’s no law that says your programming language needs to have the concept of a class at all. “(C) is incorrect because not copying the comments list leads to bugs from concurrent modification” also struck me as a bad explanation. This might well be true in Java, but it’s not true in Rust, the language that I currently use at my day job. The compiler would prevent you from accidentally introducing bugs related to concurrent modification, and force you to pick one of several strategies for sharing the comments list, which might or might not involve copying them.
When I tell you that this made me want to commit murder, that is not an exaggeration.
I thought this was referring to somehow storing the different options in arrays or something like that, which has the disadvantage of being much less clear. They literally argued against such approach in the rectangle example.
I was very surprised to see people commenting on this, as the quiz says “contains the substring ‘(140, 200, 200)’ or equivalent.” The code given contains an ordered triple of those three numbers, which is quite equivalent.
“Contains the substring” implies an exact match. I mean, we’re all programmers here, it’s not exactly surprising we think in programming terms.
If a programming language returned true when you executed
"subway sandwich string".contains("substring"), we would all call it insane.Hmmm. That’s not where we want people to go.
It’s intended that the vagueness of the answers to Question 3, where you are being asked properties of the solution instead of being offered possible solutions, would force people to first solve the question on their own, and then select the matching answer. It’s sorta like the AIME math competition, which asks for multiple choice answers between 0 and 999, but for questions about more complicated mathematical structures. So the questions tend to look like “The equation describing this system is of the form Ax^2+Bx+C. What is A+B+C?”
The question already reads “Try to come up with the best refactoring. We will not show you possible solutions, but below we test whether you thought of the best one.” So, if it’s getting the reaction that people are still trying to backsolve from the answers instead of forward-solving the question, then I’m at a loss for how to make it any better without entirely leaving the multiple-choice format.
You’ve motivated me to write a more thorough explanation of this answer, by showing me gaps in people’s understanding of the solution. Java has enums, and using them is the wrong choice. And I’ll include a link to this newsletter, which explains how this kind of change actually does eliminate a conditional: http://us16.campaign-archive.com/?u=8b565c97b838125f69e75fb7f&id=f9c7f0e02f
Which principle are you interpreting as having something to do with classes?
I’m guessing from context you’re suggesting that the Parnas Subset Criteria discussed in Question 4 has something to do with classes. But the principle was codified in 1979! Classes existed back then, but were not very common.
Heh.
Usually when people say “The biggest improvement is to not use Java,” it’s meant as a joke. But it’s also true, as you point out.
I’ve considered writing a Typescript version of the quiz. It would be exactly the same, but perhaps you’d have a different reaction just because it’s a newer language.
Do you have any example improvements other than “Rewrite the whole thing in Rust?”
Is this a software design quiz, or a Java quiz?
In type theory, an enum is any type of the form 1+1+….+1. C enums, Java enums, Typescript enums — all of them are like this. The current input to the function is
S("compact")+S("detailed"), whereS(x)denotes the singleton type consisting of all values equal to x. That means that changing it to an enum does not actually change the structure of the code.Java also has a weird construct where enums can be extended to also contain constants, so that you can define, say a MonthWithNumDays type isomorphic to
S("January")*S(31)+S("February")*S(28)+...S("December")*S(31). It is not mentioned in the solution, precisely because this is not a Java quiz.OK, so, is this a software design quiz, or a type theory quiz?
If this statement was made in jest, it worked. I chuckled.
(More seriously, I’m quite big on extracting software design lessons from type theory. https://www.pathsensitive.com/2021/03/why-programmers-shouldnt-learn-theory.html )
You’re limiting the scope of possible enums. Some other languages call their sum types enums, so if you wanted to be precise and not Java-specific, that should be unit types.
Very recent phenomenon. (Okay, 10 years; makes me feel old.) Rust and Swift are the only ones I know of that do this. (And I’m old enough to call those newfangled languages.) Enum had the same meaning for 40 years, but they decided it would be easier to change what “enum” means than to teach programmers a new word.
It does. Currently you can pass the string
"pineapple"to the method and the code will break. Enums will prevent that.Rewriting the whole thing in Scala or Kotlin is potentially tractable for a Java project, and would get you a compact algebraic data type you could stick the integer values of the display type parameters into. Rust isn’t the only good programming language that exists, it just brought a lot of good innovations into the mainstream, that other programming languages can and should adopt as well.
I do think that making a typescript version of this quiz would be helpful for you as the quiz designer to get a sense of what questions you’re asking are overly-grounded in the ways Java specifically encourages you to write software, vs more general software development concerns.
I think one of the cardinal sins of software engineering has got to be using types to model behavior.
Are you referring to the Church-encoded boolean example used by the blog post that I attack?
But they still are. Everything here still relies on context of the language, business problem, codebase, team, and company you’re working with.
Also, as another commenter points out, much of your solutions add potentially unnecessary code, classes, and abstraction.
Related: https://www.computerenhance.com/p/clean-code-horrible-performance
Reactions.
“Issue with Rectangle class?”“Incorrect” correct answer: Cache the area/perimeter, because they cannot change after construction and unlucky developers will keep calling and paying for the calculations, burning watts and killing polar bears. You don’t want to kill polar bears, do you?
“Better way to replace single quotes?”“Correct” answer: “Option 1 because Option 2 is just plain incorrect, even though it works.” Every option is misleading, and the real option of “Option 1 because it’s convenient and maintainable to handle data munging as applications of opaque data transformations” wasn’t given. Needlessly ‘gotcha’.
“Best factored code”“Incorrect” correct answer: Use a switch instead of an if. Why not something more grandiose? We all know you’re out of the codebase in a couple of years, you have other features that need to ship, and honestly making it painfully obvious for an intern that here is the switch statement they just need to add a clause to–already in source control, already tracked via CI/CD, already able to trivially be debugged via a stack trace–is a win.
Claiming that a better version uses a particular substring is absurd, since maybe the real refactor is referencing site design constants by name or using CSS. In the meantime, just grow the switch statement and go back to reading Lobsters.
“Blog post maintainability”Correct answer: God helped me we agreed on this, which is that Comments shouldn’t be in a post. Comments are a first-class entity, and hiding them inside a post makes analytics, metrics, cleaning, and migrations just a huge pain in the ass. Enforcing a single Author though is still a data-model mistake.
“XOR”“Correct” answer: “Move the XORs into two new functions!” Why? If we’re already doing bit twiddling, adding a
do_xorfunction isn’t really going to help maintainability–best keep the mess localized where it’s easier to spot bugs. To wit……if I’m not mistaken, the function doesn’t work as advertised–because XOR is commutative, any ordering of concatenated XORs will eventually come to the same answer…and that means that two differently ordered lists with the same values (which is to say: two different lists!) here will have the same output if fed to this function, which is not I think the intent.
I’m but a humble working stiff; I start my standups with a six-pack of beer and put on my programming socks one toe at a time just like anybody else–but I’m telling you, I’m really not sold that this sort of advice is correct for software as it is practiced.
Thank you for sharing this, but I really don’t think these gotcha questions are an indicator of design prowess.
I agree with your comments here. Other than the question of whether blog posts should hold comments, this was really all about purely surface issues, one level up from arguing about tabs vs spaces.
None of this matters at all (except the blog example, and even then it’ll probably work out fine).
Hi friendlysock,
Thanks for the feedback. Question for how to interpret it: Is this meant to be your initial reaction upon viewing the questions, or your actual response after viewing the question and reading the discussion. Asking because many of the statements in this comment are already addressed in the discussions. E.g.: Comparing the cost of a cache miss vs. a multiplication, it is far from obvious that caching the area and perimeter saves cycles. E.g.: “do_xor” is not the proposed way to factor out the digest and comparison operations into a function.
But I can say you’ve motivated me to write a stronger defense of the answer to question 3, now that “Everyone who sees the correct solution agrees it’s the best” is no longer true, and that you seem to be misinterpreting “Eliminates the conditional” as the only reason for its superiority. I would also like to translate the question to not be about anything graphical to eliminate the CSS question.
Additionally, the area and perimeter cache takes more memory, so even if computationally free/negligible it’s not an necessarily an improvement
The discussion of performance in the rectangle question feels a lot like empty masturbation about performance of some code that no one has fucking profiled.
Unless you have specific knowledge of the specific context (stack, architecture , compiler, environment, input data) with which some performance critical code will run, most assertions about its performance are completely baseless, and only correct by chance.
Correct! Exactly as explained in the existing discussion of this answer.
As a total tangent, I do believe that, for things this small, it is quite possible to make useful predictions about performance without profiling. But this is a semi-specialized skill, and requires knowing a lot about CPU architecture. Also, your predictions may have caveats like “It will behave this way unless another hardware-level thread on the same core is hogging the multipliers.”
Wouldn’t go as far as to say it is impossible, but think it’s a bad habit. Kinda like using go to, there might be valid use cases, but they’re quite rare, so you’re better of avoiding it entirely.
I found myself much more likely to miss a question due to the confusing way an answer was phrased rather than a fundamental disagreement about the reasoning. My reaction was usually “oh, that’s what you were trying to get at with that answer” much more than “huh, I wouldn’t have thought that” or even “I disagree”.
Outside of that issue, I said “should not be copied” for Posts/Comments mostly out of being brain-addled by Rust and interpreting that choice through the lens of ownership semantics and clicked too quickly.
The last one is my favorite by far.
Glad you enjoyed it, at least partially!
I see you’re a fellow Cantabrigian. (Perhaps we’ve run into each other at Boston Haskell?) The last question is actually inspired by an example shared by a third Cantabrigian.
Somerville these days, but I’ve been to Cambridge Haskell meetups back in the day.
Once you figure out that the key to correctly answering the questions, it’s easy to get a good score. What’s the key? Pick the option where you add more code, more complexity, more (unnecessary) abstraction, more indirection, and more obstacles for readers to truly understand how a program works.
Yow, the xor code is nonsense. You can’t fix it by refactoring the internals or parameters of the code, because the function doesn’t have the right return type. It should return a boolean: whether the target hash matches the combined input hashes. If it had the correct return type, the next refactoring would be to replace the int hashes with an opaque hash type, like jgit’s, allowing a hash of a size that one would actually use (unless one is using this to manipulate the hashes from Java’s hashCode function, in which case, one probably shouldn’t have this function at all).
One simply never wants the Hamming distance between hashes, because the whole point of hashes is that that distance will be around k/2 unless the inputs are exactly equal. If this were about some other flavor of hash, like a locality-sensitive hash, then the proposed hash-combining code would be buggy.
Agree with all of your points, but IMO the more fundamental issue with the code is that XOR (as used) doesn’t produce digests/hashes that are comparable.
I don’t know what you mean by “comparable”. It’s a common technique for combining hashes for
hashCode, but otherwise not usually a good technique for combining hashes. But to solve that problem, you don’t add another layer of abstraction. You just solve the problem.If your “fundamental” issue is that it’s a bad way to spell
==, I agree that that is bad. ButcompareHashes()is also bad.Why am I passing x and y to that rectangle at all? They’re never used.
The “
// Getters omitted” line implies there will be agetX()andgetY()that readxandy.These sorts of “gotcha” setups and answers geared to a specific theoretical teaching perspective are at best unhelpful. Additionally, they’re quite specific to the implementation language, being “whiteboard Java”.
The “fake real world” examples being used here are only to make things seem relevant, and you’re supposed to be commenting on the class implementation itself, but they also reflect on a poorly-designed ambient environment in which they are imagined to live, which is what the older more jaded of us will waste our time thinking about when seeing them.
And
Rectangleshould not contain logic about where it is. That belongs in a base class if you’re insisting on doing 1990s textbook Java, or part of the display list.Hi Sophistifunk,
Can you point to an example of something in the quiz which would be different in, say, TypeScript?
And can you explain more about your argument about the Rectangle? Can you explain what is wrong with, say, the NSRect type? https://developer.apple.com/documentation/foundation/nsrect
You seem to be misinterpreting the purpose of my comments. I’m not trying to convince you of anything in such detail, particularly given the opening text about how objectively correct you are and anybody who disagrees is clearly wrong because they haven’t spent their time explaining themselves to you.
For example take the checksum question; you ask the reader to chose between a few meaningless local changes as if one of them is clearly better, when everything about the function is wrong. And no, I will not iterate specific reasons it’s wrong. The literature on what makes a good or bad checksum is decades old.
You made several specific statements that I didn’t understand, and I asked you about them. You responded by changing the goalpost and switching to personal attacks. I can understand you being upset by someone having the audacity to claim they have objective answers, but that in no way justifies this behavior.
And if you’re going to attack a question for not being real world, then I suggest not picking the only question that actually is real-world. https://lobste.rs/s/ukoj9o/software_design_quiz#c_xdnltq
I made no personal attacks, although claiming I did probably qualifies.
You guys know that C++ and Java are bad, right? I worry that you’re missing the forest for the trees, by encouraging folks to think about imperative details of an abstract machine, rather than talking about how the computer transforms data or how the computer’s internal state machines are actually constructed.
What a dumb quiz. This is obviously fishing for a certain kind of result, while pretending it has universal applicability.
On question #1:
the rectangle class is immutable, so you can only change a rectangle by constructing a new one… there is no mention of how you do this, so the design is extremely incomplete and it’s impossible to judge the ergonomics… it also makes the answer about caching confusing
the requirement that points must be passed in as (X,Y) and size as (width,height) means you have to construct a new tuple. Given the explicitly specified use case (GUIs) it is actually extremely likely you will calculate each of these values independently in an unvectorized form, so the packing is unnecessary overhead
I stopped there.
Have these people ever built their own UI toolkit?
I don’t have broad experience with different UI frameworks but I’ve worked plenty with PDFs and websites. x and y are passed in as two integers. I haven’t seen a point class outside a college textbook or one case of Ruby code written by a Java “expert” who was clearly “programmatically challenged”.
OpenStep (including Cocoa) always passes points as NSPoint structures and passes rectangles as NSRect structures.
A few years of hard labor fixing copy-paste errors in the Siberian 3D graphics work-camps will show you the value of a good
Pointclass.[Comment removed by moderator pushcx: This is not a helpful volunteer mod comment.]
[Comment removed by moderator pushcx: Pruning meta thread.]
[Comment removed by moderator pushcx: Pruning meta thread.]
Im hoping this is subtle commentary on the egotistical developer archetype (AKA, I’m right and everyone else is an idiot) and not just another example of.
This feels like something the author will look back on in ten years and go, “Oh, yeah. Follies of youth, amirite?”
This was good, thank you.
There was some elimination / guess work involved, but I did not hate it too much. Because every day where I read someone else code (or my past own one), I need to guess a lot about the intentions of the author and even the codebase itself based on imcomplete information and attempt to make the best judgement possible. So this was a good exercise imho.
I wrote you an email about Q3 (but going to share again for everyone) where I had trouble differentiating between B and C, because they are not mutually exclusive in certain languages, including the one you have chosen for the quiz.
In Java, one could have an enum with parameters, e.g.
and reap both the benefits of avoiding conditionals when accessing the parameter, AND pattern matching, exhausting switch statements, etc (= compile time checks, yay) for other use cases. I think the question is missing an explanation why using an enum with parameters would be worse over a plain class, or needs an improvement to disambiguate B and C.
Hi Napster,
The solution you shared is quite equivalent to the official solution. Note that it does not “contain an enum
{COMPACT, DETAILED}or equivalent” but does contain the exact substring “(140, 200, 200)”. If you write it out using sum, product, and singleton types, you’ll find the solution you shared is quite close to the official solution except for being less open, and not at all isomorphic to the enum. See https://lobste.rs/s/ukoj9o/software_design_quiz#c_an0m2z for some discussion.I wanted to avoid such Java-isms in the explanation, but there’s room for it in the extra discussion, and it will be added in the next version of the quiz.
BTW, I checked spam and don’t see your E-mail.
Honestly curious — where did you get the idea that you can use XOR to produce checksums/digests like this?
From an example submitted by a student of our Advanced Software Design Course, to the question “Give an example of two functions with identical implementations but separate specs.” He works for a company that produces cloud-based engineering and manufacturing software.
Here’s the original:
It’s possible that the bitwise XOR of two digests can represent “the difference” between them, for reasonable definitions of “difference”. But I hope it’s clear that the bitwise XOR of two digests doesn’t represent a “merge” or “union” of them, in any useful sense, as it’s trivial to produce collisions.
tl;dr: compare possibly broken, merge almost certainly broken
To get a collision in the sets, you would need to find a collision or near-collision in the hashes of the underlying objects…
…unless you interpret the lists to be ordered, which at least one other commenter did
If I keep the question in its current form, then the word “list” will be replaced with “unordered set.” (The alternatives: either write a different function that uses both the compare() and merge() operations above, or pick something else from our list of student-submitted examples of “Two pieces of identical code that should not be merged.” But this XOR one is my favorite of the examples.)
Would you? XOR loses so much information, I’m not sure. For example given discrete hashes
then the set {a, b} would have the same hash as the set {c, d}, as a^b == b^a == c^d == d^c; and the set {b, c} would have the same hash as the set {a, d}, as b^c == c^b == a^d == d^a. That can’t possibly be right, right?
I could be missing something, but I would definitely expect that hash({a, b}) != hash({c, d}). This property is achieved via e.g. sha256, for example.
I think you’ve mostly shown that finding collisions in 4-bit numbers is easy, which it is.
But for different reasons, I think it actually is the case that finding collisions is easy.
Not because XOR loses information.
But because finding a subset of a list of bitmaps whose XOR equals a target hash can be reduced to solving a set of linear equations mod 2.
Looking back at the original, it appears to be a non-adversarial context. That’s a piece of missing context that affects whether the code is reasonable, although it doesn’t affect the answer. I believe collisions are still as unlikely to happen accidentally as they are with sha256.
I have more feelings about this, but I’ll focus on 1, from the xor question: extracting 1 line of code into a function, and more, THE SAME line of code into 2 DIFFERENT functions, needs a much better justification. I feel like I can die on this hill.
Then I will gladly fight you.
I’ll start: if you were to extract out the two XOR operators into a single function, what would you call it?
xor, presumably.You’re trying to conquer my hill by going down it, though. My contention is that extracting functions purely for semantic meaning is a dangerous path. If the complexity of the new function and the code around is not taking into account, you end with infinite indirection.
In the particular example in the quizz, I see no justification to reduce the original function any further.
Update: by the way, if you convince me that this wasn’t trolling or a mistake, I’ll gladly let you have the function thing: https://lobste.rs/s/ukoj9o/software_design_quiz#c_wegalw
Absolutely not trolling or a mistake. The quiz says “contains the substring ‘(140, 200, 200)’ or equivalent.” “(140, dimension(200, 200))” is equivalent to “(140, 200, 200)”.
Oh. I thought you were saying that it should be 1 function rather than two, not that it should be 0.
I think you’re thinking of a different kind of extraction. This is a case where it’s quite justified: there is two abstract operations with certain properties, whose implementations involve some technical math which is quite far from the knowledge needed for the code in the rest of the function, whose uses are less obvious than the rest of the code, and which may change. Perfect candidate.
The kind of wanton code extraction that Bob Martin and Martin Fowler do is a different beast, and quite dangerous.
I’ll quote from my Software Design Glossary, entry on “indirection.”
I don’t think the question description makes that case we’ll enough. Without extra context that makes this actually necessary, like for instance, a requirement that the xor operation might configurable, I’d still reject this change in a code review.
Not that I am an expert or anything, but I don’t think this is a fair to Martin Fowler, I find his advices/insights to be often useful.
I’ve only read Patterns of Enterprise Architecture and some of his blog posts, but for the most part I enjoy them. There are some specific pieces of advice that I disagree with in strong terms. In particular, the glossary entry I cited quotes the RevenueRecognition example shown in the Domain Model section of that book, with a screenshot of Figure 2.2 that I can’t post here. It’s a terrible example of indirection that does not create useful abstraction.
Terrible quiz. I got 3/5, with one answer “accidentally correct” and one “accidentally incorrect”:
On the first one, I thought the idea of “two arguments to the constructor” was to leave out
x, yso that the same rectangle can be rendered at different places on the screen. So I got it correct accidentally. But reading the answer makes sense and I agree that it’d be better. If the answers had better wording I’d have chosen that (but the fact I didn’t come up with it myself is telling). However, an even better answer might be “use keyword arguments”. Even withPointandDimsclasses, you can still mix up the horizontal and vertical directions to the constructors of those classes.On the second question, I understood the problem at a deep level (having spent a lot of time pondering security), but none of the answers seemed to map to the thought that perhaps you’d want to have the flexibility to change the sanitation to also escape backslashes. “Option 1 because Option 2 is just plain incorrect, even though it works” was like the last way I’d word something like that. OTOH, if you worded it in a way that would be more clearly right, the answer would be given away.
On the third question, I went with the enum because the “correct” answer depends a lot on context. If that’s the only place the sizes are plugged in, it makes more sense to leave the sizes inline and introduce an enum so that at least the caller can’t accidentally pass in the wrong
type. It also allows other kinds of dispatching elsewhere in the code, like what details to actually render depending on the display type, which might be a lot more code that would benefit from the enum improvement than merely setting the dimensions. The proposed option doesn’t allow that. It depends heavily on the rest of the code which of these choices is better.The “posts” question was just weird. I chose the “correct” answer because the other answers made even less sense. Although that last answer (a post should not be required to have an author) could be correct in a system where a user can deregister.
The digests question was “obviously good design” if you were going by a textbook and answering a teacher’s questions on an exam, but changing the return type to boolean would be the one obviously best change in practice (which isn’t offered). Also, extracting the xor operations into other methods is IMO an unnecessary abstraction in this particular case. There are only two xors in there, and moving them to helpers won’t improve the maintainability all that much, and is also likely to introduce a performance hit.
All in all, pretty bad quiz design which is aggravated by all the smugness about it. The whole “less than 99% can get these” should be changed to say “less than 99% can find exactly the one issue the authors had in mind”.
This quiz lays bare the author’s cognitive biases with regards to code organization, and uses a set of questions who’s answers map to the biases instead of some universal “truth” that the author wants to point to.
Be better than this clickbait bullshit.
I disagree. This quiz lays bare an entrepreneur’s attempts to drive more business towards his consultancy. Hence why the spam flags are coming out.
Good point!
The ideas behind the questions and the answers are great. In that sense, I got all 5 right. In reality, I got only 3 right because in one of the remaining ones I couldn’t figure out how to map my answer to an option (although the answer I came up with matched the answer given by the explanation) and in another one the answer was so poorly worded it just didn’t seem like it could ever possibly be the right answer except once you saw the explanation.
Also, so unnecessarily smug throughout.
Which were the two that you couldn’t map to an option?
Based on your comment, I’m guessing questions 2 and 3?
I liked the spirit of the quiz but I think the questions and/or answers were too vague. E.g. the one where one of the answers was “a change like (140, 200, 200)”, I think it would have been better to be explicit about what the code changes would be. In other words, “using a container object instead of discriminating on an input string to choose the relevant values.” When the questions were vague, it diminished the significance of the results of the quiz.
Possible source of false negatives: I agreed completely on their “right solution” to question three and indeed would have suggested the same but disagreed with their characterisation of it in the button text, so I ended up clicking the wrong button.
Fairly good quiz, and I’m always a fan of trying to capture and explicate what it is experts do. It would be interesting to try the quiz on people I know at various skill levels to see how strong an indicator it is.
I’m a little skeptical because I (only 8 years into my career) got them all right with no real difficulty. I’m wondering to what extent they’re really difficult and to what extent they pretend they are difficult to go viral when people feel special for doing well at it!
Some of these questions I’ve tried on a lot of people.
I don’t know the real overall numbers, as in different environments I see wildly different success rates.
E.g.: I first asked Question 2 at my Strange Loop talk, to a room of about 200 people. (I just asked Option 1 vs. Option 2, not broken down by reason.) Over 90% picked Option 1.
More recently, I asked the same question to a room of about 20 people at a workshop at a well-known cloud computing company. This time, it was a 50-50 split.
Question 3 we ask as a pre-test before our course (as a free response question, not multiple choice). Between 20% and 60% get it, varying each month.
Asked it on Reddit. Out of around 90 commenters, only 2 suggested the right answer, with many saying “Why does it need to be refactored at all?”
Hello everyone! Thanks for giving the quiz a good honest try. I know I ruffled some feathers, but take pleasure in hearing that many of you found the explanations quite educational, as well as from private commenters who were more praising than the average here. I found many of the comments here earnest, thoughtful, and well-written, and a few made quite successful attempts at humor.
I’ve now gone through all the comments to get a list of specific feedbacks. Like anyone building a thing for lots of people, I’ll be triaging them to decide how best to improve the next version. Thank you for helping me improve the world of software engineering!
Hi Lobste.rs! Excited to share the software engineering quiz we’ve been working on.
I’ve been writing and teaching about expert topics in software design for a long time; you might have seen some of them here. This is my attempt to condense many of these ideas into a small interactive format that can produce a sense of “Wow, there’s a lot of deep ideas I don’t know!”
The quiz is very short, but we’ve put a lot of work into getting a broad range of ideas into just 5 questions, and also making the correct answers ironclad in spite of the minimal context, and also trying to preemptively answer every objection that comes up, including (and especially) the idea that there are no objective answers in software design.
Is your reaction “Wow, this is interesting” or “Gawd, these guys are such know-it-alls”? Excited for your feedback!
I think the quiz is frustrating because it’s being deliberately obtuse. The “correct” choice is often worded in some manner that is tangential to the problem and only makes sense if the test taker can read your mind.
Take the very first question: “What is a design issue with this class?” The correct answer is “A) The constructor should take two parameters, not four” which is a very, very weird way to say “You should use the type system to make it harder to pass the parameters incorrectly.” The issue is not that there are four parameters, that is completely beside the point. It really feels like you’re going for a “gotcha!” moment the way the questions are worded.
I figured that this was the answer by process of elimination, but I agree that the wording is weird.
What I found annoying about this particular question is that it says “two instead of four”. Why limit ourselves to only two extra types (/s)? If a programmer could mess up the order of x, y, width, and height, they could also mess up the order of x and y when creating a point or width and height when creating a dimension! In that case, maybe what we really want is to create the types
HorizontalPosition,VerticalPosition,Width, andHeight. We could use those to create a position and dimension, but now that all components have different types, maybe our four-argument Rectangle constructor isn’t so bad since it’s impossible to mess up the order.Similarly, the methods area and perimeter both return an int even though they are different kinds of measures. Surely if a programmer can mess up the order of the parameters of a rectangle, they can most certainly use a perimeter where an area is needed, so we should protect them by introducing more types, e.g.,
Perimeter,Area,Volume, etc.I’m being intentionally cheeky because I don’t believe in universal answers to programming design problems. If the users of the Rectangle class are used to creating rectangles with
x, y, width, height, then having apos, dimconstructor would create unnecessary friction for them. If the system is to deal with a huge number of rectangles, the idea of storing rectangles in arrays (either row-ordered or column-ordered) could be a better design than a Rectangle class. Let’s not automatically create extra abstractions because some high-level principle says we should.Hi Relax,
Would it help if the wording of the question was changed to “Which of the following changes would fix the design issue in this class?”
If the answer was phrased “Use the type system to make it harder to pass the parameters incorrectly,” that would give away the answer. Writing plausible distractors is tough, and writing ones that look as plausibly correct as the “Use the type system…” is toughest.
Originally I wanted to make the quiz open-ended where it would ask for a free response, but, implementation cost aside, that would make it much more cognitively demanding on the quiz-taker.
The problem is that there isn’t any single “design issue” with the class that needs to be fixed, in the general case. There are lots of issues with the code, all with different impact and relevance depending on details of context which aren’t part of the question as stated. For example, there are plenty of situations where not caching the area/perimeter calculations represents a far larger design issue than the parameters accepted by the constructor.
It might help to reframe the question as “Identify a footgun for the end user of this class”, because it may be easier to write distractors for that without tipping your hand entirely.
Hmmm…so, something like this?
Q: Identify the most significant footgun for the end user of this class.
A. The arguments must be passed in in a certain order. B. The getArea and getPerimeter functions recompute it each time C. Oh dear, I’m having trouble coming up with more distractors thanks to the narrower scope
I see your point, and it is a tricky one pedagogically. I think the issue we all bump into here is that you’re asking folks to skip the step of identifying the problem and to jump straight to evaluating the best solution–and as you’ve seen here, a lot of people disagree with the diagnosis of the problem.
One classic technique is to take the correct answer and create a couple variants that are just a bit wrong: for example here, offering to have three arguments (anchor, width, height presumably) or having a single argument that’s just a map, or named arguments ala Python.
I’m not really sure that’s much of an improvement. It might be better to ask the taker, “What is the biggest problem you see with this class?” and adjust the answers accordingly.
Multiple choice tests are tough to do right – in avoiding giving away the answer you’ve gone completely the other direction. If that’s what you’re going for, great, but just realize it makes for a frustrating experience for the taker.
I’ve re-read the quiz with this lens in mind.
I can see that being a fair criticism of questions 3 and 4. (I have some ideas for how to improve question 3; less so for question 4.) Are you intending the criticism to apply to any of the others?
I know you cited question 1 as an example, but I don’t find that a fair cop, because the discussion explains why, yes, it really should take two parameters even if passing parameters incorrectly was not an issue. For example, if you see a C function that takes a char* and an integer for its length, hopefully we’d be agreed it would be better off taking a proper string type, even though argument order is not an issue.
This might be an argument in favor of making the quiz longer, trying to find questions that only rely on one point, as part of the lesson seems to have been lost here.
I’d say it applies to question 2 as well, although there the answer is somewhat “obvious” through the process of elimination.
Definitely question 5 too – it feels like a simplified version of something more interesting but we’re supposed to know how the more real version should be refactored.
The answer to 3 is kind of ironic considering it’s avoiding the type system solution style of question 1.
Anywho, I know this took a lot of work and is probably hard to receive criticism, so I really respect that you’re listening and trying to make it better all around.
Saying these all have objective answers, and then justifying these objective answers with one person’s courses and strangeloop talks, makes it hard for me to accept these are objective.
Just looking at the first question, I’ve lived through situations where all the answers are legitimate concerns, with the constructor being the least of them. So to say that the answer is objective is, frankly, objectively wrong.
Hi Geoff,
Can you clarify about the other situations? Are you saying that you’ve lived through situations where e.g.: caching or changing an int to a double was required, or are you specifically saying you have an example situation which is closer to the question? If so, can you elaborate?
As much as I do not want to drag this thread out…
Here are the choices presented to me for the question on a Rectange class.
The “correct” answer, as described in the quiz, is 1. This is a reasonable thing to do, but not objectively better than the other three because there simply isn’t enough information to make an informed choice.
First, there is no information at all about an
IRectangleinterface in the question. Does this make it a bad design decision? As the reader, I have no idea. For the sake of argument, let’s discount it entirely as an answer because of the lack of information. But I have been in situations where, say, the framework I’m working in requires the use of an interface, or perhaps coding standards do. Is the framework or coding standard a good design? It doesn’t matter because by not following it or attempting to subvert it in the name of supposed objective truth you are creating either massive churn or political upheaval. If such a situation applies, then making sure it follows the interface requirement is more important than the constructor.How about passing around arrays of integers? This is absolutely more important than the number of arguments to the constructor if performance really matters. Game programming is filled with these kinds of data-oriented design decisions because accessing flat arrays of raw data is considerably faster than chasing pointers. And then, maybe you want a four argument constructor (or even more!) because regularly computing the values uses too many cycles compared just looking it up. This also covers the other case about caching the area and perimeter: you may even pre-compute them in the constructor. I have applied this approach many times for highly utilized data structures.
At best, this question could be said to be “objectively correct” for the incredibly narrow use case that is presented. But to generalize and imply that other decisions are incorrect is itself incorrect. There is plenty of experience out there to refute it. Would I typically use a two argument constructor for a rectangle class? Very likely. Will I always? Don’t count on it. The same can be said for the rest of the quiz. I can’t regard this quiz as anything useful and would not recommend it to anyone studying software engineering.
Hi Geoff,
You may have noticed this question has been changed since last time you took it. The IRectangle answer replaces the “change int to double answer” (partially for consistency with the upcoming Typescript version, where int and double types don’t exist). The question now only asks “Which of the following is mostly likely to be a design improvement to this class,” and it sounds like you’re in agreement.
Definitely the latter. Making up some “deep principle” according to which some answer is correct does not make the answer objective.
OK, after doing the whole thing I feel more positive. I mostly agree with the principles. Things that irk me a bit are the way that the quiz presents itself as being the ultimate source of truth. If you change the tone of the quiz to be more about “thinking about code architecture” instead of “applying design principles” my opinion would be much more favourable.
Feedback, a bit more structured:
Honest feedback, with a little context first:
That said, the questions were very confusing and lacked necessary context. It felt like a game of “guessing what the teacher wants”. Sometimes I could guess. When I missed it was always a tossup between the correct answer and some other answer that would be correct in another context.
Two hallmarks of good code by any standard are clear communication and avoiding ambiguity, which this quiz does not achieve. Indeed, I think every single question had some amount of ambiguity:
Question 1: It’s a GUI environment – perhaps one where millions of rectangles are being drawn. How do I know the lack of caching isn’t relevant?
Question 2: This one was easier to guess right, but still the phrase “is just plain incorrect, even though it always works” threw me off.
Question 3: Correctly guessed that we needed some version of a “type” for the config, and just assumed that’s what you were getting at with “enum”. The phrase “contains the substring “(140, 200, 200)” or some equivalent” just seemed odd and I don’t think you can reasonably assume people will connect that with “a phrase you must use when constructing the type you need”.
Question 4: How do I know that the copying of what could be thousands of comments on thousands of posts at scale won’t be a problem? How do I know that concurrent modification will be an issue? Maybe the comments are read only? Again, the intended answer is a perfectly reasonable point – my problem is that there are other reasonable points here as well.
Question 5: Often inlining is the best solution. Especially when the logic is only used in one place. How are we to know it’s not? Making functions for everything, while (yes) providing the encapsulation you mention in the explanation, can also make a code base much harder to read. Depending on the situation, I might think the two function solution was ideal, and I might think it was a clear mistake.
Hey! I liked that the questions do touch on concepts I think are fundamental to software design, and force you to think about all of them in concert (and decide which supersede which on a case-by-case basis). I’ve learned from the explanations, as well as validated my own understanding.
With that said, and even keeping in mind your defense about objectivity both in the quiz and in this comment, I’d like to comment: please reconsider the use of the word “objective”. A subjective question (one based on personal experience) does not change its subjective nature regardless of how well-founded an answer is, or how unanimous people are in answering the question. One doesn’t necessarily need to take away merit from an argument just because it’s subjective, if that helps relieve the pressure of using “objective”.
Subjective questions are typically operationalized, that is, an operational definition is given which can be used as an objective question. E.g. the subjective “what soda brand is best” could be given an operational definition of “what soda brand will be voted the most when a population is asked ‘what soda brand is best’” – the answer is literally the count of votes, regardless of how people reasoned out their vote. If we think the operational definition is well-posed, we can try to form some answer to the original subjective question, but it doesn’t mean the original question is now objective – we simply have a proxy we can objectively answer.
I feel the same about “which code is best”-style questions (although the entire body of software design is so large that I’ll admit I can’t – and wouldn’t dare to – prove it to you from first principles that it is subjective). Are we sure we’re answering “which code is best”? Or are we answering an objective proxy, such as “which code patterns are employed by successful software teams, where success is measured as X” for some operational definition of success (which btw I don’t even think the quiz attempts to do, except possibly in a very implicit way)? Answers to the latter can be valuable I think, of course, but they don’t change the nature of the original question.
This absolutely isn’t meant to take away merit from the principles behind the quiz. I think there are deep principles behind it, and I think software designers should take the time to digest this instead of reflexively and defensively disagreeing if they get it wrong. Cynical takes would be “this is not objective, therefore this is wrong” or “this is subjective, so I’ll only listen to it if I agree”, both of which I wouldn’t condone. But I think the world needs more precision and nuance around language, not less – there’s already enough confusion between fact and opinion, subjective and objective.
IMO don’t just normalize “this is objective, therefore listen to me”; also normalize “subjective questions are okay; listen to this subjective answer because the principles are well-constructed”.
Then again, this entire piece (as you can probably tell from my constant “I think” hedging) is itself subjective, so take that as you will :)
Hi igemnace,
Would a fair summary of your core argument be “It cannot be objective because the actual value of the software is not objective?”
I wish I knew my philosophy well enough to name what philosopher you’re taking after. :)
These answers and the way they’re framed show very little regard for the subjectivity of context or much justification for the “objectively” correct answer other than software design principles that are just some other guys’ opinions: in other words, for different ways of knowing.
For example, you claim in the XOR explanation that the correct answer is the one in which the programmer does not have to think as much. Now, I agree with reducing mental load where possible and practical, but what even constitutes mental load is contextual, and so is the applicability of the principle. Sometimes you want the gory details spelled out in front of you, and sometimes abstracting a one-liner out to a function (let alone two) is just silly.
Or for the rectangle one, I disagree that having more constructs (Point, Dimension) is universally better. That means when I want to construct a Rectangle I first have to construct two other Things first, which could be less desirable for performance reasons, and also just strikes me as gross.
So maybe this makes me the first person to dispute that your answers are better, let alone objectively correct. Somehow, I doubt it.
None of these questions are “ironclad”. You seem very convinced of yourself.
I honestly wonder how anyone can walk away from that feeling they “learned” something.
I feel you have some good points, worth making, but these are not the examples you want. It’s 5/5 nonsense.
Calling any of this “objective” left a foul taste in my mouth.
def new_rectangle(x, y, w, h)has been fine for the past 300,000 years of human existence.