Threads for matthewfarwell

  1. 1

    If you’re interested in visualisations of PI, see Martin Krzywinski He recently did some music based on Pi with Gregory Coles, Numberphile2 did a podcast on it: (The first and last digits of PI)

    1. 5

      Really fantastic video for someone like me who has very limited formal math knowledge but quite a bit of programming experience - it’s so good because it goes into detail about the concepts and the way the approaches are being constructed with just the right amount of background and context. Really enjoyed it, thanks for posting.

      1. 7

        3Blue1Brown is always good.

      1. 2

        I once had a very confusing conversation with a recruiter who wanted 15 years of Python experience. That’s technically possible of course, but not really for $50/hr part-time contracts. I actually asked her if she found anyone, and she said yes, she found the guy who literally wrote the book on Python! Because this was in a smaller city, I pretty quickly googled him… he had written a self-published book on Python… and technically had done some Python work 15 years before…

        If someone gets really particular about years of experience, it’s best to just run the other way!

        1. 5

          Some people have 20 years of experience in a technology. Others have the same year twenty times.

        1. 3

          One pattern that has worked well for us is what we call the “instrumentation” pattern (bad name, can’t think of a better one). For instance, we use it for logins, where we want to 1) log something, 2) write an audit trail and 3) update a JMX metric. So for the class Session (session management) we have a SessionInstrumentation class containing (for instance) a failedLogin(String username) method. This does everything necessary, and keeps the Session class itself clean. The business logic stays in Session, the logging/auditing/metrics are in the SessionInstrumentation class

          1. 5

            Patching things because of a log4j CVE…..

            1. 8

              Not exactly “Web” presence, but as (some) old Usenet posts are accessible on the WWW, mine dates from October 1990.

              1. 4

                Google used to have archives of a small fraction of my Usenet and ARPAnet posts from the mid-80s … OK, here’s a post to SF-LOVERS about the recent film “Real Genius”.

                Oh, “outside of archives” … well, the unofficial FAQ of the record label 4AD was originally put together by me in the late ‘80s and is still online after going through a number of other owners.

                The image on the (disused) home page of my personal website is a picture of my then-house takin in 1994 or so.

                1. 2

                  Hail Bob!

                  I was able to find some Usenet postings of mine from 1995, but I couldn’t find any prior, which is a shame.

                  1. 1

                    Just for info, I have no connection with this:

                    1. 8

                      In order of importantance…

                      • Executable documentation on how to use this api. Always up to date with worked examples. Conversely, if you find them hard to read and understand and use as such…. that’s telling you something important about your unit test design.

                      • Compelling cases. They’re a list of use cases / required behaviours that compel the design and implementation. As a trivial example, testing !even(1) and even(2) are compelling. Anymore test cases are superfluous to compel the design.

                      • Compelling API design. If an API is hard to test, it is hard to use and reuse. Fix your API design.

                      • Excluding test cases. eg. There are an infinite supply if integer sequences which do not have 1 and do have 2. Adding test case that prove our implementation excludes large swathes of likely wrong implementations. eg. !even(1999) even(4000)

                      • Corner cases. !even(0), !even( UINT_MAX)

                      • Property tests. for 100 random i {even(i) xor even(i+1)}

                      • Defect localization. With a superbly designed api and unit test suite, tell me which test failed, and I will immediately tell you the file and line number of the bug.

                      • Capturing and locking required behaviour to prevent regressions.

                      1. 4

                        This creates some cognitive dissonance with the V-model. Are unit tests part of testing or part of the documentation?

                        I guess the logicians answer is fitting: Yes.

                        1. 4

                          I would add:

                          • testing exceptional conditions, which is usually very hard to do with integration tests
                          • speed of execution, to speed up the feedback loop. Integration tests may require a server, which needs to start up, they need an environment. Unit tests give you the opportunity to test your code more quickly.
                          1. 2

                            So much this. The human brain can stay so much more engaged and learn so much faster when the feedback loop is tight.

                            1. 1

                              Unfortunately no longer let’s me edit my post, otherwise I would add yours and /u/thirdtruck and /u/jonahx ’s points as well.

                            2. 2

                              Good list.

                              I’d add, perhaps at the top: to force you to make all your dependencies explicit. This is ofc related to the bullet you called “Compelling API Design.”

                              Another important point not mentioned in the article: To have a fast test harness. As you’re refactoring, or just developing, you can have near instant feedback when you’ve broken something with a good UT suite. This isn’t feasible with integration or e2e tests. So I’d also add “enables developer productivity.”

                            1. 2

                              Squirrel. Because I learned during the era of Embedded Squirrel, in Ingres.

                              1. 3

                                Messy by Tim Harford. Tim is the presenter of the BBC More or Less programme (, which I love, and this describes how sometimes the fact that the world is messy can promote creativity, and we shouldn’t be too hung up on trying to make everything tidy.

                                1. 3

                                  Hey, if anyone wants to have job security in the future, how about being an IoT security expert?

                                  1. 7

                                    Trust me when I say this: Nobody wants to pay for IoT security.

                                    1. 4

                                      Oh, they’re going to pay, one way or the other.

                                      1. 11

                                        They don’t pay until there’s regulations or court-level liability for them. They also don’t care until that’s true. Both embedded market in general and IoT are terrible places to try to sell people on security. Also remember that the whole reason that 8- and 16-bitters still sell to tune of billions of dollars is that management on buying end wanted to shave every dollar or penny they could per unit. It’s all extra profit for them. That’s all they care about.

                                        There are occasionally companies using secure hardware or software as a differentiator. I’ve seen switches using INTEGRITY-178B, embedded devices using Java processors, a Bitcoin wallet with tamper-resistant MCU, a workstation hypervisor in SPARK Ada, GENU builds a product line on OpenBSD, and so on. Outside defense contractors, most of them go out of business eventually or barely scrape by. People buy the insecure stuff instead.

                                        1. 8

                                          No - they won’t.

                                          Their customers will, but they won’t.

                                          When people are conditioned to buy knockoffs on amazon because “cheap electronics”, they don’t even know who the reseller is, let alone the manufacturer of the device.

                                    1. 1

                                      I know this is a really stupid question, but can you get round this by printing on yellow paper?

                                      1. 1

                                        I bet someone with the original could get the paper and ink to respond differently under different lighting. But on a scanned copy, you’d have to have a quite good, sensitive color scan to see it.

                                        As these are in most printers, your question sounds like the setup for a great afternoon experiment. I hope to upvote your blog post about it. :)

                                      1. 2

                                        There are a number of interesting things about this story. First is the phrase “Human Error”. For me, this is very similar to the story recently. There could be a staged recovery (ou’re not allowed to start all the machines at once for instance)

                                        Secondly, why wasn’t this tested? You need to test disaster recovery scenarios. Especially for something as important as this.

                                        Thirdly, I don’t like the leaking “A leaked BA email last week had pointed the finger at a single contractor.” Let’s blame someone else. I can see someone being fired for this.

                                        1. 2

                                          The contractors also deny that what BA is claiming is the cause is the cause

                                          But who knows.

                                          1. 1

                                            Have you considered support for remote+dynamic config? I’ve a co-developed such a thing for Java ( but we never found a Node equivalent. In the Java ecosystem a lot of the pieces were already there, so this is basically an API and glue. If one would want to centralise configuration using Consul, etcd or what have you, what would you suggest? What about if you need to react to runtime config changes? Do you see these features as something that would make sense in HConfig? I’m really trying to avoid having to write a “jindy.js” from scratch.

                                            1. 1

                                              Hmm, I don’t know if that would make sense in HConfig. It’s more supposed to just be a parser for a language. However, it would totally be possible to build something on top of that which uses HConfig as a configuration language. Reacting to runtime config changes would probably also be up to the application or another language to implement; just parse the file again whenever you want to reload it, and do whatever application logic is necessary to incorporate the updated configuration.

                                              1. 1

                                                We use HOCON, which, used with the typesafe config jars, has wonderful support for overriding values. The idea is that each jar can have it’s own reference.conf, and then you can add a new config file with a fallback to the default. So you can read config from consul or etcd and then use it to override the default values, We use typesafe config a lot and it makes our lives a lot easier. We use a single object which contains all of the config values which we care about. This is immutable and typed. If we need to re-read (because of changes in consul), then we re-read and we have a new immutable object.

                                                1. 2

                                                  Over the years, there have been a number of these “warts of Scala, problems with Scala, look at how bad Scala is”. I think it is really healthly to have these debates, and it is a sign of a language that people care about. Just to be clear, I am *not” attacking Haoyi in any way.

                                                  Does this happen to the same extent in other programming communities/languages?

                                                  If not, what makes Scala special in this regard? Is it where the language comes from (EPFL/Martin Odersky), what Scala is trying to do (OO / functional hybrid), or just the people that are in the community?

                                                  1. 3

                                                    We occassionally have threads like this on Haskell Reddit:


                                                    1. 3

                                                      As someone who has worked with Scala for the past 5 or 6 years, I’ll say that Scala is just poorly designed.

                                                      • It has a lot features but there’s no check to ensure that they work correctly together
                                                      • Instead of keeping Java compatibility in a useful way, Scala instead consumes all of Java’s problems
                                                      • Syntactic sugar which doesn’t desugar how you expect because of irregularity (e.g. for-comprehension and withFilter)
                                                      • Slow monadic code due to having to trampoline in a lot of cases
                                                      • Implicits
                                                      • CanBuildFrom
                                                      1. 3

                                                        What’s wrong with implicits, or CanBuildFrom, assuming you’re not referring to the silly signature it gives to functions like map?

                                                        (edit: these are not naïve questions, I’m well aware of their design reasons and know they can be problematic, but I don’t consider them as warts)

                                                        1. 2

                                                          Not OP, but the issue with CanBuildFrom (and similar “magic cookies” like ExecutionContext) is that they not only force implementations of the collections API into one specific approach, but also conflate the data you put in, the transformations you intend to run and how these operations are executed.

                                                          That’s why Scala’s collections will never be remotely efficient – CanBuildFrom prescribes that all implementations are supposed to do completely pointless work.

                                                          That design approach is also the main reason of the bizarre proliferation of almost identical, but incompatible APIs in Scala.

                                                        2. 1

                                                          Yeah it’s much more obviously painful when you’ve tried other ML languages, even F# which honestly drinks from a sicker pond.

                                                        3. 1

                                                          Popularity? Scala is probably the most popular[1] advanced language. Haskell has these debates too, but it’s not as popular, so they’re not as visible outside the community.

                                                          Such discourse is absolutely necessary for the advancement of the language, but to those not familiar to the language or the discourse, this discourse gives the impression that everything sucks and things are hopeless. But in reality, for both Scala and Haskell, things are looking up.

                                                          [1] speculation based on stackoverflow careers job postings + TIOBE + github

                                                        1. 3

                                                          Welcome to Spring.

                                                          1. 4

                                                            More drama than I’m accustomed to from the Java community. As a “normal” Java web developer, this module system was either going to have no perceptible effect on me or a bad one, depending on whether or not I believed the noise about reflection and Maven. Still, I’m surprised to see this result. Many of the ‘no’ votes here seem to be hoping for patching up but I’m not hugely optimistic about what can be done in one month, or even whether those hopes are genuine, especially from Redhat and IBM. This one may be worth scrapping and trying again from scratch.

                                                            1. 2

                                                              I agree that this is more drama than we’re used to. For me, this is a good thing, but I don’t know how Oracle will react to this. Will they say that the JCP is more trouble than it is worth? Don’t know.

                                                              1. 1

                                                                That’s a frightening thought.

                                                            1. 2

                                                              There’s a lot of stuff to address in the document, but I get the feeling their complaints are largely misguided.

                                                              Let’s go through one passage as an example.

                                                              It has been posited that the presence of circular dependencies is an anti-pattern and a design error. The primary supporting argument is that all modules which form a cycle are logically one module.

                                                              That sounds reasonable enough.

                                                              However this at best applies only to limited cycles of a small fixed number of modules which come from the same author and are produced at the same time.

                                                              Yeah, that might be!

                                                              Modern, real world applications, even relatively small ones, now consist of dozens or hundreds of distinct pieces from a multitude of sources.

                                                              Indeed they do. And this is where the author seems to be overlooking a problem.

                                                              Maven is a big part of this: by allowing application dependencies to be managed automatically, the friction of doing so has been greatly reduced, and it has been observed that including substantial dependency graphs in an application as a common practice has increased apace

                                                              Yes. Maven is a lot like NPM in this regard. Java and JavaScript applications all lug around reams and reams of dependencies, just because adding them has been so easy.

                                                              Just slap on left-pad or commons-whatever and let Maven take care of downloading everything.

                                                              So if the new module system curtails this mad proliferation of (largely pointless) dependencies, that’s not something to complain about.

                                                              The only potential downside I can think of, is if people just can’t handle cleaning things up, and the module system will cause a major Python3 -like rift that weakens the whole ecosystem for ages.

                                                              1. 3

                                                                JPMS worries me a lot.

                                                                I’ve worked at companies which have a “Not Invented Here” syndrome, and they wrote their own web servers, http clients, all sorts of things. I prefer to use apache http client, or jetty. I would find it difficult to write and maintain a correctly functioning http client, especially at the same time as actually doing work. apache-httpclient is ~33k lines of Java. The same is true of jetty, which is even more critical to our business (~43k lines of code). These figures don’t take into account the dependencies.

                                                                I don’t agree that these dependencies are pointless. left-pad, yes. apache commons, maybe (big maybe), jetty-server, no way, http-client no way. They make me much more productive for my company.

                                                                As for cleaning things up, yes, theoretically, this should be the way that things are written in the first place, but this will cause a lot of short/medium term pain. ALL of my dependencies have to be correctly modified. This modification won’t be back-ported to the versions that I use, so then I need to upgrade to the latest version. This may involve API changes. The wait for upgrade to all my dependencies is a pain which I already have to go through for Scala -Scalastyle still doesn’t have a 2.12 version because I’m waiting for a dependency to upgrade to 2.12.

                                                                My attitude comes down to the same as was quoted:

                                                                “Ok. [Forget] Jigsaw compatibility. If doing so requires use of Java 9 tools, I’ll have zero […] level interesting in support for next 2+ years” - Tatu Saloranta (Author of Jackson, WoodStox, ClassMate, TStore)

                                                                I agree with Ceki Gülki when he said:

                                                                If only the current JPMS incarnation had a convincig use case, JPMS’ barebones simplicity might be considered pure genius.

                                                                1. 1

                                                                  I’ve worked at companies which have a “Not Invented Here” syndrome, and they wrote their own web servers, http clients, all sorts of things. I prefer to use apache http client, or jetty

                                                                  I use Jetty too, but I did write a tiny http client not that long ago :)

                                                                  But there’s a massive difference between writing your own Jetty and writing your own left-pad, or whatever some random commons-turdnugget from 1998 is supposed to accomplish.

                                                                  As for cleaning things up, yes, theoretically, this should be the way that things are written in the first place, but this will cause a lot of short/medium term pain

                                                                  The Java ecosystem has been lugging around an enormous pile of mostly pointless JARs in every single project, just because that’s what we ended up doing because Maven made adding dependencies so painless.

                                                                  (Note: Before Maven came along, we used to actually care about our dependencies because we had to manually place them under the WEB-INF/lib directory of a Java web-app)

                                                                  But the situation is pretty much exactly what the front-end world is like these days. Libraries like ‘left-pad’ or ‘is-array’ should arguably not exist, but they definitely shouldn’t be (transitively) adopted by everyone, to be hauled around forever and ever in every project.

                                                                  That’s just crazy. So if the Java 9 module system manages to “force” people into being careful and judicious about their dependencies, it will be a huge benefit for the Java ecosystem in the future.

                                                                  I’d much rather endure some short-to-medium term pain in exchange for long-term happiness, than keep enduring the dirty mess we’ve got on our hands now. I’ve been scraping together my own SaaS product, and never wanted to touch Node with a 10-foot pole.. but you just can’t use React without learning some bare minimum level of Webpack and having some Node-specific config files lying around even if you’re not actually using Node.

                                                                  I’ve still managed to avoid most of the craziness, but having to wrangle with all the solutions-in-search-of-problems has been a huge waste of time, and one that wouldn’t have happened if dependencies weren’t so easy to add.

                                                                2. 1

                                                                  All this opposition to dependencies seems to be aesthetic rather than practical. Dependencies are a lot cheaper than they used to be, so it makes a lot of sense to use more of them and do less writing your own code. Even if it’s 10 lines, why write it yourself when a dependency will be less effort for you now and easier to maintain going forward?

                                                                  1. 2

                                                                    With Maven in particular, dependencies have an incredibly stupid cost, which can be trivially demonstrated with only two dependencies:

                                                                    • Project
                                                                      • DependencyA
                                                                        • Conflict:1.0.0
                                                                      • DependencyB
                                                                        • Conflict:2.0.0

                                                                    Uh-oh. If we assume correct semver, there’s no way to successfully use this project except to get lucky with the changes in Conflict between 1.0.0 and 2.0.0. Worse than that, the actual conflict will not manifest until runtime; Project successfully compiles against both DependencyA and DependencyB and the conflict won’t become apparent until A or B actually try to use a feature missing from whatever version of Conflict ends up on the classpath.

                                                                    This is made even worse because “correct semver” is a categorically false assumption for Maven projects; one that bites me often is Amazon’s AWS SDK, which frequently makes backwards-incompatible changes (often in the form of incompatible versions of transitive dependencies, making the problem very difficult to track down) between minor versions.

                                                                    (To be clear, the problem is mostly cultural. Tooling could and should make these conflicts easier to catch early, but the real issue is massive backwards-incompatible version churn.)

                                                                    Now, what’s the best way to solve this? Eliminating dependencies is one way, though as you point out the costs are very high. The proposal of Jigsaw (at least as far as I understand it, which is admittedly not very far) of letting A and B independently carry their own versions of Conflict is another. Enforcement of semver as far as possible and compile-time checking for version conflicts would at least make the problem easier to spot, though the unfortunate specter of runtime reflection makes it impossible to catch every problem. Ultimately I think the correct solution is to change the culture so people write libraries which are much more stable and versioned correctly, and major version conflicts are much rarer; but the difficulty of that solution borders on impossible.

                                                                    1. 2

                                                                      Even if it’s 10 lines, why write it yourself when a dependency will be less effort for you now and easier to maintain going forward?

                                                                      Why would a dependency be easier to maintain for me, than 10 lines of my own code that may never need to be changed?

                                                                      Sometimes a new version of ‘left-pad’ or whatever comes out, and then it’s downloaded over and over and over again all over the world, even though conceptually nothing has changed.

                                                                      The new version is supposed to do exactly the same thing as the old one, but it might even be broken somehow!

                                                                      It’s often a good idea to go through some effort in the short term, in exchange for benefits in the long term.