1. 9

    totally happy with a Microsoft ergonomic keyboard.

    1. 4

      Same here.

      I’d love to have a fancy mechanical keyboard with lots of option keys etc. but I don’t have endless spare time to research let alone configure something like that.

      1. 2

        Same on both counts

      2. 1

        The MS Natural 4000 is perfect for me: I went from pain after an hour to no pain no matter how much I type. It’s only $30! Great stuff.

        I’ve got an ergodox, but getting used to the thumb keys at work but not having it on my laptop at home was just too much to get used to.

      1. 1

        I suspect some form of glove+VR interfaces might surface in the same way violins still exist in a world where guitars exist. It’s easier to learn guitar, but there’s things you just can’t do with it, so some people still learn violin.

        But then again, no one learns the Engleberts Violin, instead they continue to learn a qwerty keyboard simply because of social velocity. Only a few learn something like devorak, and almost no one learns a chording keyboard.

        Heck, most business email I get seems to come from phones, and most I send is on a phone. We’ve already started to move towards degraded interaction in business contexts, simply because it’s so convenient.

        1. 15

          As a junior developer doing my best to learn as much as I can, both technically and in terms of engineering maturity, I’d love to hear what some of the veterans here have found useful in their own careers for getting the most out of their jobs, projects, and time.

          Anything from specific techniques as in this post to general mindset and approach would be most welcome.

          1. 33

            Several essentials have made a disproportionate benefit on my career. In no order:

            • find a job with lots of flexibility and challenging work
            • find a job where your coworkers continuously improve themselves as much (or more) than you
            • start writing a monthly blog of things you learn and have strong opinions on
            • learn to be political (it’ll help you stay with good challenging work). Being political isn’t slimy, it is wise. Be confident in this.
            • read programming books/blogs and develop a strong philosophy
            • start a habit of programming to learn for 15 minutes a day, every day
            • come to terms with the fact that you will see a diminishing return on new programing skills, and an increasing return on “doing the correct/fastest thing” skills. (e.g. knowing what to work on, knowing what corners to cut, knowing how to communicate with business people so you only solve their problems and not just chase their imagined solutions, etc). Lean into this, and practice this skill as often as you can.

            These have had an immense effect on my abilities. They’ve helped me navigate away from burnout and cultivated a strong intrinsic motivation that has lasted over ten years.

            1. 5

              Thank you for these suggestions!

              Would you mind expanding on the ‘be political’ point? Do you mean to be involved in the ‘organizational politics’ where you work? Or in terms of advocating for your own advancement, ensuring that you properly get credit for what you work on, etc?

              1. 13

                Being political is all about everything that happens outside the editor. Working with people, “managing up”, figuring out the “real requirements’, those are all political.

                Being political is always ensuring you do one-on-ones, because employees who do them are more likely to get higher raises. It’s understanding that marketing is often reality, and you are your only marketing department.

                This doesn’t mean put anyone else down, but be your best you, and make sure decision makers know it.

                1. 12

                  Basically, politics means having visibility in the company and making sure you’re managing your reputation and image.

                  A few more random bits:

              2. 1

                start a habit of programming to learn for 15 minutes a day, every day

                Can you give an example? So many days I sit down after work or before in front of my computer. I want to do something, but my mind is like, “What should I program right now?”

                As you can probably guess nothing gets programmed. Sigh. I’m hopeless.

                1. 1

                  Having a plan before you sit down is crucial. If you sit and putter, you’ll not actually improve, you’ll do what’s easy.

                  I love courses and books. I also love picking a topic to research and writing about it.

                  Some of my favorite courses:

                  1. 1

                    I’ve actually started SICP and even bought the hard copy a couple weeks ago. I’ve read the first chapter and started the problems. I’m on 1.11 at the moment. I also started the Stanford 193P course as something a bit easier and “fun” to keep variety.

              3. 14

                One thing that I’ve applied in my career is that saying, “never be the smartest person in the room.” When things get too easy/routine, I try to switch roles. I’ve been lucky enough to work at a small company that grew very big, so I had the opportunity to work on a variety of things; backend services, desktop clients, mobile clients, embedded libraries. I was very scared every time I asked, because I felt like I was in over my head. I guess change is always a bit scary. But every time, it put some fun back into my job, and I learned a lot from working with people with entirely different skill sets and expertise.

                1. 11

                  I don’t have much experience either but to me the best choice that I felt in the last year was stop worrying about how good a programmer I was and focus on how to enjoy life.

                  We have one life don’t let anxieties come into play, even if you intellectually think working more should help you.

                  1. 8

                    This isn’t exactly what you’re asking for, but, something to consider. Someone who knows how to code reasonably well and something else are more valuable than someone who just codes. You become less interchangeable, and therefore less replaceable. There’s tons of work that people who purely code don’t want to do, but find very valuable. For me, that’s documentation. I got my current job because people love having docs, but hate writing docs. I’ve never found myself without multiple options every time I’ve ever looked for work. I know someone else who did this, but it was “be fluent In Japanese.” Japanese companies love people who are bilingual with English. It made his resume stand out.

                    1. 1

                      . I got my current job because people love having docs, but hate writing docs.

                      Your greatest skill in my eyes is how you interact with people online as a community lead. You have a great style for it. Docs are certainly important, too. I’d have guessed they hired you for the first set of skills rather than docs, though. So, that’s a surprise for me. Did you use one to pivot into the other or what?

                      1. 7

                        Thanks. It’s been a long road; I used to be a pretty major asshole to be honest.

                        My job description is 100% docs. The community stuff is just a thing I do. It’s not a part of my deliverables at all. I’ve just been commenting on the internet for a very long time; I had a five digit slashdot ID, etc etc. Writing comments on tech-oriented forums is just a part of who I am at this point.

                        1. 2

                          Wow. Double unexpected. Thanks for the details. :)

                    2. 7

                      Four things:

                      1. People will remember you for your big projects (whether successful or not) as well as tiny projects that scratch an itch. Make room for the tiny fixes that are bothering everyone; the resulting lift in mood will energize the whole team. I once had a very senior engineer tell me my entire business trip to Paris was worth it because I made a one-line git fix to a CI system that was bothering the team out there. A cron job I wrote in an afternoon at an internship ended up dwarfing my ‘real’ project in terms of usefulness to the company and won me extra contract work after the internship ended.

                      2. Pay attention to the people who are effective at ‘leaving their work at work.’ The people best able to handle the persistent, creeping stress of knowledge work are the ones who transform as soon as the workday is done. It’s helpful to see this in person, especially seeing a deeply frustrated person stand up and cheerfully go “okay! That’ll have to wait for tomorrow.” Trust that your subconscious will take care of any lingering hard problems, and learn to be okay leaving a work in progress to enjoy yourself.

                      3. Having a variety of backgrounds is extremely useful for an engineering team. I studied electrical engineering in college and the resulting knowledge of probability and signal processing helped me in environments where the rest of the team had a more traditional CS background. This applies to backgrounds in fields outside engineering as well: art, history, literature, etc will give you different perspectives and abilities that you can use to your advantage. I once saw a presentation about using art critique principles to guide your code reviews. Inspiration can come from anywhere; the more viewpoints you have in your toolbelt the better.

                      4. Learn about the concept of the ‘asshole filter’ (safe for work). In a nutshell, if you give people who violate your boundaries special treatment (e.g. a coworker who texts you on your vacation to fix a noncritical problem gets their problem fixed) then you are training people to violate your boundaries. You need to make sure that people who do things ‘the right way’ (in this case, waiting for when you get back or finding someone else to fix it) get priority, so that over time people you train people to respect you and your boundaries.

                      1. 3

                        I once saw a presentation about using art critique principles to guide your code reviews. Inspiration can come from anywhere; the more viewpoints you have in your toolbelt the better.

                        The methodology from that talk is here: http://codecrit.com/methodology.html

                        I would change “If the code doesn’t work, we shouldn’t be reviewing it”. There is a place for code review of not-done work, of the form “this is the direction I’m starting to go in…what do you think”. This can save a lot of wasted effort.

                      2. 3

                        The biggest mistake I see junior (and senior) developers make is key mashing. Slow down, understand a problem, untangle the dependent systems, and don’t just guess at what the problem is. Read the code, understand it. Read the code of the underlying systems that you’re interacting with, and understand it. Only then, make an attempt at fixing the bug.

                        Stabs in the dark are easy. They may even work around problems. But clean, correct, and easy to understand fixes require understanding.

                        1. 3

                          Another thing that helps is the willingness to dig into something you’re obsessed with even if it is deemed not super important by everyone around you. eg. if you find a library / language / project you find fun and seem to get obsessed with, that’s great, keep going at it and don’t let the existential “should i be here” or other “is everyone around me doing this too / recommending this” questions slow you down. You’ll probably get on some interesting adventures.

                          1. 3

                            Never pass up a chance to be social with your team/other coworkers. Those relationships you build can benefit you as much as your work output.

                            (This doesn’t mean you compromise your values in any way, of course. But the social element is vitally important!)

                          1. 4

                            I was interested in what he was saying just up until he said

                            Some may even be lucky enough to find themselves doing Extreme Programming, also known as ‘The Scrum That Actually Works’.

                            My experience with XP was that it was extremely heavyweight and did not work well at all. It created the greatest developer dissatisfaction of any of the versions of Agile I’ve encountered.

                            1. 5

                              Couldn’t disagree more – the most successful team I was on was heavily into XP. When people say it’s heavyweight, they’re usually talking about pair programming. I’m not sure what people have against it; I’ve found it’s a great way to train junior developers, awesome for tricky problems, and generally a great way to avoid the problem of, “Oh this PR looks fine but redo it because you misunderstood the requirements.”

                                1. 2

                                  I don’t want to discount your experience, but it sounds like the issues you’ve had with pair programming are more with the odd choices your employer imposed.

                                  Both people have specialized editor configs? Sure, switch off computers or whatever too; no need to work in an unfamiliar environment.

                                  And if one person is significantly less experienced than the other, that person should be at the keyboard more often than not – watching the master at work will largely be useless.

                              1. 3

                                Why I like XP over anything else is the focus on development practices rather than business practices. Pairing, TDD, CI, <10 minute builds, WIP, whole team estimation, etc are all used to produce a better product, faster.

                                The weekly retrospective offers a way to adjust practices that aren’t working and bolster those that are.

                                1. 2

                                  Agreed 100%. It turned my head a bit when he thought Agile was too prescriptive, but then was considering an even more prescriptive methodology.

                                  1. 1

                                    What was your experience with XP? Also, scrum is heavyweight as well in my experience and doesn’t work excellently in an actually agile environment like a startup. Feels like it could work in corp. though.

                                  1. 2

                                    Very interesting, because in the average language, some can be expressed as run time checks, others as compile time checks. Getting into more dependently typed languages allows more to be expressed at compile time, but still some are quite tricky.

                                    Reminds me of the Tony Morris Tic Tac Toe game, which was surprisingly challenging, and gave me a great afternoon of fun. http://blog.tmorris.net/posts/understanding-practical-api-design-static-typing-and-functional-programming/

                                    1. 8

                                      After finishing SICP I enjoyed Programming Languages which was about building interpreters in Racket. You’ll want to follow along with the book and the course assignments, for some reason they keep them separate. http://cs.brown.edu/~sk/Publications/Books/ProgLangs http://cs.brown.edu/courses/cs173/2016/assignments.html

                                      I really enjoyed Haskell Programming From First Principles, which is about learning Haskell. Since we’re programmers, we can learn to wield both sides of the Force, Lisp AND Haskell. http://haskellbook.com

                                      Likewise, Let Over Lambda is excellent and I’d argue even more mind bending than SICP and is about DSLs in CL macros, with a final project of making the language Forth a valid DSL in CL: https://letoverlambda.com/index.cl/toc

                                      I’m only part way through, but I really like Software Foundations. It’s about dependently typed proofs in Coq. Really good stuff. Suddenly you realize there’s not just two sides to the Force. https://softwarefoundations.cis.upenn.edu/current/index.html

                                      All of these were very similar to SICP in difficulty, exercise length, and value to my ability to write and comprehend software. I can’t recommend them enough.

                                      1. 3

                                        Brown further developed the course - CSCI 1730 based on a revised version of PLAI (and an early version of typed racket) with video lectures and assignments.

                                        PLAI’s spiritual successor Programming and Programming Languages is also available based on a new language called Pyret.

                                        1. 2

                                          Oh cool! I was taking the course when they switched mid-semester to typed racket (which was a big improvement), its nice to see they’ve continued revising further.

                                      1. 10

                                        I have yet to hear anyone say that they enjoy pair programming all day, every day, forever.

                                        I love pair programming — a key part of my interviews includes a pairing session — but I quite frankly can’t pair for more than about an hour at a time, tops about four sessions per day with at least two different people. That’s my personal preference.

                                        I would be interested to know if the authors team consistently concludes during regularly scheduled retrospective sessions that everyone enjoys doing forced pairing enough to keep it a thing.

                                        1. 6

                                          Let me be the first. :) I love pair programming, and miss it whenever I have to code without it. Doing it for a year kind of ruined me for working alone. (And I’m not an extrovert! In general, I shy away from being around people.)

                                          I read about bad pair programming experiences and count myself as lucky that all the partners I had were really great people and invested in the process. I can see how it could go sideways.

                                          1. 7

                                            In the past, I’ve misunderstood what “pair programming” actually means. Could you elaborate and say more exactly about what it means to you? What does your daily process look like? How did you deal with the practical problems stated, like, say, different editor configs or such?

                                            1. 2

                                              I’m not the OP, but here is a write-up of my good experience pairing.

                                              http://deliberate-software.com/pairprogramming/

                                              1. 2

                                                OP here: I remember reading your article a while ago (it’s really good!). When I saw the below quote I realized you were talking about me.

                                                Some developers love the idea of pairing, not the practice of pairing; they often leave when they discover that distinction

                                                A lot of days I would wish I could like it, so I could enjoy my job. I felt like I gave it a fair shot too (2 years).

                                                1. 2

                                                  Do you all have a notion of “going odd”? I probably am “odd” at least 3 sessions a week, which helps for time to think and recharge.

                                                  1. 1

                                                    We did but if you’re on a team with an even headcount you’ll never go odd. I wish we could’ve gone odd more often.

                                                    1. 1

                                                      Hmm that’s unfortunate.

                                                      Even with 6-10 in our office, one pair splits most of the time to handle odd work, research, or support.

                                                      Recognizing when to split pairs while still maintaining the tenet of “all production code written by a pair” goes a long way.

                                              2. 2

                                                Sure – when I did it full time, I had the same partner for about a week at a time before switching. We worked on a project for 8 hours a day, coding for about 45 minutes at a time with 10-15 minute breaks in between. We usually were co-located, although not always. We had two keyboards and mice, and usually two monitors, and would switch off who was writing code differently depending on the partner dynamic, but it was usually pretty organic – whenever your partner was stuck on what to write and you had an idea, you’d take over.

                                                Different editor configurations were definitely a thing that could cause friction, but I don’t see that as a bad thing. I both learned a lot about editor tricks and gained an appreciation for keeping your config as simple as possible. We mainly worked in console-based editors (vi and emacs) as we were sometimes pairing remotely.

                                                Remote pairing was easier in some respects. Instead of using one person’s machine, which could have its own config issues, we spun up a VM on AWS from an image we made, so there was a standard base configuration.

                                            2. 4

                                              I would be interested to know if the authors team consistently concludes during regularly scheduled retrospective sessions that everyone enjoys doing forced pairing enough to keep it a thing.

                                              Pairing was already a part of the culture when I originally joined, so it was actually pretty hard to get people to admit they didn’t enjoy it (politically unpopular). It was never brought up in retrospectives. That said, I’d say the majority of the people who worked there did enjoy it and it worked well for them.

                                            1. 4

                                              I’ve paired full time for the last five years on complex banking software - I’d say unless you’re doing simple CRUD, it’s more like “pairing, take the productivity of two devs and multiply it by 4”.

                                              Every developer gets to work in the whole system - no silos. Everyone gets to learn everyone else’s tricks and tips - no slow fuddy-duddy who still doesn’t know about Ctrl-c, etc.

                                              Not only that, but pair rotation means we see no burnout on really hard tasks and tech debt. Normal companies I’ve worked at only have the stamina for either minor tech debt or “scrap it and start over”. Both are terrible choices for long term productivity. The real value is in massive tech debt projects accomplished over months, without a loss of functionality or features. Pairing enables that where single developers just can’t handle the boredom.

                                              Also, pairing is a skill that is as separate from programming as touch typing. It needs to be learned and practiced for months to see an ROI. Most people don’t have the chance to try it for that long, so they remember it with as much fondness as someone with a few weeks of typing remembers the keyboard ;)

                                              1. 1

                                                I respectfully disagree.

                                                I specifically wanted to avoid the conversation of productivity (happiness is a more humanitarian metric, and therefore more important to me), but here we go:

                                                I’ve paired full time for the last five years on complex banking software - I’d say unless you’re doing simple CRUD, it’s more like “pairing, take the productivity of two devs and multiply it by 4”.

                                                I was pairing on a payment gateway, so similar problem space (not easy). I found no such productivity multiplier. In fact, I was frequently (and silently) bemused when I noticed “odd” programmers completing far more work than their pairing counterparts. I’m still amazing that people try to say that dividing your work-streams in half yields higher efficacy. The math just doesn’t add up here, even in a world where everyone learned by watching people code.

                                                Not only that, but pair rotation means we see no burnout on really hard tasks and tech debt. Normal companies I’ve worked at only have the stamina for either minor tech debt or “scrap it and start over”. Both are terrible choices for long term productivity. The real value is in massive tech debt projects accomplished over months, without a loss of functionality or features. Pairing enables that where single developers just can’t handle the boredom.

                                                You’re creating a false dichotomy where you can choose between pairing or tech debt. Tech debt is equally prevalent in both XP and non-XP worlds. Also, anecdotally, I am living proof that burnout is possible with pairing. :)

                                                Also, pairing is a skill that is as separate from programming as touch typing.

                                                No argument here!

                                              1. 8

                                                Wonderful talk. Considering I use both Node and Go the first bit was hard to take, but totally understandable. I have often met professional programmers that don’t really care about how anything they use actually works. As you said, the only way to really learn is to forge ahead and attempt an understanding of whatever’s inner-workings.

                                                1. 4

                                                  A good way to learn, which schools seem to mostly ignore, is to repair something which is broken. I guess they assume you will come across enough of your own bugs, but guaranteeing that the students see certain kinds of failure is useful for making specific points and showing specific interactions.

                                                  Plus, the detective mentality is useful on its own and should be fostered wherever possible.

                                                  1. 6

                                                    My favorite programming course in school was Software Engineering 2.

                                                    The first SE course introduced us to working as a team, using version control, unit testing, and documenting our work. But we got to do all this while starting from scratch, which is an uncommon luxury.

                                                    The second SE course was centered around refactoring. Every team was provided a particularly bad SE1 project from a few years back. Ours was a checkers app written in Java with Swing. The code was just awful. Instead of matrix or even a 1D array, the authors had created individual JButtons 1 through 64 for the main gameboard. Buttons were their own ActionListeners. It was a dumpster fire.

                                                    The challenge of the SE2 project was that you could only refactor some of the app. You weren’t allowed to throw the whole thing away and start from scratch. Plus, bad as the code was, it did work.

                                                    We ended up using the newly learned Adapter and Proxy patterns to add multiplayer support to the app. It was pretty fun! And my earliest taste of what actual programming work is like.

                                                    1. 2

                                                      Back when I was in Uni, our department (mechanical physics) sometimes received crank letters with over-unity devices, etc. I thought it would be fun to have a question in a test to determine why such a device wouldn’t work.

                                                    2. 1

                                                      I have often met professional programmers that don’t really care about how anything they use actually works.

                                                      Do you meet the other kind at all? I tend to use sentences starting with “I have often met” for things like not liking triply-nested expressions. For not caring about how dependencies work, the situation seems bad enough that I tend to say: “I’ve heard in distant lands of the odd savant who understands some internals of their dependencies more often than not, but of course everyone can’t hope to meet such a person themselves.”

                                                      At best I meet people who understand the internals of some single project that they are active with the community of. But while they may have poked around inside Vim, they’d know nothing about their network stack or browser or compiler.

                                                      This assumption is kinda central to my life, so if I’ve just been really unlucky I’d really appreciate pointers on where to find such people.

                                                      1. 2

                                                        The risk of a deep understanding of an entire stack is the switching cost: what happens when that library falls out of fashion, the new company only uses X, or you switch to a new language to keep up with the changing times. In my short career I’ve gone through:

                                                        • Windows XP+Vim+Perl
                                                        • RedHat+Emacs+PHP
                                                        • Windows 7+Eclipse+Java
                                                        • Suse+ZendStudio+PHP
                                                        • Windows XP+VS+VB.NET
                                                        • Windows Server 2012+Rider+C#+F#+Vim+Java+Js

                                                        That’s an incredible amount of memorization just for language and library basics. I suspect I’m lower than the average. I don’t think it’s humanly possible for me to dive deep on all those things and still get work done. I also don’t have any motivation to try when it’s just as likely my next job will be in OpenBSD Klingon edition.

                                                        1. 2

                                                          I currently work with a really awesome lead programmer that pushes understanding rather than spouting meaningless platitudes. Still, teaching is a challenge. Things are easier for her to grasp, and her code reviews lean toward style more often than not. Nevertheless, I feel like I’ve improved professionally by seeing her work and speaking to her.

                                                          Whereas, at my previous job I had a coworker that was a “lead” in the business for nearly 10 years. I really don’t think he could code a loop on the fly. It was strange. It was depressing. Yet, he has survived by being one of those programmers that can’t program. It was not something I wanted to grow into. I’m glad I have the pleasure of working with people that seek to build maintainable codebases. Add to that the fact that they also desire to understand how the tools they use work.

                                                      1. 5

                                                        I’ve been pairing full time for five years. Having trained 12+ developers in that time, I can say with confidence that pairing is a unique skill that takes most people about 6-12 months to improve from “below average” to “above average”. I’d liken the difficulty to learning to touch type the first time.

                                                        Having put a lot of time into practicing the skill, right now I prefer to pair on most problems. I also prefer to use Vim, use TDD for most problems, and touch type. None are required to accomplish good work, but having the tool in the toolbox (with time invested to learn it) means I can judge for myself when the tool is appropriate to use.

                                                        Likewise, I’ve yet to see any believable research into programmer productivity. It’s currently impossible to measure, any studies are small sample sizes on the least possibly trained cohort (often times university students). As much as I want to believe in any study either way about TDD, pairing, static types, agile, new languages, etc, they are all just guesses made with anecdata and the fashion of the day. At least accepting the reality of the situation will let us tackle the real problem: why is programmer productivity impossible to measure?

                                                        1. 3

                                                          I am a working software engineer with no degree, but I wouldn’t recommend this approach. It is order of magnitudes easier to get a job in the field with a Bachelor’s Degree and the paid interning opportunities college generally provides. Plus, some “old-fashioned” companies are just never going to hire you without a degree, no matter how many years of experience.

                                                          1. 1

                                                            I wrote this for the many people I know who already have a college degree but cannot find sufficient employment. Either their degree was worthless, or it was in a field that has more people than jobs, or both. Many of them are willing to work hard, but just don’t even know where to start or what to do. I believe the programming field right now offers a place for those who are willing to work hard at learning and self-improvement.

                                                            Sure, if you are a high school graduate and considering this course verses going to college, I would probably lean towards a cheap CS degree. Many great CS degrees cost 30k+ a year. If that was my only option, I would skip the degree.

                                                          1. 3

                                                            One thing I’ve been wondering is: how do people make CI builds work in a monorepo? Do you rebuild everything on every checkin, or do you detect that a subproject hasn’t changed, and thus is not rebuilt? In the latter case, how does that then give you (what I assume is) the main benefit of monorepos: simplified dependency management, because you just use the dependency with the same revision.

                                                            1. 2

                                                              What I’ve done in the past is build “everything,” with build caches and aggressive dependency markings. If you’re at the scale where having a monorepo is genuinely a pain, the it’s time to get a sane build system (literally out-of-the-box Gradle works fine if you just have Java/Kotlin, for example, but something like Bazel/Pants/etc. works great for more generic workloads), at which point the builds that don’t need to happen will be no-ops, and you never need to worry about forgetting to rebuild/upgrade something.

                                                              For me, this is really coming down to trade-offs. Configuring something like Bazel or Pants (or even potentially Gradle at the scale we’re talking) does take some effort. The advantage, to me, is that you either will end up doing this anyway with multirepos (at which point, you might as well have a monorepo where you can monitor all the changes anyway), or you handle all the upgrades piecemeal and bespoke, at which point it’s gonna be really tricky to ensure you’ve actually upgraded everything. In other words, we’re trading human steps for automation: if you’re at a size where you “need” multirepos, you’re also at a scale where automating all of this will be a huge win, which in turn, IMHO, negates needing multirepos.

                                                              1. 2

                                                                The chromium project uses the analyze functionality of gn (initially implemented in gyp) to determine which test binaries are affected by a change and only build/run those tests. Since this relies on the gyp/gn dependency graph, various checks and whitelists are added to cover edge cases. This, combined with the task-deduplication abilities of Swarming heavily reduce the load of ‘building everything’ in a monorepo. For Swarming, test binaries are compiled in a deterministic way so the same code produces the same SHA, if a SHA already ran and succeeded you don’t need to run it again.

                                                                1. 1

                                                                  We use Teamcity, and it has a way to make triggers only work for a regex of directories. So, only commits to X subproject trigger builds of X.

                                                                1. 11

                                                                  This is a great post with lots of insights into the tradeoffs of different ways to split up a system. When the conclusion is basically “it depends” you know it’s a great source of information and not a marketing piece cloaked as engineering advice.

                                                                  1. 2

                                                                    At first I scoffed, but actually I appreciate the thought he put into it. I’d say I’m convinced enough to give it a try for a week.

                                                                    1. 9

                                                                      I find a lot of these type of articles leave me with little empathy for the writer, often they seem as much an aggressor in the situation as those they write about.

                                                                      Not so here - this post really pissed me off. Here is someone who was aware enough of the systemic problem to wait for a year to complain about sexual harassment so she can graduate successfully. Holy crap, I can’t fathom the determination and self-control that must’ve taken. That breaks my heart. And still they hav took at dig at her. Whew.

                                                                      Then not getting listened to? WTF is wrong with everyone?

                                                                      What can I do?

                                                                      1. 2

                                                                        What can I do?

                                                                        Avoid working in Japan, for starters.

                                                                        1. 4

                                                                          Definitely. And for a whole host of reasons. It’s not a good place for creative types to work. Society there tries to force all such people to be cogs in machines.

                                                                          1. 7

                                                                            Correct, if you don’t want to deal with a very conservative culture. Actual conservative, not the caricature most people think of in relation to, say, the Midwest. Patrick McKenzie had an interesting writeup here that covers some of the problems.

                                                                            1. 4

                                                                              To be fair to Japanese culture, there is something to be said for that sort of traditionalistic, conservative structure. Japan’s work culture is dysfunctional but it doesn’t have the ingredients that makes Silicon Valley universally reviled: corruption and evil. Japan’s work culture may be inefficient, but it’s not corrupt and it’s not evil.

                                                                              In Japan, you wouldn’t have someone spend years trying to ruin another person’s career just to amuse himself, whereas this is commonplace in the startup world. (There’s a lot of evidence that YC was originally founded to advance personal vendettas of the original partners.)

                                                                              In Japan’s work culture, you have well-intended people who end up doing harm. One example: the intent of employees to invest in their people and ensure employment, unfortunately, results in a labor market with no fluidity. A layoff that would be no big deal in the US is career-ending in Japan, to the point that unneeded people are often put on make-work for several years to avoid stigma. Another example: not wanting to let the team down, people stay in the office until 9:30 pm– and this forces the rest of the team to stay late as well.

                                                                              Silicon Valley probably has similar levels of dysfunction, but its dysfunction is opposite in origin: intentionally created, by malevolent people, toward nefarious ends.

                                                                              The Japanese work culture has an incorrect assumption in it that long work hours are productive, and this causes a lot of suffering, but there’s an earnest belief (and a benevolence) behind it. Whereas in Silicon Valley, the suffering is deliberate and usually a divide-and-conquer play coming from the founder class.

                                                                              The other thing is that Japan is homogeneous and probably the closest to being a true classless society of any at scale. (Japan, of course, has social class issues, as all societies do.) Elite graduates still have to put in the long hours like everyone else. The result of such ethnic and class homogeneity is that it can develop assumptions that aren’t entirely correct and that, even when executed by people with good intentions, lead to dysfunction.

                                                                              California, on the other hand, is fragmented and its defining trait is its lack of culture and history. The upshot of this is that it can engender creativity. The downside is that (1) it enables the worst of humanity, whether we’re talking about Charles Manson or Phil Spector or Paul Graham or Travis Kalanick; (2) the lack of established rules enables corruption, as you see with spoiled rich kids getting a huge advantage in the career game that Japanese megacorps wouldn’t give them, and (3) the heterogeneity makes the place ripe for divide-and-conquer games, which is what most people in the founder/investor class actually use to get people to work (instead of, you know, paying them fairly).

                                                                              1. 5

                                                                                I don’t feel follow YC much, so I’d like to know more about the evidence that it was used for vendettas. Could you or someone explain or show me what to search? Unfortunately, I only found irrelevant topics and a bunch of Quora posts about you.

                                                                                1. [Comment removed by author]

                                                                                  1. 1

                                                                                    I think that if someone time-traveled from 1940 to 2017 and landed in Japan, he’d see a capitalistic society but think that the communists won 45-50% of the battles. He’d see low income inequality and a focus on community and humanity in business decision-making, but he’d also see a lot of face-saving bureaucracy and inefficiency.

                                                                                    If he time-traveled to 2017 and landed in Silicon Valley, he’d see a capitalistic society but think that the fascists had won 45-50% of the battles: less of the racist shit, but dehumanizing corporate control, a lack of belief in human dignity, and increasing cohesion among the ruling class while divide-and-conquer tactics are deployed against the “good soldier folk”.

                                                                              2. 1

                                                                                Japanese salaryman employment represents just a subset of jobs and is not representative of Japanese culture as a whole. There are many opportunities for creatives across all fields here.

                                                                              3. 3

                                                                                I don’t think we can use one incident to indict Japan. There’s sexism in a lot of places, and Silicon Valley takes the cake when it comes to the aggressive, threatening kind (although other cultures may have more of the subtle, traditionalist kind) that would lead a woman to fear for her safety.

                                                                            1. 12

                                                                              Every time I find myself writing haskell.. I come to the same question: “How do people get things done in haskell!?”

                                                                              Not because of the things mentioned in the link.. but because somehow I always end up going down a rabbit hole of theory (either compsci or math).. Not saying this is bad.. I always learn a grip, but because of it - I can’t use haskell for regular things :P

                                                                              1. 13

                                                                                I think this is a pretty common pattern. Taking the “learning Haskell is like re-learning programming” metaphor seriously, this would be like learning Python but feeling infinitely compelled to dig into the VM implementation and how that all hits metal.

                                                                                The cure to both I think is just to have concrete goals of making something happen. Practice building a small project in Haskell but aim to produce a concrete project result as efficiently as possible measured in your own time and sweat. Huge aspects of Haskell will be penalized merely by you not being familiar with them—but an IO-bound implementation written in an imperative style probably is easily obtainable.

                                                                                Then from here, make an iteration and improve something. Carve off a small piece of the application and generalize it. Ask for help leveraging types and purity to constrain some pieces.

                                                                                I think an approach like that can be very successful.

                                                                                1. 2

                                                                                  As someone who is learning Haskell in this very way, it’s definitely doable. The hard parts are all up front, then it’s all downhill.

                                                                                2. 8

                                                                                  As tel notes, you really need to unlearn your existing views of computation a bit before you can start to grok haskell.

                                                                                  The problem I see is that a lot of people really have a hard time with that one thing. Haskell is oddly easy to use once you start to let go of the notion that computation has to be a series of steps. It is rewarding though, honestly I was at flipping tables stage with computation in general and learning Haskell and Idris have completely made me fall back in love with things. Just my $0.02

                                                                                1. 3

                                                                                  “This is not a near-term threat. We’ve had programs writing code for 65 years: that’s called a compile” (michaelochurch)

                                                                                  Darnit, he beat me to my counterargument again! :) Mine is that we can barely compile a functional language into something that performs as well as hand-written C. That is, in any situation where a human engineer could either write a good, functional program and a high-performance, C program. Same for straight-forward, program synthesis from logical specifications. Same for digital hardware where the binary nature gives synthesis huge advantage. Should’ve totally automated it by now in a way where hardware engineers don’t start rolling eyes when you say “High-Level Synthesis.” Same with synthesizing design from formal specifications. Same with even formally specifying every act of the system in a way a machine can understand as opposed to just some of them.

                                                                                  Over and over, the best tech humans can create for these simple tools fall short of their goals. Then, people worry that automatic programming will (a) happen, (b) happen sooner than good compilers, and © be good enough to threaten human’s jobs. Quite a leap! I suggest they figure out how to get the compiler optimizations working better before worrying about handling a system that replaces compilers and people. ;)

                                                                                  1. 2

                                                                                    Great point on the synthesis front. We get close, but despite having full control over semantics, runtime and everything else, can’t always guarantee a suitably high-level cost-free abstraction (Rust doesn’t count).

                                                                                    Related: overheard the clerks at the grocery store lamenting how their automatic doors sometimes open randomly of their own accord. They were getting extremely cold from the air blowing in. I chuckled, but then realized: if we can’t get automatic doors to function correctly 100% of the time, then we have a long way to go.

                                                                                    1. 2

                                                                                      Remember on the automatic doors that most projects screw up for economic reasons. Embedded and industrial engineers are always telling me about managers not caring about quality or pushing to cut costs however. One of reasons 8-bitters with no security are so popular: saves a few bucks to tens of dollars every unit that becomes profit.

                                                                                      Problem with automatic door is probably economic like that. Cheap sensor, hack job by developer, or not enough maintenance. There’s also a chance the algorithms are just that hard to do but I doubt it.

                                                                                    2. 2

                                                                                      I agree with this point, and can add the example of Sql. We still can’t turn ORM queries into performant Sql. Or optimization. I have been working on integer optimization for our banking software, and it falls apart after just a few thousand requirements. There are very real, very hard edges to what we can currently automate in software, and most are closer than we think. Optimization is a perfect example. If it takes my integer optimization project 45s to max which 6 cards with 2 attributes and 3 upgrades of 9 types to buy (9 constraints of 45k options each) then it is light years away from intelligently converting high level synthesis of hardware or ORM queries.

                                                                                    1. 3

                                                                                      The reason I’ll give is that Haskell’s otherwise excellent composability is destroyed at I/O boundaries, just like every other language.

                                                                                      Where we see Haskell (or more generally, typed FP) excel is for programs that have minimal surface area in contact with the outside world, but with a large amount of interesting computation happening internally.

                                                                                      This is a very good point. Most type system researchers seem to work under the wrong assumption that the only thing worth statically verifying is the memory-safety of a single running process[0]. This is why we get ridiculous “expressiveness improvements” like type-level Peano arithmetic, HLists and the like, that allow you to enforce ever more sophisticated invariants - so long as your data structures don’t cross process boundaries.

                                                                                      IMO, further research in this direction only offers diminishing returns. Far more promising results would be obtained (and probably for less effort) by using type technology to enforce that multiple processes, possibly running on different machines, exchange data safely. For example, as someone who plumbs data back and forth between database servers and GUI clients (read: writes CRUD apps) for a living, I would be delighted with a type system that checks whether the data model in my application code agrees with the data model in the database server. This is probably an utter triviality from the point of view of pure type system research; however, it would offer bigger productivity gains than yet another statically verified implementation of self-balancing search trees or whatever.

                                                                                      [0] One notable exception is type providers in F# and Idris. Kudos to their designers!

                                                                                      1. 9

                                                                                        This is why we get ridiculous “expressiveness improvements” like type-level Peano arithmetic, HLists and the like, that allow you to enforce ever more sophisticated invariants - so long as your data structures don’t cross process boundaries.

                                                                                        Far more promising results would be obtained (and probably for less effort)

                                                                                        however, it would offer bigger productivity gains than yet another statically verified implementation of self-balancing search trees or whatever.

                                                                                        I don’t know why you are bringing up type system expressiveness as if it is diametrically opposed to crossing process boundaries. Being able to enforce these properties on a single process is a prerequisite for being able to enforce them on multiple processes.

                                                                                        type technology to enforce that multiple processes, possibly running on different machines, exchange data safely

                                                                                        There is plenty of research in this area. I don’t have the time to prepare a survey, but Global abstraction-safe marshalling with hash types (ICFP 2003), its realization into a more complete PL Acute (ICFP 2005), and Proof Carrying Code (POPL 1997) are good places to start.

                                                                                        1. 6

                                                                                          I don’t know why you are bringing up type system expressiveness as if it is diametrically opposed to crossing process boundaries.

                                                                                          It isn’t. But all type systems have a complexity budget, and GADTs, type families and data kinds might not be the wisest way to spend it. The pros (mechanical verification of finer-grained specifications) have to be weighed against the cons (loss of global type inference, poor interaction with modularity).

                                                                                          Acute (ICFP 2005)

                                                                                          From a cursory read of the paper, it seems that Acute’s type system verifies the communication between Acute programs. However, in the scenario that I’m most interested in, it isn’t reasonable to rewrite an existing database server in a new programming language. I was expecting something more like a type checker that can connect to a database and verify that the client application’s data model agrees with the database schema.

                                                                                          Proof Carrying Code (POPL 1997)

                                                                                          From a cursory read of the paper, as well as what I remember from chapter 5 of ATTAPL, PCC is primarily geared towards verifying safety properties of low-level code. It seems like it would be useful primarily to systems programmers, not application programmers like me who take low-level safety for granted.

                                                                                          1. 7

                                                                                            It isn’t. But all type systems have a complexity budget, and GADTs, type families and data kinds might not be the wisest way to spend it. The pros (mechanical verification of finer-grained specifications) have to be weighed against the cons (loss of global type inference, poor interaction with modularity).

                                                                                            I agree that one must balance expressiveness with type inference, but I think this is a different claim than you originally made. I’ll note that for GADTs at least, great progress is being made on type inference. My current research involves using a language with sum kinds and polymorphic recursion and I’ve been looking into making use of that GADT result.

                                                                                            From a cursory read of the paper,

                                                                                            Indeed, both Acute and PCC require buy-in on the PL side. You may be interested in the literature on contract systems, or the literature on gradual typing. Both of these subjects consider the case where you are in a mixed typed/untyped scenario, although neither consider distributed computing explicitly.

                                                                                            For databases in particular the literature on record systems might be interesting, since a major focus is being able to precisely model relations in the type system. There’s HList which you (rightly I think) denigrated but outside of the Haskell world there is good work being done on record systems. [Ur/Web](www.impredicative.com/ur/) in particular is a standout, although that language has limited type inference.

                                                                                            1. 7

                                                                                              I always have a hard time precisely articulating my real objections to GADTs and type families (which don’t apply to polymorphic recursion) but I’ll give it a try once again. It isn’t just that they complicate type inference.

                                                                                              What I like the most about sums and products (rather than, say, unions and intersections as in Typed Racket and Ceylon) as datatype formers is that they are isomorphism-invariant, i.e., they avoid coupling the essence of what my program does from the particular encoding I choose for the data. As far as I can tell, reference cells are the only primitive type constructor in ML and Haskell that breaks isomorphism invariance, and that’s precisely what makes mutation a “hard effect” (unlike, say, exceptions and other algebraic effects, which are “soft”, i.e., syntactic sugar for pure computations).

                                                                                              GADTs and type families emphasize physical type equality and break isomorphism invariance in exactly the same way as unions, intersections and reference cells: given a type function T, and two isomorphic types Foo and Bar, you can’t tell whether T Foo and T Bar are isomorphic. This has several unfortunate consequences, of which the most damning IMO is that GADTs and type families don’t play nicely with abstract types. The former require “yes/no” answers to type equality questions. The latter require the ability to answer such questions with either “yes” or “I can neither confirm nor deny”. For example, consider the following OCaml program:

                                                                                              module S = Set.Make(String)
                                                                                              
                                                                                              type _ foo =
                                                                                                | Int : int -> int foo
                                                                                                | Str : string -> string foo
                                                                                              
                                                                                              type _ bar =
                                                                                                | One : string -> string bar
                                                                                                | Many : S.t -> S.t bar
                                                                                              
                                                                                              let test_foo : int foo -> string = function
                                                                                                | Int _ -> "success"
                                                                                              
                                                                                              let test_bar : S.t bar -> string = function
                                                                                                | Many _ -> "success"
                                                                                              

                                                                                              The type checker correctly determines that test_foo’s match is exhaustive, because the types int and string are known to be different. On the other hand, the type checker can’t correctly determine that test_bar’s match is exhaustive, because S.t is an abstract type whose underlying representation might well be string. (We happen to know that this is not the case. But this requires global reasoning, and requiring whole-program analysis for type-checking purposes doesn’t scale.) This is a failure of abstraction.

                                                                                              1. 3

                                                                                                Your thoughts mirror my own although I would lay the blame on being able to answer type disjointness questions rather than on GADTs. GADTs sans type disjointness arise naturally in any language with higher kinds via an encoding of Leibniz equality. Haskell’s issue then is that it tries to get too much mileage out of the kind–there are other kinds that have a meaningful notion of disjointness and play nicely with inference, such as sum kinds, integer kinds, name kinds, etc. bar then can be indexed by an integer kind representing length, or by the sum kind () + () with left as One and right as Many.

                                                                                                FWIW I would argue that test_foo should also flag an error. Type checking under the context Γ, int : ⋆, string : ⋆ doesn’t imply that int and string are disjoint types, for all we know integers could be implemented as strings or vice versa. Of course we know that they are disjoint because they are hard-coded into the OCaml semantics but from a type-theoretic perspective this is kind of a wart.

                                                                                                If this kind of thing interests you, you may be interested in what I’m currently working on. If you are I can give you a ping in a few months.

                                                                                                1. 2

                                                                                                  I would lay the blame on being able to answer type disjointness questions rather than on GADTs.

                                                                                                  Yep, exactly! If GADTs could be restricted to only case-analyze things that are meant to be case-analyzable (like booleans, integers, etc.), that would be perfectly fine with me. It’s case-analyzing the universe of types, which isn’t meant to be case-analyzable, that causes semantic trouble.

                                                                                                  FWIW I would argue that test_foo should also flag an error.

                                                                                                  Indeed. Depending on integers and strings being disjoint types is evil, for exactly the same reasons as mentioned in my previous comment.

                                                                                                  If this kind of thing interests you, you may be interested in what I’m currently working on. If you are I can give you a ping in a few months.

                                                                                                  I’m interested. Sign me up for updates!

                                                                                        2. 3

                                                                                          For example, as someone who plumbs data back and forth between database servers and GUI clients (read: writes CRUD apps) for a living, I would be delighted with a type system that checks whether the data model in my application code agrees with the data model in the database server.

                                                                                          Ur/Web is an interesting experiment in that direction.

                                                                                          1. 4

                                                                                            Yep, exactly! That’s the direction I want things to go. However, I’m of the idea that the database must be the ultimate source of truth, and client applications must adjust themselves to it. That’s why I asked for a type-checker that connects itself to the database and validates client application code against the real database schema. Ur/Web takes the opposite view: it generates DDL scripts so that you can adjust your database schema to your application’s data model.

                                                                                            1. 4

                                                                                              a type-checker that connects itself to the database and validates client application code against the real database schema

                                                                                              Is this the kind of thing that type providers in F# are for?

                                                                                              1. 3

                                                                                                If I understand correctly, type providers work differently from (and quite possibly better than) what I literally asked for. Rather than validating that your application’s data model agrees with the database schema, a type provider “imports” the database schema as a collection of types that your application can use, so that you never need to manually define the data model in your code in the first place.

                                                                                                1. 3

                                                                                                  I think of type providers as just a simpler way to define your data structure and parsing code. E.g instead of defining a type and parsing code for some json, just give the compiler the json itself and it’ll do it for you. But if at run time the json comes in the wrong shape…

                                                                                          1. 11

                                                                                            I committed myself to daily practice a couple years back. I decided to not break my github streak. Every day I do at least 20 minutes of katas, books, or building something outside my comfort zone. It’s been revolutionary. I’ve worked through SICP, Programming Languages, Let Over Lambda, Land of Lisp, Algorithm Design Manual, Clojure Programming, a good chunk of Data Scientist’s Toolbox, Learn You A Haskell, F# 3.0, Real World Haskell, most of @haskellbook, and I’m currently working through Software Foundations in Coq. I built, deployed, and support a school attendence suite in Clojure. I’ve done countless katas, TDD practice sessions, and little helper programs.

                                                                                            All in all, deciding to put in 20 minutes a day and tracking it on GH has completely and drastically changed my skill in just four years. I believe if you set yourself the goal of writing something, anything for 20 minutes a day, you will find plenty to keep you busy and interested. You will see your skill rapidly improve. You will get bored of things in your current comfort zone, and so you’ll have to learn new things to stay focused. I can’t recommend it enough. I just crossed 1355 days of streak, and I intend to do it as long as I write software for money.

                                                                                            1. 6

                                                                                              I agree with the “20” minutes. But I disagree with covering such broad subject matter, and so shallowly (perhaps he meant something different).

                                                                                              I think you’ll learn a lot more by doing things end to end, and start to finish. Figure out a small idea for a program you want to create. It should be highly relevant to you or someone you care about. Build the whole thing, including the UI, the backend, the database, tests, deployment, monitoring, etc etc. Use it. Polish it and make it robust. Don’t stop putting time into it until it’s actually done.

                                                                                              I think perseverance is what makes expert programmers so unique. They can hit problems they’ve never seen and push through without getting discouraged. The only way to learn that is by finishing whole projects, not starting new ones.

                                                                                              1. 5

                                                                                                I’d say we are in agreement then! I finished almost every book and course, and the attendance site I built from scratch they are using it to this day. However, I disagree that fully completing SICP and Let over Lambda as shallowly covering Lisp and FP. Completing all 350+ exercises in SICP was one of the hardest things I’ve done and stretched my skill immensely.

                                                                                                I’m actually curious completely because I want inspiration, if what I do is a shallow study, what does your personal study of it look like? I left out a lot of the books I’ve read that don’t have homework (so I don’t count in my 20 minutes a day) like Domain Driven Design, Clean Code, Code Complete, Pragmatic Programmer, Working Effectively with Legacy Code, Growing Object-Oriented Software Guided by Tests, Death March, Art of Agile Development, Planning Extreme Programming, Extreme Programming Explained, Design Patterns, Implementing Domain Driven Design, Patterns of Enterprise Architecture, Refactoring, Peopleware, Managing Humans, and Becoming A Technical Leader. Those are all fine, but can’t really be practiced so I often don’t recommend them unless the person has mastered more tactical skills.

                                                                                                I do agree that perseverance is the biggest thing. When stuck and hopeless, I am at my worst when I give up, and at my best when I take a deep breath and look for alternate solutions.

                                                                                                1. 2

                                                                                                  I want inspiration, if what I do is a shallow study, what does your personal study of it look like?

                                                                                                  What I would suggest is trying to find people who share your passion but have non overlapping skills to review your code; and do the same for them. You want to try to constantly refine and improve the code so it is better than what any of you could do alone.

                                                                                              2. 3

                                                                                                How do you balance reading a chunk of e.g. SICP and then coding, presumably something relevant, in 20 minutes? How much would you typically read? Does this only work because you do it every day? Or is it because the texts come with relevant exercises, so you don’t have to come up with your own projects?

                                                                                                1. 4

                                                                                                  Those books all have exercises along with the text. I usually will read for a few minutes, then write some samples to see what’s up with what I just read, then start on the exercises. If the exercise is hard, I’ll have a few sessions of writing some unit test cases, playing with the api, and then finishing the exercise. At first each time I sat down I’d spend 10 minutes just remembering where I was, but after a year or so I got pretty good at just “jumping in” and can sit down and be typing/reading in under 30 seconds.

                                                                                                2. 2

                                                                                                  What did you think of Data Scientist’s Toolbox? I’m curious how resources there feel. Did you have an applied math background going in?

                                                                                                  1. 2

                                                                                                    I had effectively zero math going in, but a lot of various programming languages. It was fine, I enjoyed it. I think I’d want to do it a second time without the deadlines, those really bum me out and demotivate me.

                                                                                                  2. 2

                                                                                                    The book Software Foundations came up when people were talking about learning to do stuff like seL4. I noticed it’s Coq where seL4 and many good projects are Isabelle/HOL. Curious if you or anyone else can tell me if Software Foundations' material also covers most of knowledge or skill needed for HOL, too. It plus a HOL-specific tutorial or articles. Or is it and Coq’s approach different enough to need a whole different book?

                                                                                                  1. 1

                                                                                                    I’d love to see a concrete example of such a site. I think I’d switch my React/Angular projects over to this if at all possible, but I can’t get around how much content would sent around. Would every grid be prerendered? Every grid sort a form submit?