Threads for bilal2vec

  1. 111

    I tell anyone asking me for career advice the same two things.

    The first: the deeper in the world’s dependency tree you are, the less frequently things will churn, and the longer your skills will last. TCP doesn’t change very often. Theoretical skills may be applicable for your entire career. Human skills are more durable than any technical skill. Kernels don’t change very often (but more than TCP). Databases don’t change very often (but more than kernels). There is a spectrum of skill durability, and you will burn out faster if you find that all of your skills become worthless after a very short time. Dependency forces things not to change their interface, which causes the work to shift toward performance and reliability among other things that some people find far more rewarding over time.

    The second: the more people who do what you do, the worse you will be treated, the more BS you will have to put up with, the worse your pay will be, the faster you will be fired, the harder it will be to find a job that values you, etc… etc… Supply and demand applies to our labor market, and if you want to be happier, you should exploit this dynamic as heavily as possible. Avoid competition like the plague. But don’t avoid funding. How do you avoid competition without going off into the wilderness where there is no money to be made? Hype drives funding, but it also drives a lot of competition. However, using rule #1 above, the hyped things depend on other things. Many of these dependencies are viewed as “too hard” for one reason or another. That’s the best place to be. Go where other people are afraid, but nevertheless have a lot of money depending on.

    All hyped things rely on things that for one reason or another are not commonly understood, and tend not to change quickly. That’s a good place to find work involving durable skills that tend to have lower competition. Go where the dependency is high but the competition is low, and you have a better chance of being happy than people who go where the competition is high or the dependency is low. Bonus points if it’s actually “hard” because then you won’t get bored as quickly.

    There are areas of front-end that are high-dependency, durable, slow-changing, and low-competition. That’s where engineers are likely to be happiest. But these two principles apply to every field or zooming out to any business generally. I’m pretty happy working on new distributed systems and database storage engines for the time being. But I’m always looking for the things that are viewed as hard while also receiving significant investment, as these are the things that will ultimately give me more opportunities to live life on my own terms.

    1. 10

      Go where other people are afraid, but nevertheless have a lot of money depending on.

      There is an old Yorkshire saying: “where’s there’s muck, there’s brass”.

      1. 7

        This is so true, I’ve gone to my car to fetch my laptop just to upvote and comment. It’s an exceptionally important piece of advice I wish I had understood as early as possible in life, but I didn’t.

        CS (pure) and Math degrees are so good because they teach you really basic theories that are mostly timeless. Whenever I’ve gravitated towards more trendy or applied skills, either in coursework or in jobs, there’s always been a really poor and transient ROI.

        […] using rule #1 above, the hyped things depend on other things. Many of these dependencies are viewed as “too hard” for one reason or another. That’s the best place to be.

        What are some examples of these dependencies right now, or in the near future?

        1. 6

          Thank you very much for this post. Great distillation of essential career advice, especially the part about the durability of human skills. So many developers would derive far more value from a single public speaking or improv class than from learning yet another new programming language.

          1. 3

            Oh man, the number of times I’ve echoed this exact same message to others almost verbatim in the first two paragraphs. Thanks for posting this. Thinking about my career in this way a few years ago was probably the most valuable change I made.

            1. 4

              Thank you for the kind words :)

              Large-scale ML, blockchain, IoT, serverless, k8s, etc… are all areas recently flooded by newly minted experts in the topical skills, but like the Australian poop diver who claims to have never worked a day in his life, there are great opportunities for high-respect jobs in the dirty internals of the systems :) Particularly with this set of hyped tech, there are very few people who seem to specialize in getting bugs to jump out in distributed systems. Everybody is still writing almost all of them in a way that assumes the happy path. But there are techniques for building all of these systems in ways that encourage the race conditions to jump out. The few people who take pride in building correct distributed systems will have their plates full for a while!

              Another reason why this kind of bug hunting and prevention is not all that popular may be that the sorta-similar yet way-cooler-seeming field of systems security tends to absorb a lot of the people who may have otherwise been predisposed to this type of work.

            2. 2

              love this comment. like the other commenter, do you have any examples of “hard, but trendy” in frontend or elsewhere?

              1. 8

                Spitballing, I’m going to google “hyped programming field” and see what comes up, then I’ll try to break it into areas for investigation. Ok, my results on the first page seemed to be about AI, python, golang. I mentioned high-scale ML depending on distributed systems skills and correctness skills above, so I’ll think about the others. Python feels harder for me to answer so let’s dig into that.

                Python is immensely popular and I think that becoming an expert in alleviating any friction point that people often hit will be lucrative. But what about things that SOUND really boring? When I think about things that suck about Python, the most dreadful thing that comes to my mind is migrating large applications from Python 2 to Python 3. Python 2 stuff is still everywhere, and it’s increasingly hazardous over time because its support has run out. But people don’t want to touch things that are still ticking along. Legacy systems, like geriatric medical care, becomes more important every day.

                But despite being extremely important and an excellent place to apply interesting mechanical (yet supervised) translation and correctness analysis during modernization, so many people have been burned by consent issues at work where their managers forced them to work on legacy despite them not wanting to work on them. So much of the overall programming population has been burned by non-consensual legacy engineering that almost everyone wants to stay as far away as possible. The only question is how to find the companies who realize that modernizing their legacy systems is actually something you want to pay a specialist to do instead of forcing the task on junior devs with no negotiating power. Banks are well-known for paying consultants for this kind of work, but to be honest I’m not sure.

                Maybe Python 2 modernization isn’t exactly a golden ticket due to the difficulty in finding companies who are willing to hire specialists to perform this task. Maybe it’s too easy. But may it’s not. I’m not sure. In any case, this maybe demonstrates the general search technique, and can be used to illuminate further areas for research. If general Python 2 modernization is “too easy” then you can slap on more filters. Python 2 modernization for automotive codebases. That sounds much more correctness-critical and likely to pay a specialist to accomplish.

                Anyway, the general pattern is: what feels repulsive and hard? Maybe that sense of repulsion creates a dynamic where the few people who do the thing are extremely well treated due to their scarcity. If that scarcity of talent aligns with an extremely popular overall field, there’s a good chance that there are folks who require a specialist to address this niche.

                1. 6

                  Anyway, the general pattern is: what feels repulsive and hard? Maybe that sense of repulsion creates a dynamic where the few people who do the thing are extremely well treated due to their scarcity. If that scarcity of talent aligns with an extremely popular overall field, there’s a good chance that there are folks who require a specialist to address this niche.

                  Here’s an example of scarcity working out well for an expert. Another company in my industry was converting to a different accounting system. These projects are incredibly complex and can last years. Part of the process required recreating hundreds of templates for the new system. One contractor handled the template conversion; she retired at a young age after the project was complete.

                  1. 4

                    non-consensual legacy engineering

                    Have you written about this anywhere? I’d be curious to hear more about it. It immediately struck a chord with me, and probably does with most software engineers.

              1. 2

                rewriting my jax implementation of a DCGAN from flax to the new linen api https://github.com/bkkaggle/jax-gans

                1. 1

                  i recently redesigned my website https://bkkaggle.github.io to be a lot more minimal. lmk what you think!

                  1. 2

                    I like it because it has a very distinctive design language, that reminds me of the Pebble Time smartwatch, and the duck is a nice touch! I was just disapointed to see that the blog didn’t share the same style.