1. 30
    What tools made you better programmer ask programming

As a programmer we use different tools to keep our workflow productive such as using IDEs,Personalized TextEditors with extension.

What were some of the tools that made your work more efficient and helped you become a better programmer

  1.  

  2. 27

    Every time I learn a new vim keybind it feels like I have a new superpower, even after 20 years.

    In general, I’ve paid a lot of attention to “durability” as a metric. How long is the shelf life of the skill? Text editor shortcuts, shell shortcuts and scripting for one-liners, vim, awk, sed, zsh, fuzzy finding tools like sk that I have shell shortcuts for piping results into vim.

    The things that have helped me to be a better programmer are the things that have let me better understand the things I build. It doesn’t matter how fast I can churn out garbage. It doesn’t matter how pretty code is if it wakes somebody up at night, so tools related to learning about operational characteristics of my programs are the ones that have made me a better programmer because a program is the thing that ultimately runs somewhere:

    • quickcheck paired with fault injection for deterministically replayable machine-generated tests that can avoid some of the assumptions I make while writing the code. avoids the pesticide paradox, so my code never becomes fully immune to my tests. for distributed systems, I pair this with a simulator. I hate writing tests so I really prefer to let a machine generate millions of them for me while I lazily just specify what the program is supposed to do. without a doubt this is the main tool that has improved the quality of my software the most. most bugs are in your error handling code.
    • linux internals. kernels are virtual machines and we write far better systems when we’re aware of what the VM underneath our programs is going to do to translate our use of memory, files, sockets and threads into things that happen in hardware.
    • gdb, lldb, but especially rr with chaos mode for time traveling debugging of concurrent code. shell shortcuts like this for dumping the stacks of a running process:
      stacktrace () {
        gdb --batch -ex "t a a bt" -p `pgrep $1`
      }
      
    • massif visualized through massif visualizer for understanding memory characteristics of a program
    • dhat visualized with the built-in web viewer for finding short-lived allocations that could have happened on the stack
    • perf for getting stats about instructions, cache misses, branch mispredictions and correlating it back to lines of code, paired with perf report, flamegraph and hotspot for visualization. Be wary of metrics that don’t actually make your important metrics better. More instructions is sometimes still faster due to fewer branch mispredictions. Often the most interesting metrics are max latency per request, minimum throughput for some time period, peak memory usage etc… because they have significant capacity planning implications.
    • dstat for visualizing where my system’s high-level resources are being consumed by my programs

    The mindset that has helped me to improve my skills is that you should always just roll up your sleeves and look at the running process and its code rather than wasting more than 5 minutes searching for help or pouring over documentation that’s out of date and never correct. Put a short time cap on the time you spend doing that, and then just put on your explorer hat and pull out your scary toolkit to hunt that thing down.

    1. 4

      This is a great list of tools. I like your mindset as well.

      I’m still at the junior/intermediate stage and I think the confidence to be able to “roll up my sleeves” is what’s needed to move forward.

      1. 17

        I also lacked confidence to really dive in earlier on. Two things happened at roughly the same time:

        • I learned about the Feynman technique. Just write down how the thing you’re trying to learn works (a bug, a technique, anything) until you hit a wall in your understanding. This makes the boundary of your understanding explicit. Often, it’s surprising how much you actually do understand before hitting the wall, but before you build confidence, there’s a bias to assume the amount you don’t understand is greater than it really is. Make the boundary explicit, and then you have a specific point to push forward from, rather than kind of a cloud of doubt and anxiety. This made it clear that I knew more than I felt I knew, and at the very least helped me hone in on specific questions to ask.
        • I had a couple people around me at work who would pull me over when they were having a problem, before they found the answer, just to teach me about the debugging process. They would find the bugs in languages they didn’t know at all by just simple grepping through the repo for the codebase after observing some high-level systems metric that had gone out of whack. “we get bursts of connection errors every hour”. They would apply rough binary search to the space of the unknown to make rapid progress. They would keep a short log of observed clues and observations in a text file as they went. They would make a guess about how the program maybe worked, then guess words that variables and structures may be named that related to that guess, and with kind of shocking regularity they would find the cause of the misbehavior in like 4 or 5 guesses most of the time, even if they had cloned the repo for the first time just then and there. They taught me that looking for bugs is a pretty mechanical process that doesn’t really require anything more than making guesses and keeping your fingers moving as you search for your guess. “what happens every hour? what’s in the crontab on the machine initiating the connections that fail? how many concurrent connections does it spin up? what’s the TCP backlog for the server it’s connecting to? why isn’t the server it’s connecting to closing the connections after responding to a request? what’s this streaming mode config option that’s set on the server but not the client? ohh… that’s it.”

        Digging is mechanical and just foraging around in the dark guided by your best guesses. Your guesses are better than they feel like they should be. Your guesses get better and better over time, but that doesn’t matter so much, because even with bad guesses the process is still mechanical and tends to terminate after a short number of guesses. The biggest thing is knowing that bad guesses will still drive you forward. They illuminate the negative space. Keeping a log as you go lets you use your full attention on the current question being asked, rather than trying to remember all of the clues you’ve observed up to that point (gets bigger and bigger as you go along). Mechanically apply the Feynman technique with your log until you arrive at the answer. Knowing super high-level architecture of how things are wired up is helpful to know up-front, but if not, that’s just another mechanical thing to learn to guide your search into the cause.

      2. 2

        I just wanted to chime in that this was a truly inspiring post. I fully agree with your philosophy and you’ve given me a bucket list of tools to experiment with the next time I’m stuck. There’s nothing worse for me than retrying something ten times based on shoddy out-of-date docs – why not investigate the source?

      3. 14

        Destroying the notion of “fighting the compiler” was one of my biggest steps forward. I started to view the type checker as a pair programming buddy. One that doesn’t get tired, only states facts based on information it has, and will always update its assumptions based on new information, unlike most of us. ;)

        So basically “type systems”, but only after a necessary perspective shift. Well, that and being exposed to what modern type systems can actually do. Instead of the limited assumption that they only catch errors like: “That’s not an Int!”.

        1. 2

          As someone building extremely complex, scientific systems in Pythonthis. Seriously, type systems are so undervalued.

          1. 2

            Python 3.x doesn’t have type hints/annotations?

            1. 1

              It does! But it’s not a full replacement for a static/strong typing system, but it does help. We’re also pinned to an older version of Python (3.5) which doesn’t have full support for type hinting.

              1. 2

                I’m a Java developer working with Python to automate some things. It seems like when the program is more than a few lines of code (1K) and needs to be maintained by a team, the desire for type systems grows strong.

                1. 2

                  I’ve been porting a large Python 2.x code-base to Python 3. Adding type annotations (https://mypy.readthedocs.io/en/stable/cheat_sheet.html) has been hugely helpful. The type system is quite expressive and mypy’s reporting is pretty good. After spending some time with Rust – which i really like – I appreciate the flexibility of Python’s approach.

          2. 9

            I think vim and then later emacs+evil. It’s not often I actually need all the bells and whistles that are not available in emacs, or vim, and when I do I can reach for them, but for the rest of the time I find that vim and even more so emacs are superior environments to work in because I’m allowed to define the environment exactly to my pleasing - which allows a great amount of freedom and creativity - in ways no other development environments I have tried do. It’s a bit like working in your own wood and metal shop at home was being employed somewhere and working in their shop. The latter is just very limiting to me.

            1. 8

              Karabiner for me by far. I have access to over 500 actions/hotkeys without moving my fingers at all. Caps lock is too far so is return key. i.e. my return key is mapped to s+f. That is I press s key, then f key. My control key is a key. Most map control to caps lock but why map it to caps lock when you always hover over a key already.

              It’s pretty powerful stuff. My config. https://github.com/nikitavoloboev/dotfiles/blob/master/karabiner/karabiner.edn

              1. 2

                I love Karabiner. It is an essential part of my workflow on Mac. Vim bindings, layers, media controls, screen organization, all on my 60% keyboard with Mac-compatible keys is amazing.

                1. 1

                  This looks awesome. I see that your config is EDN format but I’m seeing JSON in the docs. Do you just convert yours to json?

                  1. 2

                    Yeah its converted from EDN to Karabiner JSON using Goku (https://github.com/yqrashawn/GokuRakuJoudo)

                  2. 1

                    This looks amazing, admittedly I looked at karabiner when I had a 60% keyboard, but didn’t realize that it could do all the extra things it can do! I’m going to have to give this a try this weekend, may steal some of your config to test it out.

                  3. 8

                    Probably i3 and/or tmux

                    1. 7

                      I believe iteration speed is a huge factor for programming productivity.

                      Innermost feedback loop while typing code. An IDE can show me syntax and even some semantic or linting errors immediately. I don’t need to execute something manually. The more checks the better, but it must maintain the “immediately” feeling.

                      Next feedback loop is building/compiling (if applicable). Fast compilers like D or Go improve productivity here and C++ is one of the worst ones (this is open ended though).

                      Next feedback loop is unit tests. I find it hilarious that some people skip this step and prefer much slower tests (manual/integration/whatever) instead.

                      Optimizing these three loops is essential. Ideally, I want this to be one button to press and results within ~2 seconds. Switching tools can help here but what concrete tools does not matter that much.

                      Disclaimer: Programming is only a part of software development and usually not the most important one to optimize.

                      1. 6

                        1.) Stepping away from an IDE. Being force to learn how to manually execute commands to convert a bunch of text to get run as an executable program shows you so many doors of opportunity to let your curiosity go wild. Going back to an IDE a bit and knowing what’s going on behind the scenes helped me troubleshoot and gave me techniques not available in my IDE.

                        2.) Forcing myself to use Linux as my only OS at college (around 2008-2009). It sucked at first, but having the doors flung open to OS internals and lots of documentation and command lines (coming from Windows) taught me a lot about how computers work and helped with (1). Important: GNU tools (binutils, coreutils), ctags.

                        3.) A text editor you can use on limited environments (pick vim or emacs). It helps when all you have to a development server is an ssh connection.

                        4.) Version control (git). Learning to use git really well helps prevent losing work and allows you to experiment.

                        1. 4

                          Notepad. It’s what I started with, and it’s absolute trash, but that’s why it’s so worthwhile. I’d say that I learnt more using Notepad than I have since, relatively. Not having syntax highlighting, indentation, or anything else made me focus more on the code, as far as I can tell.

                          Man. Actually reading man pages rather than searching stuff. It’s made me more efficient and made me understand stuff more than I did when I just did a copy and paste.

                          1. 1

                            As soon as I switched from first searching the internet for a solution to checking man pages first I began to understand the console toolset available to me in much more detail.

                            1. 1

                              Totally. And it’s said a lot, but I realised this after switching to OpenBSD - having a man page that didn’t just tell me to run info made the experience so much better.

                          2. 4

                            I think not using an IDE, just using an editor with syntax highlight, made me a better programmer actually. I got used to search and read documentation, instead of just waiting for something to popup in the autocomplete. This process lead to understand how languages/APIs/programs were structured, calling conventions, patterns, etc. Of course, now I use vim and vscode (with vim plugin) but I already have most stuff I need burned into my head.

                            Version control, even a local repo, it doesn’t mind if it’s git, hg, fossil. Viewing your code history is a great process to learn from mistakes and see how you improve.

                            It’s good to have some database viewer/browser at hand, sometimes you just need to peek values without having to remember specific database backend quirks (specially when debugging some random error in a system you didn’t write).

                            Linters are good to keep the code formatted.

                            A web browser to read other’s code :)

                            There’s no magic tool to make you a better programmer, everyone has it’s own preference and that won’t make you neither good or bad.

                            1. 3

                              Google and StackOverflow. I remember having stacks of technical books and parsing through results from Altavista + Hotbot + Yahoo! and the various product specific forums I used to have to read. It was not pleasant.

                              Meetups and other avenues that forced me outside of whatever industry I was working in.

                              20 years of experience. I am where I am now because I’ve kept learning and failing.

                              Doubling down on improving communication and mentoring skills. I am where I am now because I can understand what has to happen, communicate complicated issues to non-technical folks and also delegate out small stuff to let me focus on big stuff.

                              1. 3

                                Clojure, Spacemacs, CIDER and emacs for sure.

                                1. 1

                                  Do you find that spec is a good substitute for a type system?

                                  I’m working on a system that is growing larger, and I wonder if I’m going to start finding the dynamic language scaling problems soon? I’m looking into spec as a viable solution.

                                  1. 2

                                    Yes, but as always, it all depends on your objectives and problems.

                                    From a practical point of view, without getting into the dynamic vs. static analysis rabbit hole, I use spec for data validation, documentation generation and testing. I mostly write business software (accounting, billing, payments) but these are generic activities you can do for whatever domain you are working on (ie. space travel).

                                    I value the production-quality tools available (generative testing, database testing, data validation, Swagger generation, etc.) as these have saved me years of life.

                                    My takeaway is that software correctness is achieved by defining the rules by which you will test it and running those tests. It does not have anything to do with dynamic or static typing. Spec allows you to approach defining and testing software rules in a scientific way using declarative rules, generative testing and not a lot of lines of code.

                                2. 3

                                  I have become skeptic as to whether tools have much effect on how good I am at programming. Since I discovered pen and paper, nothing significantly better has come along. Playing with tools is fun, though, which might improve morale slightly.

                                  There’s a single thing, which is not really a tool, that I think made me immediately and permanently a better programmer. When I started dabble in the python community around 2000, there was a strong unit test culture. I’m convinced that unit tests are a silver bullet. Not only is my code significantly better because of them, but I have met many problems that I don’t believe I would have solved at all without a TDD approach. Being more talented would work too, but that’s not something I can really change by my own will.

                                  edit Also, another tool that’s not really a tool is the Inbox methodology of GTD. That is, offloading everything that you don’t need to keep in your head into a trusted system. I never managed/bothered to implement rest of GTD, but that part is a solid and easy one. I could name drop what I use currently, but I tend to change tools around this thing somewhat often, so I cannot yet say I’m married to any tool for this.

                                  1. 3

                                    Curiosity

                                    1. 2

                                      Stubbornness for me. I’ve learned so much from being 100% sure that my code or software should work the way I expect it to, and fighting with it and learning what’s actually going on behind the scenes and why its behaviour is actually expected.

                                    2. 2

                                      i3

                                      spaceship prompt

                                      intellij

                                      Now without these I would be way less structured and productive than I am. It’s not just about velocity of execution, but about abstracting away more complex common operations. Without these tools I wouldn’t have been able to identify such patterns and understand that they were worth their own abstractions.

                                      In a similar vein, also Talon Voice did the same, even though I don’t use it full-time but only when I have an RSI flare

                                      1. 2

                                        I’d love to hear an elaboration on why you feel spaceship prompt, specifically, made you a better programmer!

                                        1. 2

                                          well, it’s not necessarily spaceship prompt in itself but the vast amount of tools it brings together. Spaceship prompt has the added value that I don’t need to care about configuring them when I get a new laptop and basically allows easy access to a user experience on the terminal that would have been too much effort to set up without it or the few alternatives.

                                          That said, the smart search, all the info about python, git, aws and k8s are fundamental to take off my brain the effort of checking and double checking the state of my session to be sure of what is gonna happen. It takes away the need for a lot of commands that I see used regularly by other devs just to reassess the situation every few minutes.

                                      2. 2

                                        Shell scripting, find, parallel, rsync, awk and sed, as well as their structured-data counterparts like jq and xsv, have allowed me to prototype and explore processing pipelines for data extremely quickly and effectively, even for very large amounts of data. This has come in surprisingly handy surprisingly often.

                                        Next step is having the wisdom to know when to give up on them and write a Real Processing Pipeline with numpy or something.

                                        Edit: Having a basic knowledge and familiarity with relational databases such as PostgreSQL is similar. I’m far from an expert but being able to at least consider a problem and say “Oh, this is a job for a relational database” or “this is definitely NOT a job for a relational database” is very useful.

                                        1. 2

                                          Vim, tmux, bash, Linux and the whole set of GNU/Unix commands. Maybe not better, but definitely faster and more efficient.

                                          Also, the Rust programming language gave me great insight into deeper/lower level parts of coding.

                                          1. 2

                                            I’ve been willing to write a blog on this but here’s a short summary

                                            1. Python shell - I’ve been using ptpython and xonsh shells to quickly implement custom data processing pipes as I do a lot of data programming but even if you don’t, python as a shell language is amazing.
                                              Having access to real time play environment is huge boon for productivity.
                                              I also migrated to invoke instead of make which is really awesome.
                                            2. Bin - all of the tiny live scripts you sometimes write or your occasional workflows can be optimized and saved. There are a lot of great CLI libraries that make writing CLI scripts a complete breeze so set up your ~/bin with some CLI scripts. Putting yourself in the battle rather than using some mystic one liners from stackoverflow is a brilliant exercise and keeps you in shape.
                                            3. Vim - people talked about this already but editor is really important. Try neovim, kokoune or emacs. A lot of software like Jetbrains IDE’s have vim emulations too!
                                            4. Task managers - you can be an amazing programmer but if you’re unorganized you’ll end up wasting a lot of time. Efficiency is not a swear word your boss uses but your own personal respect for your limited time. I find TaskWarrior to be a brilliant tool for micro tasks. You can do really advanced stuff with it but you can also simple fire it up as micro tasks, see my full blog post on that
                                            5. Debugger and Tests - stop debugging with print statements. Take a weekend to learn your debugger. Personally I really recommend Pycharm’s debugger. On the other end writing tests and code linters can teach you a lot valuable programming info you otherwise wouldn’t stumble upon. I use pytest, flake8, pylint and mypy for python.
                                            6. Your Desktop - finally you need to be comfortable in your machine. Try different desktop environments or even window managers. Keyboard driven window managers like (i3, awesome, qtile) can really boost your navigation. Combine that with rofi as app launcher/context swittcher and a dropdown terminal and you can really feel like flash when you peek at your colleagues with windows or macos environments.

                                            I think those would be my top 6 tips but this is a bit of a blackhole subjects. Soon you start reading about philosophy, psychology and micro-dosing psychedelics, so it’s important to remember not to get lost here :D

                                            1. 2

                                              tmux is the one for me. It really changed the game for me. Multiplexing terminals has been so crucial in delineating my work. I wear a few different hats at work so its nice to just switch between terminals with ease.

                                              1. 2

                                                I find mosh coupled with tmux is invaluable.

                                                1. 1

                                                  I’ve never given mosh a proper look but reading about it now, this seems extremely valuable.

                                                2. 1

                                                  How do deal with scrolling and copy+paste?

                                                  1. 1

                                                    You can create bindings for tmux that allow you to do vim-style copy and paste.

                                                    Scrolling is built into tmux, ctrl + [ and then just page up / page down.

                                                    1. 1

                                                      Tmux has bindings for that. Iirc the default backscroll thingie is ctrl+[

                                                      1. 1

                                                        I set my terminal emu’s scroll back to 0 and have bindings for the scroll wheel events. Setting console scrollback to 0 is important cause otherwise the terminal will try to scroll too.

                                                    2. 2

                                                      AutoHotKey. I wrote about some of my experiences with it here.

                                                        1. 2
                                                          • Various C/C++ sanitizers: ASan, UBSan, etc
                                                          • Earlier than Sanitizers: Valgrind (not working on recent macOS due to restrictions incorporated by Apple, but works on Linux)
                                                          • Fuzzers: libfuzzer, honggfuzz

                                                          I think those are the most influential tools for me, they’ve shown me that it’s nearly impossible for me to write safe code using C++ or especially C, as well as there is no such thing as easy file format parsing code ;)

                                                          1. 2

                                                            This question, like others of its kind, depends largely on what you mean by the word ‘better’ and ‘programmer’.

                                                            Going by the definition of programmer as somebody who uses computators to solve problems, and that he does so by thinking really hard, and then translate his thoughts into instructions, I would say the most powerful tools are programming languages.

                                                            1. 1

                                                              Formal modeling and simulation, especially simple techniques like decision tables, will improve your productivity and also extend value to your customers in the form of reliable, correct software.

                                                              1. 1

                                                                my personal IDE is tmux, zsh, clang-format, grep, and vim. I avoid code generation and all those fancy indexing tools. I liken it to walking through a neighbourhood rather than driving past one because you really get a different feel for everything if you slow down and build a working knowledge.

                                                                Mastery of the unix tooling is super important. Learn your way around strace, wireshark, gdb, valgrind, netcat, htop, …

                                                                1. 1

                                                                  Strict type systems, I took a break from PHP around the time 7.0 came out to work on projects with C++, Java and C#. I now primarily program in PHP 7.4, Golang and occasionally Rust and will never go back to a dynamic type system if I can help it.

                                                                  With extension to this, having an IDE with live Static Analysis helped catch far more bugs that I care to admit.