1. 14

    As someone who does ML in C++ and built a production, petabyte level, computer vision system, and interviewed many ML people and the reason is pretty obvious. ML people can’t code. seriously , many are math people, who are academic, and have very little experience building production systems. It’s not their fault, it’s not what they trained for or interested in. These high level apis exist to address their needs.

    1. 5

      I want to emphasize one thing that might make my previous comment more clear. The hard part of ML isn’t programming.

      The hard part of ML is data collection, feature selection, and algorithm construction.

      The only part where programming matters is building the training software and execution software. However most ML people care about the former, not the latter.

      1. 2

        ML has certainly been growing fast. I see this as mirroring what’s happening in CS in general nowadays, with ML simply being the foremost frontier, and a hype word to boot.

        However, I would temper your statement with an aspect of what u/zxtx said. Even those who are capable of building a project in a low-level language, won’t always want to. It’s nice to be able to dodge the boilerplate while hacking on something new. And that goes for those who understand the low-level stuff, too. So I’m not too surprised that people aren’t using low-level languages for everyday ML development. (Libraries are another story, of course.)

        Perhaps you know more about how to write ML projects in C++ without getting mired in boilerplate, though. Was this ever a problem for you? Or does low-level boilerplate generally not get in your way?

        1. 1

          Great question. I am not mired in boilerplate because C++ is a high level language. I use it because the transition from prototype to production is very smooth and natural.

          I think ultimately it’s not fashionable to learn. I’m finding most younger programmers simply don’t have proficiency in it. Meaning they haven’t developed the muscle memory so it feels slower. The computer field is all about pop culture , which I believe is the actual answer to OP’s question now that I think about it. In other words, python and R are fashionable and that’s why they are being used.

          1. 4

            I think ultimately it’s not fashionable to learn

            It’s not just fashion: it’s an incredibly complicated language. It’s so complicated that Edison Design Group does the C++ front-ends for most commercial suppliers just because they know they’ll screw it up. Some C++ alternatives are easier to learn or provide extra benefits for extra efforts.

            On top of that, it had really slow compiles compared to almost any language I was using when considering C++. That breaks developer’s mental state of flow. To test the problem, I mocked up some of the same constructs in language designed for fast compiles with it speeding way up. It was clear C++ had fundamental, design weaknesses. Designers of D language confirmed my intuition with design choices that let it compile fast despite many features and being C-like in style.

            1. 1

              It’s true the compile times are slow , but it doesn’t kill flow because you don’t need to compile while you program, only when you want to run and test. I would argue any dev style where you quickly switch between running and coding slows you down and takes you out of flow anyway.

              In regards to it being complicated, this is true. However c++17 is much more beginner friendly. Even though 1980s C++ was arguably harder to learn than today, millions learned it anyway because of fashion. Don’t underestimate the power of fashion.

              And lastly , D has it’s own design flaws like introducing garbage collection. Why in a language that has RAII do you need or want garbage collection ? Nobody writing modern C++ worries about leaking memory.

              1. 1

                “Don’t underestimate the power of fashion.”

                You just said it’s out of fashion. So, it needs to be easier to learn and more advantageous than languages in fashion. I’m not sure that’s the case. Hardest comparison being it vs Rust where I don’t know which will come out ahead for newcomers. I think reduced temporal errors is a big motivator to get through complexity.

                “And lastly , D has it’s own design flaws like introducing garbage collection.”

                You can use D without garbage collection. Article here. The Wirth languages all let you do that, too, with a keyword indicating the module was unsafe. So, they defaulted on safest option with developer turning it off when necessary. Ada made GC’s optional with defaulting on unsafe (memory fuzzy) since real-time w/ no dynamic allocation was most common usage. There are implementations of reference counting for it and a RAII-like thing called controlled types per some Ada folks on a forum.

                So, even for C++ alternatives with garbage collection, those targeting the system space don’t mandate it. Feel free to turn it off using other methods like unsafe, memory pools, ref counting, and so on.

                1. 2

                  Sorry, I had a very hard time groking your response. What I meant was that python and R are used for ML, not because of technical reasons, but because it’s fashionable. There is social capital behind those tools now. C++ was fashionable late 80s to late 90s in programming (not ML). Back then lisp and friends were popular for ML!

                  Do you mind clarifying your response about fashion ?

                  In regards to D, I still think garbage collection, even though it’s optional , is a design flaw. It was such a flaw that if you turned it off, you could not use the standard library, so they were forced to write a new one.

                  C++ is such a well designed language that you can do pretty much any kind of programming (generic, OOP, functional, structural, actor) with it and it’s still being updated and improved without compromising backwards compatibility. Bjarne is amazing. By this time, most language designers go off and create a new language, but not Bjarne. I would argue that’s why he is one of the greatest language designers ever. He was able to create a language that has never stopped improving.

                  Now WebAssembly is even getting we developers interested in C++ again!

                  1. 2

                    I was agreeing it went out of fashion. I dont know about young folks seeing as it happened during push by managers of Java and C#. They kept getting faster, too. Even stuff like Python replaced it for prototyping, sometimes production. Now, there’s C/C++ alternatives with compelling benefits with at least one massively popular. The young crowd is all over this stuff for jobs and/or fun depending on language.

                    So, I just dont see people going with it a lot in the future past the social inertia and optimized tooling that causes many to default on it. The language improvements recently have been great, though. I liked reading Bjarne’s papers, too, since the analyses and tradeoffs were really interesting. Hell, I even found a web, application framework using it.

                2. 1

                  I would argue any dev style where you quickly switch between running and coding slows you down and takes you out of flow anyway.

                  I have to disagree. REPL-driven development has only grown more & more useful over time. Now, when I say this, you may think of languages with high overhead such as Python and Clojure, and cringe. But nowadays you can get this affordance in a language that also leaves room for efficient compilation, such as Haskell. And don’t forget that even Python has tools for compiling to efficient code.

                  If you feel like having your mind opened on this matter, Bret Victor has done some very interesting work liberating coding from the “staring at a wall of text” paradigm. I think we’re all bettered by this type of work, but perhaps there’s something to be said for keeping the old, mature standbys in close proximity.

                  1. 2

                    Sorry, just to clarify. I LOVE repl development! Bret Victors work is amazing. What I mean is anything that takes you out of the editor. For example , if you have to change windows to ccompile and run.

                    REPLs are completely part of the editor and live coding systems don’t take you out of the flow. But if you need to switch out of the editor and run by hand, then it takes you out of flow because it’s a context switch.

                    1. 2

                      100% agreed. Fast compilation times can’t fix a crappy development cycle.

                      1. 2

                        I think the worst example of anti-flow programming is TDD. A REPL is infinitely better.

                        1. 1

                          Do the proponents of TDD knock REPLs? In my opinion, REPL-driven is just the next logical step in TDD’s progression.

                          1. 2

                            no, I’m knocking TDD ;-)

              2. 3

                How do you feel about platforms like https://onnx.ai/ which make it easy to write a model in a language like python, but have it deployed into a production system likely written in C++?

                1. 2

                  I think they are great but don’t go far enough. because we are entering a new paradigm where we write programs that write programs. People need to go further and write a DSL and not just a wrapper in python. I think a visual language where you connect high level blocks into a computation graph would be wonderful. And then feed it data and have it learn the parameters.

                  1. 2

                    So intuitively a DSL is the correct approach, but as can be seen with systems like Tensorflow it leads to these impedance mismatches with the host language. This mismatch slows people down and ultimately leads them systems that just try to extend the host language like pytorch.

                    1. 1

                      I guess when I think of DSL’s, I don’t think of host languages. I’m thinking more about languages that exist by themselves specific to a domain. In other words, there isn’t a host language like in Tensoflow.

                      1. 1

                        Well for Tensorflow, I mean something like python as the host language.

                        1. 1

                          wow, rereading my last sentence I can see how it had the opposite meaning than I intended. I meant I was thinking of DSLs without a host language , unlike tensorflow.

            2. 2

              Would you mind shedding some light on what this petabyte level computer vision system is? I’m very curious!

              1. 3

                It was a project within HERE maps to identify road features and signs to help automate map creation. Last time I was there it processed dozens of petabytes of LiDAR and imagery data from over 30 countries . It’s been a couple years so can’t tell you where it’s at today.

            1. 4

              Several of my peers at Arcadia.io’s Pittsburgh office are hiring.

              If you see “data ingestion pipeline” in any other job description, that’s my team. I’m building a pipeline right now for a start sometime later in Q1 next year on my team doing stuff in Scala, Groovy, Ruby, and Rust.

              1. 1

                I don’t suppose you’re looking for Remote engineers or sponsor Visas? Couldn’t find anything in the posting.

                1. 1

                  We’d consider remote for someone who fits the position perfectly or is willing to move near Pittsburgh or Boston within a few months.

                  My team is actually “local remote”: we all live in the greater Pittsburgh area but only go into the office once a week. However, the other teams are the opposite: they work in the office daily but remotely one day per week.

                  We do visa sponsorship for the right candidate.

                  1. 2

                    Sounds good. I’ll give it a shot :-)

              1. 17

                Pretty much the same experience. I went through 5 rounds with a company (all online, 1 to 1.5 hrs after my work hours, via Skype). All of them involved doing live coding (algorithms, rate limiters, multithreading problems and so on), extensive system design questions, quite a bit of time spent on work experience, past projects, scalability, microservices and everything you can think of. After every round I was given the feedback that they like me and still want to go forward.

                At the end, they decided they wanted to do one more round, in which I took too long to answer a linked list question and was rejected.

                Not only was this a mental exhaustion, but it wasted so much time back and forth b/w me and the consultant, scheduling and rescheduling calls because we were in different timezones. Quite a lot of effort.

                1. 3

                  After all of that they gave you a linked list question at the end? And then failed you on it? Was there any chance you would be implementing linked lists in that job?

                  On the face of it that all seems pretty ridiculous.

                  1. 1

                    Nah. I would’ve most likely worked on infrastructure, golang, cloud, etc etc.

                    I suspect it was simply - Engineer 1 took the interview the way they wanted it, Engineer 2 most likely didn’t collect any feedback from the previous round, and interviewed their way. In the end someone didn’t like something about me and all 5 rounds of progression wasn’t weight against the couple things I fell short on.

                    It’s fine I guess, a lesson learned. :-)

                1. 3

                  Working on a Google codejam problem. Algorithms/Data Structures are something I know well enough, but it’s always been a sticky point that I’m not super confident when a new problem comes my way. Aiming to slowly build up those skills.

                  1. 2

                    At office taking lots and lots of interviews. Sigh.

                    1. 1

                      Are there good lectures available for Distributed Systems somewhere? I find most of the classes are usually just about reading lots of papers. While that’s great and I’m doing it, it would be really fantastic to have an experienced instructor drilling down the principles and practical examples of distributed systems. Thanks :)

                      1. 1

                        Someone in #go-nuts linked me this excellent video from Linux Conf Australia: Introduction to go by Mark Smith from Dropbox. :)

                        1. 8

                          Implement a simple UNIX shell. Basically, for each command you have to fork, exec, and then wait for the process to finish. You’ll learn a lot about UNIX systems programming and process management.

                          1. 3

                            This is a great way for people to learn about syscalls and unix in general. Once there’s a shell that can handle execution of single commands, it can be interesting to implement pipes and file redirection with dup2, or shell metacharacters like $! with the wait() syscall. Around this time it’s also a good chance to learn about zombies and orphans, and how double forking can be used to daemonize a process.

                            1. 1

                              s/$!/$? :)

                            2. 2

                              I would like to learn this! Come to think of it, I would love for a good tutorial/course in systems programming. I’ve looked around and haven’t had success in finding anything useful. If anyone has any links, please post! :)

                              1. 3

                                I found working through “Advanced Programming in the Unix Environment” (and doing the exercises) to be a good start; there’s also “The Linux Programming Interface”. It’ll give you a good foundation.

                                Also, one of the things I like to do when learning a new language is to start writing the Unix utilities in them.

                                1. 2

                                  Advanced Linux Programming is a good reference for the different system calls you will have to use.

                                  I wrote a shell as the first assignment in my Operating Systems course in university. The instructions for that particular assignment are still online on the course website.

                                  1. 1

                                    To piggyback: I think Operating Systems courses should be required in all universities teaching Computer Science, there is like a treasure chest of knowledge that is hard to grasp at first - but the rewards are so worth it.

                                  2. 1

                                    Computer Systems: A Programmer’s Perspective, which was the reference for my University’s intro to systems class is one of the best textbooks I’ve ever read; the labs that come with it (many of which can be found on the internet) are excellent as well.