1. 30
  1.  

  2. 8

    Why not just teach everyone to program? I hate this ideology and I always have. Human civilization progresses in part due to specialization, that we as a society don’t have to learn to do everything in order to keep things functional.

    Learning a GUI automation tool is learning to program, just in a fit for purpose language. Maybe even a wholly visual language.

    Learning “to program” is not the same as learning “to be a dev” any more than learning “to drive stick” is the same as learning “to build and rebuild transmissions”. They involve knowledge of some ofe the same principles, but specialization is still possible and important

    1. 3

      This was also my main thought. I think it is useful to acknowledge that this is programming. I also think that it is important to acknowledge that it is useful to have a “language” for this audience. I think it is also interesting to note that some, but not all of the users will become “devs”. Many users will be happy working with this level of automation forever and that is fine.

    2. 6

      In my day job I write automated tests for web applications and APIs using selenium so a lot of what was said about the web being a bad target for automation resonates with me.

      Assigning structure to elements of a web page and giving it a structure isn’t actually that difficult if you add id attributes to elements on the page. It just seems so rare that people are interacting with their page HTML at a level where they think about that now. Every time I see a HTML mess with no structure/targetable elements it’s almost always because there is a library/framework abstracting out the process of writing HTML for the developers.

      I’ll also say that the distinction between automation and programming made in this article seems strange to me. Automating any task is programming and the people doing it are programmers in some capacity.

      1. 4

        I really like the idea of a facility for indicating that certain page elements should be considered “stable”. Even if the appearance changed, an automation tool wouldn’t need to care about that, so it actually seems pretty feasible. I suspect that if an existing automation (RPA) platform came up with a spec and started doing this, quite a few “enterprise” software products would go along with it as a (basically) free way to support their power users.

        1. 3

          While I get the desire for this, I don’t think this is the right way. Making GUI automation tools become hard to maintain, feature creeped, and just not what people wanted originally.

          Instead of better GUI automation tools, we need better education of non-programmers to programming and using more technical tools. Stop dumbing things down, and get people to learn things.

          1. 12

            Instead of better GUI automation tools, we need better education of non-programmers to programming and using more technical tools. Stop dumbing things down, and get people to learn things.

            You’re vastly underestimating how difficult this is.

            1. 10

              we need better education of non-programmers to programming and using more technical tools. Stop dumbing things down, and get people to learn things.

              I’m not sure this is realistic. You’re basically asking people to learn a second job on top of their first. As a compiler engineer, I’m very happy I don’t have to learn all the hard stuff the operating system has to do in order to allow me to, say, write to the disk. And I’m sure the application developers that use my compiler are happy they don’t have to learn what the compiler is doing in order to turn their code into an executable.

              Instead, we’re all happy to rely on a bunch of promises made to each other: “hey, here’s this interface and what it should do”. GUI automation tools are just a bunch of promises, no different from the ones my operating system is making to me. The reason GUI automation tools have a bad rap currently is that they’re unreliable, being at the very top of the stack of tools people will use, they will be impacted by bugs and changes in all layers below them. But if this problem can be solved, it should be, because it will enable all the people who already have a job but might benefit from automation to spend a minimal amount of time making their lives better.

              1. 2

                we need better education of non-programmers to programming and using more technical tools

                Something I say a lot is ‘“talking to computers” is 21st century literacy.’ Like reading and writing, being able to think in an organised and systematic fashion, and to direct a computer to execute algorithms, are fundamental to flourishing in the information economy.

                Yes, like reading and writing, not everyone is capable, and like with reading and writing, those unfortunates have a disability. Those who have the ability, though, should be properly trained.

                1. 13

                  I’m used to software developers having ridiculous misunderstandings how the difficulty or applicability of learning to program well, but “anybody who can’t learn this has a disability” is a new one to me.

                  1. 1

                    I do not think that is a terribly charitable reading of what I wrote.

                    Our society expects basic literacy from its members: the ability to write a grocery list, not the ability to write like Shakespeare. Likewise, I think that we can expect basic programming literacy: ‘being able to think in an organised and systematic fashion, and to direct a computer to execute [simple>] algorithms.’

                    I am not arguing that folks who cannot write their own shaders are disabled; I am arguing that folks who, after education, try to add ‘December’ and and a JPEG of orangutan and expect a meaningful result may be.

                  2. 2

                    I would argue understanding how to proceed with a computer that misbehaves, including reading error messages, is important. Ordering computers about, however, is not, not for most people.

                2. 3

                  I think it’s worth specifying stuff like AHK/Selenium where you’re either sending keypresses or working on selectors, or the AppleScript/VBA model of “control the graphical application(s) through specific exposed functions and objects” when you want to push this. Both have their merits for automation and very different tradeoffs. My preference is towards the latter.

                  1. 1

                    I also have a preference towards the latter, and I think it’s because really the former is an imperfect proxy for the latter. It’s much easier and natural to maintain the latter form of “hooks”.

                  2. 2

                    On Windows we don’t need better GUI automation tools. We have bunch of awesome ones. Personally I prefer AutoHotKey.

                    Here, its not only GUI, but browser automation combined with GUI automation which is more problematic since browsers do not use system GUIs.

                    1. 5

                      The article is about GUI-based automation tools, not tools for automating GUI interactions. So AHK doesn’t count here.

                    2. 2

                      Ideas in this space will continue to fail because what users are lacking isn’t programming skills, it’s a mental model of how machines work which you learn by acquiring programming skills. The solutions naive users come up with are a disastrous mess that only get worse with time no matter the tools. And there’s an answer to this that the author dismisses out of hand:

                      Why not just teach everyone to program?

                      I hate this ideology and I always have. Human civilization progresses in part due to specialization, that we as a society don’t have to learn to do everything in order to keep things functional.

                      What’s the point in teaching people to count if we have calculators? Why bother learning anything about biology and your body, your doctor will figure out what’s up anyway?

                      There are some basic prerequisites that you need to have in order to be a productive member of society that can hold down a professional job. Computational thinking is becoming one of those prerequisites, there’s no way around this.

                      1. 3

                        Because learning how to parse the garbage that is a modern websites DOM isn’t computational thinking and it isn’t programming, it’s “sorting through someone else’s garbage”.

                        1. 1

                          What’s the point in teaching people to count if we have calculators? Why bother learning anything about biology and your body, your doctor will figure out what’s up anyway?

                          I never bothered to learn how to do surgery and am still a productive member of society.

                          1. 2

                            If your job involved doing something with surgical tools and you had no idea about how they’re used, that would be pretty terrifying!

                            Most professional jobs involve dealing with computers at many levels. There’s no excuse for not teaching computer literacy skills.

                            1. 4

                              Computer literacy does not necessarily mean programming in the same way learning to drive doesn’t mean being able to change an engine.

                              1. 2

                                But what what does it mean to be “computer literate?”

                                1. 2

                                  I would say it’s having the “mental model of how machines work” that OP described, and therefore being able to navigate your way around the system. I also agree that programming is probably the best way to acquire this mental model.

                                  That being said, there’s a huge difference between having some exposure to programming and having abilities that we at Lobsters would consider “competent programming”. Say every high school includes one required programming course: I would argue that this will not teach users enough programming to be able to automate things in today’s environment, but is still sufficient to give people this mental model. A lot of the reason for this difference is because, due to the lack of platform APIs like e.g. HyperCard or Automator/AppleScript to organize things around, today automation is heavily dependant not just on your fundamental ability to program but the depth of your understanding of the specific system you’re trying to automate. E.g. say you learn Python and have a good mental model for how things like control flow work. This will not be sufficient for automating the web because you don’t know the specifics of how CSS and HTML work. There isn’t a level of abstraction to help you out.

                                  tl;dr while I totally agree with OP about the need to have a basic mental model for how the machine works - just like everyone should know e.g. a little bit about math, or science, or whatever, even if that’s not your specialty - and that you probably need to do some programming to get it, I don’t think it follows that this is a sufficient answer for the problem raised in the submission. And it shouldn’t be, because users’ time is valuable, they usually don’t have time to get past that basic model, and we should be doing better for them.

                              2. 2

                                You presumably use electricity at work; do you need to be familiar with generation and wiring standards?

                          2. 1

                            From my short experience, applications created by non programmers become a liability and not an asset. Specifically I have nightmares with Salesforce becoming a complete mess and pipedrive automations that can slow a company to a halt…

                            1. 6

                              This is a growth/scaling problem. In many cases, the ability of a user to make the computer do what they need is the difference between something getting done, or not getting done (or at least being done with hours of mundane labour instead of using the computer well).

                              Yes, when something taped together grows in importance to the point that a dev is hired to fix or replace it, it can be bad. Same as when a bunch of Junior dev code suddenly is running the core of a product. The trick is to either (a) realise this is coming early enough to invest before it’s too bad or (b) realise that failing to catch these points in time is life, and is why you can get paid so much to fix them after the fact.

                              All stopping people from using their computers will do is result in less automation and more pain. They’re not gonna shell out for a dev for every little thing, and honestly they should not need to.

                            2. 1

                              I could imagine that a competitor to Qt and Electron could win if it had built in capabilities for GUI automation from the beginning because not only would it make it easier for users to automate tasks, it would make it easier to implement completely automatic testing of GUI’s.

                              However, if such a library appears, I would also like it to have:

                              • Accessibility
                              • Localization
                              • Internationalization
                              • Animation
                              • Multi-touch
                              • Pressure (trackpads)
                              • Input Methods

                              from the beginning. (See this blog post.)

                              Looking at that list, I think I know why no competitors have appeared…