1. 12

  2. 24

    So, I had occasion in the last couple of years to put food on the table and pay rent doing chatbots–a couple of projects, one paid and one basically used to help break the ice at bars when meeting people.

    The big thing people kinda lost about chatbots is that they are basically just really shitty command line applications for normies. I did most of that work with Microsoft Bot Builder and we later slathered on some NLP/AI with Microsoft LUIS.

    Setting them up to handle SMS, setting them up to handle Facebook/Slack/etc. is annoying. Running into arbitrary restrictions on how to actually phone back users unbidden is annoying. Finding ways of managing identities across multiple sources or through a third-party (say, via Amazon Cognito) is annoying.

    And for all of that, you get a user experience that is…okay? Something that can answer direct questions, usually, and give something back to the user. It’s not as good as just sending cards with options and having users click on them, something that Microsoft, Facebook, Slack, and others have all seemingly figured out.

    Anyways, adding support for actually pulling useful entities out of the text was not terrible, at least with LUIS support. Unfortunately, they had a lot of limits in how many phrases they knew how to decode, and basically no good interface for training–ended up writing what was probably at the time the world’s best (possibly only?) good Node library to talk to LUIS just so I could do batch work from an interactive shell.

    Also unfortunately, the accuracy of the results was never as compelling as just typing in exactly what you wanted in some DSL. Even for basic questions like “How many foos are in my bar?”, you can trip it up with “How many foos does my bar have?”, “In my bar, how many foos?”, “What is the number of foos in my bar?”, etc. I’m sure people have better solutions on that, but at the time it was not great.

    The fact that the chatbot stuff landed about the time that the ML nonsense was getting into full gear made things worse–people just started expecting way too much out of what were effectively limited interfaces. I think a similar thing is going to turn out to be the trend with Amazon Alexa tasks and the like–you really only want them for a couple of types of interactions, and beyond that, they suck.

    1. 4

      The big thing people kinda lost about chatbots is that they are basically just really shitty command line applications for normies.

      This is a really important point. I’ve made a similar (but not quite identical) claim myself: the ultimate goal of any conversational interface is to have the richness, expressivity, specificity, and time-saving features (such as terseness) of a modern unix shell. What conversational platforms add on top of that, at best, is a ‘dwim’ module (which, like every dwim module, rarely actually does what you mean unless you are very boring) and some awkward but shallowly friendly-looking syntactic sugar.

      A platform optimized exclusively for beginners needs to be well and truly mutable by those beginners if it’s going to be useful for real work, otherwise it’s just a toy. This means, in the context of bots, the ability to use the natural-language-like scripting language that you speak to the bots in to reprogram those bots or create new bots. In the context of voice assistants, it means being able to define new skills by speech alone, using a language that users can get the gist of without reading documentation.

      Of course, nobody does this – least of all entrepreneurs who are interested in making people pay for their web-api-to-sms bridges.


      Back when the hype was heavy, I wrote a couple articles about the chatbot-entrepreneur situation:



      It really bugged me, since I had been part of the botally community (which has more of a focus on artbots) for a couple years and had been writing irc bots for a decade at that point, and I felt like most of the people trying to make money on chatbots were ignorant of both history & the obvious limitations of the concept (which OP goes over in part).

      Bots are great, but the only kind of utility they’re good for extracting is the serindipitous kind.

      1. 1

        A great example of fails that people get frustrated with and make them avoid chatbots and stuff is that some “commands” are not very familiar. A very good example with you quoting Alexa is the “Can you play Nicky Minaj?” that answers “Didn’t find Nicky Minaj”, and to make it work you have to specifically tell on platform you want to use (spotify/…), having to browse online to find this just prove that it’s not useful enough at that point.

      2. 5

        What happened?

        The problem is AI-hard so people decided to just use forms instead.

        1. 4

          The best kind of bots right now are the integration ones. The bots that can post updates in a group chat or alert know something important has happened as well as the ones that extend the functionality of the IM platform itself like letting letting me search online content so I can post it to a group without having to switch to a browser to find it (Telegram does this great with inline bots that can display UI and function in groups without actually sending anything to the group until you select something to send)

          The other comment about bots being crappy CLI programs is also spot on. I don’t want to have to learn the commands of your bot to find a product or search the help page unless you have the kind of AI talent that google, apple and amazon have then don’t even bother trying to make your bot smart enough to work out commands without strictly defining them.

          Until AI significantly enhances to the point where it can really understand what we mean then chatbots will always be pretty subpar.

          1. 4

            despite the title, there’s a good design/ux discussion - i wish i could promulgate thinking like that at my current org.