1. 16

Hi,

I often find what for me it’s easier to write script in js instead of bash. But I struggle with exec of other bin. Sometimes just use child_process, or install npm execa.

Do you write scripts in js? What do you use for it? What is missing?

Thanks.

PS I understand what nodejs/js is not for everybody. But I like js and nodejs/npm is okay for me.

I mostly interested in answers from people really using nodejs for scripting.

  1.  

  2. 40

    No, literally never.

    I only have nodejs installed on machines where I absolutely need it, that might mostly be because I don’t like js and the whole nodejs ecosystem, but it’s a fact. None of my personal projects rely on it and if I have to use it it’s wrapped in a docker container (with rare exceptions) - but my main argument is actually the (imho) horrible dependency handling where I either need to install globally or have the huge node_modules folder somewhere.

    I dislike bash scripts but at least I can write them to only rely on binaries that I can count on being available (i.e not literally 0 dependencies, but in reality it feels like it).

    I have experimented in writing lua scripts instead of bash with my own 20 loc-c-script runner binary, but I gave that up as well.

    1. 4

      The dependencies situation is what finally drove me to embrace Bash for all the things instead of Javascript–even though I find JS to be my main quick hack tool that I reach for.

      1. 2

        my main argument is actually the (imho) horrible dependency handling where I either need to install globally or have the huge node_modules folder somewhere

        You shouldn’t ever need to install globally unless you want to. So is size the main concern here? node_modules can definitely get super big, but in my experience typically only for application-sized projects, not for scripting use cases. Something I like about npm is the fact that you almost never have to worry about implicit peer dependencies, or shared transitive dependencies at different versions, which isn’t the case with bash. I think this is where much of the node_modules size comes from.

        1. 4

          the OP talked about “scripting” - so I instantly thought of bash - but even with a python script I maybe have 1-5 dependencies that I can somehow finagle into a virtualenv, and it never grows to half a gigabyte.

          I’m not saying anyone shouldn’t use (node)js, but I don’t see a single advantage over literally anything for me.

          1. 2

            For what it’s worth I basically agree. Unless my project is JS, I’m not reaching for it over python as a sh alternative.

        2. 1

          my main argument is actually the (imho) horrible dependency handling where I either need to install globally or have the huge node_modules folder somewhere.

          I hear you on the huge dependencies. Node is famous for this. :)

          Aside from that, you can install nodejs global packages into your home directory, much like you can install perl, python, or ruby dependencies locally. It’s just a matter of configuring npm (or yarn) to do so ; and as with other package managers, the dependencies also need to be somewhere on your disk.

          1. 2

            that’s what I meant, but maybe I’m wrong, I thought npm install -g <pkg> as non-root would install “global per user” but that’s exactly what I don’t want, then my $HOME suddenly becomes a non-defined ball of dependencies.

            I think the real thing is that the bash script lives there, on its own, for years or decades and doesn’t have to be updated ever, just copied from machine to machine (or checked into my dotfiles). This is simply a very different use case than real “software” which I “install”.

        3. 13

          Yes, all the time! #!/usr/bin/env node fits nicely at the top of a js file, and process args or stdin/stdout streams work great.

          Lots of the time I use it for data wrangling and json ETL jobs that I’m playing around with. Sometimes they graduate into being actual apps instead of throw away scripts.

          My latest was I wrote a page listener to book myself a vaccine appointment:

          #!/usr/bin/env node
          var page = "https://www.monroecounty.gov/health-covid19-vaccine";
          var selector = "#block-system-main";
          var seconds = 15;
          var warning = "It's a ready"
          
          var child_process = require("child_process");
          var cheerio = require("cheerio");
          var fetchUrl = require("fetch").fetchUrl;
          
          var last = null;
          
          var check = function() {
              fetchUrl(page, function(error, meta, body){
                  var $ = cheerio.load(body.toString());
                  var curr = $(selector).html();
                  if (last!=curr) {
                      //Page has changed!  Do something!
                      var command = `say "${warning}"`;
                      console.log("Changed!",new Date());
                      child_process.exec(command);
          
                  } else {
                      console.log("No change",new Date());
                  }
                  last=curr;
                  setTimeout(check,seconds*1000);
              });
          }
          
          check();
          
          1. 5

            At some point in time I was favoring Python over bash for scripts. But the gain in expressivity was counter-balanced by the strength of bash for piping stuffs together, and the shortness/simplicity for doing basic file stuffs, and the whole vocabulary of shell executables. In the end, I gained a lot in embracing bash.

            And when things go more complex I slowly change language: Bash -> Perl -> Python -> Compiled Language.

            How would you sell NodeJs for Linux scripting to others like me. What are the major pros ? Is your main point (which would be totally understandable and acceptable) that you are fluent in Js and want to use this over harder to read bash.

            If your convincing I’d gladly give it a try to NodeJs linux scripting. Maybe there are field where it shines. Like retrieving json over the internet and parsing it far easily than other language for example.

            1. 7

              Well it’s mostly preference and I’m a HUGE advocate of “use what you like and ignore the zealots”. It’s so hard to espouse pros and cons in a general sense for everyone, because everyone is doing different stuff in different styles.
              That being said, here’s why I personally like to use it as a shell scripting language:

              • I write python too, and use bash, and can sling perl if I have too, but js feels like the best balance between being concise and powerful (for my needs)
              • node is everywhere now (or really easy to get and I know how to get it in seconds on any OS).
              • Say what you will about node_modules, but node coupled with npm or yarn IMO set the standard in ease of using and writing packages. I also write python and writing a package for pip install is way way way more annoying compared to npm.
              • package.json is a single file that can do everything, and I loathe config split across files in other langs for tiny tiny scripts that do one thing only.
              • JSON is literally javascript, so messing with JSON in javascript is natural. Here’s how to pretty print a json file: process.stdout.write(JSON.stringify(JSON.parse(require('fs').readFileSync('myfile.json')),null,2))
              • Everyone complains about node_modules size but this ain’t webpack - it’s just a module or two. I’m a big fan of cheerio and lines. node-fetch is also quite small and very powerful

              Probably more reasons but that’s good enough :)

            2. 2

              Hello const and modules.

              1. 2

                nah ES5 for life!

              2. 1

                #!/usr/bin/env node fits nicely at the top of a js file

                At which point it’s no longer a valid Javascript file? But I suppose nodejs allows it?

                1. 4

                  https://github.com/tc39/proposal-hashbang

                  Stage 3 proposal. every engine I know of supports it.

                  EDIT: hell I just checked, even GNOME’s gjs supports it.

                  1. 2

                    Yes node allows it and I like this because it makes it clear that its meant to be a command line script and not a module to include, and allows for the familiar

                    $>./script.js

                    versus needing to invoke node explicitly:

                    $>node script.js

                2. 8

                  I think using other scripting languages instead of the shell is pretty common, but JS is an unusual choice, it’s usually either Perl or Python.

                  Now, on Windows, JavaScript is more often used for scripts instead of batchfiles or VBScript; in fact, it’s the same runtime as VBS. PowerShell kinda displaced that though.

                  1. 5

                    On macOS, JavaScript is also available as an alternative to AppleScript and also has access to all of the Cocoa frameworks, which makes it very powerful for scripting there.

                  2. 8

                    If something is too complicated for bash, I usually use Ruby and really sloppily call subprocesses using backticks. I’m not a fan of using Ruby for bigger applications (or Rails) because I find that it gets confusing, but for scripts that are more sophisticated than what I’m comfortable writing in bash, Ruby really hits a sweet spot for me.

                    I try to avoid node for everything now. Managing node installs and dependencies is too cumbersome for scripting, and the builtin facilities for calling subprocesses is lacking ergonomically. If a script needs a bunch of language-specific dependencies, I’m not a huge fan of that for one-off system scripting. For applications: keeping things up to date, dealing with dependency brittleness, changes in the ecosystem … that stuff cancels out any productivity benefit I’d get over using Go, so for applications and anything more complicated I use Go, and for little one-off scripts I use Bash or Ruby.

                    1. 2

                      Ruby backticks are one of the many things I miss about that language. I’m sure if I start using it again I’ll be reminded of all the things it did wrong, but man, it also did so much right.

                      1. 4

                        Strongly disagree that ruby backticks are right. They are annoyingly close to being right, but using the shell (rather spawning the process yourself) is wrong (for portability, security, and performance reasons).

                        Julia backticks are right though:

                        1. 2

                          it’s all true. Using backticks in Ruby is just absolutely loaded with silly pitfalls.

                    2. 7

                      Yes. But never nodejs. Deno is perfect for this.

                      For instance, in a new machine you just need to install a single binary and execute the script straight from your repo (e.g. https://raw.githubusercontent.com/ruivieira/scripts/foo.ts). It will even fetch dependencies automatically.

                      To execute processes I use https://deno.land/manual/examples/subprocess

                      1. 1

                        Your first link probably requires authentication.

                      2. 5

                        I write most of my day to day code in TypeScript/Node. I regularly use it for “scripting” tasks; we have hundreds of utility scripts at Notion written in TypeScript. I’m also fairly fluent in bash, I would rate myself in the top 5% at both Notion and Airbnb (my previous workplace).

                        Subprocesses and shelling out in Nodejs are a huge pain. The standard libraries of most “scripting” programming languages are usually a little bit wrong on the details, but Node’s is more frustrating and wrong than most. Execa makes the situation better but is unfortunately not very POSIXly correct - it papers over Node deficiencies and doesn’t handle pipes or pipelines very well. But, if you’re not used to POSIX semantics this probably won’t be annoying to you.

                        Conversely, bash struggles with complex transformations (most users would struggle with a group-by), network connections, error handling, and stateful or long-running processes - all things that are better in a more powerful scripting language.

                        My suggestion is to make a scripts “monorepo” where you can build up a library of functions that solve the problems you face with Node scripting — for me the big ones are arg parsing and shelling out. Using a monorepo means you don’t need to mess around with package.json for each new script you write, and you can spend some time once to figure out a build and distribution system if you need to execute one of your scripts on a remote host. That should defray the issues with Node. We f Node is too annoying for a specific task, you can write that part in sh or bash (and call down to it from Node).

                        1. 2

                          My suggestion is to make a scripts “monorepo” where you can build up a library of functions that solve the problems you face with Node scripting — for me the big ones are arg parsing and shelling out.

                          I’ve thought about doing something like this for the repl in python. It’s pretty deficient compared to some other environments and could benefit from a new prelude. E.g., clearing the screen is a bit too much work. Though something like xonsh might do it for me.

                        2. 4

                          I never use NodeJS for scripting, either Python or Babashka.

                          1. 4

                            Whenever I get to the point in a script where I want to do interesting things with data that isn’t plaintext, I jump ship from bash.

                            1. 5

                              “This should be small, I’ll do it in bash”

                              “Oh god, why did I use bash again?”

                              1. 1

                                For some kinds of non-plaintext data I don’t mind sticking with bash. For example, using jq in pipes is reasonably painless. The thing that really gets me is no floating point arithmetic, not even as a bash extension. Some of my scripts end up with stuff like this in them, which is not too satisfying:

                                SOME_FLOAT=0.5
                                ANOTHER_FLOAT=0.2
                                RATIO=$(echo "scale=3 ; ${SOME_FLOAT}/${ANOTHER_FLOAT}" | bc -l)
                                
                              2. 4

                                Coming back to this discussion 10h later, and seeing a few interesting responses (and I also took the question as a yes/no and not more of an “If you use it, please answer”) - I also have a question for you:

                                Do you have any scripts that you have used for a long period of time and on several machines? Like, say, 5 years - not just a few months. How often did you have to update them? Did you update them or do you keep old node versions around? Is this even a problem (because I don’t know what you’re doing in your scripts, maybe you’re actually hitting the stable parts of the API?)

                                1. 3

                                  I have one script for ~7y. Still working, has no deps. I think this is the key.

                                2. 3

                                  I don’t use it for scripting, but I had a couple scripts done with deno. The reason for deno in that case was that it was a single binary, so it was easy to move around. These days, when I need scripts I mostly using either bash or AppleScript to be honest. AppleScript is terribly underrated.

                                  1. 3

                                    This post seems like two separate generations of programmers. I wonder how many years back you would’ve gotten the same kinds of answers for python or perl?

                                    To answer your question, I don’t because I use bash and sometimes python. Deno has been on my short list for using in order to replace python scripting though.

                                    1. 3

                                      I sometimes use Lua for this

                                      1. 1

                                        Same here. I normally prefer Python, but the Lua runtime is so quick to start that I use it instead for simple stuff, like a blocking hook in my dhcp renewal process that I just want to be fast, and I don’t trust my shell scripting enough to bother.

                                      2. 3

                                        I [am] mostly interested in answers from people really using nodejs for scripting

                                        Given the atmosphere of Lobsters, it would have been good to lead with this.

                                        I haven’t really used nodejs for scripting myself, in part because of a second-hand bias against node I’d picked up from the rest of the internet, that I’m slowly working my way out of as I embrace the front end more.

                                        Probably the two main issues with using node for scripting types things are callback hell, and the dependency story. Promises and async mostly resolve callback hell, from what I can see. The dependency story is something that node shares with Ruby and Python, if with slightly different flavors. Node, and the JS ecosystem in general, are in a very different space from at lot of the sysadmin/scripty types of things, for better and for worse.

                                        One other issue that node specifically has over other scripting languages is that node is rarely quickest way to write a particular script when it comes to sysadmin or tweaking types of things, once you know other languages. Bash or Ruby will often be shorter, Python often has a library for a given thing, and Go executables can be copied around nearly anywhere. Nim and Janet have been my scripting/utility languages of choice of late.

                                        If you already have a lot of things in Node, it makes sense to attach scripts to that existing infra, if you don’t, then other options may well make more sense.

                                        That being said, Deno seems to stand as a counterpoint on the dependency front, being able to package a JS script into an executable.

                                        1. 7

                                          I avoid node like the plague, personally.

                                          1. 19

                                            Why people keep posting stuff like this? What’s the point? To show your status? Desire to kick somebody?

                                            There lots of people loving and using nodejs, and js in general everyday. So why come and post comments like this?

                                            1. 6

                                              It’s a common attitude here. They could have provided info (e.g. not wanting to deal with JS language versions / node versions / not wanting to rely on a node installation in certain environments, not finding JS expressive for their use cases), but without justification it’s indistinguishable from the whole “web bad” meming.

                                              1. 7

                                                Because it is objectly a badly designed piece of software. Forcing all IO operations to be async, all thrown in the same event loop and the results catched in callbacks has very well known problems and it is all in all a bad idea as it was proven decades before node was first released.

                                                Other than that, it is a command line interpreter for js, which is a language in the same breed as most of popular languages. I don’t see a strong case for js as a language instead of perl, python, PHP, ruby, etc. Perhaps something like coffeescript would be more aliciating in that respect. But its popularity has peaked long ago.

                                                Js command line interpreters did exist before node and not many cared about them. Windows came with a built in JScript interpreter for more than 20 years now. And there are other implementations if you look them up.

                                                But to the point of this topic… I don’t quite get the rationale of people saying “when it gets too complicated to shellscript”. Shellscript is literally optimized to call external programs and easily glue their output. Other languages are more complicated than shellscript non this task. If the task at hand wasn’t much about calling other programs, then shellscript wasn’t a good match to start with. If it was, no amiubt or complexity will be better handled by other languages. Quite the opposite.

                                                1. 11

                                                  That’s still not justification to jump into a post about “hey who here uses nodejs for bash scripting” with “no and I HATE IT”.

                                                  1. 5

                                                    Shellscript is literally optimized to call external programs

                                                    Right, and it’s very good at this.

                                                    and easily glue their output

                                                    …which it is not as good at. Unstructured textual “data” is the worst possible case for connecting programs together, because machines cannot manipulate unstructured data (without fancy ML techniques, which usually implicitly parses it into a structure that the ML model understands). In order to manipulate the output of various tools, it needs to be structured - so either it needs to come out structured (e.g. JSON) or you end up needing to parse it.

                                                    Most Unix utilities output “unstructured” data - which really means a collection of ad-hoc, poorly-designed, under-specified, brittle structured-text formats - the worst case. Meanwhile, none of the “classic” shells (sh, bsh, bash, dash, zsh, ksh, csh, tcsh, ash - excluding oil & other novel newcomers) actually give you decent tools to parse this mess (e.g. actual parsing tools with BNF & things, as opposed to text-manipulation tools like sed & awk - which aren’t even part of the shells anyway), nor do they provide any kind of structural interchange format that programs can use to exchange structured data.

                                                    Compared to that, JavaScript has some very strong advantages - most notably (1) JSON and (2) real objects.

                                                    1. 2

                                                      Forcing all IO operations to be async…

                                                      This isn’t actually true. You can use blocking IO in node, it’s just not idiomatic. In fact, scripting is typically a pretty good use case for blocking IO since you’re not generally trying to balance thousands of IO operations at the same time.

                                                      But to the point of this topic… I don’t quite get the rationale of people saying “when it gets too complicated to shellscript”. Shellscript is literally optimized to call external programs and easily glue their output.

                                                      Agreed, if all you want to do is glue together a pipeline of tools that read from stdin and write to stdout, a shell script is the right choice. I think the point others (and myself) have raised is that “scripting” is often broader than just redirecting A’s stdin to B’s stdout. When that’s the case, dynamic languages such as python, ruby, lua, and yes, javascript, are pretty handy.

                                                      1. 3

                                                        Blocking io operations on node have been marked deprecated for a decade or so. The fact that they need to be deprecated (they prevent anything else from working as expected) is exactly why node’s IO architecture is a very bad idea.

                                                        It is not my place to bash on node. I followed it since the day it was released. If it has sentimental value to some, nothing against that. But it was very clear from day one what it was. An single threaded event machine with V8 bolted into it. In all honesty, it would be more useful if they would have built V8 with basic IO and concurrency primitives like any proper language distribution out there. But obviously, the big selling point was a fast way to implement relatively performant http servers. Which they achieved.

                                                        What exactly do you refer to when you say scripting? Processing files, pick files on directories, running commands on file lists, running commands over ssh, download files from FTP or http servers…. Count occurrences… I find all this stuff an order of magnitude quicker to achieve with shellscripts than with general purpose programming language.

                                                        1. 3

                                                          In all honesty, it would be more useful if they would have built V8 with basic IO and concurrency primitives like any proper language distribution out there.

                                                          I was doing some professional Typescript programming - writing a CLI tool - shell scripting type stuff! - about a year ago and the lack of adequate threading / mutexes etc drove me up the wall (I had to query a number of things in a hurry). For whatever reason, the kexec library wasn’t building right, so I was locked on an earlier version of node for interacting with shell commands. The development system and VM was deeply, radically unfit for task, to a degree I’ve never experienced in any other language & VM as a professional. I was surprised even: I thought the system had matured and addressed all of this. This stuff was solved decades ago; it should have been reimplemented in V8 and friends. At this point, I don’t talk about roles with node; I don’t look to use it in a company, or to increase adoption in a company. If I wind up having input, I do not advise using or writing node programs or tech stacks serverside or desktopside.

                                                          1. 2

                                                            It sounds like you wanted threads but were stuck with a single threaded runtime that was designed to address io-heavy, cpu-light workloads. It sounds like a frustrating experience, but is this node being objectively bad, or just not a good fit for what you were trying to build?

                                                            1. 1

                                                              I was surprised even: I thought the system had matured and addressed all of this. This stuff was solved decades ago;

                                                              It could not have possibly addressed those because it pretty much has denied to address them at design time. It is effectively broken by design. You got one thread, concurrency is achieved by having all IO calls being async. You have one thread, block it, you block everything. If you are serving http responses on a busy website, put a blocking call and you will reduce the concurrency to one request at time.

                                                              And when you are performing IO calls say, inside a for loop, there is no way to cap the resources.

                                                            2. 2

                                                              What exactly do you refer to when you say scripting?

                                                              Yeah I think part of the disagreement and differences in preferences on this story probably have to do with us all having different use cases for “scripting” which is a broad umbrella. Often times I need to accomplish a task by communicating with APIs and services, maintaining some local state and data according to some straightforward business logic rules along the way. It’s the kind of thing I basically always want a language with actual data structures for. For small bits of automation with less logic, I’ll use shell.

                                                        2. 3

                                                          I don’t think it’s personal. They just don’t like node. The runtime is ascendant so it’s bound to attract dissenters.

                                                          1. 7

                                                            It’s not personal, but it’s also not helpful. It doesn’t add anything to the conversation.

                                                            1. 1

                                                              You’re right. I read the comment as distressed as opposed to a call out. I don’t need to soothe anyone here.

                                                      2. 2

                                                        The work project I lead and its testing tooling are in TypeScript. But when it comes to basic scripting, like our CI/CD steps, we reach for bash. I’d rather use the application’s language for business problems and the execution environment’s language for environment problems.

                                                        1. 2

                                                          Not specifically Node, but I prefer to use Not Bash wherever possible. Bash may be portable but it just irritates the everloving bile out of me.

                                                          1. 2

                                                            Personally, it varies.

                                                            If it’s automating a small set of commands, I’ll use bash. That’s what a bash script is: it executes the bash commands you’d run manually in your terminal. Common examples for me would be:

                                                            • Running a specific Docker container with certain arguments.
                                                            • Starting a development database and/or running migrations.
                                                            • Bootstrapping some other, more complicated piece of code without needing any dependencies on the system.

                                                            Using a non-bash programming language to just run some bash commands ends up feeling like overkill when it’s small.

                                                            But, bash sucks at tasks with much complexity, like if you need error handling beyond the process just failing with whatever error message the subprocess failed with, or even basic math. Importing other bash scripts is also a slight amount of pain every time you do it, because without some ceremony, the import path will be dependent on the directory of the user, rather than the path of the script being run; breaking up bash scripts into multiple files is obviously possible, but doing it a bunch is death by a thousand papercuts. If it’s more than a few dozen lines I’ll usually switch to a more standard programming language.

                                                            If the script mostly interacts with other processes, I’ll use Ruby. Node is… painful when it comes to spawning and waiting for output from processes. child_process is filled with callbacks and EventEmitters, when typically I just want to spawn a process synchronously and block until it gives me output. Usually when I’m writing scripting / automation tasks I’m trying to get something done reasonably quickly and don’t want to set up an entire environment, install dependencies, etc; I just want a script that’s reasonably-portable to other machines without doing much of anything to set it up, and so what’s in the standard library is important. For subprocesses, Node’s standard lib is pretty painful; Ruby’s is decent (mainly the open3 module).

                                                            If it’s mostly not about interacting with subprocesses and instead is about reading + transforming + outputting data, I’ll use Node. JS is a fairly comfortable language for me, it’s quite fast for an interpreted language, and I like its simple support for functional-style programming.

                                                            But really, it all depends on the project. For home use, all of the above is true; if it’s something I’m working on with other people, or at a job, I’ll use whatever is simplest for the project. If it’s a few lines I’ll still use bash, and anything larger I’ll probably use whatever language the project is written in. Most tasks can be accomplished in most programming languages, and making, for example, a Python team deal with a Ruby script is silly compared to just writing the script in Python; same goes for even compiled languages like Rust or Go.

                                                            Plus, in an existing project you’ve probably already paid the price of setting up some kind of environment and managing dependencies — so there’s no extra cost for using them. Whereas if you switch languages from the primary project’s, you’re back to either being dependent on stdlib, or asking people to set up and manage yet another environment to run your script.

                                                            1. 2

                                                              I wrote a little wrapper around DukTape to play with using JS for scripting. I haven’t really used it much, but using it as a replacement for shell scripting was one of the goals. Unlike Node, it produced a tiny (a few hundred KiB) binary that ran the scripts. I also added support for one of the Web Workers draft, so it was quite easy to write parallel programs. Having native threads and avoiding the fork / exec cost for simple things made it faster than shell scripting.

                                                              I did this as a PoC for something that could live in the FreeBSD base system and give an alternative to C or shell scripts as the two choices for things that live there. A lot of the stuff I was playing with was machine-generated JavaScript bindings from C headers. I never quite got it to a useful state, but I might go back to it at some point. The generated FFI code was significantly bigger than the core binary, which put me off the approach (this could be improved with a bit of work).

                                                              1. 2

                                                                I try to write POSIX-compliant shell syntax (without relying on GNU coreutils or other 3rd party programs if possible), and if I can’t I switch to Python. Many things can be done in a shell. You will probably find a shell (/bin/sh) on any operating system you use (I hope for you), and Python isn’t unusual to recent operating systems either (/usr/bin/env python3). Much unlike Node.js (or even, PHP, like I used to use in a similar mindset).

                                                                1. 2

                                                                  Nope. Python, I’ll reach to when I need something more complex, but not Node. I’m only ever in Node when I absolutely have to be.

                                                                  1. 2

                                                                    For throwaway code, I almost always use python or perl, where throwaway = “quick hack to solve some problem.” For anything mildly complex I almost always use nodejs instead; I have a few reasons for this:

                                                                    1. I like javascript’s more functional focus (vs other popular alternative scripting languages).

                                                                    2. node_modules is a feature, no matter how hard people imply the opposite.

                                                                      a. ^– because of the above, I can go so far as use the rich ecosystem of javascript parsers, written in javascript, to bundle my entire tool into one javascript source file.

                                                                      b. Also, people can just git clone software, much to their chagrin, and run npm install and it magically just works. Could you do it with python? probably, after you worry about what shell environment you’re executing in or what poetry/pipenv/venv/whatever-tool-of-the-week you use to manage said environment.

                                                                    3. I can share the exact same code I use in a front ended web application into the backend. I do this often and thus avoid having a lot of DRY violations around constants.

                                                                    4. Most of the slightly-more-than-quick-fixes are usually webapps in my experience as well.

                                                                    1. 1

                                                                      Yes, I write CLI scripts and some custom CI scripts with JS. I use JS whenever I need any of the following:

                                                                      • familiar APIs for making HTTP requests and transforming JSON data.
                                                                      • some familiar libraries not easily replicated in a shell script.
                                                                      • compatibility with most shells and platforms.
                                                                      • to provide a companion CLI for a low-level JS library. (I.e., JS is in the mix anyway.)

                                                                      You’ll notice “familiarity” is heavily featured. Admittedly, if I was more familiar with shell scripting, I might lean more on it. Nevertheless, I’ve needed to write three such CLIs in recent memory for work, two of which were primarily for others to use. Although the dependencies tend to be somewhat heavy, deployment/installation is pretty easy:

                                                                      1. Create a private repo in the Git host.
                                                                      2. Init an npm package and start by installing an args parser. (Lately I’ve been using cmd-ts with ts-node.)
                                                                      3. Provide instructions for executing it with npx. (No need to install anything globally. npm will cache dependencies so it should be pretty quick on subsequent uses.)
                                                                      npx githost:accountname/reponame ...args
                                                                      

                                                                      The execution of external binaries and managing processes in general is a pain for me as well. If I need to do that and I don’t need any of the above, that’s when I switch to shell scripting (and fairly basic at that).

                                                                      I’m curious to try Deno.

                                                                      1. 1

                                                                        I use it to script basic stuff. Network calls & text blob processing. Bash is too unwieldy to write

                                                                        1. 1

                                                                          To be honest I thought this was a joke when I saw the headline in the front page, but after thinking a bit more about it.. I’m somewhat sympathetic to the idea. I did write a couple of scripts in clojurescript a while ago but decided that it wasn’t what I wanted.

                                                                          It’s not that I think javascript is so terrible (although I avoid system software written in it) but rather that I dislike using tools for expedience and I’d rather fight with a learning curve in order to learn tools that are more aligned with my aesthetic or philosophy. Languages like python, perl, php, java and ruby all fall into the “javascript category” for me and I avoid using them for anything (except when the situation calls for it, I will e.g. write websites in javascript rather than introducing some compiler into the equation).

                                                                          At the moment I would rewrite a two (or more) line bash script in common lisp or scheme.

                                                                          The problem of running shell commands from whichever language you decide to use (instead of shell) for scripting is something I very often run into when I experiment with different languages, lisps have the upside of being absolutely amazing for finding a pattern that solves your immediate problem. In the past I was very invested in haskell and it took me years to get to the point of being able to use it without bikeshedding the problem to the point that I lost interest in solving it.

                                                                          1. 1

                                                                            I prefer Ruby for quick and dirty scripts, but if I’m working on a js/typescript project (which is what I do for a living) I’ll write the scripts in node. I try to avoid shelling out and prefer searching for pure js libraries to do the things I need, so that they work cross-platform without issues (I use Windows at work and frequently have problems with npm scripts that use & for launching two processes in parallel– which IMO is the wrong way to do it in the first place, but that’s beside the point). There’s plenty of pure-js reimplementations of most things, the problem is usually just finding the one that better fits your needs.

                                                                            For example just yesterday I needed to create a zip archive, and instead of shelling out to a zip binary I npm installed archiver and used its API, which also gave me better control of how things were laid out in the archive.

                                                                            And if something’s missing I can just implement it myself. I wrote the start of an AIFF parser (which I ended up not needing anyway) a while ago by looking at the spec in a few dozen lines of code with no dependencies, only using node’s builtin modules.

                                                                            A tool I really like is Jake, which is a JS reimplementation of Make. You can use it as a task runner for simple scripts (you can just require them directly from inside the Jakefile) or simply paste them in, or even better as a build system if there’s tasks that produce files that only need to be run if their dependencies change. I recommend learning to use Make or Jake, it’s a great tool that every developer should have in their toolbox.

                                                                            1. 1

                                                                              I never use JS unless I have to. It is always the last resort in any kind project.

                                                                              1. 1

                                                                                I’ve write shell scripts in node in the past for one-off tasks when the project already uses node. ej. I’ve used it to write git precommit hooks to run ESLint or CSSLint on the changed files.

                                                                                I say if you don’t know bash/sh and already know node you do you and continue using node for scripts. It doesn’t have a slow start-up and it is easy to read/write from/to STDOUT/STDIN.

                                                                                1. 1

                                                                                  I really don’t like JS in general. I usually use bash. But, honestly, I don’t really like bash either. I’ve been thinking about using Python (mostly because I want to get into Data Processing and I know it’s commonly supported) but dependency management in Python still seems yuck to me. Then again, I haven’t used Python very much in the last 5 years.

                                                                                  1. 2

                                                                                    I’m not a fan of dependency management in Python either, but it has a pretty big standard library, so for bash-like scripting purposes I find I don’t usually need to dive into dependencies. For example, here are a few of the many packages that come with base Python that I’ve found useful for scripting: os, zipfile, tarfile, csv, json, subprocess, pipes, time, fileinput.

                                                                                    1. 1

                                                                                      That’s a really good point. I hadn’t even thought of the amount of standard libs Python has.

                                                                                  2. 1

                                                                                    I typically use Python for script-like things, because the extremely expansive standard library means I don’t have to think about pulling in dependencies most of the time. Additionally, it’s super easy to pull in dependencies with Nix (I just add pythonPackages.<package name> to shell.nix) if I need them, then build a Nix derivation of my script with the dependencies if I need that.

                                                                                    1. 1

                                                                                      So I thought more about this question, and I was curious what you are having trouble with in terms of scripting? like executing random programs like bash? I honestly think Node has made it much easier recently with the introduction of async generator based streams, assuming you want to just execute stuff and process their stdout like a pipeline.

                                                                                      I hacked some example code up to sort of show what I mean. verbose, but I can’t imagine you can’t turn this into a library once it is all set up:

                                                                                      #!/usr/bin/env node
                                                                                      const { spawn } = require('child_process');
                                                                                      const { promisify } = require('util');
                                                                                      const pipeline = promisify(require('stream').pipeline);
                                                                                      
                                                                                      const Truncated = Symbol('truncated');
                                                                                      
                                                                                      function* lineize(input) {
                                                                                          let start = 0;
                                                                                          let nextLine = input.indexOf('\n');
                                                                                          while (nextLine !== -1) {
                                                                                              yield input.slice(start, nextLine);
                                                                                              start = nextLine + 1;
                                                                                              nextLine = input.indexOf('\n', start);
                                                                                          }
                                                                                          if (start < input.length) {
                                                                                              yield input.slice(start);
                                                                                              yield Truncated;
                                                                                          }
                                                                                      }
                                                                                      
                                                                                      async function main() {
                                                                                          const ls = spawn('ls', ['-1'], { stdio: [ 'ignore', 'pipe', 'ignore' ]});
                                                                                          const p = pipeline([
                                                                                              ls.stdout,
                                                                                              async function* (readable) {
                                                                                                  let lbuf = '';
                                                                                                  let lineo = 0;
                                                                                                  for await (const data of readable) {
                                                                                                      const dataStr = lbuf + data.toString('utf-8');
                                                                                                      for (const line of lineize(dataStr)) {
                                                                                                          if (line === Truncated) lbuf = line;
                                                                                                          else console.log(`${++lineo}: ${line}`);
                                                                                                      }
                                                                                                  }
                                                                                              },
                                                                                          ]);
                                                                                          await p;
                                                                                      }
                                                                                      main().then(() => console.log('done'))
                                                                                                .catch(e => console.error(e));
                                                                                      
                                                                                      1. 1

                                                                                        No, and I don’t write scripts for bash either, I just write for sh.

                                                                                        Using nodejs for your cli utility is a guaranteed way to make sure I’ll never use it.