1. 5

    My goal with infrastructure is to forget its existence, both in maintenance and on my DO invoice. Until it’s as cheap as a systemd unit on a 5$ VPS, I don’t care about it for small-scale projects.

    I think k8s is far more interesting for its API-based approaches (we’re bringing DCOM back), but I’d rather see the concepts implemented in something much simpler for again, small-scale things.

    1. 4

      imo, what’s really nice about using Kubernetes is

      • everything is an object
      • since you edit objects via an API, everything has an API
      • everything is in its own netns, cgroup

      We could absolutely build an infrastructure platform that does all of the above minus the boilerplate (and maybe even minus containers), but I don’t think that exists yet. These days I’m happy enough running Kubernetes everywhere (even single node) just so I don’t have to deal with netns myself.

      1. 2

        There is systemd which offers all of that minus boilerplate and minus containers.

        1. 2

          minus containers

          plus containers, of course https://man7.org/linux/man-pages/man5/systemd.nspawn.5.html

          1. 3

            Well, if you want, you can use them, but these aren’t required. That is why I said “minus containers”.

            1. 1

              Ah, I misunderstood you then.

          2. 1

            systemd is great but you still need something to deploy those unit files. And you have to control what goes where, which is usually the job of a scheduler.

            For a single machine, systemd over k8s any day. Anything more than a handful, it’s debatable.

            1. 1

              I was planning on writing multi-node scheduler for systemd in Erlang/Elixir. Maybe one day.

              However even in basic form you can manage that - ship everything everywhere and then use socket activation and LB in front of everything to start services as needed.

            2. 1

              Well, not quite. I don’t want to write an essay here, but here’s one example of why it really doesn’t:

              Let’s say we decide we want to create network namespaces to isolate all our services, and then selectively poke holes between them for services to communicate. Let’s look at how we would solve this in Kubernetes vs systemd.

              In Kubernetes, we could create an object type (CustomResourceDefinition) called FirewallRule. We’d then hook into an HTTP API, which notifies our program of any changes to these objects. On change, some code runs to reconcile reality with the state of the object. Of course, Kubernetes transparently handles the creation of network namespaces, and provides a builtin object type to poke holes between them (including a layer of abstraction on top where programs running on separate machines look like they’re in the same namespace), so in reality we would just use that.

              In systemd, we cannot create custom object types. Instead, to create a network namespace, we would wrap our .service with a shell script to start up the network namespace. To poke holes, we might create another .service unit, which spawns a program with some arguments that specify properties (source, destination, etc). We have to be careful to specify that the second unit depends on the first unit starting (otherwise the netns doesn’t exist).

              Let’s say opening and closing a hole in the network is an expensive operation, but modifying the port number is cheap. All we have as input in systemd is a Start and Stop, so we’d have to open and close the hole when we modify the unit file (expensive). In Kubernetes we get a whole diff of {current state, desired state} to work with, so we can choose to just edit the port (cheap). In this way, systemd isn’t really a true declarative abstraction over running your infrastructure, and is more like a shell script where you can specify a list of steps and then run those steps in the background.

              That said, I don’t think containers are the cleanest way of having object-based declarative infrastructure. Maybe something like NixOS is a better way forward long-term (the tooling and adoption are currently… ehhh). But for now, if I have to pick between writing imperative infrastructure and running containers, I’m gonna run containers.

              1. 2

                In systemd, we cannot create custom object types.

                Depends on your definition of “creation of custom types”. Because you can create “instantiated types” that allows you to create “unit templates” that you can later depend on. For example units that create named netns. Then all you need to do in your service is to add dependency on given netns like:

                Requires=netns@foo.service
                After=netns@foo.service
                

                And your service will be started after network namespace is created.

                This also solves the second problem you specified:

                All we have as input in systemd is a Start and Stop, so we’d have to open and close the hole when we modify the unit file (expensive).

                As we need to modify only our application service, not the netns service, we can restart our application without bringing the network down at all. The same goes for service that would be used for opening and closing ports in firewall.

                So the same approach is fully possible in systemd without many hiccups. Actually I think that it can be made even clearer for the Ops, as there is no “magical custom thing” like your FirewallRule, but everything is using “regular” systemd facilities.

                1. 2

                  I really like instantiated types; they make dealing with i.e. lots of site-to-site VPN links easy. I could see it used for other things; imagine describing a web service that slots into an application server as a unit like that, and have it automatically enroll in the web server’s routes/reverse proxy/whatever.

            3. 2

              Yeah, I feel the exciting part isn’t being able to build Google-scale container clusterfucks, but something like cPanel on top of clean APIs with a modern approach.

          1. 51

            I don’t believe Kubernetes and its primitives are a good fit for most companies out there. The article mentions Kubernetes, but what they are really saying is, you should make your applications containerizable and compatible with orchestrators.

            You should save yourself the pain and burden of using Kubernetes, and try out orchestrators like Nomad (from HashiCorp).

            1. 15

              I could not agree more. Any POC with k8s we did failed. I am not sure what it is for. The workloads we have perfectly fine on VMs with auto-scaling and there is no need for a 1M+ LOC solution that is looking for a problem.

              1. 7

                This is the tune I’ve been singing for a while, but I think I’m on the verge of changing it. Not because I suddenly think Kubernetes is a fantastic fit for every possible application on the planet, but because the “Kubernetes and containerization and execution environment are just different words for the same exact thing” viewpoint is becoming such an overwhelming consensus in the industry that I suspect it’s shortly going to get hard to hire people who know how to deploy code to anything else.

                It is really feeling to me like I’m dying on the wrong hill by insisting that it can be fine to use Nomad or Docker Compose or systemd to run a service at small scale. Kubernetes seems to just be winning everywhere, k8s-only tools are starting to show up more and more often, and at some point it’s going to get more expensive than it’s worth to be the odd man out.

                Makes me sad, but it’s not the first time the industry has converged on a technology that isn’t my first choice.

                1. 12

                  I am still on the “keep it simple” team and I am working on improving integration of Erlang projects with systemd. It makes deployment and configuration much simpler IMHO. I do not need to think about ingress, pods, etc. whatever there is in k8s. I just define my application to listen on port, and it will be started only when needed. I do not need anything more than that.

                  1. 3

                    Yup yup.

                    Leaning heavily on systemd units allows for a heck of a lot of ‘separation’ if you need to run different environments/versions on the same OS instance.

                    A (now) reasonably long term client originally contacted me asking about ‘setting up Dockers’ a few years ago, and the end result is that we avoided Docker/Containers (the business is essentially one website), but modernised their setup for all infra, to use:

                    • (a) 100% reproducible setup (using versioned shell scripts mostly, and a bit of Make)
                    • (b) modernised the ‘runtime’ to use systemd extensively, using a combination of instance templates, and generated unit files
                    • (c) moved local dev to a Vagrant workflow, with a custom base image built using the same scripts prod/staging/test are built with.

                    Sure, containers could have been applied to the problem, but that doesn’t mean it’s actually any better at solving the problem. You can bang a nail into a piece of wood with the back of an impact driver if you really want to. That doesn’t mean a simple hammer won’t do the job just as well.

              1. 2

                I know that this is old article, but this part:

                All modules that wish to adopt this Behaviour must:

                1. Explicitly state they wish to do so by using: @behaviour Parser.
                2. Implement parse/1 which takes a string and returns any term.
                3. Implement extensions/0 which takes zilch and returns a list of strings.

                Behaviour usage is explicit and therefore any modules adopting the Behaviour must actively state that fact by using the @behaviour SomeModule module attribute; […]

                Is completely false.

                1. You do not need to state that your module is implementing given behaviour to be considered a behaviour implementation. Some of the OTP behaviours do not even have module you can specify as a behaviour definition, for example logger handlers and formatters do not have behaviour modules defined. It is only helper for the user and it allows compiler to tell you if you omit some callbacks.
                2. You do not need to implement all functions from the behaviour. It will produce warnings during compilation (if you used @behaviour module attribute), but it will not cause compilation failure and it may not cause runtime failure. For example you can without any problems create module that will implement gen_server behaviour and you can omit handle_call/3 and handle_cast/2 callbacks, even if these aren’t marked as optional as long as you will not call them during the lifetime of the process.
                1. 10

                  I highly disagree with the concept, but I thought that it may be a good idea to post it there as a discussion starter about place of k8s in the industry.

                  1. 4

                    For checking if each commit passes tests I use git-test from Spotify.

                    1. 2

                      You can do this with plain git with git-rebase and --exec <cmd>.

                      1. 2

                        The conceptual basis - yes, but git-test will also cache results and will not rerun tests on commits that have been already checked.

                      2. 2

                        From reading the README I can’t figure out what exactly it is testing. Can you give a brief rundown please?

                        1. 2

                          It runs given test command for each commit between current head and the selected merge base. This makes sure that each commit passes test suite as most CIs by default check only head commits of request.

                          1.  

                            Makes sense. Thanks.

                            1.  

                              Makes sense. Thanks.

                        1. 1

                          Seems interesting for wearable electronics.

                          1. 4

                            You could have single-row “core” table and then create view for simulating two-row view to simplify queries (my friends are friends where user_1 = my_id). This would provide best of both of the two worlds.

                            1. 1

                              That’s true. However if you implement this view through UNION ALL it’s possible that it would be later used in some ad-hoc analytic query and the performance could be non-obvious (and hidden by the view). It’s manageable, but it needs to be kept in mind.

                            1. 11

                              Switching to using Lua for Neovim seems like it’ll be really powerful. I’m just a little scared to make that final jump, because there’s way to really go back to Vim after doing this. I’ve already leveraged a bunch of Neovim-only stuff, but this is a lot more extreme.

                              1. 7

                                The Lua support in Neovim is one of the things making me eager to look at it. Lua is a fantastic little language and there’s no reason to be stuck with something like vimscript in 2021.

                                1. 5

                                  I looked a bit at lua, and a lot of stuff seemed much more verbose/awkward to me.

                                  I kinda of like VimScript 😅 There are a few annoying parts (line continuations, ugh, and I still forget that stupid a: sometimes), but overall I find it works fairly well, especially since lambdas (already some time ago) and the method syntax (also some time ago, but never ported to Neovim) were added. It’s a fairly small language that fits its purpose reasonably well.

                                  It’s perhaps a bit of an acquired taste though.

                                  1. 4

                                    Neovim is claiming that lua is much faster than VimScript. I’ve seen VimScript chug on files with just a few hundred lines, so I’m hoping I can convert my heavier functions to lua and get a speed boost.

                                    1. 2

                                      I can’t say I ever ran in to issues with this myself; most of the times I ran in to performance issues it’s because you do something like :%s/<something rather complex on a large file>/, but I’m not sure if Lua will really help with that? From what I can tell, most of the performance issues is when people implement stuff like fuzzy matching in VimScript and similar computation-heavy stuff.

                                      The new VimScript9 is quite a bit faster (but still slower than LuaJIT), but I haven’t really looked at it.

                                    2. 1

                                      I kinda of like VimScript

                                      Wow, this is the first time in twenty years I’ve heard anyone express an emotion about vimscript other than disgust.

                                  2. 7

                                    if you keep your dotfiles in source control (which I recommend if you aren’t already), you have nothing to lose. If you make the switch and decide later that it’s not for you, just revert back to the previous version of your config that uses vimscript.

                                    1. 3

                                      That only works until you customize further in lua.

                                      1. 2

                                        I’m not sure what you mean. Sure, you lose some further custom lua config (until you port it to vimscript), but you won’t end up worse off than you were before you switched if you have your config in source control.

                                    2. 4

                                      I have recently switched not only to Lua, but to Fennel as well and I am pretty happy with result. Now I need to port rest of my config from VimL to Fennel.

                                      1. 2

                                        Wow, that’s amazing! Did not know Fennel could be used like this.

                                        1. 2

                                          do you notice any delay from having to compile the fennel files to lua on every startup?

                                          1. 4

                                            Slight, but nothing annoying me. In the past I was trying to cut load time as much as possible, but now I stopped giving a damn. I haven’t tried precompilation though.

                                          2. 1

                                            If you’re using a lisp for your editor scripting.. At what point does evil/emacs becometthe saner choice, though?

                                            1. 3

                                              As soon as Evil will learn into all my favourite quirks of Vim. I have tested Emacs in the past and there was always some small nitpick that was missing and was irritating me. Now I get the best of two worlds.

                                          3. 2

                                            Is there a reason that’s a problem? Neovim is a fork of the vim codebase and migrating to it doesn’t seem all that different from upgrading vim and then using plugins that require the new version as their minimum version. Neovim is basically just vim that is making faster progress on cool new features than mainline vim is.

                                            1. 5

                                              Neovim 0.5 lets you write your vimrc as a lua script, which Vim8 can’t parse.

                                              1. 2

                                                The big gotcha I hit with Neovim was that it uses a different format for undo files. I have this in my vimrc:

                                                set undodir=${XDG_CACHE_HOME}/vim/undo//
                                                set dir=${XDG_CACHE_HOME}/vim/swap//
                                                set backupdir=${XDG_CACHE_HOME}/vim/backup//
                                                set undofile
                                                set undolevels=1000
                                                set undoreload=10000
                                                

                                                This avoids cluttering my working directories with undo / backup / swap files and gives me effectively unlimited persistent undo, my favourite feature of vim. If I run neovim on a file that I’ve edited with vim, it complains. Similarly, if I edit with nvim and then vim, vim can’t read the undo file. I wish they’d change the name of files if they were going to have things for the same purpose as vim but with incompatible formats.

                                                1. 3

                                                  I wish they’d change the name of files if they were going to have things for the same purpose as vim but with incompatible formats

                                                  This can be achieved with something like

                                                  if has('nvim')
                                                    set undodir=${XDG_CACHE_HOME}/nvim/undo//
                                                    ...
                                                  else
                                                    set undodir=${XDG_CACHE_HOME}/vim/undo//
                                                    ...
                                                  endif
                                                  

                                                  I believe the undofile format became incompatible only very recently, when more information was added to undofiles.

                                                  1. 1

                                                    That’s what I did, but it’s annoying that I have to.

                                                    I was also quite excited by the built-in LSP support, but all of the tutorials I found required incredibly complex configs and didn’t explain how to configure the LSP integration for something like clangd for C/C++.

                                              2. 1

                                                There are heaps of incompatibilities between Vim and Neovim, and have been for years.

                                                https://news.ycombinator.com/item?id=27719968

                                              3. 1

                                                What are some neovim features that you’ve really liked? I generally see the same things trotted out by fans, like treesitter and in-build LSP, but these aren’t things I’ve felt like I was missing in Vim (I do use LSP, plugins work well for this). I’d love to hear what benefits you have found, as an experienced Vim user?

                                                1. 6

                                                  It’s a bit of a moving target, because a lot of neovim innovations are eventually implemented in vim. That’s what happened with floating windows, async jobs, and terminal emulation. So I guess the feature I really like is being a couple years ahead of everyone else :)

                                                  There’s also a few smaller things: I really like inccommand, which I don’t think has been ported over yet. And I just found out I can use the lua bindings to do this:

                                                  local nav_keys = {'h', 'j', 'k', 'l'}
                                                  for _, key in ipairs(nav_keys) do
                                                    local c_key = '<c-'..key..'>'
                                                    vim.api.nvim_set_keymap('n', c_key, '<c-w>'..key, opts)
                                                    vim.api.nvim_set_keymap('t', c_key, [[<c-\><c-n><c-w>]]..key, opts)
                                                  end
                                                  

                                                  I expect that as I get more comfortable with lua and the neovim API I’ll have more say. I write a lot of custom functions and vimL is one of the worst languages I’ve ever used.

                                                  EDIT ooh something I just realized I can do:

                                                  local set_all = function(list)
                                                  for _, item in pairs(list) do
                                                    vim.o[item] = true
                                                  end
                                                  end
                                                  
                                                  set_all {"termguicolors", "hidden", "splitright"}
                                                  set_all {"number", "relativenumber"}
                                                  

                                                  That’ll make it easy to create setting groups.

                                                  1. 2

                                                    No, no inccommand in Vim still. I was quite keen on this for a while until I discovered traces.vim which is a lot more powerful.

                                                    Yeah vimL has its quirks. The actual examples you’ve shown would look pretty similar in vimscript though … although they would use :execute, e.g. exe 'nnoremap' c_key '<c-w>..key. I’ve come to quite like vimL but I realise that puts me pretty firmly in the minority. I hate the line continuations but those are going away in vim9script.

                                                    It does look like configs will diverge now, to the lua neovim scripts and the Vim vim9script scripts. Which I suppose is good, trying to support 2 diverging editors with a single code-base is annoying, as each add different features.

                                                    1. 1

                                                      You can map stuff in loops in regular VimScript:

                                                      for k in ['h', 'j', 'k', 'l']
                                                          exe printf('nnoremap <C-%s> <C-w>%s', k, k)
                                                          exe printf('tnoremap <C-%s> <Cmd>wincmd %s<CR>', k, k)
                                                      endfor
                                                      

                                                      For a somewhat creative use of this you can see this ;-)

                                                      I guess the issue is that a lot of people just can’t be bothered to invest in learning VimScript, which is fair enough. But pretty much anything you can do in Lua you can do in VimScript, and vice versa.

                                                      1. 1

                                                        I guess the issue is that a lot of people just can’t be bothered to invest in learning VimScript, which is fair enough. But pretty much anything you can do in Lua you can do in VimScript, and vice versa.

                                                        Also, there’s more third party Lua libraries. A while back I needed to fill an autocomplete based on information in a YAML file, but couldn’t figure out how to parse YAML in VimScript. I ended up writing a python script to first convert the file to JSON, then use json_decode. It worked in that case but would have been nicer to just import a library and read the YAML directly.

                                                        1. 2

                                                          I think this - the availability of the Lua library ecosystem - is probably a “bigger deal”, looking forward, than the performance/tooling/ergonomics benefits Lua brings over VimScript. I haven’t seen too much adoption of Luarocks packages in plugins yet (though there are some examples, e.g. https://github.com/camspiers/snap), but I expect this to change now that 0.5 is stable.

                                                          I will also say that, while anything* you can do in Lua you can do in VimScript, I personally find it much easier to do most things in Lua - partially due to familiarity, but also due to the simplicity of the language and the relative lack of footguns, compared to VimScript.

                                                          *In Neovim specifically, Lua provides access to luv, which I’m not sure you can access from VimScript. In my experience, having the libuv event loop and utility functions readily at hand makes it possible to write functionality which would be at least substantially more painful without these tools.

                                                          1. 2

                                                            I haven’t seen too much adoption of Luarocks packages in plugins yet, but I expect this to change now that 0.5 is stable.

                                                            Why do you expect this to change with 0.5? It’s been possible to write Lua plugins since 0.3, with luv becoming available in 0.4. I don’t see anything (besides the hype, maybe? :^) ) in 0.5 that would change the attitude of plugin authors towards Luarocks.

                                                            1. 1

                                                              It’s true that we’ve had the ability to write Lua plugins for a while now, but there’s been a huge increase in the use of Lua during the later part of the 0.5 development cycle (the past year or so). This, to me, indicates that we may have passed an inflection point in the ease of using Lua for plugin development that will contribute to more people writing Lua plugins, and (potentially) more people using functionality from Luarocks libraries for those plugins (though I admit this also assumes that people need and know about the libraries available through Luarocks).

                                                              The hype is another factor, at least at first. And - noting that this runs dangerously close to self-promotion - I think that tools like packer.nvim‘s Luarocks integration (which you can use even if you don’t use packer for plugin management) significantly lower the complexity cost of using Luarocks packages, while also requiring features from 0.5.

                                                              1. 2

                                                                I see. In my opinion there’s a fundamental issue with the with dependencies where as a plugin author you don’t want to rely on external dependencies (otherwise you’re inevitably going to end up with people opening issues after misconfiguring the dependencies for your plugin…) but vendoring is fairly unattractive too (messes up version control history, you have to manually update everything…). This issue was already there with Vimscript and I don’t think it can be solved by anything else than Vim/Neovim implementing a full-featured plugin manager (which I don’t think it should do).

                                                                1. 1

                                                                  Yeah, that’s definitely true too. I think this is easier with Luarocks because it already supports some basic versioning specification, so you can control what versions of dependencies your users get without needing to vendor. I definitely agree that there shouldn’t be a full-featured plugin manager baked into Vim/Neovim, though - people’s preferences and needs around plugin management are too disparate for there to be a good “one size fits all” option, in my opinion.

                                                                  1. 2

                                                                    I think this is easier with Luarocks because it already supports some basic versioning specification

                                                                    My point was that the package manager that downloads from luarocks itself becomes a dependency, which you want to avoid. So luarocks doesn’t really make things easier there :)

                                                          2. 1

                                                            One way would be to use Python:

                                                            :pyx import yaml, vim; vim.vars['x'] = yaml.safe_load('{"hello": [1, 2]}')
                                                            

                                                            Or Ruby, Racket, Lua, Tcl, Perl, or even Visual Basic. It’s nice that Vim can interface with all that, but the big problem with that is that a lot of people don’t have the scripting support, and if you want things to work on Linux (all distros), macOS, and Windows is nigh-impossible. In reality It’s all pretty useless if as can never use it and have it work for everyone (the reason I didn’t try :lua is because my Vim isn’t compiled with it, and also, because I don’t really know Lua).

                                                            Generally Vim tries to minimize required dependencies, which isn’t a bad thing. The Lua interpreter is pretty small so it could be integrated like libvterm or xdiff instead of linking toliblua. But I guess it’s too late for that now :-/

                                                    1. 4

                                                      Explained in the readme

                                                      1. 3

                                                        Summary: Wikipedia has a lot more information on it than Wikidata.

                                                    1. 1

                                                      I have abandoned all my hope for PHP years ago, however I am not a fan that the mascot of this project is funded by mass murdering corporation /joke

                                                      1. 9

                                                        There is one more, huge difference. Adding new field to record is breaking change while adding new field to struct is not. This is quite important in situations like:

                                                        • Process state update, with record you need to do it in 2 steps - create new record, migrate the old record to new one, rename new record. With maps it is not a problem, as old pattern matching will still apply to the new map.
                                                        • You need to know the record structure before you can match on it. It adds additional compilation dependency as well as forces you to recompile all modules that were using given record on each update. Map-backed structures do not have that issue.
                                                        1. 1

                                                          No. Adding a new field to a record or struct requires re-compiling code. Struct definitions cannot be changed at runtime. You can get around this limitation by manually adding keys to the map at runtime, but this won’t work if you then try to use the struct to access the new field. Take for example this struct from the blog post:

                                                          defmodule Person do
                                                            defstruct [
                                                              :name,
                                                              # This expression is evaluated at compile time
                                                              created_at: DateTime.now!("Etc/UTC")
                                                              ]
                                                          end
                                                          
                                                          # in iex
                                                          iex(1)> %Person{foo: "test"}
                                                          ** (KeyError) key :foo not found
                                                              expanding struct: Person.__struct__/1
                                                              iex:2: (file)
                                                          
                                                          iex(2)> person = Map.put(%Person{}, :foo, :baz)
                                                          %{
                                                            __struct__: Person,
                                                            created_at: {:ok, ~U[2021-07-06 23:38:24.658023Z]},
                                                            foo: :baz,
                                                            name: nil
                                                          }
                                                          iex(3)> %Person{foo: binding} = person
                                                          ** (CompileError) iex:5: unknown key :foo for struct Person
                                                          
                                                          # See more examples from this page https://elixir-lang.org/getting-started/structs.html
                                                          

                                                          By manually adding a field to the map, you are bypassing some of the things that make record/structs valuable. After adding injecting a new key and value like this, the map no longer matches the struct definition, and any typespecs or types that reference the struct are now wrong.

                                                          Process state update, with record you need to do it in 2 steps - create new record, migrate the old record to new one, rename new record. With maps it is not a problem, as old pattern matching will still apply to the new map.

                                                          If you want to add/change/remove fields from a record or struct at runtime you should migrate your data. OTP behaviors have callbacks for upgrading internal state not only because it’s needed but because you should be explicit about it.

                                                          You need to know the record structure before you can match on it. It adds additional compilation dependency as well as forces you to recompile all modules that were using given record on each update. Map-backed structures do not have that issue.

                                                          Only if you treat them as maps. If you treat them as structs recompilation will always be needed.

                                                        1. 2

                                                          My favourite. Much smaller, in C, but IMHO more confusing instead of obsfucating:

                                                          #include <unistd.h>
                                                          
                                                          int main() {
                                                            uint32_t b = 0x0a646c72;
                                                            uint64_t a = 0x6f77206f6c6c6548;
                                                          
                                                            return !write(1, &a, 12);
                                                          }
                                                          
                                                          1. 5

                                                            This seems fun, and maybe a good tool for build proof of concepts. But I hardly see it as being useful for large projects. Or have I become old and grumpy?

                                                            1. 13

                                                              As a stranger on the internet, I can be the one to tell you that you are old and grumpy.

                                                              Ruby is definitely unusable without syntax highlighting… (Sadists excepted) Java is definitely unusable without code completion… (Sadists excepted) Whatever comes next will probably be unusable without this thing or something like it.

                                                              1. 9

                                                                I’m confused… Ruby has one of the best syntaxes to read without highlighting. Not as good as forth, but definitely above-average

                                                                1. 3

                                                                  Well, this is the internet. Good luck trying to make sense of every take.

                                                                  1. 2

                                                                    I used to think this way. Then I learned Python and now I no longer do.

                                                                    When I learned Ruby I was coming from Perl, so the Perl syntactic sugar (Which the Ruby community now seems to be rightly fleeing from in abject terror) made the transition much easier for me.

                                                                    I guess this is my wind-baggy way of saying that relative programming language readability is a highly subjective thing, so I would caution anyone against making absolute statements on this topic.

                                                                    For instance, many programmers not used to the syntax find FORTH to be an unreadable morass of words and punctuation, whereas folks who love it inherently grok its stack based nature and find it eminently readable.

                                                                    1. 1

                                                                      Oh, sure, I wasn’t trying to make a statement about general readability, but about syntax highlighting.

                                                                      For example, forth is basically king of being the same with and without highlighting because it’s just a stream of words. What would you even highlight? That doesn’t mean the code is readable to you, only that adding colour does the least of any syntax possible, really.

                                                                      Ruby has sigils for everything important and very few commonly-used keywords, so it comes pretty close also here. Sure you can highlight the few words (class, def, do, end, if) that are in common use, you could highlight the kinds of vars but they already have sigils anyway. Everything else is a method call.

                                                                      Basically I’m saying that highlighting shines when there are a lot of different kinds of syntax, because it helps you visually tell them apart. A language with a lot of common keywords, or uncommon kinds of literal expressions, or many built-in operators (which are effectively keywords), that kind of thing.

                                                                      Which is not to say no one uses syntax highlighting in ruby of course, some people find that just highlighting comments and string literals makes highlighting worth it in any syntax family, I just felt it was a weird top example for “syntax highlighting helps here”.

                                                                      1. 3

                                                                        Thank you for the clarification I understand more fully now.

                                                                        Unfortunately, while I can see where you’re coming from in the general case, I must respectfully disagree at least for myself. I’m partially blind, and syntax highlighting saves my bacon all the time no matter what programming language I’m using :)

                                                                        I do agree that Ruby perhaps has visual cues that other programming languages lack.

                                                                        1. 1

                                                                          ’m partially blind, and syntax highlighting saves my bacon all the time no matter what programming language I’m using :)

                                                                          If you don’t mind me asking - have you tried any Lisps, and if so, how was your experience with those? I’m curious as to whether the relative lack of syntax is an advantage or a disadvantage from an accessibility perspective.

                                                                          1. 1

                                                                            Don’t mind you asking at all.

                                                                            So, first off I Am Not A LISP Hacker, so my response will be limited to the years I ran and hacked emacs (I was an inveterate elisp twiddler. I wasted WAY too much time on it which is why I migrated back to Vim and now Vim+VSCode :)

                                                                            It was a disadvantage. Super smart parens matching helped, but having very clear visual disambiguation between blocks and other code flow altering constructs like loops and conditionals is incredibly helpful for me.

                                                                            It’s also one of the reasons I favor Python versus any other language where braces denote blocks rather than indentation.

                                                                            In Python, I can literally draw a veritcal line down from the construct and discern the boundaries of the code it effects. That’s a huge win for me.

                                                                            Note that this won’t eventually keep me from learning Scheme, which I’d love to do. I’m super impressed by the Racket community :)

                                                                        2. 1

                                                                          For example, forth is basically king of being the same with and without highlighting because it’s just a stream of words. What would you even highlight? That doesn’t mean the code is readable to you, only that adding colour does the least of any syntax possible, really.

                                                                          You could use stack effect comments to highlight the arguments to a word.

                                                                          : squared ( n -- n*n ) 
                                                                               dup * ;
                                                                           squared 3 .  
                                                                          

                                                                          For example, if squared is selected then the 3 should be highlighted. There’s also Chuck Moore’s ColorForth which uses color as part of the syntax.

                                                                    2. 6

                                                                      Masochists (people that love pain on themselves), not sadists (people that love inflicting pain on others).

                                                                      1. 2

                                                                        Ah, thank you for the correction.

                                                                        I did once have a coworker who started programming ruby in hungarian notation so that they could code without any syntax highlighting, does that work?

                                                                        1. 4

                                                                          That count as both ;)

                                                                        2. 2

                                                                          Go to source is probably the only reason I use IDEs. Syntax highlighting does nothing for me. I could code entirely in monochrome and it wouldn’t affect the outcome in the slightest.

                                                                          On the other hand, you’re right. Tools create languages that depend on those tools. Intellij is infamous for that.

                                                                        3. 6

                                                                          You’re old and grumpy :) But seriously, the fact that it’s restricted to Github Codespaces right now limits its usefulness for a bunch of us.

                                                                          However, I think this kind of guided assistance is going to be huge as the rough edges are polished away.

                                                                          Will the grizzled veterans coding exclusively with M-x butterflies and flipping magnetic cores with their teeth benefit? Probably not, but they don’t represent the masses of people laboring in the code mines every day either :)

                                                                          1. 4

                                                                            I don’t do those things, I use languages with rich type information along with an IDE that basically writes the code for me already. I just don’t understand who would use these kinds of snippets regularly other than people building example apps or PoCs. The vast majority of code I write on a daily basis calls into internal APIs that are part of the product I work on, those won’t be in the snippet catalog this things uses.

                                                                            1. 4

                                                                              I don’t doubt it but I would also posit that there are vast groups of people churning out Java/.Net/PHP/Python code every day who would benefit enormously from an AI saying:

                                                                              Hey, I see you have 5 nested for loops here. Why don’t we re-write this as a nested list comprehension. See? MUCH more readable now!

                                                                              1. 4

                                                                                The vast majority of code I write on a daily basis calls into internal APIs that are part of the product I work on, those won’t be in the snippet catalog this things uses.

                                                                                Well, not yet. Not until they come up with a way to ingest and train based on private, internal codebases. I can’t see any reason to think that won’t be coming.

                                                                                1. 2

                                                                                  Oh sure, I agree that’s potentially (very) useful, even for me! I guess maybe the problem is that the examples I’ve seen (and admittedly I haven’t looked at it very hard) seem to be more like conventional “snippets”, whereas what you’re describing feels more like a AST-based lint that we have for certain languages and in certain IDEs already (though they could absolutely be smarter).

                                                                                  1. 2

                                                                                    Visual studio (the full ide) has something like this at the moment and it’s honestly terrible. Always suggests inverting if statements which break the logic, or another one that I haven’t taken the time to figure out how to disable is it ‘highlights’ with a little grey line at the side of the ide (where breakpoints would be) and suggests changes such as condensing your catch blocks from try/catches onto one line instead of nice and readable.

                                                                                    Could be great in the future if could get to what you suggested!

                                                                                  2. 3

                                                                                    Given that GH already has an enterprise offering, I can’t see a reason why they can’t enable the copilot feature and perform some transfer learning on a private codebase.

                                                                                    1. 1

                                                                                      Is your code in GitHub? All my employer’s code that I work on is in our GitHub org, some repos public, some private. That seems like the use case here. Yeah, if your code isn’t in GitHub, this GitHub tool is probably not for you.

                                                                                      I’d love to see what this looks like trained on a GitHub-wide MIT licensed corpus, then a tiny per-org transfer learning layer on top, with just our code.

                                                                                      1. 1

                                                                                        Yeah, although, to me, the more interesting use-case is a CI tool that attempts to detect duplicate code / effort across the organization. Not sure how often I’d need / want it to write a bunch of boilerplate for me.

                                                                                  3. 1

                                                                                    it feels like a niftier autocomplete/intellisense. kind of like how gmail provides suggestions for completing sentences. I don’t think it’s world-changing, but I can imagine it being useful when slogging through writing basic code structures. of course you could do the same thing with macros in your IDE but this doesn’t require any configuration.

                                                                                  1. 2

                                                                                    2 questions:

                                                                                    • Have you taken look on Gleam? It has few of the properties you want.
                                                                                    • If you liked Elixir, then why not target BEAM instead of JVM?
                                                                                    1. 2
                                                                                      • Yes I’ve been following the development of Gleam and I’m excited to see where it goes. The main difference is that I’m currently thinking about using row polymorphism and union types rather than named structs and variant types. I’m not 100% sure if this is better in practice, but I’m gonna give it a shot.
                                                                                      • As I mentioned in the post, I am very familiar with Java, whereas I have only built a single Phoenix project with Elixir. This project is partially a learning exercise, so I’d rather focus on the JVM since it could also be useful with my day job.
                                                                                    1. 9

                                                                                      I wish Rust had a bigger standard library (“batteries” included, like python in some degree)

                                                                                      See for example sort. I realise all of us download and run programs with lots of dependencies most days but I feel like core utils should not pull non-standard dependencies.

                                                                                      1. 13

                                                                                        Note that among those 12 direct dependencies, Python’s stdlib has direct equivalents only to 4: clap, itertools, rand, tempfile. Things like unicode-width, rayon, semver, binary-heap-plus are not provided by Python. compare, fnv, memchr and ouroboros are somewhat hard to qualify Rust-isms.

                                                                                        1. 2

                                                                                          In addition, it’s worth noting that a lot of projects eschew argparse (what the alternative to clap would be) for click. If a similar project was done in python, I’d almost bet money that they’d use click.

                                                                                          rand being separate has some advantages, largely that it is able to move at a pace that’s not tied to language releases. I look at this as a similar situation that golang’s syscall package has (had? the current situation is unclear to me rn). If an OS introduces a new random primitive (getrandom(2), getentropy(2)), a separate package is a lot easier to update than the stdlib, which is tied to language releases.

                                                                                          Golang’s syscall package has (had?) a similar problem, which led to big changes being locked down, and the recommended pkg being golang.org/x/sys. There’s a lot more agility to be had to leverage features of the underlying OS if you don’t tie certain core features to the same cadence as a language release. (this is not to say that this is the only problem with the syscall package being in the stdlib, but it’s definitely one of them. more info on the move here: https://docs.google.com/document/d/1QXzI9I1pOfZPujQzxhyRy6EeHYTQitKKjHfpq0zpxZs/edit)

                                                                                          1. 1

                                                                                            argparse

                                                                                            I’d use getopt over argparse. argparse just has really abysmal parsing which is different from other shell tools, especially when dealing with subcommands.

                                                                                          2. 1

                                                                                            True. Could be just rust-lang crates like futures-rs or cargo instead of being in the stdlib.

                                                                                          3. 12

                                                                                            This problem is Software Engineering Complete (like NP-Complete - transformable to any other SW Eng Complete thing). As just one other example, Nim also struggles with what should be in the stdlib vs. external packages. Rust has just about 1000x the resources than Nim for a much more spartan core stdlib, but of course the Nim stdlib almost surely has more bugs than that Rust spartan core. So, a lot of this boils down to A) tolerance for bugs, B) resources to maintain going forward, and C) the cost of complexity/generality in the first place, and probably a factor or two I’m forgetting/neglecting. ABC relate far more to community & project management/attitudes than language details themselves. Also, presence in the stdlib is not a panacea for discoverability because as the stdlib grows more and more giant, discoverability crashes.

                                                                                            Note this is neither attack nor defense but elaboration on why this is not an easy problem.

                                                                                            1. 2

                                                                                              I wonder if the reason nim has to have a big standard library is to attract people. Rust already has the following, as you said, and people are sure to create an kinds of things. Whereas, if one was to try nim, if there wasn’t the stdlib, they would have to do everything on their own.

                                                                                            2. 3

                                                                                              Same. A big part of the learning curve for me was discovering modules like serde, tokio, anyhow/thiserror, and so on that seem necessary in just about every Rust program I write.

                                                                                              1. 3

                                                                                                Not providing a standard executor was the only complaint I had from async Rust.

                                                                                                1. 2

                                                                                                  I like that there is no built-in blessed executor - it keeps Rust runtime-free.

                                                                                                  I’ve worked on projects where using an in-house executor was a necessity.

                                                                                                  Also gtk-rs supports using GTK’s event loop as the executor and it’s very cool to await button clicks :)

                                                                                                  1. 1

                                                                                                    Yeah i’ve used it and it felt refreshing :) But for other small tools perhaps having a reference and minimal implementation would be good. I like the smol crate and I think would be perfect for this

                                                                                                2. 3

                                                                                                  All of them developed over time and became a de-facto standard. But it was always the intention that the std doesn’t try to develop these tools as you need some iterations, which won’t work on with a stability guarantee. tokio just went to 1.0 this? year, I’ve got code lying around using 0.1,0.2 and some 0.3 (and don’t forget futures etc).

                                                                                                  anyhow/thiserror ? Well there is failure,error-chain,quick-error,snafu,eyre (stable-eyre,color-eyre), simple-error….. And yes, some of them are still active as they solve different problems (I specifically had to move away from thiserror) and some are long deprecated. So there was a big amount of iteration (and some changes to the std Error trait as a result).

                                                                                                  You don’t want to end up like c++ (video) where everybody treats the std implementation of regex as something you don’t ever want to use.

                                                                                                3. 3

                                                                                                  There is problem with such approach of “batteries included” in the standard library - development of such libraries slows down or stagnates. Actually I prefer to use set of well behaved external libraries than need to replace “built-ins” because these are too simplified for any reasonable usage.

                                                                                                  1. 3

                                                                                                    The thing is that as a user I “trust” standard rust-lang crates at first glance; surely if I check the external libraries out or recognize them I will know they are well behaved and performant. Trust and trusting trust is such a big problem in software in general.

                                                                                                    1. 5

                                                                                                      Yes, that why there are some “blessed” crates as well as there is Crev project to expand the trust.

                                                                                                    2. 1

                                                                                                      just version the standard library interface, though

                                                                                                      1. 4

                                                                                                        And point me to one, just one, example where it worked? If something is merged into the core, then it will die there. Python has examples of such code, Ruby has examples of such code, etc. How often for example built in HTTP client is good enough to be used in any serious case? How often instead you pull dependency to handle the timeouts, headers, additional HTTP versions, etc. better/faster/easier?

                                                                                                    3. 2

                                                                                                      The rust ecosystem, IMO, is far too eager to pull in third party dependencies. I haven’t looked deep into this tool, but a quick glance leads me to believe that many of these dependencies could be replaced with the standard library and/or slimmed down alternative libraries and a little extra effort.

                                                                                                      1. 2

                                                                                                        Unfortunately it’s not always that simple. Let’s see the third party dependencies I pulled for meli, an email client, which was a project I started with the intention of implementing as much as possible myself, for fun.

                                                                                                        xdg = "2.1.0"
                                                                                                        crossbeam = "0.7.2"
                                                                                                        signal-hook = "0.1.12"
                                                                                                        signal-hook-registry = "1.2.0"
                                                                                                        nix = "0.17.0"
                                                                                                        serde = "1.0.71"
                                                                                                        serde_derive = "1.0.71"
                                                                                                        serde_json = "1.0"
                                                                                                        toml = { version = "0.5.6", features = ["preserve_order", ] }
                                                                                                        indexmap = { version = "^1.6", features = ["serde-1", ] }
                                                                                                        linkify = "0.4.0"
                                                                                                        notify = "4.0.1"
                                                                                                        termion = "1.5.1"
                                                                                                        bincode = "^1.3.0"
                                                                                                        uuid = { version = "0.8.1", features = ["serde", "v4"] }
                                                                                                        unicode-segmentation = "1.2.1"
                                                                                                        smallvec = { version = "^1.5.0", features = ["serde", ] }
                                                                                                        bitflags = "1.0"
                                                                                                        pcre2 = { version = "0.2.3", optional = true }
                                                                                                        structopt = { version = "0.3.14", default-features = false }
                                                                                                        futures = "0.3.5"
                                                                                                        async-task = "3.0.0"
                                                                                                        num_cpus = "1.12.0"
                                                                                                        flate2 = { version = "1.0.16", optional = true }
                                                                                                        

                                                                                                        From a quick glance, only nix, linkify, notify, uuid, bitflags could be easily replaced by invented here code because the part of the crates I use is small.

                                                                                                        I cannot reasonably rewrite:

                                                                                                        • serde
                                                                                                        • flate2
                                                                                                        • crossbeam
                                                                                                        • structopt
                                                                                                        • pcre2
                                                                                                        1. 1

                                                                                                          You could reduce transitory dependencies with:

                                                                                                          serde -> nanoserde

                                                                                                          structopt -> pico-args

                                                                                                          Definitely agree that it isn’t that simple, and each project is different (and often it’s not worth the energy, esp for applications, not libraries), but it’s something I notice in the Rust ecosystem in general.

                                                                                                          1. 4

                                                                                                            But then you’re getting less popular deps, with fewer eyeballs on them, from less-known authors.

                                                                                                            Using bare-bones pico-args is a poor deal here — for these CLI tools the args are their primary user interface. The fancy polished features of clap make a difference.

                                                                                                      2. 1

                                                                                                        Why do you think an external merge sort should be part of the Rust stdlib? I don’t think it’s part of the Python stdlib either. Rust already has sort() and unstable_sort() in its stdlib (unstable sort should have been the default, but that ship has sailed).

                                                                                                      1. 1

                                                                                                        I disagree with few of these:

                                                                                                        • Map.get/2 and Keyword.get/2 vs. Access - I prefer to just use Map and Keyword as not only it is IMHO cleaner to the reader about the expected types, it is also faster as there is no need for dynamic dispatch. Also Map.get/2 will work with structures that do not implement Access behaviour. If you want to have uniform access syntax, then you can have “frontend function” that does Map.new/1 on the options and pass the new map to other functions.
                                                                                                        • Don’t pipe into case statements - I like to do so, but only if that is the last thing in the function. That makes it much cleaner to my eyes and do not force me to create temporary variable somewhere at the top just to be ingested once.
                                                                                                        • Avoid else in with blocks - I agree with the fact that the tagged assignments are smell, but I completely disagree with the overall statement. else in with makes a lot of sense when you do not need to differ between each of the mismatches. So for example their own example that uses :fuse can be IMHO written much cleaner as:
                                                                                                          with :ok <- :fuse.check(:service),
                                                                                                               {:ok, result} <- call_service(id) do
                                                                                                            :ok = Cache.put(id, result)
                                                                                                            {:ok, result}
                                                                                                          else
                                                                                                            :blown -> Cache.fetch(id) # It is alternative to `Cache.get/1` that returns error-tuple instead
                                                                                                            {:error, _} = error ->
                                                                                                              :fuse.melt(:service)
                                                                                                              error
                                                                                                          end
                                                                                                          
                                                                                                          Yes, I fully agree, that irrecoverable errors should be exceptions. And if you are working with Plug (so it applies to Phoenix as well), then you can have :plug_status field with code that should be returned to user in case of error or you can implement Plug.Exception protocol to do the same thing.
                                                                                                        • Use for when checking collections in tests - instead, just write custom helper for that. You can also make it much better then by for example providing indices of the mismatched entries. It will also make test cleaner when you would do assert_items_match %Post{}, list instead.
                                                                                                        1. 1

                                                                                                          That is why I have created my own library for Elixir. Well, main reason was that I wanted to support “new” step definition syntax, which IMHO is much more readable, but additionaly it fixes some of the problems mentioned in the article (obviously it cannot solve all of them, as some are culture-related):

                                                                                                          1. 5

                                                                                                            Disappointed no new MBP announcement, guess I’ll just have to wait longer.

                                                                                                            The Xcode Cloud announcement is interesting. Given that… is there any reason you couldn’t develop iOS apps on Linux/Windows?

                                                                                                            1. 3

                                                                                                              Sounds like both the management of the CI/CD workflow is entirely handled through Xcode, and uses Xcode’s build system to compile, test and build apps. I imagine it’s going to be difficult to do that anywhere except a Mac any time soon.

                                                                                                              1. 3

                                                                                                                Disappointed no new MBP announcement, guess I’ll just have to wait longer.

                                                                                                                Hardware will be tomorrow.

                                                                                                                1. 7

                                                                                                                  If they announced hardware at WWDC, it was always during the keynote. As this is the only event with much media attention of WWDC, why would they announce new products on any other day.

                                                                                                                  The „leakers“ have also stated, that there won‘t be any new hardware: https://www.macrumors.com/2021/06/07/no-hardware-at-wwdc-suggests-leaker/

                                                                                                                  1. 2

                                                                                                                    Will it be streamed? What time? I can’t find any info about a hardware presentation online.

                                                                                                                1. 2

                                                                                                                  Still looking for an option to buy cask of whisky from 2020 as a gift for my son on his 18th birthday.