1. -2

    If you’re running ImageMagick this decade you’re doing things horribly wrong.

    1. 13

      I’m not sure what people would use instead.

      1. 2

        If you have a codebase using IM and you don’t want to rewrite it, the first quick fix to come to mind is to use some other library with a safer parser to decode the incoming images. Then pass something to IM which requires no parsing or trivial parsing, like a pointer to the decoded pixels in memory or a PNM.

        Using ImageMagick for input parsing is scary. Using it for output isn’t.

        1. 2

          The stb libraries are very good if you have simple requirements:

          stb_image

          stb_image_write

          stb_image_resize

          1. 0

            libpng if you want to handle PNGs, libjpeg if you want to handle JPEGs.

            1. 11

              Neither of those libraries is particularly adept at rescaling images.

              1. 1

                If you need basic image manipulation you should use something like gdk-pixbuf. I used it years ago but I’ve heard from people using it large scale production today.

              2. 1

                Uhhh

            2. 1

              Is the GraphicsMagick fork good?

              1. 3

                I assume not. The goal of being able to handle every format under the sun turns out to be (a) hard to implement securely and (b) not to line up well with actual requirements of actual software for actual users.

            1. 5

              Oh, I really want to use sway and wayland on nixos, but keep having issues :( I try every release.

              1. 3

                By the way, a Sway 1.0 beta derivation has been added to nixpkgs two days ago:

                https://github.com/NixOS/nixpkgs/commit/d8d8a0ba14a7be1aa36bcbdf66b7f2cd9df9d3ff

                Tried it yesterday, works with amdgpu by just running sway from a tty.

                1. 3

                  Yikes.

                  Package maintainers: please do not package this release.

                  1. 3

                    The person who made this derivation asked @SirCmpwn for permission. He was ok with it, as long as it was clear that it packages a beta version of Sway. The derivation is called sway-beta.

                    You also need unstable nixpkgs. So it says ‘expect problems’ on the tin twice ;).

                    1. 1

                      How else you gonna try it out on NixOS? We have to package things.

                      1. 1

                        Same way as anyone else? ./configure && make && make install? Or is that not supported with nix?

                        1. 3

                          Not the make install part.

                          1. 2

                            When installing an application is a simple ./configure && make && make install, writing a Nix derivation is trivial and has a lot of benefits:

                            • You don’t have to install the dependencies in your system or user environment.
                            • You don’t have to install the program in your system or user environment.
                            • The build is sandboxed (when sandboxing is enabled).
                            • You can manage the resulting package with the usual nix tools.
                            • Easy to reproduce the build across systems.

                            The sway-beta derivation mostly consists of metadata, package dependencies, and the GitHub username/repo/revision/SHA. But you’d have to figure out the package names of dependencies in your distribution anyway, even when doing the configure/make dance.

                      2. 1

                        thanks, I use the amdgpu driver, i will give it a go.

                      3. 2

                        I really want to use sway and wayland on nixos, but keep having issues :( I try every release.

                        I’m also interested. Did you ask for help on the nix forum?

                        1. 2

                          I was having issues with my graphics drivers and wayland, can’t remember the exact problem off the top of my head. Don’t think I asked anybody other than google for help last time.

                        2. 1

                          I used NixOS to build a VM with Wayland and Sway a long time ago. I was able to use it! Hopefully you can solve your graphics problems.

                        1. 1

                          Not sure why the DB has to be on the servers (to be monitored), and not on the client/monitoring server. Personally I try to keep the monitoring workload on the servers as minimal as possible, and do all the aggregation ++ on a separate server.

                          If they had done that, they also could have used snmpd from the base system, without the need for writing a collector daemon.

                          1. 3

                            It’s on both. The database is really just a set of circular buffers that’s transmitted in totality. But it’s on the system being monitored because hey, you might turn off the monitor. And then your history is gone. Also, even if the monitor is running, the network might fail.

                            1. 3

                              This is something I wish the Prometheus people had factored in to their design. I’d love to be able to restart the collectors or have network disruptions without losing data points. A locally spooled catch-up ring buffer is a great idea.

                              1. 1

                                A locally spooled catch-up ring buffer is a great idea.

                                Would you persist it to disk? Just in case the monitored server crashes while the buffer is not empty.

                                1. 2

                                  Persisting to disk would definitely be something I’d like to explore. I think the downside might be that it induces extra I/O load and storage pressure on what might already be a busy system.

                                  1. 1

                                    I’m not sure you’re referring to slant, but it leaves the database on the disc. The point of slant is to be a simple tool for normative hosts: if your systems are so significantly under load, you probably want to do something like snmp.

                                    That being said, I’m happy to work with folks with patches to make slant more performant. I know that ksql has lots of room for improvement!

                                    1. 2

                                      I’m not sure you’re referring to slant, but it leaves the database on the disc.

                                      Sorry, I was talking more in the abstract – we’re using the Prometheus ecosystem pretty heavily at the moment.

                                      It’s neat that slant persists to disk!

                          1. 1

                            How would be the best way to add a root certificate in a docker container using your tool? Like setting the equivalent of security.pki.certificates (configuration.nix).

                            Is there also a way to run a command while building the image? Like Dockerfile’s RUN command.

                            EDIT: I think I figured out a way by doing a dumb derivation that only copy /etc/ssl/certs/ca-certificates.crt from my workstation (which already has the CA).

                            1. 3

                              I would like nix so much more if I didn’t end up with 6 copies of gcc and a texlive install every time I installed something nontrivial.

                              1. 1

                                Yep. I wondered “where did that texlive install come from” recently too.

                                1. 2

                                  There’s a few tools for this. For example, you can ask where it’s installed:

                                  $ nix-store -q --roots $(which ghc)
                                  /nix/var/nix/profiles/per-user/brian/profile-377-link
                                  /nix/var/nix/profiles/per-user/brian/profile-378-link
                                  /nix/var/nix/profiles/per-user/brian/profile-379-link
                                  /nix/var/nix/profiles/per-user/brian/profile-380-link
                                  

                                  Or if it’s just alive from a shell or something, you can ask what referred to it:

                                  $ nix-store -q --referrers $(which ghc)
                                  /nix/store/kjxr0bm87jljb91qd7k47cqiknnxdknx-ghc-8.2.2-with-packages
                                  /nix/store/1djxkifaxmz940x571ab4ndfmvmb1lh5-env-manifest.nix
                                  /nix/store/crxhjrgkngwk8bv77sy6bxl19k6id2f3-env-manifest.nix
                                  /nix/store/50rbrla0m0819wr788gs588qi7yl6vxy-user-environment
                                  /nix/store/7wqjzgci6v9fnhgs0zsvxhhc6hzjczfv-env-manifest.nix
                                  /nix/store/i3gz4d3jkqkh4v9vscx7s2fa0q7wll3q-env-manifest.nix
                                  /nix/store/f53kz6isz60xr83yiwvl7xw0b2v5k5pg-user-environment
                                  /nix/store/wc037in1jmkdwxj1hhjhbih5arwazrcw-user-environment
                                  /nix/store/zismyry5a61xwpvnkvdidiqgvcrqwg04-user-environment
                                  
                                  1. 1

                                    Thanks, I’ll look into that. I’m curious why things even need texlive in the first place - seems like it’s unnecessary to have for a running system.

                                    Personally, I’d love to see NixOS be able to meet requirements like this - the Nix store seems like it would facilitate reduction of attack surface if derivations could either be written in a more minimalist way or if containers (or the system in general) could include only the smallest set of software they need to run.

                                    (edit)

                                    OK, I tracked it down:

                                    $ nix-store -q --referrers /nix/store/h610l0v3b3d6mqfwapgd8jb0jxilbd47-texlive-combined-2018.drv
                                    /nix/store/f0ir4m0ww8qjbialvjxrv7zzlmhsmrjc-dwarf-therapist-41.0.2.drv
                                    

                                    A package I maintain apparently used it as a build input… sigh

                                    Does this mean dwarf-therapist is still referencing texlive as far as the GC is concerned, even though it’s already built? A nix-collect-garbage -d didn’t clean it up.

                                    1. 2

                                      Seems relatively pointless, if an attacker can write source code to disk they can also just write precompiled binaries to disk (or just keep shell code in memory and never touch disk).

                                      1. 1

                                        Yeah. Requirements sometimes are that way. :-(

                                        More to the point though - it’s the same “if your users don’t need it, don’t include it” approach hank mentioned. You probably won’t need to compile programs on a production system, so why include a compiler, even one that a user would have to manually spelunk through the Nix store to use?

                                        1. 1

                                          Nix detects dependencies automatically and sometimes package maintainers don’t notice they accidentally did something to include one. It could be considered a bug with a ticket if it isn’t necessary for a program to function.

                                          1. 1

                                            I’m seeing a lot of things that use nativeBuildInputs sticking around in the Nix store on my setup.

                                            From the manual:

                                            since these packages only are guaranteed to be able to run then, they shouldn’t persist as run-time dependencies. This isn’t currently enforced, but could be in the future.

                                            And, indeed, I’m seeing them ref’ed by certain packages that needed to be compiled locally:

                                            $ nix-store -q --referrers /nix/store/imfm3gk3qchmyv7684pjpm8irvkdrrkk-gcc-7.3.0/bin/gcc
                                            /nix/store/imfm3gk3qchmyv7684pjpm8irvkdrrkk-gcc-7.3.0
                                            /nix/store/8xfm1g4vbv0mv0mn7zlny85dhmw3djvm-dfhack-base-0.44.12-r1
                                            /nix/store/fj0qxv4lqrgnva1yhn09f641373rbjvy-dwarf_fortress_unfuck-0.44.12
                                            

                                            So, it sounds like nativeBuildInputs should be GC’ed, but currently aren’t for some reason. I just assumed they would be. I’d be curious to see what’s producing extra copies of texlive for hank as well.

                                      2. 1

                                        You’re asking which derivations mention your TexLive derivation. You’re not asking what references the TexLive output. I wouldn’t worry about disk usage of these small derivations.

                                1. 3

                                  I know it wouldn’t help with caching but would it be possible to produce very small docker images if a Nix tool would only ship the right .so files and their dependencies? (like libpthread.so and a couple of others instead of the whole glibc folder).

                                  1. 4

                                    Nix doesn’t have knowledge down to that layer, but it could be possible. Another interesting thing along these lines is Nixpkgs is gaining the option to compile fully static binaries with musl for some packages.

                                  1. 1

                                    No blockchain

                                    Doesn’t scuttlebutt use one?

                                    1. 4

                                      Technically speaking your feed is an append-only log of data blocks.

                                      However, there’s typically only one node writing to it and it doesn’t have proof of anything (other than possession of a private key).

                                      1. 2

                                        No, rather than having a single centralized feed, each user has a feed that they replicate to their friends and friends of friends.

                                      1. 1

                                        The bufferbloat and quality were actually worse with the change in my case. Maybe my EdgeRouter Lite can’t handle it or something.

                                        1. 4

                                          I looked up a lot of discussion on this over in their forums…maybe some of that could help?

                                          1. 1

                                            Thanks but I’m not sure if it could help since I’m running OpenBSD on it and not the default OS.

                                        1. 5

                                          As exciting as this is, I’m wary about dependency in GNU tools, even though I understand providing an opembsd-culture-friendly implementation would require extra work and could be a nightmare maintainance, with two different codebases for shell scripts, but perhaps gmake could be replaced with something portable.

                                          1. 12

                                            This version of Wireguard was written in go, which means it can run on exactly 2 (amd64, i386) of the 13 platforms supported by OpenBSD.

                                            The original Wireguard implementation written in C is a Linux kernel module.

                                            A dependency on gmake is the least of all portability worries in this situation.

                                            1. 18

                                              While it’s unfortunate that Go on OpenBSD only supports 386 and amd64, Go does support more architectures that are also supported by OpenBSD, specifically arm64 (I wrote the port), arm, mips, power, mips. I have also implemented Go support for sparc64, but for various reasons this wasn’t integrated upstream.

                                              Go also supports power, and it used to run on the power machines supported by OpenBSD, but sadly now it only runs on more modern power machines, which I believe are not supported by OpenBSD. However, it would be easy to revert the changes that require more modern power machines. There’s nothing fundamental about them, just that the IBM maintainer refused to support such old machines.

                                              Since Go support both OpenBSD and the architectures mentioned, adding support in Go for OpenBSD+$GOARCH is about a few hours of work, so if there is interest there would not be any problem implementing this.

                                              I can help and offer advice if anyone is willing to do the work.

                                              1. 3

                                                Thanks for your response! I didn’t know that go supports so many platforms.

                                                Go support for sparc64, but for various reasons this wasn’t integrated

                                                Let me guess: Nobody wanted to pay the steep electricity bill required to keep a beefy sparc64 machine running?

                                                1. 25

                                                  No, that wasn’t the problem. The problem was that my contract with Oracle (who paid me for the port) had simply run out of time before we had a chance to integrate.

                                                  Development took longer then expected (because SPARC is like that). In fact it took about three times longer than developing the arm64 port. The lower level bits of the Go implementation have been under a constant churn which prevented us from merging the port because we were never quite synced up with upstream. We were playing a whack’a’mole game with upstream. As soon as we merged the latest changes, upstream had diverged again. In the end my contract with Oracle had finished before we were able to merge.

                                                  This could all have been preventable if Google had let us have a dev.sparc64 branch, but because Google is Google, only Google is allowed to have upstream branches. All other development must happen at tip (impossible for big projects like this, also disallowed by internal Go rules), or in forks that then have to keep up.

                                                  The Go team uses automated refactoring tools, or sometimes even basic scripts to do large scale refactoring. As we didn’t have access to any of these tools, we had to do the equivalent changes on our side manually, which took a lot of time and effort. If we had an upstream branch, whoever did these refactorings could have simply used the same tools on our code and we would have been good.

                                                  I estimate we spent more effort trying to keep up with upstream than actually developing the sparc support.

                                                  As for paying for electricity, Oracle donated one of the first production SPARC S7-2 machines (serial number less than 100) to the Go project. Google refused to pay for hosting this machine (that’s why it’s still sitting next to me as I type this).

                                                  In my opinion after being involved with Go since the day of the public release, I’d say the Go team at Google is unfortunately very unsympathetic to large scale work done by non-Google people. Not actively hostile. They thanked me for the arm64 port, and I’m sure they are happy somebody did that work, but indirectly hostile in the sense that the way the Go team operates is not compatible with large scale outside contributions.

                                                  1. 1

                                                    Having to manually follow automated tools has to suck. I’d be overwhelmed by the tedium or get side-tracked trying to develop my own or something. Has anyone attempted a Go-to-C compiler developed to attempt to side-step all these problems? I originally thought something like that would be useful just to accelerate all the networking stuff being done in Go.

                                                    1. 2

                                                      There is gccgo, which is a frontend for gcc. Not quite a transpiler but it does support more architectures than the official compiler.

                                                      1. 1

                                                        Yeah, that sounds good. It might have a chance of performing better, too. The thing working against that is the Go compiler is designed for optimizing that language with the gccgo just being coopted. Might be interesting to see if any of the servers or whatever perform better with gccgo. I’d lean toward LLVM, though, given it seems more optimization research goes into it.

                                                      2. 2

                                                        The Go team wrote such a (limited) transpiler to convert the Go compiler itself from C to Go.

                                                        edit: sorry, I misread your comment - you asked for Go 2 C, not the other way around.

                                                        1. 1

                                                          Hey, that’s really cool, too! Things like that might be a solution to security of legacy code whose language isn’t that important.

                                                    2. 1

                                                      But these people are probably more than comfortable with cryptocurrency mining 🙃

                                                    3. 3

                                                      Go also supports power, and it used to run on the power machines supported by OpenBSD, but sadly now it only runs on more modern power machines, which I believe are not supported by OpenBSD. However, it would be easy to revert the changes that require more modern power machines. There’s nothing fundamental about them, just that the IBM maintainer refused to support such old machines.

                                                      The really stupid part is that Go since 1.9 requires POWER8…. even on big endian systems, which is very pointless because most running big endian PPC is doing it on pre-POWER8 systems (there’s still a lot!) or a big endian only OS. (AIX and OS/400) You tell upstream, but they just shrug at you.

                                                      1. 4

                                                        I fought against that change, but lost.

                                                      2. 2

                                                        However, it would be easy to revert the changes that require more modern power machines.

                                                        Do you have a link to a revision number or source tree which has the code to revert? I still use a macppc (32 bit) that I’d love to use Go on.

                                                        1. 3

                                                          See issue #19074. Apparently someone from Debian already maintains a POWER5 branch.

                                                          Unfortunately that won’t help you though. Sorry for speaking too soon. We only ever supported 64 bit power. If macppc is a 32-bit port, this won’t work for you, sorry.

                                                          1. 3

                                                            OpenBSD/macppc is indeed 32-bit.

                                                            I kinda wonder if say, an OpenBSD/power port is feasible; fast-ish POWER6 hardware is getting cheap (like 200$) used and not hard to find. (and again, all pre-P8 POWER HW in 64-bit mode is big endian only) It all depends on developer interest…

                                                            1. 3

                                                              Not to mention that one Talos board was closer to two grand than eight or ten. Someone could even sponsor the OpenBSD port by buying some dev’s the base model.

                                                              1. 3

                                                                Yeah, thankfully you can still run ppc64be stuff on >=P8 :)

                                                      3. 2

                                                        This version of Wireguard was written in go, which means it can run on exactly 2 (amd64, i386)

                                                        That and syspatch make me regret of buying EdgeRouter Lite instead of saving up for an apu2.

                                                      4. 2

                                                        I’m a bit off with the dependency of bash on all platforms. Can’t this be achieved with a more portable script instead (POSIX-sh)?

                                                        1. 3

                                                          You don’t have to use wg-quick(8) – the thing that uses bash. You can instead set things up manually (which is really easy; wireguard is very simple after all), and just use wg(8) which only depends on libc.

                                                          1. 2

                                                            I think the same as you, I’m sure it is possibe to achieve same results using portable scripts. I’m aware of the conviniences bash offers, but it is big, slow, and prompt to bugs.

                                                        1. 3

                                                          OpenBSD support would be amazing.

                                                          1. 5

                                                            Want to help add it?

                                                            1. 6

                                                              Sure.

                                                              1. 7

                                                                Great! Poke me on Freenode, perhaps? I’m zx2c4 there. I can show you what needs to be implemented.

                                                          1. 3

                                                            Zulip sounds interesting.

                                                            Can you export all the data if you want to change providers, or if Zulip goes out of business/gets purchased? Can you export the discussions to a web-based format (as you can with mailing lists) for searchability?

                                                            1. 5

                                                              Zulip has already been purchased by Dropbox. And Zulip is open source so you can run your own.

                                                              1. 4

                                                                Yes, both of those things are important to me. It looks like there is an export.py tool, although the docs aren’t the greatest:

                                                                https://zulip.readthedocs.io/en/1.7.1/conversion.html

                                                                And it looks like they’re working on truly public archives:

                                                                https://github.com/zulip/zulip/pull/8135

                                                                Right now it’s a little annoying that you have to sign in to read messages, which they’re also working on changing. But at least it doesn’t require another login – you can log in with Github.

                                                                Overall, the choice was basically between Zulip and Discourse. It’s a little sad how mailing list infrastructure hasn’t kept up with the times.

                                                                1. 3

                                                                  I’m not a huge fan of the “new” style of collaboration (Slack, MSFT Teams). They try to combine the immediacy of real-time chat with some sort of structured discussion format, but there’s an implied pressure to provide an answer immediately. In a mailing list, it’s not expected for someone to reply to a message within minutes, and I believe that for most projects this is a better expectation - if nothing else than people are in different timezones.

                                                                  1. 4

                                                                    I’ve used instant messaging in different ways for 20 years, and I always felt that those tools are fundamentally asynchronous, that the answer is never expected immediately. Perhaps it would help if you never enable notifications for those tools? I never do, except for the tools that I use for personal communication with family.

                                                                    1. 4

                                                                      there’s an implied pressure to provide an answer immediately

                                                                      Maybe it’s better with good threading support since the messages don’t get lost as much and it’s less an issue if you reply later.

                                                                      1. 3

                                                                        I feel like there needs to be adequate division between channels of communication in order for it to work successfully. A bad example is Discord: a lot of servers will have a #general channel where there are often two or three conversations going on at the same time, and replying to someone’s question more than a few minutes later seems futile.

                                                                        Having used Zulip for a few months at work, I feel like the named subtopics within streams can handle slower conversations, but it might be a matter of corralling the userbase away from these catch-all channels.

                                                                        1. 2

                                                                          Well, a few people had asked for an IRC channel, which I haven’t had success with. So I hope to kill 2 birds with one stone here. It’s an experiment.

                                                                          1. 2

                                                                            Please report your findings (maybe a blog post?) at some point. I’m super interested to hear how it goes, and I imagine other folks would be too.

                                                                    1. 3

                                                                      Any NixOS user can tell me if the only way to have the latest version of everything is to use the unstable channel? Basically I want Arch but with an easy undo button so I don’t look like a fool at work when upgrades go bad.

                                                                      1. 8

                                                                        I’m an Arch -> NixOS convert and run unstable, it’s basically what you’re looking for.

                                                                        1. 2

                                                                          Thanks you very much.

                                                                        2. 3

                                                                          Even better, you can run stable, but then manually pull unstable packages out and install them. I ran 17.09 but pulled in the unstable LLVM 6.0 for example. The same would work for firefox/chrome or other packages.

                                                                          for example:

                                                                          let
                                                                          unstablePkgs = import ((import <nixpkgs> { }).fetchFromGitHub {
                                                                            owner = "NixOS";
                                                                            repo = "nixpkgs";
                                                                            rev = "Some future unstable nixos version with a version of firefox you want";
                                                                            sha256 = "XXX";
                                                                          }) { config = { }; }
                                                                          in
                                                                            unstablePkgs.firefox
                                                                          

                                                                          If you paste that into a file default.nix and run “nix-build” (or now in this release “nix build”) it will give you exactly the right firefox like magic you can also install it into your user profile with nix-env -i.

                                                                          edit: Or what you could do, is run unstable, but pull in old packages from the past if you don’t want the new one.

                                                                          1. 2

                                                                            like magic

                                                                            that is also my experience in using nix.

                                                                            1. 1

                                                                              That’s a super easy way to do it. If you wanna add unstable and have a tool track the commits, you can use nix-channel, add it then install it via nix-env and a prefix.

                                                                              1. 1

                                                                                Any NixOS user can tell me if the only way to have the latest version of everything is to use the unstable channel? Basically I want Arch but with an easy undo button so I don’t look like a fool at work when upgrades go bad.

                                                                                True, I have never tried having multiple channels, but it works too.

                                                                          1. 2

                                                                            It’s a shame it’s open-core.

                                                                            1. 7

                                                                              Spanner/F1 and FoundationDB were closed. CochroachDB was first (AFAIK) of those competing with Spanner to give us anything at all. Let’s give them credit, eh? ;)

                                                                              1. 3

                                                                                FoundationDB was open (in some form) before it disappeared into Apple.

                                                                                1. 4

                                                                                  I don’t believe any of the interesting tech was open source. It was sort of the opposite of open-core, with a proprietary core but some ancillary stuff like an SQL parser that was open source. That other stuff is what disappeared when Apple bought them (GitHub deleted, packages pulled from repos, etc.), which caused a bit of a stir as they disappeared with no warning and some people had been depending on the packages.

                                                                                  1. 2

                                                                                    I never heard that. Ill look into it further. Thanks.

                                                                                    1. 1

                                                                                      Looking into it, the core DB that was what was really valuable was closed with some peripheral stuff open. This write-up goes further to say it was kind of fake FOSS that lured people in. I don’t have any more data to go on since Apple pulled all the Github repos.

                                                                                    2. 1

                                                                                      It doesn’t seem to me that CockroachDB competes with Spanner. I’d thought of MongoDB before CockroachDB.

                                                                                      1. 8

                                                                                        It’s explicitly the origin for CockroachDB. Spanner less the cesium clocks.

                                                                                  1. 2

                                                                                    Can this be done on linux?

                                                                                    1. 5

                                                                                      There are various ORMs/mappers, but most advice you’ll find (and what I’ll say, also) is to not use them. Wrap a database connection in a struct that has methods to do what you want. Something I’ve found conditionally useful are generators to write the function to populate a struct from a database row.

                                                                                      The community will also say to not use web frameworks, and again I’d agree. The stdlib http package provides a stable foundation for what you want to do. You’ll have more luck looking for packages that do what specific thing you want, rather than thinking in terms of frameworks.

                                                                                      All that said, some coworkers like echo but I can’t for the life of me understand why. Any web-oriented package shouldn’t need to give a shit if it’s hooked up to a tty or not.

                                                                                      1. 3

                                                                                        The problem is that when you do search and filtering on various conditions (like in a shop) you don’t want to resort to sql string stitching, I wasn’t able to find anything nice when I looked at the docs, for example in gorm:

                                                                                        db.Where("name = ? AND age >= ?", "jinzhu", "22") - I expect that when you have 20 conditions with different operators and data types you will end up having a bad time.

                                                                                        1. 4

                                                                                          I’m at a small scale but I just write the query entirely and or do string concatenation. I’m having a fine time though. I just use sqlx.

                                                                                          1. 2

                                                                                            sqlx

                                                                                            https://godoc.org/github.com/jmoiron/sqlx#In is pretty useful when you need it.

                                                                                            1. 3

                                                                                              I admit I don’t understand how to use that from the doc string. Could you show a simple example or elaborate?

                                                                                              1. 4

                                                                                                It’s useful for queries using the IN keyword, like this:

                                                                                                query, params, errIn := sqlx.In("SELECT column1 FROM table WHERE something IN (?);", some_slice)
                                                                                                // if errIn != nil...
                                                                                                rows, errQuery := db.Query(query, params...)
                                                                                                
                                                                                            2. 2

                                                                                              We built something in house that is very similar in spirit to sqlx but adds a bunch of helpers.

                                                                                              https://github.com/blendlabs/spiffy

                                                                                            3. 4

                                                                                              https://github.com/Masterminds/squirrel (which kallax uses) seems somewhat akin to the SQLAlchemy expression API. (And yeah, to me, that’s a great part of SQLAlchemy; I’ve hardly used its ORM in comparison.)

                                                                                              I went from Python + heavy use of the SQLAlchemy expression API to Go and got by OK with just stdlib, but part of that was that the work in Go had far less complicated queries most of the time. So, not the best comparison maybe.

                                                                                              1. 2

                                                                                                I support the advice to not use mappers like ORMs, but I also agree with what you said. The middle ground seems to be query builders.

                                                                                                If you use Postgres as your DBMS by any chance, I advise you to make sure that the query abstraction layer of your choice doesn’t do query parameterization by string interleaving but utilizes PQexecParams underneath instead.

                                                                                                1. 1

                                                                                                  I haven’t used it but I think Dropbox built a library for that. https://godoc.org/github.com/dropbox/godropbox/database/sqlbuilder

                                                                                              1. 2

                                                                                                I’m curious about NixOS. Atomic updates sounds really nice. Can NixOS give me Arch but with atomic updates and rollback? Right now on Arch I run the 4.15 kernel but a default NixOS install seems to have 4.9. Maybe I just need to use the unstable channel but I saw that it wasn’t recommended in production.

                                                                                                1. 41

                                                                                                  Thanks to the Recurse Center for inviting me to speak and for making the video. I’m here if anyone has questions.

                                                                                                  1. 21

                                                                                                    A very non-technical question: Why should Xi “only” be an editor for the next 20 years? In terms of text editors, that’s not that long. People, like me, use Editors that are nearly twice as old as I am, and the reasons don’t seem to be tied to performance or the internal structure of the implementations, but rather a core “philosophy” regarding how things are done, or how the programmer should relate to text. What does Xi have to offer regarding these “practical” qualities, that have made, for example Emacs or Vi(m) last for so long? Does Xi see itself in such a certain tradition, having a certain ideal that you aspire to, or do you set your own terms? These could seem important if one intends to write an editor that should be practically used, which is what I gathered from the video, as opposed to being a “purely academic” experiment, which would obviously have different goals and priorities.

                                                                                                    1. 4

                                                                                                      Do you plan on doing a Linux frontend yourself and would it matter performance-wise? I saw that some people are working on a gtk+ frontend but I was wondering if it will be as fast as the mac one.

                                                                                                      1. 4

                                                                                                        In my ideal world, there’d be a cross-fertilization of code and ideas so the linux front-end would be just as nice and performant as the mac one, but it’s unlikely at this point I’ll take it on myself.

                                                                                                        1. 2

                                                                                                          I just tried xi-gtk and it’s very fast. Not sure what it’s like compared to the swift one but it’s a whole lot faster than gedit.

                                                                                                          1. 1

                                                                                                            nice, thanks!

                                                                                                        2. 4

                                                                                                          Also, here is a cool demo of async loading of big text files – you can navigate and I think even edit while loading:

                                                                                                          https://youtu.be/sPhpelUfu8Q?t=1601

                                                                                                          Using immer, Clojure-like immutable data structures in C++:

                                                                                                          https://github.com/arximboldi/immer

                                                                                                          The editor is a demo of the library: https://github.com/arximboldi/ewig

                                                                                                          1. 3

                                                                                                            I just watched the video. It looks really interesting, although a lot of it was over my head!

                                                                                                            I more or less understand the process model, async architecture, and distributed data structures. I like that part – very Unix-y.

                                                                                                            But there were a lot of rendering terms I didn’t understand. Maybe because some of it is Mac-specific. But also some of the OpenGL issues. Is there any background material on text rendering you’d recommend?

                                                                                                            Also, I don’t understand the connection to Fuschia? I was under the impression that Fuschia was more consumer-facing, and Xi is more developer-facing. That is, I imagine most consumers don’t have text editors installed. There is no text editor on Android or ChromeOS.

                                                                                                            Or is xi more general than a vi/emacs replacement – is it meant to be used as part of a browser for implementing text boxes?

                                                                                                            1. 4

                                                                                                              Glad you enjoyed the talk!

                                                                                                              Unfortunately, there really isn’t a lot of material on text processing, especially from a modern perspective. A lot of what I learned about rendering came from reading other code (alacritty in particular), and talking to people like Patrick Walton and my teammates on Chrome and Android.

                                                                                                              There is an EditText widget on Android (a good chunk of my career involved working on it), but you certainly wouldn’t want to write code (or long-form text) in it. My goal with xi is to make a core lightweight and performant enough it can be used in such cases, easily embedded in apps, yet powerful enough for cases where you really do need a dedicated editor application.

                                                                                                            2. 2

                                                                                                              I feel like it’s fairly out of my league, but I’ve been thinking about implementing a Sublime Text-like editor (multiple cursors, smart brackets/quotation marks) for arbitrary text fields in web sites. Would it be possible to use Xi as a backend for something like that? Perhaps via compilation to WASM?

                                                                                                              1. 8

                                                                                                                Eventually it is my hope that something like that could work. There are some technical details (the current implementation uses threads), so it’s not an easy project. In the meantime, the excellent CodeMirror does multiple selections, and is very widely used embedded in websites.

                                                                                                            1. 1

                                                                                                              Is Elasticsearch light enough to run on a cheap vps with 1 GB of ram?

                                                                                                              For game servers I monitor logs and run commands in real-time for certains events. Can I do something similar with Elasticsearch? It would be simpler if I’m already using it.

                                                                                                              1. 6

                                                                                                                The ELK stack is pretty heavy from my experience. If your usage is simple and Andy to get your hands dirty, I’d really advise your to look at https://github.com/oklog/oklog !

                                                                                                                1. 1

                                                                                                                  Thanks!

                                                                                                              1. 1

                                                                                                                I like it but I hate sticky headers.

                                                                                                                1. 2

                                                                                                                  That makes sense, thanks. I’ll implement that behavior that hides the header until you start scrolling up again.

                                                                                                                1. 12

                                                                                                                  For militant decentralisation, https://git.scuttlebot.io/%25n92DiQh7ietE%2BR%2BX%2FI403LQoyf2DtR3WQfCkDKlheQU%3D.sha256 seems like a cool solution. No internet required.

                                                                                                                  1. 1

                                                                                                                    You can overwrite other people’s master branch with that?

                                                                                                                    1. 3

                                                                                                                      Don’t know, I’ve never used it. http://scuttlebot.io/apis/community/git-ssb.html#usage says “You can only push to SSB git repos that you created, not ones created by other users”, but I’m guessing there’s more to the story :)

                                                                                                                      1. 3

                                                                                                                        Only if they give you the key.

                                                                                                                        Scuttlebot is built on SSB, which is a content-addressable (by Sha) distributed database, where all entries are signed by a private key, and your local replica only copies records published by a key it follows.