1. 2

    What’s wrong with current’s implementation written in ANSI C? Seems like a waste of human resources, in my opinion.

    1. 32

      What’s wrong with current’s implementation written in ANSI C?

      If you want to script a Go application, it’s a hell of a lot easier if the VM is native instead of trying to fight with C interop issues.

      Seems like a waste of human resources, in my opinion.

      Incredible that with zero insight into the business requirements or technical motivations behind a project, you still consider yourself qualified to judge the staffing allocations behind it

      1. 7

        Incredible that with zero insight into the business requirements or technical motivations behind a project, you still consider yourself qualified to judge the staffing allocations behind it.

        You are definitely right about this.

        If you want to script a Go application, it’s a hell of a lot easier if the VM is native instead of trying to fight with C interop issues.

        As @tedu commented, I understand the motivation behind it, thanks for pointing it out too.

        1. 2

          They give some further explanation of the technical requirements that led them not to pick up Shopify’s version here, which I found intersting: https://github.com/Azure/golua/issues/36.

          The TL;DR is they’ve been scripting Helm for a while using another Lua-in-Go engine, but it isn’t updating to 5.3, which they very much want, along with closer conformance to the Lua spec, plus they have some design ideas that they feel would make debugging etc easier.

          1. 3

            To each their own, but I’m a little perplexed that strict 5.3 conformance would be a driver. I’m using gopher-lua which is like 5.1 but not exactly, but it’s just fine. What I need is some scripting language to embed, it’s close enough to lua it’s not really learning a whole new whatever, and it interfaces nicely with go (nicer than a strict port of the stack api would).

            I don’t know jack about helm, but after seven seconds of research I can’t reverse engineer a requirement for lua 5.3 accept no substitutes.

            1. 1

              It’s possible that they have some need of true 64-bit integers, which only came to Lua in 5.3. That’s the only important change I can think of.

      2. 5

        Writing memory-safe C is an artform and a craft.

        Writing safe code with Go is something a trained orangutang can do.

        1. 3

          Lua bytecode is memory-unsafe in the reference implementation, for starters.

          1. 2

            Do you have a link or more details?

            1. 1

              My upload server was down, sorry I didn’t link this originally. Here’s a slide deck (pdf) of a presentation I gave at Houston Area Hackers with almost all the details you could want.

              I still have to update this; there are quite a few more tricks you can pull. I ended up creating a pure-Lua dlsym() implementation for the project that spurred all this research.

              1. 2

                Hmmm… Do reimplementations like this one remove support for lightuserdata then? I’m having a hard time imagining a lua interpreter for arbitrary bytecode that supported lightuserdata but could nevertheless guarantee that those pointers were safe.

                1. 1

                  Yeah. In gopher-lua, userdata actually looks kind of like lightuserdata, which doesn’t exist. When both host and lua state share an allocator, the difference becomes less meaningful.

                  1. 1

                    Looks like it’s unimplemented. lightuserdata is a pretty damn leaky Lua API. Bytecode itself is even more leaky in the Lua C implementation - arbitrary Lua bytecode is as expressive as C. No surprise that MS seemingly wanted to make an API/ABI compatible implementation that fixed some unnecessary leakiness.

            2. 2

              It’s annoying to interface with from go code.

              1.  

                seems like something that should be fixed in Go rather than reimplementing everything written in C.

                1.  

                  For better or worse, it’s a result of deliberate design trade-offs in go, so it’s unlikely to change.

            1. 2

              This is very cool. Just a couple of hours ago I was looking to execute what (at least in my brain) is a SQL query against a csv file. Installed this just now and I’m impressed so far! Thanks for sharing.

              1. 14

                It’s a useful little tool. Back when I worked with CSVs daily I used q, but eventually just started using SQLite directly. The .import command works fabulously. It doesn’t do full CSV parsing, just dumb split-on-delimiter, but that worked for me.

                For example, given this CSV file:

                id,name
                1,peter
                

                You can do this:

                sqlite> .separator ","
                sqlite> .import test.csv test
                

                It pulls the column names automatically, as you can see:

                sqlite> .schema
                CREATE TABLE test(
                  "id" TEXT,
                  "name" TEXT
                );
                

                Unfortunately it makes every column TEXT, which is fine until you need numeric comparison. Which you often do. So I would import into a temp table, copy the schema from the .schema command, change the types appropriately, and do an insert-select. Like this:

                sqlite> .separator ","
                sqlite> .import test.csv test_import
                sqlite> .schema
                CREATE TABLE test_import(
                  "id" TEXT,
                  "name" TEXT
                );
                sqlite> CREATE TABLE test(
                   ...>   "id" INTEGER,
                   ...>   "name" TEXT
                   ...> );
                sqlite> insert into test select * from test_import;
                

                q is faster for a small one-off, but having a full persistent SQLite database was surprisingly helpful for my work. A lot of the CSV files I was looking at had a particular schema, so in the end I had a number of schemas and queries saved as SQL scripts I would execute with .read.

              1. 5

                I last touched C++ in 1995 or so working at Fidelity Investments doing a large Defined Benefits app in MFC. The language really does seem to have changed a lot.

                Many of the things that used to annoy the crud out of me seem to have been excised or mitigated. I’m trying to bring myself up to date with it, mostly because I’m super enjoying KDE and thinking about contributing.

                1. 4

                  I recalled lots of old C++ being done like C. That caused many problems. I don’t know if that was your experience. Since I was countered on saying C/C++, I asked pjmpl on Hacker News for any descriptions of or resources on “modern” C++ to know how it’s done today. Here’s his answer with some techniques and books in case you find it useful:

                  https://news.ycombinator.com/item?id=10208786

                  Obviously, I’m interested in any Lobsters that do C++ weighing in on whether that was a good answer. Might also know some other resources. Obviously, anyone learning will have to read up on the new standards, too. They’re doing really, neat improvements.

                  1. 6
                    • references for out parameters

                    This can make code a little hard to follow though it’s traditional C/C++ style. I like returning values and trusting the compiler to use moves instead of copies. Complex return types can be tuples. I don’t know enough about optimization to know how big a performance hit this is over references.

                    1. 6

                      The C++ standard actually allows much more aggressive return value optimization than C, so it’s usually not a performance hit at all.

                      1. 3

                        The C++ standard actually allows much more aggressive return value optimization than C

                        I don’t believe that’s correct. The “return value optimisation” just amounts to storing the return value directly into its target location, rather than copying it after the fact, and that’s just as possible in C as it is in C++.

                        1. 2

                          Unfortunately I can’t dig up the source, but if I recall correctly there is some rule difference between C and C++ that lets C++ compilers perform RVO more often.

                          It may have been that C cannot RVO across compilation units. Because C++ can, without LTO, as you can see in this example. The signature as written is Foo factory() however the generated machine code has the signature Foo* factory(Foo*). I don’t think C compilers are allowed to export altered function signatures like that.

                          1. 3

                            I don’t think C compilers are allowed to export altered function signatures like that.

                            The signature is not changed; the ABI specifies how return value and parameters are passed back and forth, and the compiler is generating code according to both. The Sys V ABI (and the architectural/C++ supplement for it) specify this - the return object’s storage is allocated by the caller and a pointer to it is passed via a register. The C ABI specifies exactly the same thing, and the equivalent C compiles down to the exact same code: https://godbolt.org/z/fMBi36

                            if I recall correctly there is some rule difference between C and C++ that lets C++ compilers perform RVO more often

                            To be honest I find this hard to believe, and I can’t conceive why it would be the case - I’d be very interested if you could dig up a link to any information which confirms it.

                            (Maybe what you read was actually that the RVO is more useful/important in C++ than it is in C. This is certainly true, since it allows copy-constructor calls to be elided).

                            1. 1

                              If what you’re saying is true, that means return by value never had any performance implications in C whatsoever, since RVO essentially always happens, it’s part of the ABI. And that everyone using out parameters for structs for performance has simply been misguided / ignorant of this. So I feel like there must be more to the story there.

                              To be honest I find this hard to believe, and I can’t conceive why it would be the case - I’d be very interested if you could dig up a link to any information which confirms it.

                              I really wish I had that post too, I looked through my history for 30 minutes to no avail. If I recall correctly, which I’m now second guessing whether I do, the author demonstrated a situation where the C++ compiler generated optimized code but the C compiler did not. Oh well.

                              1. 2

                                If what you’re saying is true, that means return by value never had any performance implications in C whatsoever

                                That’s not exactly what I’m saying; the ABI is a huge factor and certain other ABIs may not lend themselves to the same optimisation.

                                Note that technically, “RVO” specifically refers to the C++ case where copy construction is elided (i.e. C doesn’t have “RVO”, it’s just that C doesn’t need the RVO optimisation because it doesn’t have copy constructors). But from your first post I’m reading your use of “RVO” as meaning “avoids an object copy” and certainly that’s possibly in both C and C++, if the ABI allows for it.

                                If the ABI doesn’t allow for in-place “construction” of the return object, then passing output-parameter structs explicitly as pointers can still be a win.

                                1. 1

                                  Does the 32-bit sys V ABI handle structs the same way? The 64-bit ABI is better in a lot of ways, if this is one such way that would explain that.

                                  1. 2

                                    Does the 32-bit sys V ABI handle structs the same way?

                                    Yes, it does.

                                    However, there’s another thing I forgot to mention: passing a struct parameter via explicit pointer (or reference) can still be an occasional win over relying on in-place return value construction, even if the ABI does allow for that - because with the latter, the compiler needs to make sure it doesn’t accidentally create an aliasing pointer. That is, if you have something like:

                                    struct foo_s { int a; int b; int c; }
                                    foo_s gfoo;
                                    foo_s foofunc();
                                    
                                    void f()
                                    {
                                        gfoo = foofunc();  // <-- here
                                    }
                                    

                                    In this case the compiler can’t pass the “hidden” return value pointer as &gfoo, because it’s possible that the foofunc() function reads values from gfoo while also storing values into its return value (via the hidden return value pointer). See https://godbolt.org/z/Zfacwj

                                    1. 2

                                      Ah yes, that makes perfect sense!

                                      I’m learning a lot today, thank you!

                      2. 5

                        Yes references hide just enough information to be dangerous in my experience.

                      3. 2

                        Speaking of return types, tuples combined with structured bindings are da bomb. It basically looks like you are writing Python - you return a pair or a tuple from a function, and then use structured bindings to retrieve them directly into variables. So sweet.

                    1. 4

                      I’m amazed by the level of detail in this post! Incredibly interesting comparisons and analysis.

                      1. 10

                        I’m amused by the level of Cupertino Kremlinology. But yes, this (like its predecessors) is a high-quality writeup by someone with some depth in both digital cartography and business models.

                        Suggested “practices” tag, because the “AI” is apparently (at least in part) 5000 new employees in the new Hyderabad office. The more you know!

                        1. 4

                          google employs similar numbers to edits its maps. When I was still at TomToms map unit (former TeleAtlas) 6 years ago google had already more people manually editing its map than TomTom had employees across all business units worldwide. Map making is suprisingly complex and requires a lot of manual work.

                          1. 2

                            I worked on the US Census Bureau’s TIGER maps over a decade ago. Fixing some of their legacy maps (like, some of them just digitized paper maps) to modern commercial maps while preserving historical data was a great summer job! We churned through it pretty quickly with software help. Flyover counties in North Dakota could be cleaned up in an hour or two, while Queens, NY took two weeks (and the beefiest machine in the house, a Sun Blade 2500).

                      1. 3

                        Um. Following this link I got redirected to some kind of spam website, that was blocked by my browser.

                        Edit: clicked through a bunch more times to try and reproduce, got something slightly different:

                        1. 2

                          I’ve seen this on compromised WordPress sites before. If it’s the same as what I investigated previously, they do something like push the spam/ad/etc. to 1% of traffic and that makes it difficult to inspect/discover.

                          1. 1

                            Does it say Comcast in there? Could that be targeted to that connection?

                            1. 1

                              That’s… worrying. It’s a bog-standard wordpress site. What happens if you go to https://zwischenzugs.com?

                              1. 1

                                I clicked through a dozen times and nothing happened. It definitely didn’t happen every time on the original link either.

                                1. 11

                                  Looks like it’s a malicious ad coming in. Hard to say which ad network it came from, since the site is loading an obscene number of them…

                            1. 2

                              supported by the Mozilla WebRender rendering engine

                              So… electron.rs? ☹️

                              But, no javascript? 😀

                              I’m so conflicted.

                              1. 12

                                So… electron.rs? ☹️

                                Doesn’t seem so: https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-fps-how-webrender-gets-rid-of-jank/ & https://github.com/servo/webrender & https://github.com/servo/webrender/wiki

                                As I seem to understand, WebRender is nowhere close to be an Electron alternative. Seems to be an efficient and modern rendering engine for GUI and provide nothing related to JavaScript/Node/Web API.

                                So it looks like you can be free of conflict and enjoy this interesting API :) . I personally definitely keep an eye on it for my next pet project and find it refreshing to have an API for UI that look both usable and simple.

                                1. 4

                                  If you’re a fan of alternative, Rust-native GUI’s, you might want to have a look at xi-win-ui (in the process of being renamed to “druid”). It’s currently Windows-only, because it uses Direct2D to draw, but I have plans to take it cross-platform, and use it for both xi and my music synthesizer project.

                                  1. 1

                                    Please, give us some screenshots! ;)

                                    1. 1

                                      Soon. I haven’t been putting any attention into visual polish so far because I’ve been focusing on the bones of the framework (just spent the day making dynamic mutation of the widget graph work). But I know that screenshot are that important first impression.

                                      1. 1

                                        Please do submit a top-level post on lobste.rs once you add the screenshots :)

                                        1. 1

                                          Will do. Might not be super soon, there’s more polishing I want to do.

                                          1. 1

                                            Thanks! :) And sure, take your time :)

                                2. 6

                                  If comparing with Chromium stack, Webrender is similar to Skia, and this is GUI toolkit on top of it, instead of on top of whole browser. BTW, there’s example of app that has whole (non-native) UI on top of Skia: Aseprite.

                                  (AFAIK, Skia is something a la Windows GDI, immediate mode, and Webrender is scene graph-style lib, more “retained-mode”)

                                  And seems that, despite there’s no components from real browser, Azul has DOM and CSS. So, Azul is something in the spirit of NeWS and Display Postscript, but more web-ish, instead of printer-ish?

                                  1. 4

                                    There is also discussion of making an xml format for specifying the dom, like html.

                                    1. 1

                                      It’s using Mozilla’s WebRender, so how about XUL?

                                      1. 1

                                        Considering that Mozilla is actively trying to get rid of XUL, doing anything new with it seems like a bad idea.

                                        But also, if I understand what XUL is correctly it’s mostly a defined list of widgets over a generic XML interface, if I understand that proposal properly it’s to make the list of widgets completely user controllable (though there will no doubt be some default ones, including HTML like ones).

                                  2. 1

                                    WebRender is basically a GPU-powered rectangle compositor, with support for the kinds of settings / filters you can put on HTML elements. It’s nowhere near the bloated monstrosity that is electron.

                                  1. 2

                                    This is interesrting, but I’m seeing / hearing SO much more lately about Clang, I wonder where GCC’s future lies.

                                    1. 13

                                      GCC isn’t going anywhere as it supports a wide range of platforms and targets that are not the focus of Clang, among other reasons.

                                      1. 7

                                        GCC also still produces faster code in most cases.

                                        1. 4

                                          That has been my experience with ldc2 vs gdc: better optimisation in gdc for the cases I cared about.

                                          1. 2

                                            Then why are the Linux kernel and some other projects switching to Clang? (Honest, stupid question. I’m WAY rusty on the C world :)

                                            1. 11

                                              Great question! But Linux actually isn’t, sort of. Most Linux distros still ship with gcc-compiled Linux, except Android. I don’t know exactly why Android switched, but I wouldn’t be surprised if clang works better when compiling to ARM, since Apple built clang to compile iOS.

                                              Linux has loads of reasons to compile with clang, even if the releases are still gcc-compiled. Just checking the warnings clang produces to compare with gcc provides a lot of value. But clang is more than a compiler, it’s a “set of reusable C++ libraries for building powerful source-level tools”, like the clang static analyzer. YouCompleteMe, the popular vim plugin does semantic code completion using libclang. So making Linux compile on clang opens up a lot of opportunities to use different tooling.

                                              Building Linux with a different compiler has other advantages too, even if that compiler wasn’t clang. Compiling on multiple C compilers helps suss out obscure undefined or implementation-defined behavior bugs, or even compiler bugs! I don’t have an references on hand but the Linux on clang effort has found incorrect Linux code that happened to work due to one or more gcc bugs. There is an alternative rust compiler written in C++, mrustc that partially exists for this same reason.

                                              Historically Linux has been gcc-only because it relies on lots of gcc features and extensions. Compiling with a totally different toolchain like MSVC is just a non-starter. Porting to clang has actually been viable because clang aims to be mostly gcc compatible, from compiler flags to special syntaxes to builtins, even down to many implementation-defined behaviors.

                                              Lastly, getting Linux to compile with clang keeps the option of switching open. Linux might switch to clang one day if clang produces faster code than gcc on amd64. It’s not like Linux could switch overnight, it’s taken years of hard work to get this far. Android made the switch only recently. And even if most Linux distros did switch to building with clang, gcc would still be supported by Linux for many of these same reasons, in addition to gcc supporting many architecture targets that clang doesn’t (as mentioned by @trn).

                                              1. 3

                                                I get it! So it’s more like Clang is a new, interesting toolchain that it makes sense to support because being compatible means people can use it for a variety of interesting analysis and optimization projects.

                                                Thanks!

                                                1. 3

                                                  https://groups.google.com/a/chromium.org/d/msg/chromium-dev/bIWc8vFMF-w/wfMJsNHvDAAJ lists why the Chromium team pushed to get Chromium to compile with Clang on Windows. I forget how much overlap there is with reasons to favor Clang over GCC (having an open-source compiler certainly doesn’t apply there).

                                              2. 5

                                                FWIW aren’t many projects like Linux aiming for compatibility with Clang and to build with both, rather than a complete switch-over?

                                                1. 4

                                                  No one else has seemed to mention licensing. I’ve always felt Clang was a response to the GPL licensing around gcc. After all, Apple has slowly been removing as much GPL code as possible from MacOS.

                                                  1. 4

                                                    clang in my book is predominantly the response to GCC not wanting to modularise their system. Apple has issues with the GPL, but I would expect that to be more of a nice addon for them.

                                                    (See https://lwn.net/Articles/582697/ for background)

                                                    1. 0

                                                      I sometimes wonder if the world would be a better place if we all just agreed that GPL-ish licenses were overly restrictive and counter productive and that BSD/MIT-ish licenses were Good Enough.

                                                    2. 2

                                                      Among other things, Clang has a lot of great security features like (Cross-DSO) Control Flow Integrity. Android is using CFI for their version of the Linux kernel already.

                                                      Also, clang (well, any LLVM-backed compiler) is always a cross-compiler (and LLD is always a cross-linker), so you can completely avoid the nightmare of toolchain management (x86_64-unknown-linux-gnu-gcc-omgwtf)

                                                      1. 1

                                                        Yeah, building cross compliation toolchains is a PITA to say the least.

                                              1. 18

                                                I’m not certain this is the best means to achieve this effect, vs function pointers. While the optimization helps at runtime, reading the code still involves tracing through this mess instead of a dedicated init function.

                                                I’d also be concerned that not every chmod variant behaves identically with respect to extended attributes, etc. By cycling through a list that can change halfway through a group of files, you may have inconsistent results depending on file order.

                                                1. 1

                                                  I suspect you are basically calling for a JIT in this case. This method might be better if function signatures might diverge.

                                                  1. 1

                                                    You’d store a pointer to a wrapper function with a common signature. You could even make the default value of the function pointer point to a “try everything” function, that calls each of the methods in sequence and sets the pointer to the first one that works. Then you simply use that pointer everywhere, and the first time you call it, it would change itself.

                                                    Something like this:

                                                    typedef int (*set_mtime_func_t)(char* path, int64_t mtime);
                                                    
                                                    set_mtime_func_t best_set_mtime;
                                                    
                                                    int set_mtime_method_1(char* path, int64_t mtime);
                                                    int set_mtime_method_2(char* path, int64_t mtime);
                                                    int set_mtime_method_3(char* path, int64_t mtime);
                                                    
                                                    int set_mtime_method_tryall(char* path, int64_t mtime) {
                                                      int rc = -1;
                                                      if ((rc = set_mtime_method_1(path, mtime)) == 0) {
                                                        best_set_mtime = set_mtime_method_1;
                                                      }
                                                      else if ((rc = set_mtime_method_2(path, mtime)) == 0) {
                                                        best_set_mtime = set_mtime_method_2;
                                                      }
                                                      else if ((rc = set_mtime_method_3(path, mtime)) == 0) {
                                                        best_set_mtime = set_mtime_method_3;
                                                      }
                                                      return rc;
                                                    }
                                                    
                                                    set_mtime_func_t best_set_mtime = set_mtime_method_tryall;
                                                    
                                                    1. 1

                                                      Which then requires two function calls, due to the wrapper function, as opposed to one with the rsync technique

                                                      1. 1

                                                        The overhead of a function call is negligible compared to the overhead of a syscall. And I’m not convinced a function call would be significantly slower than a switch branch, but feel free to prove me wrong on that one.

                                                1. 15

                                                  That’s not what you are supposed to use asserts for. This are very different things. It’s a coding horror that makes you shriek and run away when you see it. In my fact, that’s my Halloween costume this year, using asserts to validate network input.

                                                  :)

                                                  1. 21

                                                    They aren’t using asserts to check network input. Their assert_return macro returns the second value if the condition fails. It’s not actually an assert, it’s a check. It’s just a poorly named macro.

                                                  1. 21

                                                    So I think I’m a bit late for the big go and rust and garbage collection and borrow checker discussion, but it took me a while to digest, and came up with the following (personal) summary.

                                                    Determining when I’m done with a block of memory seems like something a computer could be good at. It’s fairly tedious and error prone to do by hand, but computers are good at monotonous stuff like that. Hence, garbage collection.

                                                    Or there’s the rust approach, where I write a little proof that I’m done with the memory, and then the computer verifies my proof, or rejects my program. Proof verification is also something computers are good at. Nice.

                                                    But writing the proof is still kind of a pain in the ass, no? Why can’t I have computer generated proofs? I have some memory, I send it there, then there, then I’m done. Go figure out the refs and borrows to make it work, kthxbye.

                                                    1. 18

                                                      But writing the proof is still kind of a pain in the ass, no? Why can’t I have computer generated proofs? I have some memory, I send it there, then there, then I’m done. Go figure out the refs and borrows to make it work, kthxbye

                                                      I’m in the middle of editing an essay on this! Long story short, proving an arbitrary code property is undecidable, and almost all the decidable cases are in EXPTIME or worse.

                                                      1. 10

                                                        I’m kinda familiar with undecidable problems, though with fading rigor these days, but the thing is, undecidable problems are undecidable for humans too. The impossible task becomes no less impossible by making me do it!

                                                        I realize it’s a pretty big ask, but the current state of the art seems to be redefine the problem, rewrite the program, find a way to make it “easy”. It feels like asking a lot from me.

                                                        1. 10

                                                          The problem is undecidable (or very expensive to decide) in the most general case; what Rust does is solve it in a more limited case. You just have to prove that your usage fits into this more limited case, hence the pain in the ass. Humans can solve more general cases of the problem than Rust can, because they have more information about the problem. Things like “I only ever call function B with inputs produced from function A, function A can only produce valid inputs, so function B doesn’t have to do any input validation”. Making these proofs without computer assistance is no less of a pain in the ass. (Good languages make it easy to enforce these proofs automatically at compile or run time, good optimizers remove redundant runtime checks.)

                                                          Even garbage collectors do this; their safety guarantees are a subset of what a perfect solution would provide.

                                                          1. 3

                                                            “Humans have more information about the problem”

                                                            And this is why a conservative borrower checker is ultimately the best. It can be super optimal, and not step on your toes. It’s up to the human to adjust the lifetime of memory because only the human knows what it wants.

                                                            I AM NOT A ROBOT BEEP BOOP

                                                          2. 3

                                                            Humans have a huge advantage over the compiler here though. If they can’t figure out whether a program works or not, they can change it (with the understanding gained by thinking about it) until they are sure it does. The compiler can’t (or shouldn’t) go making large architectural changes to your code. If the compiler tried it’s hardest to be as smart as possible about memory, the result would be that when it says “I give up, the code needs to change” the human who can change the code is going to have a very hard time understanding why and what they need to change (since they haven’t been thinking about the problem).

                                                            Instead, what Rust does is apply as intelligent a set of rules they could that produce consistent understandable results for the human. So the compiler can say “I give up, here’s why”. And the human can say “I know how the compiler will work, it will accept this this time” instead of flailing about trying to convince the compiler it works.

                                                            1. 1

                                                              I realize it’s a pretty big ask

                                                              I’ve been hearing this phrase lately “big ask” from business people generally, seems very odd to me. Is it new or have I just missed it up to now?

                                                              1. 2

                                                                I’ve been hearing it from “business people” for a couple years at least, I assume it’s just diffusing out slowly to the rest of society.

                                                                The new one I’m hearing along these lines is “learnings”. I think people just think it makes them sound smart if they use different words.

                                                                1. 1

                                                                  A “learning”, as a noun, is attested at least as far back as the early 1900s, FYI.

                                                                  1. 0

                                                                    This sort of comment annoys me greatly. Someone used a word incorrectly 100 years ago. That doesn’t mean it’s ‘been a word for 100 years’ or whatever you’re implying. ‘Learning’ is not a noun. You can argue about the merits of prescriptivism all you like, you can have whatever philosophical discussion you like as to whether it’s valid to say that something is ‘incorrect English’, but ‘someone used it in that way X hundred years ago’ does not justify anything.

                                                                    1. 2

                                                                      This sort of comment annoys me greatly. Someone used a word incorrectly 100 years ago. That doesn’t mean it’s ‘been a word for 100 years’ or whatever you’re implying. ‘Learning’ is not a noun.

                                                                      It wasn’t “one person using it incorrectly” that’s not even remotely how attestation works in linguistics. And of course, of course it is very much a noun. What precisely, man, do you think a gerund is? We have learning curves, learning processes, learning centres. We quote Pope to one another when we say that “a little learning is a dangerous thing”.

                                                                      To take the position that gerunds aren’t nouns and cannot be pluralized requires objecting to such fluent Englishisms as “the paintings on the wall”, “partings are such sweet sorrow”, “I’ve had three helpings of soup”

                                                                      1. 0

                                                                        ‘Painting’ is the process of painting. You can’t pluralise it. It’s also a (true) noun, the product of doing some painting. There it obviously can be pluralised. But ‘the paintings we did of the house kept improving the sheen of the walls’ is not valid English. They’re different words.

                                                                        1. 2

                                                                          LMAO man, how do you think Painting became a “true” noun? It’s just a gerund being used as a noun that you’re accustomed to. One painted portraits, landscapes, still lifes, studies, etc. To group all these things together as “paintings” was an instance of the exact same linguistic phenomenon that gives us the idea that one learns learnings.

                                                                          You’re arguing against literally the entire field of linguistics here on the basis of gut feelings and ad hoc nonsense explanations.

                                                                          1. 0

                                                                            You’re arguing against literally the entire field of linguistics here on the basis of gut feelings and ad hoc nonsense explanations.

                                                                            No, I’m not. This has literally nothing to do with linguistics. That linguistics is a descriptivist scientific field has nothing to do with whether ‘learnings’ is a real English word. And it isn’t. For the same reason that ‘should of’ is wrong: people don’t recognise it as a real word. Words are what we say words are. People using language wrong are using it wrong in the eyes of others, which makes it wrong.

                                                                            1. 1

                                                                              That linguistics is a descriptivist scientific field has nothing to do with whether ‘learnings’ is a real English word. And it isn’t. For the same reason that ‘should of’ is wrong: people don’t recognise it as a real word. Words are what we say words are.

                                                                              Well, I hate to break it to you, but plenty of people say learnings is a word, like all of the people you were complaining use it as a word.

                                                                              1. 0

                                                                                There are lots of people that write ‘should of’ when they mean ‘should’ve’. That doesn’t make them rightt.

                                                                                1. 1

                                                                                  Yes and OK is an acronym for Oll Korrect, anyone using it as a phrase is not OK.

                                                                                  1. 0

                                                                                    OK has unknown etymology. And acronyms are in no way comparable to simply incorrect grammar.

                                                                                    1. 1

                                                                                      Actually it is known. Most etymologists agree that it came from Boston in 1839 originating in a satirical piece on grammar. This was responding to people who insist that English must follow some strict unwavering set of laws as though it were a kind of formal language. OK is an acronym, and it stands for Oll Korrect, and it was literally invented to make pedants upset. Certain people were debating the use of acronyms in common speech, and to lay it on extra thick the author purposefully misspelled All Correct. The word was quickly adopted because pedantry is pretty unpopular.

                                                                                      1. 1

                                                                                        What I said is that there is what is accepted as valid and what is not. Nobody educated thinks that ‘should of’ is valid. It’s a misspelling of ‘should’ve’. Nobody thinks ‘shuold’ is a valid spelling of ‘should’ either. Is this really a debate you want to have?

                                                                                        1. 1

                                                                                          I was (mostly) trying to be playful while also trying to encourage you to be a little less litigious about how people shuold and shuold not use words.

                                                                                          Genuinely sorry for making you actually upset though, I was just trying to poke fun a little for getting a bit too serious at someone over smol beans, and I was not trying to make you viscerally angry.

                                                                                          I also resent the attitude that someone’s grammatical or vocabulary knowledge of English represents an “education”.

                                                                2. 1

                                                                  It seems like in the last 3 years all the execs at my company started phrasing everything as “The ask is…” I think they are trying to highlight that you have input (you can answer an ask with no) vs an order.

                                                                  In practice, of course, many “asks” are orders.

                                                                  1. 4

                                                                    Sure, but we already have a word for that, it’s “request”.

                                                                    1. 4

                                                                      Sure, but the Great Nouning of Verbs in English has been an ongoing process for ages and continues apace. “An ask” is just a more recent product of the process that’s given us a poker player’s “tells”, a corporation’s “yearly spend”, and the “disconnect” between two parties’ understandings.

                                                                      All of those nouned verbs have or had perfectly good non-nominalized verb nouns, at one point or another in history.

                                                                      1. 1

                                                                        One that really upsets a friend of mine is using ‘invite’ as a noun.

                                                                  2. 1

                                                                    Newly popular? MW quotes this usage and says Britishism.

                                                                    https://www.merriam-webster.com/dictionary/ask

                                                                    They don’t date the sample, but I found it’s from a 2008 movie review.

                                                                    https://www.spectator.co.uk/2008/10/cold-comfort/

                                                                    So at least that old.

                                                                3. 3

                                                                  You no doubt know this, but the undecidable stuff mostly becomes decidable if you’re willing to accept a finite limit on addressable memory, which anyone compiling for, say, x86 or x86_64 is already willing to do. So imo it’s the intractability rather than undecidability that’s the real problem.

                                                                  1. 1

                                                                    It becomes decidable by giving us an upper bound on the number of steps the program can take, so should require us to calculate the LBA equivalent of a very large BB. I’d call that “effectively” undecidable, which seems like it would be “worse” than intractable.

                                                                    1. 2

                                                                      I agree it’s, let’s say, “very” intractable to make the most general use of a memory bound to verify program properties. But the reason it doesn’t seem like a purely pedantic distinction to me is that once you make a restriction like “64-bit pointers”, you do open up a bunch of techniques for finite solving, some of which are actually usable in practice to prove properties that would be undecidable without the finite-pointer restriction. If you just applied Rice’s theorem and called verifying those properties undecidable, it would skip over the whole class of things that can be decided by a modern SMT solver in the 32-bit/64-bit case. Granted, most still can’t be, but that’s why the boundary that interests me more nowadays is the “SMT can solve this” vs. “SMT can’t solve this” one rather than the CS-theory sense of decidable/undecidable.

                                                                4. 6

                                                                  Why can’t I have computer generated proofs? I have some memory, I send it there, then there, then I’m done.

                                                                  It’s really hard. The main tool for that is separation logic. Manually doing it is harder than borrow-checking stuff. There are people developing solvers to automate such analyses. Example. It’s possible what you want will come out of that. I think there will still be restrictions on coding style to ease analyses.

                                                                  1. 3

                                                                    In my experience, automated proof generators are very leaky abstractions. You have to know their search methods in detail, and present your hypotheses in a favorable way for those methods. It can look very clean, but it can mean that seemingly easy changes turn out to be frustrated by the methods’ limitations.

                                                                    1. 4

                                                                      I’m totally with you on this. Rust very much feels like an intermediate step and I don’t know why they didn’t take it to it’s not-necessarily-obvious conclusion.

                                                                      1. 5

                                                                        In my personal opinion, it might be just that we’re happy that we can actually get to this intermediate point (of Rust) reliably enough, but have no idea yet how to get to the further point (conclusion). So they took it where they could, and left the subsequent part as an excercise for the reader… I mean, to be explored by future generations of programmers, hopefully.

                                                                        1. 4

                                                                          We have the technology, sort of. Total program analysis is really expensive though, and the workflow is still “edit some code” -> “compile on a laptop” -> repeat. Maybe if we built a gc’ed language that had a mode where you push your program to a long running job on a compute cluster to figure out all the memory proofs.

                                                                          This would be especially cool if incrementals could be cached.

                                                                          1. 4

                                                                            I’ve recommended that before. There’s millions being invested into SMT/SAT solvers for common bugs that might make that happen, too. Gotta wait for the tooling to catch up. My interim recommendation was a low-false-positive, static-analysis tool like RV-Match to be used on everything in the fast path. Anything that passes is done no GC. Anything that hangs or fails is GC’d. Same with automated proofs to eliminate safety checks. If it passes, remove that check if that’s what pass allows. If it fails, maybe it’s safe or maybe tool is too dumb. Keep the check. Might not even need cluster given number of cores in workstations/servers and efficiency improvements in tools.

                                                                          2. 4

                                                                            I think it’s because there’s essentially no chance that a random piece of code will be provable in such a way. Rust encourages, actually to the point of forcing, the programmer to reason about lifetimes and ownership along with other aspects of the type as they’re constructing the program.

                                                                            I think there may be a long term evolution as tools get better: the languages checks the proofs (which, in my dream, can be both types and more advanced proofs, say that unsafe blocks actually respect safety), and IDE’s provide lots of help in producing them.

                                                                            1. 2

                                                                              there’s essentially no chance that a random piece of code will be provable in such a way

                                                                              There must be some chance; rust is already proving memory safety.

                                                                              Rust forces us to think about lifetimes and ownership, but to @tedu’s point, there doesn’t seem be much stopping it from inferring those lifetimes & ownership based upon usage. The compiler knows everywhere a variable is used, why can’t it determine for us how to borrow it and who owns it?

                                                                              1. 17

                                                                                Rust forces us to think about lifetimes and ownership, but to @tedu’s point, there doesn’t seem be much stopping it from inferring those lifetimes & ownership based upon usage. The compiler knows everywhere a variable is used, why can’t it determine for us how to borrow it and who owns it?

                                                                                This is a misconception. The Rust compiler does not see anything beyond the function boundary. That makes lifetime checking efficient. Basically, when compiling a function, the compiler makes an reasonable assumption about how input and output references are connected (the assumption is “they are connected”, also known as “lifetime elision”). This is an assumption communicated the outside world. If this assumption is wrong, you need to annotate lifetimes.

                                                                                When compiling, the compiler will check if the assumption holds for the function body. So, for every function call, it will check if the the signature holds (lifetimes are part of the function signature).

                                                                                Note that functions with different lifetime annotations taking the same data might differ in their behaviour. It also isn’t always obvious to the compiler whether you want references to be bound together or not and that situation might be ambigous.

                                                                                The benefit of this model is that functions only need to be rechecked/compiled when they actually change, not some other code somewhere else in the program. It’s very predictable and errors are local to the function.

                                                                                1. 2

                                                                                  I’ve been waiting for you @skade.

                                                                                  1. 2

                                                                                    Note that functions with different lifetime annotations taking the same data might differ in their behaviour.

                                                                                    I wrote this late at night and have some errata here: they might differ in their behaviour wrt. lifetime checking. Lifetimes have no impact on the runtime, an annotation might only prove something safe that the compiler previously didn’t see as safe.

                                                                                  2. 4

                                                                                    Maybe I’m misunderstanding. I’m interpreting “take it to its conclusion” as accepting programs that are not annotated with explicit lifetime information but for which such an annotation can be added. (In the context of Rust, I would consider “annotation” to include choosing between &, &mut, and by-move, as well as adding .clone() when needed, especially for refcount types, and of course adding explicit lifetimes in cases that go beyond the present lifetime elision rules, which are actually pretty good). My point is that such a “smarter compiler” would fail a lot of the time, and that failures would be mysterious. There’s a lot of experience around this for analyses where the consequence of failure is performance loss due to not being able to do an optimization, or false positives in static analysis tools.

                                                                                    The main point I’m making here is that, by requiring the programmer to actually provide the types, there’s more work, but the failures are a lot less mysterious. Overall I think that’s a good tradeoff, especially with the present state of analysis tools.

                                                                                    1. 1

                                                                                      I’m interpreting “take it to its conclusion” as accepting programs that are not annotated with explicit lifetime information but for which such an annotation can be added.

                                                                                      I’ll agree with that definition

                                                                                      My point is that such a “smarter compiler” would fail a lot of the time, and that failures would be mysterious.

                                                                                      This is where I feel we disagree. I feel like you’re assuming that if we make lifetimes optional that we would for some reason also lose the type system. That was not my assumption at all. I assumed the programmer would still pick their own types. With that in mind, If this theoretical compiler could prove memory safety using the developer provided types and the inferred ownership, why would it still fail a lot?

                                                                                      where the consequence of failure is performance loss due to not being able to do an optimization

                                                                                      That’s totally understandable. I assume like any compiler, it would eventually get better at this. I also assume lifetimes become an optional piece of the program as well. Assuming this compiler existed it seems reasonable to me that it could accept and prove lifetimes provided by the developer along with inferring and proving on it own.

                                                                                      1. 3

                                                                                        Assuming this compiler existed it seems reasonable to me that it could accept and prove lifetimes provided by the developer along with inferring and proving on it own.

                                                                                        That’s what Rust does. And many improvements to Rust focus on increasing the number of lifetime patterns the compiler can recognize and handle automatically.

                                                                                        You don’t have to annotate everything for the compiler. You write code in patterns the compiler understands, and annotate things it doesn’t. So Rust has gotten easier and easier to write as the compiler gets smarter and smarter. It requires fewer and fewer annotations / unsafe blocks / etc as the compiler authors discover how to prove and compile more things safely.

                                                                                    2. 4

                                                                                      Rust forces us to think about lifetimes and ownership, but to @tedu’s point, there doesn’t seem be much stopping it from inferring those lifetimes & ownership based upon usage. The compiler knows everywhere a variable is used, why can’t it determine for us how to borrow it and who owns it?

                                                                                      I wondered this at first, but inferring the lifetimes (among other issues) has some funky consequences w.r.t. encapsulation. Typically we expect a call to a function to continue to compile as long as the function signature remains unchanged, but if we infer the lifetimes instead of making them an explicit part of the signature, subtle changes to a function’s implementation can lead to new lifetime restrictions being inferred, which will compile fine for you but invisibly break all of your downstream callers.

                                                                                      When the lifetimes are an explicit part of the function signature, the compiler stops you from compiling until you either fix your implementation to conform to your public lifetime contract, or change your declared lifetimes (and, presumably, since you’ve been made conscious of the breakage in this scenario, notify your downstream and bump your semver).

                                                                                      It’s basically the same reason that you don’t want to infer the types of function arguments from how they’re used inside a function – making it easy for you to invisibly breaking your contract with the outside world is bad.

                                                                                      1. 3

                                                                                        I think this is the most important point here. Types are contracts, and contracts can specify far more than just int vs string. Complexity, linearity, parametricity, side-effects, etc. are all a part of the contract and the more of it we can get the compiler to enforce the better.

                                                                                2. 1

                                                                                  Which is fine, until you have time or memory constraints that are not easily met by the tracing GC, which is all software of sufficient scale or complexity. At that point, you end up with half-assed and painful to debug/optimize manual memory management in the form of pools, ect.

                                                                                  1. 1

                                                                                    Or there’s the rust approach, where I write a little proof that I’m done with the memory, and then the computer verifies my proof, or rejects my program. Proof verification is also something computers are good at. Nice.

                                                                                    Oh I wish that were how Rust worked. But it isn’t. A variant of Rust where you could actually prove things about your programme would be wonderful. Unfortunately, in Rust, you instead just have ‘unsafe’, which means ‘trust me’.

                                                                                  1. 2

                                                                                    Pricing still TBD, article says:

                                                                                    ODROID-H2 price will be officially announced next month when it starts selling.

                                                                                    It must be higher than US$100 though.

                                                                                    Is that good value for just this board w/ CPU?

                                                                                    1. 4

                                                                                      Eh. This ASRock board with the same CPU is $86 on newegg.

                                                                                      The only other board I can find is this one from GIGABYTE b2b, which seems to be more industrial focused, for $169 at sg-computers.com.

                                                                                      Major differences:

                                                                                      • ODROID is by far the smallest
                                                                                      • ODROID and GIGABYTE have 2 ethernet ports, ASRock only has 1
                                                                                      • GIGABYTE has tons of extra stuff, e.g. multiple internal and external RS-232 serial ports
                                                                                    1. 19

                                                                                      “The only thing that matters is the future,” he told me after the civil trial was settled. “I don’t even know why we study history. It’s entertaining, I guess—the dinosaurs and the Neanderthals and the Industrial Revolution, and stuff like that. But what already happened doesn’t really matter. You don’t need to know that history to build on what they made. In technology, all that matters is tomorrow.”

                                                                                      What an idiot.

                                                                                      1. 6

                                                                                        I feel like you’ve just pushed CSS to the HTML, which if you’re going to do that you might as well just use CSS in HTML with react or something. Sometimes I wonder if running away from learning and using CSS proper only sets us up for rediscovering it later on.

                                                                                        1. 1

                                                                                          The author addresses this point quite plainly in the post so I’m not sure why you would feel that way.

                                                                                          1. 1

                                                                                            The author addresses inline styles, not any other component based solutions.

                                                                                        1. 4

                                                                                          The GeoIP tip was smart, I hadn’t thought of that before.

                                                                                          1. 5

                                                                                            This issue also led to tools like Speedtest connecting to the wrong test servers many states away and severely underestimating performance.

                                                                                            Semi-related note, I have been using https://fast.com lately and I’m much happier with the information it provides and how quickly. The settings can be changed after your first test is completed.

                                                                                            1. 1

                                                                                              Yeah! Used it recently to find out that I had 1 Gbit/s in my apartment. Been using it ever since. The concept is just great; enter fast.com and watch the speed.

                                                                                              1. 3

                                                                                                Fun fact: Netflix also bought slow.com for the same service :).

                                                                                                1. 1

                                                                                                  I did not, and will from now recommend that to the next person who needs a speed test. :-)

                                                                                          1. 6

                                                                                            I do largely trust applications in the App Store… but not these sketchy “cleaner” apps. I’ve always been amazed they’re even on the store.

                                                                                            1. 16

                                                                                              Some of these are really unsurprising. I mean, it’s a “WAT” that Go’s variables are block scoped? Really?

                                                                                              I think this kind of “look out for these slightly unexpected behaviors” talk is a good thing, but trying to liken them to Gary Bernhardt’s WAT talk seems a bit disingenuous.

                                                                                              1. 4

                                                                                                So I don’t know whether that is intense or not, but most of the content is covered very clearly in the Go spec, which is not exactly big, as far as specs go…. so yeah. This.

                                                                                                1. 2

                                                                                                  I mean, it’s a “WAT” that Go’s variables are block scoped? Really?

                                                                                                  It’s not just block scope, it’s block scope plus shadowing, plus allowing you to declare and assign a visible variable using the value from the variable that you’re shadowing. You could have block scope without allowing either of the latter two choices.

                                                                                                  Java doesn’t let you do this, even though it has block scope. The second declaration of a fails. I was mildly surprised that C lets you do the same thing, though given the origins of go, I shouldn’t have been.

                                                                                                  1. 1

                                                                                                    I think the link is more an appeal to popular culture.

                                                                                                    Either way, WAT8 was pretty surprising for me. I would never expect that to happen, that seems more like a quirk of code generation than an actual intentional feature.

                                                                                                  1. 7

                                                                                                    What a hill to die on.

                                                                                                    1. 2

                                                                                                      Who is dying? Certainly not Redis.

                                                                                                      1. 3

                                                                                                        It’s a metaphor.

                                                                                                    1. 2

                                                                                                      At this point, I’ve hopefully convinced you that we designed a thoughtful, careful experiment.

                                                                                                      ❤️

                                                                                                      1. 3

                                                                                                        It would be interesting to read about how the site is deployed and some performance stats. Is any of that published anywhere?

                                                                                                        1. 4

                                                                                                          The site is deployed via ansible playbook. Can you describe what you mean by performance stats? In June 2018 pushcx published activity stats and in August 2018 I gave a presentation where I further articulated that activity. Is that what you are looking for or did you have something else in mind?

                                                                                                          1. 3

                                                                                                            I would be interested in seeing you/mem stats from before and after the 5.2 migration. Pure curiosity!

                                                                                                            1. 2

                                                                                                              Alas, I don’t have any quantitative data to share here. I’ve been able to solve memory pressure by sampling and making plausible guesses from there. I can say that the memory the Ruby work queue uses is load dependent: over the day the memory size of that process set grows and shrinks based on how busy the site is.

                                                                                                              That doesn’t really answer your question though. If we were going to collect memory timeseries data, would you recommend any particular tool? collectd?

                                                                                                              1. 2

                                                                                                                I would recommend Prometheus. It’s easy to get running, suitable for collecting data and graphing, as well as full monitoring / alerting.

                                                                                                                1. 4

                                                                                                                  We already discussed about monitoring on the ansible repository. Lobsters is generously hosted by @alynpost’s company and adding Prometheus to it would add a good way to visualize metrics, but would also take cpu/memory that would have been available for Rails and MariaDB.

                                                                                                                  For the monitoring, it was then said by Alan that the Nagios setup used at Prgmr could be used, so adding monitoring without the downsides!

                                                                                                                  1. 1

                                                                                                                    Building on @jstoja’s reply: we definitely have external monitoring for lobste.rs (the username and password is ‘guest’). That Nagios instance is hooked up to prgmr.com’s pager rotation so we get paged if the site goes down.

                                                                                                          1. 2

                                                                                                            Is there a list of SQL operations that are optimal in SQL and that are not?

                                                                                                            What are the operations that SQL truly excels? What are the operations better handled with commands for text streams/files?

                                                                                                            1. 4

                                                                                                              No. There are no SQL operations that are “not optimal” in SQL. A query is a description of a result, not the execution strategy. Any execution strategy can be chosen by a SQL engine, and obviously optimizers aim to choose the optimal one.

                                                                                                              There is only one situation where using text streams is objectively better: the operation you want to do isn’t implemented efficiently by your SQL engine, which is a problem with your engine, not SQL itself. There are even SQL engines that operate on text streams directly, e.g. Hive, or q.

                                                                                                              The author has a different problem, they don’t know enough about SQL / their SQL engine to make this query work. Which the author actually states in the post.

                                                                                                              As experimenting with each suggestion could easily take at least half a day, I proceeded with a way I knew would work efficiently and reliably.

                                                                                                              This query is an “optimal” case where SQL typically excels: most SQL engines are perfectly capable of using this exact execution plan. Given that personal constraint, they made a reasonable choice to run the query outside of their database using an approach they understand.