1. 10

    Putting the finishing touches on a version of www.literature-map.com which will be used to organize scientific articles about literature.

    1. 3

      This is pretty neat. Although it may not be the intended use, I would find it useful for looking for finding new fiction authors.

      1. 4

        That is one of the main use cases of the Literature-Map. You can give it a a try by putting your favorite fiction author into the search and see what comes up.

        1. 1

          That’s really awesome, nice work!

      2. 1

        How are the author names embedded in 2-D space?

        1. 1

          How uncanny! I put in the name of an author I was reading, and up came the names of 3 authors who I had just read before, and 2 authors that I was planning on reading next. I guess I’m predictable :-)

          This is neat, thanks for sharing!

        1. 4

          Lots of gems in here; this quote stood out to me.

          Some pieces of Ninja took struggle to get to and then are obvious in retrospect. I think this is true of much of math, that once you have distilled the ideas to their essence they seem obvious. The power comes from having the right way of thinking about the problem.

          1. 1

            The author notes that this is an unofficial reference. Might I suggest calling it “a reference”, instead of “the reference”?

            There are already existing free references for Alloy. For example, here is an Alloy cheatsheet from the University of Iowa course CS:5810 Formal Methods in Software Engineering.

            There’s a parallel situation in Rust-land: Tokio markets itself as “the asynchronous runtime”. This is confusing, because async-std exists. Anyone can make a runtime. Using the definite article in this way may harm the ecosystem, because it implies that the work is uniquely authoritative, when it arguably is not.

            (The Alloy book “Software Abstractions” is delightful, and I recommend it to anyone with the money to spare. Maybe one day, it will be made available as a free download?)

            1. 13

              So… the situation right now is weird. It’s officially a first draft of the official reference, and unofficially a reference in its own right. I’m part of the Alloy board and responsible for writing the online documentation. Once I’ve polished the docs a bit more the plan is to host it on the alloytools website. It’s not the official reference, but it eventually will be.

              For example, here is an Alloy cheatsheet from the University of Iowa course CS:5810 Formal Methods in Software Engineering.

              I completely forgot to include that cheatsheet in the docs, thanks for reminding me! Though we’re gonna need to update it for Alloy 5…

              (The Alloy book “Software Abstractions” is delightful, and I recommend it to anyone with the money to spare. Maybe one day, it will be made available as a free download?)

              Fun fact: I read the entire book in two sittings and then immediately emailed Daniel Jackson raving about how good it is. One thing led to another and now I’m writing docs for him :P

              Re making it free, my understanding is that the publisher won’t let us. That’s one reason we decided to write an entirely new set of docs.

              1. 2

                I’m writing docs for him :P

                I guess you mean “for everyone” 🙃

                1. 1

                  Thanks for the clarification!

              1. 6

                I’d like to suggest an algorithmic approach to mitigating this problem. I know algorithms are not the whole answer, because technological solutions never are, but this might be an interesting place to start.

                It seems to me that we can use the structure of the user invitation tree to weight upvotes. Perhaps it is the case that the farther apart users are on the invitation tree, the less similar they are. The closer they are, the more similar they are.

                One way to think about the “meaning” of a set of upvotes is that they represent a sample of the opinions of the Lobste.rs community.

                If upvotes come from diverse places in the social graph, then maybe the story appeals to a broad collection of users. If upvotes come from a narrow cluster in the social graph, then maybe the story came from an upvoting ring.

                In the following Gist, I show an idealized invitation tree with example upvotes superimposed in various colors. After inducing a subgraph for each story’s votes, that connects its upvoters in the user invitation tree, I calculate a few distance functions. Intuitively, it seems that the following distance functions correlate to the “diverse vote” signal: the minimum spanning tree edge count, the Weiner index, and especially the mean eccentricity.

                I’d be happy to discuss this further offline, or in another thread.

                Here is a link to prototype network analysis code, distance function output, and a tree visualization: https://gist.github.com/rw/3dd53d50d2f324adac2d078beeb23410

                1. 4

                  My personal invite tree is basically “random” people I’ve invited because they have asked in the chat.

                  1. 1

                    Hmm, I don’t think that’s a problem in my model. If it is, we can come up with other measures of “distance” between users, such as by modeling their topics of interest.

                1. 4

                  The speed the author is able to achieve is a direct result of the efficiency of the underlying random number generator algorithms. But, the author only references the underlying algorithms once, by saying this:

                  I won’t bother you again other than stating that Neanderthal uses Philox and/or ARS5 RNG which is much, much, better than the stuff you get from the built-in rand.

                  From my perspective, the theory and implementation of the particular PRNG algorithm deserves most of the credit here.

                  The algorithms the author relies upon have hardware implementations that can generate gigabytes/sec of random numbers.

                  Here are two quick links:

                  1. Overview of CBRNG on Wikipedia, which includes ARS5 RNG and Philox: https://en.wikipedia.org/wiki/Counter-based_random_number_generator_(CBRNG)

                  2. Benchmarks of the CBRNG in MKL: https://software.intel.com/en-us/articles/new-counter-based-random-number-generators-in-intel-math-kernel-library

                  Furthermore, if the author used a cryptographically-secure random number generator, wouldn’t this whole thing be much slower?

                  1. 1

                    I can’t find this in the actix/actix-web docs: why did they move away from using actix itself? If I want to create an actor-based application that doesn’t necessarily do I/O, should I still consider actix?

                    1. 3

                      Here’s the content on github, if you prefer not to use JS: https://github.com/upsuper/rust-cheatsheet/blob/master/main.js

                      1. 1

                        I want them to keep the HOMEBREW_BUILD_FROM_SOURCE environment variable. I think it’s tasteful, and I love having local source code available from which I can learn and with which I can tinker. Also, building from source can help mitigate security breaches like the one that occurred with Homebrew Bottle generation in August 2018: https://www.theregister.co.uk/2018/08/08/researcher_found_homebrew_github_token_hidden_in_plain_sight/

                        1. 2

                          Pretty eager to try this as the storage format for a number of things in sled :]

                          1. 3

                            As someone who looked into flat buffers a few months ago and tried to determine its key advantages/disadvantages when compared to capnproto, could you perhaps share any insight that you might have? I found it difficult to tease out the differences. One way to make this more concrete, for example, is that I think Rust already has support for capnproto. What has prevented you (either technically or not) from using capnproto in sled?

                            1. 4

                              Here’s my understanding, please correct me if I’m off about something!

                              At the core, capnproto does not use a vtable, but flatbuffers does. capnproto messages include all values, even if they are set to the default value (if the default is all zeroes on the wire, it can be packed down using capnproto’s simple compression scheme). Default values in flatbuffers are not included in individual messages. The lack of the vtable in capnproto makes schema evolution a bit clunky over time if you’re used to protobufs. You can add new numbered fields and change their human-readable names at will. No vtable = no potentially malicious pointer validation.

                              Flatbuffers allows you to validate pointer integrity on the vtable, but this does not happen by default. Don’t accept potentially malicious flatbuffers without validating them. They assume friendly input.

                              Flatbuffers is probably the better choice if you expect to have lots of fields, possibly many having the default value.

                              Flatbuffers is a little more flexible, feels more comfortable coming from protobufs. Capnproto is simpler and you don’t have to be as mindful of security issues. Flatbuffers may have far better spatial performance if you have a lot of nonzero default values.

                              Flatbuffers lets you use structs for simple types if you want to avoid vtable hops, but you lose a lot of flexibility. Flatbuffers (in other languages, not finished yet in rust AFAIK) lets you mutate fields, as they are backed by a Vec which can grow. Capnproto is immutable.

                              sled: why flatbuffers and not capnproto? The system is changing very quickly still. I want forward compatible storage files. I am comforted by the flexibility of flatbuffers. To be honest there is not a specific case I can imagine needing flatbuffer’s flexibility for over capnproto’s ability to add numbered fields. But that’s the problem with forward compatibility, it’s hard to predict. I don’t anticipate the vtables in flatbuffers to be much of an issue, and where they are, I can migrate to flatbuffer structs. I mitigate the malicious vtable issue because I treat all data from the disk as malicious and checksum all segments and individual messages. But for me it boils down to flexibility and “good enough” performance.

                              1. 2

                                Thank you so much for that wonderfully thoughtful reply! I haven’t used capnproto or flat buffers in anger yet, but I might in the future, and want to be prepared to make the right decision. Thanks!

                                1. 2

                                  Flatbuffers (in other languages, not finished yet in rust AFAIK) lets you mutate fields, as they are backed by a Vec which can grow.

                                  Quick clarification: in-place mutation is supported in the cases where the payload already has space for the value in question. So, if you want to change the value of a stored u32, that will be supported. But, if the field’s value data is missing (indicating a default or optional value), then you won’t be able to modify the field in-place.

                                2. 1

                                  Ill add Cap n Proto is more secure than many other options.

                              1. 2

                                The Q&A get’s abruptly cut at the end. :(

                                1. 3

                                  It’s on the Mozilla livestream: https://mzl.la/Rust-Meet-Up-2018-11-13

                                  That’s a very clunky video link, though, requiring a livestream and an exception for it to create a popup.

                                  1. 1

                                    I think that was the last question :-)

                                  1. 2

                                    Sarcasm?

                                    1. 2

                                      who’ll ever know.

                                      1. 2

                                        cool kids switching from Ruby to a real programming language

                                        Kinda makes me think that. And the rest seems to be complaining that a powerful type system which allows for compile-time memory safety needs annotations.

                                      1. 1

                                        What does “& sketches” mean in the tagline? What do sketches have to do with this library?

                                        1. 2

                                          libtwiddle implements HyperLogLog, which is a sketch data structure.

                                          1. 1

                                            The author also plans to implement count-min sketches: https://github.com/fsaintjacques/libtwiddle/tree/feature/count-min-sketch

                                          1. 4

                                            As a sometimes capnproto user, are there any technical benefits to using flatbuffers vs capnproto?

                                            1. 1

                                              I think you’re closer to optimal than typical JSON or Protocol Buffers users. That said, FlatBuffers is still worth looking into. Maybe you can sift through this announcement thread with comments about FlatBuffers vs. Cap'n'Proto, and let us know what you think?

                                              https://news.ycombinator.com/item?id=7901991

                                              1. 2

                                                Thank you, great point. (Edit: I updated the question.)

                                                I guess with Apple dropping the ball with Yosemite, I am keeping an eye out for alternatives. I’m hoping someone will pick up that ball and provide the world with an easy-to-use, “intelligently designed”, innovative, secure operating system that is performant and doesn’t kill your laptop’s battery life.

                                                Who is truly innovating in desktop Operating Systems these days, in other words?

                                                I keep seeing good things about OpenBSD pop up every now and then, so I was curious. Sorry for the poorly phrased original question.

                                                1. 4

                                                  I guess with Apple dropping the ball with Yosemite, I am keeping an eye out for alternatives.

                                                  I keep hearing about apple “dropping the ball”. Yes, I too have had some problems/bugs with yosemite. I recall a few buggy early releases of previous version of OSX too though, and to my vague recollection some of those even had fewer new features.

                                                  I am not discounting doom and gloom. However, I personally find the OSX experience still rather far ahead of other current options on the laptop.