1. 23

    There are also two environment variables we need to know

    Uhhh… Kinda not really. You don’t need to know about GOROOT or GOPATH or virtualgo.

    Something like this will work.

    $ cd $HOME
    $ mkdir hashtrack && cd $_
    $ go mod init github.com/cuchi/hashtrack
    $ go get github.com/Laisky/graphql
    

    All your code and deps get scoped to the module.

    Go doesn’t have a package manager

    Oookaay, this is just wrong. It might help to read up on Go Modules. Go’s official package manager is built into the go tool. https://blog.golang.org/using-go-modules

    or official registry

    Kinda. There is an official central place to search for modules: https://pkg.go.dev/

    1. 3

      Go doesn’t have a package manager

      Oookaay, this is just wrong. It might help to read up on Go Modules.

      When did this happen? It is news to me (but then again I only try Go every now and then :-)

      1. 2

        Go 1.11. Since it wasn’t proposed until 2018, a lot of people who tried the language years ago aren’t aware of its existence.

        1. 5

          I think the author is mostly just confused about the GOPATH/Module situation. Modules are not really a “package manger” in the same sense that npm or whatnot are (it’s better!) and there’s a lot of “legacy” advice and documentation across the internet. Never mind that the tooling is rather weird as well, where things behaving in different modes depending on how/where you invoke them.

          It’s all pretty confusing especially for newcomers; I’ve seen quite a few people get confused by it. I started writing a comment about this last night, but ended up just sending it as an email to the article’s author.

    1. 4

      On holiday with my family :-)

      1. 10

        This has been discussed before, but I was wondering if anything has changed.

        Several difficulties and problems were raised … have they been addressed?

        Does anyone use it?

        1. 11

          I’ve been there for close to 2 years, and have tried to build my own SSB server from scratch (in a non-JS language). Feel free to ask any questions. For starters:

          • The low level protocol (transport encryption, RPC and discovery) is very well documented.

          • The application level protocol has almost no documentation, and what’s there is outdated. You really have to resort to reverse engineer behaviour from existing applications, or reading other’s code.

          • Replication/gossip mechanism is very inefficient, which leads to clients (especially mobile ones) spending a lot of time during the initial sync. There’s a newer gossip protocol which fixes some of these problems, but has zero documentation, and there’s only one implementation (in JS). There are no plans to port it to other languages since there’s a lot of tricky edge cases in there.

          • Yes, the JSON encoding was a mistake. There’s a new format using CBOR, but it’s still a few ways off in terms of mainstream usage in the network.

          • There are questionable decisions at the application level. For example, anyone can assign you new profile pictures or visible names, which can–and has–lead to bullying/name-calling.

          In terms of community, it’s mostly tech-centric, most discussions are either about SSB itself, or related protocols. The overall spirit is positive, focusing on sustainable living, gardening, off-grid, etc.

          However, the community is very small. This can get tiring, considering that most clients will show any replies to threads you follow at the top of your timeline (you will see the same 3 to 5 people all the time).

          1. 4

            I’ve also built a partial SSB implementation, in C++. I found a C port of the CLI tool (in an obscure Git repo hosted in SSB), which helped immeasurably with the secure handshake and packet codec. I used Couchbase Lite [of which I’m the architect] as the database. This proved a lot faster than the JS data store, but I still found that pulling all the subscribed content from one pub (on which I’m following a handful of accounts) resulted in a 600MB database. It would have helped if the protocol had an option to pull only messages back to a certain date.

            I’d love to know more about the new protocol, but not if the only way is to decipher a JS codebase.

            It’s a shame they’re so JS-centric. That’s a serious problem for iOS, which has security restrictions that disallow JITs outside of a web browser. Not to mention embedded systems. (And on a personal level I dislike doing serious programming in JS; it feels like sculpting in Jell-O.)

            1. 3

              There are two C implementations of the low level protocol: one for the secret handshake and one for the boxstream (transport encryption). There’s also some integration tests that you can run against your implementation to validate that everything works.

              As for the new replication protocol: the new CBOR format includes “off-chain” contents, which means that the actual log only contains a hash of the post content. This should make initial sync much faster, since clients only fetch the chain of hashes, without downloading anything else.

              Messages can also be downloaded out of order, so you only download what you want, if you have the hash for it. As most things, though, the only implementation is in JS.

              As for the database, I planned to use SQLite, but never got far enough to test that. I’m unconvinced that the log is a good abstraction for the kind of apps that SSB is used right now (social media). There are future plans to plug more applications on top of the log replication, but that’s in the long term, while the current use-cases are suffering from it.

              Edit: wanted to say, though, that for me the biggest block when developing an SSB implementation is the lack of documentation w.r.t. the application-level protocol, and forcing you to develop everything on top of the log abstraction. The JSON formatting can be painful, but solved via forking some json library and doing some changes (hacky, but it works).

            2. 1

              Yes, the JSON encoding was a mistake.

              Was JSON not fast enough?

              1. 4

                It’s not about JSON per-se, but more about how message signing works. In SSB, every post is represented as a JSON blob signed with your publick key, and it expects other clients to validate this, as well as produce valid JSON messages.

                The spec goes over the requirements of a valid message, as well as the steps to compute a valid signature. Unfortunately, it assumes things like key order (which the official JSON spec doesn’t say anything about), indentiation, spacing, etc. (This all goes back to how the V8 engine implements JSON.stringify()). This adds a lot of complexity when implementing SSB in another language, as most JSON libraries won’t care about specific formatting when printing, and specifically the key order requirement makes it quite complicated.

                All in all, it’s not the end of the world, but it adds enough friction to make SSB pretty dependend on the “blessed” javascript implementation.

            3. 5

              I use it regularly. Why..? After being a heavy social media user on the usual platforms, I’ve pretty much removed myself and don’t participate, but Scuttlebutt is the exception because it’s a fun place to be. Nothing more, nothing less.

              1. 3

                What difficulties and problems were raised?

                1. 12

                  There’s two big obstacles I’ve seen that seem specific to Scuttlebutt:

                  • There’s no easy way to share identities across devices. If you want to use Scuttlebutt on your computer and your phone, they’ll be separate accounts.

                  • The protocol is built around an append-only log, which I’m not convinced is a good principle for any social network. Inadvertent mistakes are forever (eg. I post an unboxing photo that has my unredacted invoice visible; I paste an embarrassing link by accident).

                    It also seems like you could grief pub servers (the Scuttlebutt “hub nodes” that federate content more widely). What happens if someone posts a bunch of illegal content to the pub? As I understand it, all the pub users will pull that content down. You might be able to blacklist certain posts in your client, and you can block users, but their content is still on your device. (Bitcoin has faced similar problems.)

                  1. 2

                    Your objection to log storage is valid, but there are ways around it. The data format could have ways to redact posts out of the log while leaving the overall integrity intact; in fact all revisions of a post other than the latest one could be redacted.

                    Of course the redactions need to be propagated, and there’s no guarantee every copy will be redacted, but that’s an intrinsic problem with most P2P protocols, since distributed caching/replication is so important for availability.

                    1. 1

                      Good points.

                      Also ironically where Facebook could have a chance to differentiate themselves, but chose to go in almost the exact different direction:

                      • “with federated networking you are trusting each and every host that your host ever federated with to delete a post, with us, once you click delete it is gone. Worldwide. Same with sharing: if you share something with your close friends it stays there. With a federated network it depends on every host around the globe sticking implementing the rules correctly and sticking to the rules.”

                      • fortunately IMO Facebook messed up massively early on and now everyone in tech now they are fundamentally untrustworthy.

                    2. 8

                      The main problem I saw when I looked into it was that it was a single program rather than a well-defined protocol that anyone in the ecosystem could implement.

                      This might have changed by now, but (for instance) there were aspects baked into the protocol that fundamentally prevented you from building a compatible client unless you used a JSON serializer with the exact same behavior as node.js, because the cryptographic identity of a post was based on a checksum of the output of that particular serializer rather than some inherent property of the post. An easy mistake to make, but one with far-reaching consequences.

                      1. 6

                        That’s my issue as well. It relies heavily on the interaction of a few dozen NodeJS repos. Different frontends all rely on the same backend code, making it not-that-diverse.

                        Also, while the protocol itself is well documented and designed, there are some obvious shortcomings. The protocol relies on hashes of pretty-printed JSON. The last time I checked for documentation on HOW to pretty-print that json, it was documented as “like v8 does it”. Tell you - it’s REALLY hard to format JSON like V8 JSON.format(x, true) does. Especially floating point numbers.

                        Now this could easily be fixed by changing the protocol from hash-of-pretty-printed-json-subobject to hash-of-blob. ({"data": "{\"foo\" 42}", "hash": ...} vs. {"data": {"foo": 42}, hash: ...} But you can’t do that without breaking compatibility. Worse, relying on hash-chains, you need to implement the old behaviour to be able to verify old hash chains.

                        1. 5

                          That’s my top issue too. Which is sad, because there’s so much prior art on canonicalizing JSON, going back to 2010 or so.

                          The spec is fine, but limited. It doesn’t cover all of the interactions between peers; there are plenty of messages, and properties in the schema, that aren’t documented.

                          A lot of the discussion and information about the protocol takes place on Scuttlebutt itself, meaning it’s (AFAICT) invisible to search engines, and accessible over HTTP only through some flaky gateways that often time out.

                          The main client is (like everything else) written in JS, so it’s a big Electron app, and in my experience very slow and resource hungry. I only joined two pubs and followed a handful of people and topics, but every time I fire up the app, it starts a frenzy of downloading and database indexing that lasts a long time. (They use a custom log-based DB engine written in JS.)

                          Part of the slowness must be because when you join a pub you’re implicitly following every user of that pub, and I believe all of the IDs they’re following. So there’s kind of an explosion of data that it pulls in indiscriminately to replicate the social graph and content.

                        2. 4

                          Reading the previous discussions shows a degree of frustration and scepticism. I’m sure it’s changed, and many of the questions will have been addressed, but the previous discussions are unconvincing.

                          Here are some links to some of them … it’s worth reading them in context:

                          https://lobste.rs/s/rgce6h/manyverse_mobile_scuttlebutt_client

                          https://lobste.rs/s/hncoad/secure_scuttlebutt

                          https://lobste.rs/s/l9vqm4/scuttlebutt_protocol_guide

                          https://lobste.rs/s/xe2z2r/scuttlebutt_off_grid_social_network

                        3. 2

                          I’m one of the core developers. Happy to answer any questions.

                          1. 1

                            Despite the fact it’s in a ‘junkyard’, I believe this issue remains unresolved, which I believe effectively means that:

                            1. scuttlebutt is not cross platform and only works on x86
                            2. It’s difficult to implement scuttlebutt libraries and clients in other languages

                            Limiting development to people who like nodejs, and limiting usage to x86 devices (when it seems like the sort of concept that should work well with mobile devices) massively reduces its appeal.

                            I would be happy to find that I’m wrong.

                            1. 3

                              You’re off on this one. I’m running SSB on ARM64 just fine, also many pubs are actually just raspberry pis on some closet.

                              I is still difficult to implement SSB in other languages mostly because of the amount of work than technical challenges. The mistakes of the past are well understood at this point even if not fixed. At the moment there are 2 Rust based implementations and one based in Go. IIRC there is also implementations in Elixir and Haskell but I am not sure how far they are. I’ve toyed with a client mixing C and Lua (just a toy prototype but it worked).

                              1. 2

                                It definitely didn’t work on arm last time I tried, so it’s good to hear they’re making some progress. It was the prototype Haskell implementation which pointed to that issue as a blocker: it looks like it hasn’t been updated since, so probably doesn’t work.

                                1. 2

                                  I know it is not what you’re looking for but the JS implementation works fine under Linux/ARM64, which is also how the mobile apps are running.

                                  My daily driver is a Surface Pro X with an ARM64 CPU. I’ve run go-ssb as native ARM32 binary and the Electron based client apps under win32 x86-32 bits emulation. The reason for it is just that for the love of all that is sacred I can’t find out how to build the native nodejs modules as ARM64 binaries.

                                2. 1

                                  Could you link to the Haskell implementation? I’d be interested in working on that!

                            1. 7

                              Nice post. It’s me or this switch back is gettting more common? I also got back to linux (but to a desktop instead of another laptop) earlier this year.

                              1. 9

                                It is not just you.

                                I wrote on my blog earlier this year (won’t link as it is just the short observation below mostly) that 2020 is the year of the Linux desktop:

                                • things just work, including Microsoft Teams
                                • people at work and with customers use Linux and nobody there lifts an eyebrow
                                • people here and on HN are behaving like they did towards Mac when Rails and Mac broke through on developer circles.
                              1. 3

                                DuckDuckGo is the way to go for this very feature.

                                1. 2

                                  I’m personally well aware of DDG and use it mostly, this happened to be on my kind of new iPhone where I haven’t changed the defaults yet.

                                  1. 2

                                    there is a whole where

                                    *hole

                                    :)

                                    1. 1

                                      Thanks, fixed :-)

                                1. 4

                                  I agree with Drew that an intelligent being in the loop is necessary - and dangerous for all kids of actually long term storage.

                                  Regarding the technical parts:

                                  For business use there there are rules that can be set in Google Cloud (and probably Azure and AWS as well) to prevent deletion, but I guess of you don’t pay your bills it goes away anyway.

                                  Personally I’ve been considering getting an mdisc compatible DVD burner and some mdiscs for photos, and I’d be happy to know if someone here has looked closer at that option and would care to share their findings.

                                  1. 49

                                    I saw this described on the IRC channel as ‘what-color-are-your-underpants threads’ - lots of easy engagement stuff, crowding more interesting stuff off the front page. My perception is that there is now a lot less of the stuff that differentiated lobste.rs from the other hundredty-dillion tech sites - it was good at bridging computer-science-proper topics and real applications, e.g. someone’s experience report of using formal verification in a real product, or how property testing uncovered some tasty bug in some avionics, or how to synthesize gateware with Z3. That sort of thing.

                                    It doesn’t have to be the case that underwear-threads exist at the cost of quickcheck threads, but as they increasingly crowd the front page and stay there, it means the slightly more cerebral stuff has less chance to be seen, and new people get a different perception of what lobste.rs is about, and so the tone of the place gradually shifts. Some people might think that’s fine, I think it’s a shame. Differentiation is good.

                                    As for ‘if it gets upvotes then by definition it belongs’, I’ve always thought that ‘just leave it to the market’ attitude is total and utter cow-dung. Of course there should be regulation. If you applied that confusion everywhere you’d have sport be replaced by gladiatorial combat, McDonalds purchasing every farm until that was all you could eat, and other kinds of dystopia that unfortunately some americans are beginning to suffer a flavor of (choosing between insulin and death, $1000 toilet roll…). There is nothing inevitable about letting upvotes decide the tone of the site, it’s not a fundamental physical force. You’re allowed to intervene, complain, and so on. It should be encouraged, I think.

                                    1. 21

                                      crowding more interesting stuff off the front page

                                      Come on, there’s very rarely more than one of these threads on the front page, how is that crowding?

                                      1. 7

                                        Well I counted three at one point today, which is over 10% of the front page. I’d like to nip this virus in the bud! It’s too facile to make corona references but regardless, we can go from ‘15 deaths out of 300 million people, no big deal’ to We Have A Problem is a fairly short space of time.

                                        One of the more useful and formative talks I watched when helping to start my business was Ed Catmull [computer graphics pioneer and former Pixar president]’s talk entitled ‘Keep your crises small’*in which he makes the case that businesses are fundamentally unstable and it’s especially hard to notice the bad stuff during periods of high growth or profitability. He contends that you must always have your hand on the tiller and make steering corrections before problems get too big. I see an analogous situation on lobste.rs.

                                        Look at my posting history here. It’s crap. I am a consumer and not a contributor. I have no right to voice my opinion really because I have not done my bit to try and steer lobsters in the direction I want. I am a mechanical engineer with no formal CS background and I stayed here merely because I learned a great deal, and my industry is one built on MScs and PhDs committing abominations in Excel and Matlab, in which a bit of solid CS and solid industrial best-practice would reduce the friction in aerospace R&D by an order of magnitude. It took me five years to get one of my customers to switch to python. Now one of them is using Hypothesis (!) and advocating its usage more widely in a reasonably large aerospace company. I am a True Believer in the value of Advocating the fruits of Computer Science in a field where most participants think the low hanging fruit lies elsewhere. All I’ve been doing is sharing the good stuff that lobsters introduced me to. And this is why I lament the fact that it’s being muscled out by what vim colorscheme do we all prefer, and why I therefore am moved to leave a comment like the grandparent.

                                        Yes, I will make more effort to upvote and comment on the bits of lobsters I value from now on.

                                      2. 11

                                        is that there is now a lot less of the stuff that differentiated lobste.rs from the other hundredty-dillion tech sites

                                        There is and it’s due to less demand. What the audience wants has changed. I was still doing submissions like you described. They rarely hit the front page. The things getting upvoted were a mix of Lobsters-like content and stuff that gets high votes on other sites. Sometimes cross-posted from those sites. I stopped submitting as much for personal reasons (priority shift) but lack of demand/interest could’ve done it by itself.

                                        1. 8

                                          I stopped submitting as much for personal reasons (priority shift)…

                                          For what it’s worth, I noticed that you have been posting less. Hope all is well.

                                          1. 12

                                            I’ll message you the details if you want. It’s been a trip with me back to the Lord, surviving some Gone Girl shit, and facing COVID workload right after. Right now, Im focused on fighting COVID and problems it causes however I can. City-wide shortage on toilet paper, soap, cleaners, etc and nurses having no alcohol/masks made me source them first. Gotta block hoarders and unscrupulous resellers, though.

                                            Gonna have to find some web developers who can build a store or subscription service. Plan to let people pick up limited quantities that I order in bulk and resell just over cost. Might also scan what’s in local stores to reduce people’s time in them. After that, maybe a non-profit version of InstaCart with advantages that may or may not be easy to code. Got an anti-microbial scanner on the way for whatever.

                                            Once everything settles, I’ll get back to my security projects. I just go where needed the most. Worse, people arent social distancing here: enveloping around me constantly. COVID can kill me. So, Im tired after work from holding my breath and dodging people up to 14hrs a day. Had no energy for doing CompSci papers either.

                                            So, there’s a brief summary of some things Ive been up to for anyone wondering.

                                            1. 4

                                              I’m sorry to hear that. I assumed that you must be busy with other stuff or taking a break, but I wouldn’t have guessed how hard of a time you were having. I hope that things start looking up for you soon.

                                              1. 4

                                                I really appreciate it. Everyone supporting these comments, too, more than I thought I’d see. I’m good, though. (taps heart) Good where I need to be.

                                                The possibilities and challenges do keep coming, though. Hope and pray those of us fighting this keep making progress both inside ourselves and outside getting things done in the real world. I’ll be fine with that result. :)

                                          2. 2

                                            Speaking of which, where do you find your papers?

                                            1. 6

                                              I applied old-school methods of using search engines to paper discovery. I pay attention to good papers that cite other work. Each sub-field develops key words that are in most of the papers. I type them into DuckDuckGo and/or Startpage with quotation marks followed by a year. Maybe “pdf” with that. This produces a batch of papers. I open all of them glancing at abstracts, summaries, and related work. I’ll go 5-10 pages deep in search results. I repeat the process changing the terms and years to get a batch for that sub-field. Then, I used to post the better ones one by one over a week or two. Then, do a different sub- or sub-sub-field next.

                                              The Lobsters didn’t like seeing it that way. Too much on the same topic. So, I started getting the batches, saving them in a file, batch another topic when I have time/energy, and trickling out submissions over time with varying topics. So, I might have 50-100 papers across a half dozen to a dozen topics alternating between them. I just pick one, submit it, and mark it as submitted. Eventually, when there’s not much left, I would just grab some more batches.

                                              1. 2

                                                Wow that’s amazing! Thank you so much for doing this! I’ve seen some really nice papers here but I didn’t realize there would be this kind of work behind the posting.

                                                1. 2

                                                  I thought people like you just happened to read extremely much.

                                                  Equally impressed now, just for a different reason.

                                            2. 2

                                              I get the idea of that. I think that what makes the distinction between good and not-so-good ask threads is the length of the responses. For share your blog - what is there to say except a link to your blog? I didn’t bother looking at that one. On the other hand, the distro question generated a ton of long responses about various Linux distros and the pros and cons thereof, interesting stuff. I wonder if there’s some way we could discourage short responses on ask threads, or ask thread topics that tend to generate short responses.

                                              1. 1

                                                Of course there should be regulation. If you applied that confusion everywhere you’d have sport be replaced by gladiatorial combat, McDonalds purchasing every farm until that was all you could eat, and other kinds of dystopia that unfortunately some americans are beginning to suffer a flavor of (choosing between insulin and death, $1000 toilet roll…).

                                                It’s not that I’m looking forward to opening a discussion about this topic, but are you sure this would be the case? Lots of pathological actions done by monopolies are the result of regulating the market in a way which effectively removes competition, leaving the power to the monopolies (in fact, lots of megacorporations that exist nowadays wouldn’t be able to grow to such sizes if it wasn’t for the help from the government). I wouldn’t be so sure that the lack of regulations is the main problem.

                                              1. 3

                                                Thanks for sharing and liking everyone.

                                                If you enjoyed this tou might also enjoy my “interview” with my internal strawman of Chrome developers from last March: Are you making a real web application? Or just a Chrome application?.

                                                1. 13

                                                  Filtering tags (ask included) is the way to go.

                                                  If they ever add the infamous domain filtering, lobster would become perfect for me. I would start filtering the few domains that I 100% hide and be done with it.

                                                  Perhaps stick to other less abused tags or keep clicking hide.

                                                  1. 15

                                                    I disagree. Filtering on tags means you miss out entirely…better for the community to just take responsibility for weeding out garbage and maintaining standards.

                                                    1. 8

                                                      Sure, you can rely on the community to weed out “garbage”, but how exactly do you plan on reaching a consensus about what qualifies as “garbage”?

                                                      Because I’ll tell you right now, none of these “low effort” posts that have been made over the past few days are what I would call “garbage”. I think that kind of content has a place on a website like this (or any site with a specialized focus).

                                                      And that seems to be the big issue here: “low effort” posts like these receive community engagement, and a lot of people like them, but then there seems to be a lot of other users who do not.

                                                      Which is exactly the kind of situation that filtering tools are great for.

                                                      Either way, I think the best solution here would not be to phase out these kinds of posts, but instead establish a recurring schedule that they can be posted on.

                                                      For instance, just like the week/weekend posts recur weekly, posts asking about tools or other “low effort” content could happen semi-annually, on different dates, so that you could regularly have one of these kinds of posts for people to engage with, but not all at the same time.

                                                      Something like “What tools do you use?” every January and June, “Share your blog” every April and October, etc.

                                                      Then, just create a recurring tag that can be applied to these kinds of posts so that users can filter them out if they want (though their frequency should be much reduced at that point, so I doubt many will want to).

                                                      That is exactly what is done over at Tildes. We have a bunch of topics that get posted (automatically by the website even) and they all recur on a schedule. Users can hide those topics by filtering out the recurring tag.

                                                      You can see all the recurring topics on the site:

                                                      https://tildes.net/?tag=recurring

                                                      There are even, for the moment, daily recurring topics about the coronavirus. These recurring topics have worked out wonderfully for the community so far.

                                                      1. 5

                                                        Then, just create a recurring tag that can be applied to these kinds of posts so that users can filter them out if they want (though their frequency should be much reduced at that point, so I doubt many will want to).

                                                        This is an excellent idea. I have had the desire to hide the “What are you doing this week/end?” threads for a long time now, but didn’t want to filter the “ask” tag as I still sometimes find value in them. I thought I needed a way to filter based on titles, but a recurring tag would work even better.

                                                      2. 3

                                                        What about implementing a recommendation system like “People who have hidden the same stories you have also hidden in the past, have also hidden this subset of stories currently in the frontpage; so we have visually de-emphasized it”?

                                                        1. 1

                                                          So just add support for filtering out people, the person who posted this complaint could have just filtered out stories by mraza instead of making another thread where people can post low effort comments.

                                                          1. 3

                                                            I don’t believe filters build good communities, least of all because they break the ability to form community norms and standards. If two members have a vastly different experience of what a community is about, they will only diverge over time.

                                                            Plus, filtering tends to mean that the average unfiltered experience for new users regresses to the mean.

                                                            1. 2

                                                              Filters are the only way to build communities, because each person has limited bandwidth and different opinions.

                                                              Saying “I don’t believe in filters for good communities” is like saying “the utopian community is an echo chamber has consensus on signal/noise evaluations” and I won’t disagree with you on that. However I will contest the viability of this “perfect consensus model” as a desirable engineering goal for building a good lobste.rs scale community. In my experience limiting features doesn’t change usage, it just pushes more of the filtering into the wetware (e.g. clicking ‘next page’ a bunch of times and scanning headlines).

                                                              To make a contrived example: I subscribe to some RSS feeds and mailing lists, this is my personal feed and although it is constructed explicitly (cherry-picking) using the basis of a vector space we call “the internet” rather than implicitly by taking the negation of some content-set in some subspace, like for example lobste.rs, it is in fact very much the same idea, the social contract is: I give up some of my control and in exchange I save some amount of work. Now our mission is to give back control (or rather the perception, via aligning opinions) but still make it cost minimal work.

                                                              So, here on lobste.rs we are sharing a single feed together and one way to give control is to allow people to “slice the feed” or “tune in to a certain frequency” (e.g. filtering out the tags they think are noisy or see all the posts of a tag you think is signaly). This means that in theory we allow for more technical higher quality discussion since in the ideal version of this model there will be an expert community at the heart of each tag and these communities compose to form the lobste.rs community. Best of all, each expert community (or “opinion population”) has roughly the “perfect consensus model”.

                                                              Regarding the unfiltered experience regressing to the mean (a la reddit’s “front page of the internet”) this is totally true IF the join of all these expert communities has no discernible qualities. So long as most of us can all tolerate the Top of our space (i.e. unfiltered front page) things will be fine. Once we reach the scale that we are pushing the foundation of the community into more and more heavily filtered sections of the space then we have failed to preserve this identity but maybe that wasn’t a good goal to begin with?

                                                              Preserving the identity means maintaining a Goldilocks zone where the conflicts can still be managed using filtering without ruining the unfiltered experience. The next level of scaling becomes a model like ActivityPub where each pub is a lobste.rs equivalent and the unity is lost but each subspace is deeper than before (but this is hard to do, for example ActivityPub does not have particularly “deep” pubs afaict).

                                                              So the central thesis is: “giving people more axis to slice along (without giving too many!) is a black magic that if done correctly should improve our community”

                                                              Okay, so lets say we know where in the hierarchy we want to place the lobste.rs identity and we leave aside any compositions that reach ActivityPub scale. Then we will never have a unfiltered experience that regresses to the mean since we will actually be filtering at the community level (banning users and flagging stories). This may create some level of spite in the surrounding ecosystem so even this is not side-effect free.

                                                              As a random anecdote; I have seen the same phenomena happen in group chats often, you have a small group of friends / acquintances, they make a group chat without any particular investment and so at the beginning people just add new people randomly and the group becomes lots of fun. Then people become afraid that they will lose the group and start making restrictions or somehow try to preserve the optimal state, this leads to stagnation and eventually the chat becomes a relic, too sacred to have fun.

                                                              Bringing it back to the ground: I’m saying you misidentify the problem as filtering, I think the problem is consensus on identity. If we allow filtering within the community to silence people that you don’t think produce valuable output then that is the internet equivalent of “agree to disagree” which is an important tool for being able to live with other people in a small space. Eventually some disagreements may grow into faction-splitting situations and in those cases we will be forced to invent the ActivityPub analogue. But hopefully the act of coexisting in a similar environment will actually resolve these conflicts as people learn how to communicate in a shorthand that others can appreciate.

                                                              The concrete example is @mraza007 and @zge, it looks like the former derives high signal from getting to know in broad-strokes what the lobste.rs community is about while the latter feels he already has a pretty good idea about this (or doesn’t care about it and just wants IT), either way, @zge seems to think @mraza007 is noise.

                                                              Now we can resolve this by taking a hard stance on not allowing one opinion in the community (consensus through violence) or we can take a softer approach like allowing to filter out people. Then the “black magic” is which part of the feature-fractal should we implement, do we want to give an option to set an upvote threshold for unfiltering a piece of information or maybe a dual construct like “trusting users” and if someone you trust upvotes someone you filter then that message gets unfiltered, giving you a chance to re-evaluate the filtered user.

                                                              Of course maybe these tools will lead to the community segmenting i.e. we implement the trust idea above and then everyone @zge trusts also filter similarly as him and the ones they filter are filtering back. This is pretty much equivalent to having subreddits and the problem you describe regarding the unfiltered experience suffering is a foregone conclusion in that case.

                                                              So the result is that you end up in feature creep as you attempt to maintain balance. Lets say you want to maybe give new users access to the different filter-perspectives but then you’ve just invented subreddits (albeit emergent subreddits rather than predefined ones, so it’s bottom up organization rather than top down, which is always good) and we’ve lost our soul…

                                                              So basically, I agree, ideally we all can coexist without losing our identity, but this is a very delicate engineering problem and while we can maybe find some nice local-optima that isn’t too violent I fear that “all good things must come to an end” is always the truth with these things.

                                                              In the end, a “chose your own delusion realiity” model is probably the best we can do.

                                                              Sorry for how long this became, I didn’t anticipate it when I started and while I would like to edit it down I need to get back to work..

                                                              1. 1

                                                                I think the problem is consensus on identity. If we allow filtering within the community to silence people that you don’t think produce valuable output then that is the internet equivalent of “agree to disagree” which is an important tool for being able to live with other people in a small space.

                                                                I like this framing of the problem, but disagree heavily on the “agree to disagree” bit.

                                                                The lowest effort solution, and one that appeals to techies because it’s an engineering solution to a people problem and one that doesn’t require scary topics like taste and judgement, is the filtering solution you advocate.

                                                                Filters don’t prevent an accumulation of garbage, which tends to leak when it isn’t labeled properly. It also tends to lead to fractured communities (see also: /pol/ or even the culture tag here). It also means that people that see Lobsters fresh are going to be more lost, with people saying “oh no osdev is a garbage fire, but the plt tags are worth following” (or whatever).

                                                                I’d rather just pull on our big kid pants and try to solve the conflict and settle on norms instead of being conflict avoidant.

                                                                1. 1

                                                                  I don’t think the level of filtering that we have now is too bad and being able to filter users wouldn’t be a big change but I also see the slippery slope so I understand your hesitation.

                                                                  Really what I want to point out is that the problem is hard and any “solution” is a tradeoff. Which tradeoffs we choose will define our collective identity, so for example; I am okay with the feed becoming too fast for me to keep up with - so long as I can just go in, find what I want, and then leave. Granted, this means I am okay with the degradation of the community which is the whole point for many of us (rather than just another faceless content source).

                                                                  In general I agree that ‘agree to disagree’ is not the best way to act in your relationships… a step up from that is ‘suspending disbelief’ and then re-evaluating periodically to see if the disagreement can be resolved. However the problem stems from the need for eventual consistency and currently I don’t see a way to keep consistency without an ever increasing supply of violence (as the user base grows so does the amount of conflicting views).

                                                                  There’s conflict avoidance but the conflict intolerance is what I worry about, like when two engineers have different ideas on a detail in a system so the manager steps in and forces a worst-of-all-worlds solution just to avoid having the conflict end with a “winner” and because there is no obvious best-of-all-worlds solution (or the conflict wouldn’t have existed in the first place).

                                                                  Conflict intolerance is like the chat group that became a relic, it’s the solution that has become politically correct (in the groups collective ethos) but makes everyone unhappy (most of the time because the group is unwilling to drop a false assumption).

                                                                  Conflict avoidance means you just have some meekness and people who will suffer rather than rocking the boat. This brings down morale but often this can be managed (I think good management is all about standing up for those who can’t stand up for themselves).

                                                                  Anyway, this exact problem is why I am trying to design a forum system organized around different primitives. The users should be able to express political consensus around a certain type of collective identity in the semantics of the system and shared views between different identities should be truely shared, collapsing duplicates into canons and making the space of collective identities continuous rather than discrete.

                                                        2. 13

                                                          Bingo.

                                                          Different people enjoy different things. I happen to think that this community contains a bevvy of super interesting people, so I quite enjoy the ‘low effort’ posts you’re harping on.

                                                          So I’d much rather give you the tools to ignore it than try to remove them by mandate.

                                                          1. 5

                                                            Totally agree. A diverse group of interesting people scattered across the world, too. Threads that bring out the activities and lifestyles of these interesting people just make it more fun for me.

                                                            1. 4

                                                              Agreed that’s one the reason why i posted these questions. Honestly seeing this post today really surprised me

                                                              1. 3

                                                                I think I agree.

                                                                There was a promising site, lamernews or some other really weird name, - just like hn and llbsters ;-)

                                                                AFAIK and IIRC some early users killed it by overpolicing it and scaring away everyone.

                                                              2. 3

                                                                Filtering works to a point, but I think a whitelist would also be a good idea.

                                                                For example, I have web blocked because most of the posts don’t interest me. Occasionally however there is a post on how someone used Lisp to write a web application or something to that effect, which gets filtered from my frontpage because it’s tagged with web as well as lisp. If there was a way for me to say I always want to see lisp posts regardless of what other tags they have, I’d be very happy. Unfortunately I don’t have enough Ruby skills to implement this.

                                                                1. 2

                                                                  Add a userscript to set display:none for li.story > div.h-entry > div.details > span.link > a[href=https://somedomain.com/*] on lobste.rs.

                                                                  1. 2

                                                                    Filtering would be the solution, if you are never interested in ask posts, but I do enjoy reading and sometimes contributing to “what are you doing this week/-end” posts. I don’t have an issue with annual or semi-annual “share you’re site” threads. I started this thread to discuss the frequency and kind of questions that are being asked, and filtering would just be overkill.

                                                                  1. 1

                                                                    erik.itland.no - old style, next to no styling, few pictures, no third party trackers (but I do look at server logs). It is very human, authentic or just plain-and-a-bit-weird and contains a mix of

                                                                    • programming,
                                                                    • tech history (a post about google+ in particular was very popular a year or two ago, and actually today a post about the “Chrome is the new IE” meme that I wrote earlier today seems to be gathering some eyeballs ),
                                                                    • observations (weather, habits of Norwegians etc)
                                                                    • comments on news from Norway etc

                                                                    Use RSS or follow on fediverse at @blog@erik.itland.no as I only post when I feel like posting. The upsite is of course less filler content ;-)

                                                                    1. 1

                                                                      My first distro was Red Hat (7.1 I think, came in the cover of a book about linux).

                                                                      I later eperimentet with a lot of distros, I remember I particularily liked Mandrake (later Mandriva) and preferred KDE before I settled with Gnome 2 on Ubuntu from 2006 until they swapped to Unity.

                                                                      After that there were a couple of years were I felt there were no really good options until I found KDE Neon, which I have used for the last 4 years, I guess one of the installations (the one I work on now) have been with me since then which is quite remarkable.

                                                                      I think what distro one use is important. For me it is incredibly important that it is fast and doesn’t get in the way. Otherwise I could have used one of the two mainstream options, but for me the micro-lagging on Windows drives me crazy, as does CMD-Tab (and back when I used it, the fact that fn and ctrl was swapped) on Mac.

                                                                      1. 1

                                                                        We were forced to use a bare bones editor in Java classes at school for reasons like this: learn the concepts, not defer learning. Only for me the easiest way towards really understanding something was to use it in a realistic context.

                                                                        I knew and enjoyed programming and got a B (second best) but decided Java was not for me and only began using it and enjoying it after I started working with someone who showed me how to use eclipse.

                                                                        I know school shouldn’t tie you to one product but teach the concepts, and I agree that knowledge of operating systems concepts and the differences between them are important but that doesn’t mean schools shouldn’t teach how to use available after they have in the first week explained how to compile from the command line. Especially when the tooling is free and open source.

                                                                        BTW: the microcontroller course I took was much more reasonable and taught assembly first, and then immediately followed up by teaching C, explaining how to inspect the generated code in the process.

                                                                        The result was vastly different: in the Java case I got a distaste that lasted for a years until someone took the time to help me. In the microcontroller case I enjoyed both assembly and C.

                                                                        1. 3

                                                                          This is interesting and well worth mentioning.

                                                                          That said: Signal has had at least one (but I think more nasty flaws) over the years and WhatsApp is constantly uploading all your data to Google Cloud in the form of unencrypted backups.

                                                                          Signal is good as far as I can see but I have a hard time taking anyone seriously who actually uncritically recommends WhatsApp while telling me how bad Telegram is.

                                                                          Finally: I’m still no Telegram fanboy and I am in the market for what I think WhatsApp was/should have become;

                                                                          • low yearly fee like before
                                                                          • API access (didn’t exist back then)
                                                                          • same encryption as today (it was bad back then)
                                                                          • bots and a few other improvements from Telegram
                                                                          • actually I realize I don’t care extremely much about bulletproof crypto (I have Signal and can use it of I need to).

                                                                          I’ll happily pay again like I did for WhatsApp as llng as there’s some kind of legal guarantee that prevents “the next WhatsApp” from selling out.

                                                                          1. 4

                                                                            That’s our fellow lobster @hyfen, pinged him in case anyone has questions! He is still actively working on this epic project.

                                                                            1. 2

                                                                              Is the source anywhere? Or if it’s closed, does he want testers at all?

                                                                              1. 1

                                                                                At least there is this page where you can sign up for email updates or even email him: https://hyfen.net/memex/

                                                                            1. 7

                                                                              The Naming Things book

                                                                              you’ll want to get the book!

                                                                              We’ll explore these in detail in the book.

                                                                              The book explores these

                                                                              these are explored in more detail in the book

                                                                              To learn more about them and see examples of how to apply them, click the button below: GET THE BOOK

                                                                              Sorry, but I flagged this as spam

                                                                              1. 3

                                                                                This is not spam, this is bad content. I’m not saying this to insult the author. It is my opinion, which I base on the following argument:

                                                                                The article lists some mistakes you can make in naming things. That list is not exhaustive. Nor does it give practical hints how to name things correctly. Nor does it provide examples. As a reader, you can get more information by reading the Wikipedia article on naming conventions.

                                                                                On my blog I have also written bad content in the past. I bet that I will write some bad content in the future. It happens, get over it. Maybe the author will be able to write a better article the next time he posts here. If you want, you can send me your article before you publish it and I will give you constructive criticism.

                                                                                1. 5

                                                                                  Bad/poor/low-effort content in itself is ok and adequate reaction to it is to ignore (maybe downvote) it and try providing constructive criticism. Bad/poor/low-effort content immediately followed by an ad-like call to action (“subscribe for real actual contents I pinky promise” in this case) is fully worth of being flagged as spam/ad. There’s a world of a difference: in the first case I can assume idealistic intentions with poor execution; in latter case I can only assume monetization intentions. I don’t claim it bad, but if only that this platform is not intended as an ads aggregator AFAIU.

                                                                                  1. 1

                                                                                    I see your point and I agree that the CTAs are unfortunate. I understand why one would decide to flag the article as spam.

                                                                                  2. 1

                                                                                    Thanks for the constructive criticism. (I’m the author.) I appreciate the feedback and agree with all of your points; I’ll use this to improve content going forward. The intention of the post was to see what people would think of these principles, but the feedback here so far has focused on other areas, so I’ll work to improve those. In case it’s helpful to know, I’m writing the book in parallel to this, and the book’s beta readers have also been giving me very valuable feedback that has some overlap with what has been mentioned here.

                                                                                  3. 2

                                                                                    I just looked at it.

                                                                                    FWIW I say this is not enough to qualify as spam. I’ve not upvoted it because it wasn’t interesting to me but not spam. And in my opinion “spam” just like “hate speech” is terms that should be reserved for actual “spam” and actual “hate speech” and not “stuff I don’t like” or “stuff I don’t agree with”.

                                                                                    It is totally ok to write less than perfect blog posts that recommends the authors book.

                                                                                    When does it cross into spam/blog spam?

                                                                                    • you get it in the form of unsolicited mail
                                                                                    • someone posts it multiple times to your forum
                                                                                    • when it is about something completely different (you search for a JS implementation of Spring Framework and get a page that contains the relevant keywords, in white text on white background)

                                                                                    That said, posting history for that domain us interesting, but I’ll leave that to the mods, I might be misreading: https://lobste.rs/domain/namingthings.co

                                                                                    1. 0

                                                                                      (I’m the author.) Thanks for the feedback on this; I agree that it was heavy-handed. I’ve updated this part of the blog post to deemphasize it and I’ve removed the CTA button.

                                                                                    1. 26

                                                                                      I think return and throw should be emphasized. I want it to be very clear if there are early exits!

                                                                                      1. -2

                                                                                        Even better:

                                                                                        Remove superfluous control-flow constructs like return, throw, for, break and continue from the language – they are pointless relics of the past anyway.

                                                                                        The slight convenience of being able to write “nicer” code is completely offset by the difficulty to read such code a week later (or a month, or a year).

                                                                                        1. 13

                                                                                          Oh gods, I could not disagree more; I think explicit control flow should be prominent. One of the languages I find most frustrating has a tonne of implicit and optional control flow that renders it super hard to read or write, as it allows for way too much personal style.

                                                                                          1. 3

                                                                                            I’m not sure how you are disagreeing with me – my suggestion is that there is only one set of control-flow keywords:

                                                                                            • if: replaces return, throw, switch/case, break, continue.
                                                                                            • while: alternative to recursion. Replaces for loops.

                                                                                            With this

                                                                                            • there is neither implicit, nor optional control flow
                                                                                            • there are no styles, because there is only one choice
                                                                                            • things become easy to read, because there are only 2 keywords to look out for, not 10
                                                                                            • things become easy to understand, because there are simply no interactions between different “levels” of control-flow anymore
                                                                                            1. 9

                                                                                              Would absolutely love to program in a language with insane nested if statements because returning early is for chumps apparently?

                                                                                              1. 2

                                                                                                Couldn’t you pick any existing language for that? It’s not like people aren’t doing it voluntarily…

                                                                                              2. 4

                                                                                                This is almost Rust. Everything-is-an-expression makes it possible to make entire function body just ifs and match blocks that exhaustively handle all possible cases, so you never need an explicit return.

                                                                                                However, in such case every last statement of every “leaf” block becomes an implicit return. If you overdo it, it’s completely inscrutable.

                                                                                                BTW: your list 100% matches brainfuck: no implicit control flow, no style choices, no superfluous keywords, only one kind of control flow. Less is not always better.

                                                                                                1. 3

                                                                                                  This is almost Rust.

                                                                                                  I don’t think this is even remotely true – there is a huge difference between “you could do that” and “there is no other way of doing it”.

                                                                                                  The latter means that the code you haven’t written (99.99% of the code) is written in that specific style; the former only means you can make your own code follow these rules (which is rather irrelevant).

                                                                                                  Rust has waaaaay to much going, it’s not a good data point.

                                                                                                  Less is not always better.

                                                                                                  At least the goal posts aren’t moving every single year about the “right” language size, like in Rust’s “more features == better language” ideology. :-)

                                                                                                  This is why my Rust libraries (very small, only about 5 million total downloads) are permanently staying on Rust 1.13 (released 2016), because I don’t feel the feature additions in the meantime have been worth the cost.

                                                                                                  your list 100% matches brainfuck

                                                                                                  Turing tarpit much?

                                                                                                2. 2

                                                                                                  while: alternative to recursion. Replaces for loops.

                                                                                                  I don’t understand this, return, throw, and so on are also alternatives to if, but we’re removing them, yet loops, which can be completely removed and replaced with simple recursion are an alternative that we’re keeping? how so?

                                                                                                  1. 3

                                                                                                    The reason is that sometimes it’s hard to write tail-recursive code (or code the compiler can turn into PTCs), so having to pick code that leaks stack frames at runtime, or having awhile loop, I’ll pick the latter.

                                                                                                    1. 2

                                                                                                      a fair enough point, I understand

                                                                                                  2. 2
                                                                                                    • if: replaces return, throw, switch/case, break, continue.
                                                                                                    • while: alternative to recursion. Replaces for loops.

                                                                                                    Would you mind showing us a code example?

                                                                                                    1. 2
                                                                                                      class HashMap[K: Identity + Hash, V]()
                                                                                                        var insertedAndDeleted: BitSet = BitSet(0)
                                                                                                        var keys: Array[K] = Array.empty()
                                                                                                        var values: Array[V] = Array.empty()
                                                                                                      
                                                                                                        var size: Int = 0
                                                                                                        var cap: Int = 0
                                                                                                      
                                                                                                        ...
                                                                                                      
                                                                                                        fun isLive(idx: Int): Bool =
                                                                                                         self.insertedAndDeleted.contains(2 * idx) &&
                                                                                                         self.insertedAndDeleted.contains(2 * idx + 1).not
                                                                                                      
                                                                                                        fun get(key: K): Option[V] =
                                                                                                          assert(self.size < self.cap)
                                                                                                      
                                                                                                          var hash = key.hash
                                                                                                          var idx = hash.bitwiseAnd(self.cap - 1)
                                                                                                      
                                                                                                          var continue = true
                                                                                                          var result = None
                                                                                                      
                                                                                                          while continue
                                                                                                          do
                                                                                                            if self.isLive(idx)
                                                                                                            then
                                                                                                              let currentKey = self.keys(idx)
                                                                                                              if currentKey.hash == hash && currentKey === key
                                                                                                              then result = Some(self.values(idx))
                                                                                                              else ()
                                                                                                              idx = (idx + 1).bitwiseAnd(self.cap - 1)
                                                                                                            else
                                                                                                              continue = false
                                                                                                      
                                                                                                          result
                                                                                                      

                                                                                                      Here is some write-up regarding control flow.

                                                                                                    2. 1

                                                                                                      I agree with most things you say, but the “for” construct is invaluable for math-heavy code where “while” feels very unnatural and verbose. I agree that the notation for “for” should be simplified to only allow an iterator over a fixed range of two constant ints. I would actually prefer to keep this “for” (which is general enough) and discard “while”. It makes your code much easier to reason about (you know exactly how many loops it is going to do, without needing to understand the logic of the program).

                                                                                                      1. 1

                                                                                                        I think you can make that case every control flow keyword; that’s basically how most languages have accumulated every control flow keyword ever invented in the last 60 years.

                                                                                                        I’m not claiming that not having some specific keyword isn’t inconvenient in cases that keyword would shine – I’m making the case that the inconvenience caused is smaller than having to learn, remember and understand how the half dozen control flow keywords interact with each other.

                                                                                                        It’s a bit like static imports in Java – sure, they are sometimes convenient (when writing code), but I’d argue that having two trivial, interchangeable ways to write the same thing is a much bigger inconvenience (when reading it).

                                                                                                  3. 2

                                                                                                    Discouraging use of control flow words makes sense for future readability, but getting rid of them completely is too strong. Sometimes you really need to use these old style control flow constructs.

                                                                                                    J gets this right – idiomatic J code are a series of one-liners using zero old-style control words (therefore allowing code to be one-way-to-do-things idiomatic, as you suggest in a reply). But if, while, try, catch, return, for, continue, goto (?), are all still there if you need them.

                                                                                                    1. 1

                                                                                                      getting rid of them completely is too strong

                                                                                                      I didn’t propose that.

                                                                                                      But if, while, try, catch, return, for, continue, goto (?), are all still there if you need them.

                                                                                                      So basically “don’t change anything”, with the result that none of the benefits are realized?

                                                                                                      1. 1

                                                                                                        I didn’t propose that.

                                                                                                        Ah, I misread. Then we’re talking about different things – you were thinking about removing some keywords and I was thinking about removing all keywords.

                                                                                                        So basically “don’t change anything”, with the result that none of the benefits are realized?

                                                                                                        I think the idea is to encourage refactoring foreign code using if, for, and while, into a more idiomatic form using combinators and no if, for, and while. Not removing these keywords from the language outright just makes it easier to get started without knowing all about J’s combinators, which is good since everyone’s likely coming from languages that use control flow keywords. Again, this is a different view from yours.

                                                                                                1. 4
                                                                                                  • Work,
                                                                                                  • Play with the kids (my wife will work Saturday night and Sunday morning.)
                                                                                                  • Be one of the hosts at a party (friends only though) at Sunday night.

                                                                                                  Now that I write it down it looks like a busy weekend

                                                                                                  1. 4

                                                                                                    One way to look at it:

                                                                                                    If you aren’t working on a hard problem there’s still room for some improvement.

                                                                                                    Something along the lines of: Churning out CRUD apps? That’s what DHH and the 37 signals folks did until they upped their game, created Basecamp and the Ruby on Rails framework that sparked innovation that changed web development even in the enterprise Java world.

                                                                                                    But they used to be a consulting/web development agency.

                                                                                                    Me, I’ll probably not do that but there’s plenty of hard problems still:

                                                                                                    • deliever my contributions faster and more predictable on existing projects while learning new stuff and maintaining a healthy work-life balance? Can be hard.
                                                                                                    • trying to improve broken processes as an individual contributor? Can be hard.
                                                                                                    • trying to contribute to open source/church/other community groups after taking care of other responsibilities? Can be hard.

                                                                                                    If you want to play life on hard there are opportunities.

                                                                                                    And: I’m lucky as is many others here I guess. For me it is optional and I only do it to dthe degree I choose it myself. For many others hard is the default setting.