1. 5

    Generally speaking, evolutionary algorithms aren’t opposed to deep learning. An evolutionary algorithm is just one kind of search over a parameter search space. A deep learning algorithm can use an evolutionary algorithm for parameter optimization, by using evolutionary search over network weights. Often gradient search get used. Other algorithms can be used as well, such as simulated annealling.

    In this case, the cited paper uses something called Cartesian Genetic Programming which creates a dynamic network of functions, determined by its “genes”. The nodes in the graph process input usng prebuilt functions determined by their genes. The prebuilt function include OpenCV image processing functions and some standard vector and scalar functions.

    In this model, the genes are really just a point in a search space. Other search algorithms could be used for this besides evolutionary algorithms, for example simulated annealling.

    What I find interesting about the algorithm used here isn’t so much the evolutionary search side of it so much as the dynamic topology of prebuilt functions, which, strictly speaking, isn’t tied to evolutionary search.

    1. 8

      After feeling some pain about how complicated all software, IT and Enterprise architecture toolsets are, I got this idea of a dead simple architecture sketching tool, that’s based on just nodes and named lines. The input for this tool is a text file that contains triples in the form of subject-verb-object, like so:

      # comments via shebang
      # declare nouns (for better typo recovery)
      internet, web front, app server, DB, Redis
      # declare verbs (perhaps not necessary and may be dropped)
      flows, proxies, reads/writes
      
      # subject-verb-object are separated by more than 1 whitespace (" " or "\t") 
      # somewhat like Robot Framework does it
      # prepositions (to, from, at, etc.) seemed redundant, so not using them
      internet     flows          web front
      web front    proxies        app server
      app server   reads/writes   DB
      app server   reads/writes   Redis
      

      I’m not married to the syntax yet, but it seems fine after a few iterations of experimentation. A tool could then read this and produce a nice graph via graphwiz. And you don’t really need a tool, you could do these sketches on a piece of paper in no time. But when you want to store and share the design, a text file + tool is a good combo.

      Tried to write a parser for this language in Rust’s pest, but that was a bit of a pain – pest’s error modes are quite odd. Seems like when my parser has a problem, Pest blows up on position 0 every time. Perhaps I’ll just do a dumb line-by-line parser by hand instead. And I’d like to implement in Crystal rather than Rust. Also thought about doing it in Swift, but that doesn’t seem to work so well outside of MacOS/Linux (I’m thinking BSDs here) yet so nope.

      Best part? The tool’s name is Architect Sketch.

      1. 5

        Reminds me of mermaid. It has shortcomings but has worked for me most of the times I’ve had to diagram something over the past few years.

        1. 4

          I’ve been working on something similar myself, with the goal of actually deploying infrastructure on it. I think this is a good avenue to explore, and pairs really well with rule engines and datalog systems.

          1. 2

            This is something I’ve been thinking about a lot as well.

            For my use case, I just want to dump, document, or understand how my multitude of web apps, daemons, and IoT devices all interact with one another.

            I had built a rough prototype with ruby and shell scripts using similar syntax, but I couldn’t ever get Graphviz to generate a “pretty” layout for the whole thing.

          1. 5

            As much as I would like a decentralised web to take over. I think I see a major issue with ActivityPub. Apparently, search doesn’t appear to be specified. And one advantage big centralised services will have is the ability to search all their network. For example, I would like to search all the network for a specific username, keyword, etc… Without that, it’s like going back to the net before google search.

            1. 2

              Yeah, I’m curious if ActivityPub can support arbitrary aggregation over multiple nodes. It seems to me that in this kind of architecture, maybe nodes ought to support publishing changes over websocket to interested listeners. You could have aggregation nodes doing things like search, or post rankings, which could attach themselves in this fashion. Plus this would have the added benefit that if you didn’t like a particular aggregator’s implementation (a hotness formula for example, or search indexing algorithm) you could switch to a different one.

              1. 1

                Usernames include the server, so I’m not sure that case makes sense.

                Not supporting keyword search means I don’t get random and bots sealioning my conversations.

                1. 1

                  I was thinking about peer tube for example. It would be very nice if I was able to search only for the current node or on all fediverse for the things I’d like to find. Like niche programming languages for example. The goal of publishing something is to be read, even by bots. Also I’m not sure a federated network would be more robust in face of bots. Pretty sure it would be the opposite, because each node would have less data to analyze for bot detection. Still by the end, I’m pretty sure the only good solution would be to have a global “Web of Trust”.

                  1. 2

                    Ahh, I see. 99% of my social media us is to connect with people I already know (or have in my extended network). For that use case, it’s a feature to be less easily discovered.

                    For publishing content it’s definitely the opposite; you want it to be found. Difficult though because now you’re competing for attention with spammers.

              1. 12

                So, this might be a good time to float an idea:

                None of this would be an issue if users brought their own data with them.

                Imagine if users showed up at a site and said “Hey, here is a revokable token for storing/amending information in my KV store”. The site itself never needs to store anything about the user, but instead makes queries with that auth token to modify their slice of the user’s store.

                This entire problem with privacy and security would go away, because the onus would be on the user to keep their data secure–modulo laws saying that companies shouldn’t (and as a matter of engineering and cost-effectiveness, wouldn’t) store their own copies of customer data.

                Why didn’t we do this?

                1. 17

                  http://remotestorage.io/ did this. I’ve worked with it and it’s nowhere near usable. There are so many technical challenges (esp. with performance) you face on the way that result of you basically having to process all user data clientside, but storing the majority of data serverside. It gets more annoying when you attempt to introduce any way of interaction between two users.

                  We did try this, saw that it’s too hard (and for some services an unsolved problem) and did something else. There’s no evil corporatism in that, nor is it a matter of making profit, even if a lot of people especially here want to apply that imagination to everything privacy-related. It’s human nature.

                  1. 2

                    basically having to process all user data clientside

                    If I go to a site, grant that site a token, couldn’t that server do processing server side?

                    It gets more annoying when you attempt to introduce any way of interaction between two users.

                    Looking at remotestorage it appears there’s no support for pub/sub, which seems like a critical failing to me. To bikeshed an example, this is how I see something like lobste.rs ought to be implemented:

                    • User data is stored in servers (like remotestorage) called pods, which contain data for users. A person can sign up at an existing pod or run their own, fediverse-style.

                    • These pods support pub/sub over websocket.

                    • A particular application sits on an app server. That app server subscribes to a list of pods for pub/sub updates, for whatever users that have given that application permission. On top of these streams the app server runs reduce operations and keeps the result in cache or db. A reduce operation might calculate something like, give me the top 1000 items sorted by hotness (a function of time and votes), given streams of user data.

                    • A user visits the site. The server serves the result instantly from its cache.

                    • Additionally the pub/sub protocol would have to support something like resuming broken connections, like replay messages starting from point T in time.

                    Anyway, given this kind of architecture I’m not sure why something like lobste.rs for example couldn’t be created - without the performance issues you ran into.

                    1. 2

                      If I go to a site, grant that site a token, couldn’t that server do processing server side?

                      If your data passes through third-party servers, what’s the point of all of this?

                      The rest of your post is to me, with all due respect, blatant armchair-engineering.

                      • The pub/sub stuff completely misses the point of what I am trying to say. I’m not talking about remotestorage.io in particular.

                      • Lobste.rs is a trivial usecase, and not even an urgent one in the sense that our centralized versions violate our privacy, because how much privacy do you have on a public forum anyway? Let’s try something like Facebook. When I post any content at all, that content will have to be copied to all different pods, making me subject to the lowest common denominator of both their privacy policies and security practices. This puts my privacy at risk. Diaspora did this. It’s terrible.

                      • Let’s assume you come up with the very original idea of having access tokens instead, where the pods would re-fetch the content from my pod all the time instead of storing a copy. This would somewhat fix the risk of my privacy (though I’ve not seen a project that does this), but:

                        • Now the slowest pod is a bottleneck for the entire network. Especially stuff like searching through public postings. How do you implement Twitter moments, global or even just local (on a geographical level, not on network topology level) trends?
                        • Fetching the data from my pod puts the reader’s privacy at risk. I can host a pod that tracks read requests, and, if the system is decentralized enough, map requests from pods back to users (if the request itself doesn’t already contain user-identifying info)

                      See also this Tweet, from an ex-Diaspora dev

                      1. 1

                        If your data passes through third-party servers, what’s the point of all of this?

                        It decouples data and app logic. Which makes it harder for an application to leverage its position as middle man to the data you’re interested in. Doing stuff like selling your data or presenting you with ads. Yet you put up with it because you are still interested in the people there. Because if data runs over a common protocol you’re free to replace the application-side of things without being locked in. For example, I bet there’s some good content on Facebook but I never go there because I don’t trust that company with my data. I wish there were some open source, privacy friendly front end to the Facebook network available, that would let me interact with people there, without sitting on Facebook’s servers, and open source. Besides that, if an application changes its terms of use, maybe you signed up trusting the application, but now you’re faced with a dilemma of rejecting the ToS and losing what you still like about the application, or accepting new crappy terms.

                        The rest of your post is to me, with all due respect, blatant armchair-engineering.

                        Ha! Approaching a design question by first providing an implementation without discussion seems pretty backwards to me. Anyway, as far as I’m concerned I’m just talking design. Specifically I’m criticizing what I perceive as a deficiency in remotestorage’s capabilities. And arguing that a decentralized architecture doesn’t have to be slow, is at least as good as a centralized architecture, and better, in many regards, for end users.

                        Let’s try something like Facebook. When I post any content at all, that content will have to be copied to all different pods,

                        No, I was saying that this would be published to subscribing applications. There could be a Facebook application. And someone else could set up a Facebook-alternative application, with the same data, but a different implementation. Hey, you could even run your own instance of Facebook-X application.

                        making me subject to the lowest common denominator of both their privacy policies and security practices.

                        If you grant an application access to your data, you grant it access to your data. I don’t see a way around that puzzle in either a centralized or decentralized architecture. If anything, in a decentralized architecture you have more choices. Which means you don’t have to resign yourself to Facebook’s security and privacy policies if you want to interact with the “Facebook” network. You could move to Facebook-X.

                        Now the slowest pod is a bottleneck for the entire network. Especially stuff like searching through public postings. How do you implement Twitter moments, global or even just local (on a geographical level, not on network topology level) trends?

                        What I was describing was an architecture where pods just store data. Apps consume and present it. If I have an app, and I subscribe to X pods, there’s no reason I have to wait for the slowest pod’s response in order to construct a state that I can present users of my app.

                        So for something like search, or Twitter moments, you would have an application that subscribes to whatever pods it knows about. Those pods publish notifications to the app over web socket, for example whenever a user tweets. Your state is a reduction over these streams of data. Let’s say I store this in an indexed lookup like ElasticSearch. So every time a user posts a tweet, I receive a notification and add it to my instance of ElasticSearch. Now someone opens my app, maybe by going to my website. They search for X. The app queries the ElasticSearch instance. It returns the matching results. I present those results to the user’s browser.

                        Fetching the data from my pod puts the reader’s privacy at risk.

                        Hmm, I’m not sure if we’re on the same page. In the design I laid out, the app requests this data, not the pod.

                        1. 2

                          With respect, “social media” and aggregator sites are red herrings here. They cant be made to protect privacy by their very nature.

                          I’m more thinking about, say, ecommerce or sites that aren’t about explicitly leaking your data with others.

                          1. 1

                            “With respect, “social media” and aggregator sites are red herrings here. They cant be made to protect privacy by their very nature.”

                            Sure they can. Starting with Facebook, they can give privacy settings per post defaulting on things like Friends Only. They could even give different feeds for stuff like Public, Friends Only, or Friends of Friends. They can use crypto with transparent key management to protect as much of the less-public plaintext as possible. They can support E2E messaging. They can limit discovery options for some people where they have to give you a URL or something to see their profile. Quite a few opportunities for boosting privacy in the existing models.

                            Far as link aggregators, we have a messaging feature that could be private if it isn’t already. Emails and IP’s if not in public profile. The filters can be seen as a privacy mechanism. More to that point, though, might be things like subreddits that were only visible to specific, invited members. Like with search, even what people are looking at might be something they want to keep private. A combo of separation of user activities in runtime, HTTPS and little to no log retention would address that. Finally, for a hypothetical, a link aggregator might also be modified to easily support document drops over an anonymity and filesharing service.

                  2. 9

                    Because the most formidably grown business of late are built on the ability to access massive amounts of user data at random. Companies simply don’t know how to make huge money on the Internet without it.

                    1. 3

                      We did. They’re called browser cookies.

                      The real problems are around an uneducated consumption-driven populous: Who can resist finding out “which spice girl are you most like?” – but would we be so willing to find out if it meant we get a president we wouldn’t like?

                      It is very hard for people to realise how unethical it is to hold someone responsible for being stupid, but we crave violence: We feel no thrill that can compare serving food, working in an office, or driving a taxi. Television and Media give us this violence, an us versus them; Hillary versus Urine Hilarity or The Corrupt Incumbent versus a Chance to Make America Great Again, or even Kanye versus anybody and everybody.

                      How can we make a decision to share our data? We can never be informed of how it will be used against us.

                      The GDPR does something very interesting: It says you’re not allowed to use someones data in a way they wouldn’t want you to.

                      I wish it simply said that, but it’s made somewhat complicated by a weird concept of “data” It’s clear that things like IP addresses aren’t [by themselves] your data, and even a name like John Smith isn’t data. Software understands data but not the kind of “data” that the GDPR is talking about. Pointing to “you” and “data” is a fair thick bit of regulation if you don’t want to draw a box around things and prevent sensible people from interpreting the forms of “data” nobody has yet thought of.

                      But keep it simple: Would that person want you doing this? Can you demonstrate why you think that is and convince reasonable people?

                      I’m doing a fair bit of GDPR consulting at the moment, and whilst there’s a big task in understanding their business, there’s also a big task getting them to approach their compliance from that line of questioning: How does this make things better for that person? Why do they want us to do this?

                      We’re not curing cancer here, fine, but certainly there are degrees.

                      1. 2

                        Browser cookies is something that crossed my mind after I suggested this, but my experience as a web dev makes me immediately suspect of them as durable stores. :)

                        I agree with your points though.

                      2. 2

                        This still doesn’t solve problems with tracking, because companies have already started to require GDPR opt-in to use their products (even when using the product doesn’t necessarily require data tracking), or to use their products without a degraded user experience.

                        See cloudflare, recaptcha, facebook, etc.

                        “You can’t use this site without Google Analytics having a K/V-auth-token”, “We will put up endless ‘find-the-road-sign’ captchas if we can’t track you”, etc.

                        1. 6

                          It’s a mistake to think you can “GDPR opt-in”. You can’t.

                          You have to prove that the data subject wants this processing. One way to do this is to ask for their consent and make them as informed as possible about what you’re doing. But they can decide not to, and they can even decide to revoke their consent at any time until you’ve actually finished the processing and erased their data.

                          These cookie/consent banners are worse than worthless; a queer kind of game people like Google are playing to try to waste time of the regulators.

                          We will put up endless ‘find-the-road-sign’ captchas if we can’t track you

                          I’ve switched to another search engine for the time being. It’s faster, the results are pretty good, and I don’t have to keep fiddling with blocking that roadblock on Google’s properties.

                      1. 11

                        It’s not true that no-one thought the early internet was rubbish. I did, and a lot of my peers did too. We just saw a slow and clunky technology filled with problems and didn’t have the imagination to see further. Strikes me that this is exactly like blockchain. Also, the title talks about Bitcoin, but then discusses blockchain, which is confusing. It’s like dismissing Yahoo, and then dismissing the internet because of it. Very odd.

                        1. 12

                          Email was already faster than physical post in 1992 when I got on the internet as a student.

                          Mailing lists and Usenet presented an awesome opportunity for interaction with people all over the world.

                          Bitcoin purports to be a better payment system. I can go online now, find a widget on Alibaba, pay for it with my credit card and get it delivered in a week or so. In what way does BTC improve on this scenario?

                          1. 2

                            Bitcoin purports to be a better payment system.

                            Bitcoin is a technology. Many people have worked on it for various reasons, and it’s used by many people for various purposes. It doesn’t make sense to talk about its purport as if that were a single unified thing.

                            At least for now it’s an alternate payment system, with its own pros and cons.

                            Cryptocurrencies are still actively iterating different ideas. Many obscure ideas are never tried for lack of a network effect. Bitcoin and its brethren are young technology. I don’t think we can truly understand its potential until people have finished experimenting with it. That day hasn’t come.

                            I think there is an innate human tendency to rush to judgment, to reduce the new to the seen before. When we do so, I think we miss out on the potential of what we judge. This is particularly true for young technology, where the potential is usually the most important aspect.

                            Email was faster than physical post in 1992 but without popular usage lacked general utility. In hindsight it all seems so obvious however.

                            1. 9

                              I wrote:

                              Bitcoin purports to be a better payment system.

                              You write:

                              Bitcoin is a technology. Many people have worked on it for various reasons, and it’s used by many people for various purposes. It doesn’t make sense to talk about its purport as if that were a single unified thing.

                              I’m going off the whitepaper here:

                              Bitcoin: A Peer-to-Peer Electronic Cash System

                              Abstract. A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution.

                              I’ve been following the cryptocurrency space since I first installed a miner on my crappy laptop and got my first 0.001 BTC from a faucet, and the discussion has overwhelmingly been about Bitcoin as a payment system, or the value of the token itself, or how the increasing value of the token will enable societal change. Other applications, such as colored coins, or the beginnings of the Lightning Network, have never “hit it off” in the same way.

                              1. 1

                                Hmm, I’m not sure how that abstract is supposed to show how bitcoin purports to be a “better” payment system, just that it was originally envisioned as a payment system.

                                Anyway, since then the technology presented in that paper has been put to other uses besides payments. Notarization, decentralized storage and decentralized computation are some examples. A technology is more than the intention of an original inventor.

                                Other applications, such as colored coins, or the beginnings of the Lightning Network, have never “hit it off” in the same way.

                                Evaluating the bitcoin technology, if that’s what we’re discussing, requires more than looking at just the bitcoin network historically. It’s requires looking at other cryptocurrencies, which run under similar principles. It also requires that we understand how the bitcoin network itself might improve itself in the future. It doesn’t make sense to write off bitcoin technology simply for slow transaction times, when there remains a chance that the community will fix it in time, or when there are alternatives with faster transaction times.

                                Besides that, there are the unthought-of uses that the technology may have in the future. And even ideas that people have had that have never been seriously tried. With all that in mind, the potential of bitcoin technology can’t really be said to be something we can grasp with much certainty. We will only understand it fully with experimentation and time.

                                1. 4

                                  Notarization, decentralized storage

                                  There was quite a bit of tech predating Bitcoin that used hashchains with signatures or distributed checking. I just noted some here. So, people can build on tech like that, tech like whatever counts as a blockchain, tech like Bitcoin’s, and so on. Many options without jumping on the “blockchain” bandwagon.

                                  1. 1

                                    Well the advantage of a cryptocurrency blockchain vs the ones you cite is that:

                                    • you have a shared, “trustless”, appendable database including an ability to resolve version conflicts
                                    • the people who provide this database are compensated for doing so as part of the protocol

                                    A cryptocurrency blockchain has drawbacks, sure, but it’s not like it doesn’t bring anything to the table.

                                  2. 3

                                    Unfortunately, what you said can be applied to every emerging tech out there. See VR and AR. The difference is that VR and AR has found enterprise-y niches like furniture preview in your home or gaming. Likewise, crypto-currency has one main use case which is to act as a speculative tool for investors. Now, crypto currency’s main use case is becoming threatened from regulation on a national level (see China, South Korea). Naturally, it’s practicality is being called into question. No one can predict the future and say with 100% certainty that X tech will become the next internet. But, what we’re saying is that the tech did not live up to it’s hype and it’s pointless to continue speculating until block chain has found real use cases.

                                    1. 1

                                      Unfortunately, what you said can be applied to every emerging tech out there.

                                      Yes, probably.

                                      The difference is that VR and AR has found enterprise-y niches like furniture preview in your home or gaming.

                                      Personally I’m skeptical that furniture preview and gaming truly explore the limits of what these technologies can do for us.

                                      Likewise, crypto-currency has one main use case which is to act as a speculative tool for investors.

                                      I mean, right now you can send money electronically with it.

                                      Now, crypto currency’s main use case is becoming threatened from regulation on a national level (see China, South Korea).

                                      You seem to be saying that regulation is going to happen everywhere. How could you know that?

                                      No one can predict the future and say with 100% certainty that X tech will become the next internet.

                                      I’m not talking about the difference between 99% certainty and 100% certainty. My argument is that we don’t understand the technology because we haven’t finished experimenting with it, and it’s through experimentation that we learn about it.

                                      But, what we’re saying is that the tech did not live up to it’s hype

                                      The life of new technology isn’t in its hype - its in its potential, something which I think we haven’t uncovered. There’s tons of crazy ideas out there that have never even seen a mature implementation - programs running off prediction markets, programmable organizations, and decentralized lambda, storage, and compute.

                                      it’s pointless to continue speculating until block chain has found real use cases.

                                      Not sure what you mean by speculating - financially speculating? I’m not advocating for that. Perhaps you mean speculating in the sense of theorizing - in that case I think there is value in that since the “real use cases” that you are demanding only get discovered through experiment, which is driven by speculation.

                              2. 1

                                And if we shift now the debate on blockchain as a whole and not just bitcoin?

                            1. 1

                              My field is ‘search’ (not SEO, but real low level search tech).

                              If I had to put a big what’s next prediction related to my field - we’re almost on the cusp of something passing a domain-specific Turing test (chatbots are really a search problem). I say domain specific - because we’re not even close to a general chatbot turing test pass yet. But when restricted to a narrow field of content and context, it’s going to happen soon.

                              1. 2

                                (chatbots are really a search problem)

                                Could you explain what you mean here?

                                1. 3

                                  For the purpose of my comment, let’s say search is more or less synonymous with Information Retrieval. IR is the science of matching a query with data in a repository for returning the correct response to the query. A response is one or more pieces of data that is most relevant given the query context, intent, and substance. Chatbots extend this to a dialog of multiple query response pairs, and typically limits the response to either a single relevant answer, or a counter interrogative to gain more context, intent, or substance.

                                  Many times the context and intent are unknown (the substance is part of the query), so the search is two part: finding the context and intent during query analysis, and then using that along with the substance to find the best response.

                                  So take for example, the interrogative in an ecommerce chatbot: “what is the status of my order?” The context can be derived from the logged in user (the person asking the question), their data in the system, and the area of the site in which they are asking the question. The intent can be derived from the domain (this is important to my prediction) and query structure as ‘Order Status’. The substance is contributing to intent derivation in this query, as ‘status of my order’ can be an altLabel of the concept ‘order status’ (an important distinction for the prediction, because you have an ontology specific to the domain). In this example, when all three pieces are available, then you can structure the search to return an order (or list of orders) in your repository that are most relevant (relevant is likely recent unfulfilled orders). If you prefer you can have the bot ask a question to narrow the result set like ‘I see these three orders…which one are you interested in?’ and then fulfilling with a search based on the response to the follow up question.

                                  The context and intent are the difficult part to solve. Restricting to a domain narrows the possibilities for both.

                              1. 2

                                Per @whymarrh’s comment I agree there are separate issues being discussed here. So, regarding how bad the localStorage API supposedly is, I have a few objections:

                                you could serialize everything including data types into local storage, but that’s an ugly hack.

                                Is it? I mean, everything that gets stored in a file or database somewhere does get serialized. I don’t see serializing the data you’re throwing into your key value store is a hack or ugly. JSON.stringify is ridiculously easy to use.

                                It is synchronous. This means each local storage operation you run will be one-at-a-time. For complex applications this is a big no-no as it’ll slow down your app’s runtime.

                                I guess if you’re reading/writing localStorage a lot in a tight loop or something this could be an issue. But to act as if this is a problem with localStorage, when it could only be a problem in particular applications - well this just seems overstated to me.

                                Later in the article, the author states you shouldn’t use localStorage if you want your app to be “ultra high performance”. The localStorage API is sufficiently fast for very many use cases. If you are only doing a read or write here and there your user literally cannot detect it. I’ve worked in codebases where performance is important but never once found localStorage to be any kind of bottleneck.

                                Additionally the fact that it’s synchronous, while not ideal for every situation, makes it very easy to use, which is advantageous in many situations.

                                1. 4

                                  It’s very interesting that many of these comments are people claiming that what they do is actually simple - you just have to be smart / educated enough to understand it. I’m a believer that if the people reading your code can’t understand it then it’s you who’ve failed, not them.

                                  1. 7

                                    Doesn’t that depend a lot on who’s reading your code? I’ve been working with a few different Java codebases and sets of collaborators lately, and people seem to differ in whether they find some of the newer Java 8 features like streams to increase or decrease readability. In a lot of cases they take what used to be normally implemented with verbose boilerplate code, and add a language-level feature doing things in a somewhat more functional style.

                                    To some people this introduces too much fancy functional stuff that makes it hard to read. To others it makes the code easier to read by reducing the amount of verbosity and boilerplate you have to wrangle with. I’m a bit more in the 2nd camp: classic Java is so verbose that it’s hard for me to read. You basically have to be good at skimming for common patterns/idioms and able to “read” it in terms of these higher-level constructs, because reading it line-by-line you get bogged down immediately. I personally find it clearer and more readable if those constructs are just reified at the language level and I don’t have to keep a bag of common Java boilerplate in my head while reading code (e.g. write .filter() instead of a nested loop that I have to recognize as doing filtering), but others obviously disagree.

                                    So sure, I agree you should write code that people can read, but to do that, you first need to know who’s going to be reading it! There’s no code that’s universally more readable…

                                    1. 5

                                      Taking this perspective further, perhaps simplicity isn’t an innate property of code, but something that is relative to the community around it.

                                      1. 2

                                        Have you watched the talk Simple Made Easy? I’d be interested in hearing you thoughts with respect to those ideas.

                                        1. 3

                                          It has been on my list for a while, and prompted by your comment I finally did so, or at least I read the transcript :) Great talk - My favorite part was how he differentiates simple from easy. This leaves room for people’s ability to learn stuff. Often when people ask for simplicity it seems like they’re expressing a desire not to have to learn something. Which I think is a way of shortchanging oneself as well as one’s code.

                                          The central conceit of this talk is that complexity is the intertwining of parts. The more that parts of a program are intertwined the more complex it is. Perhaps there are other definitions we could give complexity, and they would probably have their own pluses and minuses. But this is the the one Hickey uses.

                                          Personally I think this is great part about this talk. Rather than using the word simplicity like everyone knows what it means, he comes up with an underlying principle of simplicity and explores the consequences of this principle.

                                          Some examples he gives of the intertwining that implies complexity: value and time, function and state, inheritance and types, what is going to be done and who is going to do it.

                                          Now in terms of whether complexity is relative to the community of people around a codebase, I think that position is still defensible, even accepting his central metaphor.

                                          For starters, How could we objectively determine the degree of intertwining in a program? Surely we must first identify what’s being intertwined. Regarding abstraction as a tool to fight complexity he says:

                                          I can’t totally explain how this is done. It’s really the job of designing, but one approach you can take is just to do who, what, when, where, why, and how. If you just go through those things and sort of look at everything you’re deciding to do and say, “What is the who aspect of this? What is the what aspect of it?” This can help you take stuff apart.

                                          In other words, during the design phase we can take stuff apart so that it’s not intertwined when we write our program. It’s interesting that he says he can’t explain how this is done, even though he gives some prompts to jump-start this process.

                                          Thinking about this right now, I have a hard time imagining that there aren’t endless ways to analyze a program into parts. And depending on these analyses, we are going to have different parts, different braids, different degrees of intertwining, and thus different evaluations of complexity.

                                          Surely there must be some significance to choosing some way of cutting up the program versus others? And surely some of these cut ups are going to yield more useful estimates of complexity than others, relative to how easy it is to change the code?

                                          Going further, it may be that intertwining time and value (for example) is more significant of a problem in some programs than others. And the significance of this problem, this complexity, must be relative to the technical team changing the code, as well as relative to the users of the program and the expectations they have of it.

                                          Another way to think about this:

                                          Let’s assume for the sake of argument that simplicity is intrinsic to a program. Hickey states that the value of simplicity is that it helps us to more reliably change programs. But the reliability of a program, its ability to do what it’s what’s expected to, depends, obviously, in part on the expectations about it. And these expectations are extrinsic to the program. For example, some programs are easier to change reliably merely by virtue of there being fewer expectations around them.

                                          Regarding whether simplicity is intrinsic to a program, he says:

                                          if something is interleaved or not, that’s sort of an objective thing

                                          Well I guess my position is that the concept of simplicity is objective, but only “sort of” objective, per the quote. Given certain expectations, and a certain way of cutting up the code to analyze complexity, the complexity we estimate is objective. But these expectations depend on the community, on what’s important to the community, and these analyses depend on how we decide to cut up the program so that we can identify complexity. Even granting his central conceit, which I like, I still think it must be right to say that complexity isn’t really intrinsic to the program itself, but is relative to the community around it.

                                    1. 4

                                      So after thinking, some thoughts on the theoretical Subleq ISA:

                                      • There’s no registers, and likely no cache - you’re pegged to the speed of memory access.

                                      • You likely won’t be able to get clocks up much higher than an x86, if at all; and your IPC is going to be very low because you can’t do a lot per cycle. I doubt you could pull off many optimization tricks either.

                                      • I don’t see how you could implement interrupts - you’re going to be polling.

                                      • It’ll be hard to manage anything like a stack… or transitioning from user to kernel mode.

                                      You could probably implement a small CPU that’s more effective, less radical, yet still comprehendible. Keep any design, CISC/RISC/whatever simple enough and it wouldn’t be hard to implement.

                                      Perhaps all this complexity exists for a good reason?

                                      1. 2

                                        Perhaps all this complexity exists for a good reason?

                                        Well, maybe not all :)

                                      1. 1

                                        How do you measure cohesion?

                                        1. 9

                                          smart contract coding is so difficult and unforgiving that even one of the primary developers of Ethereum can’t do it without losing hundreds of millions of dollars to human error.

                                          Crypto advocates need to take the next logical step, and admit the possibility that irreversibility, an essential design feature of cryptocurrency blockchains, is the fatal flaw of cryptocurrency that is responsible for most cryptocurrency and smart contract disasters. Pervasive irreversibility has turned out to be a bad and stupid idea.

                                          The argument presented rests on the premise that smart contract coding is too difficult and unforgiving. But surely it must be possible to produce a coding contract language that produces programs that are easier to reason about that the current spate of Ethereum contracts.

                                          1. 10

                                            https://blockstream.com/simplicity.pdf

                                            This is essentially exactly what I’d been imagining doing if I had infinite time to work on side projects. A transaction language that’s actually good from a language design standpoint, not just some acceptably serviceable DSL (bitcoin script) or a horrendous, clueless hack (solidity)

                                            Check out the paper, it’s pretty sweet.

                                            1. 3

                                              the language is so simple that I have no idea of what can be done with it. The only example given is a full adder.

                                              1. 8

                                                That’s not true. The end of the paper gives this example:

                                                The basic signature program that mimics Bitcoin’s basic signature program

                                                basicSigVerify b c := comp (pair (witness b)
                                                    (pair pubKey (comp (witness c) sighash)))
                                                    (comp (pair checkSig unit) (case fail unit))
                                                

                                                Other, more complex programs can be built to perform multi-signature checks, to implement sophisticated smart contracts such as zero-knowledge con- tingent payments, or to create novel smart contracts.

                                                It is possible that this example program is so succinctly expressed in the language that you missed it skimming the paper.

                                                We have written the SHA-256 block compression function in Simplicity. Us- ing 256-bit arithmetic, we have also constructed expressions for elliptic curve operations over the Secp256k1 curve [9] that Bitcoin uses, and we have built a form of ECDSA signature validation [23] in Simplicity.

                                                Not only have they been implemented, but using Coq and verified implementations of the algorithms, their implementation in Simplicity has been proven correct.

                                                Did you read section 4 where they extend the core language to add combinators that would be needed to make contracts work? Namely sighash, witness, assertl and assertr, and checksig.

                                                In addition, it is stated explicitly a number of times that Simplicity is not meant to be used directly, but should be the compiled target of a higher level language.

                                                Simplicity is designed as a low-level language interpreted by blockchain software. We anticipate higher-level languages will be used for development and compiled to Simplicity. […] For the time being, generating Simplicity with our Haskell and Coq libraries is possible.

                                                Also, from the theoretical perspective

                                                While Turing incomplete, Simplicity can express any finitary function, which we believe is enough to build useful “smart contracts” for blockchain applications.

                                                Soooo, yeah, it seems it can get where it needs to go within the domain of smart contracts.

                                                1. 3

                                                  thanks a lot, totally missed this part

                                              2. 4

                                                That paper was, indeed, very sweet!

                                              3. 4

                                                I think about a year ago I was quite active in one of the Ethereum gitter channels.

                                                It’s possible yes. You don’t even need provers or anything fancy. I reasoned that a simple state machine is everything you need for most contracts. And state machines, especially deterministic, finite ones are easy to reason about. You can prove things about them, like that they never loop endlessly or that they are the most efficient and smallest implementation.

                                                I lacked the expertise to write an actual compiler but I think that might be a good direction in which to develop smart contracts. But, I wrote some documents and I think I have a small pseudo code around in which a compiler for this language could tell wether a contract can get stuck. Ever.

                                                On the other hand, DFSM’s are very simple machines and aren’t well suited for making complex things. They lack the complexity.

                                                1. 3

                                                  what are the reasoning you can have about simple state machine ?

                                                  1. 6

                                                    You can statically determine that from each state, the final/exit state is reachable. You can also determine that there are no loops in the machine that last forever (but it’s more complicated). You can also prove that the given state machine is the simplest (fewest states) machine possible and if not, optimize it to such.

                                                    Lastly, DFSM’s are very simple constructs that don’t really have a concept of anything outside themselves, in that regard they are very close to purely functional languages and you can build a lot of easy compiler checks (types) that makes the machine safer.

                                                    Compared to doing the same with proper, turing complete languages, you have less work and more, mathematically proven, securities.

                                                2. 4

                                                  Nick Szabo proposed a formal example language that wasn’t Turing-complete (that I can see).

                                                  The trouble is, Ethereum wanted uptake, so they leveraged “worse is better”. This resulted in Solidity being the language people actually use for smart contracts, on the platform people actually use for smart contracts.

                                                  The results are … less than great, given how utterly unforgiving immutable smart contracts are.

                                                  Thankfully they’re proposing various new EVM languages that are less of a horror show. But it’s still overwhelmingly Solidity. You can propose things that will theoretically work better - and many are obvious - but then you need to get anyone to use them. Solidity coders in practice don’t even follow the many official guidelines to not making their code into Swiss cheese.

                                                  Even then, I’m not convinced any human can program well enough to deal with smart contracts in an immutable environment, unless they’re coding like NASA for spacecraft. When not even Gavin Wood can write a smart contract that doesn’t proceed to lose him literally tens of millions of dollars, we might have a problem here.

                                                1. 2

                                                  I thought this was a great introduction to contemporary metaphysics. If you read this and didn’t freak out like hwayne but actually found it interesting and thought-provoking, I recommend taking a look at Deleuze’s writings.

                                                  Deleuze was probably the greatest metaphysician in recent memory. Heidegger said that Nietzsche’s statement that “god is dead and we have killed him” means the end of metaphysics. But Deleuze’s response was simple: if the old metaphysics was wrong, we should build a new one. This new metaphysics that Deleuze started building incorporates 20C mathematics and physics, nonlinear dynamics and chaos theory, probability and self-organizing systems, computability and Turing’s ideas on ordinal logics, and lots of graph theory (e.g. the “rhizome” concept from Capitalism and Schizophrenia).

                                                  Contemporary metaphysics is very similar to yet much more thorough than this article, but it is all very interesting. If it sounds interesting to you then I recommend exercising the priciple of charity and picking up some of Deleuze’s early work, Nietzsche and Philosophy is a good one to start with. Difference and Repetition is where the metaphysics gets really developed.

                                                  Edit: What is Philosophy? by Deleuze and Guattari is another good book to start with. Also I added links.

                                                  1. 3

                                                    Okay, now that I’m a bit more coherent I’d like to address one of the big things that bothers me about his argument. He says “the universe lacks structure” and everything seems so ordered (to some definition of order) because we ignore all of the messy parts of physics. The problem is that his argument is based on the idea that the fewer axioms properties the system has, the harder it is to understand. But for the vast majority of physics research, the opposite is true: the basics are, for the most part, pretty easy to model. The problems start not when you take stuff away, but when you add it.

                                                    The archetypical example is the N-body problem. Given two masses exerting gravitational pull on each other, how do they move over time? Not only is there a perfect analytic solution, you could assign deriving it as a high-school homework problem. How about three masses? Turns out a solution is impossible. But that change isn’t driven by any sort of fundamental shift in physics. It’s the same laws of physics in both situations. Adding that extra body just makes things too complicated to handle.

                                                    Obviously there are exceptions, and fairly big ones. But when we say that the universe seems ordered, what we mean is that everything obeys our current understanding of physics. A few things don’t, not because they have different physics but because there are gaps in our understanding. And many, many things do but are incomprehensible to us, because even a few particles give rise to complex emergent behavior, let alone untold nonillions.

                                                    1. 2

                                                      The problems start not when you take stuff away, but when you add it.

                                                      If that’s true couldn’t it be possible to just invert this argument and come up with a similar conclusion? For example, the universe has a high amount of complexity (stuff added to it) but “physics” is our limited, simplified understanding of this greater complexity (such that physics arises insofar as we subtract complexity that is truly present in reality).

                                                      Edit/disclaimer: I don’t have a strong opinion either way. I just think this is interesting to think about :)

                                                      1. 2

                                                        The idea isn’t that “we ignore the messy parts of physics.” It is this: we introduce structure in the very act of interpreting our observations of the universe. Interpretation in science necessarily involves partitioning the world into smaller parts - the science of chemistry only studies interactions between chemicals, for example. And in chemistry there is a certain structure to how things work that isn’t as prevalent in e.g. particle physics. We could use physics to do all of our chemistry, but that would be difficult, so we use this higher level of abstraction to interpret chemical interactions because it allows us a more convenient structure. The point is that, by creating chemical formulas and partitioning chemistry from physics, we have introduced structure into the world by way of our interpretation. This is generally true: the world is chaotic, we introduce orderly structure in our interpretations because it is easier for us to understand and make predictions with.

                                                    1. 12

                                                      I’m sure I’m gonna get flamed to death for this, but I’m against this law. Intelligence services need to collect intelligence and conduct espionage against our nation’s enemies.

                                                      In order to do that, they need to have tools. Exploits are tools.

                                                      1. 15

                                                        I’m in favour of disarmament and to stop seeing the world as being full of “enemies”.

                                                        There’s no reason why governments should be holding secrets, if we are to believe that governments are there by consent of the governed. How can we consent to things we don’t know about?

                                                        1. 4

                                                          Most of the countries in the West have spy agencies or people collecting competitive intelligence. They still compete even if peacefully. They’re not going to stop. If a country gives up its tools/techniques, then the others just get free wins on them. So, the country must keep doing that stuff to stay competitive and protect its citizens’ interests.

                                                          1. 8

                                                            So, who are those enemies? Germans? Mexicans? Russians? The Arab world? The whole world? I’m Mexican. Am I one of your enemies?

                                                            Why does the US have such a large army and why must this army be maintained? What would happen to the use without its army? What would the enemies do? Steal all their jobs? Invade the homeland? Drop a nuke on them? Destroy their intranets, ransom their data?

                                                            1. 3

                                                              The enemies mainly try to compete with us economically by stealing our IP or attempting to scheme in international negotiations or contacts. One part of government routinely catches contract rigging with help of spy agencies. They also steal military secrets. Some turn computers into bots to facilitate real life and online crime. Some want to straight up disrupt our infrastructure randomly or at key points for political reasons.

                                                              These are the enemies Im talking about. They and US have been duking it out long time with some before Internet. Gotta stay ahead. In your country, Id worry about the cartels that are such a strong threat to government that even Mexican military can stop them. Their gangs are disrupting US, too, esp on West side.

                                                              1. 5

                                                                The enemies mainly try to compete with us economically

                                                                Slightly tangential, but regarding international relations I sometimes wonder if “American interests” isn’t actually euphemism for “the interests of American corporations”, which is not really the same thing as “the interests of Americans”.

                                                                1. 3

                                                                  BINGO! You’re thinking in the right direction. I thought that initially but it’s more complex. Americans like all their content on TV (esp celebrities & artists), free services, and plentiful goods made by cheap labor here (illegal immigrants) and elsewhere (low-cost labor offshore). The vote with their wallet on these things and their political votes usually don’t care since the elected officials will continue it. So, they keep benefiting from those forms of corruption that the electorate keeps going. I think we might be able to say those are the interests of actual Americans instead of just corporations since it benefits Americans, they support it financially, and don’t want it to change.

                                                                  Then, there’s all the other stuff that’s solely about rewarding a special interest or corporation that paid politicians good money. You can usually tell in any country if the citizens are losing something, a tiny few are gaining something, the citizens didn’t ask for that, and the few paid off the people moving stuff around on the ledger. The fundamental mechanism of corruption. The Goldman Sachs response to the 2008 financial crisis (after creating it) is one of best examples in U.S. history: a trillion dollars no questions asked with criminal immunity negotiated by a regulator that was ex-CEO. It doesn’t get better than that for bad folks.

                                                                  1. 2

                                                                    Spoiler alert: Politicians’ constituents are actually corporations, not citizens.

                                                              2. 2

                                                                You can apply this competitive frame to almost anything, with the result that things become worse for everyone. You say “they’re not going to stop,” but competing nations have implemented policies that make them less competitive, such as labor protections. There’s no reason government transparency couldn’t gain traction in a similar way, especially if the U.S. took the lead.

                                                                1. 1

                                                                  If a country gives up its tools/techniques, then the others just get free wins on them.

                                                                  The whole point of requiring disclosures is to improve our own defensive capabilities. Others don’t get “free wins” if the disclosures lead to patches and increased security.

                                                                  1. 1

                                                                    I see your goal but don’t buy the method to achieve it. There’s simply too many vulnerabilities in these systems for it to be a meaningful difference. The attack crews of nation-states always have vulnerabilities in popular products. That’s despite bug hunters regularly disclosing vulnerabilities for patches. So, one side disclosing what it finds won’t meaningfully increase security if it’s a large system constantly in flux in unsafe language made by vendor who doesn’t care about security. The problem is the vendor’s development model. That’s what you fix with law. Then, we can talk about the benefit of disclosure.

                                                                    Even then, if disclosure is mandated, groups like NSA will request an exemption on national security grounds, not look for bugs in the first place since they can’t use them, or lie about not hoarding them with criminal immunity as always. I mean, we’re talking about rogue-ish, spy agencies here.

                                                                2. 3

                                                                  Way back when, when I was in college, there was a guy going around on a bike, grabbing girls butts, then riding away. This being college, the Serial Groper was cause for widespread panic. Something needed to be done, and so the police increased night patrols in the target areas.

                                                                  Now in a completely transparent society, this would have meant publicly posting “Tuesday night officer Jim will patrol up and down Maple street, and on Wednesday night he will patrol Cherry st, and etc.” After all, as an affected citizen, don’t I have the right to know what the police are up to? We surely don’t want secret police patrols.

                                                                  Of course, such disclosure makes it easy for the groper to avoid arrest. Just the price of a free society, no?

                                                                  1. [Comment removed by author]

                                                                    1. 2

                                                                      The premise was that a government should have no secrets. Not some secrets, no secrets. There’s a reasonable discussion one can have regarding which secrets there should be, but that’s difficult from a starting position of absolutely no secrets.

                                                                      1. [Comment removed by author]

                                                                        1. 3

                                                                          Sorry, the point wasn’t to draw such a direct comparison, but to establish a baseline that complete transparency is often counterproductive. Maybe I’m jaded from previous debates with people who insisted that every government document and email be made public immediately. The example was meant to be trivial and obvious, so as to avoid further side debate. (Was going to use bank robbery, but didn’t want to debate the ethics of the bank system, etc.)

                                                                          1. 1

                                                                            You framed it as a sexual harassment issue

                                                                            The parent to his remark said governments should have no secrets at all. That there was no reason for them to. The framing of “no secrets” with a sexual harassment issue is correct in that he illustrates secrets have their place to stop harm to citizens. It was also an easy to understand example for about any type of person reading. Whether we should have secrets that are mass exploitable with ease to cause same harm to everyone is another point of discussion that you’re bringing up. It wouldn’t fit his framing but that’s not what he was responding to.

                                                                        2. 2

                                                                          Please try to refrain from posting in this way. You had some good points but it is hard to see them past your hysterics.

                                                                          1. [Comment removed by author]

                                                                            1. 1

                                                                              Being triggered is no excuse for acting dickish–in such cases, removing yourself from the offending stimulus is usually the best option.

                                                                          2. 1

                                                                            Not trying to absolve the NSA (they couldn’t protect their tools, they definitely have responsibility), but Microsoft sells their OS with an expectation of security and it failed.

                                                                            In a different universe the NSA would have reported this to MSFT before the leak, and this wouldn’t have happened. We should aim for that. But there’s also another universe in which MSFT is more careful.

                                                                            It is absolutely not an inevitability that there are worm-generating bugs in Windows. We build skyscrapers that don’t fall over all the time.

                                                                            We should work on getting the NSA to get awful exploits like this fixed fast. We should have higher standards for OS issues. We should have people be able to stay updated on security releases. We should attack all angles to make this not happen again. A lot of things aligned for this to happen.

                                                                            1. 1

                                                                              It is absolutely not an inevitability that there are worm-generating bugs in Windows. We build skyscrapers that don’t fall over all the time.

                                                                              Based on past data, I can conclude that making an OS that does not have bugs is much harder than building a skyscrapers that don’t fall over.

                                                                              1. 1

                                                                                If you keep the TCB small, it cost several million over a few years for the TCB w/ the rest done in Design-by-Contract in safish language for 10-50% more. The very first one, GEMSOS, cost $15 million with the most primitive tools with R&D costs included (aka cost of partly inventing INFOSEC). seL4 was just over $2 million. A small team at Microsoft did VerveOS. Another small team did ExpressOS using same tools. These two were safe rather than secure but design/code safety is a prerequisite knocking out most code injection. Hypervisor from DeepSpec was 10+ people over a few years.

                                                                                We occasionally have to deal with new classes of vulnerabilities that are totally unrelated to existing, root causes. Hard to recall when that required total rewrites rather than a fix outside of the software. It’s hard to do a secure TCB but skyscrapers are way harder. Apathy is what stops these big companies. I mean, Microsoft could license VerveOS or Midori to those interested in building on advantages they already had. They could use that stuff internally more than they do. They just don’t care and pay politicians good money to protect their financial security. :)

                                                                            2. 1

                                                                              but exploits like used in WCRY that are wormable, FUCK NO! This is putting everyone at risk.

                                                                              Although I see the point Ted was making, the wormable nature of this exploit does make the comparison meaningless. The actual comparison to these vulnerabilities would be, say, a self-destruct button that could wreck hospitals, banks, military, or logistics. Or some subset of their operation. There’s a phone number to call to get someone to push it without asking questions. The intelligence agencies have the number, say it’s useful, and their uses justify not pulling the plug on that line or disabling that button. Now, some people have the number. They’re using it for evil.

                                                                              And so on and so forth.

                                                                            3. 0

                                                                              In a free society the girls would just carry firearms and defend themselves from physical violation with necessary force.

                                                                              What has happened instead that self-defense was outsourced to a third party who is extremely ineffective and inefficient, so it became impossible for them to work without being given MORE POWER by having the ability to now not only have monopoly on violence but also asymmetric advantage on information. So the solution to the failure of government is once again more government in a run away cycle.

                                                                              As the government gains more power, the sociopaths make their way to the top of the pyramid and we end up with comrade Stalin or dear leader.

                                                                              In the free society scenario, not only was there no need for secret, the girls would want to advertise the fact that some of them walk around with firearms with which they will defend themselves when necessary.

                                                                              1. 7

                                                                                To be clear, you’re saying that the preferred response to someone grabbing your ass without your consent is to shoot, and probably kill, the person doing it?

                                                                                1. 2

                                                                                  I like how you glossed over the phrase “necessary force” to go straight to “preferred response… is to shoot.” That’s not what LibertarianLlama said. Either you’ve seen that commenter say they prefer all small crimes resulting in killing the criminal or you’re assuming a pro-gun stance has a preference for killing. Regardless, most gun owners I know are taught to use gun as deterrence by warning an attacker, optionally a warning shot depending on what risk defender wants to take, and lethal use only if absolutely necessary. Many also carry pepper spray, stay out of risky situations, and so on to reduce odds of them having to kill people. Most people with guns, whether they like them or carry them out of necessity, don’t want to experience the risk of a violent encounter.

                                                                                  At least, that’s my experience reading surveys of gun owners across U.S. and listening to over a hundred of them in the South where people love guns. It’s something good to have but terrible to have to use. Due to the high occurrence of robbery and rape in our area, our family teaches everyone self-defense and proper use of weapons. Almost every one of us have defended ourself from an attacker. A proper defense we see as a necessity because we can’t trust people to always be good to us. Human nature…

                                                                                  1. 3

                                                                                    Fair enough. My brain did admittedly jump to the assumption that “defending themselves with necessary force” involved actually using the weapon he suggested they carry, instead of just threatening to use it against him.

                                                                                    There’s an argument to be made, though, that a gun may actually be less effective compared to, say, a non-lethal weapon such as the pepper spray you describe or a tazer, in a situation such as the one we’re talking about (where the criminal was awful but mostly non-violent). After all, if you’re so hesitant to use your weapon, your threats hold a lot less weight. And even you seem to agree that actually using the gun against him, would be a disproportionate use of force, and a court making a self-defence ruling would probably agree.

                                                                                    1. 1

                                                                                      Oh yeah. I encorage carrying a non-lethal for that reason. Also, in some cases, even shouting what they did aloud when they do it can be punishment or deterrent enough via shaming. Still good to have a lethal in the uncommon case that they escalate to violence with pepper spray not stopping them (they grab person) or among 5% (1 out of 20) immune to it. Some number like that anyway…

                                                                                      1. 1

                                                                                        After all, if you’re so hesitant to use your weapon, your threats hold a lot less weight.

                                                                                        This is such a perverse argument. So what you are saying is that criminals are more likely to attack you when you are carrying a rifle than when you carry a stick because the criminals will think that you won’t use your firearm because you will be hesitant?

                                                                                        1. 1

                                                                                          Not a stick (nice strawman there), but something like a tazer and or can of pepper spray.

                                                                                          And the answer is yes, in some cases. Not everyone is a badass who can convincingly threaten to fearlessly take a human life. To paint a stereotypical example, a trembling, inexperienced 18-year-old girl can easily be a more convincing threat with a tazer or other effective non-lethal weapon than shakily attempting to point a gun at someone.

                                                                                          (On a sidenote, the gun also escalates the level of violence far beyond that of the initial confrontation, which has its own problems. But I’d rather not get into that whole can of worms right now.)

                                                                              2. 1

                                                                                This is an ideal we should all work towards, but from where I sit human nature as it exists today makes such an ideal impossible to realize.

                                                                              3. 8

                                                                                It’s that or backdoors. If backdoors, they’re going to up their power and/or violence. Vulnerability hoarding is the lesser evil. It’s barely unethical, too, given market votes against strong security almost every time, suppliers dont give a damn, and government doesnt either due to bribes, internal politics, & lockin.

                                                                                1. 2

                                                                                  I totally agree. Backdoors weaken the armor for everyone, friend and foe alike.

                                                                                  1. 2

                                                                                    Yeah, but if you rely on exploits then you are betting on “our team” finding more exploits or quicker than “the other team”.
                                                                                    On the otherhand if you implement backdoors then you need to:
                                                                                    a) have the power to compel companies to add them and give you access, what happens when it is an over seas company?
                                                                                    b) ensure “the other team” isn’t given access or steals access to the backdoors.

                                                                                  2. [Comment from banned user removed]

                                                                                    1. 2

                                                                                      Ok, so did we just win on net neutrality given most Americans don’t want FCC trading them for a few companies? Or did the powerful few overrule the apathetic or powerless masses? And what about Patriot Act, software liability, etc? What I see in reality rather than fantasy world you’re describing is a bunch of conflicting interests we have to work through to get a compromise on what we want.

                                                                                      Im picking the choice that allows them to operate on the careless or apathetic but market, academics, or FOSS can continue to make strong security. As Schneier is doing recently, we can then continue to argue against increased laws by showing they have what they need right now. Then, people wanting SIGINT and people wanting security can continue receiving both.

                                                                                      1. 1

                                                                                        Do you disagree that we all live in a society in which the rule of law and some kind of government is probably a necessity?

                                                                                        If you do, then having some kind of intelligence service and conducting espionage against other such entities is a fact of life. This isn’t about the choices you and I make personally, it’s about the opinions we hold and the way we think our society should be run.

                                                                                    2. [Comment from banned user removed]

                                                                                      1. 7

                                                                                        I would argue that choosing to deny the pragmatic fact that the intelligence community is an important part of maintaining national security is also insane. So I guess we can all wander off to the loony bin together, eh ? :)

                                                                                        1. 4

                                                                                          I’m pretty sure people are choosing to learn the wrong things from wannacry by ignoring the inconvenient parts of the timeline. The NSA did tell MS about it, and MS did issue a patch, and then there were news stories about how important the patch was because it fixed a serious vulnerability, and still the hospitals decided not to patch.

                                                                                          It’s hard to see how any disclosure policy would have changed that. The NSA had their hand forced, but the patch was nevertheless out for two months. If the NSA had decided to disclose the vuln in 2014 out of the goodness of their hearts, does anybody think that the hospitals response would be any different?

                                                                                          1. 1

                                                                                            and still the hospitals decided not to patch.

                                                                                            It’s hard to see how any disclosure policy would have changed that.

                                                                                            The problem is clearly that they were not forced to patch by something external. Regulations or court liability are historically what accomplishes this. I lean toward former because latter becomes lawyers playground. Look at healthcare suits in the U.S. right now. All kinds of people rolling around in money-making schemes. Maybe the socialists might have it better with court-based liability for damages done if people are less sue-happy over there. I hear we’re the top place for such bullshit but I’m not sure.

                                                                                            1. 3

                                                                                              I guess the bitcoin ransom is something external forcing them to patch. Though the fine is pretty low to have much effect on a large institution, clearly the libertarian free market has provided an essential service that the statists can’t effect with all their enslaving and thieving. Another victory for crymetocurrencies! /s

                                                                                              1. 1

                                                                                                You may be facetious but honestly sometimes people just have to get burned.

                                                                                                1. 1

                                                                                                  Lol. Great example of where liability on market or criminal side ain’t exactly beneficial. Loved your original comment btw. I gotta drop it on a libertarian crowd at some point. They’ll probably just start talking Social Darwinism or something, though.

                                                                                            2. 3

                                                                                              You’re argument also supports banning software such as Windows in all such critical cases unless they can demonstrate either no vulnerabilities or strong posture of security w/ easy way to update or recover. These kinds of systems existed under DOD regulations before. Some still exist in niche sectors. However, telling intelligence agencies not to do it means it will still get done by non-cooperative agencies, black hats, and so on. The root problem is there’s no requirement to eliminate or liability for preventable defects in software. That’s should be the focus.

                                                                                              1. 3

                                                                                                There is far more risk to our nation as a result of these exploits than to any enemies of ours

                                                                                                This isn’t even remotely true. They use the same systems we do, just look at this heatmap about the same vulnerability you are talking about. Having known many intelligence people over the years I guarantee you that those people were not just sitting on an exploit for years, they were using it. I’m far from the patriot type at all, but I view this more as arms stockpiling than anything else.

                                                                                                Also the moment you allow yourself to start name calling there is a good chance that your guilty of an Ad Hominem fallacy. It does not strengthen an argument or encourage meaningful discussion. It also lowers the quality of the threads here.

                                                                                            1. 12

                                                                                              Yeah, it looks to be a bunch of BS. Better to use existing law to make a group of non-profits in a number of countries that use traditional, banking techniques with logs, decentralized checking, and some kind of corrective mechanisms. Bankers can probably already tell how to do most of that. It’s all ancient technology. For the currency end, high-assurance engineer Clive Robinson always said just tie one to the value of all kinds of useful commodities or stable currencies. There was in fact one that did it although I can’t remember its name. So, efficient databases run by non-profits chartered to not nickle and dime the customers with distributed checks and optionally a currency-like instrument tied to stable, diverse, real-world commodities/currencies.

                                                                                              EDIT: Most objections end up about what the people running the banks or governments might do. Those risks still exist for Bitcoin with a tiny number of individuals and miners having massive influence. Exchanges getting robbed all the time. Much worse than situation for traditional banking.

                                                                                              1. 4

                                                                                                Yes, if governments allowed ultra-low-overhead user-friendly automatable money transfer, that would decrease the usefulness of cryptocurrencies somewhat. But they don’t, not in the first world. Look up KYC/AML laws. A notable third-world exception is mpesa, which has done very well.

                                                                                                Those risks still exist for Bitcoin with a tiny number of individuals and miners having massive influence.

                                                                                                Their incentives are aligned with the users of Bitcoin. Miners do well if Bitcoin prospers. The same is not true with the government and my finances. The government does well if they maximize the amount they take from asset holders. Have you ever had your bank account frozen? I have, due to a paperwork error by the state comptroller. That can’t happen with Bitcoin. It’s why I started using it, actually.

                                                                                                Exchanges getting robbed all the time.

                                                                                                That’s why only idiots keep their money in exchanges. This is a non-issue. A big part of the whole point of Bitcoin is that you actually control your assets in a very literal sense, unlike with a bank.

                                                                                                Much worse than situation for traditional banking.

                                                                                                In what way? It’s cheaper, easier, more flexible, better uptime, etc. etc. The only reason I use dollars is that not everyone takes Bitcoin, and dollars have lower short-term volatility. If you know WTF you’re doing, most of your assets won’t be in dollars anyway, so it’s not like the stability of the dollar is a huge benefit when you’re (hopefully) only holding them for a short time anyway.

                                                                                                1. 3

                                                                                                  So, efficient databases run by non-profits chartered to not nickle and dime the customers with distributed checks and optionally a currency-like instrument tied to stable, diverse, real-world commodities/currencies.

                                                                                                  This is pretty much what blockchain technology is though? It’s trying to be an efficient distributed and decentralized database. If postgres released a plugin that let it be distributed and decentralized would you call that a bunch of BS?

                                                                                                  Why is blockchain tech more BS than any other technology? Are you claiming that it doesn’t work, that it can’t scale, that there is any fundamental flaw with the design? Because otherwise I don’t see how you can call a database BS.

                                                                                                  Everyone here seems to railing against blockchain technology because they disagree with some particular use of it, not because there is anything inherently wrong about the technology itself. We all want to make distributed computing easier, blockchain technology aims to do that, but everyone is saying it’s “so complicated”; that’s because distributed computing is complicated and we, the programming world, are still trying to find good solutions to the problems, I personally think blockchain tech is one of the more promising approaches to solve the problems.

                                                                                                  1. 5

                                                                                                    “ It’s trying to be an efficient distributed and decentralized database. If postgres released a plugin that let it be distributed and decentralized would you call that a bunch of BS?”

                                                                                                    That is quite a strawman. Blockchain is not a distributed postgres. Here’s some key features of popular, blockchain tech that wouldn’t happen in my model built on actual databases:

                                                                                                    1. Energy consumption of miners to create the money. My model either uses existing currencies or commodities or instruments priced against them. The creation aspect takes either nothing or calculations one computer could handle. Also, the current model increases odds that a given currency will become a pyramid scheme to shift most of the wealth to its creators. The mining model increases odds an oligopoly will form as difficulty goes up. Both happened in Bitcoin’s design.

                                                                                                    2. Commit costs are much higher than traditional, strongly-consistent databases. My MasterCard can do a transaction in one second. The bank might delay it further for some analysis. The network itself handles 30,000 a second, though. Blockchains don’t by design so far.

                                                                                                    3. Longevity. Distributed, OSS databases + nonprofits doing at least breakeven + currencies or commodities people already want is much more likely to last over time than startup model around blockchains.

                                                                                                    4. Trustworthiness. Theres several currencies that are very stable, well-managed, and stored in banks with good security. Leveraging that gives quite a headstart on secure, stable banking. The blockchain currencies haven’t been stable or secure.

                                                                                                    These are four examples where traditional tech and legal instruments are advantageous over blockchains. The mining cost, performance disadvantages, extra pyramid schemes, oligopoly pressures, and insecure exchanges all make me call BS on blockchains being the “solution” to the problems with ordinary currency and banking. So far, it’s added more problems than it solves.

                                                                                                    1. 6
                                                                                                      1. There is nothing inherent to blockchain that requires “mining”, you need transaction confirmation which is part of any consensus algorithm, it doesn’t matter if you use Ethereum or Raft or anything else, if you want consensus you need some inter-node communication.

                                                                                                      2. There’s nothing inherent in blockchain design that can’t do 30k TPS. I grant you that that speed isn’t there in most implementations yet, but there’s definitely people working on it, and condemning a technology that’s a few years old because it hasn’t had the performance tuning at the same scale as MasterCard is quite a premature condemnation. Here’s an example of a blockchain implementation that can do 8k TPS http://kadena.io/

                                                                                                      3. What? There’s nothing in the blockchain technology that’s owned by anyone?

                                                                                                      4. I’m not even talking about currencies at all.

                                                                                                      Your beef is with Bitcoin, not with anything to do with blockchain tech. Git is basically built on blockchain tech, are you saying Git is BS too?

                                                                                                      Replace “blockchain” in all your posts with “bitcoin” and I can agree with you.

                                                                                                      1. 1
                                                                                                        1. I believe it. Most Ive seen involve weird schemes for covering costs that could lead go inefficiencies, attacks, and so on. My model is simple: people pay for an account and/or companies providing service get fees for what they do. Proven model. Keeps the tech protocols simple, too. What’s the simplest, payment-oriented blockchain you know of without stuff like mining? Or even a list of them.

                                                                                                        2. Visa isnt hitting 30,000 a second due to massive optimizations: they’re using 1970’s technology throwing money at CPU’s and memory. Modern ones can do 1+ mil a second on under $100k of servers. Spanner does a crapload with strong-ish consistency on geographically separated servers but with 30s pause if you want ordering guarantee when availability takes a hit. Blockchains needing lots of optimizations to reach 1970’s mainframe performance on 2017 servers is a strike against them.

                                                                                                        3. I said longevity, not ownership. Lots of stuff coming and going with other stuff volatile in price. This is a problem dpecific to startups in general but happening extra for those using own currency.

                                                                                                        4. My bad then. Ill just stsy on blockchain points.

                                                                                                        Re Git. There are piles of threads about its problems on the Internet. It’s useful and even necessary to participate in a lot of FOSS. I certainly would argue it could be more efficient, available, and secure in its design. It was a good Worse is Better example in how it got popular.

                                                                                                        Re Bitcoin. Yeah, it’s the worst of them Ive seen. Glad we agree on af least that.

                                                                                                  2. 3

                                                                                                    this is all well and good for folks privileged enough to live in countries with benevolent, effective governments, but for those living under unjust regimes, distributed digital currency is a game changer.

                                                                                                    1. 5

                                                                                                      Currency unions seem to be a bad idea for the poorer countries. For example, Greece is unable to devalue their currency because they have the do Euro.

                                                                                                      1. 7

                                                                                                        These currencies are also a game-changer for criminals who have been driven near to extinction benevolent, effective governments.

                                                                                                        1. 6

                                                                                                          “Think of the criminals/terrorists/drug dealers/etc.”

                                                                                                          These arguments aren’t effective anymore. People are innoculated to them after they were repeated ad nauseam during the “war on drugs” and “global war on terror”. The arguments are just as hollow now as they were in the 70s.

                                                                                                          I’m also curious what makes you think that criminals were “driven near to extinction” and, beyond that, are suddenly resurging thanks to new currencies. Trends in criminology data support neither of those claims. Crime rates have been decreasing gradually, and they are still doing so.

                                                                                                          1. 1

                                                                                                            I’m actually more concerned by money launderers and corrupt government officials. The latter is indeed almost gone in the west, when was the last time you heard of someone having to slip the clerk a twenty under the table to get a driver’s license?

                                                                                                            1. 3

                                                                                                              The latter is indeed almost gone in the west

                                                                                                              Money seems to control our political system at the highest levels, even if DMV clerks rarely receive bribes.

                                                                                                              1. 1

                                                                                                                I’m not inherently concerned by money launderers; “money laundering” (attempting to anonymize ownership) does not, in and of itself, hurt anyone. It’s a victimless crime. The only reason it’s illegal is that it’s easier to prosecute than whatever the target might actually be doing with the laundered money.

                                                                                                                However, if you are concerned with money laundering, you shouldn’t be afraid of Bitcoin; multi-billion-dollar crime syndicates just use HSBC.

                                                                                                            2. 1

                                                                                                              There are no criminal people, only acts which may be labeled as​ criminal during some time period.

                                                                                                              1. 0

                                                                                                                I will note that most of my personal, hard earned, ethically earned, legally earned capital was eaten by an effective government’s currency controls.

                                                                                                                The criminals in charge of that effective government had “legal” means not accessible to the person in the street to freely move currency around.

                                                                                                                Strangely enough, currency, no matter how rotten the origin, is gladly welcome in all western countries.

                                                                                                                Equally strangely, there tends to be some pretty amazing hurdles to cross for currency to leave…

                                                                                                                I must admit my belief in the benevolence of some governments has been tainted.

                                                                                                                Their bend over backwards willingness to accept and turn a blind eye to tainted assets coming in, is matched only by their reluctance to let it leave.

                                                                                                              2. 1

                                                                                                                Those countries tend to pass laws or otherwise take action to disrupt anything effecting their control. In a lot of them, US dollars or commodities like gold are very valuable vs their own currency. My scheme is a digital version of those whose banks will be in countries like Switzerland with paper or mobile methods of using the service.

                                                                                                                That’s actually less risky than currency or banking in a poorly-governed area.

                                                                                                            1. 7

                                                                                                              I agree with the article. In fact, my article on slide design tells you pretty much the same.

                                                                                                              One additional point: Use only one font. Maybe two if you can justify it.

                                                                                                              1. 1

                                                                                                                I like your article, although the left-alignment in its CSS seems strange to me because the text is all collapsed to the left 50% of the page (for me).

                                                                                                              1. 16

                                                                                                                What would be more interesting is a list of ways in which it differs from the heaps of existing open-source federated social networks.

                                                                                                                1. 18

                                                                                                                  The key difference right now is people are using it, or at least have started to use it in the last few days. So if I were to draw a Venn diagram of “people I follow on twitter” and “people who have mastodon accounts”, it wouldn’t just be two distinct circles, which I can’t say for any of the other federated networks.

                                                                                                                  Whether they’re all still using it next week, well, we can be optimistic.

                                                                                                                  1. 8

                                                                                                                    I suspect part of it is that the people running Mastodon instances tend to have specific anti-harassment policies at a time when Twitter is getting a lot of flak for ignoring their harassment problems.

                                                                                                                    Icosahedron (a Mastodon instance) specifically calls out that “Fascism is incompatible with a free exchange of ideas”: https://icosahedron.website/about/more which is a breath of fresh air after the way Twitter has been avoiding admitting there’s even anything wrong.

                                                                                                                    I have a gut feeling that enforcing such policies would be easier on a federated network.

                                                                                                                    1. 8

                                                                                                                      Not sure how much of a factor it is, but I did notice that the majority of the instances (including all the big ones) are run by either Germans or French, which provides a different cultural and legal background compared to American-run services like Twitter. For example, whether to allow overt Nazism isn’t even really a debate in the German or French context, because it’s illegal.

                                                                                                                      1. 3

                                                                                                                        Disallowing ideas seems a bit less compatible with a free exchange of ideas to me.

                                                                                                                        1. 4

                                                                                                                          you’re free to think that, doesn’t mean you’re right though, and I guess with Mastadon you’re also free to pick a host that agrees with you rather than being up to the mercy of a totally centralized model.

                                                                                                                      2. 1

                                                                                                                        or at least have started to use it in the last few days.

                                                                                                                        Any idea why that is?

                                                                                                                        I’ve had an account for ages, but I’m seeing Mastodon everywhere today and can’t figure out why it’s suddenly a thing.

                                                                                                                        1. 3

                                                                                                                          Twitter changed how replies work and that’s became a sort of last straw for some people. Some high(ish) profile folk tweeted they had made mastodon accounts, others followed, and it’s gained traction.

                                                                                                                      3. 2

                                                                                                                        For those of us with less knowledge on this subject, can you share some examples that you have in mind?

                                                                                                                        1. 12

                                                                                                                          Some that immediately come to mind upon reading about Mastodon:

                                                                                                                          • Diaspora
                                                                                                                          • Tent.io
                                                                                                                          • Pump.io
                                                                                                                          • Friendi.ca
                                                                                                                          • Identi.ca (now GNU social?)

                                                                                                                          Edit: seems there are quite a few more I didn’t know about!

                                                                                                                          1. 3

                                                                                                                            Identi.ca is a pump.io node; it used to be based on StatusNet which is now GNU social.

                                                                                                                            1. 15

                                                                                                                              Mastodon is compatible with GNU Social fwiw. It’s an alternate server and web-UI implementation, but speaks the same protocol and can federate with GNU Social instances. The linked article doesn’t make this clear, but the GitHub repo does.

                                                                                                                              I think the linked article is targeted at Twitter users looking to switch who don’t already know anything about the history of open-source / federated networks, so avoids going into too much digression there. There’s been a huge spike in people signing up on mastodon.social the past 2-3 days, it seems due to a dislike of some recent Twitter changes that somehow it was in the right place at the right time to capitalize on. So I think this post is an attempt at writing a “hello, welcome to this new option” article for people who are seeing people on Twitter post about it and are wondering what this is all about.

                                                                                                                            2. 1

                                                                                                                              Here’s someone’s attempt to provide a short history of how all this stuff came about, and how it relates.

                                                                                                                          2. 2

                                                                                                                            The Github repo has more technical infos. Mastodon should federate with the others.

                                                                                                                          1. 4

                                                                                                                            Doesn’t the HTTPS-Everywhere campaign’s success limit the damage here to DNS queries?

                                                                                                                            1. 6

                                                                                                                              Well, the DNS queries and the IPs you’re connecting to. Equivalent information in the common case, but in theory you could use DNSCrypt to a nameserver other than your ISP’s (though almost nobody does) and in that case they’d not see your DNS queries, but still see what IPs your HTTPS connections are going to.

                                                                                                                              What I haven’t found is any kind of attempt at quantifying how significant that is. In some cases HTTPS ends up hiding basically all the relevant information. For example the information that you connected to Wikipedia or Google is nearly useless for building any kind of surveillance or marketing profile, since almost everyone does. So HTTPS hiding the actual Wikipedia articles you read or Google search queries you made accomplishes the goal nearly 100%. However, if you’re regularly connecting via HTTPS to a site with one clear type of content that isn’t co-located with a bunch of other popular sites using SNI, then the mere fact that you connected to that IP gives away a lot of information.

                                                                                                                              1. 2

                                                                                                                                Is “not co-located” necessary? I think ISPs can collect SNI information just fine, although I don’t know whether they do.

                                                                                                                                1. 1

                                                                                                                                  You’re right. For some reason I thought the SNI information was encrypted in recent versions of TLS, but I’m wrong, apparently that was proposed for TLS 1.3 but didn’t make it in.

                                                                                                                              2. 1

                                                                                                                                Only for sites that support HTTPs, right? Many do not.

                                                                                                                              1. 31

                                                                                                                                Configuration files suck, but at least they have simple syntax and declarations. I don’t have to learn to evaluate a new programming language in my head to understand each program’s config.

                                                                                                                                (Also, @crocket, please post your opinion on the story in the comments rather than in the story itself.)

                                                                                                                                1. 18

                                                                                                                                  It cannot be overstated how nice it is to be able to, over the phone, tell somebody “Hey, go to this line, change this value to say this, restart and let me know what happens”.

                                                                                                                                  Also, it’s nice to be able to consume configuration files from other languages. Having to embed a language interpreter to parse a config “file” years later is annoying at best.

                                                                                                                                  1. 5

                                                                                                                                    If they remain that simple… Great.

                                                                                                                                    But they don’t.

                                                                                                                                    They gather features until https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule holds.

                                                                                                                                    And then you have a Turing Tarpit that is as complicated as a programming language, less formally specified and really poorly documented and with no debugger support.

                                                                                                                                    Linker script is a truly horrible example.

                                                                                                                                    1. 4

                                                                                                                                      But they don’t.

                                                                                                                                      I don’t believe this is the universal rule you make it out to be. Sometimes they do stay simple. Personally I’ve never run into a single config that morphed into something that would have benefited from being written in a programming language. All the config files I’ve ever used were simple. Whereas it seems that in your experience you’ve only encountered configs that eventually become complex.

                                                                                                                                      1. 2

                                                                                                                                        I’ve never run into a single config that morphed …

                                                                                                                                        If you’d like a specific example, one that I have run into: Apache rewrite rules making use of the “skip next N rules” feature. If you have more than one of these it’s easier to write explicitly procedural code with if statements and blocks like you get in Varnish VCL or any general-purpose programming language.

                                                                                                                                        (Varnish’s VCL is a somewhat nice middle point; it’s explicitly procedural but it doesn’t have any facility for loops except for one feature which, off the top of my head, I think restarts the current request from scratch and you’re somewhat discouraged from using.)

                                                                                                                                        1. 1

                                                                                                                                          I have met a couple of proprietary gnarlies.

                                                                                                                                          The thin edge of the wedge is when “this config item + that config item must == the other”

                                                                                                                                          After screwing that up a time or ten, someone adds an addition operator.

                                                                                                                                          And maybe a loop.

                                                                                                                                          More usually it some sort of implicit goto or include.

                                                                                                                                          If somebody told me pam.conf was turing complete I wouldn’t be surprised. Appalled, but not surprised.

                                                                                                                                          The only reason why BPF isn’t, is that was a conscious design goal….. but if you told me you had worked out an insanely cunning way to do it…. I wouldn’t be (too) surprised.

                                                                                                                                          Taking a trawl through /etc/ I find a fair number of config files are indeed scripts. Shell scripts.

                                                                                                                                          css is a classic example that is now within an very very close to turing complete. (Depending on how kindly you view things, it is already).

                                                                                                                                      2. 5

                                                                                                                                        I find that simple configuration files follow the Rule of least power, a good and sound [software] engineering principle.

                                                                                                                                        That said “everything should be made as simple as possible, but not simpler” and the OP makes a fair point that configurations (and configuration languages) gets more and more complex over time but then your problem (and your domain) has moved from configuration to scripting and you have new requirements to account for and new trade-offs should be made.

                                                                                                                                        1. 3

                                                                                                                                          I don’t have to learn to evaluate a new programming language in my head to understand each program’s config.

                                                                                                                                          In all likelihood, they would use one of a fairly small number of languages.

                                                                                                                                          You could also still do what people do now, which is search for a solution on StackOverflow.

                                                                                                                                          1. 3

                                                                                                                                            I use the Awesome window manager, and its config file is written in Lua. A couple months ago an update I installed included a breaking change and, while I looked at StackOverflow, most of the resolution involved learning how Lua deals with null values, casts (or doesn’t) strings into ints, and other language minutia. None of it was “Hey, paste this in to replace what you have”, it was just programming time.

                                                                                                                                            1. 1

                                                                                                                                              I feel like “whether it embeds Lua or a JSON parser” is somewhat orthogonal to “handles breaking changes gracefully over time”.

                                                                                                                                        1. 3

                                                                                                                                          The example is much harder in statically-typed languages than dynamically-typed languages. Beyond that, it is even more difficult in a language (like Java) that is statically-typed but which doesn’t have union types. (AFAIK the closest thing to a union type in Java is an interface.) The interviewer is presumably someone with a background in a dynamically-typed language interviewing predominantly Java programmers. I think this explains why the interviewer expects it to be easy but it’s difficult for his interviewees.

                                                                                                                                          1. 3

                                                                                                                                            I largely agree with Liskov’s attitude towards monads: I always feel like Monads hide data in a secondary weird interface that you have to work around with secondary syntax.

                                                                                                                                            1. 17

                                                                                                                                              Only a fraction of monads are concerned with managing data at all. It may seem weird if you haven’t used them much, but probably only because most people don’t realize how monstrous the semantics of most imperative languages with “not weird” data management are. The type of monads you’re talking about (like State) are a semantically and syntactically simple and convenient way of doing stuff to data.

                                                                                                                                              Let’s take State as an example. If you want to represent a function that takes some arguments, takes a state, modifies the state, and returns a value, you can do it like this:

                                                                                                                                              doStuff :: args -> state -> (state, result)
                                                                                                                                              doStuff = ...
                                                                                                                                              

                                                                                                                                              Now, if you want to do a bunch of these things in sequence, you have to take the state output of one function, feed it into the next function, etc. etc. and it gets annoying fast. So instead, we just have the State monad, which is conceptually

                                                                                                                                              type State state result = state -> (state, result)
                                                                                                                                              

                                                                                                                                              So we can write this as

                                                                                                                                              doStuff :: args -> State state result 
                                                                                                                                              

                                                                                                                                              And then we can build functions for working with “State s a” values, which are just functions that take a state and return a state and a result. So instead of manually feeding the state output into the next function, you can just use the monadic bind operator (>>=) which does it for you. This lets us write things like

                                                                                                                                              doItTwice = do
                                                                                                                                                  doStuff 1
                                                                                                                                                  doStuff 2
                                                                                                                                              

                                                                                                                                              Which gets desugared using (>>=).

                                                                                                                                              This isn’t some weird detour people choose to take for no reason; this is actually the most straightforward way to do something like this in a language that has simple, equational semantics.

                                                                                                                                              1. 7

                                                                                                                                                So you take “state”, which is implicit in other programming languages; 1. make state explicit; 2. find that explicit state is awkward to pass around; and then 3. use a mechanism to hide state in an opaque data structure.

                                                                                                                                                1. 10

                                                                                                                                                  Correct. And if you think that sounds excessive, that’s a perfectly reasonable opinion!

                                                                                                                                                  1. 6

                                                                                                                                                    Yes, and you get the best of both:

                                                                                                                                                    1. Convenience
                                                                                                                                                    2. Abstraction
                                                                                                                                                    1. 1

                                                                                                                                                      The advantage, as I understand it, is that you no longer have the ability to change shared state from anywhere in a program, which makes the program easier to reason about. This seems like a huge benefit for anyone who has dealt with a spaghetti of cascading state transitions happening everywhere in a codebase (or even a single file).

                                                                                                                                                      1. 3

                                                                                                                                                        But if you are carrying around opaque state objects, you’ve simply made it harder to understand.

                                                                                                                                                        1. 4

                                                                                                                                                          It eliminates the problem where what looks like a simple accessor method turns out to be messing with some hidden global state. If you have nine functions that don’t touch state and one that does, passing an opaque state object to the latter makes your code easier to understand because you can now be confident that the functions that don’t touch state don’t touch state.

                                                                                                                                                          And if all your functions mess with global state, you already had a problem. Using monads just makes it harder for you to paper over it.

                                                                                                                                                          1. 1

                                                                                                                                                            Do you mean that the monad is opaque in that it is an abstraction the programmer needs to understand to use it? The state itself isn’t opaque (as I understand it). On the one hand you’re restricted with what you can do. On the other you can still use mutation-like code in localized areas where these kinds of mutations are easier to reason about.

                                                                                                                                                            I could understand the argument that the advantage of using a state monad is outweighed by the complexity of using an additional abstraction. I have no opinion one way or the other about that. But your argument seems to be that by using state monads to mimic state mutation you don’t get any benefits whatsoever, since the same code could be replaced with code in another language that does actual state mutation. That seems overstated since, while that is technically true that an imperative program could replace a Haskell program, by converting the code you also remove the strongly-typed restrictions on state transitions that you’re getting in Haskell, and introduce the ability for programmers to start introducing transitions of shared state in a totally unchecked way (a problem which causes serious problems for many codebases).

                                                                                                                                                          2. 2

                                                                                                                                                            Hm, Rust solves that by explicitly stating mutability and sharing in the interface.

                                                                                                                                                            Monads kind of implicitly allow that, if you know what the Monad at hand does.

                                                                                                                                                            I’m not arguing either way, but Haskell always takes the high road of abstraction with any problem and if you don’t understand the abstraction, you become lost.

                                                                                                                                                        2. 4

                                                                                                                                                          I appreciate you taking the time to write that explanation, but I understand use cases of monads - I still find it a weird detour.

                                                                                                                                                          “Instead of manually feeding the state output into the next function” – this is hiding lexical+data binding in some weird interface (namely, your type system). Why not just use something like and-let*? Then you can tie together functions that don’t share monadic types, but do share partial arguments in order.

                                                                                                                                                          (and-let* ((res1 (func1 state))
                                                                                                                                                                     (res2 (func2 state res1)))
                                                                                                                                                            (func3 res1 res2))
                                                                                                                                                          

                                                                                                                                                          I bet if you’re mildly familiar with a let expression, you can guess what’s going on. In the monadic example, I need to inspect the type of each function to understand the control flow of my program. That’s what appears weird to me. Please note I don’t think I’m going to convince you, and that’s fine, just want to publish my thoughts.

                                                                                                                                                          1. 4

                                                                                                                                                            State is a very, very limited case of what Monads (and transformers) are capable of. I used it as a pedagogical example specifically because it’s relatively easy to manipulate even without a concept of monads. However, there are several more considerations.

                                                                                                                                                            First, what you wrote is confusing and verbose. Monads via do-notation are much clearer syntactically. (Subjective, obviously)

                                                                                                                                                            Second, the trick you’ve proposed doesn’t generalize to more complex monads, such as (for example)

                                                                                                                                                            ExceptT Exception  . StateT AppState . ReaderT Config
                                                                                                                                                            

                                                                                                                                                            Which would allow you to have implicit configuration passing, managed state, and value-level exception handling, parametrized over some other arbitrary monad (which could be STM, IO, or whatever else you want).

                                                                                                                                                            1. 3

                                                                                                                                                              First, what you wrote is confusing and verbose. Monads via do-notation are much clearer syntactically. (Subjective, obviously)

                                                                                                                                                              I can actually accept either side of this particular style preference, but I don’t find either one a great argument for monads as a concept (and was therefore pretty confused when I first found it as a pedagogic example). To me, it mostly just takes you into the weeds of syntactic sugar. And there are a lot of ways to refine syntactic sugar, most of which have nothing to do with type systems.

                                                                                                                                                              There’s an old debate from the ‘90s about whether Perl makes things clearer or more confusing when it threads a magical implicit $_ state through various commonly used functions (I guess this probably is even older, but I first encountered it with Perl). Avoids the verbose explicit state passing, but at the expense of state passing without you seeing it. Some people strongly prefer explicit state-passing instead of the implicit $_ as a way of making it clearer exactly what state is being passed along which paths. The first time I ran across a monad tutorial using this approach, it read to me exactly like this Perl debate. As if the tutorial were written by someone who liked Perl’s implicit state passing, and was trying to sell Haskell as a Perl on steroids, a language perfected for magical implicit state passing, except now in all contexts! But Haskell people mostly seem to hate Perl, so I’m not sure this was the intent.

                                                                                                                                                              1. 5

                                                                                                                                                                The reason it’s a debate is that you want visibility but also conciseness, and the two are in tension. IMO monads are a great synthesis that gets you the best parts of both: almost as concise as completely implicit state, almost as visible as completely explicit state.

                                                                                                                                                        3. 6

                                                                                                                                                          My favorite description of monads is “decorators for function composition”. From that viewpoint, they’re a useful time saving convention.

                                                                                                                                                          I consider that an accurate description; I have one place to put the code for composing a bunch of operations. That code can do a pile of useful things for me.

                                                                                                                                                          1. [Comment removed by author]

                                                                                                                                                            1. 3

                                                                                                                                                              That’s the wrong way to think about it.

                                                                                                                                                              Nah, I like my way. I’ll let Barbara make her own points though.

                                                                                                                                                              The sequencing of functions can also be managed easily in languages that don’t default to lazy evaluation as long as they have the concept of function application and closures. AKA and-let* in scheme.

                                                                                                                                                            2. 2

                                                                                                                                                              Monad is largely a design pattern. I know that “design pattern” is a stigmatized phrase, but design patterns don’t have to be terrible. It’s the sloppiness endemic to corporate software engineering, and the historical limitations of certain popular languages relative to the purposes to which they were put, that resulted in the infamous “Gang of Four” mess, but design patterns recur in all aspects of life and aren’t always bad.

                                                                                                                                                              For individual monads, the fact that these types are monads is not very interesting. It’s fairly obvious. Thus, when you’re beginning to learn Haskell and not especially concerned yet with code reuse, it probably doesn’t seem to add much to talk about Monad, a type class which begins to show its value when you want to write code that’s agnostic of whether you’re doing a State s or IO or ST s action.

                                                                                                                                                              1. 1

                                                                                                                                                                I don’t know that I disagree with you at all here, but I also don’t understand why you would dislike that situation. As I read it, you’ve just described abstraction/indirection and Monadic internal languages are definitely a technique for doing that. If your goal is to avoid abstraction generally then I can see an argument for avoiding Monads very clearly.

                                                                                                                                                                On the other hand, if you’re trying to make a more refined argument about monads being a bridge too far in the process of abstraction then I think there’s also an argument to be made there… but I don’t see it yet.