1. 3

    The author claims they’re a programmer, but they still clicked 338 checkboxes manually? Sounds fishy :)

    Here’s what I’ve done on Tumblr, which also has something similar.

    for (var x of jQuery("input[checked]")) {jQuery(x).removeAttr("checked");}

    1. 11

      The author is a programmer, a software architect, an hacker, and a curious person in general.

      I can conceive several ways to fool your smart jquery script. If you cannot think of them yourself, you shouldn’t code in Javascript, actually.

      But also I’m a UI and UX designer, at work.

      I was surprised to see a nice UI with such a stupid mistake.

      I hoped the developer on the other end was cool enough to surprise me.

      After the first ten clicks I realized she was not that smart.

      I hit F12. But then I thought “my users cannot hit F12: lets walk their path and see how I feel”.

      I’m not stupid. I simply care.

      1. 2

        I can conceive several ways to fool your smart jquery script. If you cannot think of them yourself, you shouldn’t code in Javascript, actually.

        • I don’t think he was claiming his solution was a fit for all
        • So by your logic only people who know DOM JS should code in JS? ;)

        I know this was a reply to a slightly provocative comment in defense of the author, but this in particular seems a little silly

        1. 5

          I’m the author. And actually I’m sorry for the tone of the reply: I’m tired, and I didn’t intended the @janiczek’s post as a joke for me, but as an attempt to justify InfoWorld by calling me fishy.

          I’m fishy, definitely! :-)

          But I also care about users. And I’m an European fish…

          So by your logic only people who know DOM JS should code in JS? ;)

          Nobody should code in JS. Really. Nobody should.

          But yes, if you don’t know how DOM JS has been interpreted in the last 10 years, I think you shouldn’t code in JavaScript professionally. You might think I’m exaggerating to get a point, but trust me: everything is still there, under the hood. Ready to break.

          1. 2

            Thanks for the kind reply. I wasn’t trying to provoke myself, just point out something that seemed a bit off :) Professionally? Perhaps your right in a perfect world, but the fact remains there will always be code monkeys that build or maintain simple systems for a customer base that can’t pay for a seasoned developer. Regardless, I agree with the pain point of your article :)

            1. 3

              Mm, I kind of feel like as a profession we should try to have more respect for our own work. Software can cause significant harm, and we’ve all just collectively agreed that it’s okay to prop up companies that want to build broken things and not properly maintain them. Maybe companies that aren’t willing to spend the money to build safe software shouldn’t have the privilege of getting engineers to work for them.

              I know that’s a tangent and not really what you were trying to address.

              1. 3

                I completely agree with your first statement, having respect for your own work is a great virtue.

                The devil is in the details in regards to companies/individuals who provide shoddy services. Outside passionate and informed social circles, it’s customers vote with their pockets (counting data as a form of currency here), whether that be for trading for convenience or just a result of plain ignorance.

                Unfortunately there aren’t any easy remedies to this problem. Shoddy companies/individuals will find ways to work their way around regulations, and customers will quite happily dig themselves into holes in pursuit of the cheapest or quickest solution. That doesn’t mean you don’t try, in fact I personally think one of the best tactics we can use for problems such as these, is informing the general public of the consequences (though that’s another problem in itself).


                  Yes, I agree with all of that, and thank you for it.

                2. 2

                  Maybe companies that aren’t willing to spend the money to build safe software shouldn’t have the privilege of getting engineers to work for them.

                  I see your point, but to me it’s like saying that companies that aren’t willing to spend the money to write proper English shouldn’t have the privilege of getting writers to work for them.

                  They can learn how to write by themselves.

                  I prefer a different approach: turn all people into hackers.


                    Yeah, I see that point also. But, I mean, writers have historically been more willing to stand up to exploitative labor practices than hackers have… I think there’s a balance to be found, and getting to the right balance requires rethinking some things.


                      We are just like scribes from Ancient Egypt.

                      Despite the groupthink, we are still at a very early stage of information technology.

                      Just like being a scribe back then, being hackers today does not mean understanding the power and responsiblity we have. But it’s just a matter of time.

                      We will forge the future.

              2. 1

                I’m sorry if my post came as provocative! (Maybe my definition of “fishy” – as English is not my native language – is slightly off compared to your definition)

                Yeah, “I know I could do X instead of clicking, but common user can’t, so let’s walk in their shoes” is a fair motivation. Maybe I just expected the thought to be expressed in the post, given you’ve expressed you’re a programmer. But maybe that’s a silly expectation ¯_(ツ)_/¯ Thanks for the clarifications in the comments here.

        1. 3

          From the title, I first thought this might be an article from a long, long, time ago. Nah, some vulnerabilities just persist. I left Windows long time ago. Have there been any improvements on preventing or detecting these risks?

          1. 1

            Granted I’m a bit biased, but what’s the risk? I’m missing it.

            1. 3

              In principle, malware could hide in it - and probably does, but I don’t think many people are under the illusion that they can detect malware without specialized tools anyway. The most “practical” use I can think of for it is data exfiltration. I think most of the risks are around bugs rather than malicious use; it’s a somewhat bizarre feature, and not well-known, and as such it should be regarded as technical debt.

              1. 4

                I see. I’m sure it’s used by Windows though.

                It’s a pretty common feature across many filesystems: https://en.wikipedia.org/wiki/Extended_file_attributes#Implementations

                1. 3

                  Yes, you’re certainly correct that it’s not that rare a thing. As the original article notes, NTFS added it for parity with Apple’s HFS+. That doesn’t really change that it’s obscure and fragile, but certainly Windows shouldn’t bear the blame alone.

                2. 2

                  You nailed it! Hiding malware and storage channel for exfiltration are main issues.

            1. -10

              I know you get a lot of pat-on-the-backs when you implement stuff for the disabled. But I just feel like it’s rarely worth it unless you are at a large scale where the disabled population will offset the man-hours. Not to mention that different segments of the disabled have different requirements and the same special interface will not couple with all of them.

              So to me, I can’t help but think that whenever some megacorps implement these solutions, it’s more likely virtue-signalling rather than altruism or legit economic advantage.

              The problem of course, is that if we could solve this problem economically, then we would have solved it forever, but if it is virtue-signalling, then the incentive isn’t really to provide solutions, but to provide the appearance of caring, and so the mismatch will eventually result in the problem not really being solved long-term.

              1. 10

                I don’t understand this comment at all. If it’s not profitable, why do you think companies are “virtue signaling” and not caring? ISTM you’re reading an awful lot into their behavior, under the odd belief that doing something good has to be for egotistical reasons, and not because you want to help someone out.

                1. 3

                  To expand on what I believe @LibertarianLlama is saying is, it’s possible this comes out of their marketing budget as a kind of loss. The upside of this would be that the PR leads to other sells, not necessarily of this product, but others.

                  In the end it doesn’t really matter. It’s a local choice of the company, not trying to solve a problem globally in an economically sustainable way.

                  It should also be remembered that helping people can be egotistical, in which case it’s a win-win! I find it personally strange when people sometimes boycott beneficial things because they’re suspicious of the underlying motives, when the motives clearly aren’t arming belligerents in a foreign war, or something else clearly evil.

                  1. -2

                    why do you think companies are “virtue signaling”

                    because they think creating an image will give them financial rewards.

                    1. 3

                      I’m truly sorry you’ve never had the opportunity to work somewhere that prioritizes results over optics.

                  2. 9

                    Did you read the article? The controller is heavily customizable (it’s a platform, really), precisely to accommodate as many people’s needs as possible.

                    1. 13

                      I think you’re right, but I SO don’t care!

                      As a partially blind person, there is SO much of the gaming world that’s closed off to me. That’s OK. I still sleep just fine at night knowing I will never be a Call of Duty GOD :)

                      However, when game developers and console makers bother to make adaptations available to allow me and others with disabilities to enjoy the beautiful mix of art and science that is most modern video games, I really appreciate it.

                      So, virtue signaling or not, this is a laudable move on Microsoft’s part, and I for one think we should all recognize that.

                      Almost makes me want to own an Xbox again. Only problem is that I haven’t had time to play a game on any platform in ~6 months :)

                      -Chris (Aside from iPad gaming in waiting rooms sometimes)

                      1. 7

                        I think you’ve put your finger on a significant contradiction in libertarianism. You want to judge the worth of the enterprise by economic returns: success is denominated in dollars and the market is the only neutral or efficient judge of value.

                        However, the other name for “to provide the appearance of caring” is marketing, and of course good marketing enormously multiplies the returns of a product, the world being annoyingly reticent to beat a path to the door of entrepreneurial mousetrap makers. Even in the very unlikely event that sales of this controller wouldn’t cover the costs to design and manufacture it (given that video gaming is measured in the tens of billions for the U.S. and this product looks overwhelmingly superior to competitors for the mostly-untapped wallets of tens or hundreds of millions of humans with motor control injuries), Microsoft could get a positive return on investment just from the increase in warm, fuzzy feelings from the majority of the market with no need for this product if they go on to buy ever-so-slightly-more copies of OneDrive or Office. The existence of marketing and cross-promotion means that the value of these products can’t be judged solely by the invisible hand of the market discovering prices for goods and driving firms out of business. You make this point in reverse; the long-term existence of marketing points it being economically valuable. There are externalities not on the books of a single product, just like how, in reverse, the market overvalues a polluter because the externality of cleaning up toxic waste or reversing climate change isn’t charged to the company and so can’t be reflected in the stock price.

                        But whether or not the economics work, perhaps in this instance we can settle for helping make an entire art form accessible because it’s a small act of basic human decency and we’re not unthinking monsters.

                        1. 4

                          I’ll probably get downvoted, but here goes…

                          I think you’ve put your finger on a significant contradiction in libertarianism. You want to judge the worth of the enterprise by economic returns: success is denominated in dollars and the market is the only neutral or efficient judge of value.

                          However, the other name for “to provide the appearance of caring” is marketing […]

                          Libertarianism is actually about the freedom to property and its action, where the individual is his or her own property. Economics is more a description of the market that emerges from action and property. Be it a free market or not, depending on the freedom to the underlying rights.

                          So when you point out a contradiction, there really is no contradiction. It barely exists on the same plane of reality. Anyone in business, who wants to stay there, knows about marketing, cross-promotion and all that. It’s a business strategy.


                          Libertarianism is not a game of winners and losers where money is how we keep score.

                          But in a hypothetical world where it were, Microsoft would likely end up winning with this device. As would the customer demographic.

                        2. 1

                          Even if I disagree with you, I don’t understand why you are being downvoted for this argumenter opinion of your. Anyway… thank you for expressing yourself on the topic.

                          To me it’s mostly about having a customizable solution for gaming controls, that can be used for players with disabilities. If you look at Nintendo, they recently launched this thing with customizable objects in paper to enhance the gaming experience, this is just how the Microsoft gaming team is implementing it! Bold move from them!

                          1. 9

                            Even if I disagree with you, I don’t understand why you are being downvoted for this argumenter opinion of your. Anyway… thank you for expressing yourself

                            Because it’s incorrect, and baseless bloviating in order to shit on the idea of not needlessly excluding the marginalized.

                        1. 6

                          As others have alluded to, this is the classic plight of early “Web 2.0” successes where they thought they could keep their service “free” by using advertiser support. Only when nobody cared and everybody was enjoying their free lunch Twitter among many others has started to clamp down.

                          What I would LOVE to see is widespread acceptance of the idea that advertiser funding is a fatally flawed model. One way for Twitter to go with this is to offer a “pro” option which would be ad free and paid, and also allow full and open access to all of its APIs, including the ones they’ve nuked in recent years.

                          One of the things that drew me to Twitter was its diverse ecosystem of users and clients because developers had free reign to innovate using their platform. Clearly the future for this kind of innovation lies with tools like Mastodon and Pleroma, but as I say above it’s not too late for companies like Twitter to make bold moves and fix the broken model before it destroys them.

                          1. 2

                            I find the “pro” strategy appealing, but I can’t think of a big site that’s succeeded with it. I’ve seen a lot of sites try and it doesn’t really seem to last. I don’t have numbers available, but I suspect that advertising revenue substantially outweighs subscription revenue most of the time.

                            1. 4

                              Can’t remember where I heard it, but on some sites the value of a user (to advertisers) who would use a pro option exceeds what said user is willing to pay.

                              Not sure if true or not, but it has stuck in my mind.

                              1. 2

                                Yes, that’s what I was suggesting.

                                1. 1

                                  Oh! This is super interesting for a completely different discussion I’ve been having recently. Can you do me a favor and try to find out where you got that?

                                2. 2

                                  I did some googling wondering if I could find some real data on this and failed. Flickr comes to mind, which was in fact quite successful and is still much loved despite having been bought by that roving dumpster fire that is Yahoo, and recently SmugMug.

                                  1. 1

                                    This is not at all an apples-to-apples comparison, but The Guardian (a newspaper/media co) now makes more from subscribers than from advertising. It’s a far cry from saying “this model works!” (the same article notes they still posted a loss) but I think it’s promising.

                                  2. 2

                                    What I would LOVE to see is widespread acceptance of the idea that advertiser funding is a fatally flawed model. One way for Twitter to go with this is to offer a “pro” option which would be ad free and paid, and also allow full and open access to all of its APIs, including the ones they’ve nuked in recent years.

                                    This may be an unpopular opinion, but I don’t think social networks offer enough value for enough people to pay in the “pro” model. It might work on a small scale, but I don’t think it can work for a network as large as Twitter.

                                    1. 2

                                      You may be right. That would have me leaning towards the idea that behemoths like Twitter will need to go full on closed system draconian advertising for everyone and no third party anything, which will drive away the minority who really care (who should likely be seeking safe harbor in open networks like Mastodon at this point anyway.)

                                      I personally feel that if someone could make a Mastodon or Mastodon-like server simple enough to deploy that grandma could do it, Mastodon would really take off in a big way.

                                  1. 3

                                    The title of the paper is “Out of the Tar Pit” not “Common Causes of Complexity”.

                                    1. 2

                                      Thanks - fixed.

                                    1. 27

                                      What are the advantages to making it federated over the current setup?

                                      1. 7

                                        In terms of content and moderation, each instance would be kind of like a “view” over the aggregate data. If you want stricter moderation you could sign up for one instance over another. Each instance could also cater to a different crowd with different focuses, e.g. Linux vs. BSD vs. business-friendly non-technical vs. memes vs. …. Stories not fitting an instance could be blocked by the instance owner. Of course you could also get the catch-all instance where you see every type of story; it might feel like HN.

                                        The current Lobsters has a very specific focus and culture, and also locked into a specific moderation style. Federating it would allow a system closer to Reddit and its subreddit system where each instance has more autonomy, yet the content from the federated instances would all be aggregated.

                                        So of course such a system wouldn’t be a one-to-one replacement for Lobsters but a superset. Ideally an individual instance could be managed and moderated such that it would feel like the Lobsters of today.

                                        1. 18

                                          The current Lobsters has a very specific focus and culture, and also locked into a specific moderation style. Federating it would allow a system closer to Reddit and its subreddit system where each instance has more autonomy, yet the content from the federated instances would all be aggregated.

                                          If federation results in a reddit-like site, I’d much rather that lobste.rs doesn’t federate. It’s a tech-news aggregator with comments, there’s no real benefit in splitting it up, especially at it’s current scale.

                                          1. 6

                                            I get what you’re saying. I think OP framed the idea wrong. People come to Lobsters because they like Lobsters. The question is whom would the federated Lobsters benefit – it would mostly benefit people who aren’t already Lobsters users.

                                            It’s just that the Lobsters code base is open source and actively developed, and much simpler than Reddit’s old open source code. So it’s not unreasonable to want to build a federated version on top of Lobsters’ code rather than start somewhere else.

                                            1. 3

                                              it would mostly benefit people who aren’t already Lobsters users.

                                              Well that was my point. Any spammer or shiller can create and recreate reddit and hacker-news accounts, thereby decreasing the quality and the standard of the platform, and making moderation more difficult. This is exactly what the invite tree-concept prevents, which is quite the opposite of (free) federation.

                                              1. 8

                                                We do have one persistent fellow who created himself ~20 accounts to submit and upvote his SEO spam. He’s still nosing around trying to re-establish himself on Lobsters. I’m very glad not to be in an arms race with him trying to prevent him from abusing open signups.

                                                1. 1
                                          2. 2

                                            Based on my experience in community management, including here on Lobsters, I do not believe it’s possible for an individual instance in a system like you describe to have a coherent culture which is different from the top-level culture in substantial ways, unless you’re okay with participants feeling constantly under siege. The top-level culture always propagates downward, and overriding it takes an enormous amount of resources and constant effort.

                                            1. 1

                                              Have you used Mastodon at all? If that’s used as a model, it seems each instance can have a distinct personality, as Mastodon instances do today. Contrast with traditional forums, and Reddit to some extent, which do more-or-less have a tree structure and where your concern definitely applies. With federation there doesn’t necessarily need to exist a top-down structure, even if that might be the easiest to architect (although I don’t know if it is the easiest).

                                              1. 1

                                                I have used Mastodon, but not enough to have a strong opinion on it. It’s been a challenge for me to pay enough attention to it to keep up with what’s happening; it’s kind of an all-or-nothing thing, and right now Twitter is still taking the attention that I would have to give to Mastodon.

                                          3. 7

                                            Biggest argument in favor is probably for people that want to leech off of the quality submissions/culture here but who don’t want to actively participate in the community or follow its norms. That and the general meme today of “federated and decentralized is obviously better than the alternative”.

                                            Everybody wants the fruit of tilled gardens, but most people don’t want to put in the effort to actually do the work required to keep them running.

                                            The funny thing is that we’d probably just end up with a handful (N < 4) of lobster peers (after the novelty wears off), probably split along roughly ideological lines:

                                            • Lobsters for people that want a more “open” community (signups, etc.) and with heavier bias towards news and nerdbait
                                            • Lobsters for social-justice and progressive people
                                            • Lobsters for edgelords and people who complain about “social injustice”
                                            • Lobsters Classic, this site

                                            And sure, that’d scratch some itches, but it’d probably just result in fracturing the community unnecessarily and creating the requirement for careful monitoring of what gets shared between sites. As a staunch supporter of Lobsters Classic, though, I’m of course biased.

                                            1. 3

                                              So “federation” is what the cool kids are calling “forking” nowadays? Good to know ;)

                                            2. 2

                                              I’d be quite interested to see lobsters publish as ActivityPub/OStatus (so I could, for instance, use a mastodon account to follow users / tags / all stories). I don’t see any reason to import off-site activity; one of the key advantages of lobsters is that growth is managed carefully.

                                              1. 1

                                                Lobsters actually already does this with Twitter, so that seems both entirely straightforward to add and in line with existing functionality.

                                                (Note that I don’t use Twitter, so I can’t speak to how well that feed actually works.)

                                                1. 1

                                                  The feeds already exist, just have to WebSub enable them…

                                                2. 1

                                                  It won’t go away entirely if the one, special person who happens to own this system decides to make it go away for whatever reason of their own. It won’t die off if this specific instance gets sold or given to someone who can’t handle it and who runs it into the ground.

                                                1. 2

                                                  Huh, what would a federated message board look like? I guess I could see a reddit-like one where each sub could be on a different server, but you’d have a shared account around them all. Still one server per forum, so you can have consistent ordering of stories and comments. I’m not really sure what the benefit is to anyone of having a shared account among a ton of federated board servers, though. It just preserves reddit weirdness like sharing massively different karma amounts between joke boards and deep research boards.

                                                  Lobsters is meant to have one main page though. How would you do consistent ordering of the front page if stories were federated?

                                                  1. 4

                                                    what would a federated message board look like?

                                                    Usenet, I think. Threaded messages (with different people getting a different, but eventually consistent view of the thread). Each lobste.rs post would be a new top-level thread.

                                                    You’d lose voting and ranking on a straight usenet model, but that would be a small extension (usenet already supports control messages - you’d just have upvotes/downvotes propagated as a type of control message and your ‘top level’ view respecting the votes and an aging algorithm etc.

                                                    1. 1

                                                      I’m not at all convinced that you need that consistent view. Twitter doesn’t have one - everyone sees their own slice of things that they’re paying attention to.

                                                      Having some consistency is a prerequisite for a place to be a community, though, so it would certainly be a very different form of interaction.

                                                    1. 3

                                                      Good article. I particularly like that it raises the issue of unprotected length fields. I’d never thought about how security issues affect the suitability of a format for archival, but certainly if it became unsafe to use the existing libraries for the format, that would create work for archivists.

                                                      1. 3

                                                        Dumb question. How does Floyd’s operator precedence grammar (1963) fit into all this?

                                                        There’s this brief passage.

                                                        Why not use the algorithms that parse operator expressions for the whole language? Samuelson and Bauer 1959 had suggested exactly that. But, alas, operator expression parsing is not adequate for languages as a whole[49].

                                                        But it doesn’t explain why they are “not adequate”.

                                                        This is a very well researched article and I understand not everything can be included but I found a lot of the opinions included to lack justification.

                                                        I can still take it for its historical value but not a summary of what likely works or doesn’t work.

                                                        1. 5

                                                          In my experience the most common source of ambiguity in programming-language grammars, other than operator precedence, is repetition. Function parameters, struct members, array literals, statements in a block of code, chained else-if cases… these are all grammatical constructs which can occur zero to n times, and they all have to deal in various ways with making sure the parser has a way to know when one item stops and the next begins, and when the overall list stops.

                                                          Natural languages also have phenomena such as rank-shift, where for example you take an entire verb phrase and put it in a different tense and use it inside a noun or preposition phrase that’s part of a larger verb phrase. If that sentence made no sense to you, it also makes no sense to Bison. :)

                                                          Oh - and in natural language, ambiguity is a property of the language that’s there “by design”, and clever writers very often exploit it. It can actually enhance clarity in certain situations, by eliding detail that shouldn’t be the focus. It can also be the basis of humor. So to truly “parse” natural language, you absolutely must model ambiguity.

                                                          1. 4

                                                            Re: rank-shift. I finally understood rank-shift when I read the Complete Lojban Language. Lojban is a constructed language with a formal grammar which still tries to support all kinds of features of natural languages. Lojban has an explicit support for rank-shift, although naturally, in order to be machine-parseable, it needs to be explicitly marked.

                                                            Lojban rank-shift, which is called “raising”, is described in the Complete Lojban Language, 11.10.

                                                        1. 13

                                                          The author has a pretty explicit bias toward Earley parsing, and using LL(k) over LR(k) methods (which I mostly share). There’s a wealth of information here, but he’s missing a couple other significant strands of generalized parsing research outside of Marpa’s particular Earley variant (perhaps because they don’t fit within his narrative).

                                                          GLR (Generalized LR):

                                                          • Tomita, LR parsers for natural languages, 1984.
                                                          • Tomita, Efficient parsing for natural language, 1986.
                                                          • Scott and Johnstone, Generalised bottom up parsers with reduced stack activity, 2005. This segues into their later work with Earley and GLL.

                                                          GLL (Generalized LL):

                                                          • Scott and Johnstone, GLL Parsing, 2010.
                                                          • Afroozeh and Izmaylova, Faster, Practical GLL Parsing, 2015.

                                                          Outside of Earley, GLL seems like a very practical generalized parsing approach. Instaparse is one implementation.

                                                          Earley / Shared Packed Parse Forests:

                                                          • Scott, SPPF-style parsing from Earley recognisers, 2008 (and several related papers by Scott and Johnstone). Note: I’ve never been able to get the approach described in the paper implemented correctly, and not for lack of trying.

                                                          Earley Intersection Parsing (not sure if there’s a canonical name for this):

                                                          • Bar-Hillel, Perles, and Shamir, On formal properties of simple phrase structure grammars in Zeitschrift für Phonetik, Sprachwissenschaft und Kommunikationsforschung, 1961. Proves some important results about intersecting automata and context free grammars.
                                                          • Chapter 13 (“Parsing as Intersection”) of Grune and Jacobs (also cited elsewhere in his timeline), particularly 13.4 (pgs. 437-439): This describes Bar-Hillel’s Intersection Parsing approach using contemporary CS & parsing terminology, and then suggests combining it with Earley parsing. While the basic intersection parsing method produces an astonishing amount of dead-end nodes to garbage-collect later, Earley parsers limit searching to productions reachable from the start symbol. If the completer is modified to produce non-terminals in the intersection parsing format (which is easy), intersection parsing nicely handles producing the parse forest (with structural sharing, when ambiguity produces multiple derivations).

                                                          I’ve been working on a C library for Earley Intersection parsing, and an awk-like, pattern-match-directed language based on it, but working on testing them thoroughly tends to lead me to working on theft instead.

                                                          1. 3

                                                            Thanks for details about alternatives.

                                                            I have played around with Bison’s GLR option and have almost managed to create a C parser that does not require special handling of typedefs in the lexer (ambiguities still appearing at runtime for a few constructs).

                                                            Grune and Jacobs’ Parsing Techniques - A Practical Guide is sitting in a pile waiting to be read (the first edition was manageable, the second looks daunting)

                                                            1. 1

                                                              I have a print copy of the second, and it’s never struct me as daunting – reading it cover to cover, perhaps, but it’s well suited to dipping into for just specific techniques, with a well-curated bibliography for further details.

                                                            2. 2

                                                              packrattle is another example of a GLL parser.

                                                              The times I’ve read about Earley parsers, they seem like the same solution as GLL, but attacking the problem from a different angle. Hopefully someone in academia will eventually prove that.

                                                              1. 2

                                                                I’m particularly interested in Earley parsers because some use cases of mine assume ambiguity (reverse-engineering binaries, scraping damaged data), and the chart that Earley parsers build up supports several operations beyond just recognizing & building parse trees:

                                                                • what terminals/nonterminals would advance the current parse(s) in progress? (i.e., autocomplete)
                                                                • if we don’t assume where the starting position is, what overlapping instruction encodings are there?
                                                                • if we inserted a fake nonterminal here, would enough of the parse have completed to match any instances of (some structure) between here and there?
                                                                • incremental re-parsing of changing input, since earlier columns in the chart are constant once the parser has moved on.
                                                              2. 2

                                                                Thank you for adding this. I understand why everybody is excited about Earley parsing, though I personally don’t feel sufficiently grounded in it to share that excitement, but at the very least other lines of research deserve to be mentioned.

                                                              1. 7

                                                                That article links to this one: http://www.cbc.ca/news/canada/nova-scotia/concerns-teen-being-railroaded-in-privacy-breach-to-cover-government-slip-1.4616972 …which, from my point of view in the USA, seems to take place in some sort of sincerer alternative reality…

                                                                [Evan D’Entremont, software engineer,] said [that] what police and the province say is a crime in this case is something he’s done “a hundred times” himself.


                                                                [The] province says an employee found the problem by accidentally doing the same thing.

                                                                He’s referring to using a script to slurp up all the documentID=1, documentID=2, ... documentID=7000 URLs from a server.

                                                                1. 16

                                                                  Yes, well put. I remember when, in the US, we used to debate after every one of these cases whether what the person did was really wrong, and whether the entity whose lack of security was revealed should have any liability.

                                                                  That was, of course, a glitch in the general principle that the law protects the interests of capital. It was a historical anomaly that the discussion could happen at all. These days, the conversations are much shorter, and most of the cases never get publicized, because it’s already well-established that if you make a wealthy entity look bad in a way that involves computers, you’re going to jail.

                                                                  Sorry for the bleak tone. I know I’m exaggerating slightly, but I don’t have much hope on this topic anymore.

                                                                  1. 3

                                                                    You’re not exaggerating, but look at Canada and retain/regain hope!

                                                                    1. 1

                                                                      I don’t want to rhetorically put the burden of saving the world on Canada… that would be abdicating the responsibility I still feel that everyone in the US has towards their own country. These struggles parallel each other, but they do have to proceed independently.

                                                                      1. 1

                                                                        I saw it as ‘if cyber law and environs is more or less maddening depending on locale, then maddening cyber law isn’t necessarily inevitable”.

                                                                        1. 1

                                                                          That’s a totally fair view. :)

                                                                1. 8

                                                                  I’ve had this problem, and it was part of why I moved away from Lisp and on to Haskell.

                                                                  At one time in my life, I very much bought in to the idea that dynamic typing allowed faster development. Then I got a chance to compare and contrast, by implementing the same algorithms (SLR parse-table generation with GLR nondeterministic evaluation) in both Common Lisp and Haskell. I had also, earlier on, written SLR with deterministic LR evaluation in C, so I was already experienced in the area. This was all about fifteen years ago now (2003-ish).

                                                                  When I did the CL version, I started with tuples. It turned out to be very challenging to debug. These algorithms have several different concepts which involve grouping alternatives together, and I found that I had a lot of trouble identifying whether a dumped data structure was correct or not.

                                                                  When I realized this was the reason I was having trouble, I rewrote the CL code to use the defstruct construct. It made development from that point a lot faster, although it still wasn’t ideal because nothing is actually enforced. It’s a lot more verbose, and honestly very frustrating to use. If I were designing a language today, I would make sure that the safest construct is also the simplest to use, and that would among other things mean leaving untyped tuples out of it completely, since they’re somewhat of an attractive nuisance.

                                                                  I then moved on and wrote the Haskell version. Haskell records, though much criticized, were at that time still slightly less verbose to use than CL defstruct classes. They also provided much stronger guarantees of correctness. The Haskell code wound up being un-idiomatic and convoluted (I was new to the language), but I got it debugged very quickly.

                                                                  Overall, Common Lisp has a lot of very impressive and clever language constructs which I enjoy understanding the subtleties of. Writing code in CL is fun because it panders to my desire to solve interesting problems, through the wealth of well-designed tools it offers. As someone with an interest in language design, though, I think it’s probably a really bad idea to offer that complexity to the client programmer. I agree with Rob Pike on this point - maintaining code is harder than writing it, so we shouldn’t max out our cognitive resources when we do the initial writing.

                                                                  1. 5

                                                                    7:00 - 7:20 Light alarm gradually fades to on. Wake up, in a fugue state.

                                                                    7:20 - 8:35 Gentle voice reminder of who I am and that I enjoy being alive plays every fifteen minutes.

                                                                    8:30 - 9:30 Get ready for work, including 30 minutes of light meditation in the shower.

                                                                    10:00 - 10:30 Arrive at work, make coffee, review notes from yesterday, review today’s calendar to make sure it’s physically possible, write today’s to-do list.

                                                                    10:30 - 11:30 Maybe meetings, maybe code. On a bad day, email.

                                                                    11:30 - 12:00 Lunch.

                                                                    12:00 - 17:30 Mix of meetings and code, fading towards email at the end of the day.

                                                                    17:30 - 18:00 Leave notes for tomorrow.

                                                                    18:30 - 20:00 Social media, food, dissociation, video games.

                                                                    20:00 - 22:00 Work on activism and other extracurriculars.

                                                                    1. 2

                                                                      How does that voice reminder really work?

                                                                      1. 6

                                                                        It gives me the information that, in the fugue state, I am lacking. This makes it possible for me to find the necessary memories. This a phenomenon that I experience as part of general dissociative identity disorder and dissociative amnesia stuff.

                                                                        It also has a very mild, carefully-chosen hypnotic effect which results in a slight mood boost.

                                                                        I think of it as my stage1 initrd.

                                                                        1. 1

                                                                          I’ve always wanted an alarm app where I can dictate messages to it to play in the morning. I could use myself saying don’t forget you need to do something, or don’t forget you’re trying to fix your sleep schedule, don’t sleep all day.

                                                                          1. 2

                                                                            I briefly considered interpreting “how does it really work” to be about the technical aspect, but decided to focus on what I think is the more interesting part. But yeah anyway I used Tasker on Android for it. I don’t think that would work for what you want, but maybe one of the voice assistants will grow that functionality someday.

                                                                    1. 5

                                                                      Everyone here seems like a morning person…

                                                                      1. 8

                                                                        I typically get up around 10am, if it makes you feel better. :-)

                                                                        1. 1

                                                                          Hey, same as me! I kept trying to shift to an earlier schedule since bosses tend to prefer it. Brain just doesn’t agree with it. They and I are happier if they schedule me in a bit later to leave a bit later.

                                                                        2. 2

                                                                          6am isn’t early in my world. I usually get up at 4am for exercise. Have done most days for the past 18 years.

                                                                          Been starting work most days around 6am for the past 6 or 7.

                                                                          Experimenting with injecting some leisure time into my morning by starting work at 7:30-8am.

                                                                          1. 2

                                                                            By “world” you mean you live in a Nordic country?

                                                                          2. 2

                                                                            It was this comment that inspired me to write mine. ;)

                                                                            1. 1

                                                                              Frankly, I’m surprised no one posted a night schedule.

                                                                              1. 1

                                                                                if it were up to me I’d work in fits and starts from about 10am to midnight. Unfortunately an office job comes with an expectation of visibility, and an attempt to travel at the same time as other road users.

                                                                              1. 10

                                                                                I’m quite disturbed by the “killing ex-girlfriend” jokes being just dropped in there like they’re par for the course. Is it only 13 years since that seemed OK to the point where none of the many commenters there thought it worth mentioning? Did none of the (currently) 14 upvoting Lobsters not think it worth mentioning either? Was it really OK back then anyway? Just a bit “edgy”? Wouldn’t just “your ex” have done just as well for an edgy joke, unless part of the subtext was “yes I know it’s dark AF but we’re all boys together here, aren’t we, and it’s cool and funny for boys to joke about which tools are best for killing their ex-girlfriends”? In which case, eesh, no wonder women struggle in the kind of environment which tacitly accepts this sort of thing.

                                                                                I know in-groups have their own humour and acceptable levels and I’m glad they do, but … doesn’t putting that kind of thing on a public forum have a bunch of problematic implications? And then even more so on another one, 13 years later, when the obvious problems with this kind of stuff have been made manifestly clear? I hate to be a party-pooper because I can’t bear over-engineered factory-factory stuff like it’s bemoaning either, but wow, in a post in a technical forum resurrecting another one, this careless aside sure makes me feel uncomfortable.

                                                                                1. 5

                                                                                  People have short memories and a tendency to retcon righteousness into them. Oh, it’s uncool to think that now? Good thing I never thought that way! And suddenly all the formerly bad people disappear overnight.

                                                                                  1. 4

                                                                                    Yeah I was thinking about that too, because I totally had a WTF reaction as well. I was actually going to comment a few hours ago but I was caught up playing games. Oops.

                                                                                    But when I think about it, in 2005 this wouldn’t have been out of place. I’ve noticed watching movies from the early 2000s has the same jarring effect, for similar reasons.

                                                                                    1. 2

                                                                                      It barely seems out of place today. These things happen in cycles. Any time there’s a big awareness push that actually makes progress, like this year’s #MeToo, the exclusionary remarks get a little more coded for a few years. As @tedu said, people stop saying stuff, and maybe slightly reduce how they think it… but the same people are still around.

                                                                                      Then when enough time has passed that the recent movement starts to fade in everybody’s memories, the jokes get more frequent and more obvious again.

                                                                                      I want to clarify that these “jokes” never go away. They change to be more subtle, so that people who aren’t the targets don’t spot them as much. And the venues change so that exclusionary behavior happens mostly in places where either no targets are going to see it to call it out, or no non-targets are going to witness it (ie 1:1 conversations).

                                                                                      This particular example is about gender, but a similar cycle happens with all forms of bigotry.

                                                                                    1. 7

                                                                                      This has been submitted here before, both via a link to the same post in 2016, and as a video in 2017. I’ve added “(2004)” to the post title per our policy on historical articles, but I don’t see much value to actually merging the stories given the time gap - it would just prevent new discussion from taking place.

                                                                                      On a personal, non-official note, I really appreciated the commentary on the 2016 posting about how this story relates to labor practices. I gave it a bunch of thought after that thread and concluded that my emotional reaction was failing to take the consequences into account. I encourage people to go read it. :)

                                                                                      1. 3

                                                                                        I thought this was going to be Sailor Scouts and I am VERY disappointed. Which sailor scout is Ruby? Haskell? Perl? The world needs to know!

                                                                                        1. 2

                                                                                          Ruby = Mars (It’s because they both fit a red theme. Rei is unsubtle like that.)

                                                                                          Haskell = Uranus (Because of the inner sadness, and the commitment to keep going anyway.)

                                                                                          Perl = Mercury (Not necessary often, but always there and part of the team, and very powerful when she’s needed.)

                                                                                          I agree with zdsmith’s point that the gendered language is distracting. I like your approach to dealing with that, coming up with an alternate headcanon. :) So I decided to help with it. :)

                                                                                        1. 11


                                                                                          It’s my personal opinion (I haven’t synced with @pushcx about this) that allowing image embeds was a bad idea - I’d go so far as to say irresponsible, as several of you have. It opened the way to privacy violations of the type @liwakura’s post exemplifies, as well as wasting people’s bandwidth. I’m actually a bit surprised that the bandwidth is the bigger concern for most of you, but that’s my personal bias.

                                                                                          With that said, as somebody who reviews a lot of launches, I know that it’s always easy to miss things. It’s always about asking myself “what’s missing from this picture” - what part of the implications isn’t part of the write-up, which is going to be a surprise later. I wasn’t in the loop about the April Fools theme change, but I don’t blame @pushcx for not thinking about the implications of allowing embeds. It was one small detail in a much bigger effort, and it’s a lot more obvious in hindsight than it would have been while writing it.

                                                                                          Catching every negative consequence of a new feature is a lot of work, and I imagine it was overshadowed by the work of building something meant to be fun - it must have been a significant amount of engineering work to build. I hope some of you did enjoy it. I personally didn’t like the UX, but I thought it was hilarious, and I probably would have agreed with the general concept if I’d been asked.

                                                                                          I ask everyone to try not to argue with each other. Yes, mistakes were made. We’ll have to talk through what action is appropriate as mods, if any. Meanwhile, I ask people to show empathy for each other and not let this devolve into arguments. I promise that your concerns have been heard.



                                                                                          1. 7

                                                                                            This is a solid roundup. I’m sorry I didn’t think to proxy the images, I missed the privacy issue. @liwakura missed the consequences of his prank and has apologized and, no, I’m not going to ban him for it.

                                                                                            1. 6

                                                                                              In addition to what Irene said, i want to apologize for the harm i caused to several users. Mistakes were made, forgetting about mobile crustaceans was one of it.

                                                                                              For the people worrying about the data: My logs are stripped of the last 8 bits of IPv4 and last 64 bits of IPv6 addresses. The data points i have are thus not traceable to your home or phone.

                                                                                              I also want to encourage the community to keep calm, i’ll be cooperating with the staff to address open concerns.

                                                                                              1. 2

                                                                                                Thanks for taking time to address this. I’ll leave it to yall as to how.

                                                                                                Far as you wondering about data used vs stolen, many folks (me included) assume about anything online might get hit by hackers at some point. Double true if it’s not designed for security like a forum software. We just hope to be notified so we can change passwords, tell friends why they’re getting odd emails, etc. Whereas, data use on mobile is something that might cost us money directly or even cut off our ability to receive important communications.

                                                                                                So, at least for those like me, we’d find a data leak (esp non-malicious) to be eyerolling or irritating with its hypotheticals whereas massive data use might do real damage. This time I was lucky enough to have a good plan. :)

                                                                                                1. 2

                                                                                                  That explanation makes sense. Thank you.

                                                                                                2. 2

                                                                                                  I get your point, but maybe we did not need the “launch” in question at all. I personally find all these Aprils fools things super annoying. Maybe less is more and next year lobste.rs is not participating. That would be great.

                                                                                                  1. 3

                                                                                                    I definitely consider that a valid option. I feel bad telling other people not to have fun, but I’m not really a fan of April Fool’s.

                                                                                                    I can promise your view is noted and will be weighed for next year.

                                                                                                1. 2

                                                                                                  894 distinct user agents were spotted 4646 distinct IP addresses

                                                                                                  Wow…. that seems…. odd.



                                                                                                  That would suggest most of those distinct IP addresses are bots and crawlers of some ilk.

                                                                                                  Although I probably would show up as a firefox browser and a feedbro feed reader.

                                                                                                  1. 3

                                                                                                    Lobsters gets its fair share of bots and I got the impression they stepped up their crawling with so much “content change”.

                                                                                                    If folks are curious about these sorts of stats, they can write queries I’ll run on prod logs.

                                                                                                    1. 2

                                                                                                      Could also be IPv6 with privacy extensions.

                                                                                                      1. 1

                                                                                                        I think its caused by the fact that many users have identical user agent strings.

                                                                                                        1. 1

                                                                                                          Well, no, that’s what is odd.

                                                                                                          That’s about 5 ip address per user agent.

                                                                                                          If one made the reasonable assumption everybody is on maybe the one of the later firefox, internet exploder or opera browsers. Ok. Let’s be generous assume each of the major browsers each have maybe 5 versions represented… that’s about 50 different user agents.

                                                                                                          Usage share of all browsers

                                                                                                          Chrome |57.46% Safari |14.39% UC |7.91% Firefox| 5.5% Opera |3.69% IE |3.06% Samsung Internet | 2.92% Edge |1.86% Android |1.72% Others |1.47%

                                                                                                          Still suggests to me a lot of things other than humans are reading lobste.rs

                                                                                                          1. 1

                                                                                                            User agent strings are highly distinctive. They tend to include exact point releases of browsers, OSes, and often multiple shared libraries. These numbers look typical to me.

                                                                                                      1. 23

                                                                                                        GitHub URLs are pretty badly designed.

                                                                                                        For example, /contact is their contact page, and /contactt is a user profile.

                                                                                                        Apparently, there’s a hardcoded list of ”reserved words” in the code, and when someone adds a new feature, they add the word/path segment there and check that it’s not taken by a user.

                                                                                                        So it could perhaps be the case that they’re adding some feature related to malware?

                                                                                                        1. 13

                                                                                                          That could very well be the case – and I’d be totally fine with that. I understand being coded into a corner, and wanting to fix things for the greater good at the expense of a few users.

                                                                                                          I just can’t figure out why, for the sake of “privacy and security”, they don’t want to tell me.

                                                                                                          1. 16

                                                                                                            I think this is absurd behavior on GitHub’s part, and you’re right to be upset by it.

                                                                                                            Since you do seem curious, I have a guess why they’re being so evasive, and it’s pretty simple: They’re a large organization. The person you’re talking to would probably need to get approval from both legal and PR teams to tell you about their product plan before it’s launched. I have no information on how busy GitHub’s lawyers and PR people are, but I would expect an approval like that to take a few weeks. Based on what they told you about the timeframe, it sounds like they want to launch their feature sooner than that.

                                                                                                            What I’d really like to know is whether this is a one-off, or whether they’ve done it to other people before. It seems like their URL scheme will require it pretty frequently…

                                                                                                            1. 7

                                                                                                              The person you’re talking to would probably need to get approval from both legal and PR teams to tell you about their product plan before it’s launched.

                                                                                                              Which is why I didn’t single out the support representative that contacted me; they clearly were not in the decision process for any of this, and I don’t want to cause them any undue grief/trouble past my first email reply asking for clarification.

                                                                                                              To be clear: I don’t really care about the malware username, other than it’s a pretty cool name. I’m more interested in the reason behind why the forced rename.

                                                                                                              Lots of people (read: salty News of Hacker commenters) say it’s obvious (wanting to reserve the /malware top level URL) and call me dumb for even asking, but no one has given me any evidence other than theories and suppositions. Which is great! I love thinking and hypothesizing.

                                                                                                              1. 5

                                                                                                                I don’t have any documented evidence other than anecdotal, but when I worked at a similar company with an almost identical URL structure this was one of the hardest parts of launching a new top level feature. It turns out recognizable words make for good usernames… so it’s almost impossible to find one that’s still available when working on a new feature. The choice ends up being between picking a horrible URL or displacing one user to make it easier to find.

                                                                                                                It’s also worth noting that GitHub has a habit of being very secretive about what they’re working on - it’s almost impossible to get information about known bugs which have been reported before, let alone information about a potential new feature.

                                                                                                                I would be willing to bet that this is being done for something we’ll hear about in the next year or two.

                                                                                                          2. 11

                                                                                                            We made a team that was just the unicode pi symbol and GitHub assigned us the url /team/team.

                                                                                                            1. 4

                                                                                                              That’s a great unicode hack.

                                                                                                            2. 11

                                                                                                              The curse of mounting user paths directly to /. When in doubt, always put a namespace route on it.

                                                                                                              1. 6

                                                                                                                That was my thought as well. I would imagine they want it as a landing page for some new feature or product.