1. 75
  1. 15

    Great info, just a bit annoying it’s via twitter. Here’s the complete text from all the tweets:

    As of today, we have about eighteen years to go until the Y2038 problem occurs.

    But the Y2038 problem will be giving us headaches long, long before 2038 arrives.

    I’d like to tell you a story about this. One of my clients is responsible for several of the world’s top 100 pension funds.

    They had a nightly batch job that computed the required contributions, made from projections 20 years into the future.

    It crashed on January 19, 2018 — 20 years before Y2038. No one knew what was wrong at first.

    This batch job had never, ever crashed before, as far as anyone remembered or had logs for.

    The person who originally wrote it had been dead for at least 15 years, and in any case hadn’t been employed by the firm for decades. The program was not that big, maybe a few hundred lines.

    But it was fairly impenetrable — written in a style that favored computational efficiency over human readability.

    And of course, there were zero tests. As luck would have it, a change in the orchestration of the scripts that ran in this environment had been pushed the day before.

    This was believed to be the culprit. Engineering rolled things back to the previous release.

    Unfortunately, this made the problem worse. You see, the program’s purpose was to compute certain contribution rates for certain kinds of pension funds.

    It did this by writing out a big CSV file. The results of this CSV file were inputs to other programs.

    Those ran at various times each day. Another program, the benefits distributor, was supposed to alert people when contributions weren’t enough for projections.

    It hadn’t run yet when the initial problem occurred. But it did now. Noticing that there was no output from the first program since it had crashed, it treated this case as “all contributions are 0”.

    This, of course, was not what it should do.

    But no one knew it behaved this way since, again, the first program had never crashed. This immediately caused a massive cascade of alert emails to the internal pension fund managers.

    They promptly started flipping out, because one reason contributions might show up as insufficient is if projections think the economy is about to tank. The firm had recently moved to the cloud and I had been retained to architect the transition and make the migration go smoothly.

    They’d completed the work months before. I got an unexpected text from the CIO: https://pbs.twimg.com/media/EOrKh7AXsAETk4e?format=jpg&name=360x360

    S1X is their word for “worse than severity 1 because it’s cascading other unrelated parts of the business”.

    There had only been one other S1X in twelve months. I got onsite late that night. We eventually diagnosed the issue by firing up an environment and isolating the script so that only it was running.

    The problem immediately became more obvious; there was a helpful error message that pointed to the problematic part. We were able to resolve the issue by hotpatching the script.

    But by then, substantive damage had already been done because contributions hadn’t been processed that day.

    It cost about $1.7M to manually catch up over the next two weeks. The moral of the story is that Y2038 isn’t “coming”.

    It’s already here. Fix your stuff. ⏹️

    Postscript: there’s lots more that I think would be interesting to say on this matter that won’t fit in a tweet.

    If you’re looking for speakers at your next conference on this topic, I’d be glad to expound further. I don’t want to be cleaning up more Y2038 messes! 😄

    1. 15

      Would be nice if this post could be more about the 2038 issues than about twitter.

      1. 2

        Was just thinking that, every time an article twitter stream is shared more than half the comments are on the medium. While I agree its distracting I also think taking issue with the medium is not unfounded. People often take issue with the medium, be it too many ads, bloated, or some other issue. If something distracts from the content then it is an issue.

      2. 36

        Twitter is a terrible platform for (among other things) long hand writing, it’s unreadable. I wish people would stop using it for that.

        1. 27

          Unlike the other folks who are defending Twitter, I think Twitter isn’t a great format for consuming content, but I also think it reduces friction for producing content, which results in a lot of great content that wouldn’t otherwise exist. For example, foone lays out his reasons for posting on Twitter here:

          Not to humblebrag or anything, but my favorite part of getting posted on hackernews or reddit is that EVERY SINGLE TIME there’s one highly-ranked reply that’s “jesus man, this could have been a blog post! why make 20 tweets when you can make one blog post?”


          I have ADHD. I have bad ADHD that is being treated, and the treatment is NOT WORKING TERRIBLY WELL. I cannot focus on writing blog posts. it will not happen.

          if I try to make a blog post, it’ll end up being abandoned and unfinished, as I am unable to edit it into something readable and postable. so if I went 100% to blogs:

          You would get: no content

          I would get: lots of unfinished drafts and a feeling of being a useless waste

          but I can do rambly tweet threads. they don’t require a lot of attention for a long time, they don’t have the endless editing I get into with blog posts, I can do them. I do them a bunch! They’re just rambly and twitter, which some people don’t like.

          I’ve tweeted about it before, but I’ve been told this is “unprofessional” and I’m sorry if this surprises you, but I am not a professional writer. of course my hobby writing is going to be unprofessional. Cause it turns out there’s good reasons I’m not a professional writer, and the main one is that I can’t do it.

          Note that this also address the one direct criticism in the other sub-thread, broken sentences.

          Personally, if my options are “read foone’s writing on Twitter (or via something that re-writes his Twitter threads)” or “not be able to read foone’s writing”, I’ll choose the former.

          For some authors, like @jxxf, this is less extreme. He has a blog that he writes on, but the tradeoff is still the same at the margin: because he has a low friction platform available to him, he writes a lot more than he otherwise would and I’m glad he does.

          I view this the same way I view complaints that making programming easier has dumbed down programming because (for example) people no longer have to understand pointers to produce useful software. I think that’s great, on balance, even if there are some downsides.

          For people who dislike Twitter so much that they would rather have no content than content on Twitter, they can make that choice for themselves: when they see a twitter.com link, they can not click on it. Some people would prefer to make that choice for other people (as of the time of this writing, this post has downvotes for “spam”, “off topic”, and “broken link”, which are all incorrect IMO) and I don’t agree with that. From looking at lobsters content from twitter, it seems like maybe 2-4 twitter posts get traction each month? IMO, it’s preferable that people who don’t want to read content on Twitter click “hide” 2-4 times a month than to not have this content exist at all or get flagged off of lobsters.

          1. 7

            I’ve read this guy’s reasoning before and I just don’t believe it. The guy may have adhd, I have no reason to doubt him, but I think anyone capable of writing super long tweet messages and doing in-depth device break downs can be taught to write blog posts one paragraph at a time.

            Of course, the dude should write however he likes, I’m not his boss. He may believe what he says, but I think his reason is not real.

            I agree with your “don’t click on it.” approach as what I love about the internet is our ability to choose and filter.

            I don’t comment out of a wish to squash this guys stuff and I certainly don’t have to read it. But I comment because I want to understand why people write this way, and want to discourage others from writing this way, as overall it reduces the amount of content people can read, I think. Since this way takes longer to read, is hard to share and respond to thoughtfully.

            1. 2

              I have ADHD and I can relate really strongly to the sentiment. My thoughts don’t flow in a natural stream, it’s more spurts and fits. I can sit down and write a blog post, sure, but I’ll be sitting around writing it for 30+ minutes (an eon!) before I either a) get bored, and decide the point I’m trying to make isn’t worth making anyway, or b) realize my thoughts really don’t sound as good as I thought they did, even if they may have some merit. In the end, the blog post ends up getting deleted either way.

              I’d say using Twitter over a blogpost is more writer-centric than reader-centric, which obviously hinders the audience, but the way foone puts in the previous post really resonates with me:

              if I try to make a blog post, it’ll end up being abandoned and unfinished, as I am unable to edit it into something readable and postable. so if I went 100% to blogs:

              You would get: no content

              I would get: lots of unfinished drafts and a feeling of being a useless waste

              It’s all about focus, collecting thoughts, and forming them into a singular, discrete narrative - which just does not jive with ADHD behavior.

              1. 2

                Agreed as a fellow. My draft folder is a nightmare, my writing is comment sections and tweets.

                Even if I write a blog post it becomes painful, as I usually hyperfocus on it, will spend multiple days on it and end up rereading it too often out of fear for small errors. Tweet chains are much more permissive and raw.

                The guy may have adhd, I have no reason to doubt him, but I think anyone capable of writing super long tweet messages and doing in-depth device break downs can be taught to write blog posts one paragraph at a time.

                I hate to say it, but if I had a dime for every time a neurotypical person just utters disbelief about how my brain works, I’d have a ton of money on the side.

                These effects are very real.

                1. 1

                  I called out the ability to train. I understand ADHD really well. What I meant is that it’s quite possible to train on a writing technique where I write only one sentence or paragraph at a time.

                  That’s not unique to twitter. I can do the same thing in WordPress or google docs or many other tools that I can adapt to use.

                  Maybe this author can’t do it, but they can learn to produce content in a way that is still possible given their medical condition.

                  I was disputing the statement that the author can’t or that it’s impossible given ADHD. I don’t think that’s accurate. Maybe The author doesn’t want to train or learn. That’s cool, that is their prerogative. But the reason they aren’t doing it isn’t ADHD, it’s that the author likes it that way and is unwilling to change.

                2. 2

                  Eh I can believe it. I have ADHD. Sometimes the things you can do, and the things you can’t do, don’t make any goddamn sense. ADHD is an executive dysfunction disorder. Shorter tasks do help…

                3. 1

                  Not an unreasonable insight, but on the other hand if you can write 20 tweets, you can also write the same 20 tweets in a .html page?

                  By biggest gripe with Twitter is that it doesn’t work in Firefox for whatever reason. Clicking “Show this thread” just doesn’t seem to do anything. Probably due to some Firefox setting I changed or whatnot, but it’s really hard to figure out what (I discovered that copy/paste was broken in Twitter because I disabled the clipboard API, took me a long time to figure out). It can also takes ages to load (although not ridiculously slow, my connection in Indonesia is somewhat low bandwidth/high latency).

                  It’s all about what your goal is, if you want to just “tweet out in the universe” then by all means, go ahead. But if you want to reach people, then a blog post is probably more effective.

                  Perhaps this is an opportunity for creating better blogging software that combines the best of both 🤔

                4. 9

                  I like it as a format because it forces you to be concise and make every 280 characters a self-contained point. It’s a fun writing constraint.

                  1. 13

                    The endless scrolling, broken sentences (that most attempts have) and crappy twitter experience when JS is disabled doesn’t make it fun to read.

                    1. 15

                      I’d suggest maybe turning JS on to make your UX better, but I feel like you’re not going to be receptive to that idea.

                      1. 5

                        I suggest nitter, for a far more usable and less hostile experience, without requiring JS:


                        1. 1

                          Wow - JS or not, that’s a really nice UI.

                        2. 4

                          Ok, even with JS on, it’s still a bunch of poorly separated blocks of sentences and sentence fragments.

                          The comment above mentions a ‘fun writing constraint’.

                          Writers, find a different way to punish yourselves without punishing your readers. The number of hours a writer spends on a piece is almost certainly less (much, much less) than the number of combined hours that readers spend reading it. Don’t drag your readers through shit for “fun.”

                          1. 2

                            Why not? Nobody is forcing you to read it.

                            If someone wants to have fun writing in a weird format, that’s their prerogative. They don’t have to be maximising your utility.

                            That said, consider just using something like this: https://nitter.net/jxxf/status/1219009308438024200

                          2. 1

                            I disagree that JS should be required to be treated decently on a website.

                        3. 6

                          This isn’t a constraint if it’s split over 50 tweets.

                          This tweet stream is an example of how things aren’t self-obtained points. Almost all of these tweets are meaningless without context before and after.

                          This story seems made up, like a business fable and even more so because it’s tweeted out in small amounts.

                          This was interesting the first few times I saw it 13 years ago. But now it’s weird and confusing and shows how valuable friends and editors are to the writing process.

                          I read a novel in 1999 where the entire story took place in emails. It was an interesting constraint. But the same book today would not be original enough to outweigh the awkwardness of such a constraint.

                          I am not sure what’s up with these “tweet storms.” I guess they are best for online brands or something since people can like and tweet each item. This author seems quite successful with hundreds of interactions per message. So maybe it makes more money, or sells more ads, or just shows which paragraphs are best liked.

                          But I just wonder if the author is capable of setting up a blog.

                          I liked the story, but it took much longer to try to read because of all the scrolling. It’s also harder to read because the lack of paragraphs and grammar and links.

                          1. 1

                            Is Exegesis by Astro Teller the book you are thinking of?

                            1. 1

                              That looks pretty cool, but not it. I’ve added it to my to read pile.

                              I was referring to E: a Novel by Matt Beaumont. It was set in present day (2000) office place and was just a simple, relatively ok office drama.

                          2. 0

                            I am with you. There is something engaging about the dramatic separation of paragraphs that each read like headlines. Not great for details, but for short stories I find it fun.

                          3. 2

                            I think there should be automation for mirroring twitter threads in an accessible format.

                            I don’t click Twitter links anymore, because my Twitter experience typically goes as follows.

                            1. Click the twitter.com link on lobsters
                            2. “If you’re not redirected, click this link”
                            3. “It looks like you’re browsing without JS. Do you want to continue to basic site? [Yes]” (Yes is the only option provided here.)
                            4. Get redirected either to the twitter home page or a thread I clicked on 3 days ago on reddit, which was stored in a cookie and did not update correctly.
                            5. Repeat steps 0-2.
                            6. Arrive at the page, start reading the content in between the repeating author name, user name, and irrelevant timestamps.
                            7. Any other Twitter links on that page, go back to step 0.
                            8. Trying to open any images takes me back to either the author’s page, the twitter home page, or some other thread I clicked on recently.

                            I think browsing without JS is valid, and no-JS users deserve to be first-class visitors.

                            1. 1

                              @bandali says:

                              I suggest nitter, for a far more usable and less hostile experience, without requiring JS:


                            2. -1

                              Twitter threads are a de facto standard for publishing and consuming content like this on the internet, the ship has sailed, and it’s just mind-numbingly uninteresting to have to endure replies like this every time one is shared to a content aggregation platform. Get with the times, old man 👴🏻 ✊🏻 ⛅️

                              1. 3

                                De facto standard lol? Since when? Blogging is still alive and well and blog posts outnumber twitter threads 100:1 on lobsters alone.

                            3. 5

                              Old code matters. There’s a lot of old code there.

                              I’m aware this doesn’t exactly apply here, but I have a related rant brewing about this: don’t break old code. A lot of people don’t like to hear this, but it has to be heard.

                              I must enlighten the masses.

                              1. 4

                                I wonder how many engineers wept and felt their soul sink in anticipation of future problems when they saw this script in the meantime… “Heeeey, it works, why do you guys complain!” …riiight, and in fact, actually, everybody’s happy anyway, right? No real incentive to ever behave differently in the future — the original engineer cashed in so long ago that the system literally outlived them; the original managers probably also didn’t complain, the system was working, earned money for The Fund for years; the consultant flown-in got their bounty; the new manager can blame it on the old managers and also probably got a prize for “saving the day” by flying in the consultant. Others had to bite the bullet because they’re too small to complain about The Fund. Cost probably offset to clients, who cares what they even think.