1. 4

    has anyone gotten a blue lobster yet?

    1. 2

      It seems certain given our volume of traffic, but I don’t check the logs for it and they get logrotate’d out after a week or two. Might be fun to add a cron job to the Lobsters repo to grep for them.

      1. 4

        you should make it so the person who gets the blue lobster gets a blue lobster hat

        1. 2

          While I love the idea or a blue lobster hat or even a listing of who & when saw the blue lobster, this however could lead to some people trying to abuse the system to get the achievement and by the nature of how this would go, that process would generate arbitrary load on lobsters.

      2. 1

        rain1’s profile pic was a blue lobster.

      1. 3

        Does anyone know what the spikes in user sign ups correspond to?

        1. 9
          1. 2

            TIL about the invitation queue. Someone should build a site map for the site that details all these pages :-)

            1. 8

              It’s been disabled for years, though the code exists for sister sites.

          2. 2

            The other two are mentions on high-volume sites and people bringing in their buddies. An example of high-volume site that probably feeds over here is Hacker News with 20 million hits a month. We had a bit of cross-posting and shared users which increases exposure to both sites.

            I’m sure some comes from sites posted here whose members end up here. Perhaps a Lobster comments on their site saying they saw it on Lobsters. Traffic follows. I know it happens but don’t have data on what it generates.

            1. 1

              Probably when the site is mentioned/promoted somewhere else.

            1. 1

              I battle my bipolar disorder (very successfully) with physical exercise. I cannot do that anymore.

              It’s going exactly how you think it would be going. 🙃

              1. 1

                Did you mean you can’t go to the gym or something happen to you physically that prevents you from exercising?

                If the first, I found some benefit in using resistance bands. I’ve been using them on a tree outside whose angle lets me do my arms at least. You might be able to wrap them around something heavy in your house if nothing outside. Just be careful doing that.

                1. 2

                  I’ve actually been able to do some things at home (I own a set of resistance bands and a medicine ball for at-home exercising) but my primary source of exercising pre-pandemic was ice hockey which got me ~200 BPM exercises that didn’t feel like “work”. I get by with the at-home stuff, but the hockey nearly eliminated my symptoms.

                  1. 2

                    Ah that makes sense. Hopefully this stuff clears up soon so you can get back to hockey.

                    1. 2

                      I’ve had some luck with VR games for “workout without it feeling like work”. Hard to keep enough space clear though.

                1. 1

                  Physically, I put on the quarantine 15 because a vacation segued directly into self-isolation for 14 days after a likely exposure while attending a 5,000 person event from which three people were hospitalized with COVID-19 during that vacation. I’d suspended my diet for that vacation knowing that I’d put on a few pounds that I could work off in a few weeks. I did not expect to need to plan for a nigh post-apocalyptic diet as leaving the house became risky. Carbs were eaten. Many. I worked off about 5 of those pounds in the months following but I’m back up following a switch back to the keto diet I’ve ~maintained for 6+ years. I’m looking forward to fitting back into clothes I could wear when I was 15 lbs lighter for all of 2019.

                  Mentally, I’m doing OK. Some living situation changes are normalized now as a family member moved in with us because of her live-in job going away at the start of the pandemic. I’m happy to have them around but it’s showing that our house barely fits three adults. My work is slow but steady. One of my non-profits has really benefitted from the pandemic while another one is essentially suspended as the tech conference industry has gone “free and online”. My side business has lost about 40% of its revenue with no upward force in sight. We have savings that will keep our heads at the water line but if conditions continue into next year, we’ll have to drastically alter our business plan to keep the business alive. We’re already looking at other ways to generate revenue as being a primarily mendicant operation is fraught with revenue unreliability. Altogether, I’m probably about even but it’s not been without ups and downs, wins and losses, and some long conversations about the future.

                  I wish I had more time to focus on me, but servant leadership is the path I’ve chosen and it is not one easily paused or exited.

                  1. 2

                    Props for looking after others with the self-isolation and going for servant leadership. Hope and pray you do well with all this.

                  1. 3

                    Ok, so I was an ex-Christian that ditched the Bible due to science, morals, etc. That’s despite apparent miracles happening with my family at times. Turned into a good guy who woild sacrifice enormously for others but also plenty raunchy, argumentative, etc. Most people liked me. Burnout from a job was so stressful others were breaking out crying, falling out on cars, etc. I could take it despite PTSD by using a combo of breathing, positive attitude, and tough experience.

                    Was deep in burnout for long time with days blending together. Prolly had liver, heart, and cancer problems on the way. Plus, anyone real never really leaves. Called out to unknown God that if they exist and want me back to give me a little time to pull myself up and Id bring others up with me.

                    High-talent people popped up outta nowhere binded by all kinds of coincidences. Mostly went well. One was damaged and needed help which I gave. Stayed in tons of prayer. Situation kept challenging me to change super-fast to help them. I get blindsided by being disowned, then a fake stalking claim (our 1-on-1’s were a setup), and fake sexual harrassment claim. About to go to court, Lord said hold off: “I gotcha.” I did hesitantly. Within days, she ended it with a deal splitting us up with nothing on my record. My mgmt went “Wth?!”

                    I wondered what I was being prepared for. Next shift was coronavirus. Skeleton crew with hours non-stop of desparate, angry people. In Christ, I was only person at peace (stressed though!). I took the worst calls, calming them down. We made it. I’ve served as many as I can since.

                    Next tests were simpler. I handled a highly-privileged bully with patience and professionslism vs going ham. They escalated. Prayed on that. Corporate moved them in way that nobody has ever seen happen.

                    A relative had let someone move in free to have money for bail. That person turned into total bum for many months. They were too loving to kick them out. I prayed hard for them while planning a response. Like the “Then Satan entered him verses,” the guy suddenly went nuts, tried to get their landlord to evict them with wild stories, and that got him kicked out. They were confused until I said it matched my specific prayer for enemies.

                    Most were good. One that tested me was a guy got destroyed before my eyes by claim like first psycho, I forgave/blessed that enemy, and they got a house and new job out of state. Hmm… Still praying they transform then…

                    So, lots of stuff like this. I started with prayer. Professed faith again later. Back into being righteous. Using tons of energy to have servant attitude toward everyone, love even the haters, kick mental immorality common in summer, get in Scripture, give to who needs, and pray without ceasing for many I encounter.

                    Most PTSD symptoms and insomnia are minimal at the moment. I’m at this stuff from 5a-6:30a to midnight many nights. Tired but in a good way. The HR person that dealt with the people above is now my direct superior with them still here. Next test is on the way. Good that in my corner is My Heavenly Father and Lord Jesus Christ with a Holy Spirit sustaining me in 13hr sprint shifts. I’ll be blessed either way. I’ll also try to pray for any here that request it where I have time. :)

                    1. 23

                      It boggles my mind that there are more and more websites that just contain text and images, but are completely broken, blank or even outright block you if you disable JavaScript. There can be great value in interactive demos and things like MathJax, but there is no excuse to ever use JavaScript for buttons, menus, text and images which should be done in HTML/CSS as mentioned in the blog post. Additionally, the website should degrade gracefully if JavaScript is missing, e.g. interactive examples revert to images or stop rendering, but the text and images remain in place.

                      I wonder how we can combat this “JavaScript for everything” trend. Maybe there should be a website that names and shames offending frameworks and websites (like https://plaintextoffenders.com/ but for bloat), but by now there would probably be more websites that belong on this list than websites that don’t. The web has basically become unbrowsable without JavaScript. Google CAPTCHAs make things even worse. Frankly, I doubt that the situation is even salvageable at this point.

                      I feel like we’re witnessing the Adobe Flash story all over again, but this time with HTML5/JS/Browser bloat and with the blessing of the major players like Apple. It’ll be interesting to see how the web evolves in the coming decades.

                      1. 5

                        Rendering math on the server/static site build host with KaTeX is much easier than one might have thought: https://soap.coffee/~lthms/cleopatra/soupault.html#org97bbcd3

                        Of course this won’t work for interactice demos, but most pages aren’t interactice demos.

                        1. 9

                          If I am making a website, there is virtually no incentive to care about people not allowing javascript.

                          The fact is the web runs on javascript. The extra effort does not really give any tangible benefits.

                          1. 21

                            You just proved my point. That is precisely the mechanism by which bloat finds its way into every crevice of software. It’s all about incentives, and the incentives are often stacked against the user’s best interest, particularly if minorities are affected. It is easier to write popular software than it is to write good software.

                            1. 7

                              Every advance in computers and UI has been called bloat at one time or another.

                              The fact of the matter is that web browsers “ship” with javascript enabled. A very small minority actually disable it. It is not worth the effort in time or expense to cater to a group that disables stuff and expects everything to still work.

                              Am I using a framework?

                              Most of the time, yes I am. To deliver what I need to deliver it is the most economical method.

                              The only thing I am willing to spend extra time on is reasonable accommodation for disabilities. But most of the solutions for web accessibility (like screenreaders) have javascript enabled anyhow.

                              You might get some of what you want with server side rendering.

                              Good software is software that serves the end user’s needs. If there is interactivity, such as an app, obviously it is going to have javascript. Most things I tend to make these days are web apps. So no, Good Software doesn’t always require javascript.

                              1. 10

                                I actually block javascript to help me filter bad sites. If you are writing a blog and I land there, and it doesn’t work with noscript on, I will check what domains are being blocked. If it is just the one I am accessing I will temp unblock and read on. If it is more than a couple of domains, or if any of them are unclear as to why they need to be loaded, you just lost a reader. It is not about privacy so much as keeping things neat and tidy and simple.

                                People like me are probably a small enough subset that you don’t need our business.

                                1. 4

                                  Ah, the No-Script Index!

                                  How many times does one have to click “Set all this page to temporarily trusted” to get a working website? (i.e. you get the content you came for)

                                  Anything above zero, but definitely everything above one is too much.

                                  1. 3

                                    The absolute worst offender is microsoft. Not only is their average No-Script index around 3, but you also get multiple cross site scripting attack warnings. Additionally when it fails to load a site because of js not working it quite often redirects you to another page, so set temp trusted doesn’t even catch the one that caused the failure. Often you have to disable no-script altogether before you can log in and then once you are logged in you can re-enable it and set the domains to trusted for next time.

                                    That is about 3% of my total rant about why microsoft websites are the worst. I cbf typing up the rest.

                                  2. 3

                                    i do this too, and i have no regrets, only gratitude. i’ve saved myself countless hours once i realized js-only correlates heavily with low quality content.

                                    i’ve also stopped using medium, twitter, instagram, reddit. youtube and gmaps, i still allow for now. facebook has spectacular accessibility, ages ahead of others, and i still use it, after years away.

                                    1. 1

                                      My guess is that a lot of people who use JS for everything, especially their personal blogs and other static projects, are either lazy or very new to web development and programming in general. You can expect such people to be less willing or less able to put the effort into making worthwhile content.

                                      1. 2

                                        that’s exactly how i think it work, and why i’m happy to skip the content on js-only sites.

                                  3. 6

                                    The only thing I am willing to spend extra time on is reasonable accommodation for disabilities.

                                    Why do you care more about disabled people than the privacy conscious? What makes you willing to spend time for accommodations for one group, but not the other? What if privacy consciousness were a mental health issue, would you spend time on accommodations then?

                                    1. 12

                                      Being blind is not a choice: disabling JavaScript is. And using JavaScript doesn’t mean it’s not privacy-friendly.

                                      1. 4

                                        It might be a “choice” if your ability to have a normal life, avoid prison, or not be executed depends on less surveillance. Increasingly, that choice is made for them if they want to use any digital device. It also stands out in many places to not use a digital device.

                                        1. 2

                                          This bears no relation at all to anything that’s being discussed here. This moving of goalposts from “a bit of unnecessary JavaScript on websites” to “you will be executed by a dictatorship” is just weird.

                                          1. 4

                                            You framed privacy as an optional choice people might not need as compared to the need for eyesight. I’d say people need sight more than privacy in most situations. It’s more critical. However, for many people, privacy is also a need that supports them having a normal, comfortable life by avoiding others causing them harm. The harm ranges from social ostracism upon learning specific facts about them to government action against them.

                                            So, I countered that privacy doesn’t seem like a meaningless choice for those people any more than wanting to see does. It is a necessity for their life not being miserable. In rarer cases, it’s necessary for them even be alive. Defaulting on privacy as a baseline increases the number of people that live with less suffering.

                                            1. 2

                                              You framed privacy as an optional choice

                                              No, I didn’t. Not even close. Not even remotely close. I just said “using JavaScript doesn’t mean it’s not privacy-friendly”. I don’t know what kind of assumptions you’re making here, but they’re just plain wrong.

                                              1. 3

                                                You also said:

                                                “Being blind is not a choice: disabling JavaScript is.”

                                                My impression was that you thought disabling Javascript was a meaningless choice vs accessibility instead of another type of necessity for many folks. I apologize if I misunderstood what you meant by that statement.

                                                My replies don’t apply to you then: just any other readers that believed no JS was a personal preference instead of a necessity for a lot of people.

                                        2. 3

                                          The question isn’t about whether it’s privacy-friendly, though. The question is about whether you can guarantee friendliness when visiting any arbitrary site.

                                          If JS is enabled then you can’t. Even most sites with no intention of harming users are equipped to do exactly that.

                                          1. -1

                                            disabling js on a slow device is not a choice, but required for functioning. you are basically saying fuck you to all the disadvantaged.

                                            and all because you are being lazy.

                                            1. 4

                                              When you can get a quad core raspberry pi for $30 and similar hardware in a $50 phone, I really doubt that there are devices that can’t run most JS sites and someone who has a device of some sort can’t afford.

                                              What devices do you see people using which can’t run JS?

                                              The bigger question in terms of people being disadvantaged is network speed, where some sites downloading 1MB of scripts makes them inaccessible - but that’s an entirely separate discussion.

                                              1. 1

                                                how is that a separate discussion? it’s just one more scenario when js reduces accessibility.

                                                as for devices, try any device over 5 years old.

                                              2. 2

                                                I literally have the cheapest phone you can buy in Indonesia (~€60) and I have the almost-cheapest laptop you can buy in Indonesia (~€250). So yeah, I’d say I’m “disadvantaged”.

                                                Turns out, that many JavaScript sites work just fine. Yeah, Slack and Twitter don’t always – I don’t know how they even manage to give their inputs such input latency – but Lobsters works just fine (which uses JavaScript), my site works just fine as well (which uses JavaScript), and my product works great on low-end devices (which requires JavaScript), etc. etc. etc.

                                                You know I actually tried very hard to make my product work 100% without JavaScript? It was a horrible experience for both JS and non-JS users and a lot more code. Guess I’m just too lazy to make it work correct 🤷‍♂️

                                                So yeah, please, knock it with this attitude. This isn’t bloody Reddit.

                                                1. 6

                                                  “I literally have the cheapest phone you can buy in Indonesia (~€60) and I have the almost-cheapest laptop you can buy in Indonesia (~€250). So yeah, I’d say I’m “disadvantaged”. Turns out, that many JavaScript sites work just fine.”

                                                  I’ve met lots of people in America who live dollar to dollar having to keep slow devices for a long time until better hand-me-downs show up on Craigslist or just clearance sales. Many folks in the poor or lower classes do have capable devices because they would rather spend money on that than other things. Don’t let anyone fool you that being poor always equals bad devices.

                                                  That said, the ones taking care of their families, doing proper budgeting, not having a car for finding deals, living in rural areas, etc often get stuck with bad devices and/or connections. I don’t have survey data on how many are in the U.S.. I know poor and rural populations are huge, though. It makes sense that some people push for a baseline that includes them when the non-inclusive alternative isn’t actually even necessary in many cases. When it is, there were lighter alternatives not used because of apathy. I’ve rarely seen situations where what they couldn’t easily use was actually necessary.

                                                  The real argument behind most of the sites is that they didn’t care. The ones that didn’t know often also didn’t care because they didn’t pay enough attention to people, esp low-income, to find out. If they say that, the conversations get more productive because we start out with their actual position. Then, strategies can be formed to address the issue in an environment where most suppliers don’t care. Much like we had to do in a lot of other sectors and situations where suppliers didn’t care about human cost of their actions. We got a lot of progress by starting with the truth. The web has many, dark truths to expose and address.

                                                  1. 3

                                                    thank you for writing this out. the cheapest new phone in indonesia is probably much faster than your typical “obamaphone” or 3-year-old average device.

                                                    1. 1

                                                      The Obama phones are actually Android devices that also have pre-installed government malware that can’t be removed. They have Chrome and run JS fine.

                                                      1. 2

                                                        They have Chrome, and they run JS very slowly.

                                                        1. 1

                                                          Are you going to cite any devices here? Which JS do they run slowly?

                                                          My guess is that the issue is on specific documents. I’d think that the fact that JS is so often used in ways that don’t perform well is a much larger issue than this one. Sites using JS in ways that are slow is a completely different debate to be had in my opinion. Although giving someone a version of the page without JS seems a solution, it ignores the entire concept of progressive web apps and the history of the web that got us to them.

                                                          EG, would you prefer the 2008 style of having a separate m.somesite.com that works without JS but tends to be made for small devices which tends to let corporations be okay with removing necessary functionality to simplify the “mobile experience”? Generally, that’s what we got that solution.

                                                          The fact that even JS-enabled documents like https://m.uber.com allow you to view a JS map and get a car to come pick you up with reasonable performance on even the cheapest burner phones shows just how much bad programming plays into your opinion here instead of simply whether or not JS is the problem itself.

                                                          It’s also worth noting that I am strongly interested in people doing less JS and the web being JS-less, but this isn’t the hill to die on in that battle if you ask me. Not only are you going to generally find people that aren’t sympathetic to disadvantaged people (because most programmers tend to not give any fucks unfortunately) but also because the devices that run JS are generally not going to be slow enough that decent JS isn’t going to run. If we introduce some new standard that replaces HTML, it’ll likely still be read by browsers that still support HTML / JS - which means the issue still remains because people aren’t going to prioritize a separate markup for their entire site depending on devices which is the exact reason that most companies stopped doing m.example.com. The exception to this rule seems to be bank & travel companies in my experience.

                                                          1. 2

                                                            Here is an example device I test with regularly:

                                                            iPad 528LL/A, iOS 9.3.5

                                                            This iPad is less than 10 years old, and still works well on most sites with JS disabled. With JS enabled, even many text-based sites slow it down to the point of being unresponsive.

                                                            This version of iOS and Safari are gracious enough to include a JavaScript on/off toggle under Advanced, but no fine-grained control. This means that every time I want to toggle JS, I have to exit Safari, open Settings, scroll down to Safari, scroll down to Advanced, toggle JS, and then return to Safari.

                                                            Or are you going to tell me that my device is too old to visit your website? I’ll be on my way, then.

                                                            1. 2

                                                              It’s also worth noting that I am strongly interested in people doing less JS and the web being JS-less, but this isn’t the hill to die on in that battle if you ask me. Not only are you going to generally find people that aren’t sympathetic to disadvantaged people (because most programmers tend to not give any fucks unfortunately)

                                                              I think this is changing for the better, slowly but faster more recently.

                                                              but also because the devices that run JS are generally not going to be slow enough that decent JS isn’t going to run. If we introduce some new standard that replaces HTML, it’ll likely still be read by browsers that still support HTML / JS - which means the issue still remains because people aren’t going to prioritize a separate markup for their entire site depending on devices which is the exact reason that most companies stopped doing m.example.com.

                                                              I think with some feature checking and progressive enhancement, you can do a lot. For example, my demo offers basic forum functionality in Mosaic, Netscape, Opera 3.x, IE 3.x, and modern browsers with and without JS. If you have JS, you get some extra features like client-side encryption and voting buttons which update in-place instead of loading a new page.

                                                              I think it’s totally doable, with a little bit of effort, to live up to the original dream of HTML which works in any browser.

                                                              The exception to this rule seems to be bank & travel companies in my experience.

                                                              Facebook (ok, mbasic.facebook.com) MetaFilter Lobste.rs (for reading) old.reddit.com (for reading) Most blogs posted to lobsters and hn are actually nojs-friendly

                                                      2. 3

                                                        Aside from devices without a real browser, JavaScript should run fine on any device people are going to get in 2020 - even through hand-me-downs.

                                                        1. 3

                                                          I’m going to try to replace my grandmother’s laptop soon. I’ve verified it runs unbearably slow in general but especially on JS-heavy sites she uses. It’s a Toshiba Satellite with Sempron SI-42, 2GB of RAM and Windows 7. She got it from a friend as a gift presumably replacing her previous setup. Eventually, despite many reinstalls to clear malware, the web sites she uses were unbearably slow.

                                                          “When you can get a quad core raspberry pi for $30 and similar hardware in a $50 phone,”

                                                          She won’t use a geeky setup. She has a usable, Android phone. She leaves it in her office, occasionally checking the messages. In her case, she wants a nice-looking laptop she can set on her antique-looking desk. Big on appearances.

                                                          An inexpensive, decent-looking, Windows laptop seems like the best idea if I can’t get her on a Chromebook or something. I’ll probably scour eBay eventually like I did for my current one ($240 Thinkpad T420). If that’s $240, there’s gotta be some deals out there in the sub-Core i7 range. :)

                                                          1. 3

                                                            Sure, but just to clarify - we are talking about people who may need to save money to get the $30 for something like a raspberry pi. Not someone who can drop $240 on a new laptop.

                                                            1. 3

                                                              Oh yeah. I was just giving you the device example you asked for. She’s in the category of people who would need to save money: she’s on Social Security. These people still usually won’t go with a geeky rig even if their finances justify it. Psychology in action.

                                                              I do actually have a Pi 3 I could try to give her. I’d have to get her some kind of nice monitor, keyboard, and mouse for it. I’m predicting, esp with the monitor, the sum of the components might cost the same as or more than a refurbished laptop for web browsing. I mentioned my refurbished Core i7 for $240 on eBay as an example that might imply lower-end laptops with good performance might be much cheaper. I’ll find out soon.

                                                          2. 1

                                                            But what about a device people got in 2015 or 2010? Or, dare I say, older devices, which still work fine, and may be kept around for any number of reasons like nostalgia or sentimental attachment?

                                                            Sure, you can tell all these people to also stuff it, but don’t pretend they don’t exist.

                                                  2. 12

                                                    Why do you care more about disabled people than the privacy conscious?

                                                    Oh that one is easy: Its the law.

                                                    Being paranoid isn’t a protected class, it might be a mental health issue - but my website has nothing to do with its treatment.

                                                    For the regular privacy, you have other extensions and cookie management you can do.

                                                  3. 3

                                                    You have some good points. One thing I didn’t see addressed is the number of people on dial-up, DSL, satellite, cheap mobile, or other bad connections. The HTML/CSS-type web pages usually load really fast on them. The Javascript-type sites often don’t. They can act pretty broken, too. Here’s some examples someone posted to HN showing impact of JavaScript loads.

                                                    “If there is interactivity, such as an app, obviously it is going to have javascript. “

                                                    I’ll add that this isn’t obvious. One of the old models was client sending something, server-side processing, and server returns modified HTML. With HTML/CSS and fast language on server, the loop can happen so fast that the user can barely perceive a difference vs a slow, bloated, JS setup. It would also work for vast majority of websites I use and see.

                                                    The JS becomes necessary as the UI complexity, interactivity (esp latency requirements), and/or local computations increase past a certain point. Google Maps is an obvious example.

                                                    1. 3

                                                      It is interesting to see people still using dialup. Professionally, I use typescript and angular. The bundle sizes on that are rather insane without much code. Probably unusable on dialup.

                                                      However, for my personal sites I am interested in looking at things like svelte mixed with dynamic loading. It might help to mitigate some of the issues that Angular itself has. But fundamentally, it is certainly hard to serve clients when you have apps like you mention - Google Maps. Perhaps a compromise is to try to be as thrifty as can be justified by the effort, and load most of the stuff up front, cache it as much as possible, and use smaller api requests so most of the usage of the app stays within the fast local interaction.

                                                      1. 2

                                                        <rant>

                                                        Google Maps used to have an accessibility mode which was just static pages with arrow buttons – the way most sites like MapQuest worked 15 years ago. I can only guess why they took it away, but now you just get a rather snarky message.

                                                        Not only that, but to add insult to injury, the message is cached, and doesn’t go away even when you reload with JS enabled again. Only when you Shift+reload do you get the actual maps page.

                                                        This kind of experience is what no-JS browsers have to put up with every fucking day, and it’s rather frustrating and demoralizing. Not only am I blocked from accessing the service, but I’m told that my way of accessing it itself invalid.

                                                        Sometimes I’m redirected to rather condescending “community” sites that tell me step by step how to re-enable JavaScript in my browser, which by some random, unfortunate circumstance beyond my control must have become disabled.

                                                        All I want to say to those web devs at times like that is: Go fuck yourself, you are all lazy fucking hacks, and you should be ashamed that you participated in allowing, through action or inaction, this kind of half-baked tripe to see the light of day.

                                                        My way of accessing the Web is just as valid as someone’s with JS enabled, and if you disagree, then I’m going to do everything in my power to never visit your shoddy establishment again.

                                                        </rant>

                                                        Edit: I just want to clarify, that this rant was precipitated by other discussions I’ve been involved in, my overall Web experience, and finally, parent comment’s mention of Google Maps. This is not aimed specifically at you, @zzing.

                                                2. 9

                                                  It shouldn’t be extra effort, is the point. If you’re just writing some paragraphs of text, or maybe a contact form, or some page navigation, etc etc you should just create those directly instead of going through all the extra effort of reinventing your own broken versions.

                                                  1. -2

                                                    Often the stuff I am making has a lot more than that. I use front end web frameworks to help with it.

                                                    Very few websites today have just text or a basic form.

                                                    1. 10

                                                      Ok, well, that wasn’t at all clear since you were replying to this:

                                                      It boggles my mind that there are more and more websites that just contain text and images, but are completely broken, blank or even outright block you if you disable JavaScript.

                                                      Many websites I see fit this description. They’re not apps, they don’t have any “behaviour” (at least none that a user can notice), but they still have so much JS that it takes over 256MB of RAM to load them up and with JS turned off they show a blank white page. That’s the topic of this thread, at least by the OP.

                                                      1. 0

                                                        Very few websites today have just text or a basic form.

                                                        Uhh… Personal websites? Blogs? Many of the users here on Lobsters maintain sites like these. No need to state falsehoods to try and prove your point; there are plenty of better arguments you could be making.

                                                        As an aside, have you seen Sourcehut? That’s an entire freakin’ suite of web apps which don’t just function without JavaScript but work beautifully. Hell, Lobsters almost makes it into this category as well.

                                                  2. 1

                                                    Some types of buttons, menus, text and images aren’t implemented in plain HTML. These kinds should still be built in JS. For instance, 3-state buttons. There are CSS hacks to make a button appear 3-state, but no way to define behavior for them without JS. People can hack together radio inputs to look like a single multi-state button, but that’s a wild hack that most developers aren’t going to want to tackle.

                                                    1. 1

                                                      I’m trying to learn more about accessibility, and recently came across a Twitter thread with this to say: “Until the platform improves, you need JS to properly implement keyboard navigation”, with a couple video examples.

                                                      1. 2

                                                        I think that people that want keyboard navigation will use a browser that supports that out of the box, they won’t rely on each site to implement it.

                                                        1. 2

                                                          The world needs more browsers like Qutebrowser.

                                                    1. 9

                                                      Completely tech-unrelated, but I started again to learn to draw, with some more decent resources and peeps to talk about art to. Hope I’ll manage to stay motivated for long enough to not burn out this time

                                                      1. 3

                                                        I go on and off drawing partly learning with some resources and try to abstract the concept behind what I learned, partly just having fun with an idea. It is the most balanced way that I have found to keep drawing even when I take long break from it. When I draw for me, I try to draw a bit like we can do automatic writing. I don’t have a mental picture and Ior a precise idea but I just try to let out and get in the “emotional flow”.

                                                        It depends what you want to achieve with it too. More technical drawings or artistic or just to express yourself?

                                                        1. 2

                                                          I can’t draw at all. Many times I’d have loved to so others could picture what I saw or imagined. I envy folks that can do it.

                                                          Do you or anyone else here have resources for beginners who are definitely non-artists wanting to get something out? I’d appreciate them. :)

                                                          1. 1

                                                            I’ve started to pick up drawing on my Surface. Concept art is a great place to start as it can be as messy as you want it to be. I’ve been following this guy’s youtube channel

                                                            Another good tip is to practice by drawing stuff in your home / environment.

                                                            1. 1

                                                              I asked a few friends and threads about resources, here’s a list I compiled of all the resources mixed

                                                          1. 6

                                                            I think there are two aspects to this. Below, I will use now old-fashioned word RIA(Rich Internet Application) to refer to “mutated application runtime”, its functionality, not its implementation.

                                                            Replying to “HTML, which started as document markup, should never have grown into RIA”, the author basically explains RIA-less HTML wouldn’t be much simpler, nor would it be much more efficient. In other words, the post is entirely about document, not RIA.

                                                            In my experience, when the argument is brought up, it is usually about RIA, not document. HTML-less RIA, not RIA-less HTML. HTML-less RIA, legacy free RIA implementation designed from scratch for RIA need, could be simpler and more efficient. There is also no backward compatibility need here. Writing a cross platform application runtime is a big task, so it isn’t easy, but the task is not helped by need to serve document markup legacy and web compatibility burden.

                                                            Flutter is a try to create HTML-less RIA. I doubt the author thinks Flutter does not make sense; it clearly does. Now, once we have HTML-less RIA, RIA-less HTML could save time specifying and implementing endless stream of APIs necessary for RIA, and focus on its already awesome styling and layout and rendering engine of document. I agree it wouldn’t be much simpler nor much more efficient, but it would also greatly help. This is why I feel the argument and the reply in the post is talking past each other.

                                                            1. 4

                                                              I think what the author is doing is responding to the many people out there on the Internet who treat this as a throwaway line (on HN for instance). I read most of them as asking for a RIA-less-HTML, and I think this is a good criticism of that idea.

                                                              I don’t know which of us is right about what people who use this line are asking for.

                                                              1. 2

                                                                You mentioned HN, so let’s try some empiricism. This article just hit HN front page. https://news.ycombinator.com/item?id=23599734 is a typical response. Note that it is entirely about RIA and whether DOM is a good basis for RIA, not about document, as I predicted.

                                                              2. 1

                                                                HTML-less RIA, legacy free RIA implementation designed from scratch for RIA need, could be simpler and more efficient.

                                                                I think our GUI builders like VB6 and Lazaurus already implied this by their feature vs footprint compared to web offerings. For more apples to apples, I also like to bring up Sciter because it’s so much more efficient than Electron etc. We could definitely do better than HTML and web browsers if just wanting to render content efficiently. Its dominance is a legacy and/or ecosystem effect, not technical superiority, at this point.

                                                                Edit to add: I’ll add that OS’s like MenuetOS fit a whole system in a floppy. Nobody’s building RIA’s like that for various reasons, esp productivity. It does imply our platforms or supporting libraries that the RIA’s run on could be much leaner. I’m thinking something like a GUI builder combined with a runtime lean like MenuetOS.

                                                              1. 6

                                                                It’s probably only because Rust is so unreadable that they didn’t find anything. /s

                                                                On a serious note, no matter what you think about Rust, more diversity in the realm of TLS libraries is a good thing. Just like BearSSL, rustls offers a way to escape the factual OpenSSL-monoculture.

                                                                1. 2

                                                                  It’s probably only because Rust is so unreadable that they didn’t find anything. /s

                                                                  I see the /s, but is that a common criticism? I genuinely do not know how Rust is perceived other than the hype/enthusiasm.

                                                                  1. 3

                                                                    Speaking from experience, there’s technically nothing wrong with Rust’s syntax. In fact, there’s lots of great stuff about it, like types that remain human-readable even when they’re complex (nested arrays and function pointers are easy). Greppable fn keyword for function definitions is very handy too.

                                                                    However, Rust tries to look like C, but has syntax details significantly different from C. I suspect it gives an “uncanny valley” impression to users coming from C-family languages. Rust doesn’t need as many round parens, but requires more braces: if true {}. Rust has generics, which sometimes sprinkle the code with lots of <T>. This might affect overall aesthetics of the code, but I don’t find anything that would be objectively unreadable about that.

                                                                    1. 1

                                                                      Ive seen many say it’s hard to learn but don’t see much of that claim. All the changes going on in the Rust ecosystem, esp libraries, suggests folks can read the code fine. Suggests, not proves.

                                                                  1. 4

                                                                    I found this video particularly interesting given the discussion also occurring right now on the SQLite As An Application File Format thread. The conclusion of that argument on SQLite’s own webpage is:

                                                                    SQLite is not the perfect application file format for every situation. But in many cases, SQLite is a far better choice than either a custom file format, a pile-of-files, or a wrapped pile-of-files. SQLite is a high-level, stable, reliable, cross-platform, widely-deployed, extensible, performant, accessible, concurrent file format. It deserves your consideration as the standard file format on your next application design.

                                                                    Obviously, the more complex we make things, the broader the attack surface gets. I find it helpful to ask why something was created and whether what I’m trying to use it for was that. In the case of SQLite, I never would have thought twice about running a query against a SQLite file until watching this video. But putting a database in the place of other formats would seem odd to me and this video helps reinforce some of the benefits of trying to be as simple as possible. To tie in another current thread, this would be why my websites have reverted back to static HTML pages: lower attack surface and cheaper hosting due to less computing requirements.

                                                                    1. 2

                                                                      “But putting a database in the place of other formats”

                                                                      It is designed to handle issues, like filesystem failures, that most developers don’t even know how to handle. Many won’t do it. Then, SQLite became complex. Then, we saw how they tested it. We said to ourselves, “Wow! There’s no way anything we quickly throw together will be that reliable. Let’s just use SQLite.” Then, some did. :)

                                                                      “I never would have thought twice about running a query against a SQLite file”

                                                                      Almost all code is designed assuming the inputs are non-malicious. Many look for faulty input which has some overlap. Evil inputs might do that or more insidious behaviors to force the system to do what it wasn’t designed for. You should assume these two things:

                                                                      1. All systems are insecure unless designed otherwise with rigorous, proven methods with an independent evaluation by expert breakers. All apps, especially but not limited to memory-unsafe, may fail insecurely if fed malicious input. They have to be explicitly designed to enforce security properties and/or stop classes of attack.

                                                                      2. “Attacks only get better.” (Schneier) New classes of attack will occur. So, you should use mitigations like OpenBSD’s to potentially combat them, damage containment, monitor for odd behavior, have read-only backups of critical data, and a battle-tested way of restoring the system.

                                                                      Apply these principles to every piece of hardware, OS kernel, library, or application. It’s always true unless there’s counterexample Im forgetting.

                                                                      1. 1

                                                                        But putting a database in the place of other formats would seem odd to me

                                                                        I think this is unfair in the context that many file formats are attempts at reimplementing dbms with relational objects. Some formats are simple, but many are complex enough and have yielded their fair share of CVE. The way I read the SQLite As An Application File Format article is that instead of reimplementing yet another dbms, why not reuse a battle tested engine.

                                                                        On the other hand, this talk brings some question about the SQLite threat model dealing with untrusted database input. A “simple” alternative could be to split SQLite into multiple processes where a restricted “server” handle the database and the client process use a simple protocol to send queries and read results. In most case files parsing is where exploitation happens and doing it in unprivileged process it is simply good practice.

                                                                      1. 20

                                                                        Of course everyone is free to spend as much money as they like, but if you want to start a blog and self-host, and might be discouraged, please let me give you another estimate that should 100% cover your needs:

                                                                        • Cloud VPS to host your blog: 3 EUR per month (Hetzner / Scaleway / whatever)
                                                                        • Domain: 12 EUR per year.

                                                                        And then you still have plenty of resources left to run stuff on your VPS.

                                                                        1. 12

                                                                          And in case you decide to go with a static site, Netlify has an extremely generic free tier which would waive off those 3€ per month as well.

                                                                          1. 3

                                                                            Supporting your point, I have a non-optimized, web app written in Python with plain HTML and CGI serving people daily at under 30% utilization of a $5 VM. Static, cached website offloading to a CDN might be even cheaper.

                                                                            1. 3

                                                                              You can get a VPS for free (and domain as well), check out: https://matrix.org/docs/guides/free-small-matrix-server#get-a-free-server (yes, I wrote that page).

                                                                              1. 4

                                                                                If something if free, you’re the product. ;-)

                                                                                1. 7

                                                                                  This isn’t like facebook/whatsapp/google(well, some of their services) where you cannot pay for the services. It’s a freebie to get you hooked. Start using and then discover you need more but don’t have the time/effort/resources to move someplace else, so you need to start paying to grow.

                                                                                  1. 1

                                                                                    I became really disenchanted with the US engineering program I went through when I found out that they only taught us to use $1000+ software titles. Not that open source existed for some of those titles then or now, but I felt a ton like the product…

                                                                                  2. 2

                                                                                    It’s actually really impressive that Oracle gives enough to run an actual ha service. It’s the core of any system to scale from one to two. Terraform even has the free tier all coded up (copyright Oracle, obviously): https://github.com/terraform-providers/terraform-provider-oci/blob/master/examples/always_free/main.tf

                                                                                  3. 2

                                                                                    Good point.

                                                                                    You can do things even cheaper if you use plain html/css files. I paid $37 on nearlyfreespeech, but I could’ve shaved off another ~$15 if I only had one site instead of two.

                                                                                    Bandwidth has never been a concern, but if it is, Cloudflare has a free plan.

                                                                                    1. 3

                                                                                      I think a static blog can easily be hosted on netlify/git(hub|lab) pages for free

                                                                                      1. 2

                                                                                        I just now realized I didn’t specify a time frame. Whoops. That’s $37 for all of 2019, or $3 a month.

                                                                                      2. 1

                                                                                        With HTML and CSS knowledge one can just set up a static site.

                                                                                        Of course it’s not as convenient as logging into a CMS but unless you have loads of traffic it will be free, most likely forever.

                                                                                        1. 1

                                                                                          Depending on how important it is for people to self-host, one might reconsider and use services like neocities, SDF or one of the many friendly tilde communities. True, you don’t get to decide that much, but you can still learn a lot under constraints, that you can then apply if you reconsider again later on and “self-host” (though that’s not always the right term with VPS’s).

                                                                                          1. 1

                                                                                            I’ve seriously considered hand writing a blog on Neocities, but my current blog takes an enough time as it is without having to hand code the entire thing. Would be a lot of fun though.

                                                                                            1. 1

                                                                                              As if you can’t use an SSG witth neocities. ;)

                                                                                        1. 3

                                                                                          Just a quick meta-reminder from the submission Guidelines:

                                                                                          When submitting a URL, the text field is optional and should only be used when additional context or explanation of the URL is needed. Commentary or opinion should be reserved for a comment, so that it can be voted on separately from the story.

                                                                                          1. 4

                                                                                            When submitting a URL

                                                                                            To my eyes, this post is not a URL submission. I can also appreciate the concern though. But it seems like there is no main page for this project, so the best way to do it is with the text field.

                                                                                            1. 3

                                                                                              It’s a Show Lobsters. I suggested that modification through the system.

                                                                                              1. 2

                                                                                                Oops, you’re right I missed that. Ignore the above comment then.

                                                                                            1. 2

                                                                                              The Hamler 0.1 compiler was initially attempted to be implemented based on the GHC 8.10.1, but was later changed to adapt from Purescript Compiler 0.13.6’s implementation.

                                                                                              Interesting choice.

                                                                                              1. 2

                                                                                                They said more here.

                                                                                              1. 4

                                                                                                I’ve written some Go and some Rust. I feel like I usually enjoy Rust more, though I struggle to explain why.

                                                                                                I think, for Rust, I find the error handling really ergonomic. Using ? in a function that does a bunch of things that can fail is just so much nicer than having every other line be a if err == nil { return err }. I also find it easier to follow how references work in Rust, oddly enough maybe. And using modules through Cargo is just so nice, while Go modules is kind of a messy hack in comparison. Oh and the macros are just so nice too.

                                                                                                But on Go’s side, Go concurrency is really awesome and smooth, especially compared to the half-complete hacks that are tokio and the Rust async system. Did I mention how nice the built-in channels are, and how a bunch of places in the standard lib use them? And easy cross-compilation is pretty nice too. And you gotta love that massive standard library. And I suppose not having to wrestle with complex too-clever generic hierarchies is nice sometimes too.

                                                                                                1. 15

                                                                                                  But on Go’s side, Go concurrency is really awesome and smooth

                                                                                                  Concurrency is an area I feel Go really lets the programmer down. There is a simple rule for safe concurrent programming: No object should be both mutable and shared between concurrent execution contexts at the same time. Rust is not perfect here, but it uses the unique ownership model and the send trait to explicitly transfer ownership between threads so you can pass mutable objects around, and the sync trait for safe-to-share things. The only safe things to share in safe rust are immutable objects. You can make other things adopt the sync trait if you’re willing to write unsafe Rust, but at least you’re signposted that here be dragons. For example, the ARC trait in Rust (for atomic reference counting), which gives you a load of read-only shared references to an object and the ability to create a mutable reference if there are no other outstanding references.

                                                                                                  In contrast, when I send an object down a channel in Go, I still have a pointer to it. The type system gives me nothing to help avoid accidentally aliasing an object between two threads. To make things worse, the Go memory model is relaxed consistency atomic, so you’re basically screwed if you do this. To make things even worse, core bits of the language semantics rely on the programmer not doing this. For example, if you have a slice that is in an object that is shared between two goroutines, both can racily update it. The slice contains a base and a length and so you can see tearing: the length from one slice and the base from another. Now you can copy it, dereference it and read or write past the end of an array. This is without using anything in the unsafe package: you can violate memory safety (let alone type safety) purely in ‘safe’ Go, without doing anything that the language helps you avoid.

                                                                                                  I wrote a book about Go for people who know other languages. It didn’t sell very well, in part because it ended up being a long description of things that Go does worse than other languages.

                                                                                                  1. 2

                                                                                                    That’s a worthwhile point. I haven’t been bitten by the ability to write to Go object that have already been sent down a channel yet, but I haven’t worked on any large-scale long-term Go projects. I’ve found it straightforward enough to just not use objects after sending. But then, the reason why we build these fancy type systems with such constraints is that even the best developers have proved to be not very good at consistently obeying these limits on large-scale projects.

                                                                                                    I’m hoping that the Rust issues with async and tokio are more like teething pains for new tech than a fundamental issue, and that eventually, it will have concurrency tools that are both as ergonomic as Go’s and use Rust’s thread safety rules.

                                                                                                    1. 4

                                                                                                      I’ve found it straightforward enough to just not use objects after sending.

                                                                                                      This is easy if the object is not aliased, but that requires you to have the discipline of linear ownership before you get near the point that sends the object, or to only ever send objects allocated near the sending point. Again, the Go type system doesn’t help at all here, it lets you create arbitrary object graphs with N pointers to an object and then send the object. The (safe) Rust type system doesn’t let you create arbitrary object graphs and then gives strong guarantees on what is safe to send. The Verona type system is explicitly designed to allow you to create arbitrary (mutable or immutable) object graphs and send them safely.

                                                                                                  2. 15

                                                                                                    side-note: i think it’s a bit off-topic (and meme-y, rust strike force, etc. :) to compare to rust when the article only speaks of go :)

                                                                                                    Using ? in a function that does a bunch of things that can fail is just so much nicer than having every other line be a if err == nil { return err }.

                                                                                                    i really like the explicit error handling in go and that there usually is only one control flow (if we ignore “recover”). i guess that’s my favorite go-feature: i don’t have to think hard about things when i read them. it’s a bit verbose, but that’s a trade-off i’m happy to make.

                                                                                                    1. 7

                                                                                                      i really like the explicit error handling in go

                                                                                                      I would argue that Go’s model of error handling is a lot less explicit than Rust’s - even if Go’s is more verbose and perhaps visually noticeable, Rust forces you to handle errors in a way that Go doesn’t.

                                                                                                      1. 1

                                                                                                        I have just read up un rusts error handling, it seems to be rather simila, except that return types and errors are put together as “result”: https://doc.rust-lang.org/book/ch09-00-error-handling.html

                                                                                                        my two cents: i like that i’m not forced to do things in go, but missing error handling sticks out as it is unusual to just drop errors.

                                                                                                        1. 4

                                                                                                          Well since it’s a result, you have to manually unwrap it before you can access the value, and that forces you to handle the error. In Go, you can forget to check err for nil, and unless err goes unused in that scope, you’ll end up using the zero value instead of handling the error.

                                                                                                          1. 1

                                                                                                            i like that i’m not forced to do things in go, but missing error handling sticks out as it is unusual to just drop errors

                                                                                                            The thing is, while it may be unusual in Go, it’s impossible to “just drop errors” in Rust. It’s easy to unwrap them explicitly if needed, but that’s exactly my point: it’s very explicit.

                                                                                                        2. 3

                                                                                                          The explicit error handling is Very Visible, and thus it sticks out like a sore thumb when it’s missing. This usually results in better code quality in my experience.

                                                                                                          1. 2

                                                                                                            It did occur to me that it may come off like that :D It’s harder to make interesting statements about a language without comparing it to its peers.

                                                                                                            IMO, Rust and Go being rather different languages with different trade-offs that are competing for about the same space almost invites comparisons between them. Kind of like how temping it is to write comparisons between Ruby, Python, and Javascript.

                                                                                                            1. 1

                                                                                                              I think Swift fits in quite well in-between. Automatic reference counting, so little need to babysit lifetimes, while using a powerful ML-like type system in modernised C-like syntax.

                                                                                                          2. 8

                                                                                                            And using modules through Cargo is just so nice, while Go modules is kind of a messy hack in comparison.

                                                                                                            I have always found Rust’s module system completely impenetrable. I just can’t build a mental model of it that works for me. I always end up just putting keywords and super:: or whatever in front in various combinations until it happens to work. It reminds me of how I tried to get C programmes to compile when I was a little kid: put more and more & or * in front of expressions until it works.

                                                                                                            And of course they changed in Rust 2018 as well which makes it all the more confusing.

                                                                                                            1. 3

                                                                                                              Yeah, I’ve had the same experience. Everything else about Cargo is really nice, but modules appear to be needlessly complicated. I have since been told that they are complicated because they allow you to move your files around in whatever crazy way you prefer without having to update imports. Personally I don’t think this is a sane design decision. Move your files, find/replace, move on.

                                                                                                              1. 2

                                                                                                                And of course they changed in Rust 2018 as well which makes it all the more confusing.

                                                                                                                One of the things they changed in Rust 2018, FYI, was the module system, in order to make it a lot more straightforward. Have you had the same problem since Rust 2018 came out?

                                                                                                              2. 6

                                                                                                                For me Go is the continuation of C with some added features like CSP. Rust is/was heavily influenced by the ML type of languages which is extremely nice. I think ML group is superior in my ways to the C group. ADTs are the most trivial example why.

                                                                                                                1. 3

                                                                                                                  I generally agree. I like ML languages in theory and Rust in particular, but Rust and Go aren’t in the same ballpark with respect to developer productivity. Rust goes to impressive lengths to make statically-managed memory user-friendly, but it’s not possible to compete with GC. It needs to make up the difference in other areas, and it does make up some of the difference in areas like error handling (?, enums, macros, etc and this is still improving all the time), IDE support (rust-analyzer has been amazing for me so far), and compiler error messages, but it’s not yet enough to get into a competitive range IMO. That said, Rust progresses at a remarkable pace, so perhaps we will see it get there in the next few years. For now, however, I like programming in Rust–it satisfies my innate preference to spend more time building something that is really fast, really abstract, and really correct–but when I need to do quality work in a short time frame in real world projects, I still reach for Go.

                                                                                                                  1. 9

                                                                                                                    To me Go seems like a big wasted opportunity. If they’d only taken ML as a core language instead of a weird C+gc hybrid, it would be as simple (or simpler) as it is, but much cleaner, without nil or the multi-return hack. Sum types and simple parametric polymorphism would be amazing with channels. All they had to do was to wrap that in the same good toolchain with fast compilation and static linking.

                                                                                                                    1. 2

                                                                                                                      Yeah, I’ve often expressed that I’d like a Go+ML-type-system or a Rust-lite (Rust with GC instead of ownership). I get a lot of “Use OCaml!” or “Use F#”, but these miss the mark for a lot of reasons, but especially the syntax, tooling, and ecosystem. That said, I really believe we overemphasize language features and under-emphasize operational concerns like tooling, ecosystem, runtime, etc. In that context, an ML type system or any other language feature is really just gravy (however, a cluster of incoherent language features is a very real impediment).

                                                                                                                      1. 1

                                                                                                                        Nothing is stopping anyone from doing that. I’d add that they make FFI to C, Go, or some other ecosystem as easy as Julia for the win. I recommend that for any new language to solve performance and bootstrapping problem.

                                                                                                                      2. 3

                                                                                                                        Then, you have languages like D that compile as fast as Go, run faster with LLVM, have a GC, and recently an optional borrow checker. Contracts, too. You get super productivity followed by as much speed or safety as you’re willing to put in effort for.

                                                                                                                        Go is a lot easier to learn, though. The battle-tested, standard libraries and help available on the Internet would probably be superior, too.

                                                                                                                        1. 3

                                                                                                                          I hear a lot of good things about D and Nim and a few others, but for production use case, support, ecosystem, developer marketshare, tooling, etc are all important. We use a lot of AWS services, and a lot of their SDKs are Python/JS/Go/Java/dotnet exclusively and other communities have to roll their own. My outsider perspective is that D and Nim aren’t “production ready” in the sense that they lack this sort of broad support and ecosystem maturity, and that’s not a requirement I can easily shrug off.

                                                                                                                          1. 2

                                                                                                                            I absolutely agree. Unless easy to handroll, those kind of things far outweigh advantages in language design. It’s what I was hinting at in 2nd paragraph.

                                                                                                                            It’s also why it’s wise for new languages to plug into existing ecosystems. Clojure on Java being best example.

                                                                                                                  1. 6

                                                                                                                    “With an open-source implementation, you see what you get”

                                                                                                                    Just wanted to note this is not true at all for hardware. The synthesis tools, usually two in combination, convert the high-level form into low-level pieces that actually run. They’re kind of like Legos for logic. Like with a compiler, they might transform them a lot to optimize. They use standard cells that are usually secret. Then, there’s analog and RF functionality that might have errors or subversions with fewer experts that know anything about it. Finally, there’s the supply chain from masks to fab to packaging to you.

                                                                                                                    With hardware, you have no idea what you actually got unless you tear it down. If it’s deep sub-micron, you have to one or more other companies during the tear-down process. This excludes the possibility that they can make components look like other components in a tear-down. Idk if that’s possible but figure I should mention it.

                                                                                                                    When I looked at that problem, my solution was that the core or at least a checker/monitor had to be at 350nm or above so a random sample could be torn up for visual inspection. The core would be designed like VAMP with strong verification. Then, synthesis (eg Baranov’s) to lower-level form with verified transforms followed by equivalence checks (formal and/or testing). The cells, analog, and RF would be verified by mutually-suspicious experts. Then, there were some methods that can profile analog/RF effects of onboard hardware to tell if they swap it out at some point. Anyway, this is the start with open (or vetted + NDA) cells, analog, and RF showing up overtime, too. Some already are.

                                                                                                                    1. 7
                                                                                                                      1. 2

                                                                                                                        I’m not a big fan of making critiques based on stuff that is explicitly outside of their security model. From my understanding, the formal verification of side channel for RISC-V would catch Spectre-style attacks: researchers implemented Spectre-like vulnerabilities into RISC-V designs which still conformed to the specification.

                                                                                                                        Yes, you can backdoor compilers, microcode, and hardware. But that’s not far from the generic critique of formal methods based on Godel’s incompleteness theorem. seL4 is the only operating system that makes it worth our time to finally start hardening the supply chain against those types of attacks.

                                                                                                                        1. 3

                                                                                                                          I normally agree. However, they were pushing seL4 on ARM as a secure solution. You cant secure things on ARM offerings currently on market. So, it’s a false claim. The honest one is it gives isolation except for hardware attacks and/or faults. For many, that immediately precludes using it. I’d rather them advertise honestly.

                                                                                                                          A side effect is that it might increase demand in secure hardware.

                                                                                                                      1. 8

                                                                                                                        I don’t get this hype about seL4. All I see are it’s claims about it’s security and speed, but I can’t find anything about it’s usability. The communication on it’s page aggressively attacks other operating systems (e.g. “If someone else tells you they can make such guarantees, ask them to make them in public so Gernot can call out their bullshit” in the FAQ). The performance page doesn’t have any comparisons to other OS’s, yet FAQ claims that it is the fastest in the metric presented there. In general, the few times I’ve seen somebody bring up seL4, the proponents were very aggressive against other OS’s. Doesn’t really look well, does it?

                                                                                                                        1. 17

                                                                                                                          The rhetoric from the seL4 cheerleaders can indeed be cringeworthy at times. That being said, the L4 family is an interesting look into how you can start with a really minimal set of OS features and get to something useful, and seL4 is one of a very few OS kernels to be subject to rigorous formal verification. How much you value that probably tracks very closely to how much you value formal verification in general.

                                                                                                                          It isn’t particularly useful to compare seL4 to a general-purpose OS like Linux or Windows since they’re intended for such different use cases. seL4 might be a useful building block for, say, a hardened security appliance that handles signing keys or runtime monitoring on behalf of some other general-purpose OS, or a high-value industrial control system (power plants, medical devices, voting machines, etc.)

                                                                                                                          The focus on performance is AFAICT aimed mainly at the historical critique of microkernels as painfully slow for real-world workloads. That in turn largely stems from the behavior of poorly-optimized Mach-backed syscalls on commodity PCs when they were being put up against monolithic designs back in the 90s. (Mac OS still seems to carry some of this debt, as Xnu is a pretty direct descendant of Mach.)

                                                                                                                          1. 3

                                                                                                                            Is there a blog post about this? I want to know more!

                                                                                                                            1. 4

                                                                                                                              It’s not just Mach, it was also Windows NT, Minix, and others. It took the L3 and L4 family of kernels a long time to get this nailed down. Just dig around wikipedia for Microkernels and this paper for the history of L4.

                                                                                                                          2. 4

                                                                                                                            It outperformed other microkernels. It would probably host an OS like Linux in a VM alongside bare-metal or runtume-supported components. A secure middleware lets the pieces communicate. The architecture is often called Multiple Independent Levels of Security with microkernels doing it called “separation” kernels. Overall performance depends on the overheads of context switching and message passing which lead to tradeoffs in how many pieces you break system into.

                                                                                                                            This pdf on a similar system (Nizza) shows how building blocks like seL4 were to be used. INTEGRITY-178B was the first built, certified, and deployed in Dell Optiplex’s. The certification data at bottom-right shows what was required but watch out for their marketing ;). LynxSecure is used by Navy. Due to funding, complexity, and OK Labs focus on mobile, the seL4 team switched focus to embedded like military and IoT.

                                                                                                                            @Shapr, tagging you in since the Nizza paper might help you out.

                                                                                                                            1. 1

                                                                                                                              I thought version of L4 hosting Linux outperform Linux?

                                                                                                                              1. 4

                                                                                                                                It did. The benchmark might be meaningless, though. A real system would extract more and more of the Linux TCB into isolated partitions. There would be more message passing. It could also cause more accidental cache flushes on top of clearing registers and caches that already happens in separation kernels upon security context switch. We don’t know what the performance hit would be.

                                                                                                                                An example would be a web server where the kernel, ethernet, networking, filesystem, firewall, TLS, and user-facing server are all in separate partitions. Things that are mostly jumps in memory of one process become IPC across them. That could add up.

                                                                                                                          1. 18

                                                                                                                            What has been the problem with Python/Flask/SQLA?

                                                                                                                            1. 28

                                                                                                                              Python: the size of the codebase and number of moving parts has reached a point where the lack of static typing has become the main source of programmer errors in the code. There are type annotations now, but they don’t work very well IMO, are not used by most of our dependencies, and would be almost as much to retrofit onto our codebase as switching to a type-safe language would be. The performance of the Python VM is also noticably bad. We could try PyPy, but again… we’re investing a lot of effort just to stick to a language which has repeatedly proven itself poorly suited to our problem. The asyncio ecosystem helps but it’s still in its infancy and we’d have to rewrite almost everything to take advantage of it. And again, if we’re going to rewrite it… might as well re-evaluate our other choices while we’re at it.

                                                                                                                              Flask: it’s pretty decent, and not the main source of our grief (though it is somewhat annoying). My main feedback for Flask would be that it tries to do just a little bit too much. I wish it was a little bit more toolkit-oriented in its design and a more faithful expression of HTTP as a library.

                                                                                                                              SQLAlchemy: this is now my least favorite dependency in our entire stack. It’s… so bad. I just want to write SQL queries now. The database is the primary bottleneck in our application, and hand-optimizing our SQL queries is always the best route to performance improvements. Some basic stuff is possible with SQLAlchemy, simple shit like being smart about your joins and indicies, but taking advantage of PostgreSQL features is a pain. It’s a bad ORM - I’m constantly fighting with it to just do the shit I want it to and stop dicking around - and it’s a bad database abstraction layer - it’s too far removed from Postgres to get anything more than the basics done without a significant amount of grief and misery. Alembic is also constantly annoying. Many of the important improvements I want to do for performance and reliability are blocked by ditching these two dependencies.

                                                                                                                              Another problem child that I want to move away from is Celery. It just isn’t flexible enough to handle most of the things I want to do, and we have to use it for anything which needs to be done asyncronously from the main request handling flow. In Go it’s a lot easier to deal with such things. Go also allows me to get a bit closer to the underlying system, with direct access to syscalls and such*, which is something that I’ve desired on a few occasions.

                                                                                                                              For the record, the new system is not without its flaws and trade-offs. Go is not a perfect tool, nor GraphQL. But, they fit better into the design I want. This was almost a year of research in the making. The Python codebase has served us well, and will continue to be useful for some time to come, in that it (1) helped us understand the scope necessary to accomplish our goals, and (2) provided a usable platform quickly. Nothing quite beats Python for quickly and easily building a working prototype, and it generally does what you tell it to, in very few lines of code. But, its weaknesses have become more and more apparent over time.

                                                                                                                              * Almost. The runtime still gets on my nerves all the time and is still frustratingly limiting in this respect.

                                                                                                                              1. 9

                                                                                                                                Thanks for responding. I think static typing in Python works really well once configured so I’m surprised to hear you say that. I think it’s better than the static typing in most other languages because generics are decent and the inference is pretty reasonable. For example it seems better thought out than Java, C and (in my limited experience) Go. My rough feeling is that 75% of the Python ecosystem either has type annotations or has type stubs in typeshed. Where something particularly important is untyped, I tend to just wrap it and give it an explicit annotation (this is fairly rare). I’ve written some tips on getting mypy working well on bigger projects.

                                                                                                                                I don’t think you have the right intuition that asyncio would help you if your problem is speed. I pretty convinced that asyncio is in fact slower than normal Python in most cases (and am currently writing another blogpost about that - UWSGI is for sure the fastest and most robust way to run a python webservice). Asyncio stuff tends to fail in weird ways under load. I also think asyncio is a big problem for correctness - it actually seems quite hard to get asyncio programs right and there are a lot of footguns around.

                                                                                                                                Re: SQLAlchemy - I’m also very surprised. I think SQLAlchemy is a good ORM and I’ve used postgres specific features (arrays, json, user defined functions, etc) from it a great deal. If you want to write SQL-level code there is nothing stopping you from using the “core” layer rather than the “ORM” layer. There’s also nothing stopping you using SQL strings with the parameterisation, ie "select col_a from table where col_b = :something - I do that sometimes too. I have to say I have never had trouble with hand optimising a SQL query in SQLA - ever - because it gives you direct control over the query (this is even true at the ORM level). One problem I have run into is where people decide to use SQLA orm objects as their domain objects and…that doesn’t end happily.

                                                                                                                                Celery however is something that I do think is quite limited. It’s really just a task queue. I am not sure that firing off background tasks as goroutines is a full replacement though as you typically need to handle errors, retry, record what happened, etc. I think even if you were using go every serious system ends up with a messaging subsystem inside it - at least for background tasks. People do not usually send emails from their webserving processes. Perhaps the libraries for this in go land are better but in Python I don’t think there is a library that gets this kind of thing wholly right. I am working on my own thing but it’s too early to recommend it to anyone (missive). I want to work on it more but childcare responsibilities are getting in the way! :)

                                                                                                                                Best of luck in your rewrite/rework. I have not been impressed with GraphQL so far but I haven’t used the library you’re planning to use. My problems with GraphQL so far are that a) it isn’t amenable to many of the optimisations I want to do with it b) neither schema first nor code first really work that well and c) it’s query language is much more limited than it looks - much less expressive than I would like. You may not find that the grass is greener!

                                                                                                                                1. 5

                                                                                                                                  I don’t think you have the right intuition that asyncio would help you if your problem is speed.

                                                                                                                                  I don’t want asyncio for speed, I want it for a better organizational model of handling the various needs of the application concurrently. With Flask, it’s request in, request out, and that’s all you get. I would hope that asyncio would improve the ability to handle long-running requests while still servicing fast requests, and also somewhat mitigate the need for Celery. But still, I’ve more or less resigned from Python at this point, so it’s a moot point.

                                                                                                                                  I am not sure that firing off background tasks as goroutines is a full replacement though as you typically need to handle errors, retry, record what happened, etc.

                                                                                                                                  Agreed. This is not completely thought-out yet, and I don’t expect the solution to be as straightforward as fire-and-forget.

                                                                                                                                  My problems with GraphQL so far are that a) it isn’t amenable to many of the optimisations I want to do with it b) neither schema first nor code first really work that well and c) it’s query language is much more limited than it looks - much less expressive than I would like.

                                                                                                                                  I have encountered and evaluated all of the same problems, and still decided to use GraphQL. I am satisfied with the solutions to (a) and (b) presented by the library I chose, and I feel comfortable building a good API within the constraints of (c). Cheers!

                                                                                                                                2. 3

                                                                                                                                  So do you plan to keep the web UI in Python using Flask, and have it talk to a Go-based GraphQL API server? Or do you plan to eventually rewrite the web UI in Go as well? If the latter, is there a particular Go web framework or set of libraries that you like, or just the standard library?

                                                                                                                                  1. 4

                                                                                                                                    To be determined. The problems of Python and Flask become much less severe if it’s a frontend for GraphQL, and it will be less work to adapt them as such. I intend to conduct more research to see if this path is wise, and also probably do an experiment with a new Golang-based implementation. I am not sure how that would look, yet, either.

                                                                                                                                    It’s also possible that both may happen, that we do a quick overhaul of the Python code to talk to GraphQL instead of SQL, and then over time do another incremental rewrite into another language.

                                                                                                                                  2. 3

                                                                                                                                    I’m curious about why you consider that Flask does “a little bit too much”. It’s a very lightweight framework, and the only “batteries included” thing I can think of is the usage of Jinja for template rendering. But if I’m not wrong, sourcehut uses it a lot so I don’t thing this is what annoys you.

                                                                                                                                    Regarding SQLAlchemy, I totally agree with you. It’s a bad database abstraction layer. When you try to make simple queries it becomes cumbersome because of SQLAlchemy’s supposed low level abstractions. But when you want to make a fine-grained query it’s also a real pain and you end up writing raw SQL because it’s easier. In some cases you can embed some raw SQL fragment inside the ORM query, but it is often not the case (for example, here is a crappy piece of code I’m partially responsible of). Not having a decent framework-agnostic ORM is the only thing that makes me miss Django :(

                                                                                                                                    1. 8

                                                                                                                                      Regarding Flask, I recently saw Daniel Stone give a talk wherein he reflected on the success of wlroots compared to the relative failure of libweston, and chalked it up to the difference between a toolkit and a midlayer, where wlroots is the former. Flask is a midlayer. It does its thing, and provides you a little place to nestle your application into. But, if you want to change any of its behavior - routing, session storage, and so on - you’re plugging into the rails its laid down for you. A toolkit approach would instead have the programmer always be in control, and reach for the tools it needs - routing, templating, session management, and so on - as they need them.

                                                                                                                                      1. 1

                                                                                                                                        I’ve personally found falcon a bit nicer to work with than flask, as an api/component.
                                                                                                                                        That said, as a daily user for some mid-sized codebases (some 56k odd lines of code), I very much agree with what you said about python and sqlalchemy.

                                                                                                                                      2. 4

                                                                                                                                        I find that linked piece of code perplexing because converting that from string-concat-based dynamic SQL into SQLA core looks straightforward: pull out the subqueries, turn them into python level variables and then join it all up in a single big query at the end. That would also save you from having a switch for sqlite in the middle of it - SQLA core would handle that.

                                                                                                                                      3. 1

                                                                                                                                        SQLAlchemy: this is now my least favorite dependency in our entire stack. It’s… so bad

                                                                                                                                        That’s also the only thing I remember about it from when I used it years ago. Maybe it’s something everyone has to go through once to figure out the extra layer might look tasty, but in the end it only gives you stomach ages.

                                                                                                                                      4. 13

                                                                                                                                        Yeah, I’d be very interested to hear more about that too. Not that I disagree, but I think his article was light on details. What were the things that “soured” his view of Python for larger projects, and why was he “unsatisfied with the results” of REST?

                                                                                                                                        1. 11

                                                                                                                                          I found REST difficult to build a consistent representation of our services with, and it does a poor job of representing the relationship between resources. After all, GraphQL describes a graph, but REST describes a tree. GraphQL also benefits a lot from static typing and an explicit schema defined in advance.

                                                                                                                                          Also, our new codebase for GraphQL utilizes the database more efficiently, which is the main bottleneck in the previous implementation. We could apply similar techniques, but it would require lots of refactoring and SQLAlchemy only ever gets in the way.

                                                                                                                                        2. 1

                                                                                                                                          Ive been using Flask and Gunicorn. I basically do native dev before porting it to web app. My native apps are heavily decomposed into functions. One thing that’s weird is they break when I use them in web setup. The functions will be defined before “@app” or whatever it is like in a native app. Then, Gunicorn or Flask tells me the function is undefined or doesn’t exist.

                                                                                                                                          I don’t know why that happens. It made me un-decompose those apps to just dump all the code in the main function. Also, I try to do everything I can outside the web app with it just using a database or something. My Flask apps have stayed tiny and working but probably nearing the limit on that.

                                                                                                                                        1. 2

                                                                                                                                          Thank you for this classic !

                                                                                                                                          Would you recommend having prior experience with reading about mostly monolithic kernels like XV6 or is it a good starting point for someone with just a bit of unix programming ?

                                                                                                                                          1. 3

                                                                                                                                            Thank you for this classic !

                                                                                                                                            It was published three days ago. It is hardly a classic (yet).

                                                                                                                                            Would you recommend having prior experience with reading about mostly monolithic kernels like XV6 or is it a good starting point for someone with just a bit of unix programming ?

                                                                                                                                            If you’re interested in getting into operating systems, my favourite introductory book on the subject (Operating Systems, Design and Implementation, 3rd ed by Tanenbaum/Woodhull) covers the Minix3 design, which is a microkernel, multiserver system, so you can get started on the topic with something closer to the matter.

                                                                                                                                            Modern operating systems has also been refreshed recently (relatively speaking, 2014).

                                                                                                                                            These are expensive books, but have been spotted in the usual “free” online libraries before.

                                                                                                                                            Another good (and free!) resource is the Genode Foundations book, but going by tradition, a new version is likely due within a week or two.

                                                                                                                                            There’s also a lot of reading to do in Gernot Heiser’s blog.

                                                                                                                                            1. 3

                                                                                                                                              Check out Genode since they support multiple microkernels, Linux in VM, and (most important) have tools to help you glue it together. In seL4, you’ll probably use Camkes if I remember right.

                                                                                                                                              Also, programming a microkernel-based system with IPC is more like making networked apps talk to each other with instantaneous transfer and no dropped packets.

                                                                                                                                            1. 1

                                                                                                                                              For ${DAYJOB} I’m building a security appliance based on OPNsense. Hoping to deploy in production our first major build on Wednesday.

                                                                                                                                              For HardenedBSD, I wasn’t able to deliver on tmpfs enhancements like I wanted last week due to a sick wife. I’m hoping to deliver this week, but my dog is now sick.

                                                                                                                                              1. 2

                                                                                                                                                That sucks. Hope your wife and dog get better.

                                                                                                                                              1. 6

                                                                                                                                                Work: Working on our intelligence engine, a write-heavy hybrid document/relational database with facted catalogs. Trees go in, bigger trees come out, you can’t explain that.

                                                                                                                                                Home: Working on my little mostly-anaphoric scripting language. If I ever complete it, it’s going to be a nice little scripting language in 100% ANSI C that should be easily embedded in any host program and support automation and scripting/customization. Should be nice.

                                                                                                                                                1. 2

                                                                                                                                                  Working on our intelligence engine, a write-heavy hybrid document/relational database with facted catalogs.

                                                                                                                                                  Can you share any info about what kind of problems this is intended to solve?

                                                                                                                                                  1. 6

                                                                                                                                                    So it’s a centralized security intelligence engine. We inspect network traffic, grab files off of fileshares, grab stuff from web proxies, grab emails via milter or cloud integration, etc.

                                                                                                                                                    These get turned into trees like NetworkSession -> HTTPRequest -> File or EMail -> MIMEObject -> File. They get passed into the engine where we recursively expand out the tree (e.g. the File is a ZIP file, so we expand it and now there are Files under that File or we extract keywords or URLs out of that file’s contents and now there are URLs or whatever under the file).

                                                                                                                                                    We continually expand the tree until there’s nothing left to expand (every bit of intelligence is squeezed out; we have 100+ different analyzers). Once it’s all expanded, we index the tree in as many ways as makes sense. For example, a NetworkSession node would appear in the NetworkSession catalog, but would also appear (twice) in the IP Address catalog and might (given GeoIP information) appear in the Geographic Location catalog.

                                                                                                                                                    We then support real-time and on-demand queries that find different trees, so (for example) you can look for Files from Geographic Locations, even though files don’t have geographic locations, so we have to intelligently pivot between ancestors that do have geographic locations and the files that are found underneath them.

                                                                                                                                                    The trick is doing all this given fully-loaded multigigabit pipes, 6+ million files a day, etc…in an embedded, low-maintenance appliance.

                                                                                                                                                    There’s some more magic, but that’s the gist of it.

                                                                                                                                                    1. 2

                                                                                                                                                      Oh, that’s interesting, thanks for the reply. From your first comment I was imagining something like Elasticsearch (probably because I saw the word “document”) but this seems pretty different. This sounds like it’s a security product?

                                                                                                                                                      1. 2

                                                                                                                                                        Exactly. It’s a really great product, I’ve enjoyed working on it.

                                                                                                                                                  2. 2

                                                                                                                                                    Trees go in, bigger trees come out, you can’t explain that.

                                                                                                                                                    But don’t the bigger trees kind of explain the smaller trees?