1. 10

    Why do people think MS is doing all this? Do people really think a company worth 860 billion dollars has anything to give away for free? I do not want to go into MS bashing, but believing that a big company like MS is now altruistic and believing in making the world a better place is just naive. MS wants to be seen as cool and hip with the dev. crowd, esp. the young Sillicon Valley crowd, so that they can sell more Azure. They do not care about software freedom or anything like that.

    1. 12

      Goals can align. Microsoft might care about software freedom because that improves their business in some way. In this case, their goal is obviously to collect metrics about users. Almost all of the code is open though.

      1. 3

        I don’t think thats an obvious goal at all - metrics about users. A perfectly acceptable goal is to regain mindshare among developers. vscode can be seen as a gateway drug to other microsoft services, improving their reputation.

        1. 2

          I wonder what metrics from a text editor would be useful to them?

          1. 10

            I want metrics from the compilers I work on. It’d be super useful to know what language extensions people have enabled, errors people hit, what they do to fix them, etc. Sounds mundane at first, but it’d allow me to focus on what needs work.

            1. 8

              Well, VS Code doesn’t choose your compilers :)

              either way, I don’t get the paranoia. Performance telemetry, automated crash reports, stats about used configurations – not stuff that violates privacy in any meaningful way. It’s weird that this gets lumped in together in the general paranoia storm with advertisers building a profile of you to sell more crap.

              1. 8

                Issue #49161 VSCode sends search keystrokes to Microsoft even with telemetry disabled

                It’s not even paranoia so much as irritation at this point. I know my digital life is leaking like a sieve, and I’d like to plug the holes.

                1. 3

                  Kinda clickbait issue title. Yeah, keystrokes are always a lot more worrying than metrics, but this is settings search. I guess you could Ctrl+F search for something secret (e.g. a password) in a text file, but not in the settings.

                  1. 12

                    You know, there was a time when it was big news if a commercial program was caught to “phone home” at all. It didn’t matter what the content was.

                    (Today, you’d call a ‘commercial program’ a ‘proprietary application’.)

                    It’s still a big deal today if an open source/community maintained/free software application ‘phones home’, because reasons: untrusted individuals, the value of big data, and principles of privacy.

                    Now that M$ is in the game, let’s add ‘untrusted corporation’ to that last list.

                    I don’t care what the nature of the data is–I don’t want to be observed. Especially not as I ply my craft–few activities produce measurable signals from any deeper inside myself, and every one of those is definitely on my personal ‘no, you can’t watch!’ list.

                    1.  

                      For me personally, I have no problem adding telemetry to apps I maintain. But I’m sure going to make sure users know about it and can disable it if they want. I think that’s the real issue - consent.

                    2. 5

                      That’s having to think way too hard about what they’re intercepting.

              2. 4

                Platform it’s running on, type of code being edited, frequency of use for a given feature. Heuristic data about how people interact with the UI. The list goes on. Note also that none of this need be evil. It could be seen as collecting data looking to improve user experience.

            2. 3

              I’d guess they’re after a platform. They want to build a base (using organic growth) that they might later on capitalize on, either by learning from it to invite people to use (proper) Visual Studio or by limiting VSCode’s openness.

            1. 1

              I’m a bit confused by the mention of putting parts of the heap in slower memory. Shouldn’t that be the kernel + swap’s job? Or does the kernel only page out entire process address spaces (which would clearly be too coarse for what the article’s discussing)?

              1. 4

                Linux on x86 and amd64 can page out individual 4kiB pages, so the granularity of that is fine.

                It’s plausible that they might be able to get much better behaviour bybydoing it themselves instead of letting the kernel do it. Two things spring to mind:

                If they’re managing object presence in user space, they know which objects are in RAM so they can refrain from waking them up when they definitely haven’t changed. Swap is mostly transparent to user processes. You really don’t want to wake up a swapped out object during GC if you can avoid it, but you don’t know which objects are swapped out without calling mincore() for every page, which is not very fast.

                Other thing that springs to mind: AFAIK handling page faults is kinda slow and an x86 running Linux will take something like a (large fraction of) a microsecond each time a fault occurs. AFAIK the fault mechanism in the CPU is quite expensive (it has to flush some pipelines) at least. So doing your paging-in in userspace with just ordinary instructions that don’t invoke the OS or the slow bits of the CPU may be a big win.

              1. 33

                I don’t really think that you should be allowed to ask the users the sign a new EULA for security patches. You fucked up. People are being damaged by your fuck up and you should not use that as leverage to make the users do what you want so they can stop your fuck up from damaging them further.

                Patches only count if they come with the same EULA as the original hardware/software/product.

                1. 9

                  Sure - you’re welcome to refuse the EULA and take your processor back to the retailer, claiming it is faulty. When they refuse, file a claim in court.

                  Freedom!

                  1. 6

                    This suggestion reminds me of the historical floating point division bug. See https://en.m.wikipedia.org/wiki/Pentium_FDIV_bug

                    There was a debate about the mishandling by Intel. Also, there was debate over “real-world impact,” estimates were all over the charts.

                    Here, it seems that the impact is SO big, that almost any user of the chip can demonstrate significant performance loss. This might become even bigger than the FDIV bug.

                    1. 4

                      They are being sued by over 30 groups (find “Litigation related to Security Vulnerabilities”). It already is.

                      As of February 15, 2018, 30 customer class action lawsuits and two securities class action lawsuits have been filed. The customer class action plaintiffs, who purport to represent various classes of end users of our products, generally claim to have been harmed by Intel’s actions and/or omissions in connection with the security vulnerabilities and assert a variety of common law and statutory claims seeking monetary damages and equitable relief. The securities class action plaintiffs, who purport to represent classes of acquirers of Intel stock between July 27, 2017 and January 4, 2018, generally allege that Intel and certain officers violated securities laws by making statements about Intel’s products and internal controls that were revealed to be false or misleading by the disclosure of the security vulnerabilities […]

                      As for replacing defective processors, I’d be shocked. They can handwave enough away with their microcode updates because the source is not publicly auditable.

                      1. 1

                        The defense could try to get the people who are discovering these vulnerabilities in on the process to review the fixes. They’d probably have to do it under some kind of NDA which itself might be negotiable given a court is involved. Otherwise, someone who is not actively doing CPU breaks but did before can look at it. If it’s crap, they can say so citing independent evidence of why. If it’s not, they can say that, too. Best case is they even have an exploit for it to go with their claim.

                  2. 4

                    I don’t really think that you should be allowed to ask the users the sign a new EULA for security patches.

                    A variation of this argument goes that security issues should be backported or patched without also including new features. It is not a new or resolved issue.

                    Patches only count if they come with the same EULA as the original hardware/software/product.

                    What is different here is that this microcode update also requires operating system patches and possibly firmware updates. Further not everyone considers the performance trade-off worth it: there are a class of users for whom this is not a security issue. Aggravating matters, there are OEMs that must be involved in order to patch or explicitly fail to patch this issue. Intel had to coordinate all of this, under embargo.

                    1. 2

                      This reminds me of HP issuing a “security” update for printers that actually caused the printer to reject any third-party ink. Disgusting.

                      1. 2

                        I had not considered the case where manufacturers and end-users have different and divergent security needs.

                        1. 2

                          It’s worth thinking on more broadly since it’s the second-largest driver of insecurity. Demand being the first.

                          The easiest example is mobile phones. The revenue stream almost entirely comes from sales of new phones. So, they want to put their value proposition and efforts into the newest phones. They also want to keep costs as low as they can legally get away with. Securing older phones, even patching them, is an extra expense or just activity that doesn’t drive new phone sales. It might even slow them. So, they stop doing security updates on phones fairly quickly as extra incentive for people to buy new phones which helps CEO’s hit their goalposts in sales.

                          The earliest form I know of was software companies intentionally making broken software when they could spend a little more to make it better. Although I thought CTO’s were being suckers, Roger Schell (co-founder of INFOSEC) found out otherwise when meeting a diverse array of them under Black Forrest Group. When he evangelized high-assurance systems, the CTO’s told him they believed they’d never be able to buy them from the private sector even though they were interested in them. They elaborated that they believed computer manufacturers and software suppliers were intentionally keeping quality low to force them to buy support and future product releases. Put/leave bugs in on purpose now, get paid again later to take them out, and force new features in for lock-in.

                          They hit the nail on the head. Biggest examples being IBM, Microsoft, and Oracle. Companies are keeping defects in products in every unregulated sub-field of IT to this day. It should be default assumption with default mitigation being open API’s and data formats so one can switch vendors if encountering a malicious one.

                          EDIT: Come to think of it, the hosting industry does the same stuff. The sites, VPS’s, and dedi’s cost money to operate in a highly-competitive space. Assuming they aren’t loss-leaders, I bet profitability on the $5-10 VM’s might get down to nickles or quarters rather than dollars. There’s been products on market touting strong security like LynxSecure with Linux VM’s. The last time I saw price of separation kernels w/ networking and filesystems it was maybe $50,000. Some supplier might take that a year per organization just to get more business. They all heavily promote the stuff. Yet, almost all hosts use KVM or Xen. Aside from features, I bet the fact that they’re free with commoditized support and training factors into that a lot. Every dollar in initial profit you make on your VM’s or servers can further feed into the business’s growth or workers’ pay. Most hosts won’t pay even a few grand for a VMM with open solutions available, much less $50,000. They’ll also trade features against security like management advantages and ecosystem of popular solutions. I’m not saying any of this is bad choices given how demand side works: just that the business model incentivizes against security-focused solutions that currently exist.

                    2. 1

                      I think you have to be presented with the EULA before purchase for it to be valid anyway

                    1. 3

                      https://strugee.net/blog - the site is built using a static site generator that I wrote. (tl;dr: aggressively modular, built on streams, and the internals are squeaky clean and exposed to users, so you can trivially understand/change what’s going on.)

                      I write mostly about projects I maintain, the biggest of which is http://pump.io. There’s a bunch of other random stuff in there too though.

                      Warning: politics are in there too. This post is making me think I should make a more general “technology” category. I have a hidden category that feeds into Blaggregator; maybe I’ll make one for Planet Crustacean too!

                      1. 1

                        Isn’t this quite a lot of boiler-plate code for something rather simple?

                        1. 1

                          Whoops, forgot about this comment. I mean, there’s some boilerplate… but I’m not really sure what I could get rid of. I’ve seen worse.

                          1. 1

                            Well the whole node part, or am I mistaken? As far as I understand it, you’re just selecting a random file (image) from a directory to server for each HTTP request, no? It’s been a while since I used perl, but I suspect that this could be trivially done with a simple CGI script. Python and Go could even use their standard library, requiring no library dependencies, codes of conduct, contribution guidelines, package managment settings, etc.

                            1. 1

                              Oh I see. I mean, I suppose. I’m not familiar with writing CGI scripts, which is why I didn’t go that route. This could pretty easily be rewritten to not depend on Express, but I chose to use Express anyway just because it saved a little typing and because I’m used to it. The code of conduct I just copied from another project of mine and the contributing guidelines don’t really have a lot to do with the specifics of the code. I think you’d still want both those things regardless of language or whether you were using the standard library.

                        1. 11

                          This looks pretty interesting. I would love to see a line/word comparison with OpenSSH core, not including all the things that tinyssh doesn’t ship like scp etc. I also wonder what that comparison would look like if tinyssh implemented a lot of the security features OpenSSH does (e.g. authenticating untrusted clients in a chroot jail, at least IIRC), but I think that’ll be very difficult to figure out.

                          1. 6

                            I want to know where Microsoft and Apple stand on AV1. I remember when all the major players were duking it out over WebM or H.264; H.264 won (and Mozilla and Opera, who were pushing WebM, got pressured into adding patent-encumbered H.264 into their browsers by market forces).

                            AFAICT, that happened for three big reasons:

                            1. Apple and Microsoft implemented H.264 and refused to implement WebM. In retrospect I guess that made more sense for Microsoft since they were still in “we blindly hate anything with the word ‘open’ in it” mode. Apple made less sense to me.
                            2. Google promised that Chrome would drop H.264 support, but never followed through. At the time <video> was new enough, and Chrome had enough market share, that I really think they would have been able to turn the tide and score a victory for WebM if they had been serious. But apparently they weren’t.
                            3. H.264 had hardware partnerships which meant decoding was often hardware-accelerated - especially important for mobile performance. But I have no idea where I know that from so Citation Needed™.

                            I dunno, I think there’s hope for AV1 but that a lot could still go wrong. Apple I am particularly worried about due to iOS’ market share. If they refuse to implement the standard, it could seriously harm or even kill widespread adoption. But OTOH, maybe I’m just a pessimist :P

                            1. 6

                              A few months ago, Apple has announced that they joined the AV1 group and Microsoft was a founding member. That makes me much more optimistic than previous open formats.

                              I think the MPEG-LA really fucked things up with the minefield they set up for H.265.

                              https://www.cnet.com/google-amp/news/apple-online-video-compression-av1/

                              https://en.m.wikipedia.org/wiki/Alliance_for_Open_Media

                              1. 5

                                Apple and Microsoft implemented H.264 and refused to implement WebM. In retrospect I guess that made more sense for Microsoft since they were still in “we blindly hate anything with the word ‘open’ in it” mode. Apple made less sense to me.

                                Apple and Microsoft are both large corporations, and thus hydras; what one head said doesn’t necessarily reflect another. Still, they both have a foot in the game in three awful races: an attempt to be a monopoly without appearing to be such to regulators; both are heavily invested in software patents (a lose-lose game for everyone, but there’s a sunk cost fallacy problem here); heavy investment and affiliation with proprietary media companies.

                                I think the rest of your analysis on why h.264 made it in is right in gneral. Also, Cisco did the “here’s an open source h.264 implementation except if you modify it we might sue you for patent violations, so it’s not free software in practice” thing, and that was enough for various parties to check a box on their end, sadly.

                                BTW, I sat in on some of the RTCWeb IETF meetings where the battle over whether or not we would move to a royalty free default video codec on the web would happen then. I watched as a room mostly full of web activists not wanting patent-encumbered video to overtake the web were steamrolled by a variety of corporate representatives (Apple especially). A real bummer.

                                I’d like AV1 to do better… maybe it can by being actually better technology, and reducing a company’s bottom line by having a smaller bandwidth footprint, as it looks like they’re aiming for here. Dunno. Would love to hear more about strategy there.

                                1. 1

                                  Also, Cisco did the “here’s an open source h.264 implementation except if you modify it we might sue you for patent violations, so it’s not free software in practice” thing, and that was enough for various parties to check a box on their end, sadly.

                                  What exactly was happening there? IIRC Cisco basically said “we’ll eat the licensing costs on this particular implementation to fix this problem” so Mozilla/Opera(?) ended up using that to avoid the fees. Is that not what happened?

                                  I definitely remember Mozilla attempting to hold out for as long as possible. Eventually it became clear that Firefox couldn’t compete in the market without H.264 and that’s when the Cisco plugin went in.

                                  I watched as a room mostly full of web activists not wanting patent-encumbered video to overtake the web were steamrolled by a variety of corporate representatives (Apple especially).

                                  This is super gross.

                                2. 3

                                  Apple made less sense to me

                                  Apple is extremely sensitive to things that affect battery life of iOS devices. H.264 can be decoded in hardware on their devices. WebM would have to be decoded in software, so supporting it would be a worse experience for device reliability (battery would drain really fast on sites with lots of WebM content).

                                1. 26

                                  Given how many times over the years I had journald completely hose itself and freeze apps running on production systems [1] , I don’t find his arguments exceptionally compelling. Far more problems with journald/journalctl than I ever did with various syslog implementations. Yes you can still install syslog, but journald still gets the logs first, and then forwards/duplicates the data to syslog.

                                  Maybe journald is better now? Been a couple of years since I had to deal with it on high volume log systems. At the time we ended up using a program wrapper (something similar to logexec) that sent the logs directly to syslog, and avoided systemd/journald log handling entirely.

                                  [1]: app outputting some log data, journald stops accepting app output, app stdout buffer fills, app freezes blocking on write to stdout

                                  1. 7

                                    I see. Well nothing beats real world experience, so thank you very much for sharing that!

                                    1. 5

                                      For me it’s quite the opposite, I never had any issues with journald, neither in production nor in development environments.

                                      1. 4

                                        Seconded, I actually quite like that I can see all my logs the same way without setting up stuff on my side. With syslog I’d have to tell every program where to log and the systemd combo just takes away that manual burden.

                                        1. 3

                                          “works for me”

                                        2. 4

                                          I had this experience too, but that was because journald was hanging due to my disks being slow as molasses (I had deeper problems). I’m honestly not sure whether to blame journald for that.

                                        1. 8

                                          a.out binaries are smaller than elf binaries, so let’s statically link everything into one big executable ala busybox.

                                          Similarly, a modular kernel with all its modules takes up more total space than a single kernel with everything built in. So don’t even bother implementing modules. Linux 1.2 was the last great Linux before they ruined everything with modules.

                                          64-bit code takes up more space than 32-bit, so let’s build for 32-bit instruction sets. Who has more than 4GB of addressable memory anyway?

                                          Optimized code usually takes up more space, often a lot more when inlining is enabled. Let’s build everything with -Os so we can fit more binaries on our floppies.

                                          Icons are really superfluous anyway, but maybe we’ll want X11R5 or some other GUI on a second floppy. (I’d say X11R6 but all those extensions take up too much space). Make sure to use an 8-bit storage format with a common palette – 24-bit or 32-bit formats are wasteful.

                                          (I lament the bloated nature of the modern OS as much as the next Oregon Trail Generation hacker, but really – is “fits on a 1.7MB floppy really the right metric? Surely we can at least afford to buy 2.88MB drives now?)

                                          1. 5

                                            64-bit code takes up more space than 32-bit, so let’s build for 32-bit instruction sets. Who has more than 4GB of addressable memory anyway?

                                            Most programs don’t need more than 4GB of addressable memory, and those that do, know it. Knuth flamed about this some, but while you can use X32 to get the big registers and little memory, it’s not very popular because people don’t care much about making fast things.

                                            I lament the bloated nature of the modern OS as much as the next Oregon Trail Generation hacker, but really – is “fits on a 1.7MB floppy really the right metric? Surely we can at least afford to buy 2.88MB drives now?

                                            No, but smaller is better. If you can fit inside L1, you’ll find around 1000x speed increase simply because you’re not waiting for memory (or with clever programming: you can stream it).

                                            There was a time when people did gui workstations in 128kb. How fast would that be today?

                                            1. 2

                                              Most programs don’t need more than 4GB of addressable memory, and those that do, know it.

                                              All the integer overflows with values under 64-bits suggests otherwise. I know most programmers aren’t doing checks on every operation either. I prefer 64-bit partly to cut down on them. Ideally, I’d have an automated tool to convert programs to use it by default where performance or whatever allowed.

                                              “No, but smaller is better. If you can fit inside L1, you’ll find around 1000x speed increase simply because you’re not waiting for memory”

                                              Dave Long and I agreed on 32-64KB in a bootstrapping discussion for that very reason. Making that the maximum on apps kept them in the fastest cache even on lots of older hardware. Another was targeting initial loaders to tiny, cheap ROM (esp to save space for updates). Those were only memory metrics we could find that really mattered in general case. The rest were highly situation-specific.

                                              1. 1

                                                Most programs don’t need more than 4GB of addressable memory, and those that do, know it.

                                                All the integer overflows with values under 64-bits suggests otherwise.

                                                How?

                                                I know most programmers aren’t doing checks on every operation either. I prefer 64-bit partly to cut down on them. Ideally, I’d have an automated tool to convert programs to use it by default where performance or whatever allowed.

                                                What does that mean?

                                                1. 0

                                                  My point isn’t about the addressable memory: it’s about even being able to represent a number. Programs are usually designed with assumption that the arithmetic they do will work like real-world arithmetic on integers. In machine arithmetic, incrementing a number past a certain value will lead to an overflow. That can cause apps to misbehave. Another variation is a number coming from storage with many bits goes to one with fewer bits which caller didn’t know had fewer bits. That caused the Ariane 5 explosion.

                                                  Overflows happen more often with 8-16-bit fields since their range is so small. They can happen to 32-bit values in long running systems or those with math pushing numbers up fast. They either won’t happen or will take a lot longer with 64-bit values. I doubt most programmers are looking for overflows throughout their 32-bit applications. So, I’d rather just default on 64-bit for a bit of extra, safety margin. That’s all I was saying.

                                            2. 1

                                              Linux 1.2 was the last great Linux before they ruined everything with modules.

                                              https://twitter.com/1990sLinuxUser :P

                                              1. 2

                                                Why has systemd deprecated support for /usr on a different filesystem!!

                                                That issue bit me last month! I moved my /usr because it was too large, and the damned system couldn’t even boot into an init=/bin/sh shell! It dropped me into an initrd shell. I had to boot off a live CD to fix it. (If the initrd shell should have been sufficient, pardon me. I tried, but lvm wasn’t working.)

                                            1. 5

                                              Lots of things, because I’m terrible at concentrating on one thing, but in the next couple days I will probably continue working on audittool (I’m thinking of changing the name to depaudit except that dep-audit already exists…).

                                              audittool is basically a way to manage audits of (parts of) your npm dependency tree. So I can say e.g. I want to audit modules a, b and c and everything that they load for stupid behavior, because those modules are loaded as root before I drop privileges so they’re extra important. Or, I want to audit everything included by index.js to make sure that they have decent test suites so I know I can rely on the code. audittool will compute a dependency graph and then help you track exactly what you’ve checked for each module version, so when you upgrade (indirect) dependencies you can say, “help me audit only the new parts of this dependency tree for xyz property,” and the tool will know exactly what work you’ve already done and what still needs to be done.

                                              Side note, I sat down to write this last night and I was like “this actually doesn’t seem too bad; I bet I could bang out an MVP in a few hours.” Ended up sinking 4 hours into this bug, ragequitting^W giving up and going to bed, then spending 2 more hours on it today before filing that bug report. I’m pretty proud I finally found it, actually.

                                              1. 1

                                                I, for one, can only name a couple of software products that are rock solid.

                                                Does anyone know of a list or something that attempts to collect projects that are really really reliable and stable? I too can only came very few projects like this and I’d like to know of more for inspiration, examples, and to use myself.

                                                1. 2

                                                  I can share my list: TeX, Ynab Classic, Emacs and Vim (if you aren’t getting crazy with plugins), Sublime Text, OmniFocus, Devonthink (basic version at least).

                                                  I’d be interested in seeing a list of well crafted software. Maybe we could start a curated list or something?

                                                  1. 1

                                                    awesome-reliable-software, maybe? See awesome.

                                                1. 3

                                                  One nit: GPG 2.1+, I think, actually does start gpg-agent automatically on demand. But the main reason it does that is because for some unknown reason the GPG people decided to move most of the system’s functionality out of the gpg binary and into like 5 daemons that you now have to have running all the time and muddy the whole thing up. Why the old system was inadequate other than the fact that it was old and not shiny and overengineered is beyond me. (If someone knows, I would love to find out.)

                                                  That all being said I am very happy to see this experiment. PGP is awful. I’d love to see it finally die.

                                                  1. 10

                                                    GPG contains lots of engineering effort to make sure that keys are not accidentally leaked to swap, and that applications using gpg under the hood have a safe method of doing so.

                                                    I haven’t seen the GPG threat model fully documented but it’s definitely much more involved than Enchive’s.

                                                    1. 5

                                                      They split the program into communicating parts to help isolate the address spaces of the executables.

                                                      See the section of Neal’s talk starting at 0:31:45 - https://begriffs.com/posts/2016-11-05-advanced-intro-gnupg.html

                                                      1. 1

                                                        I finally made time to look at this, thanks! AFAICT there are two reasons he gave (I watched until about 40 minutes left in the video; the dumb player UI won’t show me how far in that is):

                                                        1. Each component has a separate address space
                                                        2. Future window managers with “trusted windows” could treat pinentry specially, “somehow”, because it’s forked from gpg-agent

                                                        Honestly I don’t see why either of those things couldn’t just be accomplished with the exact same architecture except using regular subprocesses instead of daemons. Can anyone give a reason that isn’t the case?

                                                        Reason 2 in particular seems like a lot of engineering to support a vaguely defined future scenario which may or may not show up, ever, and certainly does not exist now.

                                                      2. 2

                                                        They broke a whole bunch things when they moved to that new architecture and much if it was never fixed. It’s one of the reasons I stopped using GnuPG directly.

                                                      1. 1

                                                        I thought the Unix sed command at the very end (“Edit2:”) of this Microsoft article was pretty funny.

                                                        1. 5

                                                          I tried to think about what software I’ve used that is buggy and I realised it’s almost entirely websites and mobile apps. The actual applications I run on my Mac are pretty damn solid. Chrome, Sublime, WebStorm and PyCharm, Lightroom, Spotify even… They all work well. I actually couldn’t think of the last bug I ran into in any of them. There clearly are a bunch of engineers out there who care about the correctness and stability of their software and put a lot of work into it. But we don’t really hear from them too much (or, they’re buried under the endless self-promotion of the JS Framework Shootout crowd).

                                                          I work on a number of very buggy web applications right now. They’re built on (IMO) very poor decisions made by people who didn’t really know what they were doing, but who got a bunch of funding and built something anyway. In some ways that’s an impressive achievement (I’ve never built a successful company) but it’s also embarrassing. And my whole company seems to have internalised the idea that software fails. So production bugs are just normal things, as are weekly hot fixes to critical issues. It’s definitely a problem. We’re not doing anything to address the deep-rooted issues or the mistakes of the early days. No-one even mentions it. I don’t think anyone would dare to think that big.

                                                          1. 1

                                                            I work on a free software webapp project, pump.io. We try really hard not to bloat it with features and still I feel like I’m just flailing about in the dark with no real idea what I’m doing.

                                                            It’s written in JavaScript. I like JavaScript a lot, but I also wonder if that’s a big part of why I feel I have no control over the system. Maybe TypeScript will help with this.

                                                            There are an incredible amount of XXX and TODO comments scattered around. Lots were there when I took over maintenance, and lots were written by me. I wonder a lot if I’ll ever actually get to go back and fix them. I hope the answer is yes but realistically I’m guessing the answer is no because (external) things change too rapidly and I’ll never have enough stability to go do low-priority polish like that. It’s interesting and kinda sad to me that adding XXX comments to the code makes me feel better. It’s like I’d be in the exact same mess without them, but at least I feel like I have some grasp on the mess.

                                                            I don’t know. This article, and this comment bummed me out. They’re kinda spot on. The feeling I have now reminds me of Be Nice To Programmers. (I will note, though, that it is 2 AM and I’m sleep deprived.)

                                                          1. 6

                                                            down and it looks like archive.org doesn’t have the page :(

                                                            https://web.archive.org/web/*/riscv-basics.com

                                                            anyone have a copy?

                                                              1. 3

                                                                Lobsters’ built-in “cached” button worked for me just fine: https://archive.is/https%3A%2F%2Friscv-basics.com%2F

                                                                I wonder if they got embarrassed and took it down permanently.

                                                                1. 2

                                                                  It works if you prepend “https://” to it: https://web.archive.org/web/*/https://riscv-basics.com

                                                                  There’s actually quite a few snapshot over less than 48 hours, I guess there were lots of people who though this might get taken down pretty quick.

                                                                1. 2

                                                                  This reminds me very strongly of http://neveragain.tech/, right down to the point about IBM supporting the Holocaust. I tip my hat to these people for sticking to their ethics.

                                                                  1. 1

                                                                    The document doesn’t state where these clones come from. Are the Firefox numbers from a mozilla-central clone? Because if so it’s also including lots of code for Thunderbird and Lightning at least, and probably lots of legacy stuff like XULRunner, etc…. the list goes on.

                                                                    1. 31

                                                                      at this point most browsers are OS’s that run (and build) on other OS’s:

                                                                      • language runtime - multiple checks
                                                                      • graphic subsystem - check
                                                                      • networking - check
                                                                      • interaction with peripherals (sound, location, etc) - check
                                                                      • permissions - for users, pages, sites, and more.

                                                                      And more importantly, is there any (important to the writers) advantage to them becoming smaller? Security maybe?

                                                                      1. 11

                                                                        Browsers rarely link out the system. FF/Chromium have their own PNG decodes, JPEG decodes, AV codecs, memory allocators or allocation abstraction layers, etc. etc.

                                                                        It bothers me everything is now shipping as an electron app. Do we really need every single app to have the footprint of a modern browser? Can we at least limit them to the footprint of Firefox2?

                                                                        1. 10

                                                                          but if you limit it to the footprint of firefox2 then computers might be fast enough. (a problem)

                                                                          1. 2

                                                                            New computers are no longer faster than old computers at the same cost, though – moore’s law ended in 2005 and consumer stuff has caught up with the lag. So, the only speed-up from replacement is from clearing out bloat, not from actual hardware improvements in processing speed.

                                                                            (Maybe secondary storage speed will have a big bump, if you’re moving from hard disk to SSD, but that only happens once.)

                                                                            1. 3

                                                                              moore’s law ended in 2005 and consumer stuff has caught up with the lag. So, the only speed-up from replacement is from clearing out bloat, not from actual hardware improvements in processing speed.

                                                                              Are you claiming there have been no speedups due to better pipelining, out-of-order/speculative execution, larger caches, multicore, hyperthreading, and ASIC acceleration of common primitives? And the benchmarks magazines post showing newer stuff outperforming older stuff were all fabricated? I’d find those claims unbelievable.

                                                                              Also, every newer system I had was faster past 2005. I recently had to use an older backup. Much slower. Finally, performance isn’t the only thing to consider: the newer, process nodes use less energy and have smaller chips.

                                                                              1. 2

                                                                                I’m slightly overstating the claim. Performance increases have dropped to incremental from exponential, and are associated with piecemeal attempts to chase performance increase goals that once were a straightforward result of increased circuit density through optimization tricks that can only really be done once.

                                                                                Once we’ve picked all the low-hanging fruit (simple optimization tricks with major & general impact) we’ll need to start seriously milking performance out of multicore and other features that actually require the involvement of application developers. (Multicore doesn’t affect performance at all for single-threaded applications or fully-synchronous applications that happen to have multiple threads – in other words, everything an unschooled developer is prepared to write, unless they happen to be mostly into unix shell scripting or something.)

                                                                                Moore’s law isn’t all that matters, no. But, it matters a lot with regard to whether or not we can reasonably expect to defend practices like electron apps on the grounds that we can maintain current responsiveness while making everything take more cycles. The era where the same slow code can be guaranteed to run faster on next year’s machine without any effort on the part of developers is over.

                                                                                As a specific example: I doubt that even in ten years, a low-end desktop PC will be able to run today’s version of slack with reasonable performance. There is no discernible difference in its performance between my two primary machines (both low-end desktop PCs, one from 2011 and one from 2017). There isn’t a perpetually rising tide that makes all code more performant anymore, and the kind of bookkeeping that most web apps spend their cycles in doesn’t have specialized hardware accelerators the way matrix arithmetic does.

                                                                                1. 5

                                                                                  Performance increases have dropped to incremental from exponential, and are associated with piecemeal attempts to chase performance increase goals that once were a straightforward result of increased circuit density through optimization tricks that can only really be done once.

                                                                                  I agree with that totally.

                                                                                  “Multicore doesn’t affect performance at all for single-threaded applications “

                                                                                  Although largely true, people often forget a way multicore can boost single-threaded performance: simply letting the single-threaded app have more time on CPU core since other stuff is running on another. Some OS’s, esp RTOS’s, let you control which cores apps run on specifically to utilize that. I’m not sure if desktop OS’s have good support for this right now, though. I haven’t tried it in a while.

                                                                                  “There isn’t a perpetually rising tide that makes all code more performant anymore, and the kind of bookkeeping that most web apps spend their cycles in doesn’t have specialized hardware accelerators the way matrix arithmetic does.”

                                                                                  Yeah, all the ideas I have for it are incremental. The best illustration of where rest of gains might come from is Cavium’s Octeon line. They have offloading engines for TCP/IP, compression, crypto, string ops, and so on. On rendering side, Firefox is switching to GPU’s which will take time to fully utilize. On Javascript side, maybe JIT’s could have a small, dedicated core. So, there’s still room for speeding Web up in hardware. Just not Moore’s law without developer effort like you were saying.

                                                                        2. 9

                                                                          Although you partly covered it, I’d say “execution of programs” is good wording for JavaScript since it matches browser and OS usage. There’s definitely advantages to them being smaller. A guy I knew even deleted a bunch of code out of his OS and Firefox to achieve that on top of a tiny, backup image. Dude had a WinXP system full of working apps that fit on one CD-R.

                                                                          Far as secure browsers, I’d start with designs from high-assurance security bringing in mainstream components carefully. Some are already doing that. An older one inspired Chrome’s architecture. I have a list in this comment. I’ll also note that there were few of these because high-assurance security defaulted on just putting a browser in a dedicated partition that isolated it from other apps on top of security-focused kernels. One browser per domain of trust. Also common were partitioning network stacks and filesystems that limited effect of one partition using them on others. QubesOS and GenodeOS are open-source software that support these with QubesOS having great usability/polish and GenodeOS architecturally closer to high-security designs.

                                                                          1. 6

                                                                            Are there simpler browsers optimised for displaying plain ol’ hyperlinked HTML documents, and also support modern standards? I don’t really need 4 tiers of JIT and whatnot for web apps to go fast, since I don’t use them.

                                                                            1. 12

                                                                              I’ve always thought one could improve on a Dillo-like browser for that. I also thought compile-time programming might make various components in browsers optional where you could actually tune it to amount of code or attack surface you need. That would require lots of work for mainstream stuff, though. A project like Dillo might pull it off, though.

                                                                              1. 10
                                                                                1. 3

                                                                                  Oh yeah, I have that on a Raspberry Pi running RISC OS. It’s quite nice! I didn’t realise it runs on so many other platforms. Unfortunately it only crashes on my main machine, I will investigate. Thanks for reminding me that it exists.

                                                                                  1. 2

                                                                                    Fascinating; how had I never heard of this before?

                                                                                    Or maybe I had and just assumed it was a variant of suckless surf? https://surf.suckless.org/

                                                                                    Looks promising. I wonder how it fares on keyboard control in particular.

                                                                                    1. 1

                                                                                      Aw hell; they don’t even have TLS set up correctly on https://netsurf-browser.org

                                                                                      Does not exactly inspire confidence. Plus there appears to be no keyboard shortcut for switching tabs?

                                                                                      Neat idea; hope they get it into a usable state in the future.

                                                                                    2. 1

                                                                                      AFAIK, it doesn’t support “modern” non-standards.

                                                                                      But it doesn’t support Javascript either, so it’s way more secure of mainstream ones.

                                                                                    3. 8

                                                                                      No. Modern web standards are too complicated to implement in a simple manner.

                                                                                      1. 3

                                                                                        Either KHTML or Links is what you’d like. KHTML would probably be the smallest browser you could find with a working, modern CSS, javascript and HTML5 engine. Links only does HTML <=4.0 (including everything implied by its <img> tag, but not CSS).

                                                                                        1. 2

                                                                                          I’m pretty sure KHTML was taken to a farm upstate years ago, and replaced with WebKit or Blink.

                                                                                          1. 6

                                                                                            It wasn’t “replaced”, Konqueror supports all KHTML-based backends including WebKit, WebEngine (chromium) and KHTML. KHTML still works relatively well to show modern web pages according to HTML5 standards and fits OP’s description perfectly. Konqueror allows you to choose your browser engine per tab, and even switch on the fly which I think is really nice, although this means loading all engines that you’re currently using in memory.

                                                                                            I wouldn’t say development is still very active, but it’s still supported in the KDE frameworks, they still make sure that it builds at least, along with the occasional bug fix. Saying that it was replaced is an overstatement. Although most KDE distributions do ship other browsers by default, if any, and I’m pretty sure Falkon is set to become KDE’s browser these days, which is basically an interface for WebEngine.

                                                                                        2. 2

                                                                                          A growing part of my browsing is now text-mode browsing. Maybe you could treat full graphical browsing as an exception and go to the minimum footprint most of the time…

                                                                                      2. 4

                                                                                        And more importantly, is there any (important to the writers) advantage to them becoming smaller? Security maybe?

                                                                                        user choice. rampant complexity has restricted your options to 3 rendering engines, if you want to function in the modern world.

                                                                                        1. 3

                                                                                          When reimplementing malloc and testing it out on several applications, I found out that Firefox ( at the time, I don’t know if this is still true) had its own internal malloc. It was allocating a big chunk of memory at startup and then managing it itself.

                                                                                          Back in the time I thought this was a crazy idea for a browser but in fact, it follows exactly the idea of your comment!

                                                                                          1. 3

                                                                                            Firefox uses a fork of jemalloc by default.

                                                                                            1. 2

                                                                                              IIRC this was done somewhere between Firefox 3 and Firefox 4 and was a huge speed boost. I can’t find a source for that claim though.

                                                                                              Anyway, there are good reasons Firefox uses its own malloc.

                                                                                              Edit: apparently I’m bored and/or like archeology, so I traced back the introduction of jemalloc to this hg changeset. This changeset is present in the tree for Mozilla 1.9.1 but not Mozilla 1.8.0. That would seem to indicate that jemalloc landed in the 3.6 cycle, although I’m not totally sure because the changeset description indicates that the real history is in CVS.

                                                                                          2. 3

                                                                                            In my daily job, this week I’m working on patching a modern Javascript application to run on older browsers (IE10, IE9 and IE8+ GCF 12).

                                                                                            The hardest problems are due the different implementation details of same origin policy.
                                                                                            The funniest problem has been one of the used famework that used “native” as variable name: when people speak about the good parts in Javascript I know they don’t know what they are talking about.

                                                                                            BTW, if browser complexity address a real problem (instead of being a DARPA weapon to get control of foreign computers), such problem is the distribution of computation among long distances.

                                                                                            Such problem was not addressed well enough by operating systems, despite some mild attempts, such as Microsoft’s CIFS.

                                                                                            This is partially a protocol issue, as both NFS, SMB and 9P were designed with local network in mind.

                                                                                            However, IMHO browsers OS are not the proper solution to the issue: they are designed for different goals, and they cannot discontinue such goals without loosing market share (unless they retain such share with weird marketing practices as Microsoft did years ago with IE on Windows and Google is currently doing with Chrome on Android).

                                                                                            We need better protocols and better distributed operating systems.

                                                                                            Unfortunately it’s not easy to create them.
                                                                                            (Disclaimer: browsers as platforms for os and javascript’s ubiquity are among the strongest reasons that make me spend countless nights hacking an OS)

                                                                                          1. 1

                                                                                            I’m late to the party but this reminds me of Isaac Schlueter on “No”, which “stands for Node Optimized and is also gonna be what [he] says any time someone asks for a feature.”

                                                                                            1. 1

                                                                                              This is labeled security, but I don’t really understand how it’s related. Is there a security issue here that I’m missing? Or just an “ordinary” Unicode handling bug?