1. 2

    Why are they calling the new API Temporal instead of just Time?

    1. 7

      Not breaking widely used scripts. Look for Array.prototype.flatten (aka smooshgate) in your favorite search engine.

      1.  

        Totally get that, and I understand why they’re not just changing the semantics of Date. But Time isn’t in use, so why not use that?

        1.  

          With Array.flatten they had broken websites with JS libraries as old as MooTools (iirc), I wouldn’t be surprised if there was a popular libraries that breaks when there’s a built in Time. That all being said, I am just guessing.

          1.  

            Time appears to be part of it.

      1. 8

        I seem to be in writing mode, how timely!

        • We’re releasing Firefox 79 tomorrow and it was my turn to write the Security Advisories (published tomorrow).
        • I’ve just finished writing a second part of Understanding Web Security Checks in Firefox (Part 1), to be published next week or the week after. The idea is to encourage new original browser security researcher and engineering.
        • We’ve also just finished our Q2 edition of the Firefox Security & Privacy Newsletter. To be published next week.
        • I’m also writing a specification for a Javascript API that will accept a HTML string and returns sanitized (i.e., XSS-free) HTML. We’ll hopefully make it into a W3C standard. A Proof of Concept implementation shall land in Nightly anytime soon. The spec draft is at https://wicg.github.io/sanitizer-api/
        1. 28

          To be fair, it’s not just Apple, mostly Mozilla takes a similar approach: https://twitter.com/voxpelli/status/1286230638526435329

          1. 10

            So, we have the browser engine vendors today: Apple with Safari/WebKit, Mozilla with Firefox/Gecko, and Google with Chrome/Blink. Isn’t it kind of weird that so many web standards are being standardized with 2/3 of vendors unwilling to implement them? What’s the process here?

            1. 13

              They are drafts, drafts don’t necessarily get passed to be standards. For an example at hand, Geolocation API is a standard (ratified in 2016). Geolocation Sensor is not, it is a draft (last updated in 2018).

              1. 28

                Aha. From the title and article, it sounded like Apple refuses to implement standard APIs. So the real story is just that Apple and Mozilla won’t let some harmful APIs get standardized.

                1. 25

                  Yes.

                  1. 2

                    Only it doesn’t really matter. Since Chrome is so big, whatever it does is a de-facto standard because Web developers are going to use those APIs, and users are going to blame other browsers for “not working”, which is going to maintain its share because of it.

                    1. 9

                      I would think that Safari on iPhone has enough market share to force web developers to support it. It would surprise me if a commercial website intentionally disregarded MobileSafari support.

                      1. 3

                        They’ll just try to push the users to their mobile apps.

                        1. 3

                          Market share of mobile safari is actually quite poor. It’s usually supported despite the market share, as iPhone users are widely regarded valuable users (e.g., more likely to spend money online)

                          1. 3

                            I think it might also depend on where your customers are–even if iOS is only around 15% of the worldwide smartphone market, it’s 58% of the US, 51% of North America, and 26% of Europe.

                        2. 3

                          So the real story is that Google is using its near-monopoly power to circumvent the standards process? There’s some kind of irony here, but I just can’t tell WHAT.

                          1. 1

                            WHAT was a great force when Mozilla needed to pry the Web from Microsoft. It created a standard on which Firefox and later Chrome could build better browsers than IE and win users over. But then Google got big and took the process over, so here we are.

                            1. 2

                              No disagreement here! But worth pointing out that Mozilla could only make that move because Apple and Opera were backing them. I just think the important things to keep in mind about standards organizations is that they are inherently political, and that those with a seat at the table are generally large corporations who answer only to their shareholders. As such, they should be understood as turf where players jockey for competitive advantage by forming temporary strategic alliances. I think everyone paying attention to these things understands how this works, except for some programmers, who I guess are conditioned to treat even draft standards as holy writ descended directly from the inscrutable heavens, or maybe take the rhetoric about “serving users” a little too literally.

                              But as consolidation erodes consumer choice, there’s less of a game to play, and thus standards become less relevant.

                1. 6

                  I accomplished today what I was planning to do on Sunday: migrate HardenedBSD’s self-hosted gitea instance’s database from sqlite3 to mysql due to performance reasons. Who knew that hosting multiple large git repos (some with commit histories > 25 years) would carry a large database burden? ;-P

                  On Saturday, my wife and I may go to an outdoor crab feast. I don’t really want to go, especially with COVID-19 and the temperature (97F with 50-60% humidity), but we’ll see how tomorrow plays out.

                  I’ll probably get some rest on Sunday. Next week’s gonna be an incredibly busy week. I convinced the entire executive board at ${DAYJOB} to transition our entire dev team to integrate our entire product (written as Windows applications over the past couple years) into OPNsense. That means a complete rewrite. I’ll be leading that effort. :) We’re building a security appliance that can do full pcap at 40Gbps line rate in a 4U system.

                  1. 2

                    Woah, good luck on that migration.

                    Commit histories > 25 years? Git isn’t even that old.

                    1. 5

                      You can integrate history from svn, cvs etc., that’s why the Mozilla repos have history from before Firefox was a thing.

                      1. 1

                        TIL

                    1. 3

                      I find this a strong article that is relevant to us techies, but I know that this is on the verge of being off-topic and I know that I am biased (Mozilla employee).

                      Looking forward to seeing whether this gets a discussion or a downvote :-)

                      Happy Friday!

                      1. 3

                        Stories can’t be downvoted here.

                        1. 1

                          Yeah, I meant flagged as off topic. I got what I asked for :-))

                        2. 2

                          I marked this as spam because it looks like spam to me. I suppose there could be debate about whether this is spam or just off-topic?

                        1. 1

                          If you use the “current Tab” permission, your extension would be slightly less creeprier. Additionally, you may want to switch from browserAction to pageAction API, to make your UI fit with existing patterns. Essentially, sharing is an action on the current page, not on the browser. See https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/pageAction

                          1. 16

                            also annonymizes the request IPs using MD5 hashing

                            Hashing an IPv4 address, which has a very small search space, especially with something as fast as MD5, is not anonymization. On modern hardware it wouldn’t take very long to generate a rainbow table for the entirety of the IPv4 address space, which you can then look up every IP in.

                            1. 2

                              Came here to say this. Common anonymization techniques involve replacing the last half of the IP address with nulls.

                            1. 5

                              This is despite dark UX patterns in favor of allowing? Huh, as if it wasn’t worth asking and folks could just not implement tracking in the first place.

                              1. 30

                                As far as I can tell, the 9% number comes from an experiment which used an ethical dialog box.

                                I would imagine the statistics for the commonly-used unethical alternatives are not public.

                              1. 9

                                Nice idea. You’ll need to work on your security against XSS and local/remote file inclusion attacks. I’ll send you a DM.

                                1. 5

                                  Company: Mozilla

                                  Company Site: https://careers.mozilla.org/listings/

                                  Positions+Locations: Neutral Machine Translation Engineer (Berlin ONSITE), Senior Front-End Engineer (Atlanta, Denver or REMOTE US), Senior User Experience Researcher (REMOTE US, limited Contract) and more

                                  ** Description*: You might have heard of that Firefox browser, but Mozilla is bigger than that. I’m going to steal the tagline from the careers page: “Pursue your future while working to protect the future of the internet for everyone, everywhere.”.

                                  Tech Stack: lots.

                                  Contact: I’m not a hiring manager. Use the careers page and feel free to add me as a reference. Send me DMs for questions about positions/teams and I’ll try my best to answer (or find someone who will answer)

                                  1. 1

                                    That’s a lot of initiatives to eradicate memory safety bugs as a vulnerability class. Kudos!

                                    1. 6

                                      German perspective: You’re an engineer after getting a university degree in an engineering program (e.g., bachelor of science in information technology). You’re only allowed to call yourself an engineer after graduating. But then immediately.
                                      Depending on school, program that may or may not involve lots of meta/how-to engineer courses.

                                      No extra test but also no way for non-academics to become an engineer unless they go to a university. :/

                                      1. 5

                                        Similar in Sweden. The term “civilingenjör” covers all engineering, not just civil engineers (that degree is called “civilingenjör väg och vatten”) and is a protected term. You’re not allowed to present yourself as such unless you have graduated from an accredited institution (nor are you allowed to purchase the special ring you’re entitled to).

                                        1. 3

                                          Similar in France. “Ingénieur” covers all engineering and is also a protected term. But a lot of people are obsessed by degrees here in France and it kinda sucks.

                                          1. 2

                                            Not exactly.

                                            “Ingénieur” (Engineer) is not protected. It is a job title and can be given to you as long as your job corresponds.

                                            The protected constructions are things like “Diplôme d’Ingénieur”, “École d’Ingénieur” and “Ingénieur diplômé” (Engineering Degree, Engineering School and “Engineer with a degree”, respectively). They are regulated by a body called CTI.

                                        1. 4

                                          Wouldn’t it make more sense to have some kind of HTTP header and/or meta tag that turns off javascript, cookies and maybe selected parts of css?

                                          If we could get browser vendors to treat that a bit like the https padlock indicators, some kind of visual indicator that this is “tracking free”

                                          Link tracking will be a harder nut to crack. First we turn off redirects. Only direct links to resources. Then we make a cryptographic proof of the contents of a page - something a bit fuzzy like image watermarking. Finally we demand that site owners publish some kind of list of proofs so we can verify the page is not being individually tailored to the current user.

                                          1. 11

                                            The CSP header already allows this to an extent. You can just add script-src none and no JavaScript can run on your web page.

                                            1. 1

                                              very true. not visible to the user though!

                                            2. 5

                                              Browsers already render both text/html and application/pdf, and hyperlinking works. There is no technical barrier to add, say, text/markdown into mix. Or application/ria (see below), for that matter. We could start by disabling everything which already requires permission, that is, audio/video capture, location, notification, etc. Since application/ria would be compat hazard, it probably should continue to be text/html, and what-ideally-should-be-text/html would be something like text/html-without-ria. This clearly works. The question is one of market, that is, whether there is enough demand for this.

                                              1. 5

                                                Someone probably should implement this as, say, Firefox extension. PDF rendering in Firefox is already done with PDF.js. Do the exact same thing for Markdown by: take GitHub-compatible JS Markdown implementation with GitHub’s default styling. Have “prefer Markdown” preference. When preference is set, send Accept: text/markdown, text/html. Using normal HTTP content negotiation, if server has text/markdown version and sends it, it is rendered just like PDF. Otherwise it works the same, etc. Before server supports arrive, the extension probably could intercept well known URLs and replace content with Markdown, for, say Discourse forums. Sounds like an interesting side project to try.

                                                1. 8

                                                  Browsers already render both text/html and application/pdf, and hyperlinking works. There is no technical barrier to add, say, text/markdown into mix.

                                                  Someone probably should implement this as, say, Firefox extension.

                                                  Historical note: this is how Konqueror (the KDE browser) started. Konqueror was not meant be a browser, but a universal document viewer. Documents would flow though a transport protocol (implemented by a KIO library) and be interpreted by the appropriate component (called KParts) (See https://docs.kde.org/trunk5/en/applications/konqueror/introduction.html)

                                                  In the end Konqueror focused on being mostly a browser, or an ad-hoc shell around KIO::HTTP and KHTML (the parent of WebKit) and Okular (the app + the KPart) took care of all main “document formats” (PDFs, DejaVu, etc).

                                                  1. 2

                                                    Not saying it’s a bad idea, but there are important details to consider. E.g. you’d need to agree on which flavor of Markdown to use, there are… many.

                                                      1. 2

                                                        Eh, that’s why I specified GitHub flavor?

                                                        1. 1

                                                          Oops, my brain seems to have skipped that part when I read your comment, sorry.

                                                          The “variant” addition in RFC 7763 linked by spc476 to indicate which of the various Markdowns you’ve used when writing the content seems like a good idea. No need to make Github the owner of the specification, IMHO.

                                                        2. 1

                                                          What’s wrong with Standard Markdown?

                                                      2. 2

                                                        markdown

                                                        Markdown is a superset of HTML. I’ve seen this notion put forward a few times (e.g., in this thread, which prompted me to submit this article), so it seems like this is a common misconception.

                                                      3. 4

                                                        Why would web authors use it? I can imagine some small reasons (a hosting site might mandate static pages only), but they seem niche.

                                                        Or is your hope that users will configure their browsers to reject pages that don’t have the header? There are already significant improvements on the tracking/advertising/bloat front when you block javascript, but users overwhelmingly don’t do it, because they’d rather have the functionality.

                                                        1. 2

                                                          I think the idea is that it is a way for web authors to verifiably prove to users that the content is tracking free. Markdown renderer would be tracking free unless buggy. (It would be a XSS bug.) The difference with noscript is that script-y sites still transparently work.

                                                          In the invisioned implementation, like HTTPS sites getting padlock, document-only sites will get cute document icon to generate warm fuzzy feeling to users. If icon is as visible as padlock, I think many web authors will use it if it is in fact a document and it can be easily done.

                                                          Note that Markdown renderer could still use JavaScript to provide interactive features: say collapsible sections. It is okay because JavaScript comes from browser, which is a trusted source.

                                                        2. 3

                                                          Another HTTP header that maybe some browsers will support shoddily, and the rest will ignore?

                                                          1. 2

                                                            I found HTTP Accept header to be well supported by all current relevant softwares. That’s why I think separate MIME type is the way to go.

                                                          2. 2

                                                            I think link tracking is essentially impossible to avoid, as are redirects. The web already has a huge problem with dead links and redirects at least make it possible to maintain more of the web over time.

                                                            1. 2
                                                            1. 23

                                                              I have a nagging feeling that I’m missing something here. It doesn’t seem right that such an obvious solution would have been left on the table, by everyone, for decades.

                                                              Browser vendors.

                                                              They’re Why We Can’t Have Nice Things; they refuse to add UI for basic HTTP and TLS-level features and force everyone to roll their own replacements at a higher level that tend to suck. Imagine if browsers implemented HTTP Basic Auth in a way that didn’t look like it was straight out of 1996 … how much pointless code wouldn’t need to be written.

                                                              1. 7

                                                                Managing a custom CA and renewal and everything is a serious pain worth avoiding. Especially when dealing with errors for non-technical users. UX is terrible and keeping a secret file was asking too much of many people. That’s why https + username + password won in e-commerce. Lowest friction.

                                                                Enterprises are different of course. Less users to worry with. Centralized specific documentation for a reduced set of supported client software.

                                                                1. 1

                                                                  Especially when dealing with errors for non-technical users. UX is terrible and keeping a secret file was asking too much of many people.

                                                                  I can see why you wouldn’t want to use your Mozilla hat on this post..

                                                                  1. 1

                                                                    what do/did you see? Curious to hear if our understanding aligns.

                                                                    Parts of this thread are about browsers but my experience and my comment isn’t. I co-managed a tiny CA with computer security students about 10 years ago. Failure mode was hard and breaking assignment labs is a lot of bad stress. I don’t wanna know what it’s like with paying customers.

                                                                    I haven’t done any crypto related stuff at Mozilla. Mostly focusing on web/browser security. Doesn’t really make sense to use the hat, don’t you think?

                                                                    1. 3

                                                                      You state that https + username + password won because they’re the lowest friction, and you’re right. You also state that this is because of (among others) bad UX with other solutions. You’re also right there.

                                                                      Bad UX is a browser problem; no browser has done any serious work on a generic authentication UX. Basic Authentication in Firefox still presents a dialog box that looks like it’s made in the 90s. Client side certificate management is cumbersome, and using client side certificates is hard. These are not technology problems, these are UX problems.

                                                                      Our situation would be better, considering both security and UX, if browsers made authentication a first class citizen. Web developers would have it easier, users would have a more consistent experience and we would not have so many custom broken login implementations, because in that timeline letting the browser handle the authentication would have been the solution with lowest friction.

                                                                      Because of this, I see browser vendors as a big part of the problem, hence my remark about you not wearing your hat. Mozilla made a step in the right direction a while ago when they announced Persona, but it’s been discontinued for longer than it has been alive now.

                                                                      1. 1

                                                                        Whatever blame you’re trying to throw, it won’t stick. I’m not your crypto/logins guy. Anyway you might wanna try WebAuthn to solve this properly? Doesn’t have the tracking issues too.

                                                                2. 4

                                                                  “Why use existing layers as a basis to the layer above while we can replace layers below with extra layers put on top”

                                                                  We are so much used to this scheme that it looks familiar everywhere we go.

                                                                  1. 1

                                                                    Agreed: it’s a “nice” solution from a system design standpoint but sometimes IRL I re-open a browser window and 5 tabs each suddenly need my PIN number, one after the other, or they never default to the right cert, &c. Plus even when it works, it can be super laggy.

                                                                    If the user agent was a more effective key agent, it would be great!

                                                                  1. 4

                                                                    I’m surprised LMDB is not mentioned in the alternatives, I think Firefox does use it (Mozilla published a Rust binding..)

                                                                    The more I look at LMDB, the more I want all software ever to access memory-mapped data directly and avoid copies as much as possible. Is this a ridiculous obsession or am I correct? :D

                                                                      1. 2

                                                                        As part of some internal libraries building, I’ve researched LMDB as a potential application file format a few years ago. The biggest critique I have is that once the size of the LMDB set, it cannot grow (at least cannot grow while the app is active). This is problematic for mobile apps or any app want to be memory conscious. You basically have to mmap large enough upfront with limited ways to grow / shrink dynamically. That coupled with some weird accounting on mobile OSes make it a no-go (I think certain iOS versions simply don’t really evict pages that was once dirty even if you msync back to SSD).

                                                                        1. 4

                                                                          Your criticism is correct in many ways, but still requires significant clarification.

                                                                          We should distinguish between the limitations of an approach (using a single flat memory-mapped file) and the disadvantages of implementing it in the LMDB.

                                                                          POSIX mmap API supported by (almost?) all OSes, including iOS and Android. This allows to dynamically increase and shrink a mapped file with insignificant restrictions:

                                                                          • The mapping size (but not the file size) should be selected with a margin. This consumes PTE, but not RAM or disk space.
                                                                          • Subsequent mapping extension (not the file, but memory-mapping itself) require either mremap() (with free address space behind the current mapping region), or a re-creation of the mapping is required (with data access suspended during this time).
                                                                          • Reducing the size of the mapped data is easy to control by using posix_madvise(POSIX_MADV_DONTNEED) and reducing the file size. However, this requires defragmentation inside the database or calling madvise() for each page to be released (which is expensive for performance).

                                                                          The problem with LMDB is that none of the above is done, or the corresponding API is not provided. So I would like to pay attention to libmdbx, where this problem was resolved:

                                                                          About page evicting in mobile OSes I should clarify even more:

                                                                          • Basically (for iOS) a used page in RAM could be in one of the statuses: clean (could be unloaded and then re-loaded from file), dirty (modified and should be written to file or swap), compressed (compressed by OS dirty page).
                                                                          • It is no any problem to application to have a lot of clean pages, but it should avoid extra dirty and/or compressed pages.
                                                                          • In case of libmdbx (and LMDB) this could be easy avoided just not using MDBX_WRITEMAP flag (MDB_WRITEMAP for LMDB).

                                                                          P.S. As the main developer of libmdbx, I will be happy to answer any related questions.

                                                                          1. 1

                                                                            Thanks for the detailed answer. Looks interesting. I may be able to pick it up later sometime this year to re-evaluate. It was based on observations done a few years ago and certainly the landscape would change. It is also the first time I heard of libmdbx which sounds like to address some of the shortcomings.

                                                                            Ignoring the fact that I haven’t read your code yet, just out of curiosity (I am always interested in mmap techniques), if we don’t mmap it as readable, what are the techniques you use for writes? Do you just fseek and fwrite or use a journal and consolidating all writes back during compactification?

                                                                            1. 2

                                                                              if we don’t mmap it as readable, what are the techniques you use for writes?

                                                                              Thanks to the Unified Page Cache (aka Unified Virtual Memory) this is easy. I.e. if you map a local file read-only into RAM and pwrite() into this file such changes becomes immediately visible in the mapped memory region for all local processes which have mmap‘ed this file. Since 2000’x the Unified Page Cache implemented for almost all OSes, except OpenBSD, QNX and a few other embedded platforms. Without the unified page cache, a database file must be mapped read-write by each process that wants to perform write-transactions. See description of the MDBX_WRITEMAP option.

                                                                              … consolidating all writes back during compactification?

                                                                              The internals of MDBX are complex enough to be briefly explained. However, the detailed information about the internal mechanisms of MDBX and LMDB is easily found on the web, starting with https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database

                                                                              Nevertheless you should keep in mind that MDBX is a descendant of LMDB and all improvements are described mostly just in the README (https://erthink.github.io/libmdbx/#improvements-beyond-lmdb) and in the API description (https://github.com/erthink/libmdbx/blob/master/mdbx.h).

                                                                          2. 3

                                                                            Maybe mdbx is better about this?

                                                                            1. 2

                                                                              Sure :)

                                                                              See my answer below.

                                                                        1. 16

                                                                          I’ve been bitten once by the modern use of JSON files in firefox. I don’t have a bug report at hand, but the issue is that since the whole file gets rewritten on save, a crash at the wrong time can make you end up with an empty (0 bytes) file. This happened to me with the password database, and it was quite hard to fix. I ended up creating a new profile. SQLite databases tend to resist damage a lot better.

                                                                          1. 5

                                                                            That seems like a fixable problem. The trick would be to write the file with a temporary name, then fsync, then rename it over the top of the old one. Depending on when it crashes, you might lose the changes, but you shouldn’t get torn writes.

                                                                            1. 1

                                                                              rename() should be atomic on any sane posix filesystem. might be able to skip the whole fsync() call altogether.

                                                                              1. 3

                                                                                But then the atomic rename may be of an incomplete file…

                                                                                1. 1

                                                                                  Someone correct me if I’m wrong, but for my own understanding it sounds like the process is:

                                                                                  • Write output to temp file
                                                                                  • One way or another ensure temp file gets committed to disk
                                                                                  • Rename temp file to real file

                                                                                  And then in the case of a crash, start over from the beginning? That may lose you data in a crash but will always be consistent. Otherwise you need to verify that the temp file is valid before doing stage 3, or detect a crash and attempt to recover something, or something along those lines…

                                                                                  1. 1

                                                                                    Perhaps I’m mistaken and doing dumb things, but I’ve intentionally induced power faults testing my temp to rename over method without fsync and it seems to work, vs opening and editing or just overwriting the file underlying.

                                                                                    1. 3

                                                                                      iirc ext4 filesystem actually inserts an fsync for you in this case because it is such a common mistake.

                                                                                  2. 1

                                                                                    When you’re dealing with crashes, all bets are off

                                                                                  3. 1

                                                                                    Funny you say that, this is exactly what FF does for downloads.

                                                                                    1. 0

                                                                                      To encourage less SQLlite usage, providing an API to manage this (compressed JSON files, loaded+saved via tmpfile/fsync/rename) seems a sensible approach, and probably not much more effort than writing this guide.

                                                                                        1. 1

                                                                                          Thanks, interesting, the non-default version (if you provide a tmpFile and ask for ‘flush’) seems to do this safely. I wonder if a higher-level API would be of use (save/load a json object, no need to provide tmpfile name etc)

                                                                                          The description of the ‘flush’ argument suggests it is doing an fsync(), rather than using stdio and doing ‘fflush’, but if so the terminology is confusing.

                                                                                          I would check, but I don’t want to check out all of firefox and but I’m having difficulty searching up “NativeOSFileInternals.cpp” to look.

                                                                                            1. 2

                                                                                              Try searchfox.org

                                                                                    2. 2

                                                                                      A similar thing can happen if you run out of disk space. During writing the file will first get pruned, then some program will use up all the available space in the meantime and your write will fail with the end result being an empty file. This can be avoided by creating a temporary file in the same location as the destination file and then renaming it.

                                                                                    1. 3

                                                                                      besides the good survey this is also a superb intro of terms like “exploit primitive” and generally how obscure, seemingly little vulnerabilities are often times tied together to yield full remote control.

                                                                                      1. 11

                                                                                        Not everything is UTF-8, but it should be.

                                                                                        1. 3

                                                                                          Do you think should file paths on Unix systems be UTF-8 instead of simply being encoding-agnostic byte sequences terminated by 0x00 and delimited by 0x2F? I can see it both ways personally. As it stands, file paths are not text, but they’re nearly always treated as text. All text definitely should be UTF-8, but are file paths text? Should they be text?

                                                                                          1. 28

                                                                                            Paths consist of segments of file names. File names should be names. Names should be text. Text represented as a sequence of bytes must have a specified encoding, otherwise it’s not text. Now the only question left is: which encoding should we use? Let’s just go with UTF-8 for compatibility with other software.

                                                                                            I would actually put further restrictions on that:

                                                                                            • file names should consist of printable characters — what good is a name if the characters it’s made of cannot be displayed?
                                                                                            • file names shouldn’t be allowed to span multiple lines — multiline file names will only cause confusion and will often be harder to parse (not just for humans, but also for CLI programs)

                                                                                            As it is now in Unix, file names aren’t for humans. And neither are they for scripts. They’re for… file systems.

                                                                                            1. 6

                                                                                              I agree with you about those restrictions. In some sense, Windows has succeeded in this area, where Unix has failed. In Unix:

                                                                                              • File names can begin with a hyphen, which creates ambiguity over whether it is a command-line flag or an actual file (prompting the convention of -- separating flags from file arguments).
                                                                                              • File names can contain newlines, which creates almost unsolvable problems with most Unix tools.

                                                                                              In Windows, however (source):

                                                                                              • File names cannot contain forward slashes, and thus cannot be confused with command-line flags (which begin with a slash).
                                                                                              • File names cannot contain line feeds or carriage returns. All characters in the range 0-31 are forbidden.
                                                                                              • File names cannot contain double quotation marks, which means you can very easily parse quoted file names.

                                                                                              Of course, both allow spaces in file names, which creates problems on both systems. If only shell/DOS used commas or something to separate command-line arguments instead of spaces…

                                                                                              1. 1

                                                                                                Windows also doesn’t allow files named NUL, PRN, or CON. :D

                                                                                              2. 5

                                                                                                There is a long essay by David A. Wheeler about problems that are caused by weird filenames and what we might do to fix them. Including suggestions on possible restrictions that operating systems might impose on valid filenames. Prohibiting control characters (including newline) is one of the items on the list. Scroll down to the very bottom to see it.

                                                                                                1. 2

                                                                                                  Ideally you want to bring reasonable naming capabilities to folks from the world of non-latin character sets. That’s a really good driver to go beyond existing C-Strings and other “Os String” encodings.

                                                                                                  But when you say “UTF-8, but printable”, it’s not UTF-8 anymore. Also, what’s a “line”? 80 characters? 80 bytes? Everything that doesn’t contain a newline? Mh. Allowing UTF-8 will bring some issues with Right-To-Left override characters and files named “txt.lol.exe” on certain operating systems.

                                                                                                  It’s tough, isn’t it? :-)

                                                                                                  1. 7

                                                                                                    Also, what’s a “line”? 80 characters? 80 bytes?

                                                                                                    Anything that doesn’t contain a newline. The point is that filenames with newlines in them break shell tools, GUI tools don’t allow you to create filenames with newlines in them anyway, and very few tools other than GNU ls have a reasonable way to present them.

                                                                                                    Lots of stuff doesn’t allow you to include the ASCII control plane. Windows already bans the control plane from file names. DNS host names aren’t allowed to contain control characters (bare “numbers and letters and hyphen” names certainly don’t, and since domain registrars use a whitelist for extended characters, I doubt you could register a punycode domain with control characters either). The URL standard requires control plane characters to be percent encoded.

                                                                                                    1. 1

                                                                                                      \r\n or \n? ;) You get my point?

                                                                                                      1. 3
                                                                                                        1. If it includes \r\n, then it includes \n.

                                                                                                        2. If the goal is to avoid breaking your platform’s own default shell, then the answer should be “whatever that shell uses.”

                                                                                              3. 3

                                                                                                “Is” and “ought”, however, remain dangerous things to confuse.

                                                                                              1. 2

                                                                                                Now they just need to release the Ryzen 4000 models, and I may just be ready to move on from my x230…

                                                                                                1. 2

                                                                                                  I switched to an x390 and don’t regret it

                                                                                                  1. 1

                                                                                                    I just read the specs for that, and am still convinced that every new “modern” laptop now is a step backwards.

                                                                                                    My x230 has 2x the memory (16GB) and it’s soldered down (lol) on the x390 so you’re stuck with that forever. Is the hard drive at least replaceable? I guess I value repairability more than most consumers now, because I’m sick of having to throw away electronics after ~2 years. The x390 looks just like another disposable laptop. (the “17.5hr” battery life is super impressive though, but probably inflated)

                                                                                                    1. 3

                                                                                                      The disk is just a user replaceable M.2 NVMe drive. You can get 16GB of RAM in an X390 as well if you choose the i7 CPU option.

                                                                                                      1. 2

                                                                                                        from what I’ve seen, almost all SSDs are user replaceable (m.2) in modern laptops

                                                                                                        a notable exception is Apple, who uses a proprietary type of drive (because of course they do)