-
The problem turns out to be some obscure FUSE mounts that the author had lying around in a broken state, which subsequently broke the kernel namespace system. Meanwhile, I have been running systemd on every computer I’ve owned in many years and have never had a problem with it.
Does this not seem a bit melodramatic?
-
From the twitter thread:
Systemd does not of course log any sort of failure message when it gives up on setting up the DynamicUser private namespace; it just goes ahead and silently runs the service in the regular filesystem, even though it knows that is guaranteed to fail.
It sounds like the system had an opportunity to point out an anomaly that would guide the operator in the right direction, but instead decided to power through anyways.
-
-
-
-
We have monitoring and alerting around how much money is coming in, that we compare with historical data and predictions. It’s actually a very reliable canary for when things go wrong, and for when they are right again, on the scale of seconds to a few days. But you are right that things getting a little suckier slowly over a long time would only show up as real growth not being in line with predictions.
-
-
-
I tend to agree that hard failures are nicer in general (especially to make sure things work), but I’ve also been in scenarios where buggy logging code has caused an entire service to go down, which… well that sucked.
There is a justification for partial service functionality in some cases (especially when uptime is important), but like with many things I think that judgement calls in that are usually so wrong that I prefer hard failures in almost all cases.
-
-
Everybody thinks it’s a good idea “some service is surely better than no service” until it happens to them.
So if the server is over capacity, kill it and don’t serve anyone?
Router can’t open and forward a port, so cut all traffic?
I guess that sounds a little too hyperbolic.
But there’s a continuum there. At $work, I’ve got a project that tries to keep going even if something is wrong. Honest, I’m not sure I like how all the errors are handled. But then again, the software is supposed to operate rather autonomously after initial configuration. Remote configuration is a part of the service; if something breaks, it’d be really nice if the remote access and logs and all were still reachable. And you certainly don’t want to give up over a problem that may turn out to be temporary or something that could be routed around… reliability is paramount.
-
And you certainly don’t want to give up over a problem that may turn out to be temporary
I think that’s close to the core of the problem. Temporary problems recur, worsen, etc. I’m not saying it’s always wrong to retry, but I think one should have some idea of why the root problem will disappear before retrying. Computers are pretty deterministic. Transient errors indicate incomplete understanding. But people think a try-catch in a loop is “defensive”. :(
-
-
-
-
So you never had legacy systems (or configurations) to support? I read Chris’ blog regularly, and he works at a university on a heterogeneous network (some Linux, some other Unix systems) that has been running Unix for a long time. I think he started working there before
systemdwas even created. -
Why do you say that the FUSE mounts were broken? As far as we can see they were just set up in a uncommon way https://twitter.com/thatcks/status/1027259924835954689
-
-
Yes, it seems melodramatic, even to my anti-systemd ears. It’s a documentation and error reporting problem, not a technical problem, IMO. Olivier Lacan gave a great talk last year about good errors and bad errors (https://olivierlacan.com/talks/human-errors/). I think it’s high time we start thinking about how to improve error reporting in software everywhere – and maybe one day human-centric error reporting will be as ubiquitous as unit testing is today.
-
In my view (as the original post’s author) there are two problems in view. That systemd doesn’t report useful errors (or even notice errors) when it encounters internal failures is the lesser issue; the greater issue is that it’s guaranteed to fail to restart some services under certain circumstances due to internal implementation decisions. Fixing systemd to log good errors would not cause timesyncd to be restartable, which is the real goal. It would at least make the overall system more debuggable, though, especially if it provided enough detail.
The optimistic take on ‘add a focus on error reporting’ is that considering how to report errors would also lead to a greater consideration of what errors can actually happen, how likely they are, and perhaps what can be done about them by the program itself. Thinking about errors makes you actively confront them, in much the same way that writing documentation about your program or system can confront you with its awkward bits and get you to do something about them.
-
-
-
The challenging open issue I see with this sort of idea is the question of what links people will see by default when they view a page. If they see only the links the author put in, then we will generally have a situation no different than today (as most people will never change that default). If they see some additional set of links by default, then there will be ferocious competition from spammers to get their links included in that set and in general large arguments about what links will be included in it.
For better or worse, HTML and browser technology today has a simple, distributed, scalable, and clearly fair answer to the question of ‘what links appear in a page by default’, and it’s one that keeps browsers and other central parties out of disputes (and generally out of the game of influencing the answer).
(I admit that these days I look at all new protocols through the lens of ‘how can they be abused by spammers and other bad people’, but partly this is because we know spammers and other bad people are out there and will actively attempt to abuse anything they can.)
-
The challenging open issue I see with this sort of idea is the question of what links people will see by default when they view a page.
This is something we were concerned with at Xanadu during the development of XanaSpace (since we had all links in the form of loadable ODLs). The conclusion we came to was that a document author could recommend a particular set of links to go with their document, and that furthermore, people would produce and share collections of links (which, since they are not part of the content & are bidirectional, combine with transclusion to add additional context even to documents where the author is unaware of them) in sort of the same way as kottke.org or BoingBoing curates collections of other people’s web pages. Any resident links (including formatting links) would be applied when relevant (i.e., when the original source of any transcluded content overlapped with something mentioned in a link), and a person’s personal link collection would be private until shared.
This sort of mirrors the fediverse / SSB model of requiring intentional hops between independent communities. Taking advantage of default link sets for spamming purposes only makes sense when the landscape is flat – where everybody sees everything unless they take countermeasures, and thus anything, no matter the quality, scales up indefinitely with no further human input. If, on the other hand, things only spread when they are actively shared by individuals from across different communities (each acting as curator for the sake of their community), the impact of these problems becomes small and it ceases to be worth the effort for bad actors.
Ultimately, the links that appear on the page should be controllable by the person viewing the page, just like the formatting of the page should be under their control. In the case of links, that probably means trusting your friends and a handful of professional curators to have good taste & not scam you.
-
This is really interesting. I did a reasonable chunk of reading around Xanadu in my research for this project, but ended up spending more time looking at ‘open hypermedia’ stuff and didn’t have the time to dive into a lot of the nitty gritty details as much as I would have liked. The details you’ve mentioned, for instance, seem to have totally passed me by.
Curiously, although these kinds of ecosystem considerations weren’t really the main focus of my work, I think I came to somewhat similar (though less developed) conclusions in the assumptions I made for my prototyping work. If you wouldn’t mind, do you think you’d be able to point me towards a source for the design discussions you mentioned? I’d love to read more about this.
-
Most Xanadu documentation is purely internal to the project, & the stuff that gets released is usually pretty non-technical. The discussions around how ODLs would be distributed were never made public at all, as far as I can tell (and they are part of a now-abandoned subproject). However, I did a technical overview of all of the Xanadu stuff I was privy to that wasn’t under trade secret(mirrored here), and this is probably the most complete & accessible public documentation on the project.
ODL distribution isn’t covered in detail here, and XanaSpace never got to the point where it was seriously discussed in a systematic way, though there were some ideas thrown around, which I should document.
Specifically: there was the concept of a ‘xanaful’ – a tarball containing all of the files (EDLs, ODLs, links, and sourcedocs) necessary for constructing a constellation of related documents. Paths in the tar format are just strings prefixing the content blob, so we were planning to use the full permanent address of each piece of content as its path, and check all resident xanaful tarballs for those addresses before fetching them from elsewhere. The idea is that a xanaful would be a convenient way for people to share not-yet-public documents, distribute stuff on physical media to be used where network access is limited, send bookmarks to friends in big chunks, distribute private ODLs (which contain formatting – and therefore themeing – links in addition to inter-content links), and get people who are a little skeptical to try a xanadu viewer out. Sending a xanaful out-of-band (for instance, by email) would be one method of ODL distribution.
I wanted & was pushing for more of a peer-to-peer system. Specifically, I wanted individual XanaSpace applications to serve up the public parts of their caches to peers (to limit strain on hosts), and I wanted inter-peer communication to support a kind of friends network for sharing EDLs and ODLs directly with particular people. I was thinking of using gopher for this, since gopher is awfully simple to implement. (This is the system I wanted to use for transcopyright’s encryption oracle: the author’s machine or some trusted proxy would take requests for the OTP segment, check against a whitelist of authorized users, and distribute the OTP segment or a zeroed-out dummy.) There are some legal issues with this (which IPFS and SSB are discussing as well) and Ted wasn’t really comfortable with jumping into full distributed computing; also, this cut out any potential profit, and Ted still thinks of Xanadu as potentially profitable. As a result, none of these particular ideas got taken very seriously or had serious development work attached to them.
Post-XanaSpace, our translit viewers have ditched the ODL entirely in favor of sticking link addresses in the EDL. (Our code always supported intermixing the two, but there was a conceptual division that I thought was useful.) It makes things easier for newcomers to understand but I think it does so at the cost of some clarity: now, people can still pretend that the author owns all the links in their document, and this only becomes clearly untrue when two authors link transclusions of overlapping segments from two resident documents. The web-based translit viewers only support one resident document at a time, so this never happens. (Having many resident documents at once is a vital feature & so we shouldn’t expect later implementations to keep this trend except accidentally.)
-
-
-
-
tl;dw;?
-
According to the guy behind handmade hero:
The fact that we currently have hardware vendors shipping both hardware and drivers (with USB and GPUs being to major examples), rather than just shipping hardware with a defined/documented interface, a la x64, or the the computers of the 80s, is a very large contributor to the fact that we have basically 3 consumer-usable OSes, and each one is well over 15 million lines of code. These large codebases are a big part of the reason that using software today can be rather unpleasant
He proposes that if hardware vendors switched form a hardware+drivers to hardware that was well-documented in how it was controlled, so that most programmers could program it by feeding memory to/from it (which he considers an ISA of sorts), we’d be able to eliminate the need for drivers as such, and be able to go back to the idea of a much simpler OS.
I haven’t watched the whole thing yet, but that’s the highlights
-
Oh I would so, so, so, love that to happen…..
…but as a guy whose day job is at that very interface I will point this out.
The very reason for the existence of microcomputers is to soak up all the stuff that is “too hard to do in hardware”.
Seriously, go back to the original motivations for the first intel micros.
And as CPU’s have become faster, more and more things get “winmodemed”.
Remember ye olde modems? Nice well defined rs-232 interface and standardized AT command set?
All gone.
What happen?
Well, partly instead of having a separate fairly grunty/costly CPU inside the modem and a serial port… you could just have enough hardware to spit the i/q’s at the PC and let the PC do the work, and shift the AT command set interpretor into the driver. Result, cheaper better modems and a huge pain in the ass for open source.
All the h/w manufacturers regard their software drivers as an encryption layer on top of their “secret sauce”, their competitive advantage.
At least that’s what the bean counters believe.
Their engineers know that the software drivers are a layer of kludge to make the catastrophe that is their hardware design limp along enough to be saleable
But to bring their h/w up to a standard interface level would require doing some hard (and very costly) work at the h/w level.
Good luck convincing the bean counters about that one.
Of course, WinTel regard the current mess as a competitive advantage. It massively raises the barriers to entry to the market place. So don’t hold your breathe hoping WinTel will clean it up. They created this mess for Good (or Bad depending on view) reasons of their own.
-
All the h/w manufacturers regard their software drivers as an encryption layer on top of their “secret sauce”, their competitive advantage.
I thought the NDA’s and obfuscations were about preventing patent suits as much as competitive advantage. The hardware expert that taught me the basics of cat and mouse games in that field said there’s patents on about everything you can think of in implementation techniques. The more modern and cutting edge, the more dense the patent minefield. Keeping the internals secret means they have to get a company like ChipWorks (now TechInsights) to tear it down before filing those patent suits. Their homepage prominently advertises the I.P.-related benefits of their service.
-
That too definitely! Sadly, all this comes at a huge cost to the end user. :-(
-
-
-
The obvious pragmatic problem with this model is that hardware vendors sell the most hardware (and sell it faster) when people can immediately use their hardware, not when they must wait for interest parties to write device drivers from it. If the hardware vendor has to write and ship their own device drivers anyway, writing and shipping documentation is an extra cost.
(There are also interesting questions about who gets to pay the cost of writing device drivers, since there is a cost involved here. This is frequently going to be ‘whoever derives the most benefit from having the device driver exist’, which is often going to be the hardware maker, since the extra benefit to major OSes is often small.)
-
-
-
If you modify one byte in a file with the default 128KB record size, it causes the whole 128KB to be read in, one byte to be changed, and a new 128KB block to be written out.
Recordsize “enforces the size of the largest block written”1 (emphesis mine), not that all blocks are that size.
-
ZFS recordsize is extremely confusing. That description is correct at a filesystem level, but not at a file level; for a single file (or zvol), there is only one logical block size and if the file has multiple blocks, that logical block size will be the filesystem recordsize. So on a normal filesystem, if you have a file larger than 128 Kb, all its logical blocks will be 128 Kb and if you modify one byte, you do indeed rewrite 128 Kb. I had to go through an entire process of writing test files under various circumstances and dumping them with zdb to understand what was going on.
(Compression may create different physical block sizes. One potentially surprising form of this compression is compressing the implicit zero bytes at the end of files that are not exact multiples of 128 Kb. So with compression, your 160 Kb incompressible file will come out close to 160 Kb allocated on disk, instead of 256 Kb.)
-
-
Secondly, I think that compression works more efficiently when given larger blocks to compress.
It can. In some database situations it’s suggested to use a very large blocksize, like 1meg, to maximize compression.
I also wonder if the author’s problem is some sort of alignment? Their 4k image was not aligned to 4k blocks and that caused extra blocks to be written? I have no idea if that is possible in this situation.
-
My understanding is that ZFS only compresses blocks separately, since it has to be able to do random IO to them. With small block sizes on some drives, this can create a surprisingly low space savings from compression due to the fact that ZFS can only write disk-level blocks. If your disks are using 4K physical blocks (ie they’re ‘advance format’ drives) and you’re using, say, 8K logical blocks, in order to save any space from compression you need at least a 50% compression ratio, so that your original 8K block can be written in a single 4K physical block; if it doesn’t compress that much, it has to be written using two 4K blocks and you save no space. If you’re using 128 Kb logical blocks you can win from much smaller compression ratios, because you don’t have to shrink as much in order to need fewer physical blocks.
(SSDs are somewhat inconsistent in whether they report having 512 byte physical blocks or 4K physical blocks. It’s all smoke and mirrors on SSDs anyway, so the whole mess is inconvenient for ZFS.)
-
Like anything: it depends. You probably won’t notice much difference between 4k blocks and 8k blocks for storing video files. But you might notice large differences when storing logs. I think what you’re saying is right, I’m just drawing more attention to the “it depends on what you’re storing” aspect of it.
EDIT: Another place that matters too is Compressed ARC. Better compression you can get there the more data you can jam into RAM. Ramajama
-
-
-
Shamar
edited
4 months ago
|
link
| on:
Apple, Google, Microsoft, Mozilla formally object to W3C's DOM 4.1 spec
TLDR;
- In 2004 Apple, Mozilla and Opera were becoming increasingly concerned about the W3C’s direction with XHTML, lack of interest in HTML, and apparent disregard for the needs of real-world web developers and created WHATWG as a way to get control over the web standards
- they throw away a whole stack of powerful web technologies (XHTML, XSLT…) whose purpose was to make the web both machine readable and useful to humans
- they invented Live Standards that are a sort of ex-post standards: always evolving documents, unstable by design, designed by their hands-on committee, that no one else can really implement fully, to establish a dynamic oligopoly
- in 2017, Google and Microsoft joined the WHATWG to form a Steering Group for “improving web standards”
- meanwhile the W3C realized that their core business is not to help lobbies spread broken DRM technologies, and started working to a new version of the DOM API.
- in 2018, after months of political negotiations, they proposed to move the working draft to recommendation
- in 2018, Google, Microsoft, Apple and Mozilla felt offended by this lack of lip service.
It’s worth noticing that both these groups have their center in the USA but their decisions affects the whole world.
So we could further summarize that we have two groups, one controlled by USA lobbies and the other controlled by the most powerful companies in the world, fighting for the control of the most important infrastructure of the planet.
Under Trump’s Presidency.
Take this, science fiction! :-D
-
This is somewhat disingenuous. Web browser’s HTML parser needs to be compatible with existing web, but W3C’s HTML4 specification couldn’t be used to build a web-compatible HTML parser, so reverse engineering was required for independent implementation. With WHATWG’s HTML5 specification, for the first time in history, a web-compatible HTML parsing got specified, with its adoption agency algorithm and all. This was a great achievement in standard writing.
Servo is a beneficiary of this work. Servo’s HTML parser was written directly from the specification without any reverse engineering, and it worked! To the contrary to your implication, WHATWG lowered barrier to entry for independent implementation of web. Servo is struggling with CSS because CSS is still ill-specified in the manner of HTML4. For example, only reasonable specification of table layout is an unofficial draft: https://dbaron.org/css/intrinsic/ For a laugh, count the number of times “does not specify” appear in CSS2’s table chapter.
-
-
-
Disingenuous? Me? Really? :-D
Who was in the working group that wrote CSS2 specification?
I bet a coffee that each of those “does not specify” was the outcome of a political compromise.
But again, beyond the technical stuffs, don’t you see a huge geopolitical issue?
-
-
This is an interesting interpretation, but I’d call it incorrect.
- the reason to create whatwg wasn’t about control
- XHTML had little traction, because of developers
- html5 (a whatwg standard fwiw) was the first meaningful HTML spec because it actually finally explained how to parse it
- w3c didn’t “start working on a new Dom”. They copy/backport changes from whatwg hoping to provide stable releases for living standards
- this has nothing to do with DRM (or EME). These after completely different people!
- this isn’t about lobby groups, neither is this avout influencing politics in the US or anywhere.
I’m not speaking on behalf of my function in the w3c working group I’m in, nor for Mozilla. But those positions provided me with the understanding and background information to post this comment.
-
XHTML had little traction, because of developers
I remember that in early 2000s everyone started to write
<br/>instead of<br>and it was considered cool and modern. There were 80x15 badges everywhere saying website is in xhtml. My Motorola C380 phone supported wap and some xhtml websites, but not regular html in builtin browser. So I had impression that xhtml was very popular.-
-
Agreed. Worth noting that, after the html5 parsing algorithm was fully specified and libraries like html5lib became available, it became possible to apply exactly the same approach with html5 parsers outputting a DOM structure and then querying it with xpath expressions.
-
-
-
This is an interesting interpretation, but I’d call it incorrect.
You are welcome. But given your arguments, I still stand with my political interpretation.
the reason to create whatwg wasn’t about control
I was 24 back then, and my reaction was “What? Why?”.
My boss commented: “wrong question. You should ask: who?”
XHTML had little traction, because of developers
Are you sure?
I wrote several web site back then using XML, XSLT and XInclude serverside to produce XHTML and CSS.
It was a great technological stack for distributing contents over the web.
w3c didn’t “start working on a new Dom”. They copy/backport changes from whatwg hoping to provide stable releases for living standards
Well, had I wrote a technical document about an alternative DOM for the whole planet, without anyone asking me to, I would be glad if W3C had take my work into account!
In what other way they can NOT waste WHATWG’s hard work?
Wel, except saying: “guys, from now on do whatever Google, Apple, Microsoft and few other companies from the Silicon Valley tell you to do”.But I do not want to take part for W3C: to me, they lost their technical authority with EME (different group, but same organisation).
The technical point is that we need stable, well thought, standards. What you call live standard, are… working draft?
The political point is that no oligopoly should be in condition to dictate the architecture of the web to the world.
And you know, in a state where strong cryptography is qualified as munitions and is subject to export restrictions.
I’m not speaking on behalf of my function in the w3c working group I’m in, nor for Mozilla. But those positions provided me with the understanding and background information to post this comment.
I have no doubt about your good faith.
But probably your idealism is fooling you.
As you try to see these facts from a wider perspective, you will see the problem I describe.
-
XHTML was fairly clearly a mistake and unworkable in the real world, as shown by how many nominally XHTML sites weren’t, and didn’t validate as XHTML if you forced them to be treated as such. In an ideal world where everyone used tools that always created 100% correct XHTML, maybe it would have worked out, but in this one it didn’t; there are too many people generating too much content in too many sloppy ways for draconian error handling to work well. The whole situation was not helped by the content-type issue, where if you served your ‘XHTML’ as anything other than application/xhtml+xml it wasn’t interpreted as XHTML by browsers (instead it was HTML tag soup). One result was that you could have non-validating ‘XHTML’ that still displayed in browsers because they weren’t interpreting it as XHTML and thus weren’t using strict error handling.
(This fact is vividly illustrated through syndication feeds and syndication feed handlers. In theory all syndication feed formats are strict and one of them is strongly XML based, so all syndication feeds should validate and you should be able to consume them with a strictly validating parser. In practice plenty of syndication feeds do not validate and anyone who wants to write a widely usable syndication feed parser that people will like cannot insist on strict error handling.)
-
there are too many people generating too much content in too many sloppy ways for draconian error handling to work well.
I do remember this argument was pretty popular back then, but I have never understood why.
I had no issue in generating xhtml strict pages from user contents. This real world company had a couple handred of customers with pretty various needs (from ecommerce, to online magazines or institutional web sites) and thousands of daily visitors.
We used XHTML and CSS to distribute highly accessible contents, and we had pretty good results with a prototype based on XLS-FO.
To me back then the call to real world issues seemed pretestuous. We literally had no issue. The issues I remember were all from IE.
You are right that many mediocre software were unable to produce proper XHTML. But is this an argument?
Do not fix the software, let’s break the specifications!
It seems a little childish!
XHTML was not perfect, but it was the right direction.
Look at what we have now instead: unparsable contents, hundreds of incompatible javascript frameworks, subtle bugs, bootstrap everywhere (aka much less creativity) and so on.
Who gain most from this unstructured complexity?
The same who now propose the final
solutionlock-in: web assembly.Seeing linux running inside the browser is not funny anymore.
Going after incompetent developers was not democratization of the web, it was technological populism.
-
What is possible does not matter; what matters is what actually happens in the real world. With XHTML, the answer is clear. Quite a lot of people spent years pushing XHTML as the way of the future on the web, enough people listened to them to generate a fair amount of ‘XHTML’, and almost none of it was valid and most of it was not being served as XHTML (which conveniently hid this invalidity).
Pragmatically, you can still write XHTML today. What you can’t do is force other people to write XHTML. The collective browser world has decided that one of the ways that people can’t force XHTML is by freezing the development of all other HTML standards, so XHTML is the only way forward and desirable new features appear only in XHTML. The philosophical reason for this decision is pretty clear; browsers ultimately serve users, and in the real world users are clearly not well served by a focus on fully valid XHTML only.
(Users don’t care about validation, they care about seeing web pages, because seeing web pages is their goal. Preventing them from seeing web pages is not serving them well, and draconian XHTML error handling was thus always an unstable situation.)
That the W3C has stopped developing XHTML and related standards is simply acknowledging this reality. There always have been and always will be a great deal of tag soup web pages and far fewer pages that validate, especially reliably (in XHTML or anything else). Handling these tag soup web pages is the reality of the web.
(HTML5 is a step forward for handling tag soup because for the first time it standardizes how to handle errors, so that browsers will theoretically be consistent in the face of them. XHTML could never be this step forward because its entire premise was that invalid web pages wouldn’t exist and if they did exist, browsers would refuse to show them.)
-
Users don’t care about validation, they care about seeing web pages, because seeing web pages is their goal.
Users do not care about the quality of concrete because having a home is their goal.
There will always be incompetent architects, thus let them work their way so that people get what they want.Users do not care about car safety because what they want is to move from point A to point B.
There will always be incompetent manufacturers, thus let them work their way so that people get what they want.That’s not how engineering (should) work.
Was XHTML flawless? No.
Was it properly understood by the average web developers that most companies like to hire? No.Was it possible to improve it? Yes. Was it better tha the current javascript driven mess? Yes!
The collective browser world has decided…
Collective browser world? ROTFL!
There’s a huge number of browsers’ implementors that nobody consulted.
Among others, in 2004, the most widely used browser, IE, did not join WHATWG.
Why WHATWG did not used the IE design if the goal was to liberate developers from the burden of well designed tools?
Why we have faced for years incompatibilities between browsers?
WHATWG was turned into one of the weapons in a commercial war for the control of the web.
Microsoft lost such war.
As always, the winner write the history that everybody know and celebrate.
But who is old enough to remember the fact, can see the hypocrisy of these manoeuvres pretty well.
There was no technical reason to throw away XHTML. The reasons were political and economical.
How can you sell Ads if a tool can easily remove them from the XHTML code? How can you sell API access to data, if a program can easily consume the same XHTML that users consume? How can you lock users, if they can consume the web without a browser? Or with a custom one?
The WHATWG did not served users’ interests, whatever were the Mozilla’s intentions in 2004.
They served some businesses at the expense of the users and of all the high quality web companies that didn’t have much issues with XHTML.
Back then it was possible to disable Javascript without loosing access to the web functionalities.
Try it now.
Back then people were exploring the concept of semantic web with the passion people now talk about the last JS framework.
I remember experiments with web readers for blind people that could never work with the modern js polluted web.
You are right, W3C abandoned its leadership in the engineering of the web back then.
But you can’t sell to a web developer bullshit about HTML5.
Beyond few new elements and a slightly more structured page (that could have been done in XHTML too) all its exciting innovations were… more Javascript.
Users did not gain anything good from this, just less control over contents, more ads, and a huge security hole worldwide.
Because, you know, when you run a javascript in Spain that was served to you from a server in the USA, who is responsible for such javascript running on your computer? Under which law?
Do you really think that such legal issues were not taken into account from the browser vendors that flued this involution of the web?
I cannot believe they were so incompetent.
They knew what they were doing, and did it on purpose.
Not to serve their users. To use those who trusted them.
-
-
-
-
-
-
-
After 20 years of Berlusconi and with our current empasse with the Government, no Italian could ever troll an American about his current President.
It was not my intention in any way.
As @olivier said, I was pointing to this surreal situation from an international perspective.
USA control most of internet: most root DNS, the most powerful web companies, the standards of the web and so on.
Whatever effect Cambridge Analitica had to the election of Trump, it has shown the world that internet is a common infrastructure that we have to control and protect together. Just like we should control the production of oxigen and global warming.
If Cambridge Analitica was able to manipulate USA elections (by manipulating Americans), what could do Facebook itself in Italy? Or in German?
Or what could Google do in France?The Internet was a DARPA project. We can see it is a military success beyond any expectation.
I tried to summarize the debacle between W3C and WHATWG with a bit of irony because, in itself, it shows a pretty scary aspect of this infrastructure.
The fact that a group of companies dares to challenge W3C (that, at least in theory, is an international organisation) is an evidence that they do not feel the need to pretend they are working for everybody.
They have too much power, to care.
-
The last point is the crux of the issue: are technologists willing to do the leg work of decentralizing power?
Because regular people won’t do this. They don’t care. This, they should have less say in the issue, though still some, as they are deeply affected by it too.
-
No. Most won’t.
Technologist are a wide category, that etymologically includes everyone that feel entitled to speak about how to do things.
So we have technologists that mislead people to invest in the “blockchain revolution”, technologists that mislead politicians to allow barely tested AI to kill people on the roads, technologists teaching in the Universities that neural networks computations cannot be explained and thus must be trusted as superhuman oracles… and technologists that classify as troll any criticism of mainstream wisdom.
My hope is in hackers: all over the world they have a better understanding of their political role.
-
-
If anyone wonders about Berlusconi, Cracked has a great article on him that had me calling Trump a pale imitation of Berlusconi and his exploits. Well, until Trump got into US Presidency which is a bigger achievement than Berlusconi. He did that somewhat by accident, though. Can’t last 20 years either. I still think Berlusconi has him beat at biggest scumbag of that type.
-
Yeah, the article is funny, but Berlusconi was not. Not for Italians.
His problems with women did not impress much us. But for when it became clear most of them were underage.
But the demage he did to our laws and (worse) to our public ethics will last for decades.
He did not just changed the law to help himself: he destroyed most legal tools to fight the organized crime and to fight bribes and corruption.
Worse he helped a whole generation of younger people like him to be bold about their smartness with law workarounds.I pray for the US and the whole world that Trump is not like him.
-
-
-
-
An interesting link. A bit light on the sourcing and heavy on the opinion, but hey, it’s Medium.
It kind of glossed over why Canon became dominant in the later ages of film - the story goes that Nikon introduced AF (everyone was working on this, it was a Big Thing) and asked pros whether they liked it or not. Pros being pros, they said it was too slow for pro (sports) work, and Nikon said, fair enough. Canon on the other hand, decided to fix the AF problem, which they did, captured a huge share of the market, and left Nikon scrambling to catch up.
I have no sources for this, just memories and anecdotes picked up on the internet. Sorry.
Another interesting thing is that Nikon only lately, with the introduction of electronically controlled aperture mechanisms in the lens, has come to parity with the Canon EF mount - just electronic contacts, no screws or levers.
-
My understanding is that part of Canon’s edge in the 1990s ultimately came from the fact that they were willing to move to a new mount in order to enable better AF and other things (FD to EOS/EF in 1987). Arguably Canon could afford to do this because they were the underdogs; moving to a new mount and thus instantly orphaning all of their existing customers wasn’t the huge deal it would have been for Nikon, because Canon didn’t have as many customers.
(In the context of this Medium story, I think this is an important difference. It shifts the lessons from ‘Nikon not grasping things’ to ‘underdogs can afford to do disruptive things that leaders can’t’.)
-
-
Canon did have a lot of customers. The Canon FD mount cameras were very popular and successful, and they caused Nikon a lot of pain as they captured the “lower end” of the amateur market.
Ultimately the move to EF was a good choice, but it caused a lot of bad blood. Pros could afford to move eventually, but a lot of other shooters who had “invested” in the FD system were pissed. Part of it was of course because FD lenses and bodies instantly lost potential resale value, but I also think a large part of it was the feeling of being “cheated” by the company they had chosen. Fanboyism is by no means only confined to the internet age.
-
I think there’s more to it than fanboyism. Interchangeable lens cameras are a system, and when you buy into a system one of the things you’re taking a bet on is the continued life and development of the system. When Canon changed their lens mount, they killed FD as a system. Your existing FD gear would still work, but you weren’t getting any improved future cameras or any improved future lenses. People are naturally angry about a system being ended on them this way; it would be somewhat like Apple declaring out of the blue one day that they were stopping all future development of MacOS hardware and software, and what you had now was all you’d ever have. The drop in the resale value of your current hardware would be the least of your worries.
(Not entirely like it, because 1980s cameras didn’t have security vulnerabilities.)
-
That’s a good point… however, it was clear to everyone that AF was the future, and whether you shot Nikon or Canon you would have to update all your gear to take availability of the new features. Granted, many people were perfectly happy to keep on using older manual focus Nikkor glass on AF Nikons, and that path was denied to Canon shooters.
-
One difference between a gradual shift (in any sort of system) and an abrupt, all at once shift is that in the latter, you’re dependent on the new system having everything you need right away. If it doesn’t, either you have to do without, delay your shift (and do without the new system’s benefits), or use two systems for a while (in this case, carrying two cameras, using two sets of film, etc).
I don’t know how many Canon FD lenses were available in 1987 at the time that Canon announced EOS/EF, but I suspect that EOS didn’t launch with equivalents of all of them available. Although according to this source (and also) Canon does seem to have had an impressively large number of lenses available that year. Interestingly, it took until 1989 for them to put out an 85 mm prime, despite that being a common portrait focal length.
-
One of the things that probably irked Canon FD shooters was that there was no way to adapt the FD lenses to EF without an adapter with optical elements.
Canon did make such an adapter but it was special order only, and supposedly offered to owners of FD superteles so they could continue using these lenses on the new EOS system.
-
-
-
-
-
-
Hello..ouch light on sourcing :-) I included photos of magazine articles, ads, and brochures as well as my own lens ;-) What am I missing.
I tried not to gloss over this but clearly it didn’t sink in. Nikon was extremely focused on professionals and took a very conservative approach for this reason. They tended to view AF as a consumer feature and kept a very clean separation between consumer and pro cameras over concerns of both cannibalization and making pro cameras seem too gimmicky (even to the degree of worrying about having a backup mechanical shutter in the f3).
There are many magazine articles at the time debating autofocus and critical of the speed. The F3AF is horrendously slow and because it required a special finder and body and had only two slow lenses it was much more of an experiment. But the speed made it seem self-fulfilling.
it is worth noting that Nikon’s AF lens line came out the same year as the EF mount. It was also the same year as the Nikon F4 which was a pro camera with autofocus. The F4 was a big improvement over the F3 even without autofocus which led to a slower ramp up time of AF with Pros. I posted on FB a photo of the full range of AF lenses introduced at the time. Worth noting is that shortly after intro the AF lenses were all revised again to add “D” designation for distance information from lens to camera—already in the EF series.
Nikon was loathe to introduce electronic aperture since new lenses would not work at all with older cameras. It is only with the E and G lenses that the break has finally been made. Again they started with consumer lenses interestingly enough.
-
I do appreciate the article, the effort put into it, and the discussion it has engendered. Thanks!
I felt you got the main points across, but it jumped a bit from film to digital to film again during its course.
One thing worth mentioning is what I noticed leafing through old PopPhoto issues from the late 70s is that Nikon could run ads both for the pro F3 and the consumer level Nikon bodies, and in a very aspirational way - Nikon cameras were affordable and easy to use, and when you stepped up to the “big leagues”, the lenses all worked! (this was pre Canon EF, of course). The pro gear caused a “halo” effect. This is how Canon’s white glass work now.
-
According to this source, E lenses actually started with pro lenses, first with the 2008 tilt/shift PC (perspective control) refresh and moving on to some of the long telephotos. The G lenses do seem to have started with consumer lenses (some sources say the 2000 era 70-300 f/4-5.6).
The recent AF-P lenses have all been consumer lenses (and with strikingly low backward compatibility for Nikon). The cynical assumption is that removing the on-lens VR switch is largely about making them cheaper to manufacture (I say as an annoyed D7100 person).
-
-
-
-
Is this likely to also apply to to Firefox Beta? Or does it just seem to be a Nightly thing?
-
This specific study is Nightly only. In general I’d expect and hope that similar studies would only be run against Nightly for various reasons. However, as far as I can tell the current Mozilla privacy policy makes no difference between Nightly and Beta, and therefore Mozilla could consider themselves to have permission to run these opt-out studies on Beta, just as they have with Nightly. Mozilla may clarify this in the future.
On a pragmatic level, doing something like this with Beta would probably be much more visible and produce many more annoyed people and bad publicity, so I think Mozilla probably has good reasons to avoid it unless they have a study that they consider really important. Beta also has enough visibility that news about such an opt-in study would probably be widely distributed and well know, so you’d hear of it enough in advance to do something.
(I’m the author of the article.)
-
Thanks for this. I’ve been running Nightly on my mobile, and I’ve switched to Beta. (Newer Firefox versions than Stable on Android seem quite a bit faster; that’s why I care.)
-
-
-
This is why we can’t have good software. This program could literally have been an empty file, a nothing at all, a name capturing the essence perfectly.
I’m not sure I could disagree more strongly. An empty file only has the
truebehavior because of a bunch of incredibly non-obvious specific Unix behaviors. It would be equally reasonable for execution of this file to fail (likefalse) since there’s no hashbang or distinguishable executable format to decide how to handle it. At a somewhat higher level of non-obviousness, it’s really weird thattrueneed be a command at all (and indeed, in almost all shells, it’s not—trueis a builtin nearly everywhere).truebeing implementable in Unix as an empty file isn’t elegant—it’s coincidental and implicit.-
I mean, it’s POSIX specified behavior that any file that is executed that isn’t a loadable binary is passed to /bin/sh (”#!” as the first two bytes results in “implementation-defined” behavior), and it’s POSIX specified behavior that absent anything else, a shell script exits true.
It’s no more coincidental and implicit than “read(100)” advances the file pointer 100 bytes, or any other piece of standard behavior. Sure, it’s Unix(-like)-specific, but, well, it’s on a Unix(-like) operating system. :)
-
It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does. A perfectly-reasonable and nearly-equivalent specification in an alternate universe where Thomson and Ritchie sneezed five seconds earlier while deciding how executables should be handled would have precisely the opposite behavior.
On the other hand, if
read(100)did anything other than read 100 bytes, that would be extremely surprising and would not have come about from an errant sneeze.-
Black Mirror Episode: The year is 2100 and the world is ravaged by global warming. The extra energy aggregated over decades because non executables went through /bin/sh caused the environment to enter the tipping point where the feedback loops turned on. A time machine is invented, where one brave soul goes back in time with a feather, finds Thomson and makes him sneeze, saving humanity from the brink of extinction. But then finds himself going back to 2100 with the world still ravaged. Learns that it was fruitless because of npm and left-pad.
-
it’s totally coincidental that the specification says what it does.
This is true of literally all software specifications, in my experience.
-
-
Why isn’t
100an octal (or a hex or binary) constant? Why is it bytes instead of machine words? Why is read bound to a file descriptor instead of having a record size from an ioctl, and then reading in 100 records?Just some examples. :)
-
-
Pass a
mmap’ed pointer toread, and it shall write. :)
-
-
-
-
-
The current (POSIX) specification is the product of historical evolution caused in part by
/bin/trueitself. You see, in V7 Unix, the kernel did not execute an empty file (or shell scripts); it executed only real binaries. It was up to the shell to run shell scripts, including empty ones. Through a series of generalizations (starting in 4BSD with the introduction ofcsh), this led to the creation of#!and kernel support for it, and then POSIX requiring that the empty file trick be broadly supported.This historical evolution could have gone another way, but the current status is not the way it is because people rolled out of bed one day and made a decision; it is because a series of choices turned out to be useful enough to be widely supported, eventually in POSIX, and some choices to the contrary wound up being discarded.
(There was a time when kernel support for
#!was a dividing line between BSD and System V Unix. The lack of it in the latter meant that, for example, you could not make a shell script be someone’s login shell; it had to be a real executable.) -
The opposite isn’t reasonable though. That would mean every shell script would have to explicitly exit 0 or it will fail.
Every. Shell. Script.
And aside from annoying everyone, that wouldn’t even change anything. It would just make the implementation of true be exit 0, instead of the implementation of false be exit 1.
And read(100) does do something besides read 100 bytes. It reads up to 100 bytes, and isn’t guaranteed to read the full 100 bytes. You must check the return value and use only the amount of bytes read.
-
It’s not obvious to me that an empty file should count as a valid shell script. It makes code generation marginally easier, I suppose. But I also find something intuitive to the idea that a program should be one or more statements/expressions (or functions if you need main), not zero or more.
-
So if you run an empty file with sh, you would prefer it exits failure. And when you run an empty file with python, ruby, perl, et al., also failures?
Why should a program have one or more statements / expressions? A function need not have one or more statements / expressions. Isn’t top level code in a script just a de facto main function?
It’s intuitive to me that a script, as a sequence of statements to run sequentially, could have zero length. A program with an entry point needs to have at least a main function, which can be empty. But a script is a program where the entry point is the top of the file. It “has a main function” if the file exists.
-
I think whatever the answer is, it makes equal sense for Perl, Python, Ruby, shell, any language that doesn’t require main().
In my opinion, your last argument begs the question. If an empty program is considered valid, then existing is equivalent to having an empty main. If not, then it isn’t.
In any case, I don’t mean to claim that it’s obvious or I’m certain that an empty program should be an error, just that it seems like a live option.
-
-
Exactly. It sounds like arbitrary hackery common in UNIX development. Just imagine writing a semi-formal spec that defines a program as “zero characters” which you pass onto peer review. They’d say it was an empty file, not a program.
-
I guess
trueshouldn’t be considered a program. It is definitely tied to the shell it runs in, as you wouldn’t callexecv("true", {"/bin/true", NULL})to exit a program correctly. for example.truehas no use outside of the shell, so it makes sense to have it use the shell’s features. That is why now it tends to be a builtin. But having it a builtin is not specified by POSIX. Executing file on the other end, is, and the spec says the default exit code it 0 or “true”. By executing an empty file, you’re then asking the shell to do nothing, and then return true. So I guess it is perfectly fine fortrueto jist be an empty file. Now I do agree that such a simple behavior has (loke often with unix) way too many ways to be executed, ans people are gonna fight about it for quite some time! What about these?alias true=(exit) alias true='/bin/sh /dev/null' alias true='sh -c "exit $(expr `false;echo $? - $?`)"'The one true
true!-
It depends upon the system. There is IEFBR14, a program IBM produced to help make files in JCL which is similar to
/bin/true. So there could be uses for such a program.It also has the distinction of being a program that was one instruction long and still have a bug in it.
-
“That is why now it tends to be a builtin.”
Makes sense. If tied to the shell and unusual, I’d probably put something like this into the interpreter of the shell as an extra condition or for error handling. Part of parsing would identify an empty program. Then, either drop or log it. This is how such things are almost always handled.
-
-
-
-
That would mean every shell script would have to explicitly exit 0 or it will fail.
I don’t see how that follows.
Once the file is actually passed to the shell, it is free to interpret it as it wishes. No reasonable shell language would force users to specify successful exit. But what the shell does is not in question here; it’s what the OS does with an empty or unroutable executable, for which I am contending there is not an obvious behavior. (In fact, I think the behavior of running it unconditionally with the shell is counterintuitive.)
And read(100) does do something besides read 100 bytes.
You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.
-
I don’t see how that follows.
You claim the exact opposite behavior would have been equally reasonable. That is, the opposite of an empty shell script exiting true. The precise opposite would be an empty shell script—i.e. a script without an explicit exit—exiting false. This would affect all shell scripts.
Unless you meant the opposite of executing a file not loadable as an executable binary by passing it to /bin/sh, in which case I really would like to know what the “precise opposite” of passing a file to /bin/sh would be.
You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.
No. Many people assume read will fill the buffer size they provide unless they are reading the trailing bytes of the file. However, read is allowed to return any number of bytes within the buffer size at any time.
It also has multiple result codes that are not errors. Many people assume when read returns -1 that means error. Did you omit that detail for brevity, or was it not obvious to you?
-
-
-
-
Saying the behavior is totally (or even partially) coincidental is a bit strong. You’re ignoring the fundamental design constraints around shell language and giving the original designers more credit than they deserve.
Consider this experiment: you pick 100 random people (who have no previous experience to computer languages) and ask them to design a shell language for POSIX. How would all of these languages compare?
If the design constraints I’m talking about didn’t exist, then it would indeed be random and one would expect only ~50% of the experimental shell languages to have a zero exit status for an empty program.
I strongly doubt that is what you would see. I think you would see the vast majority of those languages specifying that an empty program have zero exit status. In that case, it can’t be random and there must something intentional or fundamental driving that decision.
-
I don’t care about how the shell handles an empty file. (Returning successful in that case is basically reasonable, but not in my opinion altogether obvious.) I’m stating that the operating system handling empty executables by passing them to the shell is essentially arbitrary.
-
The reason for the existence of human intelligence isn’t obvious either but that doesn’t make it random. A hostile environment naturally provides a strong incentive for an organism to evolve intelligence.
As far as the operating system executing non-binaries with “/bin/sh” being arbitrary, fair enough. Though I would argue that once the concepts of the shebang line and an interpreter exist, it’s not far off to imagine the concept of a “default interpreter.” Do you think the concept of a default is arbitrary?
-
-
-
It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does.
laughs That’s really taking an axe to the sum basis of knowledge, isn’t it?
-
-
-
yes an empty file signifying true violates the principle of least astonishment.However if there were a way to have metadata comments about the file describing what it does, how it works, and what version it is without having any of that in the file we’d have the best of both worlds.
-
-
-
-
-
-
There are a number of issues with these ideas but there are two I want to draw attention to in specific.
All byte spans are available to any user with a proper address. However, they may be encrypted, and access control can be performed via the distribution of keys for decrypting the content at particular permanent addresses.
While perpetually tempting, security through encryption keys has the major drawback that it is non-revocable (you can’t remove access once it’s been granted). As a result, over time it inevitably fails open; the keys leak and more and more people have access until everyone does. This is a major drawback of any security system based only on knowledge of some secret; we’ve seen it with NFS filehandles and we’ve seen it with capabilities, among others. Useful security/access control systems must cope with secrets leaking and people changing their minds about who is allowed access. Otherwise you should leave all access control out and admit honestly that all content is (eventually) public, instead of tacitly misleading people.
[…] Any application that has downloaded a piece of content serves that content to peers.
People will object to this, quite strongly and rightfully so. Part of the freedom of your machine belonging to you is the ability to choose what it does and does not do. Simply because you have looked at a piece of content does not mean that you want to use your resources to provide that content to other people.
-
Any application that has downloaded a piece of content serves that content to peers.
The other issue with this is what if the content is illegal? (classified government information, child abuse, leaked personal health records, etc.) There are some frameworks like Zeronet where you can chose to stop serving that content, and others like FreeNet where yo don’t even know if you’re serving that content. (These come with a speed vs anonymity trade-off of course).
I do agree with the idea that any content you fetch, you should reserve by default, maybe with some type of blockchain voting system to pass information along to all the peers if some of the content might be questionable, giving the user a chance to delete it.
-
Author of the original post here. My prototype uses IPFS, which uses (or plans to support, at least) distributed optional blocklists of particular hashes. This would be my model for blocking content. Anybody who doesn’t block what they’re asked to block becomes liable for hosting it.
-
-
-
As someone who is just starting to dive deep into operating systems, especially Unix, I’m grateful for all the writing you’ve done about the Oil project.
Oil is taking shell seriously as a programming language, rather than treating it as a text-based UI that can be abused to write programs.
One question in response to this statement is at what point does the shell language become just another programming language with an operating system interface. This question seems especially important when the Oil shell language targets users who are writing hundreds of lines of shell script. If someone is writing an entire program in shell script, what is the advantage of using shell script over a programming language? You seem to anticipate this question by comparing the Oil shell language to Ruby and Python:
…Python and Ruby aren’t good shell replacements in general. Shell is a domain-specific language for dealing with concurrent processes and the file system. But Python and Ruby have too much abstraction over these concepts, sometimes in the name of portability (e.g. to Windows). They hide what’s really going on.
So maybe these are good reasons (not sure if they are or aren’t) why Ruby and Python scripts aren’t clearly better than shell scripts. You also provide a mix of reasons why shell is better than Perl. For example: “Perl has been around for more than 30 years, and hasn’t replaced shell. It hasn’t replaced sed and awk either.”.
But again, it doesn’t seem to clearly answer why the domain language for manually interacting with the operating system should be the same language used to write complex scripts that interact with the operating system. Making a language that is capable of both should provide a clear advantage to the user. But it’s not clear that there is an advantage. Why wouldn’t it be better to provide two languages: one that is optimized for simple use cases and another that is optimized for complex use cases? And why wouldn’t the language for complex use cases be C or Rust?
-
My view is that the most important division between a shell language and a programming language is what each is optimized for in terms of syntax (and semantics). A shell language is optimized for running external programs, while a programming language is generally optimized for evaluating expressions. This leads directly to a number of things, like what unquoted words mean in the most straightforward context; in a fluid programming language, you want them to stand for variables, while in a shell language they’re string arguments to programs.
With sufficient work you could probably come up with a language that made these decisions on a contextual basis (so that ‘a = …’ triggered expression context, while ‘a b c d’ triggered program context or something like that), but existing programming languages aren’t structured that way and there are still somewhat thorny issues (for example, how you handle
if).Shell languages tend to wind up closely related to shells (if not the same) because shells are also obviously focused on running external programs over evaluating expressions. And IMHO shells grow language features partly because people wind up wanting to do more complex things both interactively and in their dotfiles.
(In this model Perl is mostly a programming language, not a shell language.)
-
Thanks, glad you like the blog.
So maybe these are good reasons (not sure if they are or aren’t) why Ruby and Python scripts aren’t clearly better than shell scripts.
Well, if you know Python, I would suggest reading the linked article about replacing shell with Python and see if you come to the same conclusion. I think among people who know both bash and Python (not just Python), the idea that bash is better for scripting the OS is universal. Consider that every Linux distro uses a ton of shell/bash, and not much Python (below a certain level of the package dependency graph).
The main issue is that people don’t want to learn bash, which I don’t blame them for. I don’t want to learn (any more) Perl, because Python does everything that Perl does, and Perl looks ugly. However, Python doesn’t do everything that bash does.
But again, it doesn’t seem to clearly answer why the domain language for manually interacting with the operating system should be the same language used to write complex scripts that interact with the operating system.
There’s an easy answer to that: because bash is already both languages, and OSH / Oil aim to replace bash.
Also, the idea of a REPL is old and not limited to shell. It’s nice to build your programs from snippets that you’ve already tested. Moving them to another language wouldn’t really make sense.
-
-
Depending on how strict one is about banishing the browser, it’s possible to treat both Firefox and Chrome more or less this way and still have fast-starting browser windows. The trick is that both browsers can rapidly spawn new windows from a running instance, so if you park a minimized empty ‘master’ window for the browser off in some dark corner you can then rapidly open new windows, interact with them for however long you need them, and then close them again.
(I have a relatively complicated setup for this to make it very low-effort to go from an URL to new window, complete with Google searches on demand and various other special URL features.)
-
I’ve done this (for a different reason; I originally started it because I wanted to use my window manager to track open pages, which is far more effective than the browser’s pitiful tab bar) but I’ve noticed similar benefits of making the browser seem like less of a “place”.
-
-
cks
9 months ago
|
link
| on:
Do you think that Go might support accepting the other types into variadic funcs w/ empty-interface
Go’s philosophy is generally to do only a limited amount of runtime work by magic behind the scenes. Converting values to interfaces is work; Go is willing to do it relatively implicitly for single values, but I expect that philosophically it is opposed to extending that work to converting an array of values into an array of interfaces, since the amount of work (and the amount of memory required) is unpredictable and potentially large.
-
alexkorban
11 months ago
|
link
| on:
Systemd on Ubuntu 16.04 can't (or won't) reliably reboot your server
I’m a complete outsider to systemd but from what I’ve read here on Lobste.rs, it seems like an excellent case study in the dangers of software rewrites. It highlights that there is a lot of complexity hidden away in the details of the code, and thus rewrites are followed by a long tail of bug reports and complaints about missing functionality. If the cause of this bug is something to do with NFS, then surely it’s something that’s been dealt with previously.
-
Bear in mind that often much of what you’ll see about anything, systemd included, is complaints; people write less praise for things for various reasons, and such praise often gets less publicity than juicy bad news like this. I say this as the original entry’s author and someone who has written about, eg, things that systemd gets right and my overall views on systemd, to go along with the various bad things I’ve written about systemd (this latest annoyance included).
The specific NFS issue involved here is not a trivial one to solve and past solutions in other init systems have had their own fragilities, where they might do things like kill necessary daemons too early because the init system didn’t know you had an unusual configuration. In one sense systemd appears to be doing exactly what it was told to do, and part of the issue is that Ubuntu created an overall configuration where they didn’t make sure that everything would happen in the right order. Systemd could do better but a reasonably general solution would require some policy decisions and possibly significantly more work.
-
-
https://github.com/golang/go/issues/17566
Some discussion of the issue here, didn’t make the cutoff for 1.9 but might make 1.10
-
There’s also https://github.com/golang/go/issues/19348; they’ve laid some groundwork to inline more, but they need to keep stack traces correct and compile-time cost reasonable.
-
This bit is interesting:
I already expressed dislike for //go:noinline, and I will firmly object any proposal for //go:mustinline or something like that, even if it’s limited to runtime.
Can anyone elaborate on this? Is it just a matter of abuse? I not-so-infrequently find myself in a position where a function i want inlined doesn’t get inlined, and I’m thankful that i have the tools to force the issue. I’ve found the other direction useful as well, particularly for profiling.
-
Can anyone elaborate on this? Is it just a matter of abuse?
I can’t think of a way in which to abuse
noinline, can ofmust, and feel I’d want an out if I know better than an analyzer. But even more, I’d like the tooling to introspect what, and why a function is or isn’t being inlined, without being force to read the intermediary code/assembly. But, honestly, there are probably only a handful of cases in a 100,000 lines of Go where I might even consider it worth attempting to inline to speed something up. In that case, better to trust the analyzer and hope that the heuristic doesn’t hurt more often than gain.-
In my code at work, yeah, I almost never care about function inlining. But in my open source/spare time work, controlling function inlining is critical in some cases for performance. I guess it would be kind of cool to learn why inlining did or didn’t happen, but I’d still want to be able to override it.
-
-
One reason I can think of to object to a general mustinline directive is that it moves Go much closer to a situation where interpreting compiler directives in comments is mandatory for any implementation, even if they are not officially part of the language specification. If enough people write code that only runs correctly (including necessary minimum performance) when inlined, then all Go compiler implementations must support //go:mustinline in practice. If Go is going to do this it should really make such compiler instructions part of the official standard, which opens up a whole can of worms. Arguably build directives have already opened this can of worms, since I think a Go implementation can’t really build Go packages in practice without understanding them (‘have users write Makefiles to control what’s compiled’ is not really an answer).
(Restricting this to the runtime is less dangerous, since the runtime is tightly coupled to the compiler. But allowing its use in the standard library outside of runtime-coupled packages would be a problem, since other Go compilers may well want to use as much of the standard library as possible.)
-
-
-
I like apenwarr’s writing in general, so it is painful and irritating that this article’s history of Ethernet networking and IP(v4) on Ethernet is utterly wrong, with events completely out of order. IP(v4) was being used on Ethernet by the early 1980s (RFC 826 for ARP dates from November 1982, for example), which is far before IPX and Netware and other LAN-only PC networking protocols were created. Even the start of IPv6 work predates much of the 1990s PC LAN networking boom (it began around 1992).
It may be apenwarr’s intention to present things as a just-so story instead of history, but if so I wish it was far more explicitly labelled as such.
-
Interesting and SLES still uses it by default. What’s going on here ?
-
There is a reasonable explanation from a former Red Hat developer who worked on btrfs. I have quoted most of his comment for convenience:
People are making a bigger deal of this than it is. Since I left Red Hat in 2012 there hasn’t been another engineer to pick up the work, and it is a lot of work.
For RHEL you are stuck on one kernel for an entire release. Every fix has to be backported from upstream, and the further from upstream you get the harder it is to do that work.
Btrfs has to be rebased every release. If moves too fast and there is so much work being done that you can’t just cherry pick individual fixes. This makes it a huge pain in the ass.
Then you have RHEL’s “if we ship it we support it” mantra. Every release you have something that is more Frankenstein-y than it was before, and you run more of a risk of shit going horribly wrong. That’s a huge liability for an engineering team that has 0 upstream btrfs contributors.
The entire local file system group are xfs developers. Nobody has done serious btrfs work at Red Hat since I left (with a slight exception with Zach Brown for a little while.)
-
Pure speculation on my part: RHEL/CentOS uses XFS now by default. I guess this is “good enough” and due to changing requirements of their customers, mostly towards virtualization, container technology, overlay filesystems etc., btrfs became a solution looking for a problem?
-
When talking about RedHats customers: many of them use RAID setups… So I asked today at my clients site, and got shown the following page:
https://btrfs.wiki.kernel.org/index.php/Status
RAID0: OK
RAID1: Mostly okay (may get stuck irreversibly)
RAID10: Mostly okay (may get stuck irreversibly)
RAID56: Unstable. Write hole still exists, parity not checksummed
So, this is the reason why btrfs was not even considered as a trial there. They had definitely looked into it.
-
-
-
My suspicion is that when Red Hat added btrfs to RHEL 6 as a technology preview they expected it to stabilize and to move to a fully supported thing within a major release or two of RHEL, but now Red Hat’s engineering no longer believes that this is going to happen soon enough. Based on a three to four year release cycle, RHEL 8 is probably going to be released within a year (RHEL 7 came out June 2014), so now is about when Red Hat could be making engineering decisions about what will be included and what won’t be.
-
-
Among other issues with this article, I don’t think it has the history of virtualization and containerization correct. The proliferation of websites on hosting providers happened well before virtualization and containerization started to be a thing, and providers tended to have reasonably well baked solutions to manage the whole complex result (and generally still do today).
I believe that it is also the case that general use of isolation is driven in part by customer demand, not hosting provider convenience. People like having an isolated system with exactly the things that they want in it, running their preferred version of this, that, and the other. They are not as enthused about sharing a host with a fixed OS, a fixed distribution, often fixed package versions, and so on.
-
So, is FastCGI just a means of ensuring that someone doesn’t expose an incomplete/brittle HTTP implementation needlessly?
-
FastCGI was from an era when putting a scalable HTTP server in your process was super expensive. This is still (largely) the case for some languages, e.g. Ruby and Python, which is why things like rack, and wsgi exist. Those solutions are just specialized, Lang specific implementations of the FastCGI ideals, really.
So, the history here, briefly:
- Documents only
- CGI for dynamic responses (with Perl being super common)
- JSP with “application servlet containers”
- mod_perl / mod_php embedding the Lang into apache
- SCGI / FastCGI (for persistent external to web server processes)
- Rack / WSGI to isolate the application from the details of how it interacts with the outside world
- Phusion passenger / mod_wsgi which adopt those Lang ideals into the web server
- PHP over FastCGI rises (for some reason?)
- Gunicorn, uwsgi, and the surge of Lang specific web servers (very similar to Java container servers, really)
- Node.js / golang / application is the web server (where we are now)
(I’m on my phone, so some liberties taken)
-
FastCGI and other similar protocols also have the advantage that they don’t muddle or confuse HTTP headers and so on from the forwarding with HTTP headers from the original request (because the forwarding is handled through a completely different protocol). Given FastCGI or SCGI, it’s very easy for a HTTP app to be given the complete original headers and to act as if it’s directly in the context of the original HTTP server; this is not so true for HTTP forwarding, where you’re going to be stitching up various things at some level (either the app or in your reverse proxy server).
-
-
Wrap on integer overflow is a good idea, because it will get rid of one of undefined behavior in C.
Undefined behavior is evil. One evil it causes is that it makes codes optimization-unstable. That is, something can work in debug build but does not work in release build, which is very undesirable. The article does not address this point at all.
To remove all undefined behaviour in C would severely impact the performance of C programs.
The post does suggest that trap-on-overflow is a superior alternative to wrap-on-overflow, and of course you could apply it to both debug and release builds if this optimisation instability concerns you. Even if you apply it just to your debug build, you’ll at least avoid the possibility that something works in the debug build but not in the release.
Note that Rust wraps on overflow and as far as I can tell this does not impact performance of Rust programs.
This is essentially the same as one of the arguments I addressed in the post. Although in certain types of programs, particularly lower-level languages (like C and Rust) where the code is written directly by a human programmer, there probably are not going to be many cases where the optimisations enabled by having overflow be undefined will kick in. However, if the program makes heave use of templates or generated code, or is produced by transpiling a higher-level language with a naive transpiler, then it could do (I’ll conceded this is theoretical in that I can’t really give a concrete example). The mechanism by which the optimisation works is well understood and it isn’t too difficult to produce an artificial example where the optimisation grants a significant speed-up in the generated code.
Also, in the case of Rust programs, you can’t really reliably assess the impact of wrapping on overflow unless there is an option to make overflow undefined behaviour. Is there such an option in Rust?
No, there is no such option, and there never will be. Rust abhors undefined behaviors. Performance impact assessment I had in mind was comparison with C++ code.
On the other hand, rustc is built on LLVM so it is rather trivial to implement: rustc calls LLVMBuildAdd in exactly one place. One can replace it with LLVMBuildNSWAdd (NSW stands for No Signed Wrap).
This cannot be entirely true. As a reducto ad absurdum, it would be possible in principle to laboriously define all the things that compilers currently do with undefined behaviour and make that the new definition of the behaviour, and there would then be zero performance impact.
C compiler writers might argue that removing all undefined behaviour without compromising performance would be prohibitively expensive, but I’m not entirely convinced; there are carefully-optimized microbenchmarks on which the naive way of defining currently-undefined behaviour produces a noticeable performance degradation, but I don’t think it’s been shown that that generalises to realistic programs or to a compiler that was willing to put a bit more effort in.
Clearly that would be absurd, and it’s certainly not what I meant by “remove all undefined behaviour”. Your “possible in principle” suggestion is practically speaking completely impossible, and what I said was true if you don’t take such a ridiculously liberal interpretation of it. Let’s not play word games here.
They might, but that’s not what I argued in the post.
There’s no strong evidence that it does, nor that it wouldn’t ever do so.
Well then, what did you mean? You said “To remove all undefined behaviour in C would severely impact the performance of C programs.” I don’t think that’s been demonstrated. I’m not trying to play word games, I’m trying to understand what you meant.
I meant “remove all undefined behaviour” in the sense and context of this discussion, in particular related to what @sanxiyn above says:
To avoid that problem, you need to define specific behaviours for cases that are currently specify undefined behaviour (not ranges of possible behaviour that could change depending on optimisation level). To do so would significantly affect performance, as I said.
I could believe that removing all sources of differing behaviour between debug and release builds would significantly affect performance (though even then, I’d want to see the claim demonstrated). But even defining undefined behaviour to have the current range of behaviour would be a significant improvement, as it would “stop the bleeding”: one of the insidious aspects of undefined behaviour is that the range of possible impacts keeps expanding with new compiler versions.
It’s not just about removing the sources of differing behaviour - but doing so with sensibly-defined semantics.
A demonstration can only show the result of applying one set of chosen semantics to some particular finite set of programs. What I can do is point out that C has pointer arithmetic and this is one source of undefined behaviour; what happens if I take a pointer to some variable and add some arbitrary amount, then store a value through it? What if doing so happens to overwrite part of the machine code that makes up the program? Do you really suppose it is possible to practically define what the behaviour should be in this case, such that the observable behaviour will always be the same when the program is compiled with slightly different optimisation options - which might result in the problematic store being to a different part of code? To fully constrain the behaviour, you’d need pointer bounds checking or similar - and that would certainly have a performance cost.
As I’ve tried to point out with the example above, the current range of undefined behaviour is already unconstrained. But for some particular cases of undefined behaviour, I agree that it would be better to have more restricted semantics. For integer overflow, in particular, I think it could reasonably be specified that the result becomes unstable (eg. behaves incosistently in comparisons), but the behaviour is otherwise defined - for example. Note that even this would impede some potential optimisations. (And that I still advocate trap on overflow as the preferred implementation).
I suspect that one issue is that compilers may manifest different runtime behaviour for undefined behaviour, depending on what specific code the compiler decided to generate for a particular source sequence. In theory you could document this with sufficient effort, but the documentation would not necessarily be useful; it would wind up saying ‘the generated code may do any of the following depending on factors beyond your useful control’.
(A canonical case of ‘your results depend on how the compiler generates code’, although I don’t know if it depends on undefined behaviour, is x86 floating point calculations, where your calculations may be performed with extra precision depending on whether the compiler kept everything in 80-bit FPU registers, spilled some to memory (clipping to 64 bits or less), or used SSE (which is always 64-bit max).)
It’s not only possible: Ive seen formal semantics that define various undefined behaviors just like you said. People writing C compilers can definitely do it if they wanted to.