They’re clearly marketing to different audiences. Rust doesn’t need to sell as a technology to engineers anymore, it needs to sell as a product to businesses. Most tech marketing sites undergo this transition the more they grow, in my experience
I think that’s not true at all. What does it that even mean, selling as a product to businesses? The way you do that is by targeting their engineers, the people who make technology choices. Ground up is by far the best strategy for a language, top down feels ridiculous. If you are actually in a position where you’re talking to bunsinesses about funding you’ve already sold to their engineers and your website is irrelevant - at that point they’re going to want to hear about what the language needs in order to succeed because the org is bought in.
Beyond that, this website contains all of the things that the Rust site does, but also code. It’s proof that you don’t need a tradeoff here - literally just put a damn snippet of the language on the site, that’s it.
The reality is that language decisions are made almost exclusively when the company starts. By engineers. Companies only ever constrain their internal support languages in the future, and it’s engineers who push for language support when that happens. Selling a language to a business through the website is just a nonsensical concept on its face.
Roc makes it clear, in text, without code, “here is why you would choose Roc” but also it shows code, very concisely right at the top.
The idea is that rust has already captured he interest of engineers. Word of mouth, blog posting, conferences etc. “Everyone” knows about rust at this point. Engineers are building hobby projects and FOSS in jot already.
What I mean by selling to a business is that ultimately management aka business needs to approve and buy into rust otherwise it will remain in hobby project obscurity.
Some engineer or group of engineers says to their boss “we want to use rust for project X” and their boss loads up the landing page because while they’ve heard of Java and python they haven’t heard of rust. This is why the first words you see are ”reliable”, “efficient”, “safe”, and “productive”. They see big recognizable company logos and testimonials by well known or high-positioned leaders in the industry.
For what it’s worth I also think it’s a good idea to put code on the homepage. You can do both, but this is why you see tech marketing sites evolve to become less and less technical over time.
The idea is that rust has already captured he interest of engineers.
The reality is that Rust is something engineers have heard of but largely know nothing about, other than some high level stuff like “fast”. A small portion of companies actually use Rust, a small portion of engineers have ever touched it.
Some engineer or group of engineers says to their boss
By far the easiest way to gain adoption is through startups or smaller companies where the engineers are already the ones who decide on language. After a few thousand employees companies always end up with someone locking down the decision making process and tiering the language support levels - at that point, again, it’s up to the engineers to work through the process to gain support. I’ve never heard of a situation where a top level manager had to go to a language website and make the call to use or not use a language, isn’t that sort of an absurd idea? But, again, Roc shows that this is not an “either, or” situation.
This is why the first words you see are
Roc shows “fast, friendly, functional” - I’m not saying you can’t have words. Roc shows that you can have these great, high level blurbs here.
They see big recognizable company logos and testimonials by well known or high-positioned leaders in the industry.
Roc also shows this. Again, I am not against the idea of content other than code on the site, I’m saying it’s ridiculous to have literally no code.
For what it’s worth I also think it’s a good idea to put code on the homepage.
I think the point is that Rust is at a stage where it’s marketing itself as an industry-ready language and ecosystem that you can build actual products on without worrying about running into a horrible bug that dooms your start up. At that stage, showing code doesn’t make too much difference, because the “industry-ready” aspect of it has so much more going for it than the surface language syntax you can showcase in a landing page.
Roc, however, is at the stage where it needs to capture the interest of curious engineers that might be interested in the language itself, without worrying about the practical applicability of it too much. (BTW, I don’t want to imply that Roc can’t be used for anything practical at this point, I don’t know, but clearly I wouldn’t bet my start up on it and I doubt anyone expects that at this stage).
What does it that even mean, selling as a product to businesses? The way you do that is by targeting their engineers, the people who make technology choices. Ground up is by far the best strategy for a language, top down feels ridiculous.
Man, as a person who sells languages (mostly TLA+) professionally, I wish that was the case. Selling to engineers is way more fun and feels more comfortable. But I’ve had cases where an entire team wanted to hire me, and cases where the team never heard of TLA+ but a manager or CTO wanted to hire me, and the second case way more often leads to business.
I’m not saying that it literally never happens, I’m saying that by far language adoption at companies is driven by engineers - either through external forces (engineers building tooling/libraries in their free time, engineers starting companies with that language, etc) or through direct forces (engineers advocating for the language at that company).
I’m sure there are plenty of extreme cases like Oracle convincing schools to sell Java, or whatever, where this wasn’t the case.
It’s all moot since Roc’s language clearly shows that a website can have code and high level information, and meld them together well.
I’m not saying that it literally never happens, I’m saying that by far language adoption at companies is driven by engineers
The guy literally said he sells languages professionally, and he’s telling us it’s driven by management, not engineers. Do you also sell languages? Or better yet, have some source of information which trumps his personal anecdotes?
In what way is Oracle an extreme case?
I don’t disagree that you can still include code, but even the high level blurbs will be different when selling to businesses, at which point, if the copy isn’t selling to developers, you might as well better use the space to more effectively sell to businesses where the code would have been (even literally empty space for a less cramped, more professional looking page).
Engineers care more about “fast (runtime)” and certainly “friendly” than most businesses do, and businesses care a lot more about “safe” than a lot of programmers do; “fast (compile time)”, ”reliable”, “efficient”, and “productive” may be mixed. Engineers also care about technical details like “functional” a hell of a lot more than businesses do, because they can evaluate the pros and cons of that for themselves.
I guess it’s much more effective to be very targeted in who you’re selling to, unless both profiles have equal decision making power.
The guy literally said he sells languages professionally, and he’s telling us it’s driven by management, not engineers.
And? I know almost nothing about them. They’ve apparently been pushing TLA+, an incredibly niche and domain specific language that the vast majority of developers never even hear of and that targets software that lives in an extreme domain of correctness requirements. It’s nothing at all like Rust. Maybe they also sell Python, or comparable languages? I don’t know, they didn’t say. Is it even evidence that selling TLA+ this way is the most effective way?
Or better yet, have some source of information which trumps his personal anecdotes?
Years of experience in this field as an engineer, advocating and getting a large company to include languages in their accepted policies, generally just being an engineer who has adopted languages.
unless both profiles have equal decision making power.
I do not buy, at all, that for a language like Rust (frankly, I doubt it’s the case for any language, but I can imagine languages like TLA+ going a different route due to their nature and lack of appeal to devs) that it’s not wildly favorable to target developers. I’ve justified this elsewhere.
The current Rust website is not ideal, but I think the general principle of convincing devs vs businesses holds.
Developers are picky about surface-level syntax. The urge to see the code is to judge how it fits their preferences. I see lots of comments how Rust is plain ugly and has all the wrong brackets.
Business-level decision makers don’t care how the code looks like. It can look like COBOL, and be written backwards. They ask what will happen if they spend resources on rewriting their code, and that better improve their product’s performance/reliability/security or devs’ productivity, not just make a neat-looking quicksort one-liner.
Readability is subjective. Syntax must be functional and clear, but devs will care about details beyond that, and die on hills of which brackets it should use, and whether significant whitespace is brilliant or terrible.
Compare “The app is slower and uses more memory, but the devs think the code is beautiful” vs “The app is faster and more reliable, but the devs think the code’s syntax looks stupid”. If you want to convince product owners, the syntax is an irrelevant distraction, and you must demonstrate outcomes instead.
In web development there’s also a point of friction in developer experience vs user experience. Frameworks and abstraction layers have a performance cost (tons of JS), but not using them is harder and more tedious.
While readability is subjective to a certain degree, it stems from human psychology, which is based on how the real world works.
In the real world, things that are inside a container are smaller than the container, so if delimiters don’t visually encompass the text they contain, it looks counter-intuitive.
And things which are physically closer together are perceived as being more strongly associated, so a::b.c looks like it should be a::(b.c), while in fact it’s (a::b).c.
Performance and safety has nothing to do with syntax. It would be entirely possible (and not difficult) to create a language exactly like Rust, but with better syntax.
I think you’re confirming what I’m saying. You’re focusing on a tiny nitpick about a part of the syntax that doesn’t cause any problems.
And yet you’re prepared to bikeshed the syntax, as if it was objectively wrong, rather than a complex tradeoff. In this case the syntax and precedence rules were copied from C++ to feel familiar to Rust’s target audience. If you just “fix” the precedence, it will require parens in common usage. If you change the separator symbols, you’ll get complaints that they’re weird and non-standard, and/or create ambiguities in item resolution.
Showing code for a language invites such shallow subjective opinions, and distracts from actually important aspects of the language.
This is literally bikeshedding. Don’t you see the pointlessness of this? If you changed Rust to use your preferred sigil, it would still produce byte for byte identical executables. It wouldn’t change anything for users of Rust programs, and it wouldn’t even change how Rust programs are written (the compiler catches the <> ambiguity, tells you to use turbofish, and you move on).
Rust has many issues that actually affect programs written in it, e.g. bloat from overused monomorphisation, lack of copy/move constructors for C++ interop and self-referential types, Send being too restrictive for data inside generators and futures, memory model that makes mmapped data unsafe, blunt OOM handling, lack of DerefMove and pure Deref, lack of placement new for Box, etc. But these are hard problems that require deep understanding of language’s semantics, while everyone can have an opinion on which ASCII character is the best.
Even Rust’s syntax has more pressing issues, like lack of control over lifetimes and marker traits of async fn, or lack of syntax for higher-ranked lifetimes spanning multiple where clauses.
You’re thinking of physics, not psychology. Concrete syntax is one of the least important parts of a programming environment, and we only place so much weight onto it because of the history of computers in industry, particularly the discussions preceding ALGOL and COBOL.
There is also https://prev.rust-lang.org, linked (not prominently) from the bottom of https://www.rust-lang.org. It doesn’t have the “eating your laundry” footnote, but the code runner works, unlike in web.archive.org.
I feel like the author was close to saying something regarding differences between a very-technical-senior and a soft-skills-senior. But we just have a kind of basic statement regarding engineering for failure.
There is definitely an aspect of Dunning-Kruger to some of the complaints, primarily driven by a lack of understanding of the sheer scale of Twitter, it’s backend (Twitter wasn’t all GCP (mostly analytics data) but largely, especially on the frontend, on-premises data centre hardware) and all of the analytics tools, machine learning crap, axillary code and such that comes with it. Anyone can write a lookalike Twitter frontend, backend given enough time. At the scale of actual Twitter comes a lot of hidden work.
I’ve worked on systems in statically typed languages, as well as dynamically typed languages, some with REPLs and fancy debugging tools, and some without.
As someone who loves lisps and also languages that are quite different from lisp, there’s a reason why homoiconicity, REPLs, and time-traveling debuggers didn’t take over the world, and it’s not because people are stupider than you.
There just isn’t one true model of computation. There isn’t one development/debugging workflow that’s globally better than all others. Humans are diverse and think and communicate in different ways. Some people are more visual in their thinking; others can fit a tremendous amount of state in their brains and think procedurally.
Regular reminder that life hacking one’s consumption habits will not change the world.
We will really buy hardware built in sweatshops, hand all our code to a company that contracts with the state, but the problem is that Microsoft looms at the keyboard shortcuts you use?
Microsoft telemetry running on your own computer is a more direct problem for you than sweatshops or Github existing, and you have more ability to change your own consumption habits to directly remove the opportunity for Microsoft to spy on you than to change how Github operates vis a vis a government or the working conditions of hardware factories in foreign countries whose languages and cultures you do not understand.
Oh yeah, totally, the problem with the internet is not enough “autonomy”. As opposed to having our personas traded by data brokers, brains melted by social media, behaviors modified by targeted advertising and influencer culture.
These techno-libertarian 80s MIT hacker derivative “autonomy” and “freedom” arguments have been such a massive distraction from a material analysis of how power actually uses technology to exploit people for profit. The spectacle of the individual atomizes people and alienates them from common cause.
But if only we were to perfect our APIs and network protocols. Then we would not be exploited by internet companies. We would have autonomy 🙄
I used to give the same advice, but I completely changed my opinion over the past 10 years or so. I eventually put in the time and learned shell scripting. These days my recommendation is:
Learn to use the shell. It’s a capable language that can take you very far.
Use ShellCheck to automatically take care of most of the issues outlined in the article.
I really don’t want to figure out every project’s nodejs/python/ruby/make/procfile abomination of a runner script anymore. Just like wielding regular expressions, knowing shell scripting is a fundamental skill that keeps paying dividends over my entire career.
Always pay attention to what version of bash you need to support, and don’t go crazy with “new” features unless you can get teammates to upgrade (this is particularly annoying because Apple ships an older version of bash without things like associative arrays).
Always use the local storage qualifier when declaring variables in a function.
As much as possible, declare things in functions and then at the end of your script kick them all off.
Don’t use bash for heavy-duty hierarchical data munging…at that point consider switching languages.
Don’t assume that a bashism is more-broadly acceptable. If you need to support vanilla sh, then do the work.
While some people like the author will cry and piss and moan about how hard bash is to write, it’s really not that bad if you take those steps (which to be fair I wish were more common knowledge).
To the point some folks here have already raised, I’d be okay giving up shell scripting. Unfortunately, in order to do so, a replacement would:
Have to have relatively reasonable syntax
Be easily available across all nix-likes
Be guaranteed to run without additional bullshit (installing deps, configuring stuff, phoning home)
Be usable with only a single file
Be optimized for the use case of bodging together other programs and system commands with conditional logic and first-class support for command-line arguments, file descriptors, signals, exit codes, and other nixisms.
Be free
Don’t have long compile times
There are basically no programming languages that meet those criteria other than the existing shell languages.
Shell scripting is not the best tool for any given job, but across every job it’ll let you make progress.
(Also, it’s kinda rich having a Python developer tell us to abandon usage of a tool that has been steadily providing the same, albeit imperfect, level of service for decades. The 2 to 3 switch is still a garbage fire in some places, and Python is probably the best single justification for docker that exists.)
While some people like the author will cry and piss and moan about how hard bash is to write, it’s really not that bad if you take those steps (which to be fair I wish were more common knowledge).
I think “nine steps” including “always use two third-party tools” and “don’t use any QoL features like associative arrays” does, in fact, make bash hard to write. Maybe Itamar isn’t just “cry and piss and moan”, but actually has experience with bash and still think it has problems?
To use any language effectively there are some bits of tribal knowledge…babel/jest/webpack in JS, tokio or whatever in Rust, black and virtualenv in Python, credo and dialyzer in Elixir, and so on and so forth.
Bash has many well-known issues, but maybe clickbait articles by prolific self-pronoters hat don’t offer a path forward also have problems?
I think there’s merit here in exploring the criticism, though room for tone softening. Every language has some form of “required” tooling that’s communicated through community consensus. What makes Bash worse than other languages that also require lots of tools?
There’s a number of factors that are at play here and I can see where @friendlysock’s frustration comes from. Languages exist on a spectrum between lots of tooling and little tooling. I think something like SML is on the “little tooling” where just compilation is enough to add high assurance to the codebase. Languages like C are on the low assurance part of this spectrum, where copious use of noisy compiler warnings, analyzers, and sanitizers are used to guide development. Most languages live somewhere on this spectrum. What makes Bash’s particular compromises deleterious or not deleterious?
Something to keep in mind is that (in my experience) the Lobsters userbase seems to strongly prefer low-tooling languages like Rust over high-tooling languages like Go, so that may be biasing the discussion and reactions thereof. I think it’s a good path to explore though because I suspect that enumerating the tradeoffs of high-tooling or low-tooling approaches can illuminate problem domains where one fits better than the other.
I felt that I sufficiently commented about the article’s thesis on its own merits, and that bringing up the author’s posting history was inside baseball not terribly relevant. When you brought up motive, it became relevant. Happy to continue in DMs if you want.
Integrating shellcheck and shfmt to my dev process enabled my shell programs to grow probably larger than they should be. One codebase, in particular, is nearing probably like 3,000 SLOC of Bash 5 and I’m only now thinking about how v2.0 should probably be written in something more testable and reuse some existing libraries instead of reimplementing things myself (e.g., this basically has a half-complete shell+curl implementation of the Apache Knox API). The chief maintenance problem is that so few people know shell well so when I write “good” shell like I’ve learned over the years (and shellcheck --enable=all has taught me A TON), I’m actively finding trouble finding coworkers to help out or to take it over. The rewrite will have to happen before I leave, whenever that may be.
I’d be interested in what happens when you run your 3000 lines of Bash 5 under https://www.oilshell.org/ . Oil is the most bash compatible shell – by a mile – and has run thousands of lines of unmodified shell scripts for over 4 years now (e.g. http://www.oilshell.org/blog/2018/01/15.html)
Right now your use case is the most compelling one for Oil, although there will be wider appeal in the future. The big caveat now is that it needs to be faster, so I’m actively working on the C++ translation (oil-native passed 156 new tests yesterday).
I would imagine your 3000 lines of bash would be at least 10K lines of Python, and take 6-18 months to rewrite, depending on how much fidelity you need.
(FWIW I actually wrote 10K-15K lines of shell as 30K-40K lines of Python early in my career – it took nearly 3 years LOL.)
So if you don’t have 1 year to burn on a rewrite, Oil should be a compelling option. It’s designed as a “gradual upgrade” from bash. Just running osh myscript.sh will work, or you can change the shebang line, run tests if you have them, etc.
There is an #oil-help channel on Zulip, liked from the home page
Thanks for this nudge. I’ve been following the development of Oil for years but never really had a strong push to try it out. I’ll give it a shot. I’m happy to see that there are oil packages in Alpine testing: we’re deploying the app inside Alpine containers.
Turns out that I was very wrong about the size of the app. It’s only about 600 SLOC of shell :-/ feels a lot larger when you’re working on it!
One thing in my initial quick pass: we’re reliant on bats for testing. bats seemingly only uses bash. Have you found a way to make bats use Oil instead?
I wouldn’t expect this to be a pain-free experience, however I would say should definitely be less effort than rewriting your whole program in another language!
I have known about bats for a long time, and I think I ran into an obstacle but don’t remember what it was. It’s possible that the obstacle has been removed (e.g. maybe it was extended globs, which we now support)
In any case, if you have time, I would appreciate running your test suite with OSH and letting me know what happens (on Github or Zulip).
One tricky issue is that shebang lines are often #!/bin/bash, which you can change to be #!/usr/bin/env osh. However one shortcut I added was OSH_HIJACK_SHEBANG=osh
Moving away from Python? Now it has my interest… in the past I skipped past know it’d probably take perf hits and have some complicaged setup that isn’t a static binary.
Yes that has always been the plan, mentioned in the very first post on the blog. But it took awhile to figure out the best approach, and that approach still takes time.
Python is an issue for speed, but it’s not an issue for setup.
You can just run ./configure && make && make install and it will work without Python.
Oil does NOT depend on Python; it just reuses some of its code. That has been true for nearly 5 years now – actually since the very first Oil 0.0.0. release. Somehow people still have this idea it’s going to be hard to install, when that’s never been the case. It’s also available on several distros like Nix.
What is the status of Oil on Windows (apologies if it’s in the docs somewhere, couldn’t find any mentioning of this). A shell that’s written in pure C++ and has Windows as a first class citizen could be appealing (e.g. for cross-platform build recipes).
It only works on WSL at the moment … I hope it will be like bash, and somebody will contribute the native Windows port :-) The code is much more modular than bash and all the Unix syscalls are confined to a file or two.
I don’t even know how to use the Windows sycalls – they are quite different than Unix! I’m not sure how you even do fork() on Windows. (I think Cygwin has emulation but there is way to do it without Cygwin)
To the point some folks here have already raised, I’d be okay giving up shell scripting. Unfortunately, in order to do so, a replacement would: […]
There are basically no programming languages that meet those criteria other than the existing shell languages.
I believe Tcl fits those requirements. It’s what I usually use for medium-sized scripts. Being based on text, it interfaces well with system commands, but does not have most of bash quirks (argument expansion is a big one), and can handle structured data with ease.
Always use #!/usr/bin/env bash at the beginning of your scripts (change if you need something else, don’t rely on a particular path to bash though).
I don’t do this. Because all my scripts are POSIX shell (or at least as POSIX complaint as I can make them). My shebang is always #!/bin/sh - is it reasonable to assume this path?
you will miss out on very useful things like set -o pipefail, and in general you can suffer from plenty of subtle differences between shells and shell versions. sticking to bash is also my preference for this reason.
note that the /usr/bin/env is important to run bash from wherever it is installed, e.g. the homebrew version on osx instead of the ancient one in /bin (which doesn’t support arrays iirc and acts weirdly when it comes across shell scripts using them)
My shebang is always #!/bin/sh - is it reasonable to assume this path?
Reasonable is very arbitrary at this point. That path is explicitly not mandated by POSIX, so if you want to be portable to any POSIX-compliant system you can’t just assume that it will exist. Instead POSIX says that you can’t rely on any path, and that scripts should instead be modified according to the system standard paths at installation time.
I’d argue that these days POSIX sh isn’t any more portable than bash in any statistically significant sense though.
Alpine doesn’t have Bash, just a busybox shell. The annoying thing is if the shebang line fails because there is no bash, the error message is terribly inscrutable. I wasted too much time on it.
https://mkws.sh/pp.html hardcodes #!/bin/sh. POSIX definitely doesn’t say anything about shs location but I really doubt you won’t find a sh at /bin/sh on any UNIX system. Can anybody name one?
That’s because the other ones are options and not errors. Yes, typically they are good hygiene but set -e, for example, is not an unalloyed good, and at least some experts argue against using it.
There are tons of pedants holding us back IMO. Yes, “set -e” and other options aren’t perfect, but if you even know what those situations are, you aren’t the target audience of the default settings.
Yup, that’s how you do it, It’s a good idea to put in the the time to understand shell scripting. Most of the common misconceptions come out of misunderstanding. The shell is neither fragile (it’s been in use for decades, so it’s very stable) nor ugly (I came from JavaScript to learning shell script, and it seemed ugly indeed at first, now I find it very elegant). Keeping things small and simple is the way to do it. When things get complex, create another script, that’s the UNIX way.
It’s the best tool for automating OS tasks. That’s what it was made for.
+1 to using ShellCheck, I usually run it locally as
shellcheck -s sh
for POSIX compliance.
I even went as far as generating my static sites with it https://mkws.sh/. You’re using the shell daily for displaying data in the terminal, it’s a great tool for that, why not use the same tool for displaying data publicly.
I went the opposite direction - I was a shell evangelist during the time that I was learning it, but once I started pushing its limits (e.g. CSV parsing), and seeing how easy it was for other members of my team to write bugs, we immediately switched to Python for writing dev tooling.
There was a small learning curve at first, in terms of teaching idiomatic Python to the rest of the team, but after that we had much fewer bugs (of the type mentioned in the article), much more informative failures, and much more confidence that the scripts were doing things correctly.
I didn’t want to have to deal with package management, so we had a policy of only using the Python stdlib. The only place that caused us minor pain was when we had to interact with AWS services, and the solution we ended up using was just to execute the aws CLI as a subprocess and ask for JSON output. Fine!
I tend to take what is, perhaps, a middle road. I write Python or Go for anything that needs to do “real” work, e.g. process data in some well-known format. But then I tie things together with shell scripts. So, for example, if I need to run a program, run another program and collect, and then combine the outputs of the two programs somehow, there’s a Python script that does the combining, and a shell script that runs the three other programs and feeds them their inputs.
I also use shell scripts to automate common dev tasks, but most of these are literally one-ish line, so I don’t think that counts.
(Although I will agree that it’s annoying that shell has impoverished flag parsing … So I actually write all the flag parsers in Python, and use the “task file” pattern in shell.)
(many code examples from others in that post, also almost every shell script in https://github.com/oilshell/oil is essentially that pattern)
There are a lot of names for it, but many people seem to have converged on the same idea.
I don’t have a link handy not but Github had a standard like this in the early days. All their repos would have a uniform shell interface so that you could get started hacking on it quickly.
Not listed but a really common reason for continuing to use Y over X is when, regardless of the original choice, the cost of switching doesn’t approach justifying any marginal differences.
This seems to be what the article meant by “Our custom internal solution is good enough”, except that that wording excluded the possibility of one’s current external solution being good enough.
I don’t have a problem with hacktivism and calling attention to injustice is generally the right thing to do. But the idea that you can call out war crimes by indiscriminately targeting disk drives of Russian citizens is really missing the point.
This morning I would had said you are right, however today I have picked up my children from their preschool where an Ukrainian boy at their age pissed under himself because he saw an unknown male (me), after having spent two days in a darkened evacuation train and having left behind everyone except his mum - and probably much more she would not tell. I have never seen a more scared being in my life. It wasn’t an anxiety, it was a kind of fear I wasn’t aware humans were able to experience. Let’s say I have a problem with that.
If this ‘malware’ will stop at least one Russian ammunition train or make one Russian more stand up, the collateral damage(money) it causes is worth much more than the collateral damage(dead kids) that may be caused by the said ammunition. I don’t blame anyone having other opinion, but I understand and would myself do what the author of the module has and had to do.
If this ‘malware’ will stop at least one Russian ammunition train or make one Russian more stand up, the collateral damage(money) it causes is worth much more than the collateral damage(dead kids) that may be caused by the said ammunition. I don’t blame anyone having other opinion, but I understand and would myself do what the author of the module has and had to do.
I see where you’re coming from. But I don’t see such methods changing the opinions or stance of a previously-indifferent (much less one who is supportive of the regime) Russian. I see frustration and anger, because they almost certainly aren’t thinking of the plight of Ukrainian families fleeing chaos, they’re thinking about their lost files.
I am not endorsing either side here, but your comment seems to presuppose that the sides of their target audience (people in Russia) are supports and opposes. I think in the view of the author of this hazardous code the sides are three: supports, opposes, and mildly in favor of the fictitious events pictured in Russian propaganda. If the theory is that the last group is the largest then they would need only small margin over 50% of that group to be pushed into the opposes group in order to justify the operation to themselves. Again, I’m not endorsing either viewpoint here – I’m just trying to highlight how views can shift depending on (mis)understanding of the subgroups and the size of those groups. Personally I don’t think we have very good estimates for these parameters but tbh I haven’t done a lot of research either.
This all depends on the broader context in which code is getting written. There’s no discussion of the incentives which lead to debt and guard against fixing it. It would be nice if our industry were to do science on quality, rather than just opinion.
This was my reaction as well. I was looking for some content or impassioned speech about being our best selves with some philosophical underpinnings. Instead we ended up with a tweet and a meme at the end.
I really like your “why should you try Emacs in 2021” list. It rings very true to my experience with emacs (and vim). Back when I used it more regularly, aside from any kool-aid I drank around the efficiency of keyboard-driven editing, if I’m being honest, the main motivation was simply that it was fun. Emacs was cool to me because I saw it as vintage, alternative, customizable, a component and expression of a hacker identity. It was an expression of programming-as-hobby as opposed to programming-as-profession. Of course I genuinely did enjoy the editing experience, and developing a healthy .emacs. I loved magit, projectile, helm, the clojure packages, etc. But I ended up in a place professionally where ultimately Emacs had no leverage over more specialized tools for what I was working on.
I personally don’t think you should be submitting posts from your company blog at all, ideally another community member would organically post it themselves if it’s interesting enough. Or the actual author of the post.
If you feel like you must promote your company blog, then I think you shouldn’t say you authored the post if you didn’t write it but you should also disclose your relationship with the company in a comment. I don’t think what you’re doing right now - saying you authored the post even though didn’t - sufficiently signals to readers the nature of the conflict and that the post is marketing for your company. There’s no need to add new site features for this either, a top-level comment on the story is sufficient.
I agree in principle, but if the content is good enough so that I don’t care that it’s on a company blog (this seems to be the case here), posting it (as not-the-author) is fine. So is an extra comment with “my coworker wrote this, happy to answer questions because *knowlede).
I see where your coming from but want to offer counterpoint.
Not every person follows every blog. So the assumption that content will get posted if it’s good enough ignores the idea that different people have are tapped into different information networks due to their social experiences, whether it’s a job or an online community.
I don’t see an inherent problem with posting content produced within the poster’s workplace. I don’t believe the stakes are remotely high enough on an aggregator site like this for there to be a conflict of interest. I also believe that on here, there’s no moral issue with a quality post coming from a company vs hobby blog, even considering the poster’s intent. There might be incidental issues, like a company blogging about ethically compromised tech, but that can be easily dealt with on a case by case basis.
Personally, I’m excited about the “melting face” emoji, as well as a few of the new gender and skin color variants that will piss off chuds. Also, the Emoji block of Unicode now has not one, but two amulets against the evil eye, which I expect will be extremely valuable for social media.
The emoji thing is so totally irresponsible. Humanity is never going to replace Unicode. We’re stuck with it until we either go extinct or go Luddite. Adding emoji based on whims is how you end up with things like this sticking around for four thousand years and counting. The Egyptians at least had the excuse that they didn’t know what computers were.
Adding emoji based on whims is how you end up with things like this sticking around for four thousand years and counting. The Egyptians at least had the excuse that they didn’t know what computers were.
Not sure what the problem is? Ancient Egyptians living thousands of years ago didn’t share your particular cultural taboos and sensitivities, which seems like an entirely valid “excuse” to me.
Right, there’s nothing that the Egyptians were doing “wrong”, because when they decided to use a penis as a letter, they had no way of knowing that for the remainder of human civilization we will have to use the penis as a letter, whether it’s culturally taboo or cool or we replace men with artificial sex bots or whatever. We however do know that Unicode is forever, and so the bar to adding a new character should be really fucking high. Like, here is an alphabet that was already in use by a non-trivial amount of people for some length of time. Not, it would be cool to make a new kind of smiley face.
A better system would be to do what is already done with flags. For flags, the flag 🇺🇸 is just
U+1F1FA 🇺 REGIONAL INDICATOR SYMBOL LETTER U
U+1F1F8 🇸 REGIONAL INDICATOR SYMBOL LETTER S
We could do the same thing for other ephemera, and not have to burden Unicode with an open ended and endless list of foods that were popular with the Unicode committee in the 21st century.
We don’t “have to use the penis as a letter” because it exists in Unicode. It’s just that it is technically representable. I’ll admit there’s nuance here - there are probably some things I’d rather see us avoid in Unicode, i.e. violence. But I’m struggling to see the harm caused in this particular case.
Who’s to say that the United States will still be around in 4,000, 1,000, or 200 years? Or that the “US” code won’t be recycled for some other country? Hell, why should our current ISO system of labelling countries even persist? Once you start talking about these kind of timeframes anything is up for grabs really.
“Forever” is a heck of a long time. I don’t think we’re stuck with Unicode for all eternity, there’s all sorts of ways/scenarios we could come up with something new. I think we should just address the issues of the day; there’s no way what the future will be like anyway; all we can do is focus on the foreseeable future.
That’s the whole point. US won’t mean 🇺🇸 forever. It will naturally change over time and when it does, the old codes will still be decipherable (flag for something called “US”) without needing to be supported anymore.
Tbh, the most likely a scenario is a RoC, PRC thing where two countries claim to be the US, and then the international community will have to pick sides. Anyway, still better than having the flag as a real emoji!
I don’t really follow how one scheme is more advantageous over the over; at the end of the day you’re still going have to map some “magic number” to some specific meaning. I suppose you could spell out “happy” or “fireman” in special codepoints, but that just seems the same as mapping specific codepoints to those meaning, but with extra steps (although “fireman” already consists of two codepoints: “man” + “fire engine”, or “person” and “women” for other gender variants).
The reason it’s done with flags probably has more to do that it’s just easier.
It’s not just that it’s easier it’s that obsolescence is a built in concept. New countries come and old countries go and ISO adds and removed country codes. Using Slack and GitHub style :name: emojis mean that you can add and drop support for specific emoji without needing to just serve up a �. It is also more forward compatible. When your friend on a new phone texts you :dotted smiley: you won’t just see �, you’ll see words that describe what is missing. Plus you aren’t using up a finite resource.
To be fair, I’ll be the first to grab popcorn when they announce that everyone and their toaster now has to adopt utf8 with 1-5 bytes. Will probably be as smooth and fast as our ipv4 to ipv6 migration.
When your friend on a new phone texts you :dotted smiley: you won’t just see �
Right, that would be useful.
Changing the meaning of specific codepoints or sequences of codepoints over time just seems like a recipe for confusion. “Oh, this 300 year old document renders as such-and-such, but actually, back then it meant something different from today” is not really something that I think will help anyone.
This already exists to some degree; e.g. “Ye olde tarvern” where “Y” is supposed to represent a capital Thorn, which is pronounced as “th”, not as Y (written as þ today, but written quite similar to Y in old-fashioned cursive writing, and early German printing presses didn’t have a Thorn on account of that letter not existing in German so people used Y as a substitute). In this case it’s a small issue of pronunciation, but if things really shift meaning things could become a lot more apt to misunderstandings in meaning.
Emoji have already shifted in ways that change their meaning. The gun emoji has become a ray gun due to political correctness and sometimes points left and sometimes right. Shoot me → zap you. The grimace 😬 was a different emotion on Android and iOS for a while. There are other documented examples of this kind of semantic shift in just a short amount of time. I think it’s a bit hopeless to try to pin them down while you keep adding stuff. The use of eggplant and peach for penis and butt is based on their specific portrayal and subject to the visual similarity being lost if they redraw them in different ways. What if President Xi demands a less sexy 🍑? Will it stick around or be a bit of passing vulgar slang from the early twentieth century? Impossible to predict.
You’re missing the point. I’m not ashamed of weiners. They are hilarious. The point is that a character can be taboo or not and we’re still stuck with it.
If it’s not something to be ashamed of, is it really taboo enough to exclude from the Unicode standard? And furthermore, why is being stuck with it an issue? It can even be valuable from an anthropological standpoint.
My dude, Unicode inherited vertical tab from ASCII. That’s my point. Things are only going to continue to accumulate for now until the collapse of civilization. It will never shrink.
It’d be nice to have some actual background on hashing in here instead of just broad generalizations and links to various hash functions. Examples:
There’s no mention of cyclic redundancy checks and why they are not valid as crypto functions (a mistake some programmers have made).
There’s no mention of avalanche effects, which is a good way of seeing how “random” a digest scheme is (with some implications for how well the output can be predicted/controlled by an attacker).
The mentioned attack on JSON hash tables in PHP (if you dig into it) would’ve been a great place to talk about trivial hashes (e.g., f(x) =0 or f(x)=x) and why they cause problems even in non-hostile environments, but that would’ve required more of an introduction to how hashing works…)
Lots of usage of jargon like “non-invertible”, “collision-resistance”, “preimage attack resistance”, etc. which is probably inaccessible if your audience is programmers who “don’t understand hash functions”.
There’s not really an explanation about the differences/similarities of crypto-strong hash functions, password hash functions, and key derivation functions, other than a mention that there is some relation but which isn’t elaborated on at all.
There’s not really any useful information at all about perceptual hashing vs other forms of multimedia digest approaches–there’s just some Apple hate.
etc.
Programmers might not understand hash functions, but infosec furries may also not understand pedagogy.
(also, can you please cool it with the inflammatory article headlines?)
Honestly I think it’s a valid concern. One of the biggest problems with the computer security world, as stated repeatedly by leading experts in the field, is communication and teaching.
A valid concern would be “infosec experts may not understand pedagogy” but why call out “infosec furries” specifically? Unless we should be concerned about infosec furries in particular vs other infosec experts?
Are these acceptable?
but infosec gays may also not understand pedagogy
but infosec women may also not understand pedagogy
but infosec people of color may also not understand pedagogy
See elsewhere for the explanation; furry bashing doesn’t enter into it, though I see why you might have read it that way. Furries are internet denizens like the rest of us, with all that entails.
I read your other comments. But you said what you said, and that undermines all your pontificating about the harm of “insulting/demeaning a group” and “the sort of microaggression/toxicity that everybody talks so much about.” Take your own advice.
The OP’s site has some pretty well reasoned and presented articles on precisely why “furry” cannot reasonably be summarized as “a kink”.
And, no, you do not “normally” have to get someone’s consent to introduce them to the idea of your kink, unless said introduction involves you engaging them in the practice of your kink.
Sorry, I didn’t realize the “furry” part was what you were opposed to. It sounded like you were upset with the implication that the infosec world is bad at teaching.
One of the things he talks about there is testing the hypothesis and seeing which title actually worked. I only clicked this link because I recognized your domain name and knew you had written interesting articles in the past and might legitimately explain something I didn’t know. If not for that, I probably would have bypassed it since the title alone was not interesting at all.
Even so, it is still possible to write clickbait titles that aren’t predicated on insulting/demeaning a group.
“Hash functions: hard or just misunderstood?”
“Things I wish more programmers knew about hashes”
“Programmer hashes are not infosec hashes”
“Are you hashing wrong? It’s more common than you might think”
“uwu whats this notices ur hash function”
How would you feel if I wrote “Gay furries don’t understand blog posting”? Even if I raise good points, and even if more people would click on it (out of outrage, presumably), it would still probably annoy a gay furry who wrote blogs and they’d go in with their hackles raised.
The difference between “Programmers don’t understand hash functions” and “Gay furries don’t understand blog posting” is quite obvious to me and I definitely don’t want to engage in whatever Internet flame is going on here. Especially since, uh, I have a preeetty good idea about what the problem here is, and I tend to think it’s about gay furries, not article titles, which is definitely not a problem that I have. (This should probably be obvious but since I’m posting in this particular thread, I wanted to make sure :P).
But I also think this title really is needlessly nasty, independent of how it might be titled if it were about other audiences. It’s a bad generalisation – there are, in fact, plenty of programmers who understand hash functions – and it’s not exactly encouraging to those programmers who want to get into security, or who think their understanding of these matters is insufficient.
I am (or was?) one of them – this was an interest of mine many, many years ago, at a time when I was way too young to understand the advanced math. My career took me elsewhere, and not always where I wanted to go, and I tried to keep an eye on these things in the hope that maybe one day it’ll take me there. Needless to say, there’s only so much you can learn about these topics by spending a couple of evenings once in a blue moon studying them, so I never really got to be any good at it. So I think the explanation is amazing, but it would definitely benefit from not reminding me of my inadequacy.
And I’m in a happy boat, actually, this is only an interest of mine – but there are plenty of people who have to do it as part of their jobs, are not provided with adequate training of any kind, have no time to figure it out on their own, and regularly get yelled at when they get it wrong.
Now, I realise the title is tongue-in-cheek to some degree, the playful furries and the clever humour scattered throughout the post sort of gives it away. If you think about it for a moment it’s pretty clear that this is meant to grab attention, not remind people how much they suck. But it’s worth remembering that, in an age where web syndication is taken for granted to the point where it sounds like a Middle English term, this context isn’t carried everywhere. Case in point, this lobste.rs page includes only the title. Some people might react to it by clicking because you grabbed their attention, but others might just say yeah, thanks for reminding me, I’ll go cry in a corner.
Even if I didn’t realise it was tongue-in-cheek, it probably wouldn’t bother me, partly because I understand how writing “competitively” works (ironically, from around the same time), partly because I’ve developed a thick skin, and partly because, honestly, I’ve kindda given up on it, so I don’t care about it as much as I once did. But I can see why others would not feel the same way at all. You shouldn’t count on your audience having a thick skin or being old enough to have given up on most of their dreams anyway.
I know this is a real struggle because that’s just how blogs and blogging work today. You have to compete for attention to some degree, and this is particularly important when a large part of the technical audience is “confined” to places like HN and lobste.rs, where you have to grab attention through the title because there’s nothing else to grab attention through. But maybe you can find a kinder way to grab it, I dunno, maybe a clever pun? That never hurt anyone. These radical, blunt (supposedly “bluntly honest” but that’s just wishful thinking) headlines are all the rage in “big” Internet media because, just like Internet trolls, they thrive on controversy, us vs. them and a feeling of smugness, but is that really the kind of thing you want to borrow?
(Edit: just to make sure I get the other part of my message across, because I think it’s even more important: title aside, which could be nicer, the article was super bloody amazing: the explanation’s great, and I like the additional pointers, and the humour, and yes, the drawings! Please don’t take any of all that stuff above as a criticism of some sort: I wanted to present a different viewpoint from which the title might read differently than you intended, not that the article is bad. It’s not!)
What if the person encountering your blog is a programmer from an underrepresented background, just barely overcoming imposter syndrome, and now here’s this scary suggestion that they don’t understand hash functions? What if they actually made one of the mistakes in the article, and feel like they’re a complete fraud, and should leave the industry? This is the sort of microaggression/toxicity that everybody talks so much about, if I’m not mistaken.
The point is: you don’t know. You can’t know.
So, err on the side of not adding more negative shit to the world accidentally in the name of pageviews–especially when there are many, many other more positive options in easy reach.
EDIT:
I wouldn’t care if it weren’t for the fact that you’re a smart dude and clearly passionate about your work and that you have good knowledge to share, and that it pains me to see somebody making mistakes I’ve made in the past.
I wouldn’t care if it weren’t for the fact that you’re a smart dude and clearly passionate about your work
I’m neither of those things :P
and that you have good knowledge to share, and that it pains me to see somebody making mistakes I’ve made in the past.
I appreciate your compassion on this subject. It’s definitely new territory for me (since forever I’ve been in the “boring headline out of clickbait adversion” territory).
Do you actually not see a difference between saying a slightly negative thing about people of a certain profession and how they engage in that profession, and an ad-hominem using sexual orientation? What a weird and bad analogy?
I’m trying to assume good intent here but all your comments make it sound like you’re annoyed at the furry pics and awkwardly trying to use cancel culture to lash out the author.
Neither the label of programmers (with which I identify) nor of gay furries (with which the author identifies, according to their writing) is being misapplied. I’m sorry you feel that a plain statement of fact is somehow derogatory–there is nothing wrong with being a proud programmer or a proud gay furry.
My point in giving that example was to critique the used construction of “ is ”. I picked that label because the author identified with it, and I picked the “bad at blogging” because it’s pretty obviously incorrect in its bluntness. If I had picked “lobsters” or “internet randos” the conjured association for the person I was in discussion with may not have had the same impact it that “programmers” had on me, so I went with what seemed reasonable.
What if the person encountering your blog is a programmer from an underrepresented background, just barely overcoming imposter syndrome, and now here’s this scary suggestion that they don’t understand hash functions?
Or they may read this and think ‘I’m glad it’s not just me!’. As a programmer who probably has a better than average understanding of hash functions, I don’t feel demeaned by this generalisation, if I were worried about my level of understanding I’d feel comforted by the idea that I wasn’t in a minority in my lack of understanding.
What if they actually made one of the mistakes in the article, and feel like they’re a complete fraud, and should leave the industry?
Or they may feel better that this mistake is so common that someone writes about it on a list of mistakes programmers make.
What if the person encountering your blog is a programmer from an underrepresented background….
While I said you’re picking a fight (and would add: “look at the thread, it’s a fight”), I see what you’re saying in this paragraph. I also value non-judgmental explanations.
My problem with the title isn’t that it’s insulting, but that it’s inaccurate. Clearly some programmers do understand hash functions, even if other programmers do not. If nothing else, @soatok, a programmer, presumably understands hash functions, or why else would he write a blog post purporting to explain the right way to use them?
Programmers don’t understand hash functions, and I can demonstrate this to most of the people that will read this with a single observation:
When you saw the words “hash function” in the title, you might have assumed this was going to be a blog post about password storage.
Specifically is wrong, at least about me, and almost certainly among other programmers as well. I don’t claim to have deep knowledge about cryptography, and I do expect that there’s probably something I could learn from this blog post, which I will read more carefully when I have a chance. But I am aware that the computer science concept of hash functions is useful for a variety of programming problems, and not just storing password-related data.
I gotta admit JS leaves me wanting when it comes to really basic stuff like list comprehensions and dictionary syntax, but being able to throw together an async-y tool to run stuff in parallel cleanly, for example, is a pretty great party trick.t
For example, you can easily make your build scripts just follow their dependencies cleanly without having to go full Bazel to get the proper speed boosts you might want
I am intrigued by the framing of the Sturm und Drang about the state of the web as being driven, to some significant degree, by politics internal to Google.
As I stated earlier this week, promo packets are what’ll do in the web.
I think a lot of developers simply lack the interest in context to process the realpolitik that shapes and distorts the fabric of spacetime for our industry.
If you refuse to understand that Google’s whole business is threatened by adblockers, you probably would be confused at the some of the changes to web extensions webRequest that make that work harder. If you don’t understand the desire to make SEO, crawling, and walled gardens easier AMP probably seemed like a great return to roots.
Other companies do this too, of course. If you didn’t know about OS/2 Warp some of the Windows APIs probably seemed weird. If you don’t know about Facebook trying to own everything you do then the lack of email signup for Oculus probably seems strange. If you invested heavily into XNA you probably got bit when internal shifts at Microsoft killed XNA off. If you don’t know about internal Canonical and RHEL shenanigans, systemd and other things probably are a surprise.
Developers need to pay as much attention to the business dependencies as the technical ones.
When you’re doing a performance review at Google, you can request a promotion. If you do this, you put together a ‘packet’ including the impactful work you’ve done. New work is rewarded heavily, maintenance less so. For senior engineers, shipping major projects with an industry wide impact is a path to promotion.
Which means Google rewards doing something new for the sake of doing something new. It’s tremendously difficult to get promoted by improving older systems. Crucially, you often need to demonstrate impact with metrics. The easiest way to do that is sunset an old system and show the number of users who have migrated to your new system, voluntarily or otherwise.
Is there any material evidence suggesting that someone’s promotion is the reason that chrome will remove alert? Obviously google will push the web in the direction that juices profit, but an individual promotion? Seems like a red herring.
It is often difficult to pick it apart as it’s rarely a single person or team. What happens in large organizations is that there is a high-level strategy and different tactics spring from that. Then, there are metrics scorecards, often based on a proxy, which support the tactics delivering the strategy. This blurs the picture from the outside and means that rarely one person is to blame, or has singular control over the successes.
I haven’t followed the alert situation very closely, but someone familiar with large organizations can get a good read from the feature blurb. There is a strong hint from the language that they are carrying a metric around security, and possibly one around user experience. This translates to an opportunity for a team to go and fix the issue directed by the metrics since it’s quantifiable. The easiest way to start might be to work back from what moves the metric, but this is a very narrow perspective.
Developers may know what the best things to work on having been a developer in that area for 10 years, but their impact tracks towards those top-level strategies. Management can’t justify promotion because someone else is very focused on metrics that drive the strategy.
In lots of places this is called alignment. Your boss may only support X amount of work on non-aligned projects, if you do at least Y amount of work on Y projects. A classic big company alignment example is a talented person in a support department. If they can fix your biggest problem at the source it’d be best to let them do this. However, metrics incentivize assigning them to solving N support cases per week and other metrics designed for lower-skilled individuals instead of working on fixing root causes. Eventually, they leave unless you have smart management taking calculated risks, manage the metrics at the team level so the team is not noticed working the way it wants, seeking paths for talented people to work on the product, etc.
Many of us understand how metrics and incentives at tech companies work. Was just pointing out that it’s a bold claim to assume that chrome is removing alert due to an individual seeking a promotion.
I think about this in terms of my time at Apple – like, people ascribed all kinds of causes to various seemingly peculiar Apple decisions that to those of us on the inside were obvious cases of internal politicking leaking out.
WHATWG is a consortium of multiple companies so I’m curious why everyone is pointing the finger at Google here, or is the assertion that Google has so much power over the WHATWG and Chrome at this point that there’s no ability for other companies to dissent? (And I mean we all know that the W3C lost and WHATWG won so a consortium of vendors is the web.)
The multiple companies are Apple, Google, Microsoft, and Mozilla (https://whatwg.org/sg-agreement#steering-group-member, section 3.1b) Of the three, only Apple develops a browser engine that is not majority funded by Google.
The browser engine Apple creates is used for a whole bunch of stuff across their platforms, besides Safari:
Mail, iMessage, Media Store fronts, App Store fronts.. Those last two alone produce revenue about 4x what Google pays Apple to make it the default.
Do I wish they’d get more people using alternatives and pass on the google money? Sure. Is there any realistic chance their ability to fund Safari and/or Webkit would be harmed by not taking the google money? Seems pretty unlikely.
Yes, this. Google’s play here is less about controlling standards per se (ed: although they do plenty of that too) and more about getting everyone to treat Whatever Google Does as the standard.
WHATWG was run at inception by a Googler and was created to give Google even more power over the standards process than the hopelessly broken W3C already gave them. That they strong armed Mozilla into adding their name or that Apple (who was using the same browser engine at the time) wanted to be able to give feedback to the org doesn’t change the Googlish nature of its existence, IMO
Like it or not, Google is the www. It is the driving force behind the standards, the implementations (other than Safari), and the traffic that reaches websites.
It would be odd if Google’s internal politics didn’t leak into the medium.
A lot of people seem to think that standards work is a bit like being in a university - people do it for the love of it and are generally only interested in doing what’s best for all.
In reality it’s a bunch of wealthy stakeholders who realize that they need to work together for everyone’s best - they’re not a monopoly, yet - but in the meantime it behooves them to grab every advantage they can get.
As mentioned in the post, standards work is hard and time-consuming, and if an organisation can assign a dedicated team to work on standards, that work will get implemented.
This basically boils down to good programming and clarifying interface boundaries. Prefer naming things over sprinkling anonymous functions around. The react hooks used in the examples provide no benefit and actually slow the code down.
Another nice project with an S3 compatible API is seaweedfs (née weedfs): https://github.com/chrislusf/seaweedfs, inspired by the haystack paper (from FB, when FB had around 1-2K photo uploads per second); we use it in production (albeit not in distributed mode). A lightning talk I did a few month ago: https://github.com/miku/haystack
if you do not mind, a question – did you find any solutions that are based on JDK 11+ (Java/clojure/scala, etc) – I am looking for a file store open source lib, but I would like it to be compatible with a JVM ecosystem.
Thank you for the follow-ups.
I would like the whole service to be packageable and accessible as a JAR that I can incorporate in our ‘uber’ JAR.
The backend I am working on, has one of its ‘features’ – a simple deployment.
In the translation, it means a single-jar + PostgresSQL.
The single-jar within it, has about 20 ‘micro-services’, essentially. So a user can start one up just one jar and have everything in ‘one host’ or start the jar with config params telling the JAR which microservices to start on which host. That configuration file is like a ‘static’ service orchestrator. It is the same for all the hosts, but there are sections for each host in the deployment.
One of the microservices (or easier to call them just services) I am going to be enhancing is a ‘content server’.
Today the content service basically needs a ‘backend-accessible-directory’.
That service does all the others things: administering the content, acting as IAM Policy Enforcement Point, caching content in a memory-mapped db (if a particular content is determined to be needed ‘often’), a non-trivial hierarchical directory management to ensure that too many files do not end up in ‘one folder’, and so on.
I need to support features where files are in a ‘remote content server’ (rather then in a locally accessible directory).
Where the content server is an S3 (or some other standard compatible system)
So I would like the ‘content server’ to be included as a 21st service in that single JAR.
Which is why, I am not just looking for a client, but for the actual server – to be compatible with JVMs.
Hope the explanation gives more color to the question I asked.
With regards to other comment where folks mention that some organizations like banks – prefer a JVM only code.
That’s true to a degree, it is a preference, not an absolute requirement though.
That’s because some of these organizations have built by themselves ‘pre-docker’ deployment infrastructures. Where it is easy to request a ‘production capacity’ as long as the deployed backend is a JAR (because those deployment infrastructures are basically JVM clusters that support migrations, software defined networks, load balancing, password vaults, monitoring, etc)
So when a vendor (or even internal team) comes in and says: for our solution we run on docker, it is OK, but they have invested millions… and now want to continue to get benefits (internal payment basically) for their self-build infrastructure management tools … Which is why there is a preference for JVM-only solutions and, perhaps, will be for some time.
And to be honest, JVM (and JVM based languages) and their tools ecosystem continues to evolve (security, code analysis, performance, etc) — it seems that the decisions back then about investing into managed infrastructure around JVM – were good decisions.
I love this website. First thing I see? Code. Amazing. It is insane how some languages hide their code from me.
And then I scroll down and… a repl? OMG. Yes.
And then I scroll down more and there’s more code??????? Did an actual developer build this website????
I remember when the Rust website was good and actually showed you code. Now it’s useless. https://www.rust-lang.org/ versus https://web.archive.org/web/20150317024950/http://www.rust-lang.org/
I feel like you’ve really hit the best of “simple, stylistic, and shows me the fucking programming language”.
They’re clearly marketing to different audiences. Rust doesn’t need to sell as a technology to engineers anymore, it needs to sell as a product to businesses. Most tech marketing sites undergo this transition the more they grow, in my experience
I think that’s not true at all. What does it that even mean, selling as a product to businesses? The way you do that is by targeting their engineers, the people who make technology choices. Ground up is by far the best strategy for a language, top down feels ridiculous. If you are actually in a position where you’re talking to bunsinesses about funding you’ve already sold to their engineers and your website is irrelevant - at that point they’re going to want to hear about what the language needs in order to succeed because the org is bought in.
Beyond that, this website contains all of the things that the Rust site does, but also code. It’s proof that you don’t need a tradeoff here - literally just put a damn snippet of the language on the site, that’s it.
The reality is that language decisions are made almost exclusively when the company starts. By engineers. Companies only ever constrain their internal support languages in the future, and it’s engineers who push for language support when that happens. Selling a language to a business through the website is just a nonsensical concept on its face.
Roc makes it clear, in text, without code, “here is why you would choose Roc” but also it shows code, very concisely right at the top.
Cool! I think it’s true at all :)
The idea is that rust has already captured he interest of engineers. Word of mouth, blog posting, conferences etc. “Everyone” knows about rust at this point. Engineers are building hobby projects and FOSS in jot already.
What I mean by selling to a business is that ultimately management aka business needs to approve and buy into rust otherwise it will remain in hobby project obscurity.
Some engineer or group of engineers says to their boss “we want to use rust for project X” and their boss loads up the landing page because while they’ve heard of Java and python they haven’t heard of rust. This is why the first words you see are ”reliable”, “efficient”, “safe”, and “productive”. They see big recognizable company logos and testimonials by well known or high-positioned leaders in the industry.
For what it’s worth I also think it’s a good idea to put code on the homepage. You can do both, but this is why you see tech marketing sites evolve to become less and less technical over time.
The reality is that Rust is something engineers have heard of but largely know nothing about, other than some high level stuff like “fast”. A small portion of companies actually use Rust, a small portion of engineers have ever touched it.
By far the easiest way to gain adoption is through startups or smaller companies where the engineers are already the ones who decide on language. After a few thousand employees companies always end up with someone locking down the decision making process and tiering the language support levels - at that point, again, it’s up to the engineers to work through the process to gain support. I’ve never heard of a situation where a top level manager had to go to a language website and make the call to use or not use a language, isn’t that sort of an absurd idea? But, again, Roc shows that this is not an “either, or” situation.
Roc shows “fast, friendly, functional” - I’m not saying you can’t have words. Roc shows that you can have these great, high level blurbs here.
Roc also shows this. Again, I am not against the idea of content other than code on the site, I’m saying it’s ridiculous to have literally no code.
Then we agree on the main issue here.
I’m not debating your tastes here? Just explaining why sites follow this trend. Hope this helps
OK and I’m explaining why the reasoning isn’t sound and it’s a bad idea.
I think the point is that Rust is at a stage where it’s marketing itself as an industry-ready language and ecosystem that you can build actual products on without worrying about running into a horrible bug that dooms your start up. At that stage, showing code doesn’t make too much difference, because the “industry-ready” aspect of it has so much more going for it than the surface language syntax you can showcase in a landing page.
Roc, however, is at the stage where it needs to capture the interest of curious engineers that might be interested in the language itself, without worrying about the practical applicability of it too much. (BTW, I don’t want to imply that Roc can’t be used for anything practical at this point, I don’t know, but clearly I wouldn’t bet my start up on it and I doubt anyone expects that at this stage).
Man, as a person who sells languages (mostly TLA+) professionally, I wish that was the case. Selling to engineers is way more fun and feels more comfortable. But I’ve had cases where an entire team wanted to hire me, and cases where the team never heard of TLA+ but a manager or CTO wanted to hire me, and the second case way more often leads to business.
Which is the norm. Because our peers unfortunately mostly have very little power to make decisions.
I’m not saying that it literally never happens, I’m saying that by far language adoption at companies is driven by engineers - either through external forces (engineers building tooling/libraries in their free time, engineers starting companies with that language, etc) or through direct forces (engineers advocating for the language at that company).
I’m sure there are plenty of extreme cases like Oracle convincing schools to sell Java, or whatever, where this wasn’t the case.
It’s all moot since Roc’s language clearly shows that a website can have code and high level information, and meld them together well.
The guy literally said he sells languages professionally, and he’s telling us it’s driven by management, not engineers. Do you also sell languages? Or better yet, have some source of information which trumps his personal anecdotes?
In what way is Oracle an extreme case?
I don’t disagree that you can still include code, but even the high level blurbs will be different when selling to businesses, at which point, if the copy isn’t selling to developers, you might as well better use the space to more effectively sell to businesses where the code would have been (even literally empty space for a less cramped, more professional looking page).
Engineers care more about “fast (runtime)” and certainly “friendly” than most businesses do, and businesses care a lot more about “safe” than a lot of programmers do; “fast (compile time)”, ”reliable”, “efficient”, and “productive” may be mixed. Engineers also care about technical details like “functional” a hell of a lot more than businesses do, because they can evaluate the pros and cons of that for themselves.
I guess it’s much more effective to be very targeted in who you’re selling to, unless both profiles have equal decision making power.
And? I know almost nothing about them. They’ve apparently been pushing TLA+, an incredibly niche and domain specific language that the vast majority of developers never even hear of and that targets software that lives in an extreme domain of correctness requirements. It’s nothing at all like Rust. Maybe they also sell Python, or comparable languages? I don’t know, they didn’t say. Is it even evidence that selling TLA+ this way is the most effective way?
Years of experience in this field as an engineer, advocating and getting a large company to include languages in their accepted policies, generally just being an engineer who has adopted languages.
I do not buy, at all, that for a language like Rust (frankly, I doubt it’s the case for any language, but I can imagine languages like TLA+ going a different route due to their nature and lack of appeal to devs) that it’s not wildly favorable to target developers. I’ve justified this elsewhere.
The current Rust website is not ideal, but I think the general principle of convincing devs vs businesses holds.
Developers are picky about surface-level syntax. The urge to see the code is to judge how it fits their preferences. I see lots of comments how Rust is plain ugly and has all the wrong brackets.
Business-level decision makers don’t care how the code looks like. It can look like COBOL, and be written backwards. They ask what will happen if they spend resources on rewriting their code, and that better improve their product’s performance/reliability/security or devs’ productivity, not just make a neat-looking quicksort one-liner.
Of course the one who cares about code readability is the one who is going to read the code. What’s wrong with that?
Readability is subjective. Syntax must be functional and clear, but devs will care about details beyond that, and die on hills of which brackets it should use, and whether significant whitespace is brilliant or terrible.
Compare “The app is slower and uses more memory, but the devs think the code is beautiful” vs “The app is faster and more reliable, but the devs think the code’s syntax looks stupid”. If you want to convince product owners, the syntax is an irrelevant distraction, and you must demonstrate outcomes instead.
In web development there’s also a point of friction in developer experience vs user experience. Frameworks and abstraction layers have a performance cost (tons of JS), but not using them is harder and more tedious.
While readability is subjective to a certain degree, it stems from human psychology, which is based on how the real world works.
In the real world, things that are inside a container are smaller than the container, so if delimiters don’t visually encompass the text they contain, it looks counter-intuitive.
And things which are physically closer together are perceived as being more strongly associated, so
a::b.clooks like it should bea::(b.c), while in fact it’s(a::b).c.Performance and safety has nothing to do with syntax. It would be entirely possible (and not difficult) to create a language exactly like Rust, but with better syntax.
I think you’re confirming what I’m saying. You’re focusing on a tiny nitpick about a part of the syntax that doesn’t cause any problems.
And yet you’re prepared to bikeshed the syntax, as if it was objectively wrong, rather than a complex tradeoff. In this case the syntax and precedence rules were copied from C++ to feel familiar to Rust’s target audience. If you just “fix” the precedence, it will require parens in common usage. If you change the separator symbols, you’ll get complaints that they’re weird and non-standard, and/or create ambiguities in item resolution.
Showing code for a language invites such shallow subjective opinions, and distracts from actually important aspects of the language.
Code readability is a problem, especially in a language that frequently features complex declarations.
It is objectively wrong.
<and>are comparison signs, not brackets. (And Rust does also use them as comparison signs, creating even more confusion.)I think using
\instead of::would be fine in terms of familiarity. Everyone has seen a Windows file path.This is literally bikeshedding. Don’t you see the pointlessness of this? If you changed Rust to use your preferred sigil, it would still produce byte for byte identical executables. It wouldn’t change anything for users of Rust programs, and it wouldn’t even change how Rust programs are written (the compiler catches the
<>ambiguity, tells you to use turbofish, and you move on).Rust has many issues that actually affect programs written in it, e.g. bloat from overused monomorphisation, lack of copy/move constructors for C++ interop and self-referential types,
Sendbeing too restrictive for data inside generators and futures, memory model that makes mmapped data unsafe, blunt OOM handling, lack of DerefMove and pure Deref, lack of placement new forBox, etc. But these are hard problems that require deep understanding of language’s semantics, while everyone can have an opinion on which ASCII character is the best.Even Rust’s syntax has more pressing issues, like lack of control over lifetimes and marker traits of
async fn, or lack of syntax for higher-ranked lifetimes spanning multiple where clauses.You’re thinking of physics, not psychology. Concrete syntax is one of the least important parts of a programming environment, and we only place so much weight onto it because of the history of computers in industry, particularly the discussions preceding ALGOL and COBOL.
And they actually explain what they claim with “What does X mean here?”.
There is also https://prev.rust-lang.org, linked (not prominently) from the bottom of https://www.rust-lang.org. It doesn’t have the “eating your laundry” footnote, but the code runner works, unlike in web.archive.org.
I feel like the author was close to saying something regarding differences between a very-technical-senior and a soft-skills-senior. But we just have a kind of basic statement regarding engineering for failure.
‘so much code, so many services & engineers. twitter is over-engineered and I could do better’
There is definitely an aspect of Dunning-Kruger to some of the complaints, primarily driven by a lack of understanding of the sheer scale of Twitter, it’s backend (Twitter wasn’t all GCP (mostly analytics data) but largely, especially on the frontend, on-premises data centre hardware) and all of the analytics tools, machine learning crap, axillary code and such that comes with it. Anyone can write a lookalike Twitter frontend, backend given enough time. At the scale of actual Twitter comes a lot of hidden work.
I’ve worked on systems in statically typed languages, as well as dynamically typed languages, some with REPLs and fancy debugging tools, and some without.
As someone who loves lisps and also languages that are quite different from lisp, there’s a reason why homoiconicity, REPLs, and time-traveling debuggers didn’t take over the world, and it’s not because people are stupider than you.
There just isn’t one true model of computation. There isn’t one development/debugging workflow that’s globally better than all others. Humans are diverse and think and communicate in different ways. Some people are more visual in their thinking; others can fit a tremendous amount of state in their brains and think procedurally.
Regular reminder that life hacking one’s consumption habits will not change the world.
We will really buy hardware built in sweatshops, hand all our code to a company that contracts with the state, but the problem is that Microsoft looms at the keyboard shortcuts you use?
Microsoft telemetry running on your own computer is a more direct problem for you than sweatshops or Github existing, and you have more ability to change your own consumption habits to directly remove the opportunity for Microsoft to spy on you than to change how Github operates vis a vis a government or the working conditions of hardware factories in foreign countries whose languages and cultures you do not understand.
Oh yeah, totally, the problem with the internet is not enough “autonomy”. As opposed to having our personas traded by data brokers, brains melted by social media, behaviors modified by targeted advertising and influencer culture.
These techno-libertarian 80s MIT hacker derivative “autonomy” and “freedom” arguments have been such a massive distraction from a material analysis of how power actually uses technology to exploit people for profit. The spectacle of the individual atomizes people and alienates them from common cause.
But if only we were to perfect our APIs and network protocols. Then we would not be exploited by internet companies. We would have autonomy 🙄
I would like to subscribe to your newsletter
I mean, we already have “autonomy” just layer your protocol on IP and away with you beasties!
I used to give the same advice, but I completely changed my opinion over the past 10 years or so. I eventually put in the time and learned shell scripting. These days my recommendation is:
I really don’t want to figure out every project’s nodejs/python/ruby/make/procfile abomination of a runner script anymore. Just like wielding regular expressions, knowing shell scripting is a fundamental skill that keeps paying dividends over my entire career.
Bingo.
My advice is:
#!/usr/bin/env bashat the beginning of your scripts (change if you need something else, don’t rely on a particular path to bash though).set -eou pipefailafter that.localstorage qualifier when declaring variables in a function.sh, then do the work.While some people like the author will cry and piss and moan about how hard bash is to write, it’s really not that bad if you take those steps (which to be fair I wish were more common knowledge).
To the point some folks here have already raised, I’d be okay giving up shell scripting. Unfortunately, in order to do so, a replacement would:
There are basically no programming languages that meet those criteria other than the existing shell languages.
Shell scripting is not the best tool for any given job, but across every job it’ll let you make progress.
(Also, it’s kinda rich having a Python developer tell us to abandon usage of a tool that has been steadily providing the same, albeit imperfect, level of service for decades. The 2 to 3 switch is still a garbage fire in some places, and Python is probably the best single justification for docker that exists.)
I think “nine steps” including “always use two third-party tools” and “don’t use any QoL features like associative arrays” does, in fact, make bash hard to write. Maybe Itamar isn’t just “cry and piss and moan”, but actually has experience with bash and still think it has problems?
To use any language effectively there are some bits of tribal knowledge…babel/jest/webpack in JS, tokio or whatever in Rust, black and virtualenv in Python, credo and dialyzer in Elixir, and so on and so forth.
Bash has many well-known issues, but maybe clickbait articles by prolific self-pronoters hat don’t offer a path forward also have problems?
If your problem with the article is that it’s clickbait by a self-promoter, say that in your post. Don’t use it as a “gotcha!” to me.
I think there’s merit here in exploring the criticism, though room for tone softening. Every language has some form of “required” tooling that’s communicated through community consensus. What makes Bash worse than other languages that also require lots of tools?
There’s a number of factors that are at play here and I can see where @friendlysock’s frustration comes from. Languages exist on a spectrum between lots of tooling and little tooling. I think something like SML is on the “little tooling” where just compilation is enough to add high assurance to the codebase. Languages like C are on the low assurance part of this spectrum, where copious use of noisy compiler warnings, analyzers, and sanitizers are used to guide development. Most languages live somewhere on this spectrum. What makes Bash’s particular compromises deleterious or not deleterious?
Something to keep in mind is that (in my experience) the Lobsters userbase seems to strongly prefer low-tooling languages like Rust over high-tooling languages like Go, so that may be biasing the discussion and reactions thereof. I think it’s a good path to explore though because I suspect that enumerating the tradeoffs of high-tooling or low-tooling approaches can illuminate problem domains where one fits better than the other.
I felt that I sufficiently commented about the article’s thesis on its own merits, and that bringing up the author’s posting history was inside baseball not terribly relevant. When you brought up motive, it became relevant. Happy to continue in DMs if you want.
You’re really quite hostile. This is all over scripting languages? Or are you passive aggressively bringing up old beef?
Integrating shellcheck and shfmt to my dev process enabled my shell programs to grow probably larger than they should be. One codebase, in particular, is nearing probably like 3,000 SLOC of Bash 5 and I’m only now thinking about how v2.0 should probably be written in something more testable and reuse some existing libraries instead of reimplementing things myself (e.g., this basically has a half-complete shell+curl implementation of the Apache Knox API). The chief maintenance problem is that so few people know shell well so when I write “good” shell like I’ve learned over the years (and
shellcheck --enable=allhas taught me A TON), I’m actively finding trouble finding coworkers to help out or to take it over. The rewrite will have to happen before I leave, whenever that may be.I’d be interested in what happens when you run your 3000 lines of Bash 5 under https://www.oilshell.org/ . Oil is the most bash compatible shell – by a mile – and has run thousands of lines of unmodified shell scripts for over 4 years now (e.g. http://www.oilshell.org/blog/2018/01/15.html)
I’ve also made tons of changes in response to use cases just like yours, e.g. https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Programs-in-the-World
Right now your use case is the most compelling one for Oil, although there will be wider appeal in the future. The big caveat now is that it needs to be faster, so I’m actively working on the C++ translation (
oil-nativepassed 156 new tests yesterday).I would imagine your 3000 lines of bash would be at least 10K lines of Python, and take 6-18 months to rewrite, depending on how much fidelity you need.
(FWIW I actually wrote 10K-15K lines of shell as 30K-40K lines of Python early in my career – it took nearly 3 years LOL.)
So if you don’t have 1 year to burn on a rewrite, Oil should be a compelling option. It’s designed as a “gradual upgrade” from bash. Just running
osh myscript.shwill work, or you can change the shebang line, run tests if you have them, etc.There is an
#oil-helpchannel on Zulip, liked from the home pageThanks for this nudge. I’ve been following the development of Oil for years but never really had a strong push to try it out. I’ll give it a shot. I’m happy to see that there are oil packages in Alpine testing: we’re deploying the app inside Alpine containers.
Turns out that I was very wrong about the size of the app. It’s only about 600 SLOC of shell :-/ feels a lot larger when you’re working on it!
One thing in my initial quick pass: we’re reliant on bats for testing. bats seemingly only uses bash. Have you found a way to make bats use Oil instead?
OK great looks like Alpine does have the latest version: https://repology.org/project/oil-shell/versions
I wouldn’t expect this to be a pain-free experience, however I would say should definitely be less effort than rewriting your whole program in another language!
I have known about bats for a long time, and I think I ran into an obstacle but don’t remember what it was. It’s possible that the obstacle has been removed (e.g. maybe it was extended globs, which we now support)
https://github.com/oilshell/oil/issues/297
In any case, if you have time, I would appreciate running your test suite with OSH and letting me know what happens (on Github or Zulip).
One tricky issue is that shebang lines are often
#!/bin/bash, which you can change to be#!/usr/bin/env osh. However one shortcut I added was OSH_HIJACK_SHEBANG=oshhttps://github.com/oilshell/oil/wiki/How-To-Test-OSH
Moving away from Python? Now it has my interest… in the past I skipped past know it’d probably take perf hits and have some complicaged setup that isn’t a static binary.
Yes that has always been the plan, mentioned in the very first post on the blog. But it took awhile to figure out the best approach, and that approach still takes time.
Some FAQs on the status here: http://www.oilshell.org/blog/2021/12/backlog-project.html
Python is an issue for speed, but it’s not an issue for setup.
You can just run
./configure && make && make installand it will work without Python.Oil does NOT depend on Python; it just reuses some of its code. That has been true for nearly 5 years now – actually since the very first Oil 0.0.0. release. Somehow people still have this idea it’s going to be hard to install, when that’s never been the case. It’s also available on several distros like Nix.
What is the status of Oil on Windows (apologies if it’s in the docs somewhere, couldn’t find any mentioning of this). A shell that’s written in pure C++ and has Windows as a first class citizen could be appealing (e.g. for cross-platform build recipes).
It only works on WSL at the moment … I hope it will be like bash, and somebody will contribute the native Windows port :-) The code is much more modular than bash and all the Unix syscalls are confined to a file or two.
I don’t even know how to use the Windows sycalls – they are quite different than Unix! I’m not sure how you even do fork() on Windows. (I think Cygwin has emulation but there is way to do it without Cygwin)
https://github.com/oilshell/oil/wiki/Oil-Deployments
I believe Tcl fits those requirements. It’s what I usually use for medium-sized scripts. Being based on text, it interfaces well with system commands, but does not have most of bash quirks (argument expansion is a big one), and can handle structured data with ease.
I don’t do this. Because all my scripts are POSIX shell (or at least as POSIX complaint as I can make them). My shebang is always
#!/bin/sh- is it reasonable to assume this path?you will miss out on very useful things like
set -o pipefail, and in general you can suffer from plenty of subtle differences between shells and shell versions. sticking to bash is also my preference for this reason.note that the
/usr/bin/envis important to run bash from wherever it is installed, e.g. the homebrew version on osx instead of the ancient one in/bin(which doesn’t support arrays iirc and acts weirdly when it comes across shell scripts using them)Reasonable is very arbitrary at this point. That path is explicitly not mandated by POSIX, so if you want to be portable to any POSIX-compliant system you can’t just assume that it will exist. Instead POSIX says that you can’t rely on any path, and that scripts should instead be modified according to the system standard paths at installation time.
I’d argue that these days POSIX sh isn’t any more portable than bash in any statistically significant sense though.
Alpine doesn’t have Bash, just a busybox shell. The annoying thing is if the shebang line fails because there is no bash, the error message is terribly inscrutable. I wasted too much time on it.
nixos has /bin/sh and /usr/bin/env, but not /usr/bin/bash. In fact, those are the only two files in those folders.
https://mkws.sh/pp.html hardcodes
#!/bin/sh. POSIX definitely doesn’t say anything aboutshs location but I really doubt you won’t find ashat/bin/shon any UNIX system. Can anybody name one?I would add, prefer POSIX over bash.
I checked, and
shellcheck(at least the version on my computer) only catches issue #5 of the 5 I list.That’s because the other ones are options and not errors. Yes, typically they are good hygiene but
set -e, for example, is not an unalloyed good, and at least some experts argue against using it.Not for lack of trying: https://github.com/koalaman/shellcheck/search?q=set+-e&type=issues
There are tons of pedants holding us back IMO. Yes, “set -e” and other options aren’t perfect, but if you even know what those situations are, you aren’t the target audience of the default settings.
Yup, that’s how you do it, It’s a good idea to put in the the time to understand shell scripting. Most of the common misconceptions come out of misunderstanding. The shell is neither fragile (it’s been in use for decades, so it’s very stable) nor ugly (I came from JavaScript to learning shell script, and it seemed ugly indeed at first, now I find it very elegant). Keeping things small and simple is the way to do it. When things get complex, create another script, that’s the UNIX way.
It’s the best tool for automating OS tasks. That’s what it was made for.
+1 to using ShellCheck, I usually run it locally as
for POSIX compliance.
I even went as far as generating my static sites with it https://mkws.sh/. You’re using the shell daily for displaying data in the terminal, it’s a great tool for that, why not use the same tool for displaying data publicly.
No, it really is ugly. But I’m not sure why that matters
I believe arguing if beauty is subjective or not is off topic. 😛
I went the opposite direction - I was a shell evangelist during the time that I was learning it, but once I started pushing its limits (e.g. CSV parsing), and seeing how easy it was for other members of my team to write bugs, we immediately switched to Python for writing dev tooling.
There was a small learning curve at first, in terms of teaching idiomatic Python to the rest of the team, but after that we had much fewer bugs (of the type mentioned in the article), much more informative failures, and much more confidence that the scripts were doing things correctly.
I didn’t want to have to deal with package management, so we had a policy of only using the Python stdlib. The only place that caused us minor pain was when we had to interact with AWS services, and the solution we ended up using was just to execute the
awsCLI as a subprocess and ask for JSON output. Fine!I tend to take what is, perhaps, a middle road. I write Python or Go for anything that needs to do “real” work, e.g. process data in some well-known format. But then I tie things together with shell scripts. So, for example, if I need to run a program, run another program and collect, and then combine the outputs of the two programs somehow, there’s a Python script that does the combining, and a shell script that runs the three other programs and feeds them their inputs.
I also use shell scripts to automate common dev tasks, but most of these are literally one-ish line, so I don’t think that counts.
This makes sense to me
FWIW when shell runs out of steam for me, I call Python scripts from shell. I would say MOST of my shell scripts call a Python script I wrote.
I don’t understand the “switching” mentality – Shell is designed to be extended with other languages. “Unix philosophy” and all that.
I guess I need to do a blog post about this ? (Ah I remember I have a draft and came up with a title – The Worst Amounts of Shell Are 0% or 100% — https://oilshell.zulipchat.com/#narrow/stream/266575-blog-ideas/topic/The.20Worst.20Amount.20of.20Shell.20is.200.25.20or.20100.25 (requires login)
(Although I will agree that it’s annoying that shell has impoverished flag parsing … So I actually write all the flag parsers in Python, and use the “task file” pattern in shell.)
What is the “task file” pattern?
It’s basically a shell script (or set of scripts) you put in your repo to automate common things like building, testing, deployment, metrics, etc.
Each shell function corresponds to a task..
I sketched it in this post, calling it “semi-automation”:
http://www.oilshell.org/blog/2020/02/good-parts-sketch.html
and just added a link to:
https://lobste.rs/s/lob0rw/replacing_make_with_shell_script_for
(many code examples from others in that post, also almost every shell script in https://github.com/oilshell/oil is essentially that pattern)
There are a lot of names for it, but many people seem to have converged on the same idea.
I don’t have a link handy not but Github had a standard like this in the early days. All their repos would have a uniform shell interface so that you could get started hacking on it quickly.
You should investigate just for task running. It’s simple like
makebut none of the pitfalls of it for task running.Not listed but a really common reason for continuing to use Y over X is when, regardless of the original choice, the cost of switching doesn’t approach justifying any marginal differences.
This seems to be what the article meant by “Our custom internal solution is good enough”, except that that wording excluded the possibility of one’s current external solution being good enough.
I don’t have a problem with hacktivism and calling attention to injustice is generally the right thing to do. But the idea that you can call out war crimes by indiscriminately targeting disk drives of Russian citizens is really missing the point.
This morning I would had said you are right, however today I have picked up my children from their preschool where an Ukrainian boy at their age pissed under himself because he saw an unknown male (me), after having spent two days in a darkened evacuation train and having left behind everyone except his mum - and probably much more she would not tell. I have never seen a more scared being in my life. It wasn’t an anxiety, it was a kind of fear I wasn’t aware humans were able to experience. Let’s say I have a problem with that. If this ‘malware’ will stop at least one Russian ammunition train or make one Russian more stand up, the collateral damage(money) it causes is worth much more than the collateral damage(dead kids) that may be caused by the said ammunition. I don’t blame anyone having other opinion, but I understand and would myself do what the author of the module has and had to do.
We must do something;
This is something;
Therefore, we must do this.
Just because it’s an action that can be taken doesn’t mean it will help. In fact, I rather suspect this hurts the overall effort.
I see where you’re coming from. But I don’t see such methods changing the opinions or stance of a previously-indifferent (much less one who is supportive of the regime) Russian. I see frustration and anger, because they almost certainly aren’t thinking of the plight of Ukrainian families fleeing chaos, they’re thinking about their lost files.
I am not endorsing either side here, but your comment seems to presuppose that the sides of their target audience (people in Russia) are supports and opposes. I think in the view of the author of this hazardous code the sides are three: supports, opposes, and mildly in favor of the fictitious events pictured in Russian propaganda. If the theory is that the last group is the largest then they would need only small margin over 50% of that group to be pushed into the opposes group in order to justify the operation to themselves. Again, I’m not endorsing either viewpoint here – I’m just trying to highlight how views can shift depending on (mis)understanding of the subgroups and the size of those groups. Personally I don’t think we have very good estimates for these parameters but tbh I haven’t done a lot of research either.
This all depends on the broader context in which code is getting written. There’s no discussion of the incentives which lead to debt and guard against fixing it. It would be nice if our industry were to do science on quality, rather than just opinion.
Some instance of this rant is posted to this website every day.
What an embarrassing thing to publish. What not-strawmen/ real world programmers is DHH referring to?
This was my reaction as well. I was looking for some content or impassioned speech about being our best selves with some philosophical underpinnings. Instead we ended up with a tweet and a meme at the end.
I really like your “why should you try Emacs in 2021” list. It rings very true to my experience with emacs (and vim). Back when I used it more regularly, aside from any kool-aid I drank around the efficiency of keyboard-driven editing, if I’m being honest, the main motivation was simply that it was fun. Emacs was cool to me because I saw it as vintage, alternative, customizable, a component and expression of a hacker identity. It was an expression of programming-as-hobby as opposed to programming-as-profession. Of course I genuinely did enjoy the editing experience, and developing a healthy
.emacs. I loved magit, projectile, helm, the clojure packages, etc. But I ended up in a place professionally where ultimately Emacs had no leverage over more specialized tools for what I was working on.I personally don’t think you should be submitting posts from your company blog at all, ideally another community member would organically post it themselves if it’s interesting enough. Or the actual author of the post.
If you feel like you must promote your company blog, then I think you shouldn’t say you authored the post if you didn’t write it but you should also disclose your relationship with the company in a comment. I don’t think what you’re doing right now - saying you authored the post even though didn’t - sufficiently signals to readers the nature of the conflict and that the post is marketing for your company. There’s no need to add new site features for this either, a top-level comment on the story is sufficient.
I agree in principle, but if the content is good enough so that I don’t care that it’s on a company blog (this seems to be the case here), posting it (as not-the-author) is fine. So is an extra comment with “my coworker wrote this, happy to answer questions because *knowlede).
I see where your coming from but want to offer counterpoint.
Not every person follows every blog. So the assumption that content will get posted if it’s good enough ignores the idea that different people have are tapped into different information networks due to their social experiences, whether it’s a job or an online community.
I don’t see an inherent problem with posting content produced within the poster’s workplace. I don’t believe the stakes are remotely high enough on an aggregator site like this for there to be a conflict of interest. I also believe that on here, there’s no moral issue with a quality post coming from a company vs hobby blog, even considering the poster’s intent. There might be incidental issues, like a company blogging about ethically compromised tech, but that can be easily dealt with on a case by case basis.
Hard agree with all of this.
Personally, I’m excited about the “melting face” emoji, as well as a few of the new gender and skin color variants that will piss off chuds. Also, the Emoji block of Unicode now has not one, but two amulets against the evil eye, which I expect will be extremely valuable for social media.
I’m torn between “melting face” and “dotted line face”, I think they’ll replace my usage of 🙃going forwards.
The emoji thing is so totally irresponsible. Humanity is never going to replace Unicode. We’re stuck with it until we either go extinct or go Luddite. Adding emoji based on whims is how you end up with things like this sticking around for four thousand years and counting. The Egyptians at least had the excuse that they didn’t know what computers were.
I actually have that one saved in my favorites in UnicodePad for Android. Of course, the modern spelling would be 🍆💦.
Not sure what the problem is? Ancient Egyptians living thousands of years ago didn’t share your particular cultural taboos and sensitivities, which seems like an entirely valid “excuse” to me.
Right, there’s nothing that the Egyptians were doing “wrong”, because when they decided to use a penis as a letter, they had no way of knowing that for the remainder of human civilization we will have to use the penis as a letter, whether it’s culturally taboo or cool or we replace men with artificial sex bots or whatever. We however do know that Unicode is forever, and so the bar to adding a new character should be really fucking high. Like, here is an alphabet that was already in use by a non-trivial amount of people for some length of time. Not, it would be cool to make a new kind of smiley face.
A better system would be to do what is already done with flags. For flags, the flag 🇺🇸 is just
We could do the same thing for other ephemera, and not have to burden Unicode with an open ended and endless list of foods that were popular with the Unicode committee in the 21st century.
We don’t “have to use the penis as a letter” because it exists in Unicode. It’s just that it is technically representable. I’ll admit there’s nuance here - there are probably some things I’d rather see us avoid in Unicode, i.e. violence. But I’m struggling to see the harm caused in this particular case.
Who’s to say that the United States will still be around in 4,000, 1,000, or 200 years? Or that the “US” code won’t be recycled for some other country? Hell, why should our current ISO system of labelling countries even persist? Once you start talking about these kind of timeframes anything is up for grabs really.
“Forever” is a heck of a long time. I don’t think we’re stuck with Unicode for all eternity, there’s all sorts of ways/scenarios we could come up with something new. I think we should just address the issues of the day; there’s no way what the future will be like anyway; all we can do is focus on the foreseeable future.
I just imagined some kind of a Unicode successor system that would have a “compatibility” block with 200k+ slots and groaned.
That’s the whole point. US won’t mean 🇺🇸 forever. It will naturally change over time and when it does, the old codes will still be decipherable (flag for something called “US”) without needing to be supported anymore.
Tbh, the most likely a scenario is a RoC, PRC thing where two countries claim to be the US, and then the international community will have to pick sides. Anyway, still better than having the flag as a real emoji!
I don’t really follow how one scheme is more advantageous over the over; at the end of the day you’re still going have to map some “magic number” to some specific meaning. I suppose you could spell out “happy” or “fireman” in special codepoints, but that just seems the same as mapping specific codepoints to those meaning, but with extra steps (although “fireman” already consists of two codepoints: “man” + “fire engine”, or “person” and “women” for other gender variants).
The reason it’s done with flags probably has more to do that it’s just easier.
It’s not just that it’s easier it’s that obsolescence is a built in concept. New countries come and old countries go and ISO adds and removed country codes. Using Slack and GitHub style :name: emojis mean that you can add and drop support for specific emoji without needing to just serve up a �. It is also more forward compatible. When your friend on a new phone texts you :dotted smiley: you won’t just see �, you’ll see words that describe what is missing. Plus you aren’t using up a finite resource.
TIL integers are a finite resource.
To be fair, I’ll be the first to grab popcorn when they announce that everyone and their toaster now has to adopt utf8 with 1-5 bytes. Will probably be as smooth and fast as our ipv4 to ipv6 migration.
Right, that would be useful.
Changing the meaning of specific codepoints or sequences of codepoints over time just seems like a recipe for confusion. “Oh, this 300 year old document renders as such-and-such, but actually, back then it meant something different from today” is not really something that I think will help anyone.
This already exists to some degree; e.g. “Ye olde tarvern” where “Y” is supposed to represent a capital Thorn, which is pronounced as “th”, not as Y (written as þ today, but written quite similar to Y in old-fashioned cursive writing, and early German printing presses didn’t have a Thorn on account of that letter not existing in German so people used Y as a substitute). In this case it’s a small issue of pronunciation, but if things really shift meaning things could become a lot more apt to misunderstandings in meaning.
Emoji have already shifted in ways that change their meaning. The gun emoji has become a ray gun due to political correctness and sometimes points left and sometimes right. Shoot me → zap you. The grimace 😬 was a different emotion on Android and iOS for a while. There are other documented examples of this kind of semantic shift in just a short amount of time. I think it’s a bit hopeless to try to pin them down while you keep adding stuff. The use of eggplant and peach for penis and butt is based on their specific portrayal and subject to the visual similarity being lost if they redraw them in different ways. What if President Xi demands a less sexy 🍑? Will it stick around or be a bit of passing vulgar slang from the early twentieth century? Impossible to predict.
Why can’t we have a little fun? What is the problem you are seeing with this?
Your body shame is a culturally specific artifact and hardly a universal experience.
You’re missing the point. I’m not ashamed of weiners. They are hilarious. The point is that a character can be taboo or not and we’re still stuck with it.
If it’s not something to be ashamed of, is it really taboo enough to exclude from the Unicode standard? And furthermore, why is being stuck with it an issue? It can even be valuable from an anthropological standpoint.
Just because a standard exists doesn’t mean we have to use all of it all the time.
Or should the ASCII maintainers be embarrassed that their standard contains Vertical Tab?
My dude, Unicode inherited vertical tab from ASCII. That’s my point. Things are only going to continue to accumulate for now until the collapse of civilization. It will never shrink.
It’d be nice to have some actual background on hashing in here instead of just broad generalizations and links to various hash functions. Examples:
f(x) =0orf(x)=x) and why they cause problems even in non-hostile environments, but that would’ve required more of an introduction to how hashing works…)Programmers might not understand hash functions, but infosec furries may also not understand pedagogy.
(also, can you please cool it with the inflammatory article headlines?)
Please don’t pick a fight. It seems more angry than friendly.
Honestly I think it’s a valid concern. One of the biggest problems with the computer security world, as stated repeatedly by leading experts in the field, is communication and teaching.
A valid concern would be “infosec experts may not understand pedagogy” but why call out “infosec furries” specifically? Unless we should be concerned about infosec furries in particular vs other infosec experts?
Are these acceptable?
No. So why furries? People need to get over it and quit furry bashing. This isn’t acceptable behavior on Lobste.rs, and I’m tired of it.
See elsewhere for the explanation; furry bashing doesn’t enter into it, though I see why you might have read it that way. Furries are internet denizens like the rest of us, with all that entails.
I agree with you that it’s a bad title.
I also think that you wouldn’t have reacted nearly this strongly to the title if it wasn’t a furry blog.
I read your other comments. But you said what you said, and that undermines all your pontificating about the harm of “insulting/demeaning a group” and “the sort of microaggression/toxicity that everybody talks so much about.” Take your own advice.
“Furry” is a kink, not an identity or protected class. And normally you have to get people’s consent before you bring them into your kink.
I don’t see any sexual imagery in this blog post.
The OP’s site has some pretty well reasoned and presented articles on precisely why “furry” cannot reasonably be summarized as “a kink”.
And, no, you do not “normally” have to get someone’s consent to introduce them to the idea of your kink, unless said introduction involves you engaging them in the practice of your kink.
Sorry, I didn’t realize the “furry” part was what you were opposed to. It sounded like you were upset with the implication that the infosec world is bad at teaching.
https://www.youtube.com/watch?v=S2xHZPH5Sng
One of the things he talks about there is testing the hypothesis and seeing which title actually worked. I only clicked this link because I recognized your domain name and knew you had written interesting articles in the past and might legitimately explain something I didn’t know. If not for that, I probably would have bypassed it since the title alone was not interesting at all.
Even so, it is still possible to write clickbait titles that aren’t predicated on insulting/demeaning a group.
How would you feel if I wrote “Gay furries don’t understand blog posting”? Even if I raise good points, and even if more people would click on it (out of outrage, presumably), it would still probably annoy a gay furry who wrote blogs and they’d go in with their hackles raised.
The important difference between what I wrote and your hypothetical is the difference between punching up and punching down.
My original title was along the same lines as “Falsehoods Programmers Believe About _____” but I’ve grown a distaste for the cliche.
The difference between “Programmers don’t understand hash functions” and “Gay furries don’t understand blog posting” is quite obvious to me and I definitely don’t want to engage in whatever Internet flame is going on here. Especially since, uh, I have a preeetty good idea about what the problem here is, and I tend to think it’s about gay furries, not article titles, which is definitely not a problem that I have. (This should probably be obvious but since I’m posting in this particular thread, I wanted to make sure :P).
But I also think this title really is needlessly nasty, independent of how it might be titled if it were about other audiences. It’s a bad generalisation – there are, in fact, plenty of programmers who understand hash functions – and it’s not exactly encouraging to those programmers who want to get into security, or who think their understanding of these matters is insufficient.
I am (or was?) one of them – this was an interest of mine many, many years ago, at a time when I was way too young to understand the advanced math. My career took me elsewhere, and not always where I wanted to go, and I tried to keep an eye on these things in the hope that maybe one day it’ll take me there. Needless to say, there’s only so much you can learn about these topics by spending a couple of evenings once in a blue moon studying them, so I never really got to be any good at it. So I think the explanation is amazing, but it would definitely benefit from not reminding me of my inadequacy.
And I’m in a happy boat, actually, this is only an interest of mine – but there are plenty of people who have to do it as part of their jobs, are not provided with adequate training of any kind, have no time to figure it out on their own, and regularly get yelled at when they get it wrong.
Now, I realise the title is tongue-in-cheek to some degree, the playful furries and the clever humour scattered throughout the post sort of gives it away. If you think about it for a moment it’s pretty clear that this is meant to grab attention, not remind people how much they suck. But it’s worth remembering that, in an age where web syndication is taken for granted to the point where it sounds like a Middle English term, this context isn’t carried everywhere. Case in point, this lobste.rs page includes only the title. Some people might react to it by clicking because you grabbed their attention, but others might just say yeah, thanks for reminding me, I’ll go cry in a corner.
Even if I didn’t realise it was tongue-in-cheek, it probably wouldn’t bother me, partly because I understand how writing “competitively” works (ironically, from around the same time), partly because I’ve developed a thick skin, and partly because, honestly, I’ve kindda given up on it, so I don’t care about it as much as I once did. But I can see why others would not feel the same way at all. You shouldn’t count on your audience having a thick skin or being old enough to have given up on most of their dreams anyway.
I know this is a real struggle because that’s just how blogs and blogging work today. You have to compete for attention to some degree, and this is particularly important when a large part of the technical audience is “confined” to places like HN and lobste.rs, where you have to grab attention through the title because there’s nothing else to grab attention through. But maybe you can find a kinder way to grab it, I dunno, maybe a clever pun? That never hurt anyone. These radical, blunt (supposedly “bluntly honest” but that’s just wishful thinking) headlines are all the rage in “big” Internet media because, just like Internet trolls, they thrive on controversy, us vs. them and a feeling of smugness, but is that really the kind of thing you want to borrow?
(Edit: just to make sure I get the other part of my message across, because I think it’s even more important: title aside, which could be nicer, the article was super bloody amazing: the explanation’s great, and I like the additional pointers, and the humour, and yes, the drawings! Please don’t take any of all that stuff above as a criticism of some sort: I wanted to present a different viewpoint from which the title might read differently than you intended, not that the article is bad. It’s not!)
How do you know that you’re punching up?
What if the person encountering your blog is a programmer from an underrepresented background, just barely overcoming imposter syndrome, and now here’s this scary suggestion that they don’t understand hash functions? What if they actually made one of the mistakes in the article, and feel like they’re a complete fraud, and should leave the industry? This is the sort of microaggression/toxicity that everybody talks so much about, if I’m not mistaken.
The point is: you don’t know. You can’t know.
So, err on the side of not adding more negative shit to the world accidentally in the name of pageviews–especially when there are many, many other more positive options in easy reach.
EDIT:
I wouldn’t care if it weren’t for the fact that you’re a smart dude and clearly passionate about your work and that you have good knowledge to share, and that it pains me to see somebody making mistakes I’ve made in the past.
I’m neither of those things :P
I appreciate your compassion on this subject. It’s definitely new territory for me (since forever I’ve been in the “boring headline out of clickbait adversion” territory).
Do you actually not see a difference between saying a slightly negative thing about people of a certain profession and how they engage in that profession, and an ad-hominem using sexual orientation? What a weird and bad analogy?
I’m trying to assume good intent here but all your comments make it sound like you’re annoyed at the furry pics and awkwardly trying to use cancel culture to lash out the author.
Neither the label of programmers (with which I identify) nor of gay furries (with which the author identifies, according to their writing) is being misapplied. I’m sorry you feel that a plain statement of fact is somehow derogatory–there is nothing wrong with being a proud programmer or a proud gay furry.
My point in giving that example was to critique the used construction of “ is ”. I picked that label because the author identified with it, and I picked the “bad at blogging” because it’s pretty obviously incorrect in its bluntness. If I had picked “lobsters” or “internet randos” the conjured association for the person I was in discussion with may not have had the same impact it that “programmers” had on me, so I went with what seemed reasonable.
What do you gain by emphasizing soatok’s sexual identity, other than this morass of objections?
that’s exactly what friendlysock is hoping for
you’re right but it’s best not to feed them
Or they may read this and think ‘I’m glad it’s not just me!’. As a programmer who probably has a better than average understanding of hash functions, I don’t feel demeaned by this generalisation, if I were worried about my level of understanding I’d feel comforted by the idea that I wasn’t in a minority in my lack of understanding.
Or they may feel better that this mistake is so common that someone writes about it on a list of mistakes programmers make.
While I said you’re picking a fight (and would add: “look at the thread, it’s a fight”), I see what you’re saying in this paragraph. I also value non-judgmental explanations.
My problem with the title isn’t that it’s insulting, but that it’s inaccurate. Clearly some programmers do understand hash functions, even if other programmers do not. If nothing else, @soatok, a programmer, presumably understands hash functions, or why else would he write a blog post purporting to explain the right way to use them?
Specifically is wrong, at least about me, and almost certainly among other programmers as well. I don’t claim to have deep knowledge about cryptography, and I do expect that there’s probably something I could learn from this blog post, which I will read more carefully when I have a chance. But I am aware that the computer science concept of hash functions is useful for a variety of programming problems, and not just storing password-related data.
Why would anyone use this? I mean, I’d much rather use Perl than javascript on my servers.
Because a great many people prefer JavaScript to Perl, or at least know JavaScript better than Perl.
Not everyone is you. Hope this helps.
A use case would be using Pulumi for provisioning and zx for configuration, deployment and automation, with common libraries between both.
I gotta admit JS leaves me wanting when it comes to really basic stuff like list comprehensions and dictionary syntax, but being able to throw together an async-y tool to run stuff in parallel cleanly, for example, is a pretty great party trick.t
For example, you can easily make your build scripts just follow their dependencies cleanly without having to go full Bazel to get the proper speed boosts you might want
I am intrigued by the framing of the Sturm und Drang about the state of the web as being driven, to some significant degree, by politics internal to Google.
As I stated earlier this week, promo packets are what’ll do in the web.
I think a lot of developers simply lack the interest in context to process the realpolitik that shapes and distorts the fabric of spacetime for our industry.
If you refuse to understand that Google’s whole business is threatened by adblockers, you probably would be confused at the some of the changes to web extensions webRequest that make that work harder. If you don’t understand the desire to make SEO, crawling, and walled gardens easier AMP probably seemed like a great return to roots.
Other companies do this too, of course. If you didn’t know about OS/2 Warp some of the Windows APIs probably seemed weird. If you don’t know about Facebook trying to own everything you do then the lack of email signup for Oculus probably seems strange. If you invested heavily into XNA you probably got bit when internal shifts at Microsoft killed XNA off. If you don’t know about internal Canonical and RHEL shenanigans, systemd and other things probably are a surprise.
Developers need to pay as much attention to the business dependencies as the technical ones.
What do you mean by promo packets? I’m not familiar with this term.
When you’re doing a performance review at Google, you can request a promotion. If you do this, you put together a ‘packet’ including the impactful work you’ve done. New work is rewarded heavily, maintenance less so. For senior engineers, shipping major projects with an industry wide impact is a path to promotion.
Which means Google rewards doing something new for the sake of doing something new. It’s tremendously difficult to get promoted by improving older systems. Crucially, you often need to demonstrate impact with metrics. The easiest way to do that is sunset an old system and show the number of users who have migrated to your new system, voluntarily or otherwise.
Ew. Thanks for the insight. But ew.
Is there any material evidence suggesting that someone’s promotion is the reason that chrome will remove
alert? Obviously google will push the web in the direction that juices profit, but an individual promotion? Seems like a red herring.It is often difficult to pick it apart as it’s rarely a single person or team. What happens in large organizations is that there is a high-level strategy and different tactics spring from that. Then, there are metrics scorecards, often based on a proxy, which support the tactics delivering the strategy. This blurs the picture from the outside and means that rarely one person is to blame, or has singular control over the successes.
I haven’t followed the alert situation very closely, but someone familiar with large organizations can get a good read from the feature blurb. There is a strong hint from the language that they are carrying a metric around security, and possibly one around user experience. This translates to an opportunity for a team to go and fix the issue directed by the metrics since it’s quantifiable. The easiest way to start might be to work back from what moves the metric, but this is a very narrow perspective.
Developers may know what the best things to work on having been a developer in that area for 10 years, but their impact tracks towards those top-level strategies. Management can’t justify promotion because someone else is very focused on metrics that drive the strategy.
In lots of places this is called alignment. Your boss may only support X amount of work on non-aligned projects, if you do at least Y amount of work on Y projects. A classic big company alignment example is a talented person in a support department. If they can fix your biggest problem at the source it’d be best to let them do this. However, metrics incentivize assigning them to solving N support cases per week and other metrics designed for lower-skilled individuals instead of working on fixing root causes. Eventually, they leave unless you have smart management taking calculated risks, manage the metrics at the team level so the team is not noticed working the way it wants, seeking paths for talented people to work on the product, etc.
Many of us understand how metrics and incentives at tech companies work. Was just pointing out that it’s a bold claim to assume that chrome is removing alert due to an individual seeking a promotion.
I think about this in terms of my time at Apple – like, people ascribed all kinds of causes to various seemingly peculiar Apple decisions that to those of us on the inside were obvious cases of internal politicking leaking out.
WHATWG is a consortium of multiple companies so I’m curious why everyone is pointing the finger at Google here, or is the assertion that Google has so much power over the WHATWG and Chrome at this point that there’s no ability for other companies to dissent? (And I mean we all know that the W3C lost and WHATWG won so a consortium of vendors is the web.)
The multiple companies are Apple, Google, Microsoft, and Mozilla (https://whatwg.org/sg-agreement#steering-group-member, section 3.1b) Of the three, only Apple develops a browser engine that is not majority funded by Google.
I’m pretty sure Apple develops a browser engine that is majority funded by Google: https://www.theverge.com/2020/7/1/21310591/apple-google-search-engine-safari-iphone-deal-billions-regulation-antitrust
That’s some pretty weird logic.
The browser engine Apple creates is used for a whole bunch of stuff across their platforms, besides Safari:
Mail, iMessage, Media Store fronts, App Store fronts.. Those last two alone produce revenue about 4x what Google pays Apple to make it the default.
Do I wish they’d get more people using alternatives and pass on the google money? Sure. Is there any realistic chance their ability to fund Safari and/or Webkit would be harmed by not taking the google money? Seems pretty unlikely.
I don’t think the stores use WebKit. They didn’t last time I investigated.
It’s true-ish. But I’m sure the most profitable company in the world probably doesn’t require that money and would be able to continue without.
You don’t become the most profitable company by turning down revenue.
Right I was just wondering if folks think the WHATWG is run solely by Google at this point. Thanks for the clarification.
The point is that many of those new APIs don’t happen in standards groups at all. Exactly because they’d require more than one implementation.
Yes, this. Google’s play here is less about controlling standards per se (ed: although they do plenty of that too) and more about getting everyone to treat Whatever Google Does as the standard.
WHATWG was run at inception by a Googler and was created to give Google even more power over the standards process than the hopelessly broken W3C already gave them. That they strong armed Mozilla into adding their name or that Apple (who was using the same browser engine at the time) wanted to be able to give feedback to the org doesn’t change the Googlish nature of its existence, IMO
Like it or not, Google is the www. It is the driving force behind the standards, the implementations (other than Safari), and the traffic that reaches websites.
It would be odd if Google’s internal politics didn’t leak into the medium.
Right, it’s just … one of those things that is obvious in retrospect but that I would never be able to state.
A lot of people seem to think that standards work is a bit like being in a university - people do it for the love of it and are generally only interested in doing what’s best for all.
In reality it’s a bunch of wealthy stakeholders who realize that they need to work together for everyone’s best - they’re not a monopoly, yet - but in the meantime it behooves them to grab every advantage they can get.
As mentioned in the post, standards work is hard and time-consuming, and if an organisation can assign a dedicated team to work on standards, that work will get implemented.
Universities work like that too now
This is sadly true.
This basically boils down to good programming and clarifying interface boundaries. Prefer naming things over sprinkling anonymous functions around. The react hooks used in the examples provide no benefit and actually slow the code down.
Another nice project with an S3 compatible API is seaweedfs (née weedfs): https://github.com/chrislusf/seaweedfs, inspired by the haystack paper (from FB, when FB had around 1-2K photo uploads per second); we use it in production (albeit not in distributed mode). A lightning talk I did a few month ago: https://github.com/miku/haystack
if you do not mind, a question – did you find any solutions that are based on JDK 11+ (Java/clojure/scala, etc) – I am looking for a file store open source lib, but I would like it to be compatible with a JVM ecosystem.
Interesting, I’d assume a JVM ecosystem would permit non JVM code. Is it a JVM client library you want?
Not the OP, but I’ve heard that some banks will refuse to deploy any code that doesn’t run on the JVM.
Wow, do you have perhaps an example, or country of a possible example?
I know crux db is on the JVM, and they can use and even encourage their object store to be on Kafka (famously JVM)
Unfortunately, no. This was just word-of-mouth from people in adjacent businesses so feel free to take it with a grain of salt.
The general contour of reasoning was that with security being a top concern, they prefer to deploy code in ways that are familiar.
Thank you for the follow-ups. I would like the whole service to be packageable and accessible as a JAR that I can incorporate in our ‘uber’ JAR.
The backend I am working on, has one of its ‘features’ – a simple deployment. In the translation, it means a single-jar + PostgresSQL.
The single-jar within it, has about 20 ‘micro-services’, essentially. So a user can start one up just one jar and have everything in ‘one host’ or start the jar with config params telling the JAR which microservices to start on which host. That configuration file is like a ‘static’ service orchestrator. It is the same for all the hosts, but there are sections for each host in the deployment.
One of the microservices (or easier to call them just services) I am going to be enhancing is a ‘content server’. Today the content service basically needs a ‘backend-accessible-directory’.
That service does all the others things: administering the content, acting as IAM Policy Enforcement Point, caching content in a memory-mapped db (if a particular content is determined to be needed ‘often’), a non-trivial hierarchical directory management to ensure that too many files do not end up in ‘one folder’, and so on.
I need to support features where files are in a ‘remote content server’ (rather then in a locally accessible directory). Where the content server is an S3 (or some other standard compatible system) So I would like the ‘content server’ to be included as a 21st service in that single JAR.
Which is why, I am not just looking for a client, but for the actual server – to be compatible with JVMs. Hope the explanation gives more color to the question I asked.
With regards to other comment where folks mention that some organizations like banks – prefer a JVM only code. That’s true to a degree, it is a preference, not an absolute requirement though.
That’s because some of these organizations have built by themselves ‘pre-docker’ deployment infrastructures. Where it is easy to request a ‘production capacity’ as long as the deployed backend is a JAR (because those deployment infrastructures are basically JVM clusters that support migrations, software defined networks, load balancing, password vaults, monitoring, etc)
So when a vendor (or even internal team) comes in and says: for our solution we run on docker, it is OK, but they have invested millions… and now want to continue to get benefits (internal payment basically) for their self-build infrastructure management tools … Which is why there is a preference for JVM-only solutions and, perhaps, will be for some time.
And to be honest, JVM (and JVM based languages) and their tools ecosystem continues to evolve (security, code analysis, performance, etc) — it seems that the decisions back then about investing into managed infrastructure around JVM – were good decisions.