1. 3

    I disagree with the first point - that there is no universal “better”. There are objectively worse and better things but the examples in the post didn’t mention them.

    The article says, for example, that someone can prefer one kind of indentation and someone else might prefer another. But the difference itself is not essential in this case, that’s why there is no better or worse. Going back to scouts - its like arguing whether to place the fire place on the left side of the road or on the right side of the road. It’s hard to see why one should be preferred over others. And so there is no “better” or “worse”. But there are different set of situations and decisions. Like: should we put out the fire before leaving the place? Should we pack the trash? This is where “better” and “worse” is clear.

    This also transfers to coding. And its not only about clear cases like bugs or new features. If you improve the execution speed of some process without changing any of its behaviour - it is for the better. If you identify a potential memory leak and fix it from happening - that is better. If you see that a variable for something has one name in one file and another name in another file, fixing them to be the same will end in better codebase. If you write a documentation for a part where it was missing… etc.

    This “nothing can be objectively better than anything else” can only logically end with not having any code at all because who can say that any code is better than no code.

    1. 1

      Very nice interface.

      Is there a reason it is hardcoded to linux news? It seems it would be quite straight forward to extend the scope.

      1. 3

        Thanks! I have several sites for topics that interest me – DevURLs for developer news, TechURLs for technology news, and MathURLs for mathematics news. I like to keep the sites separate because this way the topics are isolated and the search results are very precise. For example, if I need to find latest news about Python, I search on DevURLs (devurls.com?q=python), if I need to find latest discoveries about Fibonacci numbers, I search on MathURLs (mathurls.com?q=fibonacci), etc.

      1. 8

        I expect a technical article about the fate of superior choices, and I was disappointed to find the usual bull.

        The likes of AmigaOS and BeOS advanced the state of the art. Inferior solutions such as Windows, MacOS and later OSX were the ones most adopted.

        Now we have seL4, but it’s the same story; Technical superiority means squat. That’s the problem with software. Dumb people with decision power. Otherwise-smart people putting up with them. And we have a lot of that. It’s a wonder progress is ever made despite this fact.

        1. 22

          Maybe we have to look to economic and political factors to understand why Windows and Mac won. We shouldn’t retreat into our techie bubble and pretend those things don’t matter.

          1. 4

            Pretending they do matter is at the heart of the problem.

            Imagine if, rather than going with the flow, we used our brains and did what was right, put effort where it matters.

            There’s nothing sadder than seeing otherwise capable individuals wasting their lives by pursuing the wrong endeavors, just because they are popular.

            If anything, what is severely lacking in society is the ability to take a step back and think, as opposed as following the flow. The few people capable are often the ones that end up making a difference.

            1. 13

              There’s nothing sadder than seeing otherwise capable individuals wasting their lives by pursuing the wrong endeavors, just because they are popular.

              I suppose you would consider me guilty of that. I work at Microsoft, on the Windows accessibility team. The work I do benefits not only Microsoft’s bottom line (somewhat), but potentially millions of people. Would it be better if I implemented a screen reader for AROS, or Haiku, or some OS based on SEL4? I could design a beautiful new accessibility architecture, possibly technically superior to anything currently out there. (Then again, I’m probably not actually that brilliant.) But who would it help? That, to me, would be a waste of my time.

              Of course, this is all ignoring the fact that I probably couldn’t get paid to work on one of those obscure OS’s anyway. It would have to be a volunteer side project, and some problems are just too big to solve on nights and weekends.

              1. 5

                The work I do benefits not only Microsoft’s bottom line (somewhat), but potentially millions of people.

                Microsoft customers, perhaps. Windows is unfortunately not open source as per OSI. It doesn’t count as work for humanity.

                Those who can do paid work on actually worthy projects are few and far between.

                I am myself an AWS drone. I do whatever I want on my free time, and I do get paid well, accelerating me towards not having to work at all (so called FIRE), which will free me to do whatever I want, full time.

                or some OS based on seL4?

                By all means. You’d be proudly at the forefront of computing, advancing the state of the art.

                1. 6

                  Windows is unfortunately not open source as per OSI. It doesn’t count as work for humanity.

                  Open source is fantastic but I wouldn’t ignore closed source software like that.

                  1. 2

                    I do not ignore it. I recognize it, but due to its closed nature (source or license), it is prevented from benefiting mankind as a whole.

                    1. 5

                      The problem with that statement is that it isn’t, in any way, true. In fact, it’s downright hard to do any kind of creative thing without benefiting mankind as a whole.

                      1. 1

                        Please define what you mean by ‘benefiting mankind’.

                        1. 1

                          In this context, it was qualified “as a whole” and meant nothing more than not being restricted to Microsoft clients.

                          Of course, give it enough time and if an idea has worth, it will be replicated.

                          1. 1

                            Of course, give it enough time and if an idea has worth, it will be replicated.

                            Are you sure of this? I’m not.

                            Since this statement is conditioned on “give it enough time”, as it stands, it is untestable.

                    2. 1

                      Those who can do paid work on actually worthy projects are few and far between.

                      By your definition of ‘worthy’ or theirs?

                      Do you have a philosophical stance on https://en.m.wikipedia.org/wiki/Moral_relativism ?

                      1. 1

                        By your definition of ‘worthy’ or theirs?

                        By theirs. Only a few fortunate people feel their job is worth doing, money aside. This impression is based on the views my network of acquaintances have on their jobs, and restricted to computer science graduates.

                        Do you have a philosophical stance on https://en.m.wikipedia.org/wiki/Moral_relativism ?

                        This is a dangerous topic I’ll respectfully decline to comment on.

                        1. 3

                          By theirs. Only a few fortunate people feel their job is worth doing, money aside. This impression is based on the views my network of acquaintances have on their jobs, and restricted to computer science graduates.

                          Then it may surprise you to learn that I do believe my job is worth doing, money aside, even though I’m working for Microsoft on Windows. It’s true that my work on Windows accessibility is only available to Microsoft customers and their end-users (e.g. employees or students). But that’s still a lot of people that my work benefits.

                          1. 1

                            What can I say, but congrats for working on a job you feel worth doing.

                          2. 2

                            I think don’t I get how you or they are defining worth. Can you explain more deeply?

                            Some example guesses based on people I know:

                            • If someone meant they wouldn’t do their job if they weren’t paid for it, that would hardly be a surprise. :)

                            • Or perhaps ‘worth’ is meant as a catch-all for job satisfaction?

                            • If someone said their job is to make system X be more efficient, but finds this to ‘lack worth’, perhaps they would like to see more direct results?

                            • If someone says their job is not ‘worth’ doing, perhaps they mean they hoped for better for themselves?

                            • Perhaps someone prioritized pay or experience in the near term as a means to an end, meaning some broader notion of ‘worth’ was not factored in.

                            • Impact aside, some jobs feel draining, demotivating, or worse.

                            • Some jobs feel like backwaters that still exist for historical reasons but add little value to the organization or customers.

                            1. 2

                              If someone meant they wouldn’t do their job if they weren’t paid for it, that would hardly be a surprise. :)

                              That one. And yes, I am not joking.

                              I otherwise see working as a losing proposition, as no amount of pay is actually worth not doing whatever you want with your time, which is limited.

                              1. 1

                                I otherwise see working as a losing proposition, as no amount of pay is actually worth not doing whatever you want with your time, which is limited.

                                I’m not sure how to parse the sentence above. With regards to “otherwise” in particular: Do you mean that work (without money) “is a losing proposition”? And/or do you mean “generally, across the board”… you should simply do what you want with your time? And/or something else?

                                How do you respond to the following claim?… To the extent work helps you earn money to provide for your basic human needs and wants, it serves a purpose. This purpose gives it worth.

                                I’m trying to dig in and understand if there is a deeper philosophy at work here that explains your point of view.

                    3. 12

                      I agree that it kind of sucks, but it will always be humans using and developing software, and we cannot expect humans to be rational. We are social beings and we have feelings, and things like popularity matter, whether we like it or not.

                      You’re right that blindly following the flow is what got us into this mess. But as technologists we need to understand the humans and politics behind these decisions so that we can create our own flows for the technically superior solutions.

                      1. 1

                        In context (e.g. day-to-day work, especially in systems regarding human safety), we do want to build better technical solutions because we want them to be more reliable, which means they fail less often and do less damage to people.

                        Some of us also want better technical solutions because it makes these systems more flexible to adapting contexts, which (hopefully) means less money and time is spent rebuilding half-baked systems that is, let’s face it, not the kind of work that many of us are hoping for.

                        Now, for a broader claim: narrowly ‘technically-superior’ solutions in the service of immoral aims are not something we should be striving for.

                  2. 8

                    I really doubt it’s just “dumb people with decision power”. It’s mostly the users.

                    There is a concept called “bike shedding” with an example: if you discuss, with a group of people, plans about building a nuclear power plant and plans for building a bike shed - people will discuss the bike shed a lot more. Because that is what everyone understands. This same concept applies to most everything. Take books. Most popular books are really “dumb”. Everyone can read understand those and they become popular.

                    I think same concept transfers to the software world. We have what we have, because this is what won the “so dumb, everyone can use it” race.

                    1. 2

                      I think same concept transfers to the software world. We have what we have, because this is what won the “so dumb, everyone can use it” race.

                      And it’s still based on misconceptions, unfortunately.

                      For instance, it’s pretty well accepted that concepts of modularity make programming easier, not harder. Concepts such as abstraction (as in the abstraction of the implementation behind an interface), or isolation (user processes run sandboxed with the help of mechanisms such as pagetables).

                      However, when it comes to microkernel, multiserver operating systems, people have trouble with the idea that they are actually more tractable, rather than less. They’ll defend monoliths, even when they’re Linux-tier clusterfucks with little in terms of internal structure.

                      At times, it seems hopeless.

                      1. 2

                        Not every abstraction turns out to be helpful. Sometimes they just make it impossible to figure out what’s going on.

                        1. 1

                          Absolutely.

                          But it’s hard to argue no structure (chaos) is better than structure.

                          1. 2

                            I’ll take code that only uses simple, known-good abstractions (eg structured control, lack of global state) but is otherwise chaotic (eg code duplication with small modifications etc) over code that applies the wrong abstractions any day.

                            1. 2

                              For chaotic, try and trace function calls within the Linux kernel.

                              1. 3

                                That’s exactly the sort of thing I’m talking about - messy, but tractable with static analysis tooling. It’s a hard slog, but you can clearly see how much of a slog there is within a couple of hours investigation.

                                Compare that to my daily driver - large rails apps. Not only are static analysis tools unable to follow the control graph, but the use of templated strings to find method names means you can’t even use grep to identify a given symbol.

                                Sometimes there’s no quicker way to figure out what, if anything, uses a given method than to read 100k lines of ruby source. There’s frequently no quicker way to figure out where a method call goes than running it in a debugger.

                                1. 1

                                  As the kernel runs in supervisor mode, I’d really prefer if it was very clearly structured and the execution flows going through it were obvious and didn’t require running it on a debugger.

                    2. 4

                      I met the developers of seL4. It’s a tool intended for a very specific set of use cases, mostly embedded systems and military tech. It’s not intended to be a replacement for Windows/Mac/Linux and is not at all the “same story”.

                      1. 2

                        That’s not what they originally advertized, though. Originally, it was one of many L4-centric projects that would be used as foundation for desktop, mobile, and embedded applications. Nizza, Perseus, Genode, OKL4A, INTEGRITY-178B, LynxSecure, VxWorks MIKS, etc are all examples which did desktops by putting a Linux VM on top of the kernel. The seL4 kernel had x86 happening to support that but with initial efforts focused on ARM.

                        I guess they realized the difficulty, both technical and marketing, of doing a secure workstation for x86. Most verification funding was also going toward embedded, IoT, and military. A military company bought OK Labs. It looks like they pivoted for now to totally focus on those areas building their component architecture. They even changed their website to only talk about these things. The NICTA website talked about things in this comment.

                        It’s probably good move given the software requirements are simpler. They’ll be more likely to succeed.

                        1. 2

                          This is what it is, currently. Doing whatever necessary to get embedded applications running is absolutely a much simpler scenario than a workstation one, and currently very realistic; They do have the examples to point to.

                          But there’s nothing stopping it from going further. Genode’s sculpt manages to demonstrate this really well.

                        2. 2

                          Emotionally, I think I know where you are coming from.

                          However, there isn’t a strong argument here. Some problems I see:

                          • There is not just one problem with software.
                          • You don’t explain what you mean by ‘dumb’ — it comes across as an amorphous insult
                          • There are many kinds of intelligence
                          1. 1

                            Dumb was an unfortunate choice of words. Technically illiterate would have perhaps worked, but there’s a component of closed-mindedness or unwillingness to consider alternatives.

                            The background of the post is having seen people who are otherwise intelligent and capable put up with terrible decisions from above and derived unhappiness. It is often better to stand for your beliefs (as in, if actually sure), and oppose these decisions. Doing so allows for “I told you.”. Should the company’s climate doesn’t allow this much, I’d suggest finding another job.

                          2. 2

                            The likes of AmigaOS and BeOS advanced the state of the art. Inferior solutions such as Windows, MacOS and later OSX were the ones most adopted.

                            Technical superiority has nothing to do with the success of a platform. User experience is the ultimate arbiter in this case. MacOS has better UX than most operating systems. Windows has better UX than Linux or seL4 for a p50 user (example: my mother). People are not dumb to choose Windows or MacOS over Linux / seL4, they simply go for the better UX. If you want to create a superior platform, it has to start with superior UX, everything else is secondary.

                            1. 3

                              Windows was always in the bottom league when it came to UX, it became a winner because it had guaranteed backwards compatibility with an even worse system: MS-DOS.

                              1. 2

                                And monopoly tactics. Similar story for IBM vs better-designed mainframes such as B5000.

                              2. 1

                                Technical superiority has nothing to do with the success of a platform.

                                Do you mean ‘is less important’ rather ‘has nothing to do with’?

                                If you really mean ‘has nothing to do with’ you have the burden of proving a negative.

                                A negative claim is a colloquialism for an affirmative claim that asserts the non-existence or exclusion of something.[10] The difference with a positive claim is that it takes only a single example to demonstrate such a positive assertion (“there is a chair in this room,” requires pointing to a single chair), while the inability to give examples demonstrates that the speaker has not yet found or noticed examples rather than demonstrates that no examples exist - Wikipedia: https://en.m.wikipedia.org/wiki/Burden_of_proof_(philosophy)#Proving_a_negative

                                1. 2

                                  Ok I bite.

                                  • VMS technically superior to Unix -> Unix won
                                  • OS2 technically superior to DOS -> DOS won

                                  It appears to me that technical superiority does not have anything to do with how successful a platform is. You can prove me otherwise.

                                  1. 3

                                    I’d argue compatibility is the primary driver of success.

                                    I can run windows 95 games on windows 10; I can open an excel document from 1995 today. Getting PHP or java code from 20 years ago to run is typically no big deal, and that’s a large part of their dominance in their respective niches.

                                    1. 1

                                      The way you are making the claim is oversimplified.

                                      You’ve also shifted your language from ‘success of a platform’ to a notion of ‘winning’. But it raises the question ‘over what timeframe’? These platforms are not static, either.

                                      For example, a big reason that Windows has remained a force (relative to competitors) is that it has improved its underpinnings over time.

                                      Proving a negative is often waste of time unless you are working with precise definitions and deductive reasoning.

                                      Let me suggest your time would be better spent by clarifying what you mean rather than making absolute statements.

                                      Or maybe you want to write a thesis showing every software platform and demonstrating that in every case, technical aspects played no role in their evolution and success across various time scales? If so, go for it. :P (Be careful not to cherry pick the time scale to suit your argument. Or leave out examples that don’t fit.)

                                      I’m trying to explain why oversimplified forms of argument are not very useful to me. My goal is to understand how these factors relate not only in the past but also in the future.

                                      Your version of your argument in your head may be useful to you in some sense, but the way you’re stating it is way too blunt. I think by adding some nuance, your mental model of the situation will improve. I intend this to be taken in the spirit of constructive criticism.

                              1. 4

                                Thanks, interesting article. I read some paragraphs 4 times to really get the point about what is being said.

                                And while it is full of leftist posturing (I mean - it quotes Marx, of all things) I agree with majority of ideas expressed there. But I think there is one weakness - while each individual idea is quite strong, the connection between them, in my opinion, is missing. I still am somewhat puzzled how they all connect and have a feeling not everything presented there was necessary to make the point. In particular I don’t see the connection between “software has a legacy of many people behind it” and anything else in the article.

                                The thing that immediately presents itself after reading the post is that software is indeed held back by the “previous way of doing things”. I am not sure if we should call this “capitalism”, but the idea that we cannot freely share, say, a movie with every other person on earth, even thou we have a costless way of doing that, without each individual person having to pay for it, as if it was a scarce resource, has to end one day.

                                1. 8

                                  And while it is full of leftist posturing (I mean - it quotes Marx, of all things)

                                  That you think that quoting Marx is ‘leftist posturing’ is actually really funny. If you quoted most of Marx or explained any of what it says to almost anyone, without saying it’s Marx, they’d agree with you. But you’ve been brought up and conditioned to associate the name with extreme left-wing whatever.

                                  All Marx said is that history is the result of material conditions rather than the idealism of strong men.

                                1. 3

                                  When I opened the site at first I thought it was written on Medium. It copies the style pretty closely: has “social” media links at the top right, has an estimated minutes it takes to read the article. Even a big irrelevant picture at the top. I am not sure if that is somehow ironic or not…

                                  Besides that, in my opinion, the article itself has valid points I enjoyed reading it and learned about nofollow.

                                  But there is a counter argument to be made about costs:

                                  If you write a blog post a couple of times a month you offer ~1000$ worth of work to Medium for free.

                                  True, but compared to what? Publishing a post on your own website actually costs you money. Out of these two: 1) medium gets 1000$ for free and in return you get exposure, followers, and free hosting; 2) you pay ~10$ per month to host your stuff and get less exposure; isn’t the first option a win-win, at least for some people?

                                  1. 8

                                    If Medium was making $1,000 from every one of their bloggers there’s no way they’d have to resort to shady dark-pattern faux-wall clicks to raise more revenue.

                                    1. 2

                                      Yup that is true, I would guess the average they make on an article might be below 5$ (their subscription cost).

                                      So of course they don’t get $1,000 as value is not based on how many hours you have spent on producing it, which was how the posted article arrived at this figure. But I am saying even if they did - those $1,000 are not taken from you. Specially if the alternative is that you have to pay for putting your own article online using your own website.

                                  1. 17

                                    Imposter syndrome is endemic in our industry because there are like 50 different types of “nerd”.

                                    At first there was the IT guys at school who I adored, and I emulated them, and it was good.

                                    once I was old enough to join a company myself I found the old greybeards who shunned microsoft/apple and wrote C/Perl and arcane AWK, and they became my idols, and I did everything I could to emulate them.

                                    Once I was close to them I was aware another paradigm of people who did project Euler for fun!

                                    Once I was there I found people who made demo-scene stuff and reverse engineered binaries, and saught to emulate that.

                                    Once I was there I saw the new age of kubernetes and full stack web devs and felt like “that is what programming is”.

                                    I guess the answer here is that you don’t have to know every tool, you can’t know every tool, you can drill down in a specific area as much as you like. The tools we have are optional, mostly, so if a person extolls the virtues of emacs over vim, or Rust over C++, we owe it to ourselves to listen, but not to punish ourselves for not knowing everything or fitting into a stereotype.

                                    “You do you”, we’re all just code monkeys. It’s very difficult to have any sort of confidence in such an ever changing and young industry.

                                    1. 16

                                      I started reading the article and was sympathetic. There’s nothing wrong with using a Mac (I’m using one now, and since OS X released the idea that ‘real programmers’ don’t use a system that is a certified UNIX workstation is a bit odd). I don’t write shell scripts often and use AWK even less often, so I generally need to look up some docs whenever I do. There’s a load of stuff that a lot of programmers either don’t do often or have never needed to do in their career path that they shouldn’t feel ashamed to now know how to do without looking it up. Programmer is a pretty big field now and there’s room for experts in a large number of non-overlapping subsets. There’s also a great deal of value to the profession in people who really understand business logic and can communicate it clearly to programmers, yet would struggle to write a simple program on their own.

                                      I found that I became a lot less sympathetic as I got further down the list. It moved from things that she really had no need to know unless she chose a different career trajectory to wilful ignorance. I’d be really nervous about hiring a programmer who professed an aversion to reading and planning. I’d be pretty nervous about hiring a front-end web developer who couldn’t write CSS: it’s not difficult (though doing it well is) and even if it’s not your main area of expertise, you’re likely to be working closely with designers who live in that world and need to have a common vocabulary for communicating with them. You can’t develop that without writing some CSS and reading some HCI texts (books, articles, whatever), just as they can’t develop it without trying to write some code. I wouldn’t want to hire a designer to write the logic of an application any more than I’d want to hire a programmer to design the interface but I would want to hire designers and programmers who understood what the other jobs entailed well enough that they can work together. The second half of the list tells me that she doesn’t want to be that kind of programmer just as the first half tells me that she doesn’t want to be a greybeard who considers anything invented after 1985 a waste of time. I’m left wondering what kind of programmer she does want to be and with the sneaking suspicion that it’s one who gets paid but isn’t expected to collaborate with anyone else or write maintainable code.

                                      1. 10

                                        I has exactly the same reaction as you.

                                        First few points - Mac, ok. Homebrew - well I use macports, but same. Use mouse - sure, a lot of great programmers use mouse. Acme is even a mouse centered editor. But the next one - using Vim with FZF, and combining that with mouse is where I started to slow down. And using FZF without knowing how to use it. Why not just use a proper IDE, I would have understood it. But this seems just … I don’t know… lost? And after that things like not wanting to read. And a webdev that doesn’t know css. And finally - a webdev that doesn’t know what a server is?

                                        I definitely didn’t start thinking that the person isn’t a programmer or for some reason could not be called one. But the whole thing has something off about it. Like “I am a surgeon, but I don’t know what is a function of a heart and can’t remember what scalpel to use without a nurse telling me; I been working as a surgeon for X years thou and can push a button on a machine and it does the job”.

                                        Something is a bit off. The best way to describe my reaction is - I surely agree that everyone is different, surely agree that this person should do what they want and as long as it works all is good. But at the same time I wouldn’t want to validate or encourage the values expressed in that article.

                                        1. 1

                                          I really don’t get how “I use a package manager” is a sign of weakness? Because you don’t have patience to waste a bunch of time doing stuff that’s already been automated? IDGI

                                    1. 8

                                      In addition to cd and ls, I suggest to use pushd/popd. Built into practically every shell, so nothing to install.

                                      Apart from that, Fish autocompletion is good enough to remember my “current” project folders.

                                      1. 2

                                        Is there a way to automatically push a directory unto the directory stack? And possibly just have it remember the last 10, 20 or however much? pushd/popd are two commands I could never get used to, because I think of them after I need them, not before.

                                        1. 5

                                          I think using popd after pushd is seldom useful.

                                          What I do is prepopulate the dirs stack using pushd -n and then navigate between the “bookmarks” using tilde expansions like cd ~2 which don’t “pop” folders from the stack.

                                          1. 1

                                            I think using popd after pushd is seldom useful.

                                            Why? It would be like a “back” button in a browser/file manager.

                                            navigate between the “bookmarks” using tilde expansions like cd ~2

                                            Meh, I have a “jump” function already where I can use human-readable names.

                                          2. 2

                                            Unfortunately bash doesn’t have zsh’s autopushd. Years ago I made two aliases that suppress pushd/popd’s noisy output and alias cd/bd (“back directory”) to them - I’ve found it very handy and use it daily.

                                            # don't echo the current stack on pushd
                                            function quiet_pushd() {
                                                # preserves functionality when aliasing cd to quiet_pushd
                                                if [ $# == 0 ]; then
                                                    builtin pushd $HOME/ > /dev/null;
                                                else
                                                    builtin pushd "$@" > /dev/null;
                                                fi
                                            }
                                            
                                            # don't echo the current stack on popd
                                            function quiet_popd() {
                                                builtin popd "$@" > /dev/null;
                                            }
                                            
                                            1. 1

                                              In zsh: setopt autopushd

                                              I’m unclear about other shells, though it can be emulated by writing a function / alias for cd.

                                            2. 1

                                              I know about pushd and popd, have a few scenarios in my head where I think they might be useful, but have never used them productively in practice. Some times I note in passing that it would be nice to go back more levels into cd history than simple cd - but those moments are quite rare. And once you catch yourself wanting to do that using pushd is already too late.

                                              I secretly wish pushd would be auto implemented as cd history with a shortcut to trigger it, like maybe cd ^

                                              1. 2

                                                autopushd and pushdsilent seems to be what you want. I think oh-my-zsh enables them by default but I’m in the process of setting my shell from scratch these days (zsh + starship + cherry-picking the cool parts of oh-my-zsh) for performance reasons, and this discussion finally made me search for it.

                                                http://zsh.sourceforge.net/Intro/intro_6.html

                                                1. 1

                                                  Thanks for these.

                                                  I am somewhat on a streak for getting back to minimal (almost ascetic) setup on my machines, so things like zsh are out of the question. But will definitely keep autopushd thing in mind if someday I change my mind.

                                            1. 11

                                              Nice section titles!

                                              If you haven’t tried it already, you might want to look into fish shell or zsh autosuggestions. The idea is to automatically show the top entry from history, which matches current command.

                                              I subjectively like this UX a lot: nothing to configure, nothing to invoke explicitly, surprisingly broad applicability (cd is just one command which benefits from this). In other words, just like how dev tools should work (:

                                              1. 3

                                                Thanks!

                                                Good advice about zsh and autosuggestions. I am using bash so have no niceties like that, but others will definitely find it useful. Personally, at the current stage, I consider my problems with filesystem navigation to be more or less solved, using the simple folder structure described at the end.

                                                1. 3

                                                  Yes! Fish is a great suggestion. I will add that both Zsh and Fish support autocompleting long paths by typing something like ls ~/w/b/p/tand then hitting tab, so that it autocompletes to ls ~/work/bitbucket/project/tests or whatever. If there’s any ambiguity, you can add more characters or keep pressing tab until the right path shows up. Combine this with the suggestions and control+E in Fish and I’ve been a lot more productive.

                                                  1. 1

                                                    I will add that both Zsh and Fish support autocompleting long paths

                                                    TIL, thanks!

                                                1. 22

                                                  I have been blogging for 15+ years. I think looking for external validation is short sighted. While people have contacted me occasionally and thanked me, and I have even gotten a few $$$ from my blog, the real reason I write is intrinsic. It clears my mind and provides a way for me to realize what I am really thinking. I do it publicly because otherwise I wouldn’t (not much of a journaler).

                                                  However, if you don’t like the way the internet is currently organized, blogging is a great way to make a small change. Never been easier, all you have to commit is your time.

                                                  1. 9

                                                    This is the same with me. I typically write a post describing how I got to a certain conclusion and what choices I made along the way. It helps me internalise those and then look back and refresh my memory when needed. If it helps others who stumble upon the post - all the better. But it is not the main reason.

                                                    IMO blogging for fame or money ultimately will lead to SEO, marketing, merch and all other things of this nature in one form or another.

                                                    1. 3

                                                      You’re describing a very valid use case for a blog. It is just as much there for you as it is for someone else that might discover it and find it useful. It’s a good balance to have.

                                                    2. 8

                                                      Likewise, I’ve been doing it in some form since the late 90s, mostly in the same place.

                                                      I don’t run any analytics, but if I had to guess I’d assume that my readership spikes as high as a few dozen, once or twice a year, but mostly hovers in the range of 7 or 8. I could name most of them - friends, family, former coworkers. Writing for that handful of people feels valuable, as does documenting technical stuff in public view. Still, after publishing something on the order of 400k words I’ve got to say that the ratio of fame and fortune to time & effort expended is not exactly impressive, if that’s what I were looking for out of it.

                                                      Not that there’s no external validation: A long history of public writing has probably been narrowly better for getting jobs than not, though I’ll be extremely unsurprised if it bites me in a career-limiting way one of these years. Once in a while I get a few internet points. But I wouldn’t really advise most people to look for it. I selfishly want more blogs to read, but I also think that a whole lot of people might do just as well writing a newsletter for a handful of close connections or similar.

                                                      (I’d also point out that airing your thinking and details about your life on the public network has turned out to be a very, very different risk proposition than it seemed when I got a GeoCities page in ~1997. At a bare minimum, you’re teaching the ever-growing panopticon more about you, and there are plenty of scenarios where it leaves you more vulnerable to malicious actors, of whom there are plenty.)

                                                      1. 1

                                                        Good point about newsletters! For some it can be an even more convenient option for a platform that they own and control.

                                                      2. 4

                                                        It’s amazing when you have such a strong internal drive and motivation. And it shows in the fact that you have managed to blog for so long. It definitely works for some people. In my experience, a lot of people don’t write with intrinsic reasons as their main motivation which is perhaps why more either quit or simply choose to have a social media profile rather than a website. That external validation is so much easier to get on social media with likes, comments, views etc. With a blog, it takes more effort.

                                                        1. 8

                                                          That external validation is so much easier to get on social media with likes, comments, views etc.

                                                          100%. And I’m no saint, I like it when a post of mine gets traffic, a comment, or hits the front page of a popular site.

                                                          I just think that extrinisic motivation will fade.

                                                          1. 1

                                                            Makes sense!

                                                      1. 12

                                                        I read it but there is something I am still missing.

                                                        So say I have 10,000 notes. I write a new one. How on earth could I link it with others that I’ve forgotten about? Surely I cannot re-read all of 10,000 to see which ones I should link together. I do not get this part of it.

                                                        1. 11

                                                          Incidently, I just stumbled upon this Zettelkasten-like note: Notes should surprise you:

                                                          If reading and writing notes doesn’t lead to surprises, what’s the point?

                                                          If we just wanted to remember things, we have spaced repetition for that. If we just wanted to understand a particular idea thoroughly in some local context, we wouldn’t bother maintaining a system of notes over time.

                                                          This is why we have dense networks of links (Evergreen notes should be densely linked): so that searches help us see unexpected connections.

                                                          This is why we take Evergreen notes should be concept-oriented: so that when writing about an idea that seems new, we stumble onto what we’ve already written about it (perhaps unexpectedly).

                                                          The linked note inside starts with this:

                                                          It’s best to factor Evergreen notes by concept (rather than by author, book, event, project, topic, etc). This way, you discover connections across books and domains as you update and link to the note over time (Evergreen notes should be densely linked).

                                                          So one aspect of the answer suggested here: By writing notes about general concepts, you provoke revisiting them. Through heavy linking you now create connections between related ideas. Through further maintenance like overview notes transitive connections become more direct connections and thus closer. (As a self-refential example: This reminds me of union-find and how it merges disjoint sets).

                                                          1. 5

                                                            How does one find “related notes”?

                                                            I believe the answer lies here:

                                                            Luhmann described his Zettelkasten in different ways. Sometimes he called it a conversation partner and sometimes he described it as a second memory, cybernetic system, a ruminant, or septic tank.

                                                            You actually just keep a big chunk of the notes in your memory. You don’t memorize their literal content, but you know of their existence, so you know approximately where to look. I would say that the Zettelkasten was a tool to enhance his own memory rather than a tool to do the remembering for him.

                                                            The secret to keeping the existence of the largest part of 90.000 notes in memory is probably revisiting them regularly, by following links and browsing at random, both of which the Zettelkasten invites you to do. Luhmann’s notes were all centered around his single general focus area: philosophy and social science, so he was thinking about the full body of work all the time.

                                                            I think the brain is better at this sort of thing than we generally acknowledge. I actually believe that this may be an argument for using an analogue Zettelkasten. You come up with an idea and your brain hints that you have thought about something related before, you just don’t know when and what exactly. You might vaguely remember a particularity of a card though, maybe a tear, a stain or even the smell. So you go browsing, and soon enough you run into a connected idea, which leads you to the subweb of existing ideas that are relevant to the new idea. This rummaging around has the additional benefit of refreshing your memory of existing notes.

                                                            The efficiency on such a large scale is probably for a large part determined by your ability to leverage the hints your unconsciousness gives you, which I think is why Luhmann describes his Zettelkasten in varying, not very rational or strict ways, in an effort to capture this unconscious process.

                                                            1. 4

                                                              My interpretation is that you would have meta-notes that serve as an index for notes about a given topic. (I do that and it’s very useful) So you might not remember the existing 10,000 notes, but surely you can relate your new note to some topics you already touched, and then you can go look at the index for those, and follow the links that look relevant.

                                                              Then, I guess one assumption is that, while exploring the existing web of notes to connect it to your new note, one should dedicate a bit of time and make some new connections if they feel relevant, i.e. do some general “maintenance” of the system.

                                                              1. 2

                                                                If you completely forgot about something, then you have no trigger to even look for something to link to. However, if you have a vague memory of something, a system of indices can be helpful to refresh your memories.

                                                              2. 3

                                                                It’s about destruction and creation. You deconstruct something to use it’s parts for creating something new.

                                                                1. 2

                                                                  This is exactly the question none of my reading has answered. I haven’t read “Taking Notes the Smart Way”, but I’ve otherwise read a lot, and nothing seems to address that question.

                                                                  I have my own ideas, but I’d love to see how other people solve this.

                                                                  1. 7

                                                                    Yeah I’m starting to think that “do X the smart way” is mostly BS for most values of X.

                                                                    The more I go on the more it seems to me that the system is not important as long as you’re consistent with its use (whether it’s note taking or organising or anything else).

                                                                    Also, the most productive people I’ve seen… They don’t use fancy things that make them “super productive”. They just sit down and do the work, and they got good and fast at it by means of practice.

                                                                    The more I go on with life the stronger I feel the smell of BS.

                                                                    1. 4

                                                                      I gave it some thought and this is what I have:

                                                                      Linking problem is not solved. If it was solved there would not be any need for adding links in the first place. Why add links at all if you can easily find other notes related to the current one? So the fact that links exist leads me to the conclusion that linking is the actual hard part of this method.

                                                                      Second - the whole approach is a hyperlink data base structure before data bases became a thing. This system can be implemented in a single table having 3 columns: 1) ID, 2) Note, 3) References. We have software now so maintaining a data-base in a furniture is probably obsolete.

                                                                      But most importantly, like the article said, Zettelkasten  becomes better the more notes are added to it. OK. So the logical conclusion is to invite other people to add their notes to it. Then it grows faster. Everyone is adding their ideas, and creating links between ideas. To me it seems such a state is the ultimate goal of this system. But we have this now, it’s called the internet.

                                                                      So I think whatever made that one german scholar so productive probably wasn’t the system itself.

                                                                      1. 1

                                                                        You say that as if the internet isn’t a hugely transformative, productivity increasing tool.

                                                                        1. 1

                                                                          I would go as far as to say that for many people it’s the opposite. It can be used as productivity increasing tool and an amazing one at that. But many people use it for other things completely.

                                                                      2. 1

                                                                        I think the theory is that if you’re taking a note, you’ve probably already got a bunch of notes about the thing you’re researching handy, so you link to those. For my part that doesn’t match how I generally take notes (I often find myself making note of something I stumble on, and writing summaries/self-tutorials of stuff I’m studying to ‘learn it’ more thoroughly).

                                                                        That said, the Zettelkasten core idea seems to me to be ‘smaller notes, more often’, and also to follow the old adage: “Graphs are a set of Edges that incidentally have some Vertices attached.” Links and granularity are the key takeaways, and there are good ways to do that that aren’t exactly Zettelkasten.

                                                                        1. 1

                                                                          I’m currently trying TiddlyWiki because its UI encourages smaller notes.

                                                                          With a physical Zettelkasten you implicit see neighbor slips as you search a linked one. This is lost in a digital version where searching is delegated to computer and effectively instant. It would be easy to track all kind of implicit connections (backlinks, created before and after) but how to present that in a helpful way?

                                                                      3. 2

                                                                        I don’t think the system is supposed to answer that question. Connecting ideas is the job of the human, not the system. However, if you do connect some ideas, you don’t have to remember that connection. Your web of thinking is externalized.

                                                                        I’m sure there are strategies. Randomly showing any two cards seems as good as any strategy. This would only increase the probability (marginally) that two ideas get connected. It would still be up to you to figure out how they are connected.

                                                                        1. 4

                                                                          But to create the connection you have to know that the connection is possible, so you have to be aware of the other cards to which you can connect this one. That’s the essence of what a zettelkasten is supposed to be, and what no one is explaining.

                                                                          Here, I’ve read something, I’ve written a short, pithy note. Where do I file it? To what other cards is it connected? How do I find them?

                                                                          I have some ideas, but it’s supposed to be a solved problem via this method, and I’m seeing zero discussion of it in any of my reading.

                                                                          There’s the puzzle.

                                                                          1. 4

                                                                            I think of zettelkasten a bit differently. In database terminology, I see it as normal form applied to ideas. Ideas are often tightly coupled to their original context. This is why it’s natural to apply a hierarchical structure to notes. You just place the note under a folder which represents the original context. In zettelkasten you make the idea atomic and if you want to give it context then you have to reify that context with a link (a foreign key). Zettelkasten isn’t your schema, zettelkasten is relational algebra. You’re free to come up with any schema you want.

                                                                            Where do I file it?

                                                                            In the same folder with everything else. It’s flat. The structure is provided by the links.

                                                                            How do I find them?

                                                                            I use org-roam in emacs. A digital system helps with search. The original zettelkasten had physical organization by topic and you could traverse the links from there. I’ve only been doing this for a few weeks so maybe search might not scale.

                                                                            Btw, I think your questions are great ones. All I’m saying is I don’t think the system has an answer.

                                                                            1. 2

                                                                              Thanks for the reply … I pick up on this:

                                                                              How do I find them?

                                                                              I use org-roam in emacs. A digital system helps with search.

                                                                              I agree that search solves/avoids a lot of the problems that a purely analog system would/did have.

                                                                              But the whole point of a zettelkasten is that it helps with the search, either to avoid it, or to guide it, or to augment it. Luhmann described “having a conversation” with his zettelkasten.

                                                                              So I’m here, I have a new “card” … I can search the existing ZK for cards that have the same words, but that suffers the problem of combinatorial explosion, and it’s not using the zettelkasten in any kind of clever way.

                                                                              I think there are answers to be found, and perhaps Luhmann had some, and even his enthusiasts don’t really “get it” beyond having a huge box of index cards with some sort of indexing system. It feels from my reading that Luhmann had more than that. Otherwise it’s just a wiki with search.

                                                                              As you may be able to tell, I’ve thought about this a lot.

                                                                              1. 1

                                                                                I think the idea is to never make a note without linking to it from some context of an existing note. So you might have a note on when is the best time to do exercise. That note should be referenced from an index card on exercise, or maybe a note on daily routines.

                                                                                I think there is some hope it’ll be like a wiki, but maybe something more like tvtropes or c2 than Wikipedia where all the ideas are enmeshed are easy to move between.

                                                                            2. 3

                                                                              You connect nodes you know of. Since nodes are connected to nodes, this lets you wander the node links. No node contains every link, because that’s not their job. Their job is to link to whatever you still remember so you can later link it to what you’ve long forgotten.

                                                                              1. 2

                                                                                I have some ideas, but it’s supposed to be a solved problem via this method, and I’m seeing zero discussion of it in any of my reading.

                                                                                How to Take Smart Notes is worth a read, IMO, though it is still kind of nebulous on some points.

                                                                                There’s an “Everything You Need to Do” section. It says:

                                                                                Now add your new permanent notes to the slip-box by:

                                                                                a) Filing each one behind one or more related notes (with a program, you can put one note “behind” multiple notes; if you use pen and paper like Luhmann, you have to decide where it fits best and add manual links to the other notes). Look to which note the new one directly relates or, if it does not relate directly to any other note yet, just file it behind the last one.

                                                                                b) Adding links to related notes

                                                                                c) Making sure you will be able to find this note later by either linking to it from your index or by making a link to it on a note that you use as an entry point to a discussion or topic and is itself linked to the index.

                                                                                Presumably, in order to find things to link if you don’t have them ready to hand, you use the existing index.

                                                                                1. 1

                                                                                  The magic seems to lie here:

                                                                                  a) Filing each one behind one or more related notes … b) Adding links to related notes

                                                                                  This is the question no material or article seems to be answering: How does one find “related notes”? It refers to “the index”, but that’s rarely referred to elsewhere and seems utterly mysterious. How are things indexed?

                                                                                  I should write up my musings as best I can to further reveal my confusion and incomprehension. I’ll try to do that.

                                                                                  1. 1

                                                                                    So, full disclosure, I’ve taken notes on cards before, but at the time I gave them either meaningful names or datestamps rather than Luhmann-style sequential identifiers or what-have-you, so I just kept them in a card box sorted by alpha / date. The collection never grew substantial enough for me to think about other forms of indexing. The cards I’ve held onto are in a single cardboard file box with little alphabet dividers I bought at an office supply store.

                                                                                    That said, indexes for paper information storage are a pretty well-established technology.

                                                                                    If I were going to take a crack at it for a paper Zettelkasten, I would:

                                                                                    • Set up a large card box with alphabet dividers.
                                                                                    • When adding a note to my permanent collection:
                                                                                      • Search the index for keywords pertaining to the new note
                                                                                      • If no card exists in the index for the keyword (name, phrase, concept, etc.) I want to be able to track down again:
                                                                                        • Print the keyword on top of a fresh card
                                                                                      • Write down the ID of the related note on any relevant cards (either existing ones or the ones I’ve just created)
                                                                                      • File the cards with keywords alphabetically in my index

                                                                                    Here’s an abbreviated example from a random page of the index in the first reference volume I could find on my office bookshelf, The Chicago Manual of Style:

                                                                                    …
                                                                                    ornaments for text break, 1.56 
                                                                                    orphans (lines), 2.113, p.899
                                                                                    o.s. (old series), 14.132
                                                                                    outlines
                                                                                        basic principles, 6.121
                                                                                        parts of a book, 1.4
                                                                                        publishing process, 2.2, fig. 2.1, fig. 2.2
                                                                                        punctuation and format, 6.94, 6.126
                                                                                        See also: lists
                                                                                    Oxford comma, 6.18-21. See also commas.
                                                                                    …
                                                                                    

                                                                                    This is a ~900 page reference work with fairly dense text, and about a hundred pages of index, so that ought to give you some very rough idea what ratio is useful for a working reference system, and how far an alphabetized index can scale. The references are mostly to section numbers or figures rather than pages, which seems like a pretty useful parallel to how things could work with numbered cards.

                                                                                    The other thing you might want to research is library card catalog approaches. The paper card catalog systems I was taught at length in elementary school have all pretty well been obliterated by electronic databases by now, but there was once a range of well-developed techniques there for indexing into very large collections by author and subject matter.

                                                                                    There’s nothing to stop anyone from translating these techniques directly to software, though there are probably more automated ways to get most of the same benefits in any given system. (i.e., tagging systems, automatic keyword indexing, and good old grep.)

                                                                            3. 1

                                                                              I’m thinking of trying this out and my idea was to try some sort of transitive closure view and “show me five random notes” thingymajiggy.

                                                                              1. 1

                                                                                How on earth could I link it with others that I’ve forgotten about? Surely I cannot re-read all of 10,000 to see which ones I should link together. I do not get this part of it.

                                                                                Practically speaking:

                                                                                • in a paper system, the answer is probably an index and a sorted reference collection
                                                                                • in an electronic system, the answer is some combination of search and tagging

                                                                                Relatedly, the cards (or other unit of note-taking) aren’t intended to be an append-only log. You’re supposed to interact with the system and refine the web of connections as you go, so it may not matter if something is initially orphaned.

                                                                              1. 1
                                                                                1. Read good source code written by other people that are better than you in some way.
                                                                                2. Learn other paradigms of doing things. i.e. learning some Haskell might improve your Python.
                                                                                3. Read some time-tested books about programming in general (like “the practical programmer”).
                                                                                4. Get in the habit of not relying on StackOverflow or your IDE autocompletion for answers.
                                                                                5. Observe what works and what doesn’t. Try to articulate reasons why something doesn’t work.
                                                                                6. Dive deep in to what you are doing (e.g. data structures, algorithms, statistics, data bases, etc).
                                                                                7. Practice, practice, practice.
                                                                                1. 1

                                                                                  Well written advice, To be honest I really hate ides I tend to focus on text editors such as sublime text or vim. Furthermore, I do use outside resources such as google or stack overflow when doing something completely new.

                                                                                1. 3

                                                                                  I strive to only consume technical knowledge that is useful for the projects I am currently working on.
                                                                                  Typically for each project I have a “meta” folder which stores various ideas and documents related to it.

                                                                                  For more high quality and general artefacts like books and scientific papers I have a separate personal folder. Within this folder, each paper/book/essay lives in a separate directory. There I place the source files (typically pdf) and a simple text file with a quick summary and my comments. This allows for an easy grep search. Also it frees me from various dependencies on external tools like orgmode, vimwiki, papis, mandeley, etc.

                                                                                  To better remember/refine important ideas that are combined and distilled from multiple sources - I describe them in a short article format on my website/blog. Doing this helps to: 1) enhance my understanding of the idea 2) share it with others 3) better remember it.

                                                                                  For everything else I trust my brain to do the filtering and remember the things it thinks I will need in the future.

                                                                                  1. 11

                                                                                    I appreciate the article but feel the porn analogy undermines its effectiveness.

                                                                                    1. 2

                                                                                      Unless I’m misreading the article, it’s just irrelevant for the most part, isn’t it? It (questionably) claims that most people immediately start hyper-configuring vim, and calls style or personal values “porn”, without quite explaining it, but not returning to it either.

                                                                                      Ultimately, it seems like more of an attention-grabber than a significant idea.

                                                                                      1. 1

                                                                                        There were a few people who seemed to take the article in a very different way, for example thinking that it advocates for the remaps shown in the second section. I got some “yeah! I too remap Y” comments.

                                                                                      2. 2

                                                                                        I don’t.

                                                                                      1. 2

                                                                                        Dark mode improves the distinction between colors, while light mode improves the distinction between edges.

                                                                                        For example, when looking at syntax-highlighted code from a distance dark mode seems advantageous since you can identify blocks of code with different colors more easily. But when you want to read or write the actual text the colors seem to “shine through” the black canvas and produce a blur effect.

                                                                                        1. 1

                                                                                          For an experience software engineer wanting to explore some data modelling and statistics, do you recommend Python or R? Why or why not?

                                                                                          1. 1

                                                                                            This is a good question, but I am not qualified to answer it - I have never used Python.
                                                                                            And I am afraid not many people will see it here on this thread.

                                                                                            Maybe you should post it as a separate question with [ask] tag?

                                                                                          1. 9

                                                                                            To me these questions look OK. There are not too many of them and users can always press “hide”. Also the potential for spamming them is quite low - how often can you really ask about productivity tools. A few times per year maybe at most.

                                                                                            In addition - where else do you have a chance to ask broad technical questions about things you care about and get input from a lot of people who know what they are talking about. Nowadays Stack Overflow doesn’t accept questions like that, google search will lead you to fluff content with ramped-up SEO, on Hacker News no one will reply to you, and on Reddit you will get the same answer posted 100 times from 100 different users, with some lone different opinions downvoted and deleted.

                                                                                            But at the same time I like how sensitive to the standard of quality some users are on this site. Questions about blog posts and similar vanity threads probably should be tuned down a little. I replied on that thread with a suggestion that if you care about other people’s blogs you can find them on their user profiles.

                                                                                            The content I would really be against are question of the type “My compiler just gave me this error message, maybe someone here knows how to fix it?”. Or “Hi, rust noob here, what is the best way to start doing X”. Or “Currently I am X years old, been a musician all my life, is it too late to transition into web dev?”.

                                                                                            These types of questions are a lot more damaging. One thing - different users can ask the same questions in different permutations again and again. “Hi I am noob in Rust, help me”, “Hi, I am noob in Java, help me”, etc. Based on my experience back when I was following various subreddits (r/vim r/rstats r/bioinformatics, etc) or even with Stack Overflow, once the questions like above start appearing you can kiss any quality goodbye.

                                                                                            1. 1

                                                                                              When I want to see more from a certain user I typically just take a look at their user profile, and a link to a blog post is often there. So if you are looking for blogs about certain topics - read the discussion of that topic on lobsters and then check the user profiles of the people who commented. And it goes the other way - if you want people on lobsters to find your website/blog, be sure to add a link to it in your profile

                                                                                              1. 2

                                                                                                multimarkdown best markdown processor I’ve used. Quite conservative yet has a lot more features than standard markdown. Takes data from stdin and returns to stdout. Blazing fast. Automatically deals with tables of contents, combining multiple markdown files into one, referencing titles, footnotes, image attributes, etc etc.

                                                                                                redo best dependency build tool I’ve used. Very intuitive and simple once you “unlearn” Make. Very modular, with distinct rules for different file types living in different scripts. Language agnostic.

                                                                                                But the most productive thing I did has to be putting a stop to constantly fiddling with various config files like .vimrc, bashrc, etc. and learning to love the defaults.

                                                                                                1. 5
                                                                                                  1. I change the contents without bumping the date, Ministry of Truth-style (this mostly just falls out of what Jekyll does, rather than a conscious choice). I don’t think the specs* says what software should do if this happens, but I assume it will either silently backdate the changes (which is what I truly want) or it will completely ignore it (which is a decent enough choice, too). Sometimes it might result in odd inconsistencies within an app where different pages show different versions of the text, but I’ve never seen one actually crash because of this, and since I’ve only been correcting typos, I don’t really care. If you’re making drastic changes to a post, you need to do something different.

                                                                                                  2. You definitely should limit the size of the feed. The Atom spec has support for “pagination,” but most readers don’t do anything with it, and most sites don’t use it. You should also try to make sure your web server includes an ETag or a Last-Modified date (if this is a static file, then it will, but if you have something like a PHP script generating it, then you’ll need to do it yourself). Automated software downloads this thing over and over again; you should do everything you can to save bandwidth.

                                                                                                  3. I’ve literally never tried to maintain an RSS feed manually. I’ve used everything from hand-crafted Perl to XSLT to Jekyll to WordPress to generate it. I never wanted to potentially forget to copy stuff from the HTML to the RSS feed.

                                                                                                  4. As is typical, your best bet is “do whatever WordPress does.” I’ve seen a feed reader crash in the presence of an unknown tag in a separate namespace (I was developing a feed to interoperate with identi.ca, and was testing the feed in other readers; turns out there was no way to produce a feed that simultaneously worked in DogCatcher while also having an @mention-able name in the OStatus-verse). When developing a web page, there are about four browsers that you really need to test with, and you’ve got 99.999999999% of the users covered. This is not the case with RSS; one of the great upsides of being a properly declarative format is that it gets used for a variety of purposes and applications, all of which have their own quirks and bugs.

                                                                                                  One thing I would do, if I were you, is try to make your RSS feed “event-centric” instead of “page-centric”. RSS is a replacement for mailing lists, not a replacement for web pages. If you’re building something like Wikipedia, then your RSS feed should probably be a changelog instead of just an index of your articles.

                                                                                                  * There is more than one spec. Sorry.

                                                                                                  1. 1

                                                                                                    Appreciate the response. But I do think I am getting conflicting advice, specially between you (make RSS event centric, limit the size) and @twee who says I should put entire content within the RSS element.

                                                                                                    Are these two different “schools of thought” or am I missing something still?

                                                                                                    1. 4

                                                                                                      Well, there are two fairly commonly used expansions of the RSS acronym - Really Simple Syndication and Rich Site Summary.

                                                                                                      Some sites obviously fit a particular definition more than others. As @notriddle says, trying to store the full text of each page of a wiki-type site is probably going to end in disaster. If you write articles, however, I would always recommend putting the full text in, as that’s generally what people who use RSS want; a syndication of the sites they are interested in. Now some people might just open each article in a browser window, but others want the uniformity that a single reader provides.

                                                                                                      I suppose they could be two different “schools of thought”, but I reckon that really they’re coming from the same perspective; as was also mentioned above, RSS is used for a variety of purposes and applications, and trying to appeal to as many of those purposes and applications as possible is maybe sensible.

                                                                                                      Disclaimer: I’m biased, and I get quite sad when a site I otherwise like doesn’t have a full text feed or is incriminating in another way.

                                                                                                      1. 1

                                                                                                        Are these two different “schools of thought” or am I missing something still?

                                                                                                        Not so much “schools of thought” as just two good things being in tension.

                                                                                                        Automated software is going to poll your feed, and whenever your feed changes, it’s going to re-download the whole thing. This means you should avoid making your feed gratuitously large (compression, minification, and caching should all be enabled, and since you probably don’t want to make everyone re-download your entire website every time you make a new post, you should also limit the number of posts in your feed). RSS readers also do a fairly poor job of surfacing updates, and while that’s what you actually want for typo-fixes, if you’re making a substantial change to an existing page then you probably want it to show up as a new item. Making a new item will not only surface the change better, but also give you a chance to summarize what changed instead of making me re-read the whole thing to try to figure it out.

                                                                                                        On the other hand, if someone’s in their feed reader consuming your content, they shouldn’t be forced to switch to their web browser. Your subscribers probably prefer their feed reader’s interface over your web page’s, and even if they don’t, nobody enjoys having to make a bunch of unnecessary clicks. I actually agree with @twee that you should include entire posts rather than just excerpts or headlines. The practice that they’re complaining about is one that news websites use to try to increase ad impressions, while I’m thinking of the sort of “adapting” that comes from using blog tooling for a website that isn’t really a blog.

                                                                                                        1. 1

                                                                                                          Yep, I agree with pretty much all of this. Well said.

                                                                                                          1. 1

                                                                                                            Thanks a lot for these answers. It is all coming together. I have last few (I think) remaining questions:

                                                                                                            1. Can I mix whole-article entries and event-style entries within the same feed? - i.e. put whole articles in the element when they appear, but only use event notifications to broadcast significant updates?

                                                                                                            2. I’ve read that atom elements have to have unique international IDs. And they seem to be linked to URLs. But in the case of event entries, how would someone generate those IDs?

                                                                                                            1. 3

                                                                                                              Can I mix whole-article entries and event-style entries within the same feed? - i.e. put whole articles in the element when they appear, but only use event notifications to broadcast significant updates?

                                                                                                              In principle, I think you could, and it might even be something that could be done well, but it might conflict with people’s expectations for a feed. My inclination might be to (if possible) treat these as two separate feeds if you want to offer both for the same set of content - one for new articles and one for deltas to articles.

                                                                                                              As an example, at SparkFun Electronics, we wound up offering several feeds for products:

                                                                                                              • One for new products populated as they were added to the storefront
                                                                                                              • One for changes to existing products
                                                                                                              • One for comments on a particular product

                                                                                                              This kind of thing lets subscribers pick and choose for their use-case and can serve different audiences. You might have one customer who wants to see all the shiny new stuff, another who’s keeping an eye out for price drops, a reseller who wants to know any time a product they carry changes, or an engineer who wants to track discussion about one of their designs. Similarly, you might have some readers who just want to see when you post something new and others who want the full firehose of all your edits.

                                                                                                              1. 2

                                                                                                                I’m usually a huge proponent of “put the whole text in the feed” but that’s mostly nice for blog posts that don’t get a ton of edits. If it’s more like “a notification that this page was updated” like it sounded to be your case, then I’d like a small feed better, and not the whole text. If I’m reading this from oldest to newest I’d get all the (outdated) versions first.

                                                                                                        1. 5

                                                                                                          I agree with much of the general consensus. You should be using Atom instead of RSS1 - it’s a more modern and cleaner specification.

                                                                                                          1. If your posts are eternally work in progress, you could probably perform small updates without changing anything, and use the updated field in Atom if something significant changes.
                                                                                                          2. That’s your choice. Most feed readers download all articles they receive, so it’s not a huge issue for people using hosted solutions or desktop programs; it only possibly becomes an issue if someone moves to a new computer and has to download everything again, although even then it’s likely that they’ll have read all posts older than the most recent 10 or 20. Depending on the client, adding more posts will just make the request take longer (as it’s a bigger file). Some people just prefer to keep everything accessible via RSS.
                                                                                                          3. Yes, in the same way it’s possible to maintain the rest of your website manually. You probably don’t want to, though.
                                                                                                          4. The main thing is please, put the full text in. Headlines are bad. Excerpts are worse. Just put the full text in, which is counter-intuitive to a lot of the desire to track people more, as RSS doesn’t help with that, but the people who use RSS will appreciate you so much more2 3. Another thing for me is sometimes people alter their feed, and it makes the feed reader go funny (for example, showing all the articles again). In this case, it’s polite just to put out another post (even just to the feed) to apologise for the inconvenience, just as you would if you accidentally sent a spam email to a bunch of people. Oh, and make it easy to find the feed - the best way to do this is with a link to the feed, but if you’re unwilling to do that (please do that), use a predictable URL; the ones I try are /feed, /rss, /atom, /feed.xml, /feed.rss, /feed.atom, /atom.xml, /rss.xml, /index.xml, usually by hand (a script would probably make my life easier). It’s so much nicer just to have the link there for me - if I like your site enough to go to that effort, appreciate me ;)
                                                                                                          1. 4

                                                                                                            a script would probably make my life easier

                                                                                                            The OP should include an alternate link in their HTML <head>. Good feed readers will allow you to just paste the link to the home page itself, and will auto-discover the feed URL from there.

                                                                                                            1. 1

                                                                                                              I totally forgot that this feature existed - that can definitely go as a best practice. Thanks!

                                                                                                              1. 1

                                                                                                                My preferred reader, rss2email, cannot, remarkably.

                                                                                                                1. 1

                                                                                                                  As @twee already said, this seems like a perfect opportunity for a really simple script.

                                                                                                                  https://github.com/notriddle/find-feed

                                                                                                                  1. 1

                                                                                                                    Yes indeed, thank you. My surprise that rss2email doesn’t already is precisely because it would be quick work to add it, and it’s pretty old software, so why nobody has yet is a mystery. I may yet do it myself.

                                                                                                              2. 1

                                                                                                                Thank you for elaborate response.

                                                                                                                Everything rings true to me except for the one part where you suggest putting whole text into the feed.

                                                                                                                Is this really the standard way of doing it? If RSS readers really wanted this functionality why wouldn’t they just download the whole article automatically from the link? Also I am not even very sure what format I would have to use, should I put everything as html, and what about pictures?

                                                                                                                1. 3

                                                                                                                  It really seems to depend on the site generator and the individual. Most of the feeds I am subscribed to (uploaded to http://ix.io/2hEU if you want to check individual feeds for yourself) have full text feeds. In my first post I linked to two blog posts by individuals explaining why they personally use full-text HTML.

                                                                                                                  The content tag in atom can have the attribute type="html", and then standard HTML can be used, and will be parsed by the feed reader just as if it were HTML in a browser, so your images will look fine. I think the existence of a content tag as well as a summary tag gives a good idea as to what the Atom standard is happy with, although I haven’t checked the standard recently so am not entirely sure on the specifics of this. RSS can also take HTML through the use of CDATA or something, but as I recommend Atom I can’t advise you anymore with regards to this.

                                                                                                                  Some feed readers do download the whole article automatically, if they think it isn’t already there (I use newsboat, which does this, and I know that quiterss does too). The problem is, a lot of websites have content on each page which isn’t the article (adverts, a header, archive listing), and some don’t even have the content in the HTML (javascript apps as websites). This makes it pretty difficult to get the content without a full browser engine, and what’s the point in that when it’s trivial to just stick the article content in the RSS feed?

                                                                                                                  My opinion (which ultimately is just my opinion) is this: people who like having the full text in the feed appreciate it being there. People who don’t use that functionality probably won’t mind the added weight of it being there, as it’s not a huge amount anyway (and I say this as someone who routinely uses veeery slow connections) and they’ll likely lose the gains the moment they open one of the articles as an actual website. As Aral says in his article, the more ways your content can be propagated, the more people will read it who want or need to read it. If you take pride in the content, having people read it is definitely more important than the amount of impressions you get on your site.

                                                                                                                  Just my 5 cents or so :)