1. 19

After the last post (https://lobste.rs/s/6yo4wi/totext_py_convert_url_rss_feed_text_with) I bought a domain and made a simple homepage for the HTTP version. All the fun starts when you visit the Gopher version.

I modified PyGopherD to sort directories in reverse order, since the articles are named iso8601 date, newest ones on top. Script and pygopherd code is here: https://github.com/RaymiiOrg/to-text.py

Looking for feed suggestions, preferably without paywal or cookiewalls to workaround. This is the current list:

  • theguardian.com
  • spiegel.de
  • feeds.reuters,com
  • arstechnica.com
  • espn.com
  • cnn.com
  • bbc.co.uk
  • tweakers.net
  • raymii.org
  • nos.nl
  • rd.nl
  • nu.nl
  • ad.nl
  • security.nl

Preferably world or local news but tech is welcome as well.

I’m doubting to add the lobstres rss feed, since I’m not sure if all content will parse correctly. For now I first check before adding, and if needed implement a workaround (looking at you twitter).

I’m submitting the HTTP version since the Gopher URI throws an error in lobsters. Here’s the floodgap proxy link:

Or native Gopher:

  • gopher://txtn.ws

  2. 3

    I am not sure how many others on this list do this, but if you’re not aware, CNN has a lite version, which is text only ish

    1. 1

      There doesn’t seem to be an RSS feed for this lite version?

      1. 1

        I haven’t looked, to be honest. It would be easier to crawl this version and create one, for sure though.

    2. 2


      this is the way i would like to consume media

      i have some questions about how to determine whether feeds are appropriate

      i also have a few suggestions

      i’ve attempted a bare minimum of categorization

      (please excuse the core dump as i segfaulted in my excitement)


      Currently Existing Sites:

      • Local News: several Dutch sites that I don’t know anything about and can’t read
      • World News: Reuters, CNN, the BBC, the Guardian
      • Tech News: Ars Technica, Lobste.rs
      • Sports News: ESPN

      (yes, yes, I know, Lobste.rs /really/ belongs in the World News category since it’s the only news of any importance! Although those articles about “Firefox is the most moral browser” are really properly sports news, as everybody knows. Hundreds of upvotes. Opiate of the masses, I say. What are we supposed to say, “Safari is the second most moral browser???”) /s


      A Few Prefatory Notes Regarding Media Consumption

      I find that we are living in a time of unmitigated bias across most domains of human experience, which bias does not express itself cleanly and objectively between cultures, leading to what I would consider a “race condition” between societies that have gone through what I would consider “modern cosmopolitan liberal capitalism” (possessing, besides a thin patina of objectivity, a /desire/ for /more/ objectivity) on the one hand, and, on the other hand, a seemingly heterogeneous mix of societies that reject the very notions of tolerance and democracy as being too rigid and destructive to culturally refactor themselves along the lines of these principles (like, sorry, Hungary).

      But are there, as one public official once stated, “Good people… on both sides…”? Unfortunately the rather quaint stupidity of these “intolerant” (non-modern, non-cosmopolitan, non-liberal, non-capitalist) societies is further compounded by the far greater stupidity, really the interpretative ignorance, of political commentators who espouse universal values at the expense of the same people to whom the commentators are trying to espouse said values.

      Obviously the solution is burning books, which is why we need to spread as much knowledge around as possible to prevent the total destruction of history, prior to the inevitable event of war. That means reading the rich people’s newspapers of record, and reading the polemical newspapers, and reading the propaganda papers, and reading the liberal papers.

      Now part of a complete breakfast!


      Regarding “local” news: How local, Raymii? I, for one, would read some Russian news sites. Is that “local” or something else? Wouldn’t /everybody’s/ local be quite a lot of news? :P

      Still on the same topic: I would include additional links related to the intersection-of-arts-and-technology avante-garde type material local to San Francisco Bay Area if at least one other person is interested.


      Here is a legend for how to read the below.



      Name of Thing or Allusion to Thing’s Contents (“tagline”) (pointless editorializing)



      Foreign Policy

      Le Monde

      Foreign Affairs

      Rich People’s Newspapers of Record

      The BBC (already in there)

      The New York TImes (behind a paywall)

      The Economist (also behind a paywall)

      “Dissent” Papers

      Al Jazeera

      Public Radio International

      Democracy Now

      Project Syndicate

      Mountain Runner


      The USA.gov Site’s Literal RSS (“Government news and featured content from the U.S. government’s official website, USA.gov”)

      Unapologetic United States Propaganda

      Voice of America News

      Biased American Media Sources

      American Liberal Media (“all the news that’s fit to print that puts the DNC in a good light after editorial sanitization especially if it’s negative and relatively-morally-but-not-overtly-sexually scandalizing”)

      American Radical Conservative Media (“less than half the news that’s disproportionately relevant these days”)

      The American Presidential Twitter (USE TSCRAPE: https://codemadness.org/git/tscrape/file/README.html)

      One Lawful Good Political Commentator Whose Accuracy and Rapidity Underwrites His Clear Bias (USE TSCRAPE)

      Things That Don’t Belong Elsewhere (And Are American)


      Tom Dispatch


      The Hill

      Politico (which idiotically won’t give us all their links, and forces us to use sections)

      The Daily Caller (rather trashy in addition to being an ad-filled, biased media source whose homepage proudly says, in an American-flag-lookin’ graphic, “FIGHT BACK AGAINST / THE BIASED AMERICAN MEDIA / JOIN THE DAILY CALLER PATRIOTS TODAY / AD-FREE READING.” Who falls for this kind of crap? But we need them, because they’re both hypocrites and sadly not wrong. Confer: prefatory notes.)


      These are the kinds of magazines that will keep you on top of current trends in arts and letters. Since this audience is primarily engineers, I should note explicitly that these concerns are reflected in this community, relative to the orange site, in a greater depth of discussions of superior quality on topical issues. Might not be reversing writeups, but still pertains to being a human.

      N+1 (their RSS is broken; I’ve contacted them; they haven’t fixed it; they’re still important so…)

      The Nation

      The Baffler

      The New Republic (worth noting the recent scandal. So, this 100+ year old magazine that has been ever so fancy for ever so long had within it some old-school types, not exactly the CEO of Yahoo. Of course, media generally is in decline, in the sense of the world old-school types know being replaced by AOL literal decades ago. So they’re like, maybe we can figure out how to use the modern version of the BBS, and the highest levels announced a restructuring under some Yahoo-type of media mid-level managerial man, not exactly what the peons and scriveners were hoping for, and half the workforce just up and left, led in an exodus, and the literary community swooned, and Leon Wieseltier famously wrote unhinged rants about science being evil, and Stephen Pinker obfuscated his way to popular celebrity once again in contradiction, inanely restating time and again that “science and scientism are good,” similar to how he felt the need to state that he knew how the mind works in How The Mind Works, and at this time Leon Wieseltier was the staunch old guard of the “we shall do an Exodus from this techie evil”, but then of course his celebrity was punctured by #MeToo allegations and the whole thing screeched to a grinding halt instantly, celebrity deflated, and it turned out the Yahoo-type media mid-level managerial man wasn’t so bad and the New Republic did, in fact, continue to turn out worthy material)

      The New Inquiry


      The New Yorker

      • has no feed

      Los Angeles Review of Books (this is the most “it” of book reviews in publication, and the funniest)

      New York Review of Books

      London Review of Books

      Public Books


      Arts & Letters Daily (“philosophy, aesthetics, literature, language, ideas, criticism, culture, history, music, art, trends, breakthroughs, disputes, gossip”)

      Edge.org (“To arrive at the edge of the world’s knowledge, seek out the most complex and sophisticated minds, put them in a room together, and have them ask each other the questions they are asking themselves.”)

      Dissent Magazine

      Jacobin Magazine

      The Browser


      The Federalist

      Quillette Magazine (not necessarily to be trusted, but perspicacious – while punditry – on the topics you could expect a so-called “IDW” magazine to represent “honestly”)


      The Open Society


      Orange (“Links for the intellectually curious, ranked by readers.”) (If only the readers lived up to the publisher’s expectations!!!)

      N-Gate (“We can’t both be right.”)

      Niche Technical Forums

      A Science News Site

      Gwern (Just open the link for the humorous complaint about how he gives up on RSS. Nelson from the Simpsons would laugh.)


      LEGAL NEWS For those who want to stay up to date on what’s happening to the man in the office formerly known as the leader of the free world.

      Empty Wheel

      Here is some commentary on the issue of news: https://news.ycombinator.com/item?id=18648127

      Question: What is up with this website? Does anyone actually use this service? Are they legit? It looks good. but attractiveness doesn’t correlate with news quality in my experience. https://thecorrespondent.com/

      1. 1

        I’m in the process of checking the feeds you listed. About ten have no published date so I have to code around that. Some news like Al Jezra I added already, the rest I’m working on (parsing issues). Thanks for the huge list and your insight!

        1. 1

          Any progress?

      2. 2

        This is great! I like the way you basically have the URL of the site and the gopher displays it. It would be awesome to have a version of this for medium so one could read medium articles without any distraction.

        There is already a lobsters gopher gopher://sdf.org:70/1/users/julienxx/Lobste.rs/

        1. 2


          Thanks for that link, that is way cool. Added it to the gophermap for links just now.

          For medium.com, the script that I wrote and use for this seems to work. Here’s a picture: https://i.imgur.com/aqufUUF.png for the output of this article: https://medium.com/@way/rage-against-the-codebase-programmers-and-negativity-d7d6b968e5f3. So you could run that in your terminal and get just the text. I’m not sure if Medium has an RSS feed for everything on their site.

        2. 1

          Cool project. This seemed off:

          “This is a Gopher only site. Please use a gopher client to view this content:” “Floodgap has a public Gopher <> HTTP proxy:”

          A little inconsistent given you don’t need a Gopher client: just a web browser with a proxy. Anyone wanting to experience it authentically can use the Lynx browser in their terminal. It supports Gopher links.

          1. 1

            The two big Gopher extensions for chrome and Firefox just redirect to the floodgap proxy (due to extensions being limited since webextensions and chrome), so why not provide a direct link. Plus, not everybody is able to install lynx or extensions.

          2. 1

            This is great, thanks! I was going to suggest that the import of Lobsters automatically link to gopher versions of the articles if they exist and it’s already there!

            Except the link to this post results in “Parsing failed” :)

            1. 1
              1. 2

                Floodgap’s web proxy uses Windows-1252 encoding by default for, what I presume to be, compatibility reasons. Most clients default to UTF-8, however.

                1. 2

                  In my gopher clients it seems to work as intended: https://i.postimg.cc/dVq5pDVj/Screenshot-20190421-160850-Pocket-Gopher.png and https://i.postimg.cc/C1jDwHvB/Screenshot-20190421-161321-Diggie-Dog.png Lynx on the client as well. Have you tried a Gopher client?

                  The browser expects an encoding and I’m not sure how the floodgap proxy handles that for plain text files that don’t specify one.

                  1. 1

                    Interesting, so gopher clients assume all content is UTF-8 (or whatever encoding you’ve used here)?

                    1. 2

                      I guess so? The python script explicitly does UTF-8 and the clients I test with seem to as well. For the filenames I do strip out all except a-zA-Z.

                      1. 1

                        Why not allow numbers? Right now there are names like Queen_Attends_Easter_Service_on___rd_Birthday.

                        1. 1

                          No specific reason. I changed the regex so numbers are allowed now.

                      2. 1

                        The one I wrote assume UTF-8 as I found more pages using that than ISO-8859-1 or Windows-1252. It’s also pretty clear that UTF-8 is the way forward for encoding.

                        1. 1

                          Hey you’re behind the Boston diaries, but rumor goes, not even from, in or near Boston! I like reading your site and Gopherhole

                          1. 1

                            Thanks. There is a story behind the name.

                            1. 1

                              I’m amused to see a number of people I’ve started following on gopher I’ve already seen on Lobste.rs or Mastodon and didn’t know it.