1. 10

    Someone tell this guy about Emacs’ -nw option, at least he’ll get a uniform interface when trying to do this.

    Sometimes i think about doing this too, just to avoid distractions, but personally I just use too many PDFs that I don’t want to print out all the time, with too much mathematical notation to properly extract the text.

    w3m supports inline images (via installing the w3m-img package)—seriously, a web browser with image support, inside the terminal. The future is now.

    So was he just using terminal emulators under X, or a “proper” tty? Because the screenshots look like terminal emulators.

    1. 2

      I believe w3m supports images in the linux framebuffer, without X11. There are also some hardware terminals with escape sequences for drawing graphics

    1. 1

      I’m on zsh:

      # 1 ptman@pumba:~ (git)-[master**]-
      1013 %                                               # 2019-06-18 12:12:40 INS
      
      • 1 is exit code of previous command
      • 1013 is history line number
      • INS is vi mode

      maybe I should move timestamp to previous line, but I think RPROMPT didn’t directly support that

      1. 1

        I hope this could serve as a starting point for a more lightweight Synapse implementation, for small user-counts.

        1. 3

          While I certainly wouldn’t call synapse lightweight, I wouldn’t say that it’s prohibitively heavy either.

          I have it running on a $10/mo VPS (2GBs of RAM and 1vcpu, SSD boot disk). The synapse process generally sticks around 200MBs of active memory for my usage (2 users, and not sitting in any rooms that have more than about ~1000 people in them. I have joined some of the large rooms briefly, and they don’t seem to give my server too much trouble). The nginx and postgresql ram and cpu utilization are negligible.

          I use the VPS for other stuff so I plan to keep it the same size, but if I was running only synapse, I’d wager it would be fine on a smaller instance, provided you stayed out of large rooms such as matrixhq and synapse-admins. If you were strictly talking in local rooms to a small number of your friends, I bet a raspberry pi could handle it.

          1. 1

            You’ right, lightweight wasn’t the right tern. I was thinking more of something that along the lines that doesn’t multiple components such as a DBMS, that all have to be configured and maintained. You know, something you can just install, run a configuration script then inform your init system about it, and be done with it. After all, lowering the barrier of setting such a system up, promotes federation.

            1. 1

              Ah, yeah, I see what you’re saying. There is definitely some assembly required. I suppose if you went with sqlite, and did away with the webserver in front of synapse, you could lower the barrier of entry a bit. Though, I don’t know how much performance would suffer there.

              Debian does package synapse (https://packages.debian.org/stretch-backports/matrix-synapse), so that may help a bit in terms of letting the init system know, etc.

              1. 1

                synapse with sqlite is abysmally slow

                even on a home server with just one user there will be concurrent db access all the time while federating

        1. 0

          Have they fixed the protocol yet, in particular to trusting poisoned graphs?

          1. 3

            grins wryly that an article which is all about “hey look we fixed the protocol and released a 1.0” is met with questions about whether we’ve fixed the protocol

            1. 2

              I’m not sure what you mean by that, but matrix 1.0 and synapse 1.0 include room version 4, which includes the new state resolution algorithm, which is probably the fix to what you refer to.

              https://matrix.org/docs/spec/#complete-list-of-room-versions

            1. 17

              I am thrilled to be one of the newly announced Guardians of the non-profit foundation. I’m happy to answer any questions people may have, though bearing in mind that I’m going to be a little distracted for the next while until my son goes to bed.

              1. 5

                How were you and the other guardians for the non-profit foundation chosen? Also, what do you expect your day to day work with the foundation to be like? (I mean in this in the most prosaic sense - is there a physical office you go to, or is the work something you will mostly do over email and at occasional conferences? Is this a full-time job that they’re paying you for, or do you do something else with your time as well? These sorts of details are something I’ve wondered about for many non-profit advisory councils associated with open-source software projects, and since you’re here offering to answer questions I thought this would be a good time to do so :) )

                1. 4

                  We were chosen by Matthew and Amandine who were, by virtue of starting the foundation, the only Guardians that existed before.

                  The Guardian position is definitely not a full time job, and it is not paid. We are spread out across multiple continents, so our communications will take place mostly virtually (over Matrix, naturally) though we may get together physically from time to time. I’m sure New Vector would be happy to loan me a desk to work out if I’m in London, but there is no physical office.

                  1. 4

                    The process we went through in selecting the Guardians was to ask folks who:

                    • Are clearly philosophically aligned with the goals of the project (i.e. radical decentralisation and liberation of communication)
                    • Are widely recognised as independent experts, trusted by the community to keep the project honest
                    • Are independent of commercial factions in Matrix
                    • Ideally use Matrix already, and represent some subset of the community (e.g. Ross on the ‘personal homeserver & legal’ side, Jutta on the ‘corporate homeserver’ side, Jon on the academic side.).
                    • Have experience and understanding of the responsibilities and requirements of being non-exec directors of a non-profit

                    This narrows it down quite a lot, and we thought very carefully about who to invite to join - and happy to say that all our first choices accepted :)

                2. 4

                  Congrats on the release!

                  What is the official position on bridges?

                  Take sms bridge. If done right, can completely replace sms software for Android, that would be a huge win for freedom.

                  1. 4

                    I don’t know that the foundation has an official position on bridges, but if we did it would probably be something like \o/.

                    The more the merrier!

                    1. 2

                      What do you mean by position? There exist a couple of sms bridges

                      1. 2

                        If resources of the foundation (grant money or something) will be devoted to it.

                        1. 4

                          Currently the Foundation has very little financial resources, beyond a stack of t-shirts and the monthly donations arriving via Patreon & Liberapay. However, New Vector has one person working fulltime on bridges, plus a GSOC student and some support from the rest of the team. The main priority is on IRC, Slack and XMPP, but we try to help other bridge development as best we can too.

                          1. 1

                            That’s the answer I was looking for, thanks.

                      2. 1

                        You may be interested in jmp.chat – if you want to use it from Matrix IIRC there is work happening on a good XMPP bridge so it should be possible.

                        1. 2

                          jmp.chat

                          Wow, interesting, thanks.

                    1. 1

                      Raspberry Pis have 100Mbit networking.

                      If you need a decent CPU as well I’d get an Odroid XU4 (those even have gigabit iirc).

                      1. 4

                        I read the question as “I want my 100mbps ethernet link to be the bottleneck instead of than the CPU running the VPN software being the bottleneck”.

                        So, most or all models of raspberry pi don’t check the box, right?

                        1. 3

                          Anecdotally, I pull 100mbps through wireguard using a Raspberry Pi 3 without issue. As long as you aren’t relying on it for complicated firewall rules you should be fine.

                          1. 2

                            That’s good to know. I have an old raspberry pi at the moment that can’t crack 20mbps through WireGuard. Hence my question :)

                            1. 2

                              Older Raspberry Pis are really slow. Anything that’s not a Pi 3 (or better) won’t work (as you’ve found).

                              1. 1

                                Pi 3 is not very good either (other than the terrible I/O: it doesn’t even support AES instructions!). Try the ROCK64.

                                1. 1

                                  I guess wireguard doesn’t use AES

                                  1. 1

                                    According to this article/post strongly but tangentially related to our topic here, no, wireguard does not use AES.

                              2. 1

                                Wow. Thank you for this data point.

                          1. 6
                            • MUD server
                            • single-binary git repository server
                            • IRC bouncer with a decent web / mobile client

                            I’ve probably started all three of those (the latter two most often) at least a dozen times each, getting to various stages of completion before I decide to do something different. One day I’ll have something that’s usable…

                            1. 2

                              MUD server

                              I was about to say I think you may have just shown you age, and then I realized how incredibly presumptuous that was :) But it does seem like the idea of a MUD per se has gone the way of the dodo - now all the younger folks want to rewrite World of Warcraft :)

                              1. 4

                                Not as much as you’d think. Particularly among the indie game development community, people get into text-based games with interactive fiction (twine, inform) and then try out MUDs and MUSHs.

                                1. 1

                                  Really? That is so cool! I thought the whole MUD idea was dead dead dead. I’m a dino from wayback :)

                                  Where are these folks publishing their work?

                                2. 2

                                  Maybe; the interesting thing is that while I don’t play any MUDs anymore, I’m still fairly active in the community for one of them. The MUD server idea is probably my least fleshed out hobby project.

                                  For me, it’s not really even about making one that’s useful but more about using it as a testing ground for different ideas. Procedural generation, proxy implementations, CQRS, event sourcing, entity component systems, natural language parsing. There’s a lot of opportunities to apply interesting tech.

                                  1. 1

                                    I’ve always found that to be true myself. I suppose what’s changed is that there used to be a huge mud USER community, and now they’re a springboard for creativity for developers.

                                3. 1

                                  I desire an IRC bouncer with a good mobile client very much.

                                  1. 2

                                    Indeed. What I’ve floated around with lately is an IRC bouncer written in Go. Plugins via webassembly so they can be added/removed without recompiling. A decent web UI to configure it (maybe), plus a built-in web client and then mobile clients.

                                    1. 1

                                      That sounds pretty neat. You should be able to do it all in Go using the built-in plugin system if you want to stick with one language and can run on Linux or macOS.

                                      1. 1

                                        My understanding is that the Go plugin implementation leaves a lot to be desired, but I can’t seem to find the articles I read about it in the first place. Granted, that was from awhile ago when the support was first launched.

                                        Based on a cursory glance it doesn’t seem like pkg/plugin is very widely used.

                                      2. 1

                                        Plugins via webassembly so they can be added/removed without recompiling

                                        Which Go package would you have in mind to presumably interpret/run the wasm blobs, or would you write one for that project, or depend on some external interpreter/runner?

                                        1. 2

                                          This one seems pretty well-done, but I haven’t actually played around with it yet: https://github.com/perlin-network/life

                                          Another option may be to use the plugin framework built by hashicorp which uses protobufs rpc I believe, but I haven’t looked into it much.

                                      3. 2

                                        I’ve started using matrix.org/riot.im and irc bridges.

                                    1. 2

                                      I wonder why they chose IRC over some other open and federating protocol, like XMPP or Matrix.

                                      1. 6

                                        Probably because they already had IRC servers. Darenet has been around for years.

                                        1. 1

                                          IRC is a bit over 30 years old, relatively stable, very simple protocol with lots of support.

                                          1. 8

                                            They could have responsibly disclosed instead of being an asshat, stealing information and posting a ton of github issues from a fresh account.

                                            1. 3

                                              stealing… information?

                                              1. 2

                                                I’m both supportive of and we participate in the responsible disclosure process for Xen, even those times we don’t make the cut for pre-disclosure. I’m sad someone would go to the effort they have here in a criminal manner when there is more [market] demand for the skillset on display here than I have ever seen before.

                                              2. 7

                                                Why the hell did github allow people to remove issues? This is annoying.

                                                1. 4

                                                  It appears the issues were removed by GitHub when a third party reported the user that posted the issues.

                                                  1. 2

                                                    Unfortunate that GitHub was powerless to prevent nuking their account after being reported.

                                                2. 4

                                                  I was telling a coworker about this and similar writeups and it turns out he wasn’t aware of the Hacking Team writeup from 2016. It’s detailled and very interesting. I would advise anyone to read it: https://pastebin.com/0SNSvyjJ .

                                                  1. 1

                                                    A 0day in an embedded device seemed like the easiest option, and after two weeks of work reverse engineering, I got a remote root exploit.

                                                    thanks a lot, the whole walkthrough is quite amazing and insighful with a wide variety of tools used

                                                  2. 3

                                                    Did you get a copy of them? They’re deleted now :(

                                                    1. 10

                                                      They’ve been reposted here: https://github.com/matrix-org/matrix.org/issues/371 (and this site has been archived here)

                                                      1. 2

                                                        Thanks!

                                                      2. 1

                                                        I think web archive has some of them. Maybe not every comments.

                                                      3. 1

                                                        Concerning #358, what is “Flywheel” in this context?

                                                        Side-note: I hate locked threads on free software projects.

                                                        Update: I think it’s a hostname of one of their machines?

                                                        1. 1

                                                          Seems like it’s the hostname of their jenkins build slave

                                                          1. 2

                                                            yup, it was the hostname of the jenkins build slave.

                                                          2. 1
                                                        1. 2

                                                          There’s https://www.opencvs.org/ as well, but luckily rsync is a smaller (I think) project to replicate

                                                          1. 4

                                                            Not to mention that Andrew Tridgell’s PhD thesis is a nice and short read.

                                                            1. 1

                                                              ‘Roughly 100 pages’ is short ?

                                                              1. 1

                                                                :) Well, the core part of the algorithm is short - see page 49 onwards.

                                                          1. 3

                                                            I would have gone for Ethernet, personally. I’ve definitely had to do SLIP before, on a laptop where I didn’t have a PCMCIA ethernet card; I set up an NT 4 VM to do it. Once I got the Ethernet card, I decided to use ZModem to transfer the drivers for it over.

                                                            1. 3

                                                              I picked up an old ethernet card that was new in box, but it expected Windows for Workgroups. This machine only had the standard home edition of 3.1. I never was able to get the card running, which is unfortunate given the slow speeds of the serial line.

                                                              1. 1

                                                                Now that you do have Internet, you could transfer windows 3.11 to the machine.

                                                            1. 1

                                                              So, to ask the question implied in the blog post, why do we need IPV6? I mean, the End-to-End principle is not worth the billions we are spending on it right?

                                                              1. 19

                                                                Because the blog post is wrong about IPv6 being unneeded. IPv4 is not sustainable for the public internet. If you’re American, you probably have no concept of this. But less than 45% if IPv4 addresses belong to organizations outside the US.

                                                                Furthermore, NAT subnetting prevents a ton of useful technologies. Unless your mobile carrier blows, your phone is perfectly capable of operating continuous connections as you walk around. That’s entirely because your phone can have the same IPv6 address as long as you’re on your carrier’s network. If things were using NAT, you’d probably get a new IP address every time you changed towers. Which happens a lot.

                                                                IPv4 doesn’t work for this because NAT isn’t infinitely scalable. A NAT gateway can only open as many connections as it has ports, and there are only ~60k usable ports. With web browsers opening 6 connections per domain, phones persisting a connection to their respective push notification services, and other apps using data, it’s safe to assume 10 connections will be used per active user, meaning about 6000 active users per IPv4 address. A lot fewer if those users are accessing more than one website at once.

                                                                So you’d need to load balance active users across IPv4 addresses. And synchronize NAT state across routers on your network to allow in-network tower roaming, i.e. solve distributed consensus at the router level.

                                                                Or use IPv6.

                                                                Now armed with knowledge about the limitations of IPv4, I think you can understand why many carriers choose IPv6, even in the US where IPv4 addresses are relatively abundant.

                                                                1. 1

                                                                  It might still make sense to change the IPv6 address when changing towers, since tracking which IPv6 address lives at which tower is needed to route properly, and it can be costly.

                                                                  1. 2

                                                                    That’s something that’s ordinarily handled at the switching layer, and therefore pretty cheap, compared to stateful per-connection TCP/UDP packet mangling (NAT).

                                                                2. 5

                                                                  There are a lot of reasons why the end-to-end principle is really important - often those reasons are understated. When you don’t have end-to-end reachability, you often have to rely on middlemen servers to proxy your connections or to help “punch holes” (which is often tricky in itself, or in some networks, impossible).

                                                                  Take for example a Voice-over-IP call (but in reality any peer-to-peer application is exactly the same). You use some SIP server, or similar, to signal the call and to set it up. After that point, you really just want to send voice traffic directly to the remote phone, not via some gateway which could add significant latency or may be congested or badly configured or evil/malicious or any other number of things.

                                                                  With NATs things get complicated because it’s entirely possible that both phones are behind NATs and neither of the phones are aware of that fact, so then the signalling server needs to make allowances for this (additional complexity) and clients have to implement hole-punching functionality (which may or may not work). Even worse is when you might end up behind more than one NAT, as is becoming increasingly common-place with Carrier-Grade NAT (CGNAT) as a method of coping with IPv4 exhaustion.

                                                                  If everything on the internet is end-to-end reachable, which is really how the Internet was designed, the question of how to “punch holes” through NATs becomes irrelevant because you can know reliably that the address that you have for a device actually is a one-to-one mapping to that device. The full range of TCP, UDP etc ports are available for use between any two points, your applications don’t need to implement awful mechanisms for dealing with NATs, you can be fairly happy that your traffic won’t be mangled by some router on-path and it makes real peer-to-peer applications much easier to develop without depending on third-party infrastructure.

                                                                  IPv6 makes it substantially easier to give every device in the world end-to-end reachability (and the various associated superpowers) because there are simply more addresses available and therefore address space exhaustion is not a problem like it is with IPv4.

                                                                  1. 4

                                                                    I mean, the End-to-End principle is not worth the billions we are spending on it right?

                                                                    If you think about it, the E2E principle has wide implication in our lives beside the business perspective. It allows every country or citizen to be part of a larger network without having to adapt to proprietary or cultural specific implementations or conventions. Sure it costs a lot, but is the cost the real issue with it?

                                                                  1. 1

                                                                    How does it handle spam in comments?

                                                                    1. 2

                                                                      Uses Akismet, which has a huge database of spam/not spam and is quite good at detecting spam. There are also fairly extensive moderation tools, so you can also do stuff like manually approve each comment before it appears publicly (you can choose to receive email notifications whenever a comment is pending moderation).

                                                                    1. 3

                                                                      there’s also pinfo that I’ve used with great success for years

                                                                      1. 2

                                                                        Yeah, I’m aware of pinfo, but it still suffers from some of the same issues as regular info. Basically, I just want my pager of choice (in my case, less(1), because that’s what I’m used to, but any pager will work).

                                                                        There’s also the issue that I can’t select text from it for some reason. Perhaps there’s a fix, but I couldn’t be bothered to find out. Instead I attempted to replace info with a small shell script, figuring that would take the same amount of time as figuring out how to get pinfo to behave (eventually I spent more time on it since it turned out info pages spread out over multiple files was too tricky to support with a shell pipeline, hence the Go program).

                                                                      1. 4

                                                                        I’ve always had mixed feelings about storing plaintext passwords on my personal computer. I suppose that there is nothing inherintly wrong with this, but something gives me the willies about opening a file and seeing my password in all it’s glory.

                                                                        1. 1

                                                                          What do you think about storing encoded, but not encrypted? Like docker stores registry passwords base64 encoded in .docker/config.json?

                                                                          1. 4

                                                                            It’s the same.

                                                                            The issue with storing password in plain text is that all the programs that you run can potentially grab and exfiltrate those password. It doesn’t matter if they are obfuscated or not.

                                                                            • any shell script or build system you git clone from github
                                                                            • most language-specific package managers like npm install run arbitrary code
                                                                            • any website that you visit if your browser suffers from file traversal attacks
                                                                            • if the HDD gets stolen and there is no full-disk encryption

                                                                            It’s better to have the passwords encrypted at rest inside of a key store. Or even better have the decrypt key stored on another device like a YubiKey.

                                                                            1. 1

                                                                              I’m going to 90% agree with @zimbatm on this one. For my use case, I definitely don’t get any additional benefit from it being encoded vs plaintext. That being said, I believe this stems from the fact that I generate my passwords randomly with a password manager. This means that I’m not using words/phrases from my native language. The situation where this would be the most helpful though is when:

                                                                              1. You are using passwords (or especially passphrases) that are composed primarily of native language phrases
                                                                              2. Your threat model is focused more on shoulder surfing than malware

                                                                              In that case, encoding does actually make a difference for the user, because now they can open that file, and the likelihood that someone shoulder surfing can memorize their password is dramatically decreased. I believe a non-trival number of people meet this criteria (even if they don’t actively know they fall into the second category).

                                                                              tl;dr: Don’t make people save their passwords in unencrypted files on the disk, but if you’re going to, add an option to specify an encoded (base64) password.

                                                                          1. 3

                                                                            Excellent! So CTEs are no longer an optimisation fence? No reason to avoid CTEs for performance reasons.

                                                                            Assuming this is coming to Pg12?

                                                                            1. 2

                                                                              They seem to inline CTEs as Sub-SELECTs if possible, and then optimize these with the outer query as before. The updated documentation has more info: select.sgml diff (I don’t know how to link to a specific line number in the diff…)

                                                                            1. 9

                                                                              I was pleasantly surprised by the Hetzner cloud (I’ve had terrible experience with their hardware offerings). It’s cheap, stable and it works.

                                                                              I stil have some VPS at strato.de - they’re not such a good bang for the buck but they’ve been very stable for me (multiple VPS, 10 years).

                                                                              There’s also tilaa (I haven’t tried them but heard some good things) and scaleway - not sure where the company is really based.

                                                                              1. 3

                                                                                Oh nice Hetzner looks good, + points for being German!
                                                                                Prices for their small vps’s are comparible to scaleways.
                                                                                How was customer support?

                                                                                1. 2

                                                                                  Can’t really comment, haven’t used any support iirc.

                                                                                  Had a problem once with one of those cloud VMs where it wouldn’t boot, but I could get the rescue system up and running and so I could do a quick check if I needed to back up something (was purely a test instance, so I didnt set up backups). But if you accept cloud instances as somewhat ephemeral, I don’t see that as a problem. This has happened to me on AWS and GCP as well.

                                                                                  1. 1

                                                                                    Looks like they have the option of hosting it in Finland. Anyone know how well it works to run openvpn from a VPS given that it is significantly cheaper than a VPN? I used to pay for a VPN to get Finnish TV from abroad but the IP would get blocked from time-to-time. Are cloud providers more immune to that? Or do they detect you in other ways such as via DNS?

                                                                                    1. 1

                                                                                      If anything the Hetzner Finland IPs are shown to be in Germany, since the blocks are owned by Hetzner. But it’s cheap to test, minimum pay isn’t a whole month.

                                                                                1. 10

                                                                                  Ugh, scattering crap across .local and .cache and .hootenanny is possibly worse. Now I have to go digging around in several places to find what’s wrong.

                                                                                  1. 17

                                                                                    I’m actually kind of a fan of .cache because in theory I can delete the entire thing at any time, and avoid backing it up.

                                                                                    1. 5

                                                                                      oh, you mean like /bin, /usr/bin, /usr/local/bin and /opt/… ?

                                                                                      1. 5

                                                                                        I’ve had fewer problems with binaries corrupting themselves.

                                                                                    1. 8

                                                                                      The arch linux wiki has a nice page documenting how to fix various software: https://wiki.archlinux.org/index.php/XDG_Base_Directory

                                                                                      1. 5
                                                                                        • Accepting total revocation/hard deletion of customer data
                                                                                        • (Hint from company lawyer:) do not provide free text fields in admin interfaces, to avoid having “stuff that mustn’t leak”, or irrelevant-for-the-purpose, laying around
                                                                                        1. 5

                                                                                          (Hint from company lawyer:) do not provide free text fields in admin interfaces, to avoid having “stuff that mustn’t leak”, or irrelevant-for-the-purpose, laying around

                                                                                          We would have liked to provide a freeform notes or description field for user accounts, but decided against it because of this.