1. 3
    • Watching the International 9 (TI9) finals, the most important Dota 2 tournament of the year.
    • Working on my automation system for projects, tasks and window management in Lisp.
    1. 1

      Working on my automation system for projects, tasks and window management in Lisp.

      Which Lisp? What kinds of automation systems? I’m (very) slowly growing a set of tools to enable a workflow loosely based around the Getting Things Done system, perhaps we can collaborate?

      1. 1

        Common Lisp/SBCL, not really trying to make it portable as for now it is a very opinionated personal project.

        For now it’s mostly about automating all the ancillary setup I have to do manually every time I boot the computer or change current project. Things like moving windows around to where I like them, setting up multiple windows and tabs in Kitty, starting several daemons/services/compilers in a watch loop using watchexec. I want to be able to write a file per project that defines all of these and to be able to say “this is my current project” and all of that is automatically set. It’s mostly a tight integration between window management, terminals and jobs.

        It seems yours is more about the actual task management for projects, completing them and such? That would be very nice too to have it integrated in my system. Would love to get an explanation on where you are right now, what objectives you have with it and how could it be integrated! Let’s collaborate!

    1. 3

      Working on reimplementing my file-browsing tool in Common Lisp (working on a generic “narrowing” module at the moment), with breaks to watch games of The International 9.

      1. 1

        The International 9

        Yeah! Doing that also, so excited for the finals tomorrow. Who are you rooting for?

        1. 1

          OG! A bit uninteresting of a pick, as they’re not an underdog anymore, but they seem to take the game very light-heartedly and keep the fun in it - contrasting the very militant attitude I see in a lot of other teams. Things like Ana’s position 1 Io pick vs NiP are really enjoyable for me to watch and keep me spectating Dota even though I haven’t played in months.

          1. 1

            Nah man, I understand, same here. They have a very nice approach to the game, they are also very fun to watch.

            Really looking forward to today games. Woke up at 4am. Hoping to get a TL vs OG finals :D

      1. 3

        @WORK: Continuing to develop the map/quest editor for our game

        @HOME:

        • Trying to build the map/quest editor in Clojurescript to see if it is a better fit
        • Learning Japanese
        • Learning to Draw
        • Yoga
        1. 2

          Rewriting a React/Typescript app to Re-frame/Clojurescript and having a blast. It’s the map/quest editor for work, although this change is more of a personal weekend challenge, if the challenge goes well maybe I’ll actually switch to this stack as I am finding it a lot faster and it matches the way I think.

          Also have a birthday party tonight.

          1. 6

            I’m trying to distance myself a bit from programming in my free time. I’m already doing a lot of exhausting programming at work building the Map/Quest editor for our game and I don’t want to end up burnt out (although tonight I’ve been learning Racket through Beautiful Racket, can’t help myself :( ).

            Like this past weeks I’ll be focusing on:

            • Learning to draw using Drawabox
            • Learning to model in 3D with Blender using CGCookie
            • Continue learning Japanese using NihongoShark
            • Climbing, went rock climbing for the first time recently after some months bouldering in a climbing gym and my heights phobia destroyed my brain, need to put the sucker to sleep.
            • Want to introduce some yoga everyday at night with the Yoga with Adriene videos on YouTube
            1. 5

              A great example of “doing” devops. But I continue to see this ideal that we “do” devops in companies like we “do” agile. And places are continuing to miss the core cultural changes that these systems are designed to bring. You end up with companies that are “using pipelines” to deploy code but they are about as devopsy as a battle ship is nimble. Are we ever going to realize that devops like lean is more of a culture shift then a job position and actually use it, or is it destined to ride of into the sunset of derision that agile seems to be riding off into?

              1. 1

                I hope I didn’t actually talk about “doing” devops. I agree that it’s a culture, not a job title, but that wasn’t my experience in the environment where I came up with this presentation/blog post.

                1. 1

                  What would you say are the core culture changes that a company switching to DevOps should look into, or should pay special attention to?

                  1. 3

                    You have to make several changes.

                    1. Remove silos. At my current company, the Dev team has to ask the devops team for a pipeline, then they have to ask the infrastructure team for the cloud resources they need to be provisioned. When they should be able to do all of that internally to the team.

                    2. Hire generalist developers. Generalists can get you 80% of the way there, and then in most cases a team of generalists can get you the next 20% that need if you even need that full 20% to be successful. And if you tackle every 20% problem with a team the whole team gets better.

                    3. Remove the idea of non working managers. I’m currently working to instill this idea of working managers that are active in pulling tasks of the queue because then if they know the pain that comes directly to them from decisions then they will make better decisions and push back on upper management and the business to make smarter decisions.

                    4. Actually do agile/lean and when a problem arises the whole team owns it and fixes it. That way everybody gets better at the problems and they become less of a problem because developers know to look out for them in the future.

                    I’m actively working at a mid sized corporation to work to do these things and honestly we are having to look at pulling a team completely out of the existing IT monolith to be under the CTO directly because the powers are so entrenched they will Never be able to make these changes in the existing monolith. So I’ve proposed tackling it like you would micro services. Small teams that are pulled out and rolled into this new more startup like structure. We shall see if my experiment is successful or if they fire me.

                1. 2

                  As I’m spanish/catalan I usually say these things in a mix of catalan and english:

                  1. ɛ te ˈsɛ
                  2. li.ˈβ
                  3. ˈʧaɾ
                  4. effe check
                  5. eskema (eskemas)
                  1. 1

                    I too mix spanish and english, altough with /etc as etcetera.

                    I think the worst from me might be ce ache oun

                    1. 1

                      hahahaha, yes “ce ache oun” is a classic, I usually say “choun” tho, “ele ese” too.

                  1. 4

                    Thanks for this! I’ve been looking to learn a bit about statistics and it’s been a bit overwhelming, this takes some of the work off my shoulders to find good resources

                    1. 2

                      I know of a shitty morale boosting gesture through a friend of mine. He was working in a startup and they were staying a bit late one day in order to get a release out. The CTO kept walking between tables and giving cookies. “You’re doing a great job, here, a cookie!”

                      1. 3

                        Basically web is made for displaying documents not doing games

                        1. 1

                          ‘Computers are for company data, sorting and classifying not doing games’

                          Platforms evolve and change. I always see here in Lobste.rs some kind of nostalgia for computers like the C64 because you had direct access to the framebuffer and it was super easy to draw to the screen. The canvas API offers something similar in my opinion. It’s the easiest way to get a pixel on the screen nowadays. And the ability to distribute your game worldwide for free is nothing short of amazing.

                          Yes, the evolution of javascript and the web platform has brought some really big pain points like websites that should be static that now consume more resources than photoshop, and that is something that should definitely change, but let’s not sacrifice the amazing multi-platform development and distribution platform that the web is because some people abuse it, not all of us have to pay for the crappiness of certain devs.

                          1. 1

                            You are extrapolate a lot around what I say. (I confess that I was unclear)

                            More or less what I want to say is that web platform was not designed for games and games is not the main use, as such this article saying more or less it is broken because of this design and use is very strange.

                            1. 1

                              Ah, I see, I read it as saying “and that is how it should be”. My bad, maybe jumped too quick to conclusions.

                        1. 18

                          I continue being amazed both by how fragile the security of our systems is and the ingenuity of the security researchers. It seems it’s impossible for anyone to completely understand all the implications of every design decision. Even the ECC correction is not enough in this case by exposing yet another side-channel in the latency of reads, giving the attacker the information it needs to know if there has been a flip or not.

                          What could be done in order to mititgate side-channels systematically? Is it to go back to simpler, even if slower systems? I don’t think even that would help, right? Is security really a completely unaittenable goal for computing systems? I know that the general idea is that perfect security doesn’t exist and the level of security depends on tradeoffs, but hardware side-channels are very scary and I don’t think it is that much about trade-offs anyway (although I am far from knowledgeable in this).

                          I used to have this trust in hardware, don’t know really why, but more and more I’m scared of the amount of ways to get secret information there are (even if impractical).

                          I think we humans got into levels of complexity we were completely unprepared for, and we will pay it badly very soon.

                          1. 11

                            I continue being amazed both by how fragile the security of our systems is and the ingenuity of the security researchers. It seems it’s impossible for anyone to completely understand all the implications of every design decision.

                            Sort of. Applying covert-channel analysis to Intel CPU’s in the mid-1990’s showed pervasive vulnerability. If you do it at system level, you’d see even more of these problems. I’d seen folks on HN griping about QA being a low priority when they worked at RAM companies. The problems were mostly ignored due to market and management’s economic priorities: make things faster, smaller, and with less power at max profit. That leads to less QA and more integration instead of separation. Both apathetic users and companies supplying their demand got here willingly.

                            The attacks have been really clever. There were always clever defenses that prevented many of them, too. Companies just don’t use them. There’s a whole niche of them dedicated to making RAM untrusted. They define SoC itself as security boundary, try to maintain confidentiality/integrity of pages, and typically take a performance hit from the crypto used to do that. Another strategy was using different DIMM’s for different applications with separation kernels flushing the registers and caches on a switch. The RAM controller would get targeted next if that got popular. Others suggested building high-quality RAM that would cost more due to a mix of better quality and patent royalties RAM cartel would sue for. It has to be high volume, though, if nobody wants to lose massive money up-front. I was looking at sacrificing RAM size to use SRAM since some hardware people talked like it had less risks. I’d defer to experts on that stuff, though.

                            “What could be done in order to mititgate side-channels systematically?”

                            Those of us worried about it stuck with physical separation. I used to recommend small-form PC’s or high-end embedded (eg PCI cards) tied together with a KVM switch. Keep untrusted stuff away from trusted stuff. Probably safest with a guard for what sharing needs to happen. Most people won’t know about those or be able to afford them. However, it does reduce the problem to two things we have to secure at users’ end: a KVM switch and a guard. Many guards have existed with a few high security. I think Tenix making a security-enhanced KVM. It’s a doable project for open source, small company, and/or academia. It will require at least two specialists: one in high-security with low-level knowledge; one doing EMSEC, esp analog and RF.

                            1. 11

                              Is security really a completely unattainable goal for computing systems?

                              Well, yes. Not because they are computer systems, but because they are physical systems.

                              Let’s take fort-building techniques and materials as an analogy. Suppose you want to protect a crown. There was a pre-fort era: anybody could walk up and take the crown, if they knew where it was. Think dialup access to a prod system; no password. Early forts were a single, short, unconnected wall (designed to halt the progress of foes coming at you from a single point) and they were trivial to defeat: think of a front end with a password and a backend database with no password, also connected to the internet. Let’s fast forward…

                              Modern forts have moats and observation towers and doors that are armored and that armor is engineered to be stronger than the walls–which provides a sort of guarantee that they ain’t gonna breach that door–it’s cheaper for them to go through the wall. Modern forts have whole departments dedicated to simply determining ahead of time how powerful the foe’s strongest weapon is and making sure the armor is at least strong enough stop that weapon.

                              You see where I’m going. A fort is never “done”. You must continue to “fortify”, forever, because your foe is always developing more powerful weapons. Not to mention, they innovate: burrowing under your walls, impersonating your staff, etc.

                              That said, there are some forts that have never been breached, right? Some crowns that have never been stolen? This is achieved by keeping up with the Jones, forever. It’s difficult and it always will be, but it can be done.

                              What about physics? Given infinite time, any ciphertext can be brute-forced, BUT according to physics, the foe can not have infinite time. Or, given infinite energy, any armor can be pierced, BUT, according to physics, the foe can not have infinite energy. Well, this isn’t my area, but.. does physics say that the foe can not better at physics? Better keep up…

                              The horror we’re facing now with all these side channel attacks is analogous to the horror that the king in that one-wall fort must have felt. “Oh crap, we’re playing on a massive plane, rather than a single line between them and me. I’m basically fort-less right now.”

                              (EDIT: moved my last paragraph up one and removed the parens that were wrapping it.)

                              1. 3

                                What could be done in order to mititgate side-channels systematically?

                                Systematic physical separation of everything.

                                Provision a new Raspberry Pi for each browser tab :D

                                (more practically, never put mutually untrusted processes on the same core, on the same DRAM chip, etc. maybe?)

                                1. 4

                                  There’s not that much unpractical about it, I do it on a daily basis - though Pine64 clusterboard turned out a bit cheaper (~300usd / for 7 tabs) than the PIs. Ramdisk boot chromium (or qemu, or android or, …) as a kiosk in a “repeat-try connect to desktop; reboot” kind of loop. Have the DE allow one connection everytime you want to spawn your “tab”. A bit more adventurous is collecting and inspecting the crashes for signs of n-days…

                                  1. 3

                                    Provision a new Raspberry Pi for each browser tab :D

                                    Ah yes, the good old “Pi in the Sky” Raspberry Pi Cloud

                                    1. 3

                                      Power usage side channels will still leak data from one Raspberry Pi to another. The only larger point I could tie that to is that perfect defense is impossible, but sebboh already said that quite eloquently, so I’ll leave it at that.

                                      1. 6

                                        Most of the more esoteric side channels are not readily available to other systems however. Even physically colocated systems aren’t hooked into the same power monitor to watch each other.

                                        There will be a never ending series of cpu/ram performance side channels because the means of measurement is embedded in the attack device.

                                        1. 3

                                          Separate battery systems (power), everything stored at least 30cm apart (magnets) in a lead-lined (radiation) soundproof (coil whine) box. Then you’ll want to worry about protecting the lines to the keyboard and monitor…

                                          1. 1

                                            is it possible to protect monitor cables / monitors for remot scanning. From what I’ve gathered there is hardware that can get a really clear picture of what’s on screen from quite the distance. Faraday’s cage around the whole unit and or where you are sitting or what?

                                            1. 2

                                              From my fairly basic knowledge of the physics, yes. Any shifting current in a wire will make that wire act a little like an antenna and emit radio waves, which is how these attacks work. It’s usually undesirable to have the signal you’re trying to send wander off into the ether, so cables are designed to minimize this, but it will always happen a little. Common coax cables already incorporate braided wire mesh or foil around the signal-carrying bits, for example.

                                              But, it can never eliminate it completely. So, it’ll always be another arms race between better shielding and more sensitive detectors.

                                              1. 1

                                                ah so they work against the cable and not the display itself right? Does this mean that say a tablet or a laptop is less susceptible to this kind of attack than a desktop computer?

                                                Also to really be foolproof would it be useful to build faraday’s cages into the walls? I’ve heard that if the metal rods stabilizing the concrete in buildings gets in contact with water that grounds them, creating a faraday’s cage and this explains why cell phones can get really bad reception in old big concrete houses. Wouldn’t it be a sensible measure for large companies to do exactly this but on purpose. For cell reception they could have repeaters inside where that would be needed. Wifi is supposed to stay indoors anyways and yeah chinese spies with tempest equipment shouldn’t get their hands on any radiation either.

                                                1. 2

                                                  They’re called emanation attacks. The defense standards are called TEMPEST. Although they claim to protect us, civilians aren’t allowed to buy TEMPEST-certified hardware since they’d have harder time spying on us. You can find out more about that stuff here (pdf), this history, this supplier for examples, and Elovici et al’s Bridging the Airgap here for recent attacks.

                                                  The cat and mouse game is only beginning now that teams like Elovici’s are in the news with tools to develop attacks cheaper and more capable than ever. It’s why Clive Robinson on Schneier’s blog invented concept of “energy gapping.” All types of matter/energy that two devices share is potentially a side channel. So, you have to mitigate every one just in case. Can’t just buy a product for that. ;)

                                                  1. 2

                                                    yeah I heard about TEMPEST there was this fun program that let you broadcast FM or AM via your CRT that I played with forever ago tempest for eliza or something.

                                                    messed up that they make laws against things like that.

                                                    My thinking is to protect the whole house at once or why not cubicle depending on how much you are willing to spend on metal of course

                                                    1. 1

                                                      This?

                                                      Far as whole house, they do rooms and buildings in government operations. A lot of the rooms don’t have toilets because the pipes or water might conduct the waves. Air conditioning is another risk. Gotta keep cellphones away from stuff because their signal can bounce off the inside of a passively-secured device, broadcasting its secrets. All sorts of issues. Safes/containers and SCIF-style rooms are my favorite solutions since scope of problem is reduced.

                                                      1. 1

                                                        Yeah that’s the one.

                                          2. 2

                                            I always recommended EMSEC safes with power filters and inter-computer connections being EMSEC-filtered optical. So, yeah, it’s a possibility. That said, some of these systems might not have the ability for firmware, kernel code, or user code to measure those things. If none are this way, new hardware could be designed that way with little to no modifications of some existing hardware. Then, a compromise might just be limited to whats in the system and whatever the code can glean from interactions with hardware API’s. On the latter, we use ancient mitigations of denying accurate timers, constant-time operations, and masking with noise.

                                            I think there’s potential for making some of those attacks useless with inexpensive modifications to existing systems. Meanwhile, I’m concerned about them but can’t tell you the odds of exploitation. We do need open designs for EMSEC safes or just containers (not safes), though.

                                        2. 3

                                          I used to have this trust in hardware, don’t know really why, but more and more I’m scared of the amount of ways to get secret information there are (even if impractical).

                                          As long as there’s physical access to a machine, that access will be an attack vector. As long as there’s access to information, that information is susceptible to being intercepted. It comes down to acknowledging and securing against practical attack vectors. Someone can always cut my brakes or smash my windows and take my belongings from my car, but that doesn’t mean I operate in fear every time I park (of course this is a toy analogy: it’s much easier and far less risky to steal someone’s digital information, EDIT: and on second thought, you would immediately know when your belongings have been tampered with).

                                          From the paper:

                                          We now exploit the deterministic behavior of the buddy allocator to coerce the kernel into providing us with physically consecutive memory

                                          Does the Linux kernel currently have any mitigations like randomization within its allocators? I believe this is orthogonal to ASLR.

                                          1. 2

                                            Hardware is cheap; use that as your security boundary between trust domains. On-device process separation, virtualization, still makes a lot of sense for other reasons (compatibility, performance, resilience), but it is about as alive as a parrot in a monty python sketch when it comes to security. Rowhammer should have been the absolutely last straw in that respect - there were plenty of indicators well before then. What sucks is that the user-interfaces and interaction between hardware separated tasks (part of the more general ‘opsec’ umbrella) is cumbersome at the very best. Maybe that is easier to fix than multiple decades of opaque hardware…

                                            1. 4

                                              Consumer grade hardware may be cheap; Power and hardware with ECC RAM support is not so much. With dedicated hardware you are burning a lot more power for useful computations performed.

                                              For this particular attack, AMD’s Secure Encrypted Virtualization (SEV) is an actual solution and is mentioned as such in the paper. Intel’s Multi-Key Total Memory Encryption (MKTME) should be too when it comes out. Unfortunately software support is not really what I would call complete yet.

                                          1. 6

                                            I have a cousin’s wedding and we are planning a trip to Japan, anyone that has been there has any advice or tips? No tech related stuff probably.

                                            1. 4

                                              Hey! I currently live and work in Tokyo. Do you have questions about a specific topic?

                                              1. 2

                                                Oh! That’s super cool! I have some questions, yes:

                                                • What would you say are some must-see things all around japan? I’m mostly interested in seeing the old japanese culture, art and towns. How easy/hard is that to find?
                                                • Is rail the best way to move around the country? What would you recommend we do between renting a car or going everywhere by train?

                                                And now one unrelated to the trip, how did you find work there and how was the visa process like?

                                                1. 2

                                                  Unless you are really comfortable driving on the left side in dense unfamiliar urban environments I’d recommend against renting a car there.

                                                  1. 1

                                                    I can only speak for Tokyo area, since it is the only town I know.

                                                    Is rail the best way to move around the country? What would you recommend we do between renting a car or going everywhere by train?

                                                    The rails network is very good here. I only know the Kanto area rails network (mostly operated by JR East). All train companies here are very well integrated with Google Maps. You can track every train in real time for each station. This is super useful when you want to be sure that you are on the right platform for the right direction.
                                                    There are several private companies operating trains in the same area. With one pass (the “Suica” pass), you can access all networks, but avoid inter-network exchanges, since you’ll pay a base fee each time you access a network (~160JPY). So, try to stick with one company when you take the train/metro.

                                                    What would you say are some must-see things all around japan? I’m mostly interested in seeing the old japanese culture, art and towns. How easy/hard is that to find?

                                                    The Enoshima Island is great, Kamakura’s temple is fine, too. However, they are a bit touristic. If you want to visit some non-mainstream places, be prepared to speak and read Japanese! ;)

                                              1. 29

                                                The hypocrisy of this article is staggering.

                                                Google uses advanced techniques to violate your privacy, even if you are trying to maintain it. It uses browser finger printing techniques to identify you even if you are blocking cookies. If you use Chrome, then there is an interface in Settings for blocking cookies on a per-site basis, but these settings are ignored if you try to block Google cookies.

                                                It is very difficult to have privacy when using Google services. First of all, don’t create a Google account or log in to Google. So obviously don’t use Gmail. In order to have privacy when using Google search or when viewing videos on Youtube, the only effective technique is to use the Tor browser.

                                                1. 8

                                                  Yep. Also the Kafkaesque situation that has been developing with ReCaptcha. Where in some cases you cant succeed with any amount of solves, but if you log in to Gmail first it works right away.

                                                  https://recaptcha-demo.appspot.com/recaptcha-v3-request-scores.php

                                                  1. 3

                                                    Google blocks Tor on their search service.

                                                    1. 1

                                                      I just tried it right now. This time, I had to go through a reCAPTCHA dialogue, then my search worked. That doesn’t always happen. If you are being blocked, you may need to change to a new TOR circuit.

                                                      1. 1

                                                        Huh, I’ve never gotten past it. Maybe it’s just blocked without JavaScript.

                                                        1. 1

                                                          without js recaptcha doesn’t work so you can’t get past the block

                                                    1. 30

                                                      This response isn’t a denial. I think folks should notice that. He goes out of his way to diminish the author without actually denying it. He’s mad about the post, not that he’s falsely accused, because the post is true.

                                                      This response is also filled with red flags:

                                                      Saying Tom didn’t want money and then saying all future donations will be split with Tom is a weird contradiction. The quote “work on the advancement of requests” seems like a way to differentiate between maintenance (which he wasn’t really doing while others were) and “advancement” (which is whatever he’s doing). Including the news that the library will changing its backend sounds like one of those sudden, made-up decisions people do try and make their accuser seem unqualified. How interesting the timing on that! Talking about the small set of “real collaborators” excludes someone who he explicitly says he was collaborating with is gaslight-y. And the “just don’t fucking work with me” has such a long history of being said by people who really did awful things and don’t want to admit that.

                                                      1. 3

                                                        Saying Tom didn’t want money and then saying all future donations will be split with Tom is a weird contradiction

                                                        But Tom is not njs.

                                                        1. 1

                                                          Including the news that the library will changing its backend sounds like one of those sudden, made-up decisions people do try and make their accuser seem unqualified.

                                                          You make some good points. In terms of timing, feel like this was mentioned ahead of PyCon on an episode of Talk Python, but I was only half-listening to that the first time.

                                                        2. 23

                                                          All that being said, I’m not sure why this person feels the need to attack my character, including curating a list of quotes (what?) from “collaborators”.

                                                          I’d just like to point out that Kenneth has lists of quotes…about himself…on his website.

                                                          1. 0

                                                            Kenneth has lists of quotes…about himself…on his website.

                                                            While I’m in no way defending Kenneth or his actions, mocking someone for stating their opinions (in quote or any form) on their own website is not in the spirit of engineering or science. If you feel the need to be petty, please find another place to dunk on people.

                                                            1. 5

                                                              I was highlighting the irony of the journal entry expressing incredulity about nj’s inclusion of a list of quotes from collaborators, as KR knows all about including quotes from “collaborators” (or sycophants, everyone can make up their own mind).

                                                              As far as “scoring on people”, I’d suggest that you are the one who is attempting to do so, with your virtue signalling and calling me petty.

                                                          2. 3

                                                            It always amazes me (and scares me) how different people percieve reality (if that is even an achievable thing) and how the same situation can be read completely differently by two different brains. It is super scary to me. In this case I believe neither of them had anything malicious going on, and still, both of them have a completely different grasp of the situation.

                                                              1. 8

                                                                This is why I think the original article was bad form. I know neither of the people involved. I’ve never even heard of them. I wouldn’t know who to believe even if I knew them.

                                                                Tag this one as “call out culture.” If there’s something to be done, it should probably be done within that community and with discretion, precisely because there are two sides to every story and people are biased toward the first/best expositor regardless of whatever actually happened.

                                                                I think it would be great if the mods banned personal call out articles on this basis. And, again, I know neither of these people. I’m not in the Python community.

                                                                1. 10

                                                                  it should probably be done within that community and with discretion, precisely because there are two sides to every story and people are biased toward the first, best expositor.

                                                                  I don’t disagree in general, but how do you do that in the context of an open source community? There is no real central authority, and people can essentially just do what they want.

                                                                  1. 18

                                                                    I actually can contextualize Nathaniel’s post with my own interactions with reitz (which were a lot less involved) but they verify my impression.

                                                                    So I an glad Nathaniel posted this. It helps me stay clear of unproductive conflicts for the future.

                                                                    A few helpful and engaged members of the python community have signaled that it matches some of their observations.

                                                                    If you keep such things private and secretive is hard to go through with community actions (like removing someone from boards, etc). If you make it public discourse people complain about character assassination or whatever. At the end of the day I believe in a victims right to discuss their case publicly of they want to.

                                                                2. 2

                                                                  I do think that’s true, but I think that one of the author’s central points, and part of the reason I posted this, is that it’s important to be aware that when money is involved there is a whole different level of accountability that comes into play.

                                                                  This is why the legal system exists. This is why scrupulously detailed contracts arbitrated by lawyers exist.

                                                                  Moreover, this is why foundations like the PSF exist - they handle the ‘dirty’ work of distributing money in a way that’s free of legal entanglement and less likely to engender this kind of mis-understanding.

                                                              1. 10

                                                                I have had the view that safety is typically not the most important goal of a project (user satisfaction is). In my experience, safety can get in the way of writing software that is useful (the real world is really messy). It’s great to see a real-life case with Rust from someone who obviously tried very hard but ultimately had safety get in the way of writing the software they wanted. Hopefully, Rust will evolve to interface with the “unsafe” world in a more ergonomic way.

                                                                1. 9

                                                                  I think many people obsess over Rust’s idea of safety and lose track of the point of safety. The point of safety is to create software that does not leak (performance or memory) and does not crash.

                                                                  It is entirely possible to write software that does not leak or crash, in C. It requires good practice and good tooling to enforce that good practice, as does programming in literally any other programming language (You can write spaghetti code in any language, after all).

                                                                  1. 6

                                                                    That’s absolutely true, but how much faith do you have that Way Cooler will be one of the C projects that uses good practice and good tooling to minimize the amount of memory-safety bugs?

                                                                    1. 1

                                                                      If it’s written by somebody that is very familiar with Rust (and clearly it will be) then actually I have a lot of confidence in Rust. Writing a lot of Rust is like writing a lot of any language: it infects your mind and way of doing things, and I suspect it will have infected the author’s mind with thinking about memory management constantly, which will hopefully mean they think about it constantly during their C programming too.

                                                                      1. 1

                                                                        This has actually happened to me, the more I write Rust the more I can predict what the compiler will complain about. And I think about that a lot when writing C too. It’s very… interesting? when I’m writing C and I think about ownership.

                                                                    2. 4

                                                                      I actually think a more encompassing tool that also deals with safety, but not just safety, all other forms of bugs is Design by Contract. I’m excited to see DbC will be part of the C++ language standard for 2020. Rust had a discussion about DbC as part of the language but many feel this library fills most of the needs. While a library approach is better than nothing, having DbC as part of the language (after all, types ARE a contract) makes more sense to me. Hopefully, the Rust folks keep the discussion going and make DbC part of the language. Maybe more discussion happened but I haven’t seen it.

                                                                      1. 3

                                                                        Out of curiosity, how is Design by Contract different/better than just sprinkling assertions at the start/end of your functions? Most of the examples I’ve seen for it are like those in the readme of the library you linked, which are fairly trivial. How do you express more complicated contracts like, say, “The sublist returned by this list search routine is a subset of the list passed in and shares the same memory”? Do you have suggestions for more in depth things I can read?

                                                                        1. 3

                                                                          Most languages don’t have first-class support for Design by Contract so most implementations are glorified assertions. To express more complicated contracts you would use higher level functions in your assertion. For example, you can define a “is_subset” function and use that in your assertions.

                                                                          I do many C++ projects and use DbC for all of them, and before C++2020 I used a small library I wrote Which I copy and past in all projects. When compilers will support native contracts in C++, I’ll use those instead of the library.

                                                                          1. 2

                                                                            My Dafny is a little rusty, but assuming you mean modifying the list to get the sublist, it would be something like

                                                                            modifies array
                                                                            ensures exists lo, hi :: out == old(array[lo..hi])
                                                                            
                                                                        2. 3

                                                                          How about memory vulnerabilities leading to RCEs? Do you suggest apps with those are safe? Why “has leaks” is unsafe, but “has remote code execution vuln” is safe? I totally don’t get it. Can you explain, other than just claiming “because I say so”?

                                                                          1. 1

                                                                            Can you explain your position better?

                                                                            Are you saying that GCC is guaranteed to generate code that has remote code execution vulnerabilities? And Rust doesn’t have this?

                                                                            It’s my understanding that memory leaks are what cause RCEs (At least, the main cause of them). It is also my understanding that there is nothing inherent about C’s memory model that causes presumably safe (As tested by tooling and humans) code to magically appear with RCEs.

                                                                            1. 4

                                                                              RCEs are not caused by memory leaks, but most commonly by accidental occurences of:

                                                                              • use after free (this is something different than memory leaks — more like opposite of memory leaks: mem leak is when you forget to free(), use after free is when you do free() and yet dereference the pointer afterwards)
                                                                              • out-of-bounds array reads/writes (most notoriously as a result of off-by-one errors, but not only)
                                                                              • Undefined Behavior (good intro: [1] [2] [3])

                                                                              Especially the much-too-many, casually unknown UB “fine print” cases are “something inherent about C that causes presumably safe (as tested by tooling and humans) code to magically appear with RCEs”. The most infamous example that basically proves that human/tooling testing is not enough, is the recent RCE in SQLite, a project that is often perceived as the most covered by tests & tooling & “human review” open-source C codebase in the world, with any contenders being far behind. Other than that, the PVS-Studio’s website is a great reference showcasing the proliferation of (often RCE-grade) errors in widely used open-source C codebases. (The common understanding being, that closed-source codebases are usually even worse.) Their blog is also worth a read.

                                                                              Rust is specifically designed with an aim towards completely eliminating whole classes of such errors. Not all errors, mind you. But especially the RCE-grade ones resulting from the reasons I listed above. IOW, yes, I’m saying that GCC/Clang is guaranteed* to generate code that has remote code execution vulnerabilities, and Rust doesn’t have this**.

                                                                              * — Unless: (a) maybe if the codebase were written by 1 person, who is flawless; but there is no such person; (b) also, there’s some chance military-grade hardened C code might have no RCEs (this requires super expensive and tedious dev process); (c) code written in Frama-C or similar “verified C” dialects, potentially.
                                                                              ** — You can still write RCEs in Rust, but it’s levels of magnitude harder to do accidentally, and close to impossible if you’re not using unsafe blocks (barring bugs in the compiler/standard library).

                                                                              Edit: Also, please note that I’m fully aware Rust is super annoying to work with. I myself tried, and gave up for now. I’m not claiming it’s a panaceum for any and all problems. But C, and esp. C++, those are paths I followed with devotion, deep enough that I touched the lurking madness, and came back shaken and changed forever.

                                                                              1. 2

                                                                                The supposed RCE in sqllite is dubious. Sqlite is a database that can accept executable code as an input. If your application allows unrestricted access to that database and someone then provides an input to the DB with executable code in it and the DB executes it, then the error is in the application, not the DB.

                                                                                1. 3

                                                                                  If you let a user write queries, it’s expected behavior that they can modify the database.

                                                                                  It’s not expected behavior that they can read/write unrelated files or perform network activity.

                                                                                  1. 2

                                                                                    that is exactly the intended functionality of the sqlite full text search

                                                                                    1. 1

                                                                                      Did you somehow reply to the wrong comment?

                                                                                      How is full text search intended to let me overwrite /etc/passwd?

                                                                                      The text search index is stored in the same database file as everything else. I expect users with access to use it to read/write that file.

                                                                                  2. 2

                                                                                    Sorry, but it feels like “blame the victim” mentality to me. Did SQLite at least come with warnings of “do not let users write SQL queries, as we don’t audit such scenarios for security”?

                                                                                    1. 2

                                                                                      It’s not at all “blame the victim”. It is: “understand what the tool does”. SQLite doesn’t even have user accounts. It’s an SQL engine and SQL is a powerful language.

                                                                                  3. 0

                                                                                    use after free (this is something different than memory leaks — more like opposite of memory leaks: mem leak is when you forget to free(), use after free is when you do free() and yet dereference the pointer afterwards)

                                                                                    This is a simple error to avoid, though. Alter free() to check for NULL and don’t free() if the input is NULL (Something that is nonsensical and a check that should be done anyway), then after every single free, set the pointer to NULL. It becomes very, very difficult to use-after-free using this method. In the same way, set file descriptors to negative values before assignment and check for negative values before close(), etc.

                                                                                    out-of-bounds array reads/writes (most notoriously as a result of off-by-one errors, but not only)

                                                                                    Every single array should store and check against the maximum number of elements, and in some cases the current number of elements in the array.

                                                                                    Undefined Behavior (good intro: [1] [2] [3])

                                                                                    This is probably the only case that causes errors. However, there are tools to check for UB (One of them, PVS-Studio, you linked to. Another is available here: http://css.csail.mit.edu/stack/). It is possible to remove the UB from C, but there is a lot of resistance to it for the reason that it is useful. I personally don’t agree entirely with them, and I do not see the need to rehash those arguments here.

                                                                                    Formal verification is something I find deeply interesting, but at some point you need to balance ‘shipping’ against ‘total safety’. I also don’t see distrusting {GCC / Clang}’s code generation as a ‘useful’ stance, given that most ‘safe’ languages eventually, somewhere down the chain, rely on the code generation from {GCC / Clang}. A case in point, Rust’s first version was written in OCaml, which descends from OCaml Light, which was written in C.

                                                                                    If you truly believe that “GCC/Clang is guaranteed to generate code that has remote code execution vulnerabilities” (Emphasis mine), then you must extend that distrust to the OCaml Light code generator, which was generated by {Clang/GCC}, which means you have to distrust OCaml (A language cannot be safe without safe code generation), and therefore you can hardly trust Rust to be safe!

                                                                                    1. 2

                                                                                      Ok, from what you wrote here, I am surprised to see that we seem to be kinda coming to an agreement. Meaning, theoretically, I certainly agree, that it’s possible to have safe code emitted from GCC, iff the input C source code is perfectly flawless (as I explained in the footnote). I stand by my claim however, that in real life, i.e. practically, this cannot really be achieved (with the exceptions I stated previously). Moreover, I suppose even PVS-Studio is not perfect; I don’t know it very well, I admit, so maybe I’m in error, but I’d be really (and positively!) surprised if they claimed to eliminate all UB-based errors.

                                                                                      Interestingly, as to generated C code (i.e. output of compilers such as OCaml Light, Nim, etc.), I believe it can be actually much easier to keep safe! The trick here is that the generated C code will probably be a relatively small/finite subset of predesigned C patterns/snippets. The author of the compiler should thus be able to enforce using only safe C constructs; interactions between the limited number of patterns can be scrutinized much better; and finally, extra protections such as bounds checks can be added fully automatically to every snippet that needs them, leaving no space for an occasional human error in perusing them. (Including the common hubris such as not setting a freed pointer to NULL “because obviously it won’t be used anywhere further, so why waste the coveted CPU cycles”.) OTOH, if a compiler author does introduce some UB, I imagine there would be a higher chance that it will get repeated automatically by the compiler, thus hopefully making it “louder”/more frequent and therefore easier to notice and fix.

                                                                                      Edit: Also, please note, that with all the checks you suggest (in free, in arrays), you’re actually already not talking about basic C, but some special dialect of C-with-protections! Notably, some of them (e.g. bounds checks) are often laughed away by C programmers, as “costly, reducing effectiveness/speed”. And with this notion of adding reasonable protections, you can slowly get to trying to unknowingly reimplement Rust! Or Ada, more probably. Which is indeed seen as much sager language than C. (Though also more annoying, I believe even than Rust.)

                                                                                      1. 1

                                                                                        you’re actually already not talking about basic C, but some special dialect of C-with-protections!

                                                                                        Wow, I didn’t realise that coding properly was called coding in a different language! Thanks! I didn’t realise that ‘Python with style-guides and tooling’ was a different language to Python! That’s amazing! /s

                                                                                        The trick here is that the generated C code will probably be a relatively small/finite subset of predesigned C patterns/snippets. The author of the compiler should thus be able to enforce using only safe C constructs; interactions between the limited number of patterns can be scrutinized much better; and finally, extra protections such as bounds checks can be added fully automatically to every snippet that needs them, leaving no space for an occasional human error in perusing them.

                                                                                        So what you’re essentially saying is that, you can code safely in C, albeit extremely carefully. Yes, we do seem to be agreeing.

                                                                                      2. 1

                                                                                        This is a simple error to avoid, though. Alter free() to check for NULL and don’t free() if the input is NULL

                                                                                        That only works if they’re not only freeing the same spot in memory, but also freeing it through the same pointer. That’s not where most of the use-after-frees come from. The use-after-frees all come from aliased pointers, like in this code.

                                                                                        void consume_object(object *ptr) {
                                                                                            do_something_with(ptr);
                                                                                            free(ptr);
                                                                                            ptr = NULL;
                                                                                        }
                                                                                        void main_whatever() {
                                                                                            object *ptr = malloc(sizeof(object));
                                                                                            init_object(ptr);
                                                                                            consume_object(ptr);
                                                                                            /* ptr is not NULL, in spite of what `consume_pointer` did, because reasons */
                                                                                            free(ptr);
                                                                                            ptr = NULL;
                                                                                        }
                                                                                        
                                                                                        1. 1

                                                                                          because reasons

                                                                                          But if you’re writing code like that, you don’t understand pointers or argument passing. If I write code in Haskell and expect it to be strict-evaluation, then there will be huge problems with it. That’s not a fault of the language. That’s a fault of me.

                                                                                          Besides, as I have previously said, these problems are caught with appropriate tooling. Both cppcheck and scan-build point out this error.

                                                                                        2. 1

                                                                                          Alter free() to check for NULL and don’t free() if the input is NULL (Something that is nonsensical and a check that should be done anyway),

                                                                                          Two problems:

                                                                                          1. Developers regularly don’t do it enough to catch everything.

                                                                                          2. There might be a performance penalty.

                                                                                          Rust’s method increases odds they’ll have to deal with it while eliminating the need for runtime checks. If runtime checks are fine, there’s way to do those, too, while getting other benefits for memory safety.

                                                                                  4. 2

                                                                                    “It is entirely possible to write software that does not leak or crash, in C.”

                                                                                    It will be a lot harder in most cases. Especially given memory safety applies on all inputs. Getting that in C usually means hand-inserting checks everywhere and/or running it through a sound static analyzer that might cost five digits.

                                                                                    “ It requires good practice and good tooling to enforce that good practice, as does programming in literally any other programming language (You can write spaghetti code in any language, after all).”

                                                                                    This is a false equivalence. If the code is safe Rust, your mistakes will not usually lead to code injection. Programmers will make mistakes, esp if overly casual or hurried. Field evidence indicates most will do this regularly with careful experts doing it some of the time. In C, those mistakes will increase number of successful hacks or leaks.

                                                                                    So, you could say using something that converts most hacks into panics is using “good tooling” to get good results. Aside from borrow checker, you get most of that safety without even trying. That boosts productivity on that kind of code. If borrow checker is too much, one can downgrade to reference counting or unsafe with the other risks still mitigated automatically.

                                                                                    1. 1

                                                                                      Getting that in C usually means hand-inserting checks everywhere and/or running it through a sound static analyzer that might cost five digits.

                                                                                      From personal experience, a lot of the tooling that helps you avoid these mistakes is free. But, don’t let that stop your fearmongering, please.

                                                                                      If the code is safe Rust, your mistakes will not usually lead to code injection.

                                                                                      [citation needed].

                                                                                      No, really. As far as I know, there are no long-term studies on Rust projects. I don’t think you can argue with ‘field evidence’ until you can show that ‘field evidence’ proves that Rust improves safety. After all, as you say, programmers are lazy. For all you know, they could just be doing the equivalent of dumping unsafe { ... } everywhere the borrow checker complains and calling it a day.

                                                                                      1. 1

                                                                                        a lot of the tooling that helps you avoid these mistakes is free.

                                                                                        Are the tools for total memory safety in C as easy as a Rust compile offering same guarantees? Especially no temporal errors or races on all inputs?

                                                                                        It’s a trick question: I’ve submitted more tooling like that here than anybody. I read their performance and effectiveness evaluation sections. It usually takes more CPU/RAM with less certain benefits than just using Rust. Then, we have the constant vulnerabilities supporting my position. Amateurs are doing better in Rust so far since it’s just immune to a lot of those problems. The compiler yells at them and/or apps just panic in bad situation.

                                                                                        “I don’t think you can argue with ‘field evidence’ until you can show that ‘field evidence’ proves that Rust improves safety. “

                                                                                        The design of the language makes it immune to many classes of errors. Outside a compiler error, about any code in safe Rust will inherit those properties. If you’re talking compiler errors, that would be weird since C compilers, esp optimizing, have been buggy as hell without C programmers showing up dismissing potential benefits of C until their compilers are proven correct. Virtually no use of CompCert in GPL’d software either. Be a double standard there.

                                                                                        “For all you know, they could just be doing the equivalent of dumping unsafe { … } everywhere the borrow checker complains and calling it a day.”

                                                                                        Btw, the borrow checker is just one of many safety mechanisms in Rust. It covers temporal errors that show up even in OpenBSD. I mean, if they can’t avoid them…

                                                                                        What you say might be true, is worth looking for in field data, and might give a different picture of average app in safety. That said, even if they were doing that, the code would still be safer than C given it has no protections against those same risks vs Rust apps likely combining pre-existing, borrow-checked libraries with new code, some of which is unsafe. Attack surface gets lowered just because the lazy path in Rust is pre-existing, safe libraries. The harder path, creating borrow-checked code, is strongly encouraged by its ecosystem to get code into stdlib’s, etc. Better default than C again.

                                                                                1. 5

                                                                                  I’ve seen this more and more lately, accounts where all posts and comments they submit are about their own content and I’m not sure what the general stance is here, but even if they are kinda relevant topics I’m not comfortable with people interacting with the community only for self-promotion. Apart from this, it seems like a very low-effort post where there is a typo in a 5 word title, I don’t think we should allow this here.

                                                                                  1. 2

                                                                                    This week we open a beta for schools of our videogame to learn programming. It’s been a long journey and we still have a long way ahead but this week will be crucial.

                                                                                    I will be finishing some testing and setting up some metrics trackers for our servers and apps so we know if everything is going fine.

                                                                                    Excited!

                                                                                    In the future we will open it for competitive AI programming too, in case any of you want to try :D

                                                                                    1. 1

                                                                                      tiling terminal app similar to iTerm

                                                                                      We usually use a terminal + a terminal multiplexer (like tmux).

                                                                                      If you want a single package that does both there is the Kitty terminal.

                                                                                      1. 2

                                                                                        We are about to open our product to schools! So finishing some details and optimizing a bit the infrastructure so we don’t die next week. Just finished a pretty decent optimization of a dynamic texture atlas creation service we have, went from 200ms/req to under 20ms/req (the target machines are shitty school computers, so we need all performance we can squeeze out of them, we generate texture atlases dynamically for each quest execution to reduce the memory footprint to the minimum).

                                                                                        I may do a writeup of how it works soon-ish if any of you is interested.