Threads for jvns

1

chbarts 3 months ago | link | on: Some possible reasons for 8-bit bytes

the byte size is the smallest unit you can address.

This is only true on some hardware, although compatibility with existing code written for byte-addressable hardware will cause C compilers, for example, to emit enough shift-and-mask code to make word-addressable systems look like they’re byte-addressable to software.

Apparently on x86 the word size is 16 bits, even though the registers are 64 bits.

Thou hast defeated me. I am well and truly confused by what she could possibly mean here.

Word size is register size is pointer size on modern general-purpose hardware.
1. 11
  
  jvns edited 3 months ago | link
  
  I also thought that word size = register size = pointer size until today, but I looked at the Intel® 64 and IA-32 Architectures Software Developer Manuals (section 4.1, “FUNDAMENTAL DATA TYPES”) and it says pretty clearly that a word is 16 bits. So now I’m not sure what to think.
  1. 6
    
    adam_d_ruppe 3 months ago | link
    
    I think it is just a historical accident. They used the word “word” in various places in the 16 bit days, then the 32 bit (and later, 64 bit) extensions had to maintain compatibility and that included those old names and documentations.
    1. 3
      
      andyc 3 months ago | link
      
      Yeah I think that’s likely … that’s why we’re all confused about what a “word” is – because it doesn’t really matter for modern computing :-)
      
      I’m pretty sure that “word” means nothing in C / C++ / Rust as well. The language specs don’t need that concept. They do need to know about bytes and pointer alignment, etc. but not “words”.
      1. 1
        
        breadbox 3 months ago | link
        
        Also, remember that for some time after the first 32-bit x86 chip, the OSes most widely used (i.e. MSDOS and Windows) continued to run in 16-bit mode. It wasn’t until 1995 that a non-16-bit environment became commonplace. By that time, people had been using “word” to mean 16-bit values for so long that re-defining it to just mean a 16-bit value was actually the path of least resistance.
2. 8
  
  icefox edited 3 months ago | link
  
  “Word” is casually redefined by every single CPU architecture out there to be “whatever size we feel like for the context of this CPU”. For x86_64 this means “registers were 16 bits on the 8086 so a “word” is 16 bits forevermore.” For ARM this means “registers were 32 bits on the first ARM chip so a “word” is 32 bits forevermore”. For Itanium, if I’m reading the right source, this means “registers are always 64 bits so a word is 64 bits, except registers have an extra flag for invalid values so they’re really more like 65 bits.” For RISC-V this means “a word is 32 bits because we say so”. On the PDP-10, of course, a “word” was 36 bits, and registers were 36 bits wide, but a pointer was 18 bits.
  
  It’s a pretty dumb bit of legacy terminology, and I wish hardware people would stop using it.
3. 4
  
  x64k edited 3 months ago | link
  
  the byte size is the smallest unit you can address.
  
  This is something that’s kind of correctly quoted, only without context and really outdated, hence the trouble you’ve mentioned :-D.
  
  First, for the modern use: a byte is just a group of bits treated as a unit. That’s the IEC definition, which doesn’t hinge on any practical implementation, because the IEC learned that’s a really bad idea a long, long time ago, and we’re still trying to disentangle some of the mess they made before they learned that.
  
  As for the addressing part…
  
  The first instance of use of the word “byte” as we know it today (i.e. which evolved into what we now call a byte) comes from IBM. It referred specifically to the smallest unit that the I/O unit would process. The shift matrix of the IBM 7030 (described in section 7 here) operated in units smaller than a word; the smallest such unit had 8 bits. Anything less than 8 bits would be padded. A memo (Memo 45) from the same project kind of clarifies that:
  
  The maximum input-output byte size for serial operation will now be 8 bits, not counting any e r r o r detection and correction bits. Thus, the Exchange will operate on an 8-bit byte basis, and any input-output units with less than 8 bits per byte will leave the remaining bits blank. The resultant gaps can be edited out later by programming
  
  (Emphasis mine).
  
  This is how the “smallest addressable unit” came to be. A few years later, when Brooks (that’s the same Brooks in the Mythical Man-Month), Blaauw and Buchholz published “Processing Data in Bits and Pieces”, which was the first public use of the term and what we now use today, that was the “internal” meaning it had at IBM.
  
  Reading that paper is extremely confusing to a modern audience (I can’t link it here since it’s not publicly available anywhere but that site which carries scientific papers may or may not have it, wink wink):
  
  A data-handling unit is described which permits binary or decimal arithmetic to be performed on data fields of any length from one to sixty-four bits. Within the field, character structure can be further specified: these processing entities, called bytes, may be from one to eight bits long. Fields may be stored with or without algebraic sign.
  
  (Emphasis mine). Some of the sub-modules of the data-handling unit work with eight-bit bytes. Others, like the arithmetic unit, use eight-bit bytes for binary arithmetic, and four-bit bytes for decimal-coded arithmetic. There’s actually a figure in that paper that literally shows a word composed of 10 bits that aren’t addressed as a unit so they aren’t part of a byte, seven one-bit bytes, four 4-bit bytes, and five 6-bit bytes.
  
  A few years later, the same authors published a book that was really well-received (Planning a Computer System - Project Stretch), which talked about the IBM Stretch at length. This was the definition of byte for the next thirty years or so. It also spelled out the definition of word:
  
  Byte denotes a group of bits used to encode a character, or the number of bits transmitted in parallel to and from input-output units. A term other than character is used here because a given character may be represented in different applications by more than one code, and different codes may use different numbers of bits (i.e., different byte sizes). In input-output transmission the grouping of bits may be completely arbitrary and have no relation to actual characters.
  
  For practical reasons, the number of bits transmitted in parallel to and from I/O units was also the smallest addressable size on many computers of that era. Addressable units smaller than what the I/O units could handle was kind of pointless (you’d just waste extra hardware to zero out unused bytes, otherwise things like the zero flag would be ambiguous), and addressable units larger than what the I/O units could handle was efficient for some operations but not all.
  
  So in practice, lots of hardware designers in that age just stuck to byte-addressable memory. Even systems with word addressing, like the PDP-6, had special instructions that tl;dr allowed you to address variable-length bytes, although not as efficiently as a word.
  
  Consequently, lots of reference manuals of that era referred to a byte as either “whatever it takes to encode a character” or “the smallest unit we can address”. Both were effectively referencing Brooks’ definition, which was presumably sufficiently well-known that it didn’t need a separate reminder.
  
  For extra shits and giggles, that is also how we ended up with a word. The Project Stretch book also said…
  
  A word consists of the number of data bits transmitted in parallel from or to memory in one memory cycle. Word size is thus defined as a structural property of the memory.
  
  That’s why we can have registers and words of different sizes. In the beautiful golden age of CISC there was no problem having 32-bit registers for 16-bit words – but it took (at least) two cycles to load one.
  
  At some point, though, that… just kind of lost significance, just as it happened with the byte. A byte, or a word, was whatever the reference manual said it was.
4. 2
  
  david_chisnall 3 months ago | link
  
  I prefer the definition of a byte as the larges unit for which the in-memory representation is not observable. You can’t tell if bytes are stored most or lesser significant bit first. You can tell the order of bytes within a word.

21

jaculabilis 3 months ago | link | on: Some notes on using nix

I was complaining yesterday about how there are no Nix packaging tutorials, and lo and behold, someone blogging their Nix experience that actually has packaging in it!

Replace the first statement with import <nixpkgs> {}; I don’t know why you have to do this, but it works.

I basically just wrote this by copying and pasting from other packages in the nixpkgs repository.

I figured out how to set installFlags just by running rg PREFIX in the nixpkgs repository

replace with stdenv.lib to with lib for some reason.

I figured out how to run this by running rg 'mv ' in the nixpkgs repository and just copying and modifying something that seemed related.

Yeah, I guess that’s what I should have expected.

Random thought, but I wonder if Nix needs to have fewer defaults in stdenv. Currently, it seems like all the Nix packaging tutorials use hello or some other simple C program because stdenv is C-oriented. This makes it look like it has fewer moving parts than it really does, which does new users a disservice. It’s harder to understand, precisely, what things Nix is doing when all the builder steps are hidden, and understanding those is critical to having generalizable knowledge about using Nix to build software. This is doubly important when new users try to package something in a language other than C, because then they have to set up things themselves that were not only never explained, but never even appeared in the mkDerivation calls in their tutorials.

Of course, the die has been cast, since the source code in “search for source code that solves an adjacent problem” is already written with the existing defaults.
1. 8
  
  abathur 3 months ago | link
  
  I think part of the problem with all of these are that they assume any familiarity with existing software norms around building/installing/packaging software in different ecosystems.
  
  Disemboweling the stdenv and showing the gore in the derivation would make it much clearer what’s going on for people who know how to read the entrails, but I imagine it’s still going to be greek to someone who doesn’t know the make/configure/automake patterns, compiler/linker flags, build system, etc. in use.
  
  I guess there’s tension between baiting people in by making golden-path things easy, and intimidating them from ever trying by requiring an understanding of the toolchains for the basics. That understanding is obviously helpful once the leaky abstractions leak, and I too have spent plenty of time rage-flailing my way through problems with projects that don’t square with undiscussed/taken-for-granted patterns.
  
  I’ve had a thought that might be related, which is whether it’d be a good time investment for the Nix community to ~compile a broad resource on toolchain/packaging patterns for well-behaved software in all ecosystems, both as a resource for people in the community who need to understand the patterns (and some anti-patterns…) to package something, and maybe as a resource to help project maintainers in many ecosystems quickly figure out how to structure their project to “go with the flow”?
  
  It might be cool to have something a bit like explainshell for a Nix derivation that can create an interactively-explorable hierarchy, from the expression level down to the shell functions the builder will be using, so that people can drill down into what it’s doing and why. Sounds like a lot of work though :)
  1. 10
    
    jaculabilis 3 months ago | link
    
    Disemboweling the stdenv and showing the gore in the derivation would make it much clearer what’s going on for people who know how to read the entrails, but I imagine it’s still going to be greek to someone who doesn’t know the make/configure/automake patterns, compiler/linker flags, build system, etc. in use.
    
    I just want to compliment your choice of metaphor here.
    
    You’re right that someone with no packaging experience, whose problem requires adding a compiler flag, will likely be just as confused by “where do I add the compiler flag in mkDerivation” as “what is a compiler flag” and “which compiler flag do I need”. But maybe a tutorial for “How to package with Nix” should resign itself to teaching the “how to package” parts as well as the “with nix” part. There’s got to be enough general teachable material about building, installing, libraries, documentation, etc. to cover without having to get into area-specific bits of wisdom.
    
    It might be cool to have something a bit like explainshell for a Nix derivation
    
    I think I’d settle for any kind of intellisense. Maybe I haven’t found the right extensions, but I feel like the most assistance I get writing Nix is syntax coloring, which just feels bad in 2023.
    1. 7
      
      jvns edited 3 months ago | link
      
      I’d personally love to see a guide to nix packaging that starts by going into the guts. I’m not an autotools expert or anything, but I’ve compiled many things in my life and it would be very helpful to see what’s going on under the hood. I think there are a lot of people out there who know what a compiler flag and a linker flag are but do not understand nix :)
      
      I tried to look through this 1600-line setup.sh bash script that seems to be at the heart of nix builds, but it’s a lot to process.
      1. 4
        
        jaculabilis 3 months ago | link
        
        I think ianthehenry’s series of learning Nix with only the manuals and experimentation did a fair job of diving into internals when he needed to understand something, though it’s far from being a systematic walk through all the aspects of nix packaging. I don’t think I could have figured out what he did without asking someone for help.
      2. 1
        
        rokgarbas 3 months ago | link
        
        If you want to understand how (and why) things work under the hook your best bet is to go over the “Nix pills” -> https://nixos.org/guides/nix-pills/
  2. 8
    
    pmc edited 3 months ago | link
    
    I think it would be reasonable to require autotools-based packages to have to use a pkgs.buildAutotoolsPackage wrapper (a la pkgs.buildPythonPackage) instead of having stdenv just magically do things when using mkDerivation. That way, you can direct new packagers to see how buildAutotoolsPackage is implemented using the mkDerivation primitive.
    
    Or, maybe setup hooks should be replaced with a “mixin” system so they aren’t magically called when you add a package to buildInputs, and the existing magic in stdenv should be moved to separate mixins. (e.g. pkgs.stdenv.mkDerivation { mixins = [ pkgs.autotools.passthru.mixin ]; })
    1. 3
      
      asymmetric 3 months ago | link
      
      I think this mixins idea is how Guix does it, IIRC, and it seems much more explicit and sane.
  3. 3
    
    rski 3 months ago | link
    
    I think part of the problem with all of these are that they assume any familiarity with existing software norms around building/installing/packaging software in different ecosystems.
    
    one of the tings that really helped me with getting how to build packages with nix was learning about autotools in depth and the norms they implement, which definitely supports this idea. Unfortunately, https://www.gnu.org/software/automake/manual/automake.html as the first step to getting nix builds is quite far from appetizing.
  4. 1
    
    andyc edited 3 months ago | link
    
    Does a Nix derivation evaluate to a shell script that calls some pre-baked shell functions? i.e. the ones pointed at below:
    
    https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh
    
    I sort of got that idea from some comments, but not sure if it’s exactly true.
    
    If it’s true, it would probably help my mental model a little.
    
    I agree that the core distro-building patterns, aside from Nix, are extremely confusing themselves … I think a big mystery to me has been pkg-config. It seems like an informal set of conventions.
    
    And I just built a tool uftrace from scratch. It uses pkg-config , but that’s actually WRONG on Debian/Ubuntu, because it gets libpython3.7.so, while the actual library name is libpython3.7m.so.1.0 (note both the m for pymalloc, and 1.0 version, which I guess is ABI version)
    
    Instead you are supposed to use python3-config on those distros ! Not even sure where that comes from, or when it arose
    
    I patched this when building from source, but I need to file a bug about this …
    
    So I have been using Python for ~20 years now, and this still utterly confuses me … i.e. loading the Python interpreter as a shared library on a distro. When you add in Nix, it gets even more confusing
    1. 1
      
      abathur 3 months ago | link
      
      Does a Nix derivation evaluate to a shell script that calls some pre-baked shell functions?
      
      Pretty much. Here’s a gist (https://gist.github.com/abathur/917d722a16c65824663d762cdd2f4d22) with:
      
      a console log where I “built” oil (it just fetched from cache instead of building and this created a result symlink in my current directory to the oil in the nix store)
      
      a derivation.json that shows the output of nix show-derivation result/
      
      a default-builder.sh that’s a copy of what you can see specified at the top of the derivation in the args/builder keys
      
      a stdenv_setup.sh (a bit of a fictional name for gist restrictions and syntax highlighting) that shows what the default-builder.sh would be sourcing
      
      This isn’t the whole story. The rest of the derivation also factors into how the environment is set up before running, for example.
      1. 1
        
        andyc 3 months ago | link
        
        Thanks, that’s very helpful. Definitely should be a blog post :) I have never seen that despite years of Nix-related stories on lobste.rs.
        
        Some people might not be able to read shell, but there are many who can.
        
        Though the setup script is a bit messy, it doesn’t look as bad as I thought. I would say it’s closer to generating JSON than generating shell, though both are involved.
        
        I can sort of see the JSON is used to set up what’s “visible” to the build.
        
        It also points to the tiny default-builder.sh, which sources some attributes, then sources $stdenv/setup – and I guess the latter is fixed across all packages?
        
        And I guess you can choose genericBuild or some other function to call from it, on a per package basis.
        
        And there are lots of other hooks to override in the package.
        
        I’m actually building a mini containerized “semi-distro” now for Oil’s build deps :-) And I’ve look at a lot of other distros.
        
        3
        
        abathur 3 months ago | link
        
        Definitely should be a blog post :)
        
        Some parts of the oft-linked Nix Pills overlap with this. Particularly:
        
        6: https://nixos.org/guides/nix-pills/our-first-derivation.html
        
        7: https://nixos.org/guides/nix-pills/working-derivation.html
        
        8: https://nixos.org/guides/nix-pills/generic-builders.html
        
        19: https://nixos.org/guides/nix-pills/fundamentals-of-stdenv.html
        
        It also points to the tiny default-builder.sh, which sources some attributes, then sources $stdenv/setup – and I guess the latter is fixed across all packages?
        
        It isn’t fixed, though I think it is probably close in practice? Most derivations don’t explicitly set their own builder, but there are still a fair number that do.
        
        Of those, most still source $stdenv/setup and use at least part of it. But I could also find at least two special cases that didn’t without looking too hard.
        
        2
        
        abathur 3 months ago | link
        
        Julia of course turned around and wrote a very approachable version of this :)
        
        https://lobste.rs/s/isligm
        
        2
        
        andyc 3 months ago | link
        
        Yup, great post! I think the difference is that it shows all the error messages you get if you DON’T do the right thing, which helps comprehension
        
        I’m used to seeing all those same errors when building software from scratch

6

proctrap 1 year ago | link | on: How do you tell if a problem is caused by DNS?

It’s not DNS -> it’s never DNS -> it can’t be DNS -> it’s DNS
1. 5
  
  kqr 1 year ago | link
  
  Huh, in my experience it’s almost always DNS.
  1. 2
    
    jvns 1 year ago | link
    
    what do you mean by this? I’m curious because i hear “it’s always DNS” a lot and that’s not my experience at all – I’ve found that some networking issues are caused by DNS and some aren’t.
    
    It does feel like weird/inexplicable issues are disproportionately likely to be caused by DNS though, because non-DNS problems are more straightforward to diagnose.
  2. 1
    
    joshka 1 year ago | link
    
    It’s not DNS -> it’s never DNS -> it can’t be DNS -> it’s DNS
    
    This was possibly a reference to https://www.cyberciti.biz/humour/a-haiku-about-dns/
    
    It’s not DNS
    
    There’s no way it’s DNS
    
    It was DNS

6

mayli 2 years ago | link | on: The Case of the 50ms request

Nice game to demonstrate a common weird issue. Is there a framework or it’s built from scratch?
1. 17
  
  jvns 2 years ago | link
  
  It uses Twine. The source is here https://github.com/jvns/twine-stories and I wrote a blog post about the project here https://jvns.ca/blog/2021/04/16/notes-on-debugging-puzzles/
  1. 5
    
    hwayne edited 2 years ago | link
    
    …this is giving me a lot of teaching ideas.

1

learnbyexample 2 years ago | link | on: A tool to spy on your DNS queries: dnspeep

Tried it, haven’t heard/used such a tool before. Going to log for a few days and then analyse. Seems like it doesn’t work for private/incognito browser session though.

Is there a tool that’ll also log how much bandwidth is being used per site? Need not be too accurate, just want to get an overview.
1. 5
  
  jvns 2 years ago | link
  
  That’s surprising – I’d definitely expect it to work in an incognito session (unless your incognito session is using DNS-over-HTTPS).
  1. 1
    
    learnbyexample 2 years ago | link
    
    I just checked again and now it works. I already had a private session open yesterday, I think I refreshed the site and it didn’t give me any results. Sorry for the confusion.

2

zladuric 2 years ago | link | on: A little tool to make DNS queries

I like how the article started:

This is great if you’re used to reading it and if you know which parts to ignore and which parts to pay attention to, but for many people this is too much information.

Now, I thought it’s going to be a bit more in that perspective, but it looks more like a simplified version of, say, intodns.com or similar tools that already exist. They cleaned the information up, but what else can you do with it now?

So it should either stay exactly the same: super simplified look into the DNS records, for info purposes for muggles - or it should add more features to be actually useful for something else.

I do bet that the author had a lot of fun making the tool though :)
1. 10
  
  jvns 2 years ago | link
  
  hi, author here! I’m curious about what you were expecting – I’m considering adding more features (like maybe reverse DNS), but for me just looking up the value of a given record for a domain is all I want to do 90% of the time.
  1. 2
    
    sanxiyn 2 years ago | link
    
    I think choosing a name server would be useful.
    1. 2
      
      jvns 2 years ago | link
      
      I’ve been thinking about that! Can you give an example of a different nameserver you’d like to use instead of 8.8.8.8? (like do you want to query the authoritative namerserver for the domain? or do you want to just look at the results from a different public recursive dns server?)
      1. 2
        
        Hail_Spacecake 2 years ago | link
        
        Maybe you want to make requests against your own ISP’s nameserver, whatever it is. Or make requests against the mullvad public non logging DNS server at an IP address I can’t remember off the top of my head. Or if you are setting up your own nameserver and want to test it.
      2. 2
        
        0x2ba22e11 2 years ago | link
        
        One DNS issue I’ve had to debug before is where there are several DNS servers and one of them is responding with the wrong value. So I look up the relevant NS records and then query each of them individually to see if there’s a mismatch.
        
        Another thing I’ve cared about in the past is split horizon DNS, where a particular server has one A record on the public internet and a different A record inside our LAN so it could be seen from both. (In retrospect, perhaps doing this with routing entries to make the external IP address with on our LAN would have been better?)
        
        It comes up rarely but “I’m testing our in house custom DNS server software before we deploy it” (yeah that sounds like a bad idea, doesn’t it? It was.) so obvs I wanna send queries to it instead of the live NS servers.
        
        The last common use I can think of was “dig +trace $name” to quickly get a view of the whole chain from TLD on down. Used to use this to diagnose issues where there were 3 oe more levels of domain servers.
      3. 2
        
        fs111 2 years ago | link
        
        some people simply do not like to use google infrastructure, so at least having 9.9.9.9 would be nice.
      4. 1
        
        sanxiyn 2 years ago | link
        
        I am mainly interested in 1.1.1.1 and 9.9.9.9.

9
hauleth 2 years ago | link | on: ninja: a simple way to do builds
Is it really that complex to write Makefile for such simple project? And in many projects it can became even simpler as there are implicit rules in GNU Make for C/C++/TeX/etc.
```
OUTPUTS = variables.pdf

.SUFFIXES: .svg .pdf
.PHONY: all

all: ${OUTPUTS}

clean:
	rm -rf *.pdf

.svg.pdf:
	inkscape "$<" --export-text-to-path --export-pdf="$@"
```
1. 12
  
  friendlysock 2 years ago | link
  
  The makefile is simple, but knowing which incantations to use to make it simple is harder.
2. 3
  
  mattgreenrocks 2 years ago | link
  
  Yeah, I was a bit surprised at Julia writing off make as arcane when she’s covered things like ptrace and kernel hacking elsewhere. Poorly documented is a better descriptor for make, and other people have done a decent job of fixing that.
  
  Maybe she will see this and write another article on how make isn’t too arcane. :)
  1. 3
    
    jvns 2 years ago | link
    
    I’m (obviously) always interested in learning about things I thought were arcane and complicated are not actually that complicated! I’ve never read anything about make that I thought was approachable, but I haven’t tried too hard :)
    
    I think the thing that throws me off about Make is that by default it seems to decide how to build something based on its file extension, and I find this pretty hard to reason about – what if I have 2 different ways to build PDFs files? (which I do).
    1. 2
      
      Boojum 2 years ago | link
      
      I’m no Makefile expert, but the things with the file extensions are implicit rules and are basically there to save you from having to copy/paste the same basic rule template for each file. You can always override them with your own explicit recipe since those take precedent:
      
      special.pdf: special.svg do-this-command # indent via TAB! then-that-command # ...
      
      For simple one-off uses, I usually ignore all the details of the implicit rules and just spell everything out explicitly like that. I usually don’t even bother with automatic variables like $^ and $@ since I can never remember which is which. I’ll just copy/paste and search and replace all the recipes, or write a quick Python script to generate them. It’s hard to get much simpler than that. Just remember that the first target in the Makefile is the default if you type make with no targets.
    2. 2
      
      spc476 2 years ago | link
      
      I found the GNU Makefile manual to be very good about describing how make works.
      
      The two different ways you make PDF files—do both methods use the same input files? If not, implicit rules can work just as well if the inputs are different (at work, I have to convert *.lua files to *.o files, in addition to *.c files to *.o).
  2. 3
    
    andyc Oil Shell Developer 2 years ago | link
    
    FWIW I wrote 3 substantial makefiles from scratch, and “gave up” on Make. I use those Makefiles almost daily, but they all have some minor deficiencies.
    
    I want to move to a “real language” that generates Ninja. (either Python or the Oil language itself :) )
    
    Some comments here but there are tons of other places where I’ve ranted about Make:
    
    https://www.oilshell.org/blog/2017/10/25.html#make-automatic-prequisites-and-language-design
    
    I think my summary is that the whole point of a build system is to get you correct and fast incremental and parallel builds. But Make helps you with neither thing. Your makefile will have bugs.
    
    I never condensed my criticisms in to a blog post, but I think that’s the one-liner.
  3. 1
    
    gecko 2 years ago | link
    
    I hear you, and I know that make is the standard make-type things are a bit more legitimate than the article I’m referencing.
    
    That said, the amount of inexplicable boilerplate in something like this (from up in the thread) are why I’m super-empathetic to people who find Makefiles too complicated. And that’s ignoring things like the tab-based syntax or the fact that Makefiles effectively yoink in all of sh (or, honestly, these days, bash) as a casual comprehension dependency.
    
    Makefiles can be very simple; I absolutely grant you that. They generally aren’t. And even when they are, they require a certain amount of boilerplate that e.g. ninja does not have.
3. 2
  Boojum 2 years ago | link
  That’s about how I’d write it for a good hand-written yet scalable Makefile, but it’s worth noting that you can go even simpler still here:
  
  pdfs/variables.pdf: variables.svg inkscape variables.svg --export-text-to-path --export-pdf=pdfs/variables.pdf
  
  Easy enough that you can do it from memory. This is also the style that I go for when writing a quick script to generate a Makefile.

8

friendlysock 4 years ago | link | on: New zine: Help! I have a manager

I love her work, but this is naked product advertising. Flagged.
1. 13
  
  jvns edited 4 years ago | link
  
  I also find it weird (as the author) for this to be on lobsters – to me if something costs money to read, I’m not sure that it makes sense for it to be posted to a site like this, because most folks won’t be able to read it and then it’s not possible to have a discussion about the content :)
  1. 3
    
    friendlysock 4 years ago | link
    
    Indeed. Congratulations again though on your work! :)
  2. 1
    
    phaer 4 years ago | link
    
    Thanks a lot for your work, I am a big fan myself and distributed a bunch of print copies of your CC-licensed work among friends and colleagues! :)
    
    I could imagine that the fact that gumroad seems to support only credit card payments hinders some customers who, like myself, live in a region were credit cards are not as common as they seem to be in the US. I bought a pre-paid card just for your zines, but supporting at least something like paypal would be great.
    1. 2
      
      wunderseltsam 4 years ago | link
      
      Gumroad now has paypal support for me. (I live in Europe, maybe it’s a regional thing?) When I click to the payment dialogue, it shows the credit card one, but also a button where I can pay with paypal.
      
      I tried that out last week, when I bought the Bite size linux zine. :)
      1. 1
        
        phaer 4 years ago | link
        
        You are right, that works for me too. I just had missed the paypal button, thanks for the hint!

10

o5r 5 years ago | link | on: Working remotely, 4 years in

I’m actually going to send this article to a few peers, maybe even my manager. I live about 1h away from the office when there’s no traffic. If I want to encounter no traffic, I have to leave home at the latest at 5 AM, or no sooner than 8:45. This essentially means working from 6:00-6:30 to 15:00, or from 9:45-10:00 to 18:00. There’s also traffic on the way back.

I don’t want to get closer to the office because: this is my childhood neighborhood, I live right across the street from my parents (which is great when you have 4 kids and/or a cordial relationship with your parents), and the land + house that we have would be worth 2.5 ~ 3 times as much if I moved to a closer location.

Logistics aside, this is also much better for my concentration, I get much less often interrupted too. I need more of this.
1. 14
  
  jvns 5 years ago | link
  
  one really cool side effect of having a remote-friendly team is that it makes it normal for people to work from home occasionally – one person on my team has kids and he’ll pretty frequently work from home so that it’s easier for him to pick up his kids from school or something, and it’s not an issue at all since half the team is remote anyway :)
2. 5
  
  dfrey 5 years ago | link
  
  My commute isn’t quite as bad as yours. When I started my current job, I negotiated the expectation that I would be working from home about half of the days. I now work from home 2 days a week some weeks and 3 days a week on other weeks. My commute is still terrible half the time, but on average it’s not a bad commute. I would phrase it as “I’m feeling burnt out because of my commute and I don’t think I can keep doing it 5 days a week for much longer. I would like to begin working X days per week from home so that I can continue to work for the company”. This doesn’t send an aggressive ultimatum, but it signals that you’re probably going to quit soon if they don’t accommodate you. If that doesn’t work, it’s probably time to find a new job.

2

michaelochurch edited 5 years ago | link | on: Writing performance reviews with positive constructive feedback

Ugh. I hate performance reviews, whether writing or receiving. There is literally no upside. You’re giving HR words that may be used against someone– or against you.

Feedback should be verbal and direct. You should be able to get a sense of who can handle tough feedback and who might respond better to gentler language. There’s no formula for it.

As for written feedback and the futility thereof, management knows (or should know) who the high performers are and will reward and promote them. If they get it wrong, there’s nothing you can do as a grunt to change their minds. The only thing that comes from putting stuff in writing and handing it over to management is risk.

Of course, if you’re a manager, you have to use performance reviews to protect your people so they can focus on their work rather than politics. It’s part of your job. You can’t not write reviews because you don’t like doing it. But reviewing peers is, in my view, inadvisable. If you have something to say, deliver it privately.
1. 4
  
  jvns 5 years ago | link
  
  this is an interesting comment! At my job “just don’t write peer reviews” isn’t really something I can do, which is why I wrote this post – this is a small but mandatory part of my job, so how do I handle it when it comes up?
  
  The only thing that comes from putting stuff in writing and handing it over to management is risk.
  
  Part of the reason I wrote this post is that I feel like putting negative feedback in writing where I have no idea who will see it is a bit risky! But I feel like writing down positive feedback is a lot less risky, and so my hope is that if I mostly stick to that then the outcomes will be okay, and maybe I’ll even say something useful by accident.
  1. 1
    
    michaelochurch 5 years ago | link
    
    I’m late to come around to this; I’ve been in Mexico for the past week.
    
    I agree. Ratting someone out to management is bad form and a way to make a permanent enemy, but putting positive feedback (or even having a conversation with the person about what you should be saying) is not a bad idea. What this ought to be, instead of a way for workers to slag each other (to executives’ benefit), is a way for them to help each other out and build up each other’s stories. If used properly, these tools can be used for good (i.e., in workers’ favor instead of executives’).

2

inactive-user 5 years ago | link | on: How do you spy on a program running in a container?

I don’t know enough about Linux namespaces. Couldn’t this just be handled with chroot?
1. 2
  
  jvns 5 years ago | link
  
  I don’t know what you mean by “couldn’t this just be handled with chroot” (what solution are you suggesting exactly?), but mount namespaces are not the same as chroot! here are a couple of articles that might be helpful: http://man7.org/linux/man-pages/man7/mount_namespaces.7.html, https://lwn.net/Articles/689856/
  1. 1
    
    inactive-user 5 years ago | link
    
    You can’t access the cgroup namespaces from the host system? That’s crazy. This is such a crucial feature of jails and zones.

9

apy 5 years ago | link | on: I think I found a Mac kernel bug?

She should be able to use dtrace to investigate deeper.
1. 13
  
  jvns 5 years ago | link
  
  Where would you start with using dtrace to investigate this? I already know the responsible system call – is the idea that I could use dtrace somehow to trace what resource is under contention inside the kernel?
  
  I’ve basically never used dtrace except running dtruss occasionally to look at system calls so it’s pretty unclear to me where to start.
  1. 12
    apy 5 years ago | link
    
    I’m not a dtrace master, or even low-level amateur but this might be a first start:
    
    sudo dtrace -n ':::/pid == $target/{@[stack()] = count();} tick-5s {exit(0);}' -p PID_OF_YOUR_STUCK_PROGRAM
    
    That will sample the kernel stack of the process any time something happens with it and count them, then after 5 seconds it’ll print the kernel stacktraces out in ascending order of count. You can also look at -c instead of -p
    
    That’ll probably give you way too much stuff, though. You can only get the stack traces when it does a syscall with:
    
    sudo dtrace -n 'syscall:::/pid == $target/{@[stack()] = count();} tick-5s {exit(0);}' -p PID_OF_YOUR_STUCK_PROGRAM
    
    Maybe what you could do is use dtruss to figure out what it’s stuck in (probably a syscall?) and then use dtrace to see what’s going on there. For example, to see what syscalls were called in doing sleep 10 I did (on FreeBSD):
    
    sudo dtrace -n 'syscall:::/pid == $target/{}' -c "sleep 10"
    
    And I got a bunch of output, where it clearly sat for 10 seconds on:
    
    2 80973 nanosleep:entry
    
    Then, to see exactly what goes on in the kernel for this process between nanosleep:entry and nanosleep:return, I did:
    
    sudo dtrace -n 'BEGIN {trc = 0} syscall::nanosleep:entry /pid == $target/ {trc = 1} syscall::nanosleep:return /pid == $target/{trc = 0;} ::::/pid == $target && trc == 1/{@[stack()] = count();}' -c "sleep 10"
    
    Kind of hard to read but if you pull the stuff in quotes out you can see it’s using a variable called trc and for this pid, when nanosleep is entered it sets the variable to 1, and when nanosleep returns it sets trc to 0. Then, for any probe for this pid if trc is 1 then record the kernel stack. I got a bunch of output in that.
    
    Hopefully that is helpful. I’m not near a trace wizard so I’m sure there is some much more clever things one can do, but that might be a start to digging.
    
    You can see probes available to you with dtrace -l.

8

jvns 5 years ago | link | on: How Raft Works

One can imagine a scenario where one of the nodes has a latency higher than it’s election timeout, causing it to continuously start elections

I have used Raft in production and can confirm this is a real thing that happens. Here’s an issue on the etcd repo discussing this problem https://github.com/coreos/etcd/issues/7970. basically: “a 5-node cluster can function correctly if two nodes are down. But it won’t work if one node has slow disk.”. Which is a weird failure mode!

also I found this post valuable, I’d never read the raft website/paper but this appeared in my RSS reader and it caused me to actually learn more about how Raft works!
1. 3
  
  irfansharif 5 years ago | link
  
  Using Raft’s PreVote extension should alleviate this issue, though etcd’s current PreVote implementation could be more stable. See https://github.com/coreos/etcd/issues/8501, https://github.com/coreos/etcd/pull/8517, https://github.com/coreos/etcd/pull/8288 and https://github.com/coreos/etcd/pull/8334.

3

lain 6 years ago | link | on: julia's drawings

I don’t understand why these, frankly, childish drawings of things you can learn from reading a wikipedia article have reached this level of popularity.
1. 24
  jvns edited 6 years ago | link
  I’d be the first to point out that I’m not a particularly talented artist. So why do I spend my time drawing comics, when I’m not even that good at drawing?
  
  it’s because I’ve found they’re, in some situations, such a great way to communicate that I feel silly not taking advantage of it
  
  A few reasons:
  
  Comics are great for summarizing. I wrote a blog post at work about how we do service discovery. I started out with just writing 1200 words, and trying to be as informative as possible. I thought it was pretty good. But then I drew a comic summarizing a main ideas! A really common reaction was “wow, the blog post took me 15 minutes to understand, but after reading the comic I understood the main ideas in 30 seconds”. This is awesome because somebody can see the main ideas and decide whether they want to continue reading.
  
  I can trick people into reading about concepts that they might otherwise think are too hard or not useful or not fun. I wrote a zine called “linux debugging tools you’ll love” (here). This zine discusses a lot of tools which are traditionally considered a little bit advanced. Because of that, a lot of people don’t even know they exist, or might not consider trying them! All kinds of people have told me “julia, i read your zine, and the next day I used a tool in it to fix a bug at work”.
  
  They’re an awesome way to introduce people to new ideas. Anyone can learn about /proc! It is not that complicated. But a lot of people don’t even know /proc is interesting. This comic about /proc took me maybe 15 minutes to draw. I posted it on twitter and it got 180,000 impressions. A lot of people replied to me “wow, that’s so useful, I didn’t even know that that existed, I’m going to go learn more about that now!!”. They will probably learn more about it by reading the man page or Wikipedia or some more traditional means. But they learned about it from a silly drawing :)
  
  Basically, I do it because I love telling people things about computers that help them (“omg julia, i had no idea about this, this, this is so useful”), and silly drawings turn out to be a pretty effective way to do that.
  1. 2
    
    lain 6 years ago | link
    
    Thanks for the great answer, but as I’ve already been downvoted as a ‘troll’, I won’t comment any more.
2. 8
  
  danielrheath 6 years ago | link
  
  Because they have a carefully chosen tone that maintains a sense of psychological safety. If you’ve never been told off for learning I doubt it’d make a lot of sense.
  1. 2
    
    lain 6 years ago | link
    
    Yeah, it doesn’t make much sense for me to need psychological safety when learning about /proc.

4

nickpsecurity 6 years ago | link | on: A few drawings about Linux

I think someone making books for Linux beginners should consider offering her a job or gig making imagery for both common and complex stuff. She beautifully presents system calls, proc, signals, and killing processes. Her graphics might help the learning process for visual learners.
1. 8
  
  jvns 6 years ago | link
  
  i’m a much better professional developer than professional artist, or at least I hope so given my art skills :) (and the job market is much better <3)
  
  probably going to stick to writing software.
  1. 1
    
    nickpsecurity edited 6 years ago | link
    
    Well keep up your hobby. We definitely enjoy it. Might even turn into something profitable on the side one day if you’re lucky. :)
  2. 1
    
    jfb 6 years ago | link
    
    Stick to writing software, by all means, but please please please continue to write and draw about it. I know I’m not alone in admiring your work.

2

jvns 9 years ago | link | on: Should we adopt a small set social rules for our community?

I went to Hacker School and I found that the social rules worked very well.

I’m not sure the Lobsters needs to be a place free of discussions of sexism. That works well at Hacker School for reasons that I can’t quite articulate, but there’s certainly a place for discussions of bigotry in the world at large.

3

jm 9 years ago | link | on: Tag proposal: 'community'

Could call it “Dev Culture”
1. 1
  
  jvns 9 years ago | link
  
  I like “dev culture”.
2. 1
  
  jm 9 years ago | link
  
  But, so far, all tags are one word. So maybe not.

5

SeanTAllen 9 years ago | link | on: Questions I'm asking my technical interviewers

I do an awful lot of interviews. Sadly, few people ask questions like this.

It’s too bad because at least with me, I’m far more impressed with candidates who turn the tables on me and interview me as well. Far too many end up just trying to be “amendable” and “impress”. My advice to people interviewing is, don’t worry about fucking it up. Think about what you want in a job and make sure the place you are looking at will give you what you want. If the place is really good, the interviewer is probably going to be delighted that you asked more probing questions.
1. 5
  
  jvns 9 years ago | link
  
  I worry a bit about this advice – asking tons of questions is working well for me right now, but it also depends on the market and how much privilege/power you have as an interviewee.
  1. 2
    
    SeanTAllen 9 years ago | link
    
    and if you don’t and end up somewhere were you only got the job because you didn’t ask questions? it fills an immediate economic need but we are talking about software engineering jobs. the ratio of supply to demand is greatly skewed towards demand.
    1. 5
      
      kelsey 9 years ago | link
      
      Eh, I’m totally on board the ask-lots-of-questions train–I have been known to ask more questions of my interviewer than they asked me!–but let’s not pretend there’s not risk involved. For example, a company’s parental leave policies are a GREAT (maybe one of the best) indicators of their approach to gender relations and work-life balance, but as a woman of childbearing age there is no way in hell I’m risking an offer by bringing that up in an interviewing situation, even though I know that legally and in a perfect world that is not supposed to influence hiring decisions. (FFS, I feel the need to disclaim that I have no interest in taking advantage of anywhere’s paternal leave anytime soon, in case someone I know in real life is reading this!) Most of the questions under “quality of life” have similar implications that can potentially be negative; if you’re working against various implicit biases to begin with the risks involved in even planting that seed of doubt may not be acceptable. There’s other skews than demand over supply in the software engineering world…
      1. 3
        
        SeanTAllen 9 years ago | link
        
        i wasn’t trying to say there is no risk. i’m suggesting that not asking those questions mean you risk ending up somewhere you aren’t going to want to work and that has to be balanced again the need for gainful employment. we are fortunate as engineers that we can be more selective. unfortunately, due to various biases, some of us can afford to be more selective than others, but as a profession, we are blessed that we can regularly take this into account.
        
        3
        
        kelsey 9 years ago | link
        
        Agreed! I’d add that if you’re on the hiring/recruitment side it’s worth thinking about how your potential hires are weighing that risk vs information calculation. If the answers to any of those more risky questions are a selling point for your company, getting out ahead of that dilemma by volunteering that info up front or highlighting it in your job listings is a great idea.
        
        2
        
        SeanTAllen 9 years ago | link
        
        i try to do that as the vast majority of the time, i’m on the hiring/recruiting side.
  2. 1
    
    journeysquid 9 years ago | link
    
    It doesn’t need to be tons of questions, but 3 good probing questions can easily give you a feel for what the company and environment are really like.
2. 1
  
  lambda 9 years ago | link
  
  My advice to people interviewing is, don’t worry about fucking it up.
  
  I think I understand the intended mindset this is meant to advocate, but I wonder if this works when I know I’m interviewing for my dream job? How do I not worry about the interview, when I, as the interviewee, already know the job is precisely what I want? This seems like a special case where my primary objective has to be to impress and possibly show I’m amenable (we’re talking about a dream job after all).
  
  Further, what if I give the interviewer the wrong impression by asking questions that could either be interpreted as pedantic or intrusive? What if I give the interviewer the impression I don’t trust the company to get even the basics right? Leading with the impression that I’m concerned at once with small details most decent places are bound to get right or showing mistrust doesn’t seem like something an employer is likely to respond positively to. Conversely I do see how some of these questions might be interpreted positively, as showing an interest in the particulars of the job and how they relate to you as a potential employee–certainly that should be a positive signal to the interviewer.
  
  However, if I know I’m interviewing for my dream job, I’m probably not wondering if they use version control, for instance. And asking a question like that seems like it could be a bit of a red flag or at best noise that could be avoided in favor of more relevant discussion. So maybe this list is potentially less applicable to an interview you know you can’t fuck up. In which case, I’m not sure I can walk into an interview without worrying if I’m going to make a misstep.