1. 10
  1.  

  2. 3

    Ah yes, the beauty of x86 instruction encodings :)

    I’ve been working here and there on an assembler for a bit now. The encoding is ridiculous. I pursued a semi-exhaustive testing strategy based on the instruction tables, and have already found a few bugs in nasm and gas as a result.

    Additionally, because of the compositional nature of the encoding, there are many undocumented combinations which appear to do something instead of simply #UD. Most disassemblers will decode these, too, often in inconsistent ways. Nice way to hide your payload: stick in an instruction that has a different length according to the disassembler vs the hardware so you get misaligned. Defensively, you can’t win in the general case: there are differences between intel and amd!

    And of course there are the infamous xh registers which for some reason no extant assembler warns you about using even though they are slow and have errata…¯\_(ツ)_/¯

    1. 2

      ‘Beautiful’ indeed, I guess it is what you get with multiple layers of backwards compatible additions :P. It feels like it was designed by a mad genius under time pressure.

      1. 1

        P..S. to the OP (I guess you wrote this?) I really suggest switching to a table-based approach if you want anything approaching full coverage. It’s worth it, promise. Currently my tables are 2x the loc of my encoder, and it will probably be closer to 5-10x by the time I’m done; doing that in-like would be a massive bloat. Even tcc uses tables.

        1. 1

          OP (I guess you wrote this?)

          Yep.

          I really suggest switching to a table-based approach if you want anything approaching full coverage.

          Link to your assembler? I kind of use tables embedded in the code - though It would be nice to do it in a more principled way. I have full coverage of instructions that cproc generates, so in theory I can assemble all the C code in a system, but i definitely would like to be able to handle more instructions in an easy way.

          1. 1

            Not open yet, But I mean a declarative approach where you say ‘thusly is ADD encoded’ and then use that declaration to figure out how to encode an instruction which happens to be called ADD. Instead of saying ‘ok, if the instruction is ADD then do this; otherwise do that’.

            Such an approach makes optimization easier. Given an instruction (such as ADD), there are many possible encodings, all in terms of different source operand types. So you can identify all the encodings which satisfy the given sequence of operands, and then pick the shortest one (or err if no encoding satisfies).

            E.G. here are nasm’s tables, from which c code is generated. Here’s tcc, which does it directly in c (using preprocessor macros for easier expression).

            1. 1

              If you say something is better but do not let us see the code - then I don’t know how to reply to that. The nasm table is slightly disheartening by the sheer size. I think the set of instructions people actually use must be significantly shorter than that - I would be curious to know exactly how many instructions are used in the linux kernel for example.

              Minias itself is less code than the tool used to process the nasm table, so a table might make things worse if the project only aims to support the listed C compilers. That being said, a simplified or ‘mini’ table may be all that is needed to make things even better.

              1. 3

                If you say something is better but do not let us see the code - then I don’t know how to reply to that

                I don’t understand what you’re getting at here. My code does not do anything special that other assemblers don’t; I linked code of two other assemblers using similar techniques, which are representative.

                Minias itself is less code than the tool used to process the nasm table, so a table might make things worse if the project only aims to support the listed C compilers

                In fairness, nasm is famously bloated :)

                I do use a preprocessing script—albeit a much smaller one—but given your goals a tcc-like approach is probably more appropriate.

                1. 1

                  As an update - I rewrote minias to be more tabular/closer to the intel manual - it is about 1kloc for the code, and 1kloc for the parser/tables.

                  Overall I am more happy with it - thanks!

                  1. 1

                    :)

                  2. 1

                    I don’t understand what you’re getting at here

                    I just meant, I can’t really evaluate what you are saying is true because everyone is biased that their own approach is best and I can’t see the code myself. I do want a good table system if it makes the code simpler though.