That’s not a great selling point, as X.509 is widely reviled. (Peter Gutmann has a masterful slow-burn takedown disguised as a “style guide”.)
That’s not entirely ASN.1’s fault, but I don’t know any good reason to use it unless you’re forced to mess with certs. There are newer, better formats like FlexBuffers and CapnProto that do the same things.
FlexBuffers and CapnProto are different things from ASN.1. These are encoding formats, that also have IDL. ASN.1 is not encoding format, just IDL. Encodings are separate from ASN.1 and technically it would probably be possible to use subset of ASN.1 to describe the FlexBuffers, CapnProto, ProtoBuffers, or other encoding format you like. ASN.1 already supports for example JSON and XML.
To anyone reading this thread: after years and years of experience with attempting ASN.1 interop, STAY AWAY from ASN.1!
Run, hide, lie, do whatever is necessary to avoid this design-by-committee monstrosity.
Yes, it looks just fine on paper, but in actual implementation you will enter a never-ending hell of “why isn’t that working?” followed by the most persnickety language-lawyering from everyone involved.
Read Peter Gutmann’s tirade about X.509 with the mindset of “How would I interoperate with this?” and you might begin to understand.
I use it everyday at work (telco), so much I can read DER format ASN.1 from hex dumps directly. (Newly designed protocols and formats also choose to use it, it’s not all old).
Yep, ASN.1 is used for GSMA TAP3 standard (and the older TAP 2).
TAP3 is a way for telco carriers to exchange Call Detail Records (CDRs) amongst each other for the purposes of billing each other (The use cases I am familiar with - were for Roaming).
I think, also, this standard was used when a billing system (for a telco) would use a 3rd party bill printing solution.
The CDRs would be encoded in ASN.1 and sent to print vendor (together with other invoice related data).
Although that part is now quite faint in my memory :-).
A number of ASN.1 use cases are ‘hidden’ behind a higher level protocol spec.
Which is why searching for, say TAP3 , bring more ‘business use case links’.
We were in the same class with the author! A really brilliant person! And the post reminded me of a post in Slashdot which I cannot find now that claimed that ASN.1 was a better format than XML.
Such statement would be on the “less brilliant” end of spectrum. This would be like saying that DTD is better than HTML - not comparable at all. XML is encoding, while ASN.1 is IDL, in fact, you can use them together in form of XER.
Ah, yes, ASN.1, whose encodings were so opaque and whose implementations were so painful that RFC 2692 had as a requirement ‘No library code should be required for the packing or parsing of SPKI certificates. In particular, ASN.1 is not to be used.’
Granted, RFCs 2692 & 2693 were completely ignored along with SPKI in general, and instead the world works on cobbled-together hacks atop X.509 certificates (defined in ASN.1), so … maybe ASN.1 had certain important survival characteristics.
I think Erik Naggum had good things to say about it, so it probably isn’t completely terrible, but my own opinion is that, like JavaScript, JSON, C and XML its greatest impact may be the alternatives which wilted in its shadow.
Where has this been all my life, it sounds awesome!
Also: Again with the same schtick of starting the article out with a couple of pages of what to not do (use architecture-specific memo dumping as if it were serialization) before we get to what to do (establish a deserialization schema so you can efficiently use DER or PER). That’s my latest hangup & pet peeve.
There’s also Thrift from Apache (#changethename) and a few others. I love this kinda stuff.
But it is worth to remind that ASN.1 is from 1984, while JSON is from around 2000, and Rust is 2010. And… That is a full text search, which doesn’t imply the problem is with ASN.1 (or JSON, or Rust), but that the keyword shows up in any text of the CVE, which my include the description of the product affected.
So, a modern analog might be something like google’s protocol buffers? Both start from a type definition which gets “compiled” into auto-generated source code in your favorite language. Then there are other binary encoding formats like CBOR which start from your own source code and move towards a compatible serializer / deserializer by specifying numerical field IDs for each serialized property on a type / struct.
Never surprises me when I find out something “new” is just a rewrite of something quite old :)
Protocol buffers serve a similar function, but for cryptographic applications it’s important to keep in mind that they don’t provide a unique encoding. In particular, a lot of things under the hood in protos are variable-length integers, and these integers are not required to be represented minimally. ASN.1 DER, in contrast, guarantees that any permissible value will have a unique encoding.
That is the thing - CBOR and ProtoBuffers are encodings that also offer some kind of IDL, while ASN.1 (that is why it is called Abstract Syntax One) is just IDL with multiple encodings. Technically it should be possible to use ASN.1 for defining ProtoBuffs, CBOR, or any other encoding you like. It already supports JSON (JER) and XML (XER).
The reason why we now have a lot of different concepts like ASM.1 that use their own IDL is that ASN.1 grown in a lot of cruft and the spec is abysmal. It would be great to have something like “ASN.2” that would remove all the cruft and would start from mostly “clear slate”, but I do not think that it will become a thing any time near. Too many people focused on looking what Google will publish next.
CBOR is just “binary JSON”, it is a self-describing encoding. There is no IDL in CBOR. There is an external CDDL spec (or good old JSON Schema) but.. meh.
Protobuf and the performance focused alternatives are actually IDL-first. They just happen to have one standard encoding because there are benefits to developing IDL+encoding together as a cohesive system. You could use alternative encodings, but the only reason anyone actually does is to have a human-readable JSON representation for debugging.
a modern analog might be something like google’s protocol buffers?
No. protobuf is just another encoding. You can almost certainly define a protobuf-like encoding rule for ASN.1. This is the problem with all of these flavor-of-the-day formats. All they do is fracture this space.
Protobuf is IDL first, encoding second. The main part of protobuf is the proto(2/3) language for describing data structures with deep support for format evolution (backward/forward compatibility).
Also, “flavor of the day”? Really? Protobuf has been public / open source for 13 years now.
As far as I can tell protobuf provides nothing that ASN.1 does not already provide. In my experience it’s worse. Try doing polymorphism in protobuf to see what I mean.
You could define a new wire format for it. But why do that when ASN.1 already has the infrastructure for all this?
ASN.1 is really a cool and under appreciated technology. We use it at work to transfer data in BER encoding. Unfortunately there are not that many open source implementations available. There is one python lib that can convert encodings on the fly.
It’s still in use, by you, every day: X.509 certificates, such as those used for TLS, are defined using ASN.1 and encoded using DER.
That’s not a great selling point, as X.509 is widely reviled. (Peter Gutmann has a masterful slow-burn takedown disguised as a “style guide”.)
That’s not entirely ASN.1’s fault, but I don’t know any good reason to use it unless you’re forced to mess with certs. There are newer, better formats like FlexBuffers and CapnProto that do the same things.
FlexBuffers and CapnProto are different things from ASN.1. These are encoding formats, that also have IDL. ASN.1 is not encoding format, just IDL. Encodings are separate from ASN.1 and technically it would probably be possible to use subset of ASN.1 to describe the FlexBuffers, CapnProto, ProtoBuffers, or other encoding format you like. ASN.1 already supports for example JSON and XML.
Also a lot of your calls may be at some point routed using h323 which is also asn.1 defined.
To anyone reading this thread: after years and years of experience with attempting ASN.1 interop, STAY AWAY from ASN.1!
Run, hide, lie, do whatever is necessary to avoid this design-by-committee monstrosity.
Yes, it looks just fine on paper, but in actual implementation you will enter a never-ending hell of “why isn’t that working?” followed by the most persnickety language-lawyering from everyone involved.
Read Peter Gutmann’s tirade about X.509 with the mindset of “How would I interoperate with this?” and you might begin to understand.
I use it everyday at work (telco), so much I can read DER format ASN.1 from hex dumps directly. (Newly designed protocols and formats also choose to use it, it’s not all old).
Yep, ASN.1 is used for GSMA TAP3 standard (and the older TAP 2).
TAP3 is a way for telco carriers to exchange Call Detail Records (CDRs) amongst each other for the purposes of billing each other (The use cases I am familiar with - were for Roaming).
I think, also, this standard was used when a billing system (for a telco) would use a 3rd party bill printing solution. The CDRs would be encoded in ASN.1 and sent to print vendor (together with other invoice related data). Although that part is now quite faint in my memory :-).
A number of ASN.1 use cases are ‘hidden’ behind a higher level protocol spec. Which is why searching for, say TAP3 , bring more ‘business use case links’.
We were in the same class with the author! A really brilliant person! And the post reminded me of a post in Slashdot which I cannot find now that claimed that ASN.1 was a better format than XML.
Such statement would be on the “less brilliant” end of spectrum. This would be like saying that DTD is better than HTML - not comparable at all. XML is encoding, while ASN.1 is IDL, in fact, you can use them together in form of XER.
Indeed I am not as brilliant as the author of the post.
Ah, yes, ASN.1, whose encodings were so opaque and whose implementations were so painful that RFC 2692 had as a requirement ‘No library code should be required for the packing or parsing of SPKI certificates. In particular, ASN.1 is not to be used.’
Granted, RFCs 2692 & 2693 were completely ignored along with SPKI in general, and instead the world works on cobbled-together hacks atop X.509 certificates (defined in ASN.1), so … maybe ASN.1 had certain important survival characteristics.
I think Erik Naggum had good things to say about it, so it probably isn’t completely terrible, but my own opinion is that, like JavaScript, JSON, C and XML its greatest impact may be the alternatives which wilted in its shadow.
Where has this been all my life, it sounds awesome!
Also: Again with the same schtick of starting the article out with a couple of pages of what to not do (use architecture-specific memo dumping as if it were serialization) before we get to what to do (establish a deserialization schema so you can efficiently use DER or PER). That’s my latest hangup & pet peeve.
There’s also Thrift from Apache (#changethename) and a few others. I love this kinda stuff.
https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1
That is an unfair, and low effort, point: do the same search for JSON, and the list is way bigger.
Even the darling of the moment, Rust, has a list bigger than ASN.1.
But it is worth to remind that ASN.1 is from 1984, while JSON is from around 2000, and Rust is 2010. And… That is a full text search, which doesn’t imply the problem is with ASN.1 (or JSON, or Rust), but that the keyword shows up in any text of the CVE, which my include the description of the product affected.
In summary:
As you noted, it’s good to click through and read the details, certainly before going off the beaten grammar and encodings road.
So, a modern analog might be something like google’s protocol buffers? Both start from a type definition which gets “compiled” into auto-generated source code in your favorite language. Then there are other binary encoding formats like CBOR which start from your own source code and move towards a compatible serializer / deserializer by specifying numerical field IDs for each serialized property on a type / struct.
Never surprises me when I find out something “new” is just a rewrite of something quite old :)
Protocol buffers serve a similar function, but for cryptographic applications it’s important to keep in mind that they don’t provide a unique encoding. In particular, a lot of things under the hood in protos are variable-length integers, and these integers are not required to be represented minimally. ASN.1 DER, in contrast, guarantees that any permissible value will have a unique encoding.
That is the thing - CBOR and ProtoBuffers are encodings that also offer some kind of IDL, while ASN.1 (that is why it is called Abstract Syntax One) is just IDL with multiple encodings. Technically it should be possible to use ASN.1 for defining ProtoBuffs, CBOR, or any other encoding you like. It already supports JSON (JER) and XML (XER).
The reason why we now have a lot of different concepts like ASM.1 that use their own IDL is that ASN.1 grown in a lot of cruft and the spec is abysmal. It would be great to have something like “ASN.2” that would remove all the cruft and would start from mostly “clear slate”, but I do not think that it will become a thing any time near. Too many people focused on looking what Google will publish next.
IDL – https://en.wikipedia.org/wiki/Interface_description_language
CBOR is just “binary JSON”, it is a self-describing encoding. There is no IDL in CBOR. There is an external CDDL spec (or good old JSON Schema) but.. meh.
Protobuf and the performance focused alternatives are actually IDL-first. They just happen to have one standard encoding because there are benefits to developing IDL+encoding together as a cohesive system. You could use alternative encodings, but the only reason anyone actually does is to have a human-readable JSON representation for debugging.
No. protobuf is just another encoding. You can almost certainly define a protobuf-like encoding rule for ASN.1. This is the problem with all of these flavor-of-the-day formats. All they do is fracture this space.
No! Stop! Where do all of you get that idea?!
Protobuf is IDL first, encoding second. The main part of protobuf is the proto(2/3) language for describing data structures with deep support for format evolution (backward/forward compatibility).
Also, “flavor of the day”? Really? Protobuf has been public / open source for 13 years now.
As far as I can tell protobuf provides nothing that ASN.1 does not already provide. In my experience it’s worse. Try doing polymorphism in protobuf to see what I mean.
You could define a new wire format for it. But why do that when ASN.1 already has the infrastructure for all this?
ASN.1 is really a cool and under appreciated technology. We use it at work to transfer data in BER encoding. Unfortunately there are not that many open source implementations available. There is one python lib that can convert encodings on the fly.