Re: Welcome to the Internet
@Warm Brew,
"To be fair, JSON is in part an answer to the problem that ASN.1 is (the clue is in the name) an Abstract specification."
Well, I'm not sure that the term abstract in this sense is a real problem. Both JSON, ASN.1 and XML can represent any data in a manner in a programming language neutral way. The schema languages follow a similar philosophy (i.e. allowing one to say everything there is to know about data that will be passed). The only real difference in outcome is that JSON and associated tools are not as well pinned down as, say, ASN.1 or XSD.
"After that you have to pick an "encoding rule" and, while these are rather better defined than JSON, they're not without their problems: there can be different ways of encoding the same thing (potentially awkward if you're trying to do digital signatures) and encodings that can't encode all possible data (GSER - which is why it's not widely used)."
Personally speaking I find the wide variety of encoding rules available under ASN.1 very liberating; there's one for every occasion. Mixing and matching between them can be a pain in the arse, and understanding exactly which variant is in use is indeed problematical. But at least they're rigidly defined. GSER sounds, well, unpleasant...
"The binary encodings are prone to programming errors if hand-crafted, but tried-and-tested libraries can be a fairly heavyweight addition to a simple project and are not immune to bugs even now."
Agreed, handcrafting ASN.1 serialisation code is definitely a mugs game. I've found that a couple of the commercial toolchains / libraries are pretty good, and have for me at least been excellent value for money.
I've had interesting discussions with the vendors concerning the "heavyweightedness" of their tools, particularly C++. These days one would imagine that they'd be leveraging all the nice containers provided by the STL to give a pleasant programming interface to things like lists, sequences, etc. but there's a strong resistance to doing so. Part of the issue is that they've all built up non-STL C++ implementations which, conveniently, they can support on platforms of lighter weight (e.g. C++ on microcontrollers where STL is a rare beast).
And yes, there can be bugs. I've come across some myself. But in theory every bug identified and fixed which results in a whole lot of code everywhere becoming better with minimal effort on the part of the system developer. They just get an updated toolchain, rebuild, et voila. Certainly it's better than having to fix numerous pieces of handcrafted code every time an interface is found to be crufty.
"Having said that, there does seem to be an irrational prejudice in the Unix-derived world to exchanging data in any form other than western-culturally-specific string formats which don't seem to be either time- or space-efficient."
It's definitely a UNIX thing. Example: the howls of rage with SystemD not storing logs in human readable text, requiring a journal control programme to access log data. Part of the Unix world's preference for text is "ease of debugging", but I think that's false economy. Having exchanges easily read by a human is all very well and good, but if you end up having to do that a lot because the interface spec is poorly written and poorly implemented then it's a waste of time. Far better indeed to go for a rigid interface definition system of any sort and prevent having to spend endless hours pouring over text files / streams, etc.
And for tools like ASN.1 there's some excellent debugging capabilities anyway. With the commercial tools you can get some excellent data viewers which will interpret data and show you what it contains. The online ASN.1 Playground is a useful freebie for doing this. Wireshark also includes an ASN.1 parser - capture a TCP stream, import the ASN.1 schema, see what data is being exchanged. Wireshark can also do the same for Google Protocol Buffers.
So I think in this day an age the idea that things have to be human readable to be debuggable is well and truly debunked.
"And to anything arising from ISO/IEC/ITU. So a complex standard with at least different options for encoding the actual data is going to struggle, even if it has burrowed into some of the further corners of the Internet (e.g. SNMP and the widespread use of X.509 certificates)."
Perhaps. Let's not forget LDAP too, something that no one has been able to displace. Some of these things have come out of X500 rather than out of Unix, which may explain their nature.