Python has an entire module decimal that handles basically all of these issues. The only one it doesn’t that I see at first glance is serialization. Most languages haven’t bothered with a proper decimal data library and are stuck doing lots of stuff like the OP talks about.
Neat! I didn’t realize that changed in py3.3. It originally was just pure python. Glad they sped it up, and even wrote cdecimal for people < py3.3. Thanks for the link!
And worth noting that it indeed implements IEEE 754 floats - just of the decimal variety. It is still prone to create or destroy money - just less so with the normal values that you get when dealing with money (division and sometimes multiplication are the more problematic operations). You can still both underflow (goes to zero by default) and overflow (throws an exception by default). And of course there still is the minimal quantization issue, where you will need to handle that yourself in the code.
Decimal floating point does not solve everything, but it can help you deal with some problems with careful design.
The problem isn’t that the math creates or destroys money, it’s that you have to do it the same way as your accountants/auditors/money laws expect. It’s not like their math is any different than ours, but they have rules around money math, and you have to follow their rules. Your organization will have standards they have to follow, you just have to make sure you follow them, whatever they are. Every jurisdiction makes their own rules about that sort of thing, so there isn’t some universal truth.
$ python3
Python 3.9.10 (main, Jan 15 2022, 11:48:04)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal as d
>>> pi = d('3.14159')
>>> area = pi * 2 * 2
>>> str(area)
'12.56636'
Sure, except you have to know it’s a decimal on both sides, and treat it so. Try shoving python’s Decimal through JSON(which has no idea what a decimal number is). Pretty much all serialization formats completely punt on decimal numbers, and just assume they are floats(at best). Of course then you have float -> decimal problems, which can get stupidly annoying.
Hence serialization and de-serialization are not as simple as it sounds at first glance.
Pushing it through as a string is usually your best option, as you point out. Of course then you have to ensure you handle it properly everywhere, which nobody does.
That’s why you need a schema with preferably autogenerated serialization/deserialization functions, like Rust’s serde. In any case the type safety of serialization over the wire is an orthogonal issue to the efficacy of the decimal type. You face the same issue with any type that’s not trivial to represent in JSON, e.g. timestamp with time zone.
I clearly should never have typed the word JSON in my comment, it has almost nothing to do with JSON, I apologize for even bringing it up.
As for rust, it doesn’t have a library to deal with decimal math does it? (It didn’t last I looked). If that’s still the case, you can have all the magic for serialization/deserialization one wants, but it’s pointless if you don’t have a safe place to put the decimal number once you get it.
That was basically my point, you have to treat it like the decimal number it is on both ends. Most people never, ever bother, because they aren’t even aware decimal math is a thing in computers. This is because most languages have basically the bare bones types(strings, integers, and some sort of array or KV store). You probably get floats, but that’s not guaranteed, you maybe get dates and/or times, but even if you do, you probably just barely get those(timezones are almost always completely dropped, different calendar support is ridiculously rare, etc). If the language is popular, a 3rd party library usually shows up and tries to sort of handle some of the more common date and time math.
Decimal numbers are not even recognized as needed most of the time. Look at the OP, they are likely ignorant of decimal numbers in computers, as they wouldn’t have ran into all of these horrors if they had just switched their data type to one that can handle money. Instead they got to wander down a list of horrors instead.
When pretty much any SQL database has more data types than your programming language, one of those things is broken. Obviously I think it’s the programming language.
OK but ‘I don’t know this library exists’ is not a problem the library can realistically be expected to solve. My point is serialization is not an issue when everyone agrees on the schema of the data being exchanged. This is not specific to decimal types, schemas solve this problem in general. If you send me a string and I try to treat it as a boolean on my end, it doesn’t matter that both our languages have great support for strings and booleans, I’m still going to get an error.
This was not very good. Use a fixed point type or package, and use a consistent rounding strategy. Make sure these line up with your data store. It may be worth considering pushing as much logic into the datastore as possible if it is well done like an RDBMS which will handle both of these conditions.
It’s not quite that easy, as you have to follow the rules of your jurisdiction, whatever they are. They all have their own rules about that sort of thing. But yes you basically have the gist of it.
Money has even more complications. Were one traversing down a financial application world, one could do worse than read the old RogueWave Money++ library manual at https://help.perforce.com/sourcepro/legacy-hpp/mnyug/index.html. This was a popular library from the 1980s attempting to handle multiple currencies, exchange issues, correct rounding in different scenarios, and the like.
Did you read the article? Your comment indicates not. This is about how numbers that represent money are processed by computers, not about funding OSS.
I think climate change will force economists to acknowledge that the traditional “solution” to the tragedy of the commons (i.e. make it into a monopoly) does not work in practice. Once that has been accepted then the problem can be managed with open eyes and we will quickly start moving towards p2p. I don’t think “money” (as we recognize it) will work (for this reason) and therefore a lot of questions (that we currently agonize over) will become irrelevant with this shift in foundations.
Python has an entire module decimal that handles basically all of these issues. The only one it doesn’t that I see at first glance is serialization. Most languages haven’t bothered with a proper decimal data library and are stuck doing lots of stuff like the OP talks about.
IBM created a spec for it.
And this Python module is based on the mpdecimal library, which you can use through any C FFI system.
Neat! I didn’t realize that changed in py3.3. It originally was just pure python. Glad they sped it up, and even wrote cdecimal for people < py3.3. Thanks for the link!
And worth noting that it indeed implements IEEE 754 floats - just of the decimal variety. It is still prone to create or destroy money - just less so with the normal values that you get when dealing with money (division and sometimes multiplication are the more problematic operations). You can still both underflow (goes to zero by default) and overflow (throws an exception by default). And of course there still is the minimal quantization issue, where you will need to handle that yourself in the code.
Decimal floating point does not solve everything, but it can help you deal with some problems with careful design.
The problem isn’t that the math creates or destroys money, it’s that you have to do it the same way as your accountants/auditors/money laws expect. It’s not like their math is any different than ours, but they have rules around money math, and you have to follow their rules. Your organization will have standards they have to follow, you just have to make sure you follow them, whatever they are. Every jurisdiction makes their own rules about that sort of thing, so there isn’t some universal truth.
Serialization seems pretty trivial:
Sure, except you have to know it’s a decimal on both sides, and treat it so. Try shoving python’s Decimal through JSON(which has no idea what a decimal number is). Pretty much all serialization formats completely punt on decimal numbers, and just assume they are floats(at best). Of course then you have float -> decimal problems, which can get stupidly annoying.
Hence serialization and de-serialization are not as simple as it sounds at first glance.
Pushing it through as a string is usually your best option, as you point out. Of course then you have to ensure you handle it properly everywhere, which nobody does.
JSON always represents numbers as strings, so the only problem is if your serialisation is buggy.
I apologize I wasn’t clear, I wasn’t trying to limit my comment to only JSON. I agree JSON sends it across as strings.
That’s why you need a schema with preferably autogenerated serialization/deserialization functions, like Rust’s
serde. In any case the type safety of serialization over the wire is an orthogonal issue to the efficacy of thedecimaltype. You face the same issue with any type that’s not trivial to represent in JSON, e.g. timestamp with time zone.I clearly should never have typed the word JSON in my comment, it has almost nothing to do with JSON, I apologize for even bringing it up.
As for rust, it doesn’t have a library to deal with decimal math does it? (It didn’t last I looked). If that’s still the case, you can have all the magic for serialization/deserialization one wants, but it’s pointless if you don’t have a safe place to put the decimal number once you get it.
That was basically my point, you have to treat it like the decimal number it is on both ends. Most people never, ever bother, because they aren’t even aware decimal math is a thing in computers. This is because most languages have basically the bare bones types(strings, integers, and some sort of array or KV store). You probably get floats, but that’s not guaranteed, you maybe get dates and/or times, but even if you do, you probably just barely get those(timezones are almost always completely dropped, different calendar support is ridiculously rare, etc). If the language is popular, a 3rd party library usually shows up and tries to sort of handle some of the more common date and time math.
Decimal numbers are not even recognized as needed most of the time. Look at the OP, they are likely ignorant of decimal numbers in computers, as they wouldn’t have ran into all of these horrors if they had just switched their data type to one that can handle money. Instead they got to wander down a list of horrors instead.
When pretty much any SQL database has more data types than your programming language, one of those things is broken. Obviously I think it’s the programming language.
OK but ‘I don’t know this library exists’ is not a problem the library can realistically be expected to solve. My point is serialization is not an issue when everyone agrees on the schema of the data being exchanged. This is not specific to decimal types, schemas solve this problem in general. If you send me a string and I try to treat it as a boolean on my end, it doesn’t matter that both our languages have great support for strings and booleans, I’m still going to get an error.
Why is there no mention of IEEE fixed point numbers?
This was not very good. Use a fixed point type or package, and use a consistent rounding strategy. Make sure these line up with your data store. It may be worth considering pushing as much logic into the datastore as possible if it is well done like an RDBMS which will handle both of these conditions.
It’s not quite that easy, as you have to follow the rules of your jurisdiction, whatever they are. They all have their own rules about that sort of thing. But yes you basically have the gist of it.
This is the second time today I’ve been linked to Cliff Stoll’s coffee beans. Something in the air?
I am sorry, what do you mean?
This was linked in the post.
Money has even more complications. Were one traversing down a financial application world, one could do worse than read the old RogueWave Money++ library manual at https://help.perforce.com/sourcepro/legacy-hpp/mnyug/index.html. This was a popular library from the 1980s attempting to handle multiple currencies, exchange issues, correct rounding in different scenarios, and the like.
[Comment removed by author]
Did you read the article? Your comment indicates not. This is about how numbers that represent money are processed by computers, not about funding OSS.
Oh, yes. Sorry, it was meant https://lobste.rs/s/ikwovy/preparing_for_wave_open_source_funding.
This article is great, too. No hot takes on it though.
Perhaps this was meant to be a reply to https://lobste.rs/s/ikwovy/preparing_for_wave_open_source_funding?
Yeah thanks, I had two two tabs open on both articles.
I think climate change will force economists to acknowledge that the traditional “solution” to the tragedy of the commons (i.e. make it into a monopoly) does not work in practice. Once that has been accepted then the problem can be managed with open eyes and we will quickly start moving towards p2p. I don’t think “money” (as we recognize it) will work (for this reason) and therefore a lot of questions (that we currently agonize over) will become irrelevant with this shift in foundations.