Thank you @soapdog, the HN thread also made me feel tired. Your post helped me remember there are others like me who do understand and appreciate the choices made by the authors of the language. And more people who understand that it’s not a zero-sum game, where one language being good (for some purposes) automatically means others must be “bad”.
Thanks for the kind words. Someone posted that article on HN as a “real entry” instead of a comment reply like I did. Now, the amount of people devoting a ton of energy on 0-based indexes vs 1-based indexes, makes me tired. :-)
I find that “batteries” are a mixed blessing anyway; Python has batteries like sunau; how many people actually use that? And all code, even “dead” code, impose some amount of maintenance burden on the Python devs. There are a whole bunch of packages like that, and I believe it was the source of some conflict in the Python community as well last year.
Standard libraries also tend to have stricter compatibility and quality requirements. These are good things, of course, but it’s also a trade-off. Want to make an incompatible “v2” or “v3” of your library? Well, that’ll be trickier. Want to add some more experimental interface or support some standard that’s not common (yet)? Probably not in the standard library. I believe it was Russ Cox who observed that “the standard library is where good code goes to die”. This is why so many people using requests instead of Python’s standard library for HTTP requests.
Don’t get me wrong, I like that Python comes with stuff like SQLite and TKinter; I’ve gotten good mileage out of it. But it’s clearly not some sort of perfect solution or the only way to build a useful language/ecosystem.
0-based indexing is supposed to be better for some things, according to some anyway. ABC – Python’s predecessor – used 1-based indexing as well, but it was switched to 0-based for Python. Guido wrote a bit about this and why he felt it was easier.
Array indexes are not just about counting and telling your friend that “you are first in line”. Our calendar is essentially a 1-based array, and it’s kind of weird, no? When I was 15 in 2000 and even more of a smartass than I am now I loved telling everyone they were ackchyually celebrating the millennium in the wrong year since the first year was 1CE and there is no year 0. I sure knew how to make myself popular… The astronomical calendar does have year 0 by the way, because it’s more convenient.
The PostgreSQL docs even has a snarky comment on this by the way:
The first century starts at 0001-01-01 00:00:00 AD, although they did not know it at the time. This definition applies to all Gregorian calendar countries. There is no century number 0, you go from -1 century to 1 century. If you disagree with this, please write your complaint to: Pope, Cathedral Saint-Peter of Roma, Vatican.
I also think there’s some value in sticking what $most_other_languages are doing too (although blindly following this would be foolish of course).
I never worked much with Lua or other languages that use 1-based indexing (I just wrote a few Wikipedia templates in Lua), so I don’t have super strong opinions or anything, but it does seem a bit more complex than you mentioned in your post.
I believe it was Russ Cox who observed that “the standard library is where good code goes to die”.
I find this interesting because IME Go’s standard library is something languages should aspire to: tightly scoped, high quality, and guaranteed to be stable.
Want to make an incompatible “v2” or “v3” of your library? Well, that’ll be trickier. Want to add some more experimental interface or support some standard that’s not common (yet)? Probably not in the standard library.
Good.
I’ve compiled old Go code for work (last built against 1.3 iirc) and thankfully the developers back then made the conscious, documented decision to go stdlib only. My career has given me a few opportunities to build code whose maintenance stopped before my children were born, but only Go makes it relatively pleasant.
There are a bunch of packages that are “frozen” and effectively (un)dead, such as net/smtp, encoding/gob, text/tabwriter, archive/tar, log/syslog, net/rpc, net/rpc/jsonrpc, and perhaps some others. In hindsight, adding these to the standard library was a mistake. While these are certainly all useful packages, supporting the full range of SMTP in all its complexity and extensions or all the different tar flavours isn’t really something that’s best done in the standard library.
There are also a bunch of “mistakes” in the standard library that are hard to fix. A well-known example is the log package which doesn’t have a logger interface (only a log.Logger struct) and doesn’t offer a lot of flexibility in general, which is why so many people are using logrus or whatnot. There’s a few more mistakes like this, and is all quite hard to change now in a compatible way (although, with modules, there could be a /v2 for stdlib packages too at some point).
Things like contexts and the (upcoming) fs.FS changed a number of paradigms, and made a number of methods obsolete (like the Cancel channel on net.Dialer), as well as introducing a bunch of inconsistencies and a whole slew of duplicated functions (I’m reminded by Microsoft’s *Ex() functions). When generics arrive next year it’ll be even worse.
Third-party code isn’t going to stop working, and if you vendor the code then you can be guaranteed it’s not going to disappear either. If your Go 1.3 project would have vendored any external code then the project would compile and run today as well without any changes.
My point is, all this strong compatibility is great and I love it too, but also comes with some serious drawbacks. You really want to think carefully about where you want to apply it.
Nit pick: I tend to see a lot of people assume you need to define a type Fooer interface{} in order to use an object as an interface. Just a reminder that Go isn’t Java and you don’t need a type LogInterface interface{} to use log.Logger{} as an interface.
For example, these two different projects create their own log interface. They only specify the methods of log.Logger{} they actually care about. You can use log.Logger{} or whatever other implementation you’d like.
The biggest issue is that the log struct/package/interface is quite limited and not really adequate for a lot of use cases, which is why there are a bunch of “competing” logging interfaces. And some stdlb references log.Logger directly as well, such as ErrorLog on http.Server. You can work around this, but it’s all rather ugly. The problems run a bit deeper than just “log.Logger isn’t an interface”.
There should probably be an entirely approach actually; people have proposed these huge interfaces and I’m not a fan. There are a few interesting links to other discussions/proposals in that issue as well, such as this one.
At any rate, this story is about Lua and not Go 🙃 But suffice to say Go is stuck with a rather suboptimal logging solution in the stdlib IMHO.
The big reason to favour 0-indexed arrays is that you are almost certainly going to need to interoperate with a system that uses them and so you’re asking for bugs if you index from 1 (and bugs that happen at an interop boundary are the most painful to debug). That said, I actually do prefer them having learned to program on language with 1-indexed arrays. Cardinal and ordinal numbers are different things. English conflates them but a lot of other human languages do not. I’d really like a language that used fundamentally different types for these things, they’d solve a lot of bugs. They make a bunch of things simpler:
If I want to concatenate two 0-indexed things, the index of the first element of the second one is the length of the first, not the length of the first + 1.
I can write index < length to check if something is in bounds, not index <= length or index < (length+1).
If I have a type like std::vector that lets me insert at the end, I can insert at the index that is the length, not the length + 1.
If I ever need to mix array indexing and pointer arithmetic, I don’t need to add a +sizeof(T) to the pointer arithmetic or -1 to the array indexing.
Anything that requires the programmer to add +1 or -1 is likely to be forgotten some of the time and cause logic errors. Consider how many C bugs exist because people forgot about needing to add a +1 to the strlen result to account for null terminators. Just the first affordance on this list is enough to justify 0-indexed arrays.
That said, Lua is Smalltalk with weird naming, so it’s not surprising that it adopts Smalltalk arrays.
I don’t think the examples you gave are actually simpler.
If I want to concatenate two 0-indexed things, the index of the first element of the second one is the length of the first, not the length of the first + 1.
Even as a seasoned C++ dev, I actually find 1-indexing more intuitive here. Conceptually, the start of the second list is length+1 elements into the concatenated list. In real code you would write list[length], but that really means list[(length-1)+1].
Thus, and admittedly anecdotally, I find complex 1-indexed code more legible than 0-indexed code. A binary search, for example.
I can write index < length to check if something is in bounds, not index <= length or index < (length+1).
Typing <= length is hardly a burden, and I’d argue it makes more sense. Yes, as people familiar with 0-indexed languages we know that < length does include the last element, but “up to and including the last element” is <= logically.
Lua’s built in constructs work this way, eliminating any possible inconvenience of typing the additional = in <=. To iterate over an array table:
for i = 1, #t do ... end
Looking at the opposite operation makes 0-indexing look even more silly. Checking for out of bounds becomes if i >= length, compared to if i > length for 1-indexing.
Likewise Lua reverse iteration is much cleaner:
for i = #t, 1, -1 do ... end
Compare to C:
for (int i = length - 1; i >= 0; i--) { ... }
I had to think about it to get it right.
If I have a type like std::vector that lets me insert at the end, I can insert at the index that is the length, not the length + 1.
This just rehashes the “indexes are pointer arithmetic” argument. If you have a std::vector, you can insert at the end with v.push_back(item_to_append). In Lua, table.insert(t, item_to_append).
If I ever need to mix array indexing and pointer arithmetic, I don’t need to add a +sizeof(T) to the pointer arithmetic or -1 to the array indexing.
I believe this is the only true utility of 0-indexing.
Consider how many C bugs exist because people forgot about needing to add a +1 to the strlen result to account for null terminators.
This is more a problem with C null terminators, a genuinely terrible idea in every respect. I doubt this translates to Lua 1-indexing. As I think I have demonstrated, 1-indexing is more intuitive and therefore less likely to make anyone ponder whether they need +1 or -1, unless their brain is already tuned for 0-indexing and they have to unbend their thinking to get it right. While that certainly occurs, I don’t think it’s a good argument for 0-indexing.
That said, Lua is Smalltalk with weird naming, so it’s not surprising that it adopts Smalltalk arrays.
I haven’t heard that before. To me, Lua and Smalltalk seem quite different. Smalltalk emphasizes the importance of objects, whereas Lua merely provides some syntax sugar for passing self to otherwise ordinary functions.
You get used to the 1-indexing. I do a lot of interfacing with C code with Lua, both for personal projects and for work (sorry, can’t show those) and I haven’t encountered any real issues with switching from 1-indexing to 0-indexing. But I’m willing to accept that it might just be the type of work I do (dealing with telephone calls, which is mostly parsing SIP messages anyway).
@soapdog: I think you linked to the wrong HN comment when talking about Janet? You linked to the parent of it?
I was the one who wrote that, my point wasn’t that Janet would replace Lua, but that for people that are looking for “Lua semantics, but with more baked in by default”, Janet may well end up filling that role for some.
@yumaikas, oops sorry. I will fix the link later. There is too much traffic right now and I use rsync to update the static files. I don’t want to touch the server right now. Sorry for the mistake.
I like Janet a lot too, I even tried contributing some patches to make it work on Windows on ARM. “Lua semantics, but with more baked in by default” I understand you better now, I agree with you. As I said on the post, I was not singling Janet or your comment, it was just that it reminded me of all the times someone told me that some language will replace Lua. Sorry if my free association was bad for you.
No worries. I don’t think the association will hurt me.
I’ve also used Lua (specifically one of the Go-based implementations), for a CMS in the past, and I had a good run of using it with Love2D to make video games. It’s a nifty language.
Walmart uses embedded lua (in a Go proxy server) to handle all of their L7 routing and to generate full responses and segmentation for high throughput analytics servers (their equiv to Google Analytics). This logic can run all the way at their edge CDN. It’s been a really powerful tool for them and it was relatively easy to embed in a multi-tenant proxy server.
For inexplainable reasons Lua brings me so much joy. Its the most fun I’ve ever had in a programming language & I’ve barely done much in it. It is so beautifully small, dynamic, quirky & fun.
We usually start to count from one. When a baby completes its first roundtrip around the Sun, they’re one year old, not zero years old.
Amusingly, they got this backward. In English, ages are 0-based, not 1-based. Is a baby that was born a month ago 0 or 1? We say they’re 0 years old (or, since that’s awkward, 1 month old). IIRC, in China you would say they’re 1.
If a baby completes its first round-trip around the Sun, the question isn’t whether they’re zero or one. The question is whether they’ve just started their year 1, or just started their year 2.
I think that you can only appreciate Lua when you embed it on your own software. Until what you are building is a combination of some code in other language (such as C) and Lua, the power of Lua will not click for you. Not only the power, but the way of Lua will not click for you. If all you’re using is 100% Lua source code, it will feel just like another scripting language – not unlike Python, or Ruby
This is absolutely not true. Lua stands head and shoulders above Python and Ruby even as a standalone language. The design is so much more coherent and sensible. It’s simultaneously dramatically simpler and significantly more powerful.
Thanks for the kind words. That post was written as a reply to that Lua vs Python post, folding might be a good idea. I don’t know how folding works here to be honest.
Lua is a great example of being the “right tool for the right job”. Because it really is meant to be used in specific ways inside of specific applications/domains. Using Lua means you understand the system it is being used inside of. And, honestly, thinking this way about any tool has helped me gain a lot more perspective about the systems I’ve been given to work with or I’m currently building.
Thank you @soapdog, the HN thread also made me feel tired. Your post helped me remember there are others like me who do understand and appreciate the choices made by the authors of the language. And more people who understand that it’s not a zero-sum game, where one language being good (for some purposes) automatically means others must be “bad”.
Thanks for the kind words. Someone posted that article on HN as a “real entry” instead of a comment reply like I did. Now, the amount of people devoting a ton of energy on 0-based indexes vs 1-based indexes, makes me tired. :-)
Spite writes are the best writes!
I find that “batteries” are a mixed blessing anyway; Python has batteries like
sunau; how many people actually use that? And all code, even “dead” code, impose some amount of maintenance burden on the Python devs. There are a whole bunch of packages like that, and I believe it was the source of some conflict in the Python community as well last year.Standard libraries also tend to have stricter compatibility and quality requirements. These are good things, of course, but it’s also a trade-off. Want to make an incompatible “v2” or “v3” of your library? Well, that’ll be trickier. Want to add some more experimental interface or support some standard that’s not common (yet)? Probably not in the standard library. I believe it was Russ Cox who observed that “the standard library is where good code goes to die”. This is why so many people using
requestsinstead of Python’s standard library for HTTP requests.Don’t get me wrong, I like that Python comes with stuff like SQLite and TKinter; I’ve gotten good mileage out of it. But it’s clearly not some sort of perfect solution or the only way to build a useful language/ecosystem.
0-based indexing is supposed to be better for some things, according to some anyway. ABC – Python’s predecessor – used 1-based indexing as well, but it was switched to 0-based for Python. Guido wrote a bit about this and why he felt it was easier.
Array indexes are not just about counting and telling your friend that “you are first in line”. Our calendar is essentially a 1-based array, and it’s kind of weird, no? When I was 15 in 2000 and even more of a smartass than I am now I loved telling everyone they were ackchyually celebrating the millennium in the wrong year since the first year was 1CE and there is no year 0. I sure knew how to make myself popular… The astronomical calendar does have year 0 by the way, because it’s more convenient.
The PostgreSQL docs even has a snarky comment on this by the way:
I also think there’s some value in sticking what $most_other_languages are doing too (although blindly following this would be foolish of course).
I never worked much with Lua or other languages that use 1-based indexing (I just wrote a few Wikipedia templates in Lua), so I don’t have super strong opinions or anything, but it does seem a bit more complex than you mentioned in your post.
I find this interesting because IME Go’s standard library is something languages should aspire to: tightly scoped, high quality, and guaranteed to be stable.
Good.
I’ve compiled old Go code for work (last built against 1.3 iirc) and thankfully the developers back then made the conscious, documented decision to go stdlib only. My career has given me a few opportunities to build code whose maintenance stopped before my children were born, but only Go makes it relatively pleasant.
There are a bunch of packages that are “frozen” and effectively (un)dead, such as
net/smtp,encoding/gob,text/tabwriter,archive/tar,log/syslog,net/rpc,net/rpc/jsonrpc, and perhaps some others. In hindsight, adding these to the standard library was a mistake. While these are certainly all useful packages, supporting the full range of SMTP in all its complexity and extensions or all the different tar flavours isn’t really something that’s best done in the standard library.There are also a bunch of “mistakes” in the standard library that are hard to fix. A well-known example is the
logpackage which doesn’t have a logger interface (only alog.Loggerstruct) and doesn’t offer a lot of flexibility in general, which is why so many people are using logrus or whatnot. There’s a few more mistakes like this, and is all quite hard to change now in a compatible way (although, with modules, there could be a /v2 for stdlib packages too at some point).Things like contexts and the (upcoming) fs.FS changed a number of paradigms, and made a number of methods obsolete (like the
Cancelchannel onnet.Dialer), as well as introducing a bunch of inconsistencies and a whole slew of duplicated functions (I’m reminded by Microsoft’s *Ex() functions). When generics arrive next year it’ll be even worse.Third-party code isn’t going to stop working, and if you vendor the code then you can be guaranteed it’s not going to disappear either. If your Go 1.3 project would have vendored any external code then the project would compile and run today as well without any changes.
My point is, all this strong compatibility is great and I love it too, but also comes with some serious drawbacks. You really want to think carefully about where you want to apply it.
Nit pick: I tend to see a lot of people assume you need to define a
type Fooer interface{}in order to use an object as an interface. Just a reminder that Go isn’t Java and you don’t need atype LogInterface interface{}to uselog.Logger{}as an interface.For example, these two different projects create their own log interface. They only specify the methods of
log.Logger{}they actually care about. You can uselog.Logger{}or whatever other implementation you’d like.The biggest issue is that the log struct/package/interface is quite limited and not really adequate for a lot of use cases, which is why there are a bunch of “competing” logging interfaces. And some stdlb references log.Logger directly as well, such as ErrorLog on http.Server. You can work around this, but it’s all rather ugly. The problems run a bit deeper than just “log.Logger isn’t an interface”.
There should probably be an entirely approach actually; people have proposed these huge interfaces and I’m not a fan. There are a few interesting links to other discussions/proposals in that issue as well, such as this one.
At any rate, this story is about Lua and not Go 🙃 But suffice to say Go is stuck with a rather suboptimal logging solution in the stdlib IMHO.
The big reason to favour 0-indexed arrays is that you are almost certainly going to need to interoperate with a system that uses them and so you’re asking for bugs if you index from 1 (and bugs that happen at an interop boundary are the most painful to debug). That said, I actually do prefer them having learned to program on language with 1-indexed arrays. Cardinal and ordinal numbers are different things. English conflates them but a lot of other human languages do not. I’d really like a language that used fundamentally different types for these things, they’d solve a lot of bugs. They make a bunch of things simpler:
index < lengthto check if something is in bounds, notindex <= lengthorindex < (length+1).std::vectorthat lets me insert at the end, I can insert at the index that is the length, not the length + 1.+sizeof(T)to the pointer arithmetic or-1to the array indexing.Anything that requires the programmer to add +1 or -1 is likely to be forgotten some of the time and cause logic errors. Consider how many C bugs exist because people forgot about needing to add a +1 to the
strlenresult to account for null terminators. Just the first affordance on this list is enough to justify 0-indexed arrays.That said, Lua is Smalltalk with weird naming, so it’s not surprising that it adopts Smalltalk arrays.
I don’t think the examples you gave are actually simpler.
Even as a seasoned C++ dev, I actually find 1-indexing more intuitive here. Conceptually, the start of the second list is length+1 elements into the concatenated list. In real code you would write
list[length], but that really meanslist[(length-1)+1].Thus, and admittedly anecdotally, I find complex 1-indexed code more legible than 0-indexed code. A binary search, for example.
Typing
<= lengthis hardly a burden, and I’d argue it makes more sense. Yes, as people familiar with 0-indexed languages we know that< lengthdoes include the last element, but “up to and including the last element” is<=logically.Lua’s built in constructs work this way, eliminating any possible inconvenience of typing the additional
=in<=. To iterate over an array table:Looking at the opposite operation makes 0-indexing look even more silly. Checking for out of bounds becomes
if i >= length, compared toif i > lengthfor 1-indexing.Likewise Lua reverse iteration is much cleaner:
Compare to C:
I had to think about it to get it right.
This just rehashes the “indexes are pointer arithmetic” argument. If you have a
std::vector, you can insert at the end withv.push_back(item_to_append). In Lua,table.insert(t, item_to_append).I believe this is the only true utility of 0-indexing.
This is more a problem with C null terminators, a genuinely terrible idea in every respect. I doubt this translates to Lua 1-indexing. As I think I have demonstrated, 1-indexing is more intuitive and therefore less likely to make anyone ponder whether they need +1 or -1, unless their brain is already tuned for 0-indexing and they have to unbend their thinking to get it right. While that certainly occurs, I don’t think it’s a good argument for 0-indexing.
I haven’t heard that before. To me, Lua and Smalltalk seem quite different. Smalltalk emphasizes the importance of objects, whereas Lua merely provides some syntax sugar for passing
selfto otherwise ordinary functions.As far as I know, the Lua devs reasoning has nothing to do with Smalltalk. I recall reading somewhere that they chose 1-indexing because it makes more logical sense. This Stack Overflow answer indicates Lua inherits this from Sol, for similar reasons.
You get used to the 1-indexing. I do a lot of interfacing with C code with Lua, both for personal projects and for work (sorry, can’t show those) and I haven’t encountered any real issues with switching from 1-indexing to 0-indexing. But I’m willing to accept that it might just be the type of work I do (dealing with telephone calls, which is mostly parsing SIP messages anyway).
@soapdog: I think you linked to the wrong HN comment when talking about Janet? You linked to the parent of it?
I was the one who wrote that, my point wasn’t that Janet would replace Lua, but that for people that are looking for “Lua semantics, but with more baked in by default”, Janet may well end up filling that role for some.
@yumaikas, oops sorry. I will fix the link later. There is too much traffic right now and I use rsync to update the static files. I don’t want to touch the server right now. Sorry for the mistake.
I like Janet a lot too, I even tried contributing some patches to make it work on Windows on ARM. “Lua semantics, but with more baked in by default” I understand you better now, I agree with you. As I said on the post, I was not singling Janet or your comment, it was just that it reminded me of all the times someone told me that some language will replace Lua. Sorry if my free association was bad for you.
No worries. I don’t think the association will hurt me.
I’ve also used Lua (specifically one of the Go-based implementations), for a CMS in the past, and I had a good run of using it with Love2D to make video games. It’s a nifty language.
Walmart uses embedded lua (in a Go proxy server) to handle all of their L7 routing and to generate full responses and segmentation for high throughput analytics servers (their equiv to Google Analytics). This logic can run all the way at their edge CDN. It’s been a really powerful tool for them and it was relatively easy to embed in a multi-tenant proxy server.
For inexplainable reasons Lua brings me so much joy. Its the most fun I’ve ever had in a programming language & I’ve barely done much in it. It is so beautifully small, dynamic, quirky & fun.
Amusingly, they got this backward. In English, ages are 0-based, not 1-based. Is a baby that was born a month ago 0 or 1? We say they’re 0 years old (or, since that’s awkward, 1 month old). IIRC, in China you would say they’re 1.
If a baby completes its first round-trip around the Sun, the question isn’t whether they’re zero or one. The question is whether they’ve just started their year 1, or just started their year 2.
This is a great article, but I have one nitpick.
This is absolutely not true. Lua stands head and shoulders above Python and Ruby even as a standalone language. The design is so much more coherent and sensible. It’s simultaneously dramatically simpler and significantly more powerful.
[Comment removed by author]
Good post, I learned a bit more about Lua.
@pushcx, consider folding this into https://lobste.rs/s/2lpxqj/lua_python
Thanks for the kind words. That post was written as a reply to that Lua vs Python post, folding might be a good idea. I don’t know how folding works here to be honest.
Thank you for this. It’s nice to see support for mixing langauges and using small languages.
Lua is a great example of being the “right tool for the right job”. Because it really is meant to be used in specific ways inside of specific applications/domains. Using Lua means you understand the system it is being used inside of. And, honestly, thinking this way about any tool has helped me gain a lot more perspective about the systems I’ve been given to work with or I’m currently building.