Threads for atilaneves
-
Always, unless you can’t (legacy codebases, sigh).
-
Another person who can’t tell the difference between their own preferences and universal experiences.
-
Please attempt to make an argument rather than a single-sentence hot-take, especially one that isn’t grounded in the article and is instead just lazy character assassination.
The author introduces the rest of the writeup by saying:
I want to make a developer-centric argument that the current state of majority remote working is bad, not because it is bad for your company or for your salary but because it is not best for yours and others mental well being.
If you want to argue with any of the core claims, I’d expect something addressing the parts about:
- Knowing our colleagues making our work more meaningful (and the implied assertion that this is not doable for remote colleagues)
- Working remote causing less of a feeling of “belonging” in an org
- Working remote has blurred the boundaries of what is work and what is leisure time
- Working remote allows for too much distraction
It is totally reasonable to refute or disagree with any of those claims from the article-but on the other hand, a lazy dismissal is corrosive to the discussion culture of this site.
-
This isn’t lazy character assassination, my analysis of this article is that the author cannot distinguish between their preferences and universal experiences. It may be the case that your experiences match the author, and I hope both of you can enjoy the office without those of us who never wanted to be in the office.
Are you sure your comment to me isn’t a reflection of personal dislike?
-
my analysis of this article is that the author cannot distinguish between their preferences and universal experiences.
Please show me this analysis. What statements in the article lead you to that belief? What reasoning are you using?
I mean this quite honestly and without attempt at meanness: do you not see why I made my objection?
-
Neither you nor anyone else is owed the chain of reasoning by which someone arrived at their opinion on an article. And as I pointed out in a different reply to you, this topic is one that gets extremely repetitive extremely quickly. So if you truly and genuinely are interested in learning more about the arguments, you will have little difficulty finding previous iterations of discussion on similar articles across a wide variety of sites, courtesy of your preferred search engine.
For the moment, your behavior comes across as, at best, back-seat moderating. If you believed the comment violated site rules, you should have simply flagged it and moved on.
-
-
-
At this point the “working remote is bad for you!” argument is just incredibly repetitive. It basically boils down to: yes, we understand that the author is the sort of person who relied on the traditional office environment to make them happy and effective, but not everybody thrived in that environment and not everybody suffers now as a result of it no longer being the automatic default.
What more is there to say after that, really? The author seems to be yet another in a long line of authors of these articles, not saying anything particularly new or different and not really breaking out of the mold of extrapolating from I liked and thrived on these aspects of office culture to therefore everyone must have thrived on them and suffers now that they’re gone, when the latter is not at all true.
At this stage it’s clear that forcing everyone back to the office full-time is a non-starter. I suggest that the author learn to accept this, make the most of available “hybrid” environments, and perhaps along the way develop some empathy for those of us who did not thrive in traditional office environments but were forced into them for years on end anyway, and who wouldn’t have been taken seriously if we wrote “why working in an office is bad for you!” articles.
-
Yet I think there are some good points in there. Especially the risk of burnout/overworking is elevated when working remotely in my own experience.
It is good to bring out the possible negatives of this arrangement, even if most of the people WFH are happy about the change.
I think it will take up to 5-10 years for people to realize what they have lost with WFH.
-
-
-
I wonder how much of that is based on the novelty of remote work for a lot of people. I’ve been working remotely for most of a decade and over time, as remote work has become a totally routine, unremarkable part of my lifestyle, I’ve found it’s gotten much easier to switch out of work mode at the end of the day without having to physically move to a different part of the house. (Muting Slack notifications at the end of the day helps!)
-
This is why I made a point of having a room in my flat that’s the designated office that I never spend time in unless I’m working. Out of the home office, brain off for the day.
-
The problem with WFH is that not everyone can afford to pay the high rent of an extra room (and another for your spouse) which is a cost saving for your employer but a big cost for you unless you are prepared to move to a rural or suburban area for the benefit of your employer.
-
-
-
You could, and you’d be incorrect. The article makes claims about “tech workers” in general, and therefore your comment would make sense there. I’m talking about my experiences, and at no point in time did I generalise to anyone else. This is what works for me, which refutes “what tech workers are missing”. If going to the office works for someone else, good for them.
-
-
As discussed in other branch, not everybody has access to a extra office room at their home, nor have peaceful enough environment for concentrated work.
For me the office is much more quiet place to focus on work and I have much better computer setup there. I don’t really even use computers at home that often.
-
-
-
-
-
-
-
-
I didn’t lose anything, I’ve only gained. The issues with vídeo calls can be solved by having fewer of them, especially since most meetings are useless anyway.
Social isolation? My job is to read and write code, and for that I want to be in flow. I see people after work, at the pub, like $DEITY intended.
-
-
Nowhere near half of my time is interacting with anyone, and 99% of the time that I do it’s better to do it asynchronously via slack/email than getting interrupted in a noisy office.
My job is to read and write code. Sometimes I have to interact with other people to do so.
-
-
-
-
I laughed at the headline since if you look at my open source libraries you’ll find a few files bigger than that: https://github.com/adamdruppe/arsd/ simpledisplay is 21.8k, nanovega.d is 15.1k, minigui.d is 14.4k, cgi is 11.1k…
I find larger files easier to work with than collections of smaller files, all other things equal, and I like having complete units of functionality.
But “all other things equal” does a lot there - the article’s description of “It looked like the entire file would execute through from top to bottom”… That’s what is really scary: not the size of the file, but rather that this appears to be a large single function. See, if I open simpledisplay.d, I’m not looking at the whole file. I’m just interested in SimpleWindow.close() or EventLoop.impl. The rest of the file isn’t terribly important; I open it, jump straight to the function I want to work on, and do what needs to be done. Then the individual function is pretty ordinary.
So I push back against file size by itself as mattering - a file is just a container. You actually reason about functions or classes or whatever so that’s what you want to keep easy to follow (and note that smaller is not necessarily easier to follow, I also prefer long, simple code to short, clever code, and I’d prefer direct local use to indirect things through multiple layers).
-
I find larger files easier to work with than collections of smaller files, all other things equal
That’s interesting, I’m very much the opposite: I try to keep my source files under 500 lines, and each file has a specific purpose like one class, or a set of constants used in one module. Makes it a lot easier to jump to a specific part of the code, by just clicking on a filename or tab. And when I search I’m limited to the appropriate context, like the specific class I’m working on.
What is it you prefer about single big files?
-
I’m also team big files. I hate how many subjective decisions you have to make when you split things across files “Hey we split up foo and bar, now where do we put this function that was used by both foo and bar? Into foo.js, bar.js, or helpers.js?” or “Hey do we group models together and controllers together, or do we group by feature?”.
Whatever organizational decisions you make, some of them will ultimately prove unsatisfying as your codebase evolves and you’ll face the constant temptation to spend a bunch of energy reorganizing, but reorganizing your code across files doesn’t actually make your code more modular or more adaptable or improve it in any way other than maybe making it a little bit more navigable for somebody whose muscle memory for navigating codebases revolves around filenames.
I default to large files because it requires the least energy to maintain.
-
-
It seems what you really want is to store all “source code” in a database instead.
When you think about it, how can it be possible that storing program code in a bunch of plain text files (ASCII, UTF-8) is in any way optimal for comprehension and modification? Text files are very much a least-common-denominator representation. We continue to use it because of the ecosystem of operating systems, version control, text editors and such allow us to use and interchange this information. So there is a very good reason why they persist to this day.
But I can imagine some kind of wacky Matrix-y (in the William Gibson, Vernor Vinge sense) 3D representation of programs, which makes great use of colored arrows, shapes and more to represent program operations and flow.
Do I have the slightest idea of where to start making such a programming “language”, and what exactly it looks like? No, I do not. Until we have better 3D systems (something akin to a floating hologram in front of me), that allows me to easily grab and manipulate objects, I don’t think I’d want to use such a system anyway. But this is the direction I think things will go in… eventually. That will likely take a long time.
Also, do we want to design a programming system that is optimized for human comprehension? Or something that is optimized for AI to use?
-
Also, do we want to design a programming system that is optimized for human comprehension? Or something that is optimized for AI to use?
Well, my vote is always for humans. I have no stock in AI-produced code every being a good thing.
It seems what you really want is to store all “source code” in a database instead.
Actually now that I think about it, NDepend is pretty similar to this. Warning: that page autoplays a video with sound.
-
-
Those are heading in the right direction, but I’m thinking about something more comprehensive. The link from /u/amw-zero about NDepend is very interesting.
-
-
-
-
I’m not sure all or even most of these decisions are subjective - I think that ideally one would want to reduce coupling throughout, and to limit the visibility of implementation details.
I tend to think in terms of build system DAGs - it profoundly annoys me when everything needs to get rebuilt for no actual good reason. Which is another reason to prefer smaller files, I think - less rebuilding of code.
-
I agree. For one, I don’t think the decisions are subjective or objective in itself . But the fact that I do have to spend time thinking about code before I start is a clear benefit to me, not a disadvantage. Yes, sometimes it’s annoying, but mostly in pays off in, as you say, simpler code, clearer boundaries, less stuff tied into one big implementation.
-
Splitting things into classes or modules can limit coupling/visibility. Some programming languages enforce one-module-per-file, but many (golang, ruby) don’t and in these languages there is no encapsulation benefit to putting things in different files.
Optimizing the build is a good point, though. If that’s the criterion for file divisions, and not subjective developer feelings about what belongs where, that eliminates the problem for me…
-
-
At some point in my career, I realized I had crossed a threshold where the writing & testing of the code was no longer the hard part: the hard part is organizing the code so it “makes sense” for what it’s doing, and I’ll be able to figure it out in 5 years when I come back to it.
My mental line is 500 lines, too. Once I hit that, and I don’t immediately know how to break it up, it’s usually a sign that I need to take a hike and think about the structure at a higher level. Most of the time, this mental refactoring unlocks a lot of future features, and the invested “thinking” time pays itself off multiple times over.
(None of this is a new insight, btw. I think it’s been written about since before the transistor.)
-
-
What is it you prefer about single big files?
Basically everything. It is easier to find what I’m looking for, since I can just search inside the file rather than having to find more. Perhaps if I used an IDE and project files I’d feel differently, but I don’t. And even if I did, sometimes I browse projects online, and online, you often click links that lead to single files to see in the web browser. So it is easier to work with here and easier to browse online. It is also easier for users (which includes me doing a quick ad-hoc build or test run): I can say “download cgi.d and list it on your build command” and it just works for them, no complication in mirroring the directory and figuring out which list of files belong in the build, etc. D’s encapsulation is set at the file layer too, so I can actually define smaller, better defined API surfaces this way than with a variety of files, since the implementations are all wrapped up, without being tempted to expose
packageprotection or similar to share bits that need to be shared across files (or worse yet, marking somethingpublicbecause I need it from a different file in my own project instead of committing to supporting it long term to the users, which is what I think public SHOULD me).When working with other people’s projects, I find the file organization almost never helps. I can never guess where something is actually found from the directory/file organization and just have to
grep -Rit.So I believe in the old gmail slogan: don’t organize, search! And then you start working on a more abstract level - classes, functions, etc. - instead of files anyway, so just take one more complication out of the way.
-
Perhaps if I used an IDE and project files I’d feel differently, but I don’t.
It’s not just in IDEs. Any good programming editor ought to offer a file hierarchy view and support multi-file search. I can’t even imagine doing nontrivial coding without that, just like I can’t imagine not having Undo.
I don’t mean to sound condescending, but it really is worth it to look into more powerful tools.
-
What possible benefit could I get from merging my file browser into my editor? I have a file browser, I have an editor, they know how to talk to each other.
-
-
-
-
I generally like the large file as well, if I don’t have really good navigation in my IDE, because it puts that much more within easy reach of the search function of my editor. And since I spend a lot more time looking for and reading code than jumping confidently to places in a codebase that biases me to wanting longer files.
-
Does your IDE not have good ‘search this entire project’ support?
-
Yes, thus my statement “when I have good IDE navigation support.” The logical conclusion of that is Smalltalk where there are no files. But if I am sitting in a terminal with cli tools and vi, large files are easier since I don’t have to keep Ctrl+z’ing to run grep and fg’ing again.
-
-
-
There’s no right / wrong answer here, because obviously people’s individual brains are different. But lots of small files absolutely kills my productivity, because working on large pieces of functionality ends up requiring changing 5-10 files. There’s no good way to look at 10 files simultaneously, I don’t care how large your monitor is. So the benefit of larger files is that all of the code that I need to understand something is localized within something that’s on my screen right now, and it’s pretty easy to use something like the sidebar in Sublime or VSCode to scroll quickly to the part of the file that you need. Or text search for a specific function name to jump right to it. The benefit is not really needing to do anything to find the code that I need.
Also:
Makes it a lot easier to jump to a specific part of the code, by just clicking on a filename or tab
This isn’t unique to the small-file approach, if your IDE / editor has “jump to definition” support, it works just as well within a file. Like I said, while this can be subjective and you may prefer one way or the other, I find this often with people who prefer small files - there’s no actual tangible reason or benefit, it just feels more organized (to proponents) when code is factored into small pieces.
It may be a limitation of our tools, but I find too many files to be a cognitive cost. After a while I have too many editor tabs open, I can’t get to the part of code that I wanted without backtracking, etc. And, what is the downside of larger files? “Large things are bad” is not an axiom. “Large files are bad because they are large” is circular logic.
All that being said, I don’t care all that much. I can navigate around most codebases whatever the structure.
-
-
-
The problem is not, in my experience, large files, but the lack of separation of concerns. Large file sizes can be a symptom of a lack of separation of concerns but they’re the symptom, not the problem. I started work on clang in 2008 because I was working on GNUstep and wanted to use the shiny new Objective-C features that Apple had shipped. Apple had their own fork of GCC and no one merged their changes into the main branch[1] and so Objective-C on non-Apple platforms had a NeXT-era feature set.
I looked at GCC to see how much effort it would be to update it. All of the Objective-C code in GCC was contained in a single file,
objc-act.c, which was around 10K lines. It didn’t have any clear separation between the compiler stages and it was littered withif (next_runtime)everywhere. Some of the new features needed a new runtime, so all of those would need auditing and extending to provide a different implementation and become exciting switch cases.At the time, clang had mostly working codegen for C (it miscompiled
printfimplementations, but a lot of C code worked). It also had parsing and semantic analysis support for Objective-C, but no code generation. I started by adding an abstraction layer separating the language-specific parts from the runtime-specific parts. That’s still there: there is an abstractCGObjCRuntimeclass with a bunch of different subclasses (Apple has two significantly different runtimes and a bunch of different variants of the newer one, so has made a lot of use of this abstraction). For a while, clang had better support for Objective-C on non-Apple platforms than on macOS.Clang now has a bunch of source files that are larger than
objc-act.c, but they’re cleanly layered. Parsing, semantic analysis, and IR generation are all in separate files. Objective-C runtime-agnostic IR generation is mostly in one file, Apple runtimes in another, non-Apple runtimes in a third. If you want to navigate the codebase and modify the Objective-C support, it’s easy to find the right place.[1] The FSF used to point to Objective-C as a big win for the GPL. I consider it a great example of failure. NeXT was forced to open source their GCC changes but not their runtime, which made the changes useless in isolation. Worse, the NeXT code was truly awful. If NeXT had offered to contribute it to GCC, I strongly suspect that it would have been rejected, but because the FSF had made such a big deal about forcing NeXT to release it, it was merged.
-
andy@ark ~/d/zig (master)> wc -l (find src/ -name '*.zig') | sort -nr 186923 total 23226 src/Sema.zig 11146 src/AstGen.zig 7805 src/codegen/llvm.zig 6745 src/clang_options_data.zig 6716 src/link/MachO.zig 6586 src/translate_c.zig 6359 src/arch/x86_64/CodeGen.zig 6187 src/type.zig 5554 src/Module.zig 5247 src/value.zig 5181 src/Compilation.zig 5171 src/arch/arm/CodeGen.zig 5134 src/main.zig ...shrug
-
Maybe some of those source files are too big and ought to be broken up into smaller subcomponents?
-
-
-
I worked on a C project with a function with 10kSLOC and 46 parameters. Then someone decided to add another parameter because at that point, I guess why not.
-
-
Tests should be at the bottom for me. For multiple reasons, but one I’d like to highlight is they handle the API semantics anyway.
-
it has some serious, even prohibitive limitations when it comes to applications that require short run-times and low latency … Julia also suffers from another limitation it shares with many other languages: garbage collection
Short runtime + low latency requirements would indicate to me that a language compiled to native code with a GC would be the best choice - both C++ and Rust, if used properly, would “waste” time deallocating at the end just to hand back the memory to the OS anyway. With a GC, you get the benefits of never deallocating unless you use up too much memory, which is unlikely in a short-lived program.
On Zig:
Instead, there are structs dedicated to managing memory known as allocators. One intriguing consequence of this is that different programs are free to take completely different approaches to memory management
One can use allocators if one wants to even in C, C++ has them as template parameters in its own standard library, and D even has pre-built ones.
-
This was from 2007. Do people still see this problem nowadays? Admittedly all the interviewing I’ve done recently has been somewhat removed from the front lines, and I’m hiring for positions where people would be working in a somewhat obscure language (Clojure) but I’ve never seen anything like this.
-
In 2014 i had a group project at uni. The group had 12 people, and the top-grade student wasn’t able to implement a histogram computation for a grayscale image. Which is basically “foreach(pixel) array[pixel] += 1” He also wasn’t able to ask for help in the two days he struggeled with a solution. So my N=1 sample here is that it was necessary 7 years later, and it probably still is
-
I have had my first experience at hiring programmers this year and although I have not had the issue described in the article (we do not advertise publicly and are not hiring entry level people at this time), I have also had issues with people who won’t ask for help. For me this is the worst possible failure. If a person came and said “I actually don’t know how to write a loop in this language” I would be patient enough to evaluate how fast they can learn, knowing they would need to do a lot of it. But people who get stuck on things that there is no way they could possibly know (e.g. internal company design decisions we forgot to inform them about) and then just waste time trying to puzzle it out without the required information… I can’t work with people like that. You send them some work with a two week deadline, and they say they are making progress, and then 2 hours before the deadline you find out they have done nothing at all.
-
-
Not to defend the student, who should have asked for help, I suspect that the idea you can do array[pixel] probably just didn’t occur to him, so he got stuck. Similarly for fizzbuzz, I think many coders just don’t know about % because you don’t use it very often. A problem for junior/poor coders is you get stuck in certain places, and then there’s nowhere to go but to try some xyproblem.info nonsense.
-
I think many coders just don’t know about % because you don’t use it very often.
Maybe it depends on the level of abstraction, but I use % (or, more often, bit-twiddling special cases of it) quite a lot in C/C++. That said, you don’t actually need % for fizzbuzz. The pattern repeats every 15 elements, so if you don’t know about modulo arithmetic (which is surprising, since it’s a pretty important part of a computer science degree, even if you don’t know that there’s a built-in operator for it) then you can write something like this:
long i=1; while (1) { printf("%ld\n", i++); printf("%ld\n", i++); printf("Fizz\n"); i++ printf("%ld\n", i++); printf("Buzz\n"); i++; printf("Fizz\n"); i++; printf("%ld\n", i++); printf("%ld\n", i++); printf("Fizz\n"); i++; printf("Buzz\n", i++); printf("%ld\n", i++); printf("Fizz\n"); i++; printf("%ld\n", i++); printf("%ld\n", i++); printf("FizzBuzz\n"); i++; }This implementation doesn’t require modulo arithmetic at all. If I saw this in an interview, I’d then ask how to do it in a single printf call in the loop. This would then hit an interesting corner case in C, because argument order is undefined and so a correct solution couldn’t use
i++in the arguments and would have to look like this:long i=1; while (1) { printf("%ld\n%ld\nFizz\n%ld\nBuzz\nFizz\%ld\n%ld\Fizz\nBuzz\n%ld\nFizz%ld\n%ld\nFizzBuzz\n", i+1, i+2, i+4, i+7, i+8, i+11, i+13, i+14); i += 15; }So you can actually write C FizzBuzz implementation with a single statement in the loop body and no arithmetic other than addition.
This isn’t a particularly optimised FizzBuzz, but it’s very simple and the required knowledge to build it in any language is:
- Loops
- Addition
- Outputting strings and integers
- The rules of FizzBuzz
- The ability to spot a repeating pattern in a trivial sequence
-
-
-
Yes. I regularly interview candidates for dev roles. Somebody handed in this:
def oh_god_why(): return MyClass.__init__() class MyClass(object): def __init__(self): # notice the lack of `pass` hereI don’t even dare guess what they meant.
That’s one example. I’ve… seen things.
-
You’re misjudging “this person has spent about 5 minutes to google for python syntax, a language they probably never used before” for “this person has never programmed”. Sure it could be true, but it’s not a string indicator - especially the
passis something that is very weird to someone being used to C-like languages.void foo() {}is perfectly fine.-
I’m not sure I can agree with this. An average developer proficient in C should not be so confused that they would submit something like that above (assuming that’s a verbatim submission).
I wouldn’t expect perfection, or even idiomatic usage or the language, but if you’re sending in code to apply for a job the code should at least make sense. I would not want to hire someone willing to send in code that doesn’t make sense - though I would also accept submissions in nearly any language that a candidate is familiar with.
-
-
-
It wasn’t in realtime, they had to refactor a tiny codebase in their own time and submitted that.
-
Well…then…I think I would ask why submit it at all?
-
I would ask the same… but the person in question emailed that in.
-
-
-
-
-
I’d expect someone who says they know Python in their CV to well… know Python. And I’m not claiming this is proof of “has never programmed”, I’m saying it’s proof that people that can’t program apply for programming jobs.
-
-
-
-
The nice thing about Go is that when it is verbose, like for the list deletion example, it highlights that the computer is doing more work. If you are constantly deleting items in the middle of an array (or if you’re doing this at all), it might not be the best choice of data structure.
-
This sounds like a post hoc rationalization to me. You can always hide arbitrary computations behind a function call.
I believe this is explained by the lack of generics, and I predict that, if/when generics are implemented, go stdlib will gain a bunch of slice-manipulating functions a-la
reverse.-
-
This sounds like a post hoc rationalization to me.
This was always pretty reliably cited as the motivation for a lot of design decisions, back when Go was just released.
You can always hide arbitrary computations behind a function call.
Yes, but one nice thing about Go is that functions are essentially the only way to hide arbitrary computation. (And, relatedly, the only mechanism that Go has to build abstraction.) When you’re reading code, you know that
a + bisn’t hiding some O(n²) bugbear. That’s valuable.-
Yup, I agree with the general notion that reducing expressive power is good, because it makes the overall ecosystem simpler.
But the specific example with list manipulation is “a wrong proof of the right theorem”, and this is what i object to.
Incidentally, a + b example feels like a wrong proof to me as well, but for a different reason. In Go, + can also mean string concatenation, so it can be costly, and doing + in a linear loop can be accidentally quadratic. (I don’t write Go, might be wrong about this one).
-
In Go, + can also mean string concatenation, so it can be costly
How do you mean? String concatenation is O(1)…
doing + in a linear loop can be accidentally quadratic. (I don’t write Go, might be wrong about this one).
How’s that?
-
String concatenation is O(1)…
Hm, I don’t think that’s the case, seem to be O(N) here:
14:48:36|~/tmp λ bat -p main.go package main import ( "fmt" "time" ) func main() { for p := 0; p < 10; p++ { l := 1000 * (1 << p) start := time.Now() s := "" for i := 0; i < l; i++ { s += "a" } elapsed := time.Since(start) fmt.Println(len(s), elapsed) } } 14:48:40|~/tmp λ go build main.go && ./main 1000 199.1µs 2000 505.49µs 4000 1.77099ms 8000 3.914871ms 16000 14.675162ms 32000 49.782358ms 64000 182.127808ms 128000 661.137303ms 256000 2.707553408s 512000 11.147772027s-
You’re right! O(N). Mea culpa.
-
-
-
-
-
-
it highlights that the computer is doing more work
It seems strange to me to trust in the ritual of doing strange and obnoxius, bug-prone, programming to reflect the toil the computer will be burdened with. Doubly strange when append() is generic magical go function built-in which can’t be implemented in Go so it’s hard to even say, without disassembly and studying the compiler, what the code would even end up doing at runtime; I guess we can make very good educated guesses.
-
IMHO, no performance concerns are valid* until a profiler is run. If it wasn’t run, the program was fast enough to begin with, and if it was, you’ll find out where you actually have to optimise.
- Exception: if there are two equally readable/maintanable ways to write code, and one is faster than the other, prefer the faster version. Otherwise, write readable code and optimise when/if needed.
-
I feel this statement is a bit too generic, even with the exception. Especially when you’re talking about generic stdlib(-ish) functions that may be used in a wide variety of use cases, I think it makes sense to preëmptively think about performance.
This doesn’t mean you should always go for the fastest possible performance, but I think it’s reasonable to assume that sooner or later (and probably sooner) someone is going to hit some performance issues if you just write easy/slow implementations that may be “fast enough” for some cases, but not for others.
-
-
-
Source code verbosity is poorly correlated with runtime cost, e.g. bubble sort code is shorter and simpler than other sorting algorithms.
Even in the case of removing items from arrays, if you have many items to remove, you can write a tiny loop with quadratic performance, or write longer, more complex code that does extra bookkeeping to filter items in one pass.
-
Then don’t build your language to hide what’s computationally expensive. Fortunately, most languages don’t do any sort of hiding; folding something like list deletion into a single function call is not hiding it - so this statement isn’t relevant to the discussion about Go (even if true in a vacuum). Any function or language construct can contain any arbitrary amount of computational work - the only way to know what’s actually going on is to read the docs, look at the compiler, and/or inspect the compiled code. And, like @kornel stated, “Source code verbosity is poorly correlated with runtime cost”.
-
-
-
If you’re referring to compilation or code-generation - that work is trivial. If you’re referring to unnecessary computation hidden in function calls - writing the code by hand is an incredibly bad solution for this. The correct solutions include writing efficient code, writing good docs on code to include performance characteristics, and reading the function source code and/or documentation to check on the performance characteristics.
As @matklad said, “You can always hide arbitrary computations behind a function call.”e
-
-
-
-
Good lord, how is it elegant to need to turn your code inside-out to accomplish the basic error handling available in pretty much every other comparable language from the last two decades? So much of Go is well-marketed Stockholm Syndrome.
-
I don’t think that responding with a 404 if there are no rows in the database is that any language supports out of the box. Some frameworks do, and they all have code similar to this for it.
-
And sadly so often error handling is often done in a poor manner in the name of abstraction, though really bad one that effectively boils down to ignore that things can go wrong, meaning that one ends up digging through many layers when they actually do go wrong.
People eventually give up and copy paste StackOverflow[1] solutions in the hopes that one of them will work, even when the fix is more accidental and doesn’t fix the root cause.
The pinnacle was once checking code that supposedly could not fail. The reason was that every statement was wrapped in a try with an empty catch.
But back to the topic. Out of the box is all good and nice until you want to do something different which in my experience happens more often than one would think. People sometimes create workarounds. In the example of non existing rows, for example doing a count before fetch, so doing two queries instead of one, just to avoid cases where a no rows error would otherwise be thrown.
Now i am certainly not against (good) abstractions or automation, but seeing people fighting against those in many instances makes me prefer systems where they can be easily be added and can easily be reasoned about, like in this example.
[1] Nothing against StackOverflow, just blindly copy pasting things, one doesn’t even bother to understand.
-
-
In what way is Go’s error handling turning my code inside out?
-
-
-
compiles unusually fast
There is nothing unusual about it. It’s only C++ and Rust that are slow to compile. Pascal, OCaml, Zig and the upcoming Jai are decent. It’s not that Go is incredible, it’s that C++ is really terrible in this regard (not a single, but a lot of different language design decisions made it this way).
-
For single files, I agree. But outright disallowing unused dependencies, and designing the language so that it can be parsed in a single pass, really helps for larger projects. I agree on Zig and maybe Pascal too, but in my experience, OCaml projects can be slow to compile.
-
-
My impression from hanging out in
#zigis that the stage 1 compiler is known to be slow and inefficient, and is intended as a stepping-stone to the stage 2 compiler, which is shaping up to be a lot faster and more efficient.Also there’s the in-place binary patching that would allow for very fast incremental debug builds, if it pans out.
-
-
-
Don’t forget D.
-
-
My experience with Go is that it’s actually very slow to compile. A whole project clean build might be unusually fast, but it’s not so fast that the build takes an insignificant amount of time; it’s just better than many other languages. An incremental build, however, is slower than in most other languages I use; my C++ and C build/run/modify cycle is usually significantly faster than in Go, because its incremental builds are less precise.
In Go, incremental builds are on the package level, not the source level. A package is recompiled when either a file in the same package changes, or when a package it depends on changes. This means, most of the time, that even small changes require recompiling quite a lot of code. Contrast with C, where most of the time I’m working on just a single source file, where a recompile means compiling a single file and re-linking.
C’s compilation model is pretty bad and often results in unnecessary work, especially as it’s used in C++, but it means that you can just work on an implementation by just recompiling a single file every build.
-
I have not encountered many packages that take more than one second to compile. And the Go compiler typically parallelizes compilation at the package level, further improving things. I’m curious to see any counter examples, if you have them.
-
-
-
-
-
Srsly, I was looking for simpler, and was disappointed by the false promise.
-
-
atilaneves
2 years ago
|
link
| on:
Modern IDEs are magic. Why are so many coders still using Vim and Emacs?
Most IDEs create entire worlds where developers can create, but creating requires configuration. It takes time to adjust that world, to play god, to create shortcuts and hotkeys, to get used to different command structures and UI.
The authors seem to be describing Emacs.
With code completion, Git control, and even automatic deployment systems, modern IDEs are a Swiss Army Knife of features
Do people just not know what can be done in Emacs? Git control???
-
Emacs is amazing, but has a horrible issue with discoverability. “Distros” like Doom or Spacemacs have tried to address that, but, regardless of my personal feelings on them, they haven’t really broken into the mainstream enough to be noticed. (And as someone who used Emacs for years, I think the discoverability bit is very real.)
-
-
emacsclient --alternate-editor=''Really, Emacs in daemon mode is faster than anything else. Even vim is slow to start in comparison. I don’t understand why Emacs is the only editor which does this. I guess, you need to be reeeealy slow for someone to actually implement this feature :-)
-
-
Yeah, I actually have the same experience, getting useful setup of daemon mode took me several tries. Like, you need to pass an empty string as an argument to
--alternate-editorto have it auto-spawn the daemon if it isn’t running yet. That’s right next to sending SIGUSR1 to du in my chart of horrible CLI interfaces :)
-
-
-
-
-
-
Emacs is amazing, but has a horrible issue with discoverability.
I do want to specifically point out that the parent comment emphasized the mention of “Git control”, when one of the aspects of the Emacs ecosystem a bystander is most likely to have heard praise for is magit (another being org-mode). So it’s not so much a matter of whether it’s possible to discover magit’s existence in a vacuum – it’s that having access to one of the best interfaces to Git, period, is something often evangelized to non-Emacs-users as a point in Emacs’ favor.
-
-
C-hand friends are great, but you kind of have to know what the options are to get the help you want. For example, you can do a lot of Git manipulation in dired, but you need to know that exists, or at least to think of maybe dired might have some stuff in that area, to even know what you should be asking for. Experienced users, sure, I think this is sufficiently discoverable, but new users (and I mean that fairly broadly), it’s trickier.
-
-
-
-
Nim is roughly in the same space as Rust, but with somewhat better ergonomics in the language.
-
Yes, and zig and D-lang. All three of these deserved a mention if Ada gets a spot, in my opinion.
-
Just as my own sanity check, D is not memory safe: https://run.dlang.io/is/FxoDZr. I haven’t tried this, but I believe Ada rejects such code.
-
Neither is Zig. Nim is memory safe thanks to GC, and possibly thanks to a borrowing system once they get it polished.
Incidentally, Araq just hours ago posted that async seems to be working with it ORC: https://forum.nim-lang.org/t/6549#42774
-
That won’t compile if you add
@safe:to the top of the module or individually to each function: https://run.dlang.io/is/G9fIev
-
-
-
Nim has a garbage collector… I wouldn’t call it the same space as Rust. More like same space as Go?
-
-
Maybe a GC would be fine for those cases, but by choosing rust one is choosing to avoid a GC. That’s the thing that makes rust special and different and worth talking about at all. Rust with a GC would just not be a big conversation buzz. It would be just another language.
-
-
-
-
-
-
Editors are not a group you can reasonably compare. A JPEG editor and a WAV editor have nothing in common and exactly the same applies to “text” editors. You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
If you squint a little you probably would want to edit all programs in the same way but that unfortunately doesn’t work because you just cannot refactor Python like you refactor Java. So even trying to do such a comparison for IDEs is tricky.
All of these discussions about how far away the arrow keys - or your mouse - are, are completely pointless in comparison. This article almost caught on to this fact and gives a list of editors with their strength but doesn’t make it to the conclusion that text editors are just not a reasonable thing to discuss anymore. Nobody edits “text”. And the people who do use Word.
-
All of these discussions about how far away the arrow keys - or your mouse - are, are completely pointless in comparison. This article almost caught on to this fact and gives a list of editors with their strength but doesn’t make it to the conclusion that text editors are just not a reasonable thing to discuss anymore. Nobody edits “text”. And the people who do use Word.
That’s amusing. Just last night my roommate had a problem where all the cells in her document were set to “£0.00” after saving, it would have taken about 8 hours for her to recover it.
Luckily she already had the list, she just wanted it in a Libreoffice table. I opened vim (Because raw sed is line-based), and just used the expressions:
:%s/$/"/,:%s/ \r/, "/. I saved it as a .csv and opened it in Libreoffice Calc, copied the cells, then imported it into Writer by doingC-Vand selecting “RTF”.It took five minutes, not 8 hours. A handful of small adjustments that added 2 minutes on to it, and the table was exactly as before.
You are severely underestimating the power and use of the tools programmers have been using for the last 60 years. The arrogance on display is absolutely staggering.
The purpose of a tool like Vim, is to make your editing commands become muscle memory, so you do not have to think about what you want to do, and then do it. You can simply do it reflexively. I’ve been using it for 8 years on all kinds of information, and it’s extremely powerful.
I wonder, when I hit
d}in vim, am I still editing text? The operation itself runs over a paragraph. What aboutdi)which deletes the contents of a set of brackets, do you think you can find a use for that in programming? If your answer here is “paragraphs and argument lists are all text”, that means you’re coming close to the realization that “XML is text”, “S-Expressions are text”, and pretty much every format you can think of is a form of text.Is ParEdit, a mode for editing the structure of Lisp documents in Emacs, still editing text? What about rope, a refactoring tool for Python with both Emacs and Vim plugins, when you use those plugins, are you still editing text? Vim provides an interface for “text objects”, which allow you to manipulate XML rather easily, are you editing text, or the objects? Of course you are editing text. Because everything that isn’t a binary format is text.
In addition, both Emacs and Vim have binary editors built in. They both have extensive sets of plugins for editing programming languages while retaining the power of each environment, and the benefit of a uniform interface that you have experience and will log 100s of hours in.
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
You’re right, however, that you don’t want to edit all programs the same way. However, most modern text editors have functionality specific to what you’re editing. Hell, the exact commands used for alphabetically sorting the contents of CSS directives in Vim (
:%g/{/ .+1,/}/-1 sort) dates back to before Vim existed – it was a feature ofed(1)– which isn’t even visual!-
So, I think your argument is: we are not editing text we are editing all kinds of things in a text representation. Which is also what I was trying to say.
I think we just disagree about the effectiveness of this approach. I agree that the editors we have are often better than nothing but I wouldn’t advertise vim for it’s ability to edit tabular data using :s. Neither do I think the incantation in your last line is something that I should have to know. I just want a CSS editor and then press the sort button.
#editorflamewar2020
-
So, I think your argument is: we are not editing text we are editing all kinds of things in a text representation.
No, my point was that we are all editing text, we just smooth it over so we don’t have to care about the specifics. Text editors do this too, except they provide more power if you wish it. From your comment here and your other comments about this, you seem grossly uneducated about the capabilities of text editors as tools. I’ve linked you articles in another comment – please read them.
I just want a CSS editor and then press the sort button.
You don’t have to know it.
nnoremap <leader> s :%g/{/ .+1,/}/-1 sortMy leader key is set to
f. So I can just typef sand the document is sorted. In emacs, I can set a mode switch, so that this keybinding is only available for CSS documents.-
In emacs, I can set a mode switch, so that this keybinding is only available for CSS documents.
FYI, you can do the same in Vim with ftplugins.
-
No need for anything complex, just use autocmd
-
-
-
You are not expected to know this incantation. It is not a word, it is a sentence. Do you «know» the sentence «Neither do I think the incantation in your last line is something that I should have to know.» ?
The reason we prefer to use a single editor with a language to think about editing in [without agreeing which one] is that things that are just out of reach of the functionality supplied by the editor/plugins/etc. are actually within reach if you express them in the slang of the editor.
Learning such a language probably doesn’t pay off if handling the things typically represented as text is not an activity taking up a large share of your computer use. And maybe not if you just need five representations and the single-use options available for these representations are good enough from your point of view.
But as there are more (and slightly less popular) things to handle, no thanks, I will take a small set of tools I know well and can push one step beyond over sorting through special-case solutions, with many of them missing «trivial» features I have come to depend on.
-
-
-
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
Why not? As an Emacs user, I appreciate that different modes interpret concepts such as paragraphs, words, symbols, top level structures in their own appropriate ways, so that keybindings and other commands trivially adapt to the kind of file I’m working with. Sure, not everything fits perfectly into the abstract, primitive categories, but it’s still preferable to having n different editors and work environments for n different problems.
-
“Why not?”
Because having a dedicated editor is more efficient. PyCharm is better for Python and IntelliJ better for Java and Kile better for Latex and Trello better for Todos and Thunderbird better for Mails.
The reason you can use Emacs reasonably well for all of those is that it actually tries to be n different editors for n different problems. org-mode barely has anything in common with other modes. Paredit as well, etc…
-
Because having a dedicated editor is more efficient
Having a dedicated editor may be more efficient for a single thing, but most of the projects I work on involve ‘text’ files with at least half a dozen different languages (programming languages, build system scriting, markup languages). If I had to switch between editors for each one then each editor would have to be a lot more efficient to offset the cost of the cognitive load from having different tooling for each file type.
-
Well I always feel a lot less productive in IntelliJ editors, and find the Emacs equivalents a lot more comfortable. I guess that’s a difference in attitude, but I think it should be considered as a factor.
The reason you can use Emacs reasonably well for all of those is that it actually tries to be n different editors for n different problems
Major modes are interfaces, of sorts, connecting and implementing the files to existing subsystems (completion-at-point, imenu, xref, …), not their own Editors. Sure, some major modes have more additional keybindints, such as Org mode, but it’s not unrelated: I can still use isearch, avy,
highlight-symbol-at-point, etc. or any other function that wasn’t written for a specific mode in mind. Paredit is a minor mode for S-Expression based file formats, so I don’t quite get your point here…
-
-
-
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
I most certainly want to do exactly that.
-
Nobody edits “text”. And the people who do use Word.
Thanks for making this point clear. I think programmers (usually systems programmers who post to Lobsters or The Orange Site) have basically boxed themselves into a “fake reality” where talking about plain text email and using vi for prose actually matter. I’m basically the only systems programmer on my team (excluding maybe one or two others), and talking to clients (who almost are all non-technical industries) or even other team members - they have big HTML signatures with images (and use formatting in their messages!), reply on top, use IDEs or maybe Notepad++ on Windows. They don’t tweak their window manager or Emacs config, they’re busy whatever line-of-business application.
-
I’m not sure that applies to everyone. I have proselint jacked into vim and I use it frequently. The ability to perform operations on sentences or paragraphs without having to move your hand to the mouse is underappreciated.
-
-
-
I had never heard of or thought of using colours to highlight which parens close which and now it seems so obvious in hindsight I want this yesterday.
-
It’s been a thing (at least in Emacs) for a while: https://www.emacswiki.org/emacs/RainbowDelimiters
-
-
Can anyone recommend some material describing concrete motivations for adding generics to Go? I’m aware of the abstract idea that you need generics in order to build data structures that work with different types, but is there a real-world setting where this is actually better than just copying code and doing s/type1/type2/g? My sense is that projects that use complex data structures almost always customize the data structures in a way that depends on the data type being stored.
-
I hope it’s not that hard to imagine wanting different data structures than hash maps; maybe your problem fits better into a binary search tree for example.
Well, I for one usually don’t feel like implementing my own red-black tree, so I would like to just grab a library. That library will be much nicer to use if I can just make an
RBTree<string, MyFoo>. I certainly wouldn’t want to copy/paste an int->string red-black tree into some file(s) and judiciously search/replace until I have a string->MyFoo tree (and then do the same when I need an int->float tree).-
That makes sense, but I am still looking for some grounding in actual programming practice. Is there a use of a red-black tree that would not warrant customizing it for the storage type? Or one where it would make sense to add a library dependency rather than copying the RB tree code?
-
How do you write a library that provides a Red-Black tree that can in principle work with many different client types without generics? This isn’t a rhetorical question, I don’t know Go and I genuinely don’t know how you would implement this kind of library in Go without generics.
-
-
Right, that’s a great example. Because it uses interface{}, it:
- Requires all keys and values to be heap allocated, leading to worse performance, worse memory usage, and worse memory fragmentation. Requiring two heap-allocated ints to store one value in an int->int concurrent hash map is unacceptable for many uses.
- Is less ergonomic, requiring a cast every time you want to use a value.
- Provides no type safety. (I imagine this one will be the least convincing to Go programmers, since Go generally expects the programmer to just not make mistakes)
-
-
I’m using a sync.Map (for its concurrency support - I have many goroutines writing map entries, and another goroutine periodically ranging over the entire map to dump it to a json file).
However I know the types I write to the map, I have no need for
interface{}.Am I better off with a real typed map + using sync.RWLock/mutex/etc. directly (in a custom struct)? Performance-wise.
-
I don’t know, you would have to benchmark or measure CPU or memory usage. The sync.Map documentation suggests that using a regular map + mutexes could be better though: https://golang.org/pkg/sync/#Map
The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content.
The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex.
If your usage falls outside of the two use cases which sync.Map is optimized for, it would absolutely be worth looking into replacing your sync.Map with a regular map and a mutex.
I suppose it becomes a question of which has the biggest performance penalty for you, heap allocation + indirection with sync.Map or lock contention with regular map + mutex?
(Also, in most cases, this probably doesn’t matter; make sure you’re not spending a long time improving performance in a part of your code which isn’t actually a performance issue :p)
-
Right - the code “just works(TM)” and it takes around 0.5 seconds to render the JSON file every minute (which I track with metrics just to be safe) so it should be fine to keep as is. This is just a for-fun conversation.
or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex.
I definitely remember reading this sentence and it made me choose sync.Map because it sounds like my usecase. But like you say if I don’t measure it’ll be hard to tell.
-
-
-
-
-
Here is a popular Go RB tree implementation https://github.com/emirpasic/gods/ that uses Interface{} for the key and value types. Just search github for uses of it… With generics, users of this library would get greater typesafety.
https://github.com/search?q=%22github.com%2Femirpasic%2Fgods%2Ftrees%2Fredblacktree%22&type=Code
-
-
To short-circuit this:
At a previous job, I worked on a tool that started various services. The services had different dependencies, each of which needed to be started before the service. We wanted to be able to bring them up with as much parallelism as possible, or have a flag to launch them serially.
A simple approach to doing this correctly is modeling the dependencies as an acyclic graph (if it’s a cyclic graph, you’ve got a problem — you can never bring the services up, because they all depend on each other). To launch them in parallel, launch each one that has its dependencies met. To launch them serially, topologically sort the graph into an array/list/whatever and launch them one by one.
A generic graph implementation would be very useful, as would a topological sort that worked on generic graphs. With Go, you can’t have one that’s type-safe.
Another great use case for graphs: visualizing dependency graphs! You can have an open source graph visualization library, build a graph of whatever it is you’re trying to visualize, and pass it to the library and get a nice visualization of the data.
Graph data structures can be quite useful. Supporting generics makes them type-safe, so you catch errors at compile time instead of runtime. Some other examples of the usefulness of graphs:
- Graphs of friends at a social network (I currently work at one, and we use generic graph data structures all over the place — graphs of people to people, graphs connecting people and photos they’re tagged in, etc)
- Network topology graphs
- Graphs of links between documents
etc.
And it’s not just graphs. How do you write a type-safe function that takes in a list of possibly-null items, and returns a new list with the nulls stripped out? How about a function that takes a map and returns the list of its keys? In Golang, the answer is always copy-paste or give up type safety. In languages with generics, you can trivially write these functions yourself if they’re not in the standard library.
-
-
Huh. It had not occurred to me that github search would require a login.
-
-
-
-
-
-
-
To turn the question around, why would you want to manually copy/paste code all over the place when the compiler can do it for you? And while I personally think “DRY” can be over done, not having the same (or very similar) code copy/pasted all over the place seems like a big practical win.
As far as customizing specific data structure and type combinations, most languages with generics have a way to do that, and I’d bet the Go designers thought of it.
-
Copy / paste has got it’s own problems, but it lets you avoid a ton of complexity in the toolchain.
Toolchain development is all about tradeoffs. For instance, I use Typescript; the reference implementation is featureful, but slow to boot, so it keeps a background process alive to cache the heavy lifting, which accumulates state and introduces subtle confusions (eg type errors that don’t exist) until it’s restarted.
For some problem spaces, the problems introduced by copy/paste pale in comparison to the problems introduced by slow, stateful compilers.
-
Copy/paste vs generics is unrelated to compiler bugginess.
If you carefully pick TypeScript as the comparison point, you can make the case that a buggy toolchain is bad (not that most users care, they just restart the compile process when it starts to go bad).
But if you were to pick say ReasonML for comparison, you could say that it’s possible to have a solid generics implementation (much less copy-pasting) and a super-fast, accurate compiler.
I.e. you can have both buggy and non-buggy compilers supporting generics. Hence, unrelated.
-
ReasonML is great!
That said, while the relationship is indirect, it’s there. Adding complexity is never free. It didn’t cost ReasonML speed or reliability, but it costs maintainers time and makes every other feature more difficult to add in an orthogonal way.
-
-
IMO, there’s such a thing as a right time to do it.
In the early years it’s more important to keep the simplicity - there aren’t that many users and you’re still figuring out what you want the language to be (not every feature is compatible with every approach to generics).
Once you’re ready to start generics you need to answer questions like - do you want monomorphisation or lookup tables? Is boxing an acceptable overhead for the improved debugging ergonomics?
-
-
-
-
Being able to use lots of existing OCaml bits is a huge advantage.
Typescript has been able to compete due to the sheer number of contributors - MS pays quite a large team to work on it (and related stuff like the excellent Language Server Protocol, VScode integration).
However, large teams tend to produce more complex software (IMO due to the added communications overhead - it becomes easier to add a new thing than find out what existing thing solves the same problem).
-
I should clarify my comment was more about comparing performance of the two languages.
OCaml is a well optimized language that targets native machine code so tooling built in OCaml should be more performant than tooling built in Typescript. As a result, it’s hard to compare the complexity of either tool by the performance of the tool. It’s apples and oranges.
-
-
isn’t Typescript self hosted, whereas ReasonML is written in OCaml? It seems like Typescript would have a very hard time competing.
That’s a strange argument. If it were very hard for them to compete why would they not use OCaml as well, especially since its contemporary alternative, Flow, was written in OCaml too? Or why would they not make TypeScript as good as a language for writing TypeScript in as OCaml is?
-
My comment was more about performance, but it wasn’t very clear. It’s hard for Typescript, which is compiled to Javascript and then interpreted/JITed, to create tooling that’s as fast as a language that builds optimized native code.
Given that Typescript is self hosted it has the advantage that community involvement is more seamless and I don’t want to downplay the power that brings.
-
-
-
-
-
-
-
I think Typescript is a straw man. Does this Go implementation of generics slow down the compiler a noticeable amount? There’s nothing inherent to generics that would make compiling them slow.
On the other hand, copy/pasted code is an ever increasing burden on developer and compile time.
-
-
-
You are imagining a code base where the same complex data structure is instantiated with two different types. Is that realistic?
Realistic enough that the Linux kernel developers went through the hassle of developing generic associative arrays, circular buffers, and other generic data structures using void*.
And so did Gnome with GLib, which provides generic lists, hash tables, and trees, along with several others structures, also using void*.
And the standard libraries of most modern languages include reusable and generic sequence and associative data types, and some times significantly more than that.
For most data structures, though, focusing on a single code base gives too narrow of a view. Generics allow libraries of data structures to be created, so even though a single code base only use one R* tree (or whatever), that R* tree library can be used as-is by any number of projects.
-
-
-
The Abstract and Background sections of the draft design doc touch on the motivations. Additionally, each section describing a dimension of the design usually mentions, at least briefly, the motivation for that feature.
-
Here is an example that I’ve wanted for ever, and can finally do. Higher order combinators that you can leverage first class functions with!
-
-
“Can’t you just …” goes forever. “Can’t you just write your for loop with labels and jumps in assembly?”^^
For me, it’s all about abstraction. Having low level combinators, like this, that I can compose to build higher level abstractions in a generic way is wonderful.
^^: See also whataboutism.
-
I’m not sure that composing from higher level abstractions is always such a good idea. I like both Go (hobby projects) and Rust (work!) but I still fell that most of the time I prefer this level of abstraction:
type Server struct { ... Handler Handler // handler to invoke, http.DefaultServeMux if nil ... } type Handler interface { ServeHTTP(ResponseWriter, *Request) }from this:
pub fn serve<S, B>(self, new_service: S) -> Server<I, S, E> where I: Accept, I::Error: Into<Box<dyn StdError + Send + Sync>>, I::Conn: AsyncRead + AsyncWrite + Unpin + Send + 'static, S: MakeServiceRef<I::Conn, Body, ResBody = B>, S::Error: Into<Box<dyn StdError + Send + Sync>>, B: HttpBody + 'static, B::Error: Into<Box<dyn StdError + Send + Sync>>, E: NewSvcExec<I::Conn, S::Future, S::Service, E, NoopWatcher>, E: H2Exec<<S::Service as HttpService<Body>>::Future, B>, { ...Don’t get me wrong, i like type level guarantees and I can see flexibility here, but in my experience with c++, rust and haskell is that generic programming often ends up complicating things to a degree that I personally don’t like.
-
I think this is going to be a balance that the community has to find. I don’t regularly program in rust, but I’d be quite surprised if it wasn’t possible to get something close to the Go http API in it. The example you pasted seems complicated for the sake of being complicated. In theory, the Go community has been drilled into thinking in terms of the littlest abstraction that’ll work, which maybe makes it possible to generally avoid generic APIs that don’t actually need to be?
-
-
“Can’t you just” does not go forever. It is a simpler way to say that the alternative is not significantly harder than what’s proposed. Is there some type of task that would be doable using a generic map but unreasonably hard using for loops?
I feel like Go was designed from the ground up to be written in an imperative style, and composing first order functions is more of a functional style of coding. If I understand, without generics you would be nesting for loops rather than composing map functions, which is no more difficult to understand or write.
I don’t follow the connection to whataboutism.
-
I think it’s fine for your style of writing code to be to use loops and conditionals instead of map and filter. I think it’s a fine way to code that makes more sense in an imperative language. Straight for loops and while loops with if statements inside them is just better, more easily understandable code in an imperative language, in my opinion, than
.map(...).filter(...).map(...)etc. -
Incidentally there is a repo wherein Rob Pike expresses his attitude towards this style of coding:
https://github.com/robpike/filter/
I wanted to see how hard it was to implement this sort of thing in Go, with as nice an API as I could manage. It wasn’t hard.
Having written it a couple of years ago, I haven’t had occasion to use it once. Instead, I just use “for” loops.
You shouldn’t use it either.
-
I mean… that’s like … one man’s opinion… man. See also.
Generics are going to create a divide in the Go community, and it’s going to be popcorn worthy. There’s no point of adding Generics to the language if this filter thing “shouldn’t be used,” and the community rejects the abstractions that Generics provide.
This divide is easily already seen in the community as it relates to test helpers. On the one hand, there’s a set of developers that say “stdlib testing is more than enough.” On the other hand, there are people who want the full testing facilities of junit, with matchers, lots of assert style helpers, etc. Who is right? They all are, because those things work for their respective teams and projects.
This general dogmatic approach to language idioms is why I call it “idiotmatic” Go.
-
-
-
-
You can’t pass a for loop to anything, you can only write it where you need it. Sure, toy examples look like toy examples, but the fact remains that Go has first-class functions, which should be a nice thing, but it doesn’t actually have a type system rich enough to express 90% of the things that make first-class functions worth having.
-
You can’t pass a for loop to anything, you can only write it where you need it.
right, so the example code could be done with a for loop no problem. is there a more motivating example?
it doesn’t actually have a type system rich enough to express 90% of the things that make first-class functions worth having.
how do you mean?
-
-
-
-
That works in some cases, but it’s pretty easy to find a counter example, too.
-
-
-
-
-
In terms of collections, the truth is most of the time a map/slice is a good option. Here’s my top two favorite use cases for generics in go:
- Result<T> and functions that compose over them.
- Typesafe versions of sync.Map, sync.Pool, atomic.Value, even a rust like Mutex
-
Oh man. I hadn’t even considered a better way to do error handling, eg. a Result type. People are gonna get so mad.
-
Generics isn’t enough to do what people want to do with error handling 99.99% of the time, which is to return early. For that, you either need a macro, such as the aborted
tryproposal, or syntactic sugar for chaining such “functions that compose over them” (like Haskell’sdonotation).Otherwise you end up with callback hell à la JavaScript, and I think nobody wants that in Go.
-
I was more thinking of something where the
if errpattern is enforced via the type system. You’re still not getting there 100%, you could get reasonably close, with a genericResulttype that panics when the wrong thing is accessed, forcing you to check always or risk a panic.r := thing() if r.HasError() { handleError(r.Err()) } else v := r.Val() { handleSuccess(v) }And of course it’s easy to question why this is interesting until you do chaining of things, and get a full on, type safe
Resultmonad.r := thing().andThen(func(i int) { ... }).andThen(func(i int) { ... }) if r.IsErr() { handleErrForWholeComputation(r.Err()) } else { handleSuccessForWholeComputation(r.Val()) }The alternative can be seen in things like this where you skirt around the fact that you can’t generically accept a value in one of those called functions. This is also why I said people are going to get so mad. These things are confusing to people who haven’t dealt with them before, and will make Go much more expressive, but less easy to grok without effort.
-
-
-
but is there a real-world setting where this is actually better than just copying code and doing s/type1/type2/g
All of them. Copying code manually is one of the worst things you can do in software development. If it weren’t, why even bother writing functions, ever?
My sense is that projects that use complex data structures almost always customize the data structures in a way that depends on the data type being stored.
The fact that libraries exist that don’t customise in such a way in languages with generics would disprove that notion.
-
-
but is there a real-world setting where this is actually better than just copying code and doing s/type1/type2/g
All of them. Copying code manually is one of the worst things you can do in software development. If it weren’t, why even bother writing functions, ever?
I disagree with your implication that the use of functions means code should never be copied. For example if you want to use strlcpy() in a portable C program, it makes more sense to put a copy in your source tree rather than relying on an external library. An extra dependency would add more headache than just copying the code.
My sense is that projects that use complex data structures almost always customize the data structures in a way that depends on the data type being stored.
The fact that libraries exist that don’t customise in such a way in languages with generics would disprove that notion.
That’s why I said “almost always.” And remember that the existence of a library doesn’t mean it is used with any frequency.
-
-
Suppose you have a structure parametrizable by types T1, T2. You’re writing it in Go, so you assume that it’s ok if T1=string, T2=int. Also, in some of the places, you were using int for purpose unrelated to T2 (ie. if T2=foo, then there are still ints left in the source). Another programmer wants to copy-paste the code and change some types. How does he do it?
-
-
-
Defer was unique to Go but Swift has since adopted it as well, although it is subtly different.
-
deferwasn’t unique to Go even when it was introduced given that D has hadscope(exit)for longer. -
-
-
ZIg also has
defer, anderrdefer, which I haven’t seen elsewhere.-
-
Sorry to mention D again in this thread, but
scope(failure)is pretty much similar toerrdefer.
-
-
-
This is interesting, and I think I agree with many arguments when it comes to the reasons java, OCaml, Haskell, Go, etc. haven’t replaced C. However the author cites rust only briefly (and C++ not at all?) and doesn’t really convince me why rust isn’t the replacement he awaits: almost all you can do in C, you can do in rust (using “unsafe”, but that’s what rust is designed for after all), or in C++; you still have a higherer-level, safer (by default — can still write unsafe code where needed), unmanaged language that can speak the C ABI. Some projects have started replacing their C code with rust in an incremental (librsvg, I think? And of course firefox) because rust can speak the C ABI and use foreign memory like a good citizen of the systems world. Even C compilers are written in C++ these days.
To me that’s more “some were meant for no-gc, C ABI speaking, unsafe-able languages” than “some were meant for C”. :-)
-
Besides Rust, I think Zig, Nim, and D are strong contenders. Nothing against Rust, of course, but I’m not convinced it’s the best C replacement for every use case. It’s good to have options!
Nonetheless, I imagine C will linger on for decades to come, just due to network effects and economics. Legacy codebases, especially low-level ones, often receive little maintenance effort relative to usage, and C code is incredibly widespread.
-
I love Rust, but I think Zig and D (in the ‘better C’ mode and hopefully their new borrow checker) are closer to the simplicity and low-level functionality of C. Rust is a much nicer C++, with simpler (hah!) semantics and more room to improve. C++ is, unfortunately, a Frankenstein monster of a language that requires a 2000 page manual just to describe all the hidden weird things objects are doing behind your back. Every time I have to re-learn move semantics for a tiny project, I want to throw up.
-
-
The lot of them! It was kind of a large gap between C and Python/Perl/Ruby/Java.
-
-
You might want to read this brief follow-up:
-
Maybe I’m the archetype of a C-programmer not going for Rust. I appreciate Rust and as a Mathematician, I like the idea of hard guarantees that aren’t a given in C. However, Rust annoys me for three main reasons, and these are deal-breakers for me:
- Compile time: This is not merely the language’s fault, and has more to do with how LLVM is used, but there doesn’t seem to be much push to improve the situation, either. It annoys me as a developer, but it also really annoys me as a Gentoo user when a Firefox compilation takes longer and longer with each subsequent Rust release. Golang is a shining example for how you can actually improve compilation times over C. Admittedly, Rust has more static analysis, but damn is it slow to compile! I like efficiency, who doesn’t? Rust really drops the ball there.
- Standard library/external libraries: By trying to please everyone and not mandating certain solutions, one is constantly referred to this or that library on GitHub that is “usually used” and “recommended”. In other cases, there are two competing implementations. Sure, Rust is a young language, but for that reason alone I would never choose it to build anything serious on top of it, as one needs to be able to rely on interfaces. The Rust developers should stop trying to please everybody and come up with standard interfaces that also get shipped with the standard install.
- Cargo/Package management: This point is really close to the one before it: Cargo is an interesting system, but really ends up becoming a “monosolution” for Rust setups. Call me old-fashioned, but I like package managers (especially Gentoo’s) and Cargo just works around it. When installing a Rust package, you end up having to be connected to the internet and often end up downloading dozens of small crates from some shady GitHub repos. I won’t make the comparison with node.js, given Cargo can be “tagged” to a certain version, but I could imagine a similar scenario to leftpad in the future. Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.
To put it shortly: What I like about C is its simplicity and self-reliance. You don’t need Cargo to babysit it, you don’t need dozens of external crates to do basic stuff and it doesn’t get in the way of the package manager. I actually like Rust’s ownership system, but hate almost anything around it.
-
-
See, I feel the exact opposite when it comes to Cargo vs. system’s package manager: managing versions of libraries using your system’s package manager is a royal pain in the ass or outright impossible when you have multiple projects requiring different versions of a library. In my experience with C and C++, you’ll end up using CMake or Meson to build exactly the same functionality that Cargo deploys for you, at a much higher cost than just adding one line in a configuration file.
In fact, my biggest gripe with C and C++ is that they still depend on a 3-step build system (preprocessing, compiling, linking) each of which requires you to specify the location of a group of files. I get why having header files was attractive in the 1970s when you counted your computer’s memory in KBs, but it makes writing and maintaining code such a pain in the ass when compared with a modern module system.
The funniest bit is I used to consider all these things as ‘easy to deal with’ when all I did was write C/C++ code 15 years ago. Nowadays, having to switch from Go, Rust or any other language to C for a small project makes me want to cry because I know I’ll spend about 20% of the time managing bullshit that has nothing to do with the code I care about.
-
Build systems in C give a feeling of craftsmanship. It takes a skill to write a
Makefilethat correctly supports parallel builds, exact dependencies, interruptions and cleanups, etc. And so much work into making it work across platforms, and compilers.And then Cargo just makes it pointless. It’s like you were king’s best messenger trained to ride the fastest stallions, and Cargo’s like “thanks, but we’ve got e-mail”.
-
LOL, I guess it’s a matter of age. When I first started programming, I’d love all that stuff. I can’t count how many libraries and tools I re-implemented or extended because they didn’t do something exactly the way I wanted it. Or the nights I spent configuring my Linux machine to work just right. Or the CPU time I spent re-encoding all of my MP3 collection to VBR because it’d save 5% of storage.
Now, I learned Cargo for a tiny project and I keep swearing every time I have to start a Python virtualenv because it’s just not easy enough, goddammit!.
-
-
-
This is a fair criticism of C, personally I would love to see a set commonly used data structures added to the C standard library. However, currently in the C world you either write your own or use something like glib, neither of these cases require the equivalent of Cargo.
-
However, currently in the C world you either write your own or use something like glib, neither of these cases require the equivalent of Cargo.
Neither does using the Rust standard library, which also has a hash table implementation (and many other useful data structures). You can just use
stdand compile your project withrustc.-
-
But you and FRIGN are complaining about the Rust standard library compared to C. The Rust standard library is much more comprehensive than the C standard library or the C standard library + glib. So, the whole point seems to be void if C is the point of comparison.
If you are comparing to the Java standard library, sure!
-
-
That’s just because it’s convenient and most people don’t really care that it happens. But it’s not inherent to the tooling:
$ git clone -b ag/vendor-example https://github.com/BurntSushi/ripgrep $ cd ripgrep $ cargo build --releaseOther than the initial clone (obviously), nothing should be talking to GitHub or crates.io. You can even do
cargo build --release --offlineif you’re paranoid.I set that up in about 3 minutes. All I did was run
cargo vendor, setup a.cargo/configto tell it to use the vendor directory, committed everything to a branch and pushed it. Easy peasy. If this were something a lot of people really cared about, you’d see this kind of setup more frequently. But people don’t really care as far as I can tell.where as a typical C application does not
When was that last time you built a GNU C application? Last time I tried to build GNU grep, its build tooling downloaded a whole bunch of extra goop.
-
-
TIL that a GNU C application is not a “typical” C application. Lol.
-
None of the C code I’ve worked on was written by GNU, and most of the C code out in the real world wasn’t written by GNU either. I find it frankly bizarre that you are seriously trying to suggest that GNU’s practices are somehow representative of all projects written in C.
-
You said “a typical C application.” Now you’re saying “representative” and “what I’ve worked on.”
If the implementation of coreutils for one of the most popular operating systems in history doesn’t constitute what’s “typical,” then I don’t know what does.
Talk about bizarre.
Moreover, you didn’t even bother to respond to the substance of my response, which was to point out that the tooling supports exactly what you want. People just don’t care. Instead, you’ve decided to double down on your own imprecise statement and have continued to shift the goal posts.
-
and most of the C code out in the real world wasn’t written by GNU either.
^ typical
I don’t have the time or patience to debate semantics though.
As for your other point, see FRIGN’s comment for my response. (It doesn’t matter what’s possible when the reality is random crates get pulled from github repos)
-
-
-
-
-
-
-
-
-
-
-
C doesn’t even have a hash table.
Why do you say “even”? There are many hash table implementations in C, with different compromises. It would be untoward if any of them made its way into the base language. There are other things missing in C which are arguably more fundamental (to me) before hash tables. It is only fair if all of these things are kept out of the language, lest the people whose favorite feature has not been included feel alienated by the changes.
-
-
but there doesn’t seem to be much push to improve the situation
Definitely not true. There are people working on this and there has been quite a bit of progress:
$ git clone https://github.com/BurntSushi/ripgrep $ cd ripgrep $ git checkout 0.4.0 $ time cargo +1.12.0 build --release real 1:04.05 user 1:51.42 sys 2.282 maxmem 360 MB faults 736 $ time cargo +1.43.1 build --release real 19.065 user 2:34.51 sys 3.101 maxmem 740 MB faults 0That’s 30% of what it once was a few years ago. Pretty big improvement from my perspective. The compilation time improvements come from all around too. Whether it’s improving the efficiency of parallelism or micro-optimizing rustc itself: here, here, here, here, here, here or here.
People care.
The Rust developers should stop trying to please everybody and come up with standard interfaces that also get shipped with the standard install.
That’s one of std’s primary objectives. It has tons of interfaces in it.
This criticism is just so weird, given that your alternative is C. I mean, if you want the C experience of “simplicity and self-reliance,” then std alone is probably pretty close to sufficient. And if you want the full POSIX experience, bring in libc and code like its C. (Or maybe use a safe interface that somebody else has thoughtfully designed.)
When installing a Rust package, you end up having to be connected to the internet
You do not, at least, no more than you are with a normal Linux distro package manager. This was a hard requirement. Debian for example requires the ability to use Cargo without connecting to the Internet.
and often end up downloading dozens of small crates from some shady GitHub repos.
Yup, the way the crates.io model works means the burden of doing due diligence is placed on each person developing a Rust project. But if you’re fine with the spartan nature of C’s standard library, then you should be just fine using a pretty small set of well established crates that aren’t shady. Happy to see counter examples though!
but I could imagine a similar scenario to leftpad in the future.
The leftpad disaster was specifically caused by someone removing their package from the repository. You can’t do that with crates.io. You can “yank” crates, but they remain available. Yanking a crate just prevents new dependents from being published.
Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.
… like C? o_0
-
This point is really close to the one before it: Cargo is an interesting system, but really ends up becoming a “monosolution” for Rust setups. Call me old-fashioned, but I like package managers (especially Gentoo’s) and Cargo just works around it.
C has just been in the luxurious position that its package managers have been the default system package managers. Most Linux package managers are effectively a C package managers. Of course, over time packages for other languages have been added, but they have mostly been second-class citizens.
It is logical that Cargo works around those package managers. Most of them are a mismatch for Rust/Go/node.js packages, because they are centered around distributing C libraries, headers, and binaries.
but I could imagine a similar scenario to leftpad in the future.
Rust et al. certainly have a much higher risk, since anyone can upload anything to crates.io. However, I think it is also an illusion that distribution maintainers are actually vetting code. In many cases maintainers will just bump versions and update hashes. Of course, there is some gatekeeping in that distributions usually only provide packages from better-known projects.
Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.
You mean a large standard library like… C?
-
C has just been in the luxurious position that its package managers have been the default system package managers.
This just isn’t true, if you look at how packages are built for Debian for example you will find that languages such as Python and Perl are just as well supported as C. No, the system package managers are for the most part language agnostic.
-
This just isn’t true, if you look at how packages are built for Debian for example you will find that languages such as Python and Perl are just as well supported as C.
Most distributions only have a small subset of popular packages and usually only a limited number of versions (if multiple at all).
The fact that most Python development happens in virtual environments with
pip-installed packages, even on personal machines, shows that most package managers and package sets are severely lacking for Python development.s/Python/most non-C languages/
No, the system package managers are for the most part language agnostic.
Well if you define language agnostic as can dump files in a global namespace, because that’s typically enough for C libraries, sure. However, that does not work for many other languages, for various reasons, such as: no guaranteed ABI stability (so, any change down the chain of dependencies needs to trigger builds of all dependents, but there is no automated way to detect this, because packages are built in isolation), no strong tradition of ABI stability (various downstream users need different versions of a package), etc.
-
No, most development happens in virtualenv because python packaging is so broken that if you install a package you cannot reliably uninstall it.
If we didn’t have a package manager for each language then the packages maintained by the OS would be more comprehensive, by necessity. Basically having a different packaging system for each programming language was a mistake in my view. I have some hope that Nix will remedy the situation somewhat.
edit: it’s also difficult to reply to your comments if you substantially edit them by adding entirely new sections after posting…
-
No, most development happens in virtualenv because python packaging is so broken that if you install a package you cannot reliably uninstall it.
I have no idea what you mean here. Can’t you use dpkg/APT or rpm/DNF to uninstall a Python package?
If we didn’t have a package manager for each language then the packages maintained by the OS would be more comprehensive, by necessity.
We are going in circles. Why do you think languages have package managers? Technical reasons: the distribution package managers are too limited to handle what languages need. Social/political reasons: having the distributions as gatekeepers slows down the evolution of language ecosystems.
I have some hope that Nix will remedy the situation somewhat.
Nix (and Guix) can handle this, because it is powerful enough to implement the necessary language-specific packaging logic. In fact, Nix’
buildRustCrateis more or less an implementation of Cargo in Nix + shell script. It does not use Cargo. Moreover, Nix can handle a lot of the concerns that I mentioned upthread: it can easily handle multiple different versions of a package and ABI-instability. E.g. if in Nix the derivation of say the Rust compiler is updated, all packages of which Rust is a transitive dependency are rebuilt.As I said, traditional package managers are built for a C world. Not a Rust, Python, Go, or whatever world.
-
-
-
-
-
The first problem is a technical one, unless Rust is doing things such that it can’t be compiled efficiently, but the latter two are cultural ones which point up differences in what language designers and implementers are expected to provide then versus now: In short, Rust tries to provide the total system, everything you need to build random Rust code you find online, whereas C doesn’t and never did. Rust is therefore in with JS, as you mention, but also Perl, Python, Ruby, and even Common Lisp now that Quicklisp and ASDF exist.
I was going to “blame” Perl and CPAN for this notion that language implementations should come with package management, but apparently CPAN was made in imitation of CTAN, the Comprehensive TeX Archive Network, so I guess this goes back even further. However, the blame isn’t with the language implementers at all: Packaging stuff is one of those things which has been re-invented so many times it’s bound to be re-invented a few more, simply because nobody can decide on a single way of doing it. Therefore, since language implementers can’t rely on OS package repos to have a rich selection up-to-date library versions, and rightly balk at the idea of making n different OS-specific packages for each version of each library, it’s only natural each language would reinvent that wheel. It makes even more sense when you consider people using old LTS OS releases, which won’t get newer library versions at this point, and consider longstanding practice from the days before OSes tended to have package management at all.
-
Therefore, since language implementers can’t rely on OS package repos to have a rich selection up-to-date library versions, and rightly balk at the idea of making n different OS-specific packages for each version of each library, it’s only natural each language would reinvent that wheel.
This is right on the mark.
Sorry if this is a bit of a tangent, but I think it is not just a failing of package sets – from the distributor’s perspective it is impossible to package every Rust crate and rust crate version manually – but especially of package managers themselves. There is nothing that prevents a powerful package management system to generate package sets from
Cargo.lockfiles. But most package managers were not built for generating package definitions programmatically and most package managers do not allow allow installing multiple package versions in parallel (e.g.ndarray 0.11.0andndarray 0.12.0).Nix shows that this is definitely feasible, e.g. the combo of
crate2nixandbuildRustCratecan create Nix derivations for every dependency in aCargo.lockfile. It does not usecargoat all, compiles every crate into a separate Nix store path. As a result, Rust crates are not really different from any other package provided throughnixpkgs.I am not that familiar with Guix, but I bet it could do the same.
-
-
Build times are important, and you’re right, Rust takes a while to compile. Given the choice between waiting for rustc to finish and spending a lot longer debugging a C program after the fact, I choose the former. Or, better yet, use D and get the best of both worlds.
-
rust can be damn fast to compile, most rust library authors just exercise fairly poor taste in my opinion and tend not to care how bad their build times get. sled.rs compiles in 6 seconds on my laptop, and most other embedded databases take a LOT longer (usually minutes), despite sled being Rust and them being C or C++.
rust is a complex tool, and as such, you need to exercise judgement (which, admittedly, is rare, but that’s no different from anything else). you can avoid unnecessary genericism, proc macros, and trivial dependencies to get compile times that are extremely zippy.
-
-
-
I though about using rust to implement a byte-code compiler / vm for a gc’d language project, but I assumed that this would require too much fighting to escape rust’s ownership restrictions. Do you have any insight into how well suited rust is for vm implementation? I haven’t used the language much but I’d love to pick it up if I though I could make it work for my needs.
(I see that there’s a python interpreter written in rust, but I’m having trouble locating its gc implementation)
-
I honestly don’t know, I have never written an interpreter. You probably can fall back on unsafe for some things anyway, and still benefit from the move semantics, sum types, syntax, and friendly error messages. I’m doing a bit of exploring symbolic computations with rust and there are also some design space exploration to be done there.
-
-
Also, just saw this GC for Rust on the front page: https://github.com/zesterer/broom/blob/master/README.md
Looks like it’s designed specificly for writing dynamic languages in Rust.
-
Oh cool! This looks super useful
-
-
-
I wrote a ST80 VM in rust just to play around; the result was beautifully simple and didn’t require any unsafe code, though it doesn’t currently have any optimization at all. The result was still reasonably snappy, but I suspect that a big part of that is that the code I was running on it was designed in, well, 1980.
-
-
Rust works well for VM implementation. For GC, you may want to look at https://github.com/zesterer/broom.
-
-
-
I never understood the advantages of ninja with respect to make. It seems to boil down to things like that the makefiles do not use tab characters with semantic value, that the -j option is given by default, or that the syntax is simpler and slightly better. But apart from that, what are the essential improvements that would justify a change from make to ninja? If ninja is slightly better than GNU make, I tend to prefer GNU make that I know and that it is ubiquitous and it avoids a new build dependency.
-
The article discusses how it’s really a low-level execution engine for build systems like CMake, Meson, and the Chrome build system (formerly gyp, now GN).
So it’s much simpler than Make, faster than Make, and overlapping with the “bottom half” of Make. This sentence is a good summary of the problems with Make:
Ninja’s closest relative is Make, which attempts to encompass all of this programmer-facing functionality (with globbing, variable expansions, substringing, functions, etc.) that resulted in a programming language that was too weak to express all the needed features (witness autotools) but still strong enough to let people write slow Makefiles. This is vaguely Greenspun’s tenth rule, which I strongly attempted to avoid in Ninja.
FWIW as he also mentions in the article, Ninja is for big build problems, not necessarily small ones. The Android platform build system used to be written in 250K lines of GNU Make, using the “GNU Make Standard Library” (a third-party library), which as far as I remember used a Lisp-like encoding of Peano numbers for arithmetic …
-
# ########################################################################### # ARITHMETIC LIBRARY # ########################################################################### # Integers a represented by lists with the equivalent number of x's. # For example the number 4 is x x x x. # ---------------------------------------------------------------------------- # Function: int_decode # Arguments: 1: A number of x's representation # Returns: Returns the integer for human consumption that is represented # by the string of x's # ---------------------------------------------------------------------------- int_decode = $(__gmsl_tr1)$(if $1,$(if $(call seq,$(word 1,$1),x),$(words $1),$1),0) # ---------------------------------------------------------------------------- # Function: int_encode # Arguments: 1: A number in human-readable integer form # Returns: Returns the integer encoded as a string of x's # ---------------------------------------------------------------------------- __int_encode = $(if $1,$(if $(call seq,$(words $(wordlist 1,$1,$2)),$1),$(wordlist 1,$1,$2),$(call __int_encode,$1,$(if $2,$2 $2,x)))) __strip_leading_zero = $(if $1,$(if $(call seq,$(patsubst 0%,%,$1),$1),$1,$(call __strip_leading_zero,$(patsubst 0%,%,$1))),0) int_encode = $(__gmsl_tr1)$(call __int_encode,$(call __strip_leading_zero,$1))-
-
-
Yup exactly, although the representation looks flat, it uses recursion to turn
4intox x x x! The__int_encodefunction is recursive.It’s what you would do in Lisp if you didn’t have integers. You would make integers out of cons cells, and traverse them recursively.
So it’s more like literally Greenspun’s tenth rule, rather than “vaguely” !!!
-
-
Yes, so I guess its main advantage is that it is really scalable. This is not a problem that I have ever experience, my largest project having two hundred files that compiled in a few seconds, and the time spent by make itself was negligible. On the other hand, for such a small project you may get to enjoy the ad-hoc GNU make features, like the implicit compilation .c -> .o, the usage of CFLAGS and LDFLAGS variables, and so on. You can often write a makefile in three or four lines that compiles your project; I guess with ninja you should be much more verbose and explicit.
-
He mentions that the readme explicitly discourages people with small projects from using it.
I suspect it’s more that ninja could help you avoid having to add the whole disaster that is autotools to a make-based build rather than replacing make itself.
-
I suspect it’s more that ninja could help you avoid having to add the whole disaster that is autotools to a make-based build rather than replacing make itself.
Sure; autotools is a complete disaster and a really sad thing (and the same thing can be said about cmake). For small projects with few and non-configurable dependencies, it is actually feasible to write a makefile that will compile seamlessly the same code on linux and macos. And, if you don’t care that windows users can compile it themselves, you can even cross-compile a binary for windows as a target for a compiler in linux.
-
-
You don’t (or better, shouldn’t!) write Ninja build descriptions by hand. The whole idea is something like CMake generates what Ninja actually parses. I’ve written maybe 3 ninja backends by now.
-
-
-
we use ninja in pytype, where we need to create a dependency tree of a project, and then process the files leaves-upwards with each node depending on the output of processing its children as inputs. this was originally done within pytype by traversing the tree a node at a time; when we wanted to parallelise it we decided to instead generate a ninja file and have it invoke a process on each file, figuring out what could be done in parallel.
we could doubtless have done the same thing in make with a bit of time and trouble, but ninja’s design decisions of separating the action graph out cleanly and of having the build files be easy to machine generate made the process painless.
-
-
I use it in all my recent projects because it can parse cl.exe /showincludes
But generally like andyc already said, it’s just a really good implementation of make
-
-
Fearless Concurrency actually delivers.
Until you write a deadlock, which I managed to do within 30min of writing Rust code.
-
Deadlocks are easy compared to data races. You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation. That is a walk in the park compared to some bytes somewhere sometimes briefly having wrong value, which might not even be reproducible with a debugger attached.
I’ve had my share of deadlocks in Rust, and I’ve been able to solve them all. OTOH in my last C project I’ve had a data race, and the best I could do was to give up and remove all non-trivial OpenMP uses. Was it my bug? Was it compiler bug? I couldn’t know. I couldn’t even reproduce the data race myself.
-
-
Deadlocks are easy compared to data races. You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation. That is a walk in the park compared to some bytes somewhere sometimes briefly having wrong value, which might not even be reproducible with a debugger attached.
I’ve had my share of deadlocks in Rust, and I’ve been able to solve them all. OTOH in my last C project I’ve had a data race, and the best I could do was to give up and remove all non-trivial OpenMP uses. Was it my bug? Was it compiler bug? I couldn’t know. I couldn’t even reproduce the data race myself.
distributed system? Everything is more complicated with micro-services and other distributed systems. Rust can’t save you there.
-
-
-
The nice thing about deadlocks is that they’re eminently fixable but still sufficiently annoying that after the third time you start asking “why am I getting deadlocks all the time?” and rekajigger your design so that it can’t happen any more. In my experience it’s another case of confused ownership, just expressed differently.
-
Did you try clang thread sanitizer? Last time I used it was over 5 years ago and it very effectively told me exactly which threads and exactly what structures / bytes had a race.
I use address sanitizer every time I code C and C++ now and it has changed how I code… I would call it “fearless memory management”
-
-
I would call it “fearless memory management”
I wouldn’t. I’ve used valgrind since the early 2000s and asan since I knew it existed (probably around 2010/2011 but I’m not sure). I’ve lost count of how many memory management bugs went undetected until a segfault led me to investigate.
asan is huge for C and C++. But it’s only as good as your test suite and its coverage, and even then…
-
-
Deadlocks are easy compared to data races
Even assuming I agree, I don’t see how that changes my point that if I can write deadlocks it’s not fearless.
You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation
That’s not been my experience. And that’s if the bug is easily reproducible, which tends to not be the case with concurrency.
OTOH in my last C project I’ve had a data race
Why compare to C? C is terrible at writing concurrent programs. If the bar for Rust is “better than C”…
-
Compile-time verification that the whole program (and dependencies!) are free from data races is without a doubt a major improvement over C, C++ and Go. It is a way stronger guarantee than most other languages offer with “just be careful” and runtime sanitizers (and Rust also supports thread sanitizer). On top of that Rust has a rich ecosystem of libraries with robust concurrency primitives, so you can make programs multi-threaded without even touching any lock yourself.
But your entire argument hangs only on a literal interpretation of an advertising slogan. Of course a slogan doesn’t contain every possible caveat. And arguing about it is arguing about who fears what, which is unconstructive (there’s always someone afraid of everything, but this doesn’t change what Rust does).
The reasonable interpretation is that we all know data races are a major source of hard-to-fix bugs that are commonly feared, and Rust improves in this area so much that addition of concurrency to programs is no longer a major risk (but Rust does not guarantee bug-free programs, and never claimed that).
-
major improvement over C, C++ and Go
I don’t think Go is even close to being in the same category as C or C++ in this regard.
rich ecosystem of libraries
I don’t think any existed at the time I wrote the deadlock (I looked). I ended up using mio directly. Things certainly seem to have change for the better since then.
we all know data races are a major source of hard-to-fix bugs that are commonly feared,
In certain languages, sure. I’ve written more concurrency bugs than I can count in C++, and I’m glad that if I write Rust those wouldn’t be possible. But there are other languages which make such bugs rare or non-existent. Rust is literally the only language I’ve every written a deadlock in and I did it on my first day.
-
Again, you’re hung up on that one deadlock, as if it invalidated everything else. “Fearless” doesn’t mean “can be totally careless” or “guaranteed bug-free”.
I don’t think you will find any language that can use low-level
mio/libuvthat can do better here, so let’s not pit real shipping Rust against an imaginary ideal.And you don’t have to use
mio. You can use Rust with data-bound parallelism where you can’t make that bug. You can use Rust with an actor framework. Probably you could even hook it up to the Pony runtime.BTW, Pony is deadlock-free on a technicality. It doesn’t solve the problem, but merely reshapes it to “two actors will send a message only after receiving a message from each other”.
-
-
-
-
-
I always assume that Rust’s “fearless” is more about “your thread will not accidentally use data that it shared with someone else and it was freed in the meantime” than preventing you from writing deadlocks – just like its regular safety features will prevent you from breaking the memory-related stuff, not from actual logic bugs.
-
I don’t think that’s a fair comparison. I can trust Rust to not let me violate memory safety. I can’t trust it to prevent me from writing deadlocks. In one case, it’s fearless memory management, in the other it’s fearless concurrency except for deadlocks.
Plenty of languages make it easy to not accidentally used data that is shared with someone else.
-
-
How many languages Erlang
I think you partially answered your own question. From my personal experience, also Haskell and D. From reading the tutorial, Pony. The fact that Erlang doesn’t share memory is a good thing.
-
- Does D protect you from data races if you use mutex?
- Does pony have locks? It seems to me it’s only about channels and sharing immutable references (which you can also do in rust, no need for locks if you share
& Foo, just have to ensure it lives long enough) - Erlang doesn’t share memory at all, which is good, except when you try to write something very efficient I guess, like a shared cache. Besides you can get deadlocks with actors too, if two actors are waiting for messages from one another.
- Haskell provides shared memory abstractions, but apart from the STM I don’t think it does anything special for the deadlock issue.
In this list, I still don’t see anything that offers what rust does (race-free shared memory concurrency, statically guaranteed, where multiple threads can still modify the shared structures concurrently). Channels don’t protect from deadlocks at a higher level, and even then, rust has them too if they fit your needs.
-
Does D protect you from data races if you use mutex?
That’s not the usual way of writing concurrent code, which is to send messages between threads. You can only send immutable or shared data. The former guarantees a lack of data races, the latter can’t really be used without locks. That’s an oversimplified explanation though.
Does pony have locks?
No.
but apart from the STM
I think STM is their secret weapon when it comes to concurrency.
-
That’s not the usual way of writing concurrent code, which is to send messages between threads.
But in rust you can do that too, and the move semantics even allows you to send mutable things since you give up their ownership. And you can still get deadlocks. I fail to see what D does that rust doesn’t.
-
-
-
-
-
-
Detecting deadlocks statically is equivalent to solving the halting (heh) problem, so it’s never gonna be 100%.
That being said, it is possible to detect deadlocks once they’ve occurred or are about to…how does Rust handle a deadlock situation? Does it do runtime deadlock detection and panic, or just lock up?
(Forgive me if I sound pedantic. Not my goal. Low on coffee and it’s a pain to buy more right now, what with the apocalypse and all…)
-
Rust as a language doesn’t even know that locks exist. They’re purely a library feature. Data-race safety of the locked data is expressed via
Send/Synctraits, but there isn’t more to it. The stdlib just wraps pthread mutexes, but you’re free to use whatever implementation you want.parking_lotfrom WebKit is a popular alternative. -
You can constraint the programmer such that deadlocks are impossible by construction. X10 started like this. Later they introduced old-fashioned locks via library so the guarantee can be broken (like unsafe in Rust).
X10 has
async { }to spawn of an activity (think thread). There is “finish { }” which waits for all (recursively) spawned activities inside the block at the end of the block. You also get some barrier synchronization with a smart API design. If that is all you use, then you cannot have deadlocks. -
Detecting deadlocks statically is equivalent to solving the halting (heh) problem, so it’s never gonna be 100%.
The Pony programming language guarantees absence of deadlocks.
That being said, it is possible to detect deadlocks once they’ve occurred or are about to
Not deterministically.
-
The Pony programming language guarantees absence of deadlocks.
Right, I was referring to classic free-for-all threads-with-locks models; there are ways to prevent deadlocks by construction or restriction.
Not deterministically.
I’m rapidly approaching the edge of my knowledge, so absolutely correct me if I’m wrong, but at least for traditional threads-with-locks, you can generate resource-acquisition graphs to determine if a deadlock has occurred, right?
You can’t determine a priori if a deadlock will occur by static analysis of an arbitrary threads-and-locks, at least not in the general case, though, you’re right; that’s what I was alluding to above.
-
-
-
Most schemes to tame concurrency still seem to be able to deadlock and livelock. My favorite, though, is still SCOOP. Heck, I”ll throw in a submission on it since SCOOP appears portable across languages.
-
Eiffel is the best of all the OOP languages. It’s got so much going for it, it’s sad that it isn’t more popular.
-
Seemed like a great language for maintainable, business applications. Took a while for me to understand the non-technical reasons languages like it don’t get much adoption.
-
-
Carl Sassenrath, is that you?!?
stares wistfully at REBOL
-
-
-
I tried to learn it, because I’m a huge fan of contracts, only to find out you couldn’t get it up and running without purchase, so I binned the plan.
-
There were a few open source compilers over the years but, as far as I know, none are still under much active development.
-
D has contracts.
-
-
-
You know me, I’m a technical Luddite. Anything newer than 25 years is inexcusable newfangled decadence.
But, to answer your question, no, I don’t think it’s kept up. Last time I looked at the “official” Eiffel implementation, they were mainly focusing on interop and compiler speedups, though that was several years ago.
-
-
-
Pony doesn’t allow deadlocks to happen.
-
Well, it uses the actor model instead of multithreading with shared state that I recall. Great that it stops deadlocks, though.
-
-
-
No. There are a lot of things that are common to those two and only those two languages*. Sometimes I say C, sometimes I say C++, and sometimes I say C/C++. It depends.