Threads for Lt_Riza_Hawkeye
-
Volatile reads and writes should have been function calls that take pointers. It would have been a straightforward interface that doesn’t raise all the questions that implicit magic of volatile types does.
-
In retrospect, I agree, but with a minimally optimising compiler, on a single-core system with no caches (i.e. the original C deployment scenario),
volatileis easy to understand: the compiler may not elide loads or stores to those variables. It wasn’t until C11 that C got a memory model that even allowed you to write async-signal safe using standard C (volatilereads and writes can be reordered with respect to other operations, includingvolatilereads and writes of other objects), let alone threading.The rule of thumb for
volatileis: if you’re using it for anything other than memory-mapped I/O, you’re doing it wrong.-
Rules of thumb are hard to make all-encompassing. You also need volatile when you reading and writing variables between the main program and a signal handler. Otherwise your compiler might optimize
while (!g);into a single read, resulting in an infinite loop if your program hasn’t gotten the signal yet.-
You also need volatile when you reading and writing variables between the main program and a signal handler
This is an example of incorrect use of
volatile(though the closest portable approximation of something that would mostly work that was possible prior to C11). The compiler may not elide the loads or stores to thevolatilevariable, but it is completely allowed to reorder them with respect to other loads and stores and so your program may have an observed execution order that is very surprising to someone reading the source. For this purpose, you want an_Atomic-qualified variable, with the correct memory order to inhibit whichever kind of move you want. The best option if the variable isn’t shared across threads is a relaxed-consistency atomic op combined with an atomic signal fence.
-
-
-
-
And I’m in the habit of using the names VCC for the power and GND for, well, the ground. But the datasheet uses VDD for power and VSS for ground. And I’d mixed up these two connections. Gah!
This bit me too on my first PCB just earlier this month! Except I made the mistake that “VSS” = “Source” must be the “source” of power, instead of the “source” of electrons :(
-
If memset is a hot instruction (that is, it’s being called frequently), why didn’t that get moved to hardware? Something like an instruction to memory to zero out big aligned, power of two chunk if memory?
-
If memset is a hot instruction (that is, it’s being called frequently), why didn’t that get moved to hardware?
Block copy and clear instructions have a long history in hardware; here’s a comp.arch thread that discusses some of the difficulties involved with implementing them.
Also, the ARM instruction set was recently modified to add memcpy (CPYPT, CPYMT, CPYET) and memset (SETGP, SETGM, SETGM) instructions.
-
-
why didn’t that get moved to hardware?
It did. See x86’s ‘rep’ family of instructions, ‘clzero’, etc. Rep has a high startup cost; it has gotten better but it is still not free. (Clzero is very specialized and lacks granularity.) The technique implemented by the linked post aims to improve performance of small-to-medium-sized memsets, where you can easily beat the hardware. The calculus is complicated by second-order effects on i$/btb (e.g. see here, sec. 4.4, and note that ‘cmpsb’ is never fast). My own implementation is slower at very small sizes, but ~half the size/branches of the linked version.
Such small sizes are empirically very rare; but application-side specialization can nevertheless clean up the chaff. Dispense with ‘memset’ entirely and statically branch to memset_small memset_medium clear_page etc. Overgenerality is the bane of performance. Compare the performance of malloc vs your own purpose-built allocator, or dumb virtual dispatch vs closed-world matching or inline-cached JIT (which amounts to the same thing).
-
-
repis an interesting command, but I think I was not clear in my question. I was wondering why the option to clear out chunks of memory didn’t move to memory itself? Repeating something from the CPU still takes a lot of roundtrips on the memory lane, and latencies add up. If something is performance critical, why not do it on the very edge, which in this case is the memory chip/board itself.-
Heh, everybody wants their problems to run directly on memory!
-
‘Shows up on a profiler’ ≠ ‘performance critical’. The complexity is just not worth it, esp. as it is far-reaching (what are the implications for your cache coherency protocol?)
-
Again, the things that were showing up on the profiler were not bandwidth-limited; they fit comfortably in L1. Touching main memory at all would be extremely wasteful
-
There are some bandwidth-limited problems. The most obvious example is kernels needing to zero memory before handing it out to applications. But the performance advantage is not there; memory is written to many more times than it is mapped. Dragonflybsd reverted its idle-time zeroing
-
-
DDRwhatever sticks are simple memory banks with no logic in them, you can’t move anything into them.
The memory controller is in your SoC (was in the northbridge in the old times). Moving the command operation just into the controller I guess doesn’t win much.
Now, this might make some sense if you move memory controller to be remote again, talking over a higher-latency serial link (hello IBM) I guess.
-
You often don’t want this to move to the memory directly because you’re setting the contents of memory that you’re about to use or have just used. In either of those cases it either wants to be, or already is, in the cache. At a minimum, you’d need CPU instructions that invalidated the cache lines that were present and then told memory to set the pattern in a range.
-
-
-
I think
rep stosqwill work on x86. But that doesn’t mean it’s fast.-
It is guaranteed to be quite fast if your cpuid has the ERMS flag (Enhanced REP MOVSB). That would be >=IvyBridge on the Intel side, and only >=Zen3 on AMD.
-
-
-
Lt_Riza_Hawkeye
1 year ago
|
link
| on:
The pedantic checklist for changing your data model in a web application
I do think this could be a bit overly pedantic. I work at , and version 3: read from the new representation (while still writing to the old) is something we would typically skip - we would go from 2 to 4 using a knob/killswitch/rollout config/whatever you want to call it. They deploy (just about) instantly to all machines, so there is not really any downtime. But without the infrastructure to deploy configs that fast, I see how it could be necessary.
-
when would you do your data migration for older records? It does depend on the velocity of your changes (this flow is really important when working on your primary data model, for example, where you have near-constant reads and writes, and migrations take time to run), but I feel like just a knob seems a bit dangerous.Of course if you’re only looking at a lower velocity table, or stale reads aren’t a big deal, then you can just move stuff, then like… run the backfill twice. Bit data dependent.
There are some alternatives though! For example you can track the “version” of your data on a row-basis. For example your JSON structure embeds a
versionparameter. That way your read code can on-the-fly transform to the latest version, and backfilling switches things over on the fly as well.I’ve found that strategy to be fiddly if you aren’t just using a loosly-validated JSON though.
-
Basically I’m saying version 2 would read old, write old & new, and also have another codepath (behind a killswitch to enable it) that would read new, write new. No need for an intermediate version that writes to both old and new. You would backfill the “new” data for existing records while v2 is running in production, then flip the switch as soon as you are done with the backfill. You can then reap the knob (another version, which would correspond to v4 in the article) and then drop the old column(s)
The version tag is interesting. We use something similar to gRPC internally, and we use the same approach for our databases: basically guessing the version based on which fields are present/missing, and relying on every engineer to clean up after any migration they start. The version tag might be a better alternative.
-
-
-
Our application consumes 2–4 gigabytes of memory but we run it with 25G heap size
Wondering if this means they are provisioning servers with more than 25Gb of RAM to run a 2–4Gb. If so, that seems… not ideal, and expensive.
-
It’s possible they just tune the vm overcommit way up
-
-
Definitely an interesting article, I learned a few things. But I think it is addressing a very specific type of “service”. For example, the author mentions wanting to be able to perform authentication in nginx, and then proxy the connection to an existing instance of the server running locally, which uses HTTP over unix sockets. I have never run such a setup, though I suspect it isn’t uncommon. But when I think of “running a service” I typically think of it running on another remote machine, not locally, and communicating with it over a structured protocol like gRPC, Thrift, dbus, or JSON-RPC, and being queried from my application code (for example, $service->getAdvertisementForUser). Many of these requested behaviors don’t seem very applicable for this type of service.
-
What you describe is exactly what is described in the article. The only difference is the addition of nginx (or equivalent) as an intermediary.
-
Hi, author here: I definitely focused on HTTP, it is so widely used that it makes sense as a default. I agree that these other protocols can be great choices as well, especially for internal services where you can stick to the same stack across an organization. However I think most of the same concerns apply, even if the mechanism is different. For example you probably still want to offload auth from the service itself and into a proxy (or common library). Rate limiting may not be required for internal services but if you need it I would again recommend providing the same sort of hooks so that it can be done by a proxy or common library. Even receiving your socket from your caller is still relevant if you utilize static ports as it ensures that services can’t bind each other’s port (by error or malice) and you don’t need to start them with elevated permissions. For setups like docker it isn’t as relevant (although you could view listening on Docker’s isolated network interface as basically equivalent, because at the end of the day it is the Docker daemon which binds your listening socket to the outside world).
-
-
Lt_Riza_Hawkeye
1 year ago
|
link
| on:
you're naming a file, but the name doesn't matter. what do you usually call it?
“temp”
-
In the Arduino world, everything is done in C++, a language which is almost never used on 8-bit microcontrollers outside of this setting because it adds significant complexity to the toolchain and overhead to the compiled code.
I don’t buy this. C++ is C with extra features available on the principle that you only pay for what you use. (The exception [sic] being exceptions, which you pay for unless you disable them, which a lot of projects do.)
The main feature is classes, and those are pretty damn useful; they’re about the only C++ feature Arduino exposes. There is zero overhead to using classes unless you start also using virtual methods.
The C++ library classes will most definitely bloat your code — templates are known for that — but again, you don’t have to use any of them.
(Aside: can someone explain why anyone’s still using 8-bit MCUs? There are so many dirt cheap and low-power 32-bit SoCs now, what advantage do the old 8-but ones still have?)
-
(Aside: can someone explain why anyone’s still using 8-bit MCUs? There are so many dirt cheap and low-power 32-bit SoCs now, what advantage do the old 8-but ones still have?)
They’re significantly cheaper and easier to design with (and thus less pretentious in terms for layout, power supply parameters, fabrication and so on). All of these are extremely significant factors for consumer products, where margins are extremely small and fabrication batches are large.
Edit: as for C++, I’m with the post’s author here – I’ve seen it used on 8-bit MCUs maybe two or three times in the last 15 years, and I could never understand why it was used. If you’re going to use C++ without any of the ++ features except for classes, and even then you still have to be careful not to do whatever you shouldn’t do with classes in C++ this year, you might as well use C.
-
- RAII is a huge help in ensuring cleanup of resources, like freeing memory.
- Utilities like unique_ptr help prevent memory errors.
- References (&) aren’t a cure-all for null-pointer bugs, but they do help.
- The organizational and naming benefits of classes, parameter overloading and default parameters are significant IMO.
stream->close()vs having to rememberIOWriteStreamClose(stream, true, kDefaultIOWriteStreamCloseMode). - As @david_chisnall says, templates can be used (carefully!) to produce super optimized type-safe abstractions, and to move some work to compile-time.
- Something I only recently learned is that
for (x : collection)even works with C arrays, saving you from having to figure out the size of the array in more-or-less fragile ways. - Forward references to functions work inside class declarations.
I could probably keep coming up with benefits for another hour if I tried. Any time I’m forced to write in C it’s like being given those blunt scissors they use in kindergarten.
-
The memory safety/RAII arguments are excellent generic arguments but there are extremely few scenarios in which embedded firmware running on an 8-bit MCU would be allocating memory in the first place, let alone freeing it! At this level RAII is usually done by allocating everything statically and releasing resources by catching fire, and not because of performance reasons (edit: to be clear, I’ve worked on several projects where no code that malloc-ed memory would pass the linter, let alone get to a code review – where it definitely wouldn’t have passed). Consequently, you also rarely have to figure out the size of an array in “more-or-less fragile ways”, and it’s pretty hard to pass null pointers, too.
The organisational and naming benefits of classes & co. are definitely a good non-generic argument and I’ve definitely seen a lot of embedded code that could benefit from that. However, they also hinge primarily on programmer discipline. Someone who ends up with
IOWriteStreamClose(stream, true, kDefaultIOWriteStreamCloseMode)rather thanstream_close(stream)is unlikely to end up withstream->close(), either. Also, code that generic is pretty uncommon per se. The kind of code that runs in 8-16 KB of ROM and 1-2 KB of RAM is rarely so general-purpose as to need an abstraction like anIOWriteStream.-
I agree that you don’t often allocate memory in a low-end MCU, but RAII is about resources, not just memory. For example, I wrote some C++ code for controlling an LED strip from a Cortex M0 and used RAII to send the start and stop messages, so by construction there was no way for me to send a start message and not send an end message in the same scope.
-
That’s one of the neater things that C++ allows for and I liked it a lot back in my C++ fanboy days (and it’s one of the reasons why I didn’t get why C++ wasn’t more popular for these things 15+ years ago, too). I realise this is more in “personal preferences” land so I hope this doesn’t come across as obtuse (I’ve redrafted this comment 3 times to make sure it doesn’t but you never know…)
In my experience, and speaking many years after C++-11 happened and I’m no longer as enthusiastic about it, using language features to manage hardware contexts is awesome right up until it’s not. For example, enforcing things like timing constraints in your destructors, so that they do the right thing when they’re automatically called at the end of the current scope no matter what happens inside the scope, is pretty hairy (e.g. some ADC needs to get the “sleep” command at least 50 uS after the last command, unless that command was a one-shot conversion because it ignores commands while it converts, in which case you have to wait for a successful conversion, or a conversion timeout (in which case you have to clear the conversion flag manually) before sending a new command). This is just one example but there are many other pitfalls (communication over bus multiplexers, finalisation that has to be coordinated across several hardware peripherals etc.)
As soon as you meet hardware that wasn’t designed so that it’s easy to code against in this particular fashion, there’s often a bigger chance that you’ll screw up code that’s supposed to implicitly do the right thing in case you forget to “release” resources correctly than that you’ll forget to release the resources in the first place. Your destructors end up being 10% releasing resources and 90% examining internal state to figure out how to release them – even though you already “know” everything about that in the scope at the end of which the destructor is implicitly called. It’s bug-prone code that’s difficult to review and test, which is supposed to protect you against things that are quite easily caught both at review and during testing.
Also, even when it’s well-intentioned, “implicit behaviour” (as in code that does more things than the statements in the scope you’re examining tell you it does) of any kind is really unpleasant to deal with. It’s hard to review and compare against data sheets/application notes/reference manuals, logic analyser outputs and so on.
FWIW, I don’t think this is a language failure as in “C++ sucks”. I’ve long come to my senses and I think it does but I don’t know of any language that easily gets these things right. General-purpose programming languages are built to coordinate instruction execution on a CPU, I don’t know of any language that allows you to say “call the code in this destructor 50us after the scope is destroyed”.
-
-
-
-
-
-
You can get Cortex M0 MCUs for about a dollar, so the price difference isn’t huge. Depending on how many units you’re going to produce, it might be insignificant.
It’s probably a question of what you’re used to, but at least for me working with a 32 bit device is a lot easier and quicker. Those development hours saved pay for the fancier MCUs, at least until the number of produced units gets large. Fortunately most of our products are in the thousands of units…
-
-
The license fee for a Cortex M0 is 1¢ per device. The area is about the size of a pad on a cheap process, so the cost both of licensing and fabrication is pretty much as close to the minimum cost of producing any IC.
-
-
A quick look at the Arm web site tells me I’m out of data. This was from Arm’s press release at the launch of the Cortex M0.
-
-
-
-
-
-
you only pay for what you use
Unfortunately many arduino libraries do use these features - often at significant cost.
-
I’ve not used Arduino, but I’ve played with C++ for embedded development on a Cortex M0 board with 16 KiB of RAM and had no problem producing binaries that used less than half of this. If you’re writing C++ for an embedded system, the biggest benefits are being able to use templates that provide type-safe abstractions but are all inlined at compile time and end up giving tiny amounts of code. Even outside of the embedded space, we use C++ templates extensively in snmalloc, yet in spite of being highly generic code and using multiple classes to provide the malloc implementation, the fast path compiles down to around 15 x86 instructions.
-
-
One thing I’ve read over and over is “don’t store your files in a database”. I assume there are caveats, and times when this does make sense, but could anyone care to make the case for why this is a good or a bad idea in this particular scenario?
-
In general: Storing files in the database generates a ton of IO load on the database server, which is usually already IO-bound. If your database is busy doing other stuff (unlike, say, IMGZ which doesn’t do anything else) that’s going to degrade performance.
On the flip-side, it’s terribly operationally convenient when you can backup a single database and actually have all your data, and having consistency between the data-store & the filesystem store is nice so you can’t refer to files that have been deleted / failed to upload / whatever.
-
Generally, considerations for files are different than for other data. E.g. you almost never need to filter files by actual contents, or sum them, or do any of the other things databases are good at, you just want to be able to store them, retrieve them and delete them. If you saved everything in a database, it would be more expensive, just because of the type of guarantees that you need for data, which you don’t need for files.
That means you’d unnecessarily be paying all the costs that are associated with what we normally want to do for data, but not need any of the benefits.
-
Sqlite did an analysis on this and found that for files under a certain size, it is actually faster to read them from a database (based on the database page size): http://0x0.st/iFUc.png
-
It’s a bad idea in this scenario. But you’re not likely to care too much at very low scale.
One solid reason to care is that in PostgreSQL the maximum “row” size is 16KiB. If you go over that then “the next” 16KiB goes into what’s called a “TOAST” table, which has access characteristics similar to a singly linked list. (So, 1 request for a 64KiB row will cause 4 queries to the storage engine and so on.)
Other characteristics of the TOAST table is that it compresses on input, which can quickly saturate your CPU.
-
-
You basically have three options with Postgres: you can put the file contents in the row, you can use the large object facility, or you can write the file to disk somewhere and store the path.
Putting the file contents in the row is simple, and is the only option that gives you for-free the notion that deleting a row will get rid of the content of the file. It has the disadvantages that others have discussed, although I don’t think TOAST is so bad for performance.
The large object facility is basically file descriptors in the database. You have to take extra care to remove them when you’re done with them, and most ORMs have poor support for it. I have never seen the large object facility used in the wild, and it’s not a tool I would reach for personally.
The third option is probably the best. The filesystem will always be better at storing files than your layer on top of the filesystem. But you have integrity concerns here (deleting a row does not cause the corresponding file to disappear), and you have to back it up separately from the database.
-
-
The amount of breaking changes in GNOME have traditionally been to a meme-worthy level, so I’m not sure what to make of this.
A diminishing number of veterans is doing an increasing share of the work. Although recruitment is stable, newcomers don’t seem to be hitting their stride in terms of commits.
So the same people that used to experiment and change everything all the time now don’t do this anymore?
-
I think there’s a number of different aspects to this – since the introduction of GNOME 3.0, the project as a whole has become more consolidated around the idea of a consistent desktop environment, rather than, say, a desktop shell plus loose collection of apps, as GNOME 2.x was. This idea is driven all the way from how the Design Team handles everything from the HIG, the design of core parts such as GNOME Shell itself, down to the minutae of how “core” GNOME applications are maintained (there was some recent drama around Gedit).
That is to say, the scope of the project has increased dramatically, while autonomy has, in some ways, diminished. This has allowed for an increase in the rate of experimentation and iteration, especially around the UX of the Shell itself (which has changed quite a bit over the years, compared to GNOME 2.x). This has also led to issues with compatibility, mostly around GNOME Shell extensions (that I’m aware of, anyways), that has led to perhaps unnecessary frustration on the part of external contributors.
As a bystander and long-time user of GNOME, I think the situation here has led to a marked increase in quality of both core applications and the ecosystem as a whole – usable GUI applications that actually meshed well with the rest of the system, whatever that means, were pretty rare back in the day. Unfortunately, it also means the barrier to entry is higher for developers, especially if one is looking to make some sort of outstanding contribution.
-
-
-
-
-
I don’t know what soc was referring to, but what stuck with me was when Gnome went to the bug tracker of Transmission (a popular bittorrent client) and opened a bug asking them to remove support for notification area icons, because they would not be displayed in gnome 3, so there was no need to keep them around:
Transmission has an option in the Desktop tab of the preferences to “Show Transmission icon in the notification area”. This should probably be removed.
In response, it was brought up that removing this would only benefit gnome 3 users and would be removing useful functionality for users of gnome shell, unity, and XFCE - on top of the fact that GTK had made many breaking changes to this API in the past as well, requiring many compile-time flags and different distributions. The response was:
I guess you have to decide if you are a GNOME app, an Ubuntu app, or an XFCE app unfortunately. I’m sorry that this is the case but it wasn’t GNOME’s fault that Ubuntu has started this fork. And I have no idea what XFCE is or does sorry. It is my hope that you are a GNOME app
-
Thanks for bringing this issue up. I had never heard of this, but it seems like the Transmission/Gnome3 thing was a kerfuffle indeed.
I guess you have to decide if you are a GNOME app, an Ubuntu app, or an XFCE app unfortunately. I’m sorry that this is the case but it wasn’t GNOME’s fault that Ubuntu has started this fork. And I have no idea what XFCE is or does sorry. It is my hope that you are a GNOME app
Indeed, out of context that feels tactless and unsympathetic. But let’s look at the previous comment in the chain [1]:
So now we can have three builds of Transmission that decide at compile time whether to use AppIndicator?, GtkStatusIcon?, or nothing at all, over such a stupid feature? Removing it altogether, as you suggest, will hurt XFCE users. I wish GNOME, Canonical, and everyone else involved would settle on one consistent API for this and stop fucking the app developers over. In order for this ticket to move forward, I’d like you to tell me what change should be made to Transmission that will make it work properly, out of the box, on GNOME Shell, Unity, and XFCE.
In that light, we have to ask, what should Gnome do? Should Gnome consult other DEs before shipping its features? Then the Gnome project loses its autonomy. Should Transmission conform to the whims of the 3 DEs? Then Transmission loses its autonomy. The problem is, each of these are independent projects with different users, goals, and ideas. Gnome does not want to be beholden to Ubuntu/Unity, XFCE does not want to be beholden to Gnome, and Transmission does not want to have to dance around 3 DEs, in which case, who budges? Why should it be Gnome in this case? If anything, this thread just shows why “worse is better” is the eventual shakeout of loose coupling in the open source world; it’s because the lowest common denominator is what wins when you have multiple actors with only occasionally concordant desires and goals.
And remember, if the authors of Transmission felt that this change in Gnome was not supporting, they could have simply closed the bug and not worked on it. In the Gedit case linked above, Gedit is considered part of “Gnome core” and so Gnome feels greater pressure to make it fall in line, but Transmission could just ignore this change in API and wait for Gnome3 to land and then make the changes gradually or not at all without much pushback.
[1]: https://trac.transmissionbt.com/ticket/3685 for the entire contents of the thread
-
That was a long time ago – it still gets brought up because, while it was a long time ago, the Gnome project still regularly treats other developers, and users, with condescension and snark.
XFCE may not be the most famous project but I find it really hard to believe the person who posted that didn’t know what it was or what it did. Even assuming it were so, when a developer is voicing concerns about their users’ environment, you can’t just wave your hand and make them go away. Just because you don’t know what something is or does doesn’t make it go away for everyone else.
Should Gnome consult other DEs before shipping its features? Then the Gnome project loses its autonomy. Should Transmission conform to the whims of the 3 DEs? Then Transmission loses its autonomy
That was way too long ago for me to remember the technical details of that discussion, status icons and systrays have always been a bit of tarpit on Linux DEs, but as far as I recall, a third option – consult with application developers – would’ve likely helped…
If, for whatever reason, you come up with your own API or your own way of doing something, that’s great. But if you want other people to start using it, opening a bug asking for the removal of an application feature just because that feature doesn’t really work with your new thing – even though it works everywhere else, and it worked fine with the API you’re deprecating! – is probably not the most elegant way to go about it.
-
-
-
-
-
-
-
-
-
This never occurs if you statically link against libc, right? I’m pretty sure the problem here is dynamic linking having no runtime checking whatsoever (which is certainly not a bad thing in all cases)
-
A really great article. The concepts apply just the same for x86_64 as well. You need those memory barriers or std::atomics for the correct behavior, no matter what.
Another example that might be easier for people to understand is this (taken from Herb Sutter’s std::atomic<> weapons talk): if one thread does this:
bool g; std::atomic<bool> a; g = 1; a = 1;And the other thread does this:
if (a) assert(g)the assert must NEVER fail - that is, ALL writes performed by a thread must be visible to ALL other threads before the other threads see the “after” value of an atomic write. This is the same thing the author tries to explain in the article, but is a little simpler of an example.
Now, perhaps you don’t really need all of those writes to be visible, just the ones that relate to the atomic variable you’re writing to. Couldn’t that be a little cheaper? That’s when you start diving into the
std::memory_order_consumerabbit hole :)
-
Writing web-facing applications in C is one of those
cgi-bindays that went away, and probably for good reason. I’d understand wanting better performance, but perhaps there are better tools.-
-
I think BCHS recommends compiling individual executables for each page, and executing them using regular old CGI. Kore is a web framework that generates a shared object file with all your code, that you can then run with “kore run” (or alternatively, generates a single executable)
-
-
-
Editors are not a group you can reasonably compare. A JPEG editor and a WAV editor have nothing in common and exactly the same applies to “text” editors. You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
If you squint a little you probably would want to edit all programs in the same way but that unfortunately doesn’t work because you just cannot refactor Python like you refactor Java. So even trying to do such a comparison for IDEs is tricky.
All of these discussions about how far away the arrow keys - or your mouse - are, are completely pointless in comparison. This article almost caught on to this fact and gives a list of editors with their strength but doesn’t make it to the conclusion that text editors are just not a reasonable thing to discuss anymore. Nobody edits “text”. And the people who do use Word.
-
All of these discussions about how far away the arrow keys - or your mouse - are, are completely pointless in comparison. This article almost caught on to this fact and gives a list of editors with their strength but doesn’t make it to the conclusion that text editors are just not a reasonable thing to discuss anymore. Nobody edits “text”. And the people who do use Word.
That’s amusing. Just last night my roommate had a problem where all the cells in her document were set to “£0.00” after saving, it would have taken about 8 hours for her to recover it.
Luckily she already had the list, she just wanted it in a Libreoffice table. I opened vim (Because raw sed is line-based), and just used the expressions:
:%s/$/"/,:%s/ \r/, "/. I saved it as a .csv and opened it in Libreoffice Calc, copied the cells, then imported it into Writer by doingC-Vand selecting “RTF”.It took five minutes, not 8 hours. A handful of small adjustments that added 2 minutes on to it, and the table was exactly as before.
You are severely underestimating the power and use of the tools programmers have been using for the last 60 years. The arrogance on display is absolutely staggering.
The purpose of a tool like Vim, is to make your editing commands become muscle memory, so you do not have to think about what you want to do, and then do it. You can simply do it reflexively. I’ve been using it for 8 years on all kinds of information, and it’s extremely powerful.
I wonder, when I hit
d}in vim, am I still editing text? The operation itself runs over a paragraph. What aboutdi)which deletes the contents of a set of brackets, do you think you can find a use for that in programming? If your answer here is “paragraphs and argument lists are all text”, that means you’re coming close to the realization that “XML is text”, “S-Expressions are text”, and pretty much every format you can think of is a form of text.Is ParEdit, a mode for editing the structure of Lisp documents in Emacs, still editing text? What about rope, a refactoring tool for Python with both Emacs and Vim plugins, when you use those plugins, are you still editing text? Vim provides an interface for “text objects”, which allow you to manipulate XML rather easily, are you editing text, or the objects? Of course you are editing text. Because everything that isn’t a binary format is text.
In addition, both Emacs and Vim have binary editors built in. They both have extensive sets of plugins for editing programming languages while retaining the power of each environment, and the benefit of a uniform interface that you have experience and will log 100s of hours in.
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
You’re right, however, that you don’t want to edit all programs the same way. However, most modern text editors have functionality specific to what you’re editing. Hell, the exact commands used for alphabetically sorting the contents of CSS directives in Vim (
:%g/{/ .+1,/}/-1 sort) dates back to before Vim existed – it was a feature ofed(1)– which isn’t even visual!-
So, I think your argument is: we are not editing text we are editing all kinds of things in a text representation. Which is also what I was trying to say.
I think we just disagree about the effectiveness of this approach. I agree that the editors we have are often better than nothing but I wouldn’t advertise vim for it’s ability to edit tabular data using :s. Neither do I think the incantation in your last line is something that I should have to know. I just want a CSS editor and then press the sort button.
#editorflamewar2020
-
So, I think your argument is: we are not editing text we are editing all kinds of things in a text representation.
No, my point was that we are all editing text, we just smooth it over so we don’t have to care about the specifics. Text editors do this too, except they provide more power if you wish it. From your comment here and your other comments about this, you seem grossly uneducated about the capabilities of text editors as tools. I’ve linked you articles in another comment – please read them.
I just want a CSS editor and then press the sort button.
You don’t have to know it.
nnoremap <leader> s :%g/{/ .+1,/}/-1 sortMy leader key is set to
f. So I can just typef sand the document is sorted. In emacs, I can set a mode switch, so that this keybinding is only available for CSS documents.-
In emacs, I can set a mode switch, so that this keybinding is only available for CSS documents.
FYI, you can do the same in Vim with ftplugins.
-
No need for anything complex, just use autocmd
-
-
-
You are not expected to know this incantation. It is not a word, it is a sentence. Do you «know» the sentence «Neither do I think the incantation in your last line is something that I should have to know.» ?
The reason we prefer to use a single editor with a language to think about editing in [without agreeing which one] is that things that are just out of reach of the functionality supplied by the editor/plugins/etc. are actually within reach if you express them in the slang of the editor.
Learning such a language probably doesn’t pay off if handling the things typically represented as text is not an activity taking up a large share of your computer use. And maybe not if you just need five representations and the single-use options available for these representations are good enough from your point of view.
But as there are more (and slightly less popular) things to handle, no thanks, I will take a small set of tools I know well and can push one step beyond over sorting through special-case solutions, with many of them missing «trivial» features I have come to depend on.
-
-
-
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
Why not? As an Emacs user, I appreciate that different modes interpret concepts such as paragraphs, words, symbols, top level structures in their own appropriate ways, so that keybindings and other commands trivially adapt to the kind of file I’m working with. Sure, not everything fits perfectly into the abstract, primitive categories, but it’s still preferable to having n different editors and work environments for n different problems.
-
“Why not?”
Because having a dedicated editor is more efficient. PyCharm is better for Python and IntelliJ better for Java and Kile better for Latex and Trello better for Todos and Thunderbird better for Mails.
The reason you can use Emacs reasonably well for all of those is that it actually tries to be n different editors for n different problems. org-mode barely has anything in common with other modes. Paredit as well, etc…
-
Because having a dedicated editor is more efficient
Having a dedicated editor may be more efficient for a single thing, but most of the projects I work on involve ‘text’ files with at least half a dozen different languages (programming languages, build system scriting, markup languages). If I had to switch between editors for each one then each editor would have to be a lot more efficient to offset the cost of the cognitive load from having different tooling for each file type.
-
Well I always feel a lot less productive in IntelliJ editors, and find the Emacs equivalents a lot more comfortable. I guess that’s a difference in attitude, but I think it should be considered as a factor.
The reason you can use Emacs reasonably well for all of those is that it actually tries to be n different editors for n different problems
Major modes are interfaces, of sorts, connecting and implementing the files to existing subsystems (completion-at-point, imenu, xref, …), not their own Editors. Sure, some major modes have more additional keybindints, such as Org mode, but it’s not unrelated: I can still use isearch, avy,
highlight-symbol-at-point, etc. or any other function that wasn’t written for a specific mode in mind. Paredit is a minor mode for S-Expression based file formats, so I don’t quite get your point here…
-
-
-
You don’t want to edit HTML and Java and Latex and ToDos and emails in the same way.
I most certainly want to do exactly that.
-
Nobody edits “text”. And the people who do use Word.
Thanks for making this point clear. I think programmers (usually systems programmers who post to Lobsters or The Orange Site) have basically boxed themselves into a “fake reality” where talking about plain text email and using vi for prose actually matter. I’m basically the only systems programmer on my team (excluding maybe one or two others), and talking to clients (who almost are all non-technical industries) or even other team members - they have big HTML signatures with images (and use formatting in their messages!), reply on top, use IDEs or maybe Notepad++ on Windows. They don’t tweak their window manager or Emacs config, they’re busy whatever line-of-business application.
-
I’m not sure that applies to everyone. I have proselint jacked into vim and I use it frequently. The ability to perform operations on sentences or paragraphs without having to move your hand to the mouse is underappreciated.
-
-
-
This is not incorrect, but broad enough that I’d probably get more use out of the inverse of this list. For example, listing ‘cat’ as a program that when run with the suid bit set, will read privileged files is.. erm.. noise. ‘It runs with elevated privileges under sudo’ is also similarly applicable to basically everything.
And as a result, programs that give surprising shell access, like patch(1), which can invoke ed(1), which has the ability to filter text through the shell, may go unnoticed in this list, when they should be ringing alarm bells.
For that matter, this list doesn’t even include patch(1).
-
I think that’s a fair criticism, and a good point!
-
Haha yeah I saw “cobc runs in a privileged context” and immediately went to check if it was setuid on my system… turns out they just run it with sudo in the example.
-
-
-
I like the article, I think it’s valuable, but please…
Since time immemorial PostgreSQL supports JSON fields and can even index them. By immemorial I mean …
I perfectly remember the world where PostgreSQL had no JSON support
why yes, that is exactly the opposite of “immemorial”! And yes, I know that the phrase “since time immemorial” is almost always used hyperbolically, but what’s the use of a hyperbole that you unceremoniously shoot down just one sentence later?
-
I interpreted that comment as a joke
-
-
Great read! I use
scpquite a lot. It sounds like I should learnsftp.-
-
This. Also, while rsync can be used like a drop-in scp replacement, it can do a lot more than that and for copying or syncing whole trees or filesystems there’s nothing better. Just as ssh is the swiss-army knife of secure connections, rsync is the swiss-army knife of file tree copying / moving / duplicating. It’s well worth becoming thoroughly familiar with its command-line options.
-
rsync is a bit painful about when it copies files or directories and where exactly they will end up in what situations. I came to prefer https://github.com/DragonFlyBSD/cpdup because its UI is safe enough for me.
-
You only have to remember one basic Unix rule: foo is the directory itself and foo/ is the contents of the directory
-
-
-
-
No server required on remote, but yes you do need the binary on remote. I haven’t had any problems with it not being installed places I need, but you may have some different targets than I do. There are a couple of statically compiled copies of rsync you can drop in for different arches.
-
-
-
Can’t say I’ve ever used -3. So without -3 it sounds like it tries to copy directly from one remote to another, but with -3 it’ll copy to the local machine and then pass the file through to the remote? If I understand correctly, then I believe you’re correct, rsync has no builtin capabilities for handling copying between two remotes. You’ll have to setup port forwards.
-
-
-
sftpbehaves very similarly toftpwhen it comes to commands, plus any recent releases come with tab completion for both local and remote files.All you’ll probably ever need is
put filename,get filename,put -r folder,get -r folder,ls,lls,cd,lcd. You can also run commands locally like “!vim filename”.It’s very easy to get used to.
-
-
-
This looks great, let’s please replace PGP with it everywhere. :D
-
Yes, let’s replace a system which has been tested and proven and worked on since the 1990s with a random 47-commit project someone just tossed up on GitHub. Because good encryption is easy.
/s
-
File encryption is in fact kind of easy, thanks to everything we learned in the last 30 years.
-
Yes, actually.
-
I don’t see the point in sarcasm. PGP does many things and most of them are handled poorly be default. This is not a PGP replacement, it’s a tools with single purpose: file encryption. It’s not for safe transfers, it’s not for mail. It’s got a mature spec and it’s designed and developed by folks who are in the crypto community and there are two ref implementations. It does one thing and does it well which is everything PGP isn’t.
-
I guess even the author of PGP would be up for that: https://www.vice.com/en_us/article/vvbw9a/even-the-inventor-of-pgp-doesnt-use-pgp
-
In a cryptography context, “since the 1990s” is basically derogatory. Old crypto projects are infamous for keeping awful insecure garbage around for compatibility, and this has been abused many many times (downgrading TLS to “export grade” primitives anyone?)
-
I think icefox’s comment was already being sarcastic
-
-
-
This should answer your question better than I ever will.
-
-
-
-
-
-
-
Disclosure timeline:
- 13th September 2019: We submitted the issue to product-security@apple.com
- 18th September 2019: Apple asked us the resend the screen shots
- 10th October 2019: Apple told us that they were planning to address this issue in a future update
- 30th October 2019: Apple released version 12.10.2 of iTunes but did not fix the issue
- We asked several times about this case but no answer from Apple
- 9th December 2019: We informed Apple that we would release a public post about this issue on 12th of December (90 days since the initial submission)
- 11th December 2019: Apple released version 12.10.3 of iTunes but did not fix the issue
- 12th December 2019: still no answer, post has been published
Is this normal?
-
Unfortunately it feels like it’s getting more common.
I think a lot of new websites doing server side rendering are using react/jsx on the server side. Definitely not all, but I would disagree with the author that there has been a “lack of adoption” of AST-based templating. Of course PHP and ERB are both older and have a huge number of websites using them - but that doesn’t necessarily mean that adoption of AST-based templating systems has not also occurred.