Of all of these, the most important is text handling. This is the biggest reason that macOS apps provide a clean and consistent UI: NSTextView is sufficiently powerful to do everything from displaying a label to rendering multi-page documents with tables and figures inline. Almost equally importantly, it’s coupled with a rich text abstraction that allows coupling arbitrary bits of metadata to a range of characters so users can attach semantic markup information and then generate instructions for the text renderer with the same data structure.
The WebView is similarly complicated. If you’re building your own widget set then you either need to make your web view use them or it will feel weird. When Google forked WebKit to make Blink, they ripped out a load of the UI toolkit abstraction layers, so you’re probably stuck with either WebKit or Gecko, both of which have tremendously exciting build systems.
For input, it’s really worth looking at Taligent, which had a fantastic model that decoupled raw UI events from higher-level inputs and provided mechanisms for adding custom ones easily. Things like gestures are trivial to add in this model.
macOS is also about the only system I’ve used that gets drag-and-drop right (I think X11 does? Wayland gets it wrong): Drag and drop is fundamentally different from copy-and-paste because copy-and-paste has to handle the case where the source program has quit before the paste, whereas drag-and-drop must be completely responsive. This means that a drag should provide a list of possible types and nothing else, you should not require the actual data to be provided (which can take a few seconds if you’re dragging something like a video) until the drop event.
X drag and drop works very similarly to the X clipboard. Now I know a lot of people love to hate the X clipboard since it doesn’t persist when the program exits but I actually like it a lot but regardless, like you said,t hat model works really well for drag and drop anyway.
The dragging application advertises that it has something and tells windows it passes over that it is passing over, then the window can ask for the list of types you offer. If they like one, they tell you what operations they will allow (the dragger application is always responsible for updating the cursor so it needs to know). If the user drops, then the drop application asks the drag information for a particular format from the advertised list and the data is transferred in incremental chunks. All these interprocess asks are done through the X server as the middle man rather than strictly peer to peer, to ensure it works even if the applications are running on separate closed-off machines.
X copy paste is basically that same process, just without the dialog with the dragger. The “copy” application advertises it has something, which takes ownership away from whatever previous app (if any) had it. Then the pasting application knows it can ask the “copying” application for the list of formats. If it likes one, it asks for the data and it is transferred over. (I used scare quotes because nothing is actually copied until the transfer of a specific format requested. This is why people hate it but also why I like it - it is flexible and efficient, once you get to know it at least.)
The Windows way for drag and drop actually can be really similar to the X way btw: you provide a COM interface that has methods that provide formats on-demand. One method lists the available options and another method requests a copy of one of the formats. You don’t need to actually copy things up front…
EDIT: I guess lobsters doesn’t let me do two comments on separate topics to the same parent. oh well
Of all of these, the most important is text handling.
I think the text handling thing is why HTML did so well. You can pretty easily mix and match rich text and in-flow controls. That kind of thing was a legit pain to do in old desktop apis - they’d tend to offer something high level like a pre-packaged RTF widget and something low level like the text metrics…. but that middle level of html’s mix took a lot of work. The RTF with OLE components is about as close as you got but like html is soooo much simpler.
That kind of thing was a legit pain to do in old desktop apis - they’d tend to offer something high level like a pre-packaged RTF widget and something low level like the text metrics…. but that middle level of html’s mix took a lot of work. The RTF with OLE components is about as close as you got but like html is soooo much simpler.
NSTextView with NSText (NSAttributeString) gives you a very similar level of control to HTML. It’s not a coincidence. The original WorldWideWeb was a very thin wrapper around NSTextView that stripped the tags from a string and converted them into NSAttributedString tags before passing the result to the NSTextView for rendering.
Similarly, Win32 used to have its RichEditEx decades ago, although not as able.
Windows also had a universal web view component. It forms some of the most jarring additions to Windows 98 and mainly 2000.
I don’t see how Wayland does it wrong: https://wayland.app/protocols/wayland#wl_data_device:request:set_selection.
X11 and Wayland have fully “responsive” DnD and selections, which is what you seem to be concerned about. On the other hand, they share the aspect of the selection disappearing with its owner, so in both you need a clipboard manager to retain them, preferably for text/images only–it’s a decision to make.
Similarly, Win32 used to have its RichEditEx decades ago, although not as able.
It’s been around for ages, but it’s not very useful. It didn’t have a real abstraction over the rich text (it kind-of does now) so if you wanted to use it as anything other than an opaque source or consumer of RTF data, you ended up having to write your own RTF parser and generator.
I am led to believe (with no insider knowledge, through things I’ve read in the press) that in the ‘90s the Office team was deeply concerned that a more powerful rich text widget would make it easy for people to write a Word competitor. With NSTextView, you can write something that’s got the core feature set of Word ’97 (styles, multi-column layout, live spell checking, inline images, printing, and so on) in around a thousand lines of code and something quite polished in about 10K. Keeping a bunch of those features Office-exclusive probably led to a lot of the growth of web apps early on: It was easier to write a web app and delegate UI text layout to IE than it was to deal with RichTextEx.
I don’t see how Wayland does it wrong:
Oh, great! It looks as if this has been fixed since I last looked at Wayland in detail (which, I now realise, was probably at least 10 years ago, back when I was actively contributing to GNUstep). Back then, their protocol was built on top of copy-and-paste: you put every format that you could provide onto a pasteboard (I can’t remember the Wayland terminology) at the start of the drag. To be honest, getting this so fundamentally wrong was the main reason I stopped paying attention to Wayland. It’s probably been fixed for a long time without my noticing.
I also used to be angry at Wayland, but it’s developing, albeit very slowly.
In particular, I thought it wouldn’t be possible to monitor all selected text, though Sway, which is what I’d naturally use for a WM, now supports both Xwayland synchronisation, and some unstable protocols to do it natively. But, e.g., I still don’t get 10-bit colour or indicators like nm-applet, so it’s plainly worse than X.org + i3, and will likely stay that way for a few years to come.
My dislike of Wayland has hence shifted towards this complexity. The core is tiny, and you need to have the extensions you want supported by the compositor. Not much can be depended on.
Something about throwing the baby out with the bathwater.
My dislike of Wayland has hence shifted towards this complexity. The core is tiny, and you need to have the extensions you want supported by the compositor. Not much can be depended on.
Well, it’s good to know that they didn’t replicate the problems with X11, which has a tiny core and implements everything in different extensions (some in the X server, some provided by the window manager or compositing manager) that don’t always compose well and are not universally supported.
I’m not a person that uses tray icons, but I think the nm-applet weirdness only applies to swaybar, which is a specific status bar and not anything inherent to the protocol itself. It doesn’t reflect on Wayland any more than a buggy GNOME status bar would reflect on X.
That being said, the “tiny core + extensions” thing bothers me. Though wlroots does appear to give a decent baseline for stuff.
Sway and the bar are just indicative of the practical usability of Wayland–it still hasn’t caught up–also, the protocol doesn’t live in a vacuum. More examples can be found. E.g., someone on IRC had a problem with Wayland not supporting QWindow::requestActivate(), and when I looked into it, the protocol in staging (for once not unstable) meant to support this kind of functionality still can’t be used to implement this method. I taught him how to force the program to run under XWayland.
Oh yeah, that’s a fair point. I misread you as blaming it on the protocol itself; my bad.
I’ve been using it to work on, but screen recording was iffy for a while, and I miss all the fancy tools X has. I hear it’s better in GNOME-land, but that’s because they have their own protocols for everything… bleh. And not having window urgency or a way for the currently focused window to transfer that focus is just bizarre.
(I used sway until very recently; I switched to river because I like its layout model more, and because of frustration with sway’s “no features that aren’t in i3” philosophy.)
It’s about 15 years since I last read the docs, back when we were designing EtoileUI. The API docs were online somewhere, but I didn’t bookmark them and I’ve no idea if they’re still there now…
Oh, also: whatever approach you choose, you are going to also need to provide an ergonomic, performant animation API.
This is where I died inside. Just no. Animations are one of the most despicable developments of recent UIs. I don’t want to keep either waiting for the computer to do its trivial jobs, or distract me in general altogether. Want to make your phone faster? Disable animations!
You need to support emoji.
Also no, this is a largely pointless complication.
Async You do have nice ergonomic async support, don’t you?
What does this even mean? Win32/X11/Qt/GTK+ are all async by their sheer nature.
He’s describing an opinionated, everything-but-the-kitchen-sink approach. But a decent overview nonetheless.
Complaints about animations an emoji sound very much like “old man yells at cloud”.
Emoji (and generally Unicode support for more than BMP) is now an expected feature of modern software.
UI animations can be done well and be helpful, and hint how UI items relate to each other spatially. It’s really weird that even low-end mobile devices have GPUs that can effortlessly animate millions of 3D triangles, but sliding a menu is considered a big effort.
I’ve honestly never seen a single helpful UI animation in my life, other than spinning wheels and the likes, indicating that the application/system hasn’t frozen up completely. Instead of “click, click, click, be done”, things tend to shift into “click, have your attention grabbed, click, have your attention grabbed, throw your computer out of the window and go live in the woods”. GNOME Builder and GIMP 3 settings dialogues are spectacular examples of GNOME keeping digging its grave.
One would expect the computer to be a tool, rather than a work of art.
For example, macOS has generally well-done animations that don’t get in the way. It tends to animate after an action, but not before. Animations are very quick, unless they’re used to hide loading time. There are also animations to bring your attention to newly created/focused elements. Reordering of elements (menu icons, tabs) is smooth. IMHO they usually add value.
Touchscreen OSes also animate majority of elements that can be swiped or dragged. The animation is not on a fixed timer, but “played” by your finger movements, so you may not think about it as an animation, but technically it is animating positions and sizes of UI elements. Users would think UI is broken if it instantly jumped instead of “sticking” under the finger.
Eh, macOS has sort of jumped the gun on animation, too. I’ve activated the “Reduce motion” for it because, by default, the transition between full-screen apps is done by having windows “slide” in and out of view. It’s not just slow (it takes substantially more time to do the transition that in takes to Cmd-Tab between the applications), it legit makes you dizzy if you keep switching back and forth.
I imagine it’s taken verbatim from iPad (last macOS version I used before Big Sur was… Tiger, I think? so I’m not sure…) where it makes some sense, but I can’t for the life of me understand what’s the point of it on a thing that has a keyboard and doesn’t have a touch screen.
I can’t for the life of me understand what’s the point of it on a thing that has a keyboard and doesn’t have a touch screen.
Probably because most people will use their MacBook trackpad to swipe between full screen apps/desktops. I only rarely use the Ctrl-arrow shortcuts for that. A lot of macOS decisions make more sense when you assume the use of a trackpad rather than a mouse (which is why I’ve always found it weird they don’t ship a trackpad with iMacs by default)
You can still cmd-tab between full-screen applications, which is what a lot of people actually do – a “modern” workplace easily gets you an email client, several Preview windows, a Finder window, and a spreadsheet. Trackpad swiping works great when you have two apps open, not so much when you got a bunch of them.
When you’re on the seventh out of the fifteen invoices you got open, you kindda want to get back to the spreadsheet without seeing seven invoices again. That’s actually a bad example because full-screen windows from the same app are handled very poorly but you get the point: lots of apps, you don’t always want to see all of them before you get to the one you need…
These animations are helpful, and have been shown to be so. It’s not something some asshole down at Cupertino just cooked up because he thought it would look cool. Cues as to the spatial relations of things (as the desktops have an order and you use left/right gestures to navigate them) are very valuable to a lot of people, and they even let you turn them off, I don’t really see anything worth complaining about.
I mean there’s a lot of questionable things Apple is doing these days, but that’s not one of them.
I’m not talking about desktops, but “maximized” applications (i.e. the default, full-screen maximisation you get when you press what used to be the green button).
You get full-screen sliding animations when you Cmd-Tab between apps in this case, even though there’s no spatial relations between them, as the order in which they’re shown in the Cmd-Tab stack has nothing to do with the order in which they’re shown in Mission Control (the former is obviously mutable, since it’s in a stack, the latter is fixed).
In fact, precisely because one’s a stack and the other one isn’t, the visual cue is wrong half the time: you switch to an application to the right of the stack, but the screen slides out to the left.
Animation when transitioning between virtual desktops is a whole other story and yes, it makes every bit of sense there.
and have been shown to be so.
Do you have a study/some data for that? I know of some (I don’t have the papers at hand but I can look it up if you’re curious), but it explicitly expects only occasional use so it doesn’t even attempt to discuss the “what if you use it too much” case. So it’s not even close to applying to the use case of e.g. switching between a spreadsheet and the email client several times a minute.
(Edit: just for the fun of it, I tried to count how often I Cmd-Tab between a terminal window and the reference manual after I ended up ticking the reduce animation box in accessibility options. I didn’t automate it so I gave up after about half an hour, at which point I was already well past 100. Even if this did encode any spatial cues, I think spatial cues are not quite as valuable as not throwing up my guts.)
There are many legitimate uses for animations. Yes loading spinners are one of them, unless you think that every single operation can be performed instantly. Sliding and moving elements around on mobile especially is another one. A short animations to reveal new elements after a user action can also improve the UX.
Not everything has to be one extreme or another - it’s not pointless animations everywhere, or no animations at all. When used well they can improve usability.
Loading spinners are far from ideal though. A progress bar would be generally better so you have some idea of if it is actually still working and how far is left to go. Or anything else that provides similar information.
I’ve seen so many times when a loading spinner sits there spinning forever because, for example, the wifi disconnected. The animation then is misleading, since it will never complete.
That’s an entirely different loading element. You can have animated progress bar. And when progress is indeterminate a spinner makes the most sense. A spinner not stopping on error is a UI bug, not a problem with the concept. If you want to get mad at bad design, how about programming languages and paradigms that don’t make you explicitly handle errors to get in this state.
The standard way of indicating length is to say so to the user, and possibly add a progress bar, for when progress can be sensibly measured. Like adding a spinner, it needs to be done explicitly. Revealing new elements can be done responsively by improving the design–the standard way is by turning an arrow.
The notion of phones invading GUIs, as x64k hinted at, is interesting here (though not new). Transplanting things that do not belong, just because of familiarity.
Going back to the article, it said I needed to. I don’t. And still, I can make anything I desire with the toolkit, with no real change to the required effort. Except for “modern” IM, when I don’t care to implement pictures as pseudo-characters.
One would expect the computer to be a tool, rather than a work of art.
The computers I remember most fondly all had some qualities of art in it. Sometimes in the hardware, others in the software, and the ones I like the most had it in both.
Animations are important in devices in which there is mostly visual feedback. Most computers don’t have haptics, and we often get annoyed at audio feedback. Visual cues that an action was performed or that something happened are important. There is a difference between UI animation and a firework explosion in the corner of the app.
One would expect the computer to be a tool, rather than a work of art.
You, me, and everybody in this thread is in the top 0.001% of computer power users. What’s important to us is totally different than what’s important to the rest of the population. Most normies I talk to value pleasing UIs. If you want to appeal to them, you’re going to need animations.
I’d argue even further that the 0.001% of computer power users also need animations. When done well, animations really effectively convey state changes in a way that speaks to our psychology. A great example is that when I scroll a full page in my web browser, the brief scroll animation shows my brain how where I am now relates to where I was a split second ago. Contrast this to TUIs which scroll by a full page in a single frame. It’s easy to consciously believe that we can do without things like animations, but I’m pretty sure that all the little immediate state changes can add up to a subperceptual bit of cognitive load that nevertheless can be fatiguing.
Absolutely. I love the subtle animations in iOS. Like when I long-press on the brightness slider to access more controls like the dark mode toggle. I’m already making an action that takes more time than a simple tap, and the OS responds with a perfectly timed animation to indicate that my action is being processed.
On the other hand, animations can be very easily abused. Plenty of examples, like today’s Grumpy Website post, show animations that hinder accessibility. I think the cases where animation goes wrong are where it was thrown in only because “it’s modern” rather than primarily as a means to convey information.
I respectfully disagree with your opinion about animations. There are lots of times when I genuinely feel that the animation is important and actually conveys information. For example, the Exposè-style interface in macOS and GNOME is much better since the windows animate from their “physical” locations to their “overview” locations; the animation provides important context about which windows went where, so your eyes get to track the moving objects. It also helps that those animations track your finger movements on the trackpad perfectly, with one pixel of movement per tiny distance travelled across the trackpad (though the animation also has value when triggered using a hotkey IMO).
But there’s definitely a lot of software which over-uses animations. The cardinal sin of a lot of GUI animations is to make the user wait for your flashy effects. A lot of Apple’s and GNOME’s animations do fit this description, as well as arguably most animations in general. So I think a GUI framework needs a robust animation system for when it’s appropriate, but application programmers must show much more discretion about when and how they choose to use animations. For example, I’m currently in Safari on macOS, and when I do the pinch gesture to show an overview of all the tabs, I have to wait far too long for the machine to finish its way too flashy zoom animation until I actually get the information I need in order to interact further.
What does this even mean? Win32/X11/Qt/GTK+ are all async by their sheer nature.
Not really. It is very easy to block the event loop with various other calls (including things like clipboard which like the thing said is async under the hood on X, but qt and gtk don’t present it that way). The gui thing needs to have some way for other operations to hook into the event loop so you can avoid blocking.
Not terribly difficult but still you do need to at least think about it, it isn’t fully automatic. (Even on Windows, where it is heavenly bliss compared to unix, you do still need to call a function in your loop like SleepEx to opt into some additional sync processing.)
GTK+ does present clipboard retrieval asynchronously, see https://docs.gtk.org/gtk3/method.Clipboard.request_contents.html which contains a callback argument–that much I remember. Setting clipboard contents can be done blindly, you can have ownership snatched at any moment.
Going the way of recursive event loops requires caution that I would avoid imparting on users as much as possible, in particular because of callbacks vs state hell. Typically, this is reserved for modal dialogues, and the API user knows what they’re dealing with.
There’s also the possibility of processing the event pump selectively, though that’s another thing you don’t want to do to yourself.
Want to make your phone faster? Disable animations!
This is generally the point of animations as a UX feature - they mask the slow operation of applications with a silly animation to keep you distracted/indicate a process is occurring. Want to notice when your app is using a jpeg to pretend it’s loaded? Disable animations!
It’s not only that. The human visual system is very good at tracking movement, much worse at noting that a thing has disappeared and reappeared. If a thing disappears, you’ll typically notice quickly but not immediately be aware of what has disappeared. If you animate movement then there’s lower cognitive load because a part of your brain that evolved to track prey / predators is used rather than anything related to understanding the context of the application.
I don’t think OP is talking about things like progress bars and spinning indicators, which are pretty legitimate everywhere, but things like “gliding” page transitions between application screens. If a framework is indeed so slow that you notice it rendering widgets, an animation API will help now and then, but won’t make that big a dent. (Edit: also, unless you’re loading things off a network, loading JPEGs cannot be a problem anymore, it hasn’t been a problem in twenty years!)
I do think this piece could’ve been more aptly titled “so you want to write a GUI framework for smartphones”. Animations are important on touch screen driven by gestures (e.g. swiping) – gestures are failure-prone, they need incremental feedback and so on, plus nobody who’s not high on their 2013-era post-PC supply expects efficiency out of phone UIs.
But they are pretty cringy on desktop. E.g. KDE’s Control Center (and many Kirigami apps) has an annoying misfeature, where you click “back” and the page moves out of view as if you’d swiped it. But you didn’t swipe. Regardless of what you think about animation, it’s not even the right animation.
That’s why so many people see them as useless eye candy. If you go all Borg on it and only think in absolute terms, you get a very Borg user experience.
Edit: yes, I know, “a modern GUI toolkit” should have all these things. The point is you can drop a lot of them and still write useful and efficient applications. Just because Google is doing something on Android doesn’t mean everyone has to do it everywhere.
It’s funny you mention page transitions. I have my ebook reader set up to do a 200ms-ish animation when I tap the ‘next page’ button where the current page slides off to the left and the next one slides in from the right. It has an option to disable it, but I actually find that disorienting in this vague way I can’t explain. But on my desktop, it’s fine with no animations.
Empirically, that is not how it’s used. This masking is a minority of use cases, and even then it’s bad. To some people that aren’t me, it might be better described as “eye candy” and “smoothness”.
Being able to disable this irritation is being lucky, e.g. it’s CSS-hardcoded in Firefox and GTK+ (/org/gnome/desktop/interface/enable-animations only works partially).
What does this even mean? Win32/X11/Qt/GTK+ are all async by their sheer nature.
I think they’re talking about Rust async, since this is all in the context of writing a cross-platform GUI toolkit in Rust. This is more of a problem than it seems because if you’re doing a cross-platform toolkit that uses native widgets, it’s not at all trivial to impedance-match whatever model the native widget toolkit uses behind the scenes with your toolkit, which exposes an async model.
(Edit: there are some additional complications here, too. For example there are toolkits that (generally) async, but still do some operations synchronously. The author mentions the copy-paste API as an example.)
One might conclude that it’s better to not do any of that and instead expose the platform’s intended programming model, as second-guessing thirty year-old code bases tends to backfire spectacularly, but maybe I’m just being grumpy here…
One potential limitation is that compute shaders are a relatively new feature, and are only available in GPUs made in the last five-or-so years.
Something to watch out for with GPU-based 2D rendering in general is that it might unnecessarily make the application unusable on older hardware. I’m reminded of a sad case involving WPF about a decade ago: a Bible study application where the choice of video card actually mattered. I know about this particular case because it affected my brother when he tried to run that program on an older laptop. I find it sad that, as shown by the linked forum thread, people had to take time away from their real work to discuss video cards. And bear in mind that the application in question was basically a specialized text viewer and editor.
Well, it’s a Christian forum rather than one on ethics in general. Many Christians aren’t concerned much about those things due to the sovereignty if God. As in, how He exerts His will on all things to accomplish His purposes regardless of the schemes of men. From the past (eg Ninevah in Nahum or reforming Israel) to future (eg Revelation), He’s not stopped by much worse oppression and corruption than software licensing. Ex: The number of believers growing massively in China despite the Chinese regularly jailing them. Similary, but smaller numbers, in North Korea. His Word and Will are unstoppable.
We’re also to be judged for how much we loved/served God and other people. The love others is things like feeding them, equipping them, being kind, listening, etc. Obviously, bringing them to Christ as well for redemption, peace, and purpose. I’ve personally found I can’t do those things while simultaneously spending lots of time tinkering with FOSS. I’ve cut way, way back on tech to invest in both God’s Word and other people with Him moving consistently.
Finally, there’s the fact that believers in Christ get the Holy Spirit to transform us but we’re still human. Our human nature remains till death. Centering on Christ supernaturally transforms in gradual way us to give more of His traits if we stay on it. Jesus says His people will bear His fruits (i.e. character). 1 John 3 has stronger words. If they don’t or are casual about it, human traits dominate with a group of them acting like any other. For instance, not caring about software freedom and griping about electronics is extremely normal. We are to call out evil, though. Grey area on graphics cards. ;)
I’ve been slowly doing my own gui code. Its name is simpledisplay.d for the basic functions and minigui.d for the buttons, checkboxes, etc. widgets…. but I keep adding stuff that comes up. simpledisplay is now 20,000 lines and minigui, built on top of it, is now up to 12,000 lines. And I use the native controls on Windows to get some of the stuff for free! For cross platform I mostly define the apis to be async then on Windows, it just immediately calls like when drag data comes in. Works fairly ok.
It is legitimately a kinda big job. But on the other hand, I’ve been semi-successful with it all. Everyone likes to tell me it is impossible but…. I’ve been working on it on-and-off for ten years, sure, but I’m getting it done.
I just wanted something for myself, but I actually do use it for (some) day job work too so it kinda contributes to other things.
I was annoyed at how big and complicated the other options were and wanted to keep it simple and small… just as time went on, I wanted more and more bits so it grew. Still simpler and smaller than most the things though. And then on Linux I get to do everything my way which is slow progress but educational and gives nice programs for me.
Of all of these, the most important is text handling. This is the biggest reason that macOS apps provide a clean and consistent UI:
NSTextViewis sufficiently powerful to do everything from displaying a label to rendering multi-page documents with tables and figures inline. Almost equally importantly, it’s coupled with a rich text abstraction that allows coupling arbitrary bits of metadata to a range of characters so users can attach semantic markup information and then generate instructions for the text renderer with the same data structure.The WebView is similarly complicated. If you’re building your own widget set then you either need to make your web view use them or it will feel weird. When Google forked WebKit to make Blink, they ripped out a load of the UI toolkit abstraction layers, so you’re probably stuck with either WebKit or Gecko, both of which have tremendously exciting build systems.
For input, it’s really worth looking at Taligent, which had a fantastic model that decoupled raw UI events from higher-level inputs and provided mechanisms for adding custom ones easily. Things like gestures are trivial to add in this model.
macOS is also about the only system I’ve used that gets drag-and-drop right (I think X11 does? Wayland gets it wrong): Drag and drop is fundamentally different from copy-and-paste because copy-and-paste has to handle the case where the source program has quit before the paste, whereas drag-and-drop must be completely responsive. This means that a drag should provide a list of possible types and nothing else, you should not require the actual data to be provided (which can take a few seconds if you’re dragging something like a video) until the drop event.
X drag and drop works very similarly to the X clipboard. Now I know a lot of people love to hate the X clipboard since it doesn’t persist when the program exits but I actually like it a lot but regardless, like you said,t hat model works really well for drag and drop anyway.
The dragging application advertises that it has something and tells windows it passes over that it is passing over, then the window can ask for the list of types you offer. If they like one, they tell you what operations they will allow (the dragger application is always responsible for updating the cursor so it needs to know). If the user drops, then the drop application asks the drag information for a particular format from the advertised list and the data is transferred in incremental chunks. All these interprocess asks are done through the X server as the middle man rather than strictly peer to peer, to ensure it works even if the applications are running on separate closed-off machines.
X copy paste is basically that same process, just without the dialog with the dragger. The “copy” application advertises it has something, which takes ownership away from whatever previous app (if any) had it. Then the pasting application knows it can ask the “copying” application for the list of formats. If it likes one, it asks for the data and it is transferred over. (I used scare quotes because nothing is actually copied until the transfer of a specific format requested. This is why people hate it but also why I like it - it is flexible and efficient, once you get to know it at least.)
The Windows way for drag and drop actually can be really similar to the X way btw: you provide a COM interface that has methods that provide formats on-demand. One method lists the available options and another method requests a copy of one of the formats. You don’t need to actually copy things up front…
EDIT: I guess lobsters doesn’t let me do two comments on separate topics to the same parent. oh well
I think the text handling thing is why HTML did so well. You can pretty easily mix and match rich text and in-flow controls. That kind of thing was a legit pain to do in old desktop apis - they’d tend to offer something high level like a pre-packaged RTF widget and something low level like the text metrics…. but that middle level of html’s mix took a lot of work. The RTF with OLE components is about as close as you got but like html is soooo much simpler.
NSTextViewwithNSText(NSAttributeString) gives you a very similar level of control to HTML. It’s not a coincidence. The original WorldWideWeb was a very thin wrapper aroundNSTextViewthat stripped the tags from a string and converted them intoNSAttributedStringtags before passing the result to theNSTextViewfor rendering.I’m not sure, but I’m guessing this isn’t a compliment…
I thought it was excellently put tho :)
Similarly, Win32 used to have its RichEditEx decades ago, although not as able.
Windows also had a universal web view component. It forms some of the most jarring additions to Windows 98 and mainly 2000.
I don’t see how Wayland does it wrong: https://wayland.app/protocols/wayland#wl_data_device:request:set_selection. X11 and Wayland have fully “responsive” DnD and selections, which is what you seem to be concerned about. On the other hand, they share the aspect of the selection disappearing with its owner, so in both you need a clipboard manager to retain them, preferably for text/images only–it’s a decision to make.
It’s been around for ages, but it’s not very useful. It didn’t have a real abstraction over the rich text (it kind-of does now) so if you wanted to use it as anything other than an opaque source or consumer of RTF data, you ended up having to write your own RTF parser and generator.
I am led to believe (with no insider knowledge, through things I’ve read in the press) that in the ‘90s the Office team was deeply concerned that a more powerful rich text widget would make it easy for people to write a Word competitor. With
NSTextView, you can write something that’s got the core feature set of Word ’97 (styles, multi-column layout, live spell checking, inline images, printing, and so on) in around a thousand lines of code and something quite polished in about 10K. Keeping a bunch of those features Office-exclusive probably led to a lot of the growth of web apps early on: It was easier to write a web app and delegate UI text layout to IE than it was to deal with RichTextEx.Oh, great! It looks as if this has been fixed since I last looked at Wayland in detail (which, I now realise, was probably at least 10 years ago, back when I was actively contributing to GNUstep). Back then, their protocol was built on top of copy-and-paste: you put every format that you could provide onto a pasteboard (I can’t remember the Wayland terminology) at the start of the drag. To be honest, getting this so fundamentally wrong was the main reason I stopped paying attention to Wayland. It’s probably been fixed for a long time without my noticing.
I also used to be angry at Wayland, but it’s developing, albeit very slowly.
In particular, I thought it wouldn’t be possible to monitor all selected text, though Sway, which is what I’d naturally use for a WM, now supports both Xwayland synchronisation, and some unstable protocols to do it natively. But, e.g., I still don’t get 10-bit colour or indicators like nm-applet, so it’s plainly worse than X.org + i3, and will likely stay that way for a few years to come.
My dislike of Wayland has hence shifted towards this complexity. The core is tiny, and you need to have the extensions you want supported by the compositor. Not much can be depended on.
Something about throwing the baby out with the bathwater.
Well, it’s good to know that they didn’t replicate the problems with X11, which has a tiny core and implements everything in different extensions (some in the X server, some provided by the window manager or compositing manager) that don’t always compose well and are not universally supported.
I’m not a person that uses tray icons, but I think the nm-applet weirdness only applies to swaybar, which is a specific status bar and not anything inherent to the protocol itself. It doesn’t reflect on Wayland any more than a buggy GNOME status bar would reflect on X.
That being said, the “tiny core + extensions” thing bothers me. Though wlroots does appear to give a decent baseline for stuff.
Sway and the bar are just indicative of the practical usability of Wayland–it still hasn’t caught up–also, the protocol doesn’t live in a vacuum. More examples can be found. E.g., someone on IRC had a problem with Wayland not supporting QWindow::requestActivate(), and when I looked into it, the protocol in staging (for once not unstable) meant to support this kind of functionality still can’t be used to implement this method. I taught him how to force the program to run under XWayland.
Oh yeah, that’s a fair point. I misread you as blaming it on the protocol itself; my bad.
I’ve been using it to work on, but screen recording was iffy for a while, and I miss all the fancy tools X has. I hear it’s better in GNOME-land, but that’s because they have their own protocols for everything… bleh. And not having window urgency or a way for the currently focused window to transfer that focus is just bizarre.
(I used sway until very recently; I switched to river because I like its layout model more, and because of frustration with sway’s “no features that aren’t in i3” philosophy.)
Can you point me to a resource that describes the way Taligent did this? I’m curious about how this works…
It’s about 15 years since I last read the docs, back when we were designing EtoileUI. The API docs were online somewhere, but I didn’t bookmark them and I’ve no idea if they’re still there now…
This is where I died inside. Just no. Animations are one of the most despicable developments of recent UIs. I don’t want to keep either waiting for the computer to do its trivial jobs, or distract me in general altogether. Want to make your phone faster? Disable animations!
Also no, this is a largely pointless complication.
What does this even mean? Win32/X11/Qt/GTK+ are all async by their sheer nature.
He’s describing an opinionated, everything-but-the-kitchen-sink approach. But a decent overview nonetheless.
Complaints about animations an emoji sound very much like “old man yells at cloud”.
Emoji (and generally Unicode support for more than BMP) is now an expected feature of modern software.
UI animations can be done well and be helpful, and hint how UI items relate to each other spatially. It’s really weird that even low-end mobile devices have GPUs that can effortlessly animate millions of 3D triangles, but sliding a menu is considered a big effort.
I’ve honestly never seen a single helpful UI animation in my life, other than spinning wheels and the likes, indicating that the application/system hasn’t frozen up completely. Instead of “click, click, click, be done”, things tend to shift into “click, have your attention grabbed, click, have your attention grabbed, throw your computer out of the window and go live in the woods”. GNOME Builder and GIMP 3 settings dialogues are spectacular examples of GNOME keeping digging its grave.
One would expect the computer to be a tool, rather than a work of art.
Maybe that’s just a failing of GNOME?
For example, macOS has generally well-done animations that don’t get in the way. It tends to animate after an action, but not before. Animations are very quick, unless they’re used to hide loading time. There are also animations to bring your attention to newly created/focused elements. Reordering of elements (menu icons, tabs) is smooth. IMHO they usually add value.
Touchscreen OSes also animate majority of elements that can be swiped or dragged. The animation is not on a fixed timer, but “played” by your finger movements, so you may not think about it as an animation, but technically it is animating positions and sizes of UI elements. Users would think UI is broken if it instantly jumped instead of “sticking” under the finger.
Eh, macOS has sort of jumped the gun on animation, too. I’ve activated the “Reduce motion” for it because, by default, the transition between full-screen apps is done by having windows “slide” in and out of view. It’s not just slow (it takes substantially more time to do the transition that in takes to Cmd-Tab between the applications), it legit makes you dizzy if you keep switching back and forth.
I imagine it’s taken verbatim from iPad (last macOS version I used before Big Sur was… Tiger, I think? so I’m not sure…) where it makes some sense, but I can’t for the life of me understand what’s the point of it on a thing that has a keyboard and doesn’t have a touch screen.
Probably because most people will use their MacBook trackpad to swipe between full screen apps/desktops. I only rarely use the Ctrl-arrow shortcuts for that. A lot of macOS decisions make more sense when you assume the use of a trackpad rather than a mouse (which is why I’ve always found it weird they don’t ship a trackpad with iMacs by default)
You can still cmd-tab between full-screen applications, which is what a lot of people actually do – a “modern” workplace easily gets you an email client, several Preview windows, a Finder window, and a spreadsheet. Trackpad swiping works great when you have two apps open, not so much when you got a bunch of them.
When you’re on the seventh out of the fifteen invoices you got open, you kindda want to get back to the spreadsheet without seeing seven invoices again.That’s actually a bad example because full-screen windows from the same app are handled very poorly but you get the point: lots of apps, you don’t always want to see all of them before you get to the one you need…These animations are helpful, and have been shown to be so. It’s not something some asshole down at Cupertino just cooked up because he thought it would look cool. Cues as to the spatial relations of things (as the desktops have an order and you use left/right gestures to navigate them) are very valuable to a lot of people, and they even let you turn them off, I don’t really see anything worth complaining about.
I mean there’s a lot of questionable things Apple is doing these days, but that’s not one of them.
I’m not talking about desktops, but “maximized” applications (i.e. the default, full-screen maximisation you get when you press what used to be the green button).
You get full-screen sliding animations when you Cmd-Tab between apps in this case, even though there’s no spatial relations between them, as the order in which they’re shown in the Cmd-Tab stack has nothing to do with the order in which they’re shown in Mission Control (the former is obviously mutable, since it’s in a stack, the latter is fixed).
In fact, precisely because one’s a stack and the other one isn’t, the visual cue is wrong half the time: you switch to an application to the right of the stack, but the screen slides out to the left.
Animation when transitioning between virtual desktops is a whole other story and yes, it makes every bit of sense there.
Do you have a study/some data for that? I know of some (I don’t have the papers at hand but I can look it up if you’re curious), but it explicitly expects only occasional use so it doesn’t even attempt to discuss the “what if you use it too much” case. So it’s not even close to applying to the use case of e.g. switching between a spreadsheet and the email client several times a minute.
(Edit: just for the fun of it, I tried to count how often I Cmd-Tab between a terminal window and the reference manual after I ended up ticking the reduce animation box in accessibility options. I didn’t automate it so I gave up after about half an hour, at which point I was already well past 100. Even if this did encode any spatial cues, I think spatial cues are not quite as valuable as not throwing up my guts.)
There are many legitimate uses for animations. Yes loading spinners are one of them, unless you think that every single operation can be performed instantly. Sliding and moving elements around on mobile especially is another one. A short animations to reveal new elements after a user action can also improve the UX.
Not everything has to be one extreme or another - it’s not pointless animations everywhere, or no animations at all. When used well they can improve usability.
Loading spinners are far from ideal though. A progress bar would be generally better so you have some idea of if it is actually still working and how far is left to go. Or anything else that provides similar information.
I’ve seen so many times when a loading spinner sits there spinning forever because, for example, the wifi disconnected. The animation then is misleading, since it will never complete.
That’s an entirely different loading element. You can have animated progress bar. And when progress is indeterminate a spinner makes the most sense. A spinner not stopping on error is a UI bug, not a problem with the concept. If you want to get mad at bad design, how about programming languages and paradigms that don’t make you explicitly handle errors to get in this state.
The standard way of indicating length is to say so to the user, and possibly add a progress bar, for when progress can be sensibly measured. Like adding a spinner, it needs to be done explicitly. Revealing new elements can be done responsively by improving the design–the standard way is by turning an arrow.
The notion of phones invading GUIs, as x64k hinted at, is interesting here (though not new). Transplanting things that do not belong, just because of familiarity.
Going back to the article, it said I needed to. I don’t. And still, I can make anything I desire with the toolkit, with no real change to the required effort. Except for “modern” IM, when I don’t care to implement pictures as pseudo-characters.
The computers I remember most fondly all had some qualities of art in it. Sometimes in the hardware, others in the software, and the ones I like the most had it in both.
Animations are important in devices in which there is mostly visual feedback. Most computers don’t have haptics, and we often get annoyed at audio feedback. Visual cues that an action was performed or that something happened are important. There is a difference between UI animation and a firework explosion in the corner of the app.
You, me, and everybody in this thread is in the top 0.001% of computer power users. What’s important to us is totally different than what’s important to the rest of the population. Most normies I talk to value pleasing UIs. If you want to appeal to them, you’re going to need animations.
I’d argue even further that the 0.001% of computer power users also need animations. When done well, animations really effectively convey state changes in a way that speaks to our psychology. A great example is that when I scroll a full page in my web browser, the brief scroll animation shows my brain how where I am now relates to where I was a split second ago. Contrast this to TUIs which scroll by a full page in a single frame. It’s easy to consciously believe that we can do without things like animations, but I’m pretty sure that all the little immediate state changes can add up to a subperceptual bit of cognitive load that nevertheless can be fatiguing.
I think good animations are ‘invisible’, but bad ones aren’t. So people remember the bad more than the good.
Absolutely. I love the subtle animations in iOS. Like when I long-press on the brightness slider to access more controls like the dark mode toggle. I’m already making an action that takes more time than a simple tap, and the OS responds with a perfectly timed animation to indicate that my action is being processed.
On the other hand, animations can be very easily abused. Plenty of examples, like today’s Grumpy Website post, show animations that hinder accessibility. I think the cases where animation goes wrong are where it was thrown in only because “it’s modern” rather than primarily as a means to convey information.
I respectfully disagree with your opinion about animations. There are lots of times when I genuinely feel that the animation is important and actually conveys information. For example, the Exposè-style interface in macOS and GNOME is much better since the windows animate from their “physical” locations to their “overview” locations; the animation provides important context about which windows went where, so your eyes get to track the moving objects. It also helps that those animations track your finger movements on the trackpad perfectly, with one pixel of movement per tiny distance travelled across the trackpad (though the animation also has value when triggered using a hotkey IMO).
But there’s definitely a lot of software which over-uses animations. The cardinal sin of a lot of GUI animations is to make the user wait for your flashy effects. A lot of Apple’s and GNOME’s animations do fit this description, as well as arguably most animations in general. So I think a GUI framework needs a robust animation system for when it’s appropriate, but application programmers must show much more discretion about when and how they choose to use animations. For example, I’m currently in Safari on macOS, and when I do the pinch gesture to show an overview of all the tabs, I have to wait far too long for the machine to finish its way too flashy zoom animation until I actually get the information I need in order to interact further.
Bad news, even smooth scrolling is a kind of animation.
I’ll admit, this is an improvement in the browser. Bumping my counter to one case of a useful animation.
Not really. It is very easy to block the event loop with various other calls (including things like clipboard which like the thing said is async under the hood on X, but qt and gtk don’t present it that way). The gui thing needs to have some way for other operations to hook into the event loop so you can avoid blocking.
Not terribly difficult but still you do need to at least think about it, it isn’t fully automatic. (Even on Windows, where it is heavenly bliss compared to unix, you do still need to call a function in your loop like SleepEx to opt into some additional sync processing.)
GTK+ does present clipboard retrieval asynchronously, see https://docs.gtk.org/gtk3/method.Clipboard.request_contents.html which contains a callback argument–that much I remember. Setting clipboard contents can be done blindly, you can have ownership snatched at any moment.
Going the way of recursive event loops requires caution that I would avoid imparting on users as much as possible, in particular because of callbacks vs state hell. Typically, this is reserved for modal dialogues, and the API user knows what they’re dealing with.
There’s also the possibility of processing the event pump selectively, though that’s another thing you don’t want to do to yourself.
This is generally the point of animations as a UX feature - they mask the slow operation of applications with a silly animation to keep you distracted/indicate a process is occurring. Want to notice when your app is using a jpeg to pretend it’s loaded? Disable animations!
It’s not only that. The human visual system is very good at tracking movement, much worse at noting that a thing has disappeared and reappeared. If a thing disappears, you’ll typically notice quickly but not immediately be aware of what has disappeared. If you animate movement then there’s lower cognitive load because a part of your brain that evolved to track prey / predators is used rather than anything related to understanding the context of the application.
This is it exactly. The human visual system evolved in a world where things don’t instantly flicker out of existence or appear out of nothing.
I don’t think OP is talking about things like progress bars and spinning indicators, which are pretty legitimate everywhere, but things like “gliding” page transitions between application screens. If a framework is indeed so slow that you notice it rendering widgets, an animation API will help now and then, but won’t make that big a dent. (Edit: also, unless you’re loading things off a network, loading JPEGs cannot be a problem anymore, it hasn’t been a problem in twenty years!)
I do think this piece could’ve been more aptly titled “so you want to write a GUI framework for smartphones”. Animations are important on touch screen driven by gestures (e.g. swiping) – gestures are failure-prone, they need incremental feedback and so on, plus nobody who’s not high on their 2013-era post-PC supply expects efficiency out of phone UIs.
But they are pretty cringy on desktop. E.g. KDE’s Control Center (and many Kirigami apps) has an annoying misfeature, where you click “back” and the page moves out of view as if you’d swiped it. But you didn’t swipe. Regardless of what you think about animation, it’s not even the right animation.
That’s why so many people see them as useless eye candy. If you go all Borg on it and only think in absolute terms, you get a very Borg user experience.
Edit: yes, I know, “a modern GUI toolkit” should have all these things. The point is you can drop a lot of them and still write useful and efficient applications. Just because Google is doing something on Android doesn’t mean everyone has to do it everywhere.
It’s funny you mention page transitions. I have my ebook reader set up to do a 200ms-ish animation when I tap the ‘next page’ button where the current page slides off to the left and the next one slides in from the right. It has an option to disable it, but I actually find that disorienting in this vague way I can’t explain. But on my desktop, it’s fine with no animations.
Empirically, that is not how it’s used. This masking is a minority of use cases, and even then it’s bad. To some people that aren’t me, it might be better described as “eye candy” and “smoothness”.
Being able to disable this irritation is being lucky, e.g. it’s CSS-hardcoded in Firefox and GTK+ (/org/gnome/desktop/interface/enable-animations only works partially).
I think they’re talking about Rust async, since this is all in the context of writing a cross-platform GUI toolkit in Rust. This is more of a problem than it seems because if you’re doing a cross-platform toolkit that uses native widgets, it’s not at all trivial to impedance-match whatever model the native widget toolkit uses behind the scenes with your toolkit, which exposes an async model.
(Edit: there are some additional complications here, too. For example there are toolkits that (generally) async, but still do some operations synchronously. The author mentions the copy-paste API as an example.)
One might conclude that it’s better to not do any of that and instead expose the platform’s intended programming model, as second-guessing thirty year-old code bases tends to backfire spectacularly, but maybe I’m just being grumpy here…
Something to watch out for with GPU-based 2D rendering in general is that it might unnecessarily make the application unusable on older hardware. I’m reminded of a sad case involving WPF about a decade ago: a Bible study application where the choice of video card actually mattered. I know about this particular case because it affected my brother when he tried to run that program on an older laptop. I find it sad that, as shown by the linked forum thread, people had to take time away from their real work to discuss video cards. And bear in mind that the application in question was basically a specialized text viewer and editor.
Interesting to see that even in a forum of people that probably care a lot about ethics, you’ll see this comment:
I thought at least these people would care more about freedom than about performance. But I guess not.
Well, it’s a Christian forum rather than one on ethics in general. Many Christians aren’t concerned much about those things due to the sovereignty if God. As in, how He exerts His will on all things to accomplish His purposes regardless of the schemes of men. From the past (eg Ninevah in Nahum or reforming Israel) to future (eg Revelation), He’s not stopped by much worse oppression and corruption than software licensing. Ex: The number of believers growing massively in China despite the Chinese regularly jailing them. Similary, but smaller numbers, in North Korea. His Word and Will are unstoppable.
We’re also to be judged for how much we loved/served God and other people. The love others is things like feeding them, equipping them, being kind, listening, etc. Obviously, bringing them to Christ as well for redemption, peace, and purpose. I’ve personally found I can’t do those things while simultaneously spending lots of time tinkering with FOSS. I’ve cut way, way back on tech to invest in both God’s Word and other people with Him moving consistently.
Finally, there’s the fact that believers in Christ get the Holy Spirit to transform us but we’re still human. Our human nature remains till death. Centering on Christ supernaturally transforms in gradual way us to give more of His traits if we stay on it. Jesus says His people will bear His fruits (i.e. character). 1 John 3 has stronger words. If they don’t or are casual about it, human traits dominate with a group of them acting like any other. For instance, not caring about software freedom and griping about electronics is extremely normal. We are to call out evil, though. Grey area on graphics cards. ;)
I’ve been slowly doing my own gui code. Its name is simpledisplay.d for the basic functions and minigui.d for the buttons, checkboxes, etc. widgets…. but I keep adding stuff that comes up. simpledisplay is now 20,000 lines and minigui, built on top of it, is now up to 12,000 lines. And I use the native controls on Windows to get some of the stuff for free! For cross platform I mostly define the apis to be async then on Windows, it just immediately calls like when drag data comes in. Works fairly ok.
It is legitimately a kinda big job. But on the other hand, I’ve been semi-successful with it all. Everyone likes to tell me it is impossible but…. I’ve been working on it on-and-off for ten years, sure, but I’m getting it done.
What is your criteria for success? Are you able to make a living off of it?
I just wanted something for myself, but I actually do use it for (some) day job work too so it kinda contributes to other things.
I was annoyed at how big and complicated the other options were and wanted to keep it simple and small… just as time went on, I wanted more and more bits so it grew. Still simpler and smaller than most the things though. And then on Linux I get to do everything my way which is slow progress but educational and gives nice programs for me.
Minor correction: Edge is crossplatform now.
Really? Other than other Windows devices?
Edit: maybe not at all any more? https://www.business-standard.com/article/technology/microsoft-ends-support-for-original-cross-platform-edge-browser-121031000318_1.html
This still offers Edge for Mac: https://www.microsoft.com/en-us/edge I think maybe they killed the version of Edge that contained an IE11 mode.