The Newton had a bunch of features that I still miss from modern systems. The two that most spring to mind:
The drawing app had really good shape recognition. The MS Whiteboard app is now maybe 60% as good on a CPU at least an order of magnitude more powerful. If you drew a square, a circle, a line between them, and wrote in the square, you’d end up with vector shapes that connected properly (so dragging the square or circle would keep the line / arrow attached and where the text was associated with the shape that you drew in).
The other thing is copy and paste. It annoyed me that the iPhone launched without this feature when the Newton was the only small-screen device to do it well. You could select something on the Newton and drag it to the edge of the screen. You’d then get a little tab containing it as a clipping. You could then switch to another app and drag it back. Direct manipulation everywhere, no abstract clipboard idea for people to understand, and an interface that scaled trivially to multiple concurrent clipboards. I wish newer hand-held devices would do this instead of their current clumsy copy-and-paste (which only works reliably for text, just like ’80s-era X11).
It’s sort of remarkable how bad input has been on mobile devices without triggering any kind of competition/experiments to make it better.
There used to be a great keyboard on Android called 8pen which was definitely in the “experiment” category. It’s a bit hard to describe, but the gist was:
The end result was that your finger stayed in fairly rapid, fairly fluid motion, and only stopped at the end of a word. It definitely required learning, but it was remarkably easy (and forgiving–the quadrants were all large, and there was optional haptic feedback every time you crossed a boundary) once you knew it. I could routinely type full, correct text messages without actually looking at my phone. I used it as my keyboard for years, and loved it.
Sadly, the company that made it shuttered, and it’s no longer available.
edited to add: Apparently in the last couple years somebody made an open-source clone called 8vim. I am delighted, and encourage everyone to try it out.
The Newton OS was particularly brave in jettisoning the filesystem concept, which nothing else seems to have managed since (except devices too tiny to have one.)
Soups were shared between apps, letting them access each others’ data, so your contacts and calendar and to-do list were all integrated and any app could access them without needing custom APIs. (Nowadays we’d leap to the security/privacy issues that entails, yeah, but still a worthwhile concept.)
I thought I might be interviewed for this article, but wasn’t — but it’s remarkably accurate anyway! :)
It’s fascinating how the KEY feature of the Newton and all the other handhelds of the period was cutting-edge handwriting recognition, and it turned out we just needed better on-screen QWERTY keyboards.
I would argue that we are still waiting for better on-screen keyboards…the point and tap approach leaves a lot to be desired.
an on-screen keyboard that could be configured to your hands, and provide haptic feedback would be great. I think that the Newton was remarkable in the fact that it could do handwriting recognition considering the limitations of the hardware.
I think one of the keys to Palm’s success was that they didn’t try for cutting-edge handwriting recognition. Instead, users used Graffiti for input. That let them avoid needing an on-screen keyboard while also giving them an out on handwriting recognition. It meant smaller, more capable devices, at the cost of users learning a new input type.
BlackBerry showed up around the turn of the century and started to slowly nudge Palm out of the top spot. Much of that was better integration with office software suites, but a nontrivial part was the keyboard. It was physical, and tiny (though still far dominating the profile of the device), but didn’t have the learning curve of Graffiti.
And then the iPhone showed up with its on-screen keyboard, and the game was up.
Still, there’s a definite market for devices that recognize handwriting. The reMarkable devices come to mind, but there are others, too. And the handwriting recognition has definitely improved over 1993.
IIRC, Graffiti was a third-party package that Palm bought or licensed; I think it was also available on the Newton, but that could just be brainworms at this point.
I miss my Newton. I loved it unreservedly.
Palm developed it, but I believe they developed it for other devices. That is, Palm started out as a software company, and it was a few years before they released their own PDA (the original PalmPilot). So Graffiti was available for the Newton before it was on their own hardware. Also several other portable devices that didn’t have the same level of success.
Someone probably still owns the Graffiti IP (I think Palm was acquired by HP, so maybe them?), so it’s possible they could release Graffiti as a keyboard for Android or iPhone.
This is broadly the case.
I hope that my attempt to précis the history won’t get the details too wrong.
• NewtonOS 1.0 needed to learn your handwriting. It was thus impossible to demo in-store; it needed a week or so to get to know you. But the vision was there and it was very impressive.
• Jeff Hawkins saw a way to fix this and launched his own simplified handwriting-entry system. You need to learn to type, so why not re-learn how to write? Simplified single-stroke letters, one on top of the other, in a defined area with zones and hotspots for capitals and numbers so that the app didn’t have to guess. Don’t make the device learn the human’s ways, as they are bad at that: make the human learn the device’s ways, as humans are good at that.
• Graffiti 1 launched as a Newton app.
• Hawkins realises that the Newt is too big and ambitious. He feels that what people really need is a shirt-pocket sized device, that runs for weeks on a few AAA cells. Enter your data on a PC or Mac, sync it very easily to the pocket device: plug in, press 1 button, done. Only make minor edits on the device.
• He prototypes the device as a block of wood to get the size right. He buys in an OS kernel, he and a small team wrap a simple GUI around it with the existing Graffiti as the sole text-entry system.
• Result: Palm Pilot. Big hit.
• NewtonOS 2.0 can read hand-printed letters, i.e. non-cursive. No learning needed. This gets around the need-to-learn-the-owner’s-writing problem. Print at first, write in longhand when you have time to teach it. But it’s still big and unwieldy, while the Palm range are tiny. They only do 20% of the functionality but it’s the important 20%.
• Palm’s IPO hits problems when Xerox sues over Graffiti. Palm loses.
• Palm intros Graffiti 2 but you have to relearn it again.
• Ruling overturned but it’s getting late.
[Links are from my employer but from long before my time.]