I don’t think it’s a gimmick, but it’s not really a specific thing either. As I understand it, Watson is basically a brand name for all of IBM’s AI-related services. They are trying to portray it as an integrated thing with the personified name “IBM Watson”, and there is some effort to actually integrate the bits and pieces so they can interoperate, but the level of that integration is probably less like “a smart man in the cloud” image they’re trying to give it, and closer to something like AWS, a suite of services that are being offered together, with some attempt to make them play nicely.
It is my understanding that the services don’t really integrate super well, and that it’s almost more a consulting/services deal than a tangible product.
It is also my experience that in medicine, at big cancer centers like MD Anderson, Watson is just another bit of whiz-bang technology used to signal how awesome the hospital system is to other hospital systems and doctors. It’s basically a prop used for dick-waving and dressing up so the medicos can pretend they are serious researchers.
I think we’re in a better position now to actually use such systems, but more because of social and general-technology changes than AI ones. There has been AI progress, but the bigger impediment was that even in areas of medical diagnosis where AI did work decades ago it was never deployed, because either people were very uncomfortable with the idea, or there were all kinds of practical questions about how you’d use it: would there be a computer in every doctor’s office, who’d maintain the software, etc. There are still some questions around that kind of thing, but doctors using computers as part of their work has gotten more common, even for routine things like scheduling and patient records, which it wasn’t in previous eras.
One example that’s interesting to me is the late-70s system MYCIN, which had good results diagnosing specific types of blood infections. It worked well, actually beating most human doctors, because: 1) there was well-codified knowledge about how such infections should be diagnosed, essentially a few hundred rules covers the vast majority of cases, but 2) applying those rules accurately requires using conditional probabilities, which most people (even doctors) are fairly bad at, especially past two or three variables, whereas computers are very good at this. But the system never got deployed anywhere.
They generated those rules through clustering IIRC. Not sure how far such methods would scale if we bring in more conditions but there’s research on that now with ML techniques. I remember MYCIN being one of my early inspirations during my short stint messing with AI in late 90’s to early 2000’s. My guess is that the tool was too specific and manual. Just imagine if we had one for about everything a doctor might encounter that was similarly hard to diagnose for humans.
An ideal one for this sort of thing might get information from nurses, doctors, or tests on all patients then run MYCIN-like analysis on them all in background to notify someone if a high-probability candidate comes up. It could run it for meningitis or any number of other things. It wouldn’t surprise me if these already exist but this was my proposal way back then. We just didn’t have computing power, algorithms & labor necessary.
If I had to guess where it’d be most likely to get something fairly specific like this deployed, it’d be on the analysis-lab rather than doctor’s side. You send off sample to the lab, it runs some standard analysis and comes back with information. Traditionally that information was very low-level measurements of chemical properties. But I believe they have already started including higher-level information like “tests positive for [marker X]”, i.e. running the low-level data through a set of rules to annotate whether the sample matched some well-known markers or not, rather than leaving it purely for the doctor to interpret. MYCIN is not much more than a fairly large set of those kinds of markers. Although there is a bit of a social difference in that, afaik, the current lab annotations stop short (probably deliberately so) of claiming to perform “diagnosis”.
Good guess as I did find some papers doing stuff like that with a quick Google. However, the bigger find was a thorough treatment on the problems of getting it right by experts from various perspectives. I decided it was worth a submission:
It’s still only doing one narrow function that Prolog was good at. Here’s a comment from Hacker News where I told a Prolog, AI user why I didn’t think it was good in general based on prior AI uses:
“That most AI problems deal with uncertainty that’s harder to deal with in first-order logic. You gave a good example with NLP where the early stuff I looked at tried logic approaches to find they fell flat due to context & language’s probabilistic nature. All kinds of things in game AI turned out that way. Even some stuff that would seem true or false, like "am I being attacked?”, wasn’t so clear when bluffing was involved. I couldn’t imagine how to handle a Poker game in Prolog with any success. That’s significant given a combo of math, human BS, and luck it brings is common in many AI problems. For these reasons, people were moving from first-order logic even in my day to fuzzy logic, probabilistic models, neural networks, and machine learning techniques."
Still true. Anything Prolog can do well I can do with a DSL or library on top of better language. I know Allegro Common LISP still incorporates Prolog engine as such. Racket Scheme has one too. Leads to another benefit of avoiding Prolog for something like LISP: can use DSL’s that match each part of AI system best, even auto-generating 3GL code for acceleration (C/C++) or 3rd party libs (eg R lang).
Bad article, little content, and is niche news.
Oh, and Watson is a stupid marketing gimmick being pushed on everyone.
This is not my understanding. Can you elaborate?
I don’t think it’s a gimmick, but it’s not really a specific thing either. As I understand it, Watson is basically a brand name for all of IBM’s AI-related services. They are trying to portray it as an integrated thing with the personified name “IBM Watson”, and there is some effort to actually integrate the bits and pieces so they can interoperate, but the level of that integration is probably less like “a smart man in the cloud” image they’re trying to give it, and closer to something like AWS, a suite of services that are being offered together, with some attempt to make them play nicely.
It is my understanding that the services don’t really integrate super well, and that it’s almost more a consulting/services deal than a tangible product.
It is also my experience that in medicine, at big cancer centers like MD Anderson, Watson is just another bit of whiz-bang technology used to signal how awesome the hospital system is to other hospital systems and doctors. It’s basically a prop used for dick-waving and dressing up so the medicos can pretend they are serious researchers.
[Comment removed by author]
I think we’re in a better position now to actually use such systems, but more because of social and general-technology changes than AI ones. There has been AI progress, but the bigger impediment was that even in areas of medical diagnosis where AI did work decades ago it was never deployed, because either people were very uncomfortable with the idea, or there were all kinds of practical questions about how you’d use it: would there be a computer in every doctor’s office, who’d maintain the software, etc. There are still some questions around that kind of thing, but doctors using computers as part of their work has gotten more common, even for routine things like scheduling and patient records, which it wasn’t in previous eras.
One example that’s interesting to me is the late-70s system MYCIN, which had good results diagnosing specific types of blood infections. It worked well, actually beating most human doctors, because: 1) there was well-codified knowledge about how such infections should be diagnosed, essentially a few hundred rules covers the vast majority of cases, but 2) applying those rules accurately requires using conditional probabilities, which most people (even doctors) are fairly bad at, especially past two or three variables, whereas computers are very good at this. But the system never got deployed anywhere.
“But the system never got deployed anywhere.”
They generated those rules through clustering IIRC. Not sure how far such methods would scale if we bring in more conditions but there’s research on that now with ML techniques. I remember MYCIN being one of my early inspirations during my short stint messing with AI in late 90’s to early 2000’s. My guess is that the tool was too specific and manual. Just imagine if we had one for about everything a doctor might encounter that was similarly hard to diagnose for humans.
An ideal one for this sort of thing might get information from nurses, doctors, or tests on all patients then run MYCIN-like analysis on them all in background to notify someone if a high-probability candidate comes up. It could run it for meningitis or any number of other things. It wouldn’t surprise me if these already exist but this was my proposal way back then. We just didn’t have computing power, algorithms & labor necessary.
If I had to guess where it’d be most likely to get something fairly specific like this deployed, it’d be on the analysis-lab rather than doctor’s side. You send off sample to the lab, it runs some standard analysis and comes back with information. Traditionally that information was very low-level measurements of chemical properties. But I believe they have already started including higher-level information like “tests positive for [marker X]”, i.e. running the low-level data through a set of rules to annotate whether the sample matched some well-known markers or not, rather than leaving it purely for the doctor to interpret. MYCIN is not much more than a fairly large set of those kinds of markers. Although there is a bit of a social difference in that, afaik, the current lab annotations stop short (probably deliberately so) of claiming to perform “diagnosis”.
Good guess as I did find some papers doing stuff like that with a quick Google. However, the bigger find was a thorough treatment on the problems of getting it right by experts from various perspectives. I decided it was worth a submission:
https://lobste.rs/s/mojiqe/why_is_machine_learning_not_more_widely
It’s still only doing one narrow function that Prolog was good at. Here’s a comment from Hacker News where I told a Prolog, AI user why I didn’t think it was good in general based on prior AI uses:
“That most AI problems deal with uncertainty that’s harder to deal with in first-order logic. You gave a good example with NLP where the early stuff I looked at tried logic approaches to find they fell flat due to context & language’s probabilistic nature. All kinds of things in game AI turned out that way. Even some stuff that would seem true or false, like "am I being attacked?”, wasn’t so clear when bluffing was involved. I couldn’t imagine how to handle a Poker game in Prolog with any success. That’s significant given a combo of math, human BS, and luck it brings is common in many AI problems. For these reasons, people were moving from first-order logic even in my day to fuzzy logic, probabilistic models, neural networks, and machine learning techniques."
Still true. Anything Prolog can do well I can do with a DSL or library on top of better language. I know Allegro Common LISP still incorporates Prolog engine as such. Racket Scheme has one too. Leads to another benefit of avoiding Prolog for something like LISP: can use DSL’s that match each part of AI system best, even auto-generating 3GL code for acceleration (C/C++) or 3rd party libs (eg R lang).