I don’t know where all those fake Scotsmen who ruin programming come from, but I frequently read articles from across software engineering (not just machine learning) that make it clear that the whole field would be more $property if only everyone knew the same things as $author.
Is the barrier of entry that low? You need to have a certain mathematical maturity to understand the papers, and most of the new results are about stuff in papers. If you can do that, what’s the big deal? More people working in an area means more research directions get to be explored. Now if you are a charlatan out fooling non-experts, how is that any different from any other field? The same thing has happened with software in general.
The comments seem to get obsessed with how need engineers more than researchers but that feels like a strawman as well. Unless your problem was already solved in a paper published at ICLR/NIPS/ICML/etc you are going to do at least a little bit of research to adapt the technology to your problem. This is going to require at least some creativity and intuition for these statistical models. You might not get a publication out of the work, but you’ll still work damn hard to get things working.
I can whip up a non-trivial ML solution in about 15 minutes using commoditized tools, like TensorFlow. That system will even work pretty well for my sample test data- well enough that I think I’ve just built a system that accurately catalogs sentiment or identifies a trait in a photograph. I can run to production with that, and get pretty good results… until I don’t.
If I need to do something more complicated, I can snag a lightweight, pop-press ML book that guides me through some of the statistical concepts. Again, I get a solution that looks good, under a cursory inspection. I can roll that package out to production and let people start consuming its results, and its flaws will manifest in subtle, difficult to detect ways.
That might not be so bad. Like I don’t think that is any worse than any other bugs that somehow only manifest in production. As long as you have good monitoring in place, I think you can get pretty far on that attitude. More cynically, a colleague once told me, “These models usually fail a little after you’ve been promoted so it’s not your problem”.
I think there are real ethical concerns. I mean, simple stuff- like that research team that put pictures as “beautiful” into their training set, and ended up creating essentially a robotic racist. A lot of our assumptions about how the world works, when encoded via machine learning, magnifies what are normally minor issues into an industrial scale.
I completely agree. But Fairness and ethics in machine learning is not something you can fix with barriers. That’s something that needs to be out there as an idea. I’m not even sure how to go about doing that.
I rather feel that deep learning marketing is a bit “too good”. As in people expect deep learning to be a silver bullet, and we all know that doesn’t exist.
To the contrary, a lot of knowledge out there about judging prediction models and improving preprocessing steps for conventional machine learning pipelines are kind of drowned out in the noise.
Another issue is the quality of training they give. I attended a tensorflow training at a conference and they only went through 3 examples in 3 hours. All that could be gotten from their basic tutorial.
I don’t know where all those fake Scotsmen who ruin programming come from, but I frequently read articles from across software engineering (not just machine learning) that make it clear that the whole field would be more $property if only everyone knew the same things as $author.
Is the barrier of entry that low? You need to have a certain mathematical maturity to understand the papers, and most of the new results are about stuff in papers. If you can do that, what’s the big deal? More people working in an area means more research directions get to be explored. Now if you are a charlatan out fooling non-experts, how is that any different from any other field? The same thing has happened with software in general.
The comments seem to get obsessed with how need engineers more than researchers but that feels like a strawman as well. Unless your problem was already solved in a paper published at ICLR/NIPS/ICML/etc you are going to do at least a little bit of research to adapt the technology to your problem. This is going to require at least some creativity and intuition for these statistical models. You might not get a publication out of the work, but you’ll still work damn hard to get things working.
I can whip up a non-trivial ML solution in about 15 minutes using commoditized tools, like TensorFlow. That system will even work pretty well for my sample test data- well enough that I think I’ve just built a system that accurately catalogs sentiment or identifies a trait in a photograph. I can run to production with that, and get pretty good results… until I don’t.
If I need to do something more complicated, I can snag a lightweight, pop-press ML book that guides me through some of the statistical concepts. Again, I get a solution that looks good, under a cursory inspection. I can roll that package out to production and let people start consuming its results, and its flaws will manifest in subtle, difficult to detect ways.
That might not be so bad. Like I don’t think that is any worse than any other bugs that somehow only manifest in production. As long as you have good monitoring in place, I think you can get pretty far on that attitude. More cynically, a colleague once told me, “These models usually fail a little after you’ve been promoted so it’s not your problem”.
I think there are real ethical concerns. I mean, simple stuff- like that research team that put pictures as “beautiful” into their training set, and ended up creating essentially a robotic racist. A lot of our assumptions about how the world works, when encoded via machine learning, magnifies what are normally minor issues into an industrial scale.
I completely agree. But Fairness and ethics in machine learning is not something you can fix with barriers. That’s something that needs to be out there as an idea. I’m not even sure how to go about doing that.
I rather feel that deep learning marketing is a bit “too good”. As in people expect deep learning to be a silver bullet, and we all know that doesn’t exist.
To the contrary, a lot of knowledge out there about judging prediction models and improving preprocessing steps for conventional machine learning pipelines are kind of drowned out in the noise.
The barrier to entry is very low, the companies pushing their tools try to make it as easy as possible to get an answer (any answer).
http://shape-of-code.coding-guidelines.com/2015/11/23/machine-learning-in-se-research-is-a-bigger-train-wreck-than-i-imagined/
Another issue is the quality of training they give. I attended a tensorflow training at a conference and they only went through 3 examples in 3 hours. All that could be gotten from their basic tutorial.
did you mean to tag this entry in the thread?