I believe that there’s a very real chance that a reëvaluation of scientific research — particularly the social sciences, which are so soft as to border on pseudoscience to begin with — in light of this new understanding is very likely to yield unexpected & revolutionary results.
The social sciences are not “so soft as to border on pseudoscience”. Yes P-Hacking and over reliance on p-values are an issue, but they’re an issue unilaterally. It matters just as much in physics as it does in social science. The idea that social sciences are somehow less rigorous than say physics is I think fallacious and without merit. Physics is still ultimately rooted in experimental evidence which is statistical in nature. You realize you’re discrediting an entire set scientific fields with literally no evidence and likely with very little knowledge of the subject. It’s not like there’s been literally no replication in the social sciences. What an absolutely absurd claim.
I have to agree with @bargap here. Physics is a poor example to hold up against the social sciences. Physics results require a level of significance that is mind boggling. Of course those piddly experimentalists do get things wrong sometimes. But a better comparison is perhaps life sciences. The basic problem is that the system is so complex that it is hard to figure out all the moving parts and make sure you’ve bolted down all but a few.
There is a running joke about “Doctors recommend … “ (take your pick - babies should sleep on their back/belly, this food is bad/good for you, this causes/cures cancer, this causes/does not cause autism …) precisely because we’ve been burned so many times by studies that just did not have enough N, or more insidiously (and forgivably), had a biased N.
Social sciences study PersONS which are much more complex than protONS or electrONS but their tooling and standards are, paradoxically, lower, not higher and society seems to have an expectation of what the right answer should be, allowing social biases to play a disproportionate role in biasing reports in these fields.
If you look in spaces like chemistry or spintronics you’ll find a lot of papers that people end up not being able to reproduce because it turned out that the machine that the writers were using had issues.
In situations where there are only 5 or so machines in the world that can do an experiment, if 1 of them has an issue then it’s not always easy to validate. Especially when the research methodology is “cartesian product of every material with every technique => see if something happens”. And even absent that, people’s interpretations of measurements change over time too!
Agree that people are harder to study, especially in long term stuff. But we don’t do many physics experiments over 20 years either! It’s mainly that time is hard, and preconditions are hard to set up. You can totally do experiments in social sciences over a small time scale correctly.
Instead of comparing the foundations of physics to the edges of social sciences you should be considering the edges of physics, after all it doesn’t make sense to compare things which have a thousand years of replication to ones with tens of years. We’re still not sure that dark matter actually exists. Classical bias risk is pervasive in theoretical physics and is a constant threat.
The analogy is instead saying “Medical science borders on pseudoscience, we should cast doubt on the entire field including claims such as “Arsenic is poisonous” because some of the newer claims on cancer haven’t yet been replicated”.
To be entirely clear you haven’t been burned by science, you’ve been burned by irresponsible reporting which has confidence from a single study.
The concession is that there hasn’t been enough replication but a recent study reported on by ars which I don’t have on hand right now says that the replication rate of social science studies were around 70%, which is yeah not great but it’s hardly pseudoscience.
To be entirely clear you haven’t been burned by science, you’ve been burned by irresponsible reporting which has confidence from a single study.
That’s a classic defense. “I never said that! They blew the press release all out of proportion”. Sometimes it is true.
I know of many papers where inferences and conclusions are not warranted by the data and have seen this in different fields of study. I infer that this is common human behavior to game a the system that rewards visibility and volume.
You have conveniently dodged the main premise of my argument. Sure I’ll concede there are plenty of bad actors, but they aren’t magically condensed in the social sciences.
The social sciences are definitely one of the areas where bad studies are harder to root out because of lower standards of rigor combined with the complexity of the subject and the fact that “truthiness” is easier to determine and therefore forms an extra social pressure not to challenge a particular study if it conforms to current social biases.
I don’t think its actually reasonable to say that the newest physics are any more rigorous, any less complex, or any less rife with bias. They both use studies with statistical evidence so they both have a lot of rigor but it’s hard to beat replication. They both are very very complex. They both have lots and lots of human bias, from a bias towards classicalism, conservativism, anti-classicalism and modernism. That doesn’t mean that either of them are anywhere near pseudoscience. The claim is just uninformed.
I believe that there’s a very real chance that a reëvaluation of scientific research — particularly the social sciences, which are so soft as to border on pseudoscience to begin with — in light of this new understanding is very likely to yield unexpected & revolutionary results.
The social sciences are not “so soft as to border on pseudoscience”. Yes P-Hacking and over reliance on p-values are an issue, but they’re an issue unilaterally. It matters just as much in physics as it does in social science. The idea that social sciences are somehow less rigorous than say physics is I think fallacious and without merit. Physics is still ultimately rooted in experimental evidence which is statistical in nature. You realize you’re discrediting an entire set scientific fields with literally no evidence and likely with very little knowledge of the subject. It’s not like there’s been literally no replication in the social sciences. What an absolutely absurd claim.
I have to agree with @bargap here. Physics is a poor example to hold up against the social sciences. Physics results require a level of significance that is mind boggling. Of course those piddly experimentalists do get things wrong sometimes. But a better comparison is perhaps life sciences. The basic problem is that the system is so complex that it is hard to figure out all the moving parts and make sure you’ve bolted down all but a few.
There is a running joke about “Doctors recommend … “ (take your pick - babies should sleep on their back/belly, this food is bad/good for you, this causes/cures cancer, this causes/does not cause autism …) precisely because we’ve been burned so many times by studies that just did not have enough N, or more insidiously (and forgivably), had a biased N.
Social sciences study PersONS which are much more complex than protONS or electrONS but their tooling and standards are, paradoxically, lower, not higher and society seems to have an expectation of what the right answer should be, allowing social biases to play a disproportionate role in biasing reports in these fields.
If you look in spaces like chemistry or spintronics you’ll find a lot of papers that people end up not being able to reproduce because it turned out that the machine that the writers were using had issues.
In situations where there are only 5 or so machines in the world that can do an experiment, if 1 of them has an issue then it’s not always easy to validate. Especially when the research methodology is “cartesian product of every material with every technique => see if something happens”. And even absent that, people’s interpretations of measurements change over time too!
Agree that people are harder to study, especially in long term stuff. But we don’t do many physics experiments over 20 years either! It’s mainly that time is hard, and preconditions are hard to set up. You can totally do experiments in social sciences over a small time scale correctly.
Instead of comparing the foundations of physics to the edges of social sciences you should be considering the edges of physics, after all it doesn’t make sense to compare things which have a thousand years of replication to ones with tens of years. We’re still not sure that dark matter actually exists. Classical bias risk is pervasive in theoretical physics and is a constant threat.
The analogy is instead saying “Medical science borders on pseudoscience, we should cast doubt on the entire field including claims such as “Arsenic is poisonous” because some of the newer claims on cancer haven’t yet been replicated”.
To be entirely clear you haven’t been burned by science, you’ve been burned by irresponsible reporting which has confidence from a single study.
This article is also relevant here https://news.harvard.edu/gazette/story/2016/03/study-that-undercut-psych-research-got-it-wrong/
The concession is that there hasn’t been enough replication but a recent study reported on by ars which I don’t have on hand right now says that the replication rate of social science studies were around 70%, which is yeah not great but it’s hardly pseudoscience.
That’s a classic defense. “I never said that! They blew the press release all out of proportion”. Sometimes it is true.
I know of many papers where inferences and conclusions are not warranted by the data and have seen this in different fields of study. I infer that this is common human behavior to game a the system that rewards visibility and volume.
You have conveniently dodged the main premise of my argument. Sure I’ll concede there are plenty of bad actors, but they aren’t magically condensed in the social sciences.
The social sciences are definitely one of the areas where bad studies are harder to root out because of lower standards of rigor combined with the complexity of the subject and the fact that “truthiness” is easier to determine and therefore forms an extra social pressure not to challenge a particular study if it conforms to current social biases.
I don’t think its actually reasonable to say that the newest physics are any more rigorous, any less complex, or any less rife with bias. They both use studies with statistical evidence so they both have a lot of rigor but it’s hard to beat replication. They both are very very complex. They both have lots and lots of human bias, from a bias towards classicalism, conservativism, anti-classicalism and modernism. That doesn’t mean that either of them are anywhere near pseudoscience. The claim is just uninformed.