Science Isn’t Broken

If you’ve seen the (relative) deluge of journal retractions over the past couple of years, you’re probably wondering why science is broken. FiveThirtyEight is here to tell you that it’s not, explaining the peer review process and the various ways in which it works or doesn’t.

Of particular note are the issues concerning either purposeful statistical manipulation of data or just plain old confirmation bias:

If you tweaked the variables until you proved that Democrats are good for the economy, congrats; go vote for Hillary Clinton with a sense of purpose. But don’t go bragging about that to your friends. You could have proved the same for Republicans.

The data in our interactive tool can be narrowed and expanded (p-hacked) to make either hypothesis appear correct. That’s because answering even a simple scientific question — which party is correlated with economic success — requires lots of choices that can shape the results.

This is interesting from a startup perspective as well. Traction is one of those nebulous things that isn’t all that well defined and can be shaped to fit the ways in which your company is doing well while ignoring the ways in which it might be broken.

I do wonder how much confirmation bias and statistical “tweaks” make it into investment decks - my suspicion is quite a lot - and does anyone ever take a closer look at that data before investing?

Startup-talk aside, it’s an interesting article all by itself. You can see my Highly summary as well.


Wed, Aug 19, 2015