It really is a crapshoot: Conclusion

So you were probably wondering what all that (1 | 2 | 3) was about. The coin that lands heads up 55 percent of the time is clearly an abstraction. It has been concocted to help our loyal readers think about the significance of significance.

This coin is most definitely biased. The problem is that you’re going to have to toss it quite a number of times before that bias reveals itself. The coin is like a weak relationship between something unpleasant (like being promiscuous in the strictest pharmacological sense) and something you can measure (like lipophilicity). Powered by enough data even the weakest trends become significant.

To note the significance of a trend without showing the underlying variation in the trend is like simply noting that a coin is biased. In this world all biased coins are equal. These include our 55 percent coin and the one lands heads up 999 times out of 1000. So think about this the next time that you’re desperately seeking significance.

The 20-player game is a little more difficult to place in a drug discovery context but we won’t let that stop us. We know our readers are depending on us. Let’s return to that weak relationship between the unpleasant and the measurable. Allowing your chemists to make more lipophilic compounds increases the risk of it going belly up later. However, drug discovery is a risky business and the risk of choking in development is just one risk you face. You can’t win by just not choking in development. You need to get your drug market before your competitors or it needs to be better or cheaper than what has already got there. The 20 player game tells us that the risks cannot be assessed in isolation. The risks depend on what the other players are doing.