It’s time to get stuck back into Solubility Forecast Index (SFI) although we do admit to lacking enthusiasm for this. Earlier, we tried find out whether too many aromatic rings are a liability in drug design and were greatly amused to learn that one of the authors of that offering has asked whether medicinal chemists should do molecular modelling. Perhaps ‘should medicinal chemistry design experts do data analysis?’ would have been a good subtitle for our previous blog post.
Let’s get straight to business. We’d like you to take a look at Figure 8 which is described as a "pie chart matrix representation of solubility category as a function of ACD log DpH7.4 and aromatic ring count". The first point that we’d like you take note of is that the distribution of data is not nearly as even as this figure suggests. Have a look at the number on top of each pie chart to see how many data points were used to generate each pie chart and you’ll see what we’re getting at.
You’ll notice Figure 8 has a line drawn on it and this line defines SFI. We’ll leave to the authors to explain:
“This compounding effect is further emphasized by the diagonal split between regions of predominantly high and low solubility in Figure 8. In fact, the division between these regions can reasonably be described by the diagonal of c log D pH7.4 + #Ar = 5”
Now when you see a line drawn like this, you should be asking about how the line was positioned. There are techniques (discriminant analysis | support vector machine) that allow you to find the best split between regions and some of these have been around for a very long time even if the authors choose to ignore them. If we were going to use something like SFI for making decisions, we’d want to make sure that it actually represented the best split between the high and low solubility regions. As far as we can tell, the authors have drawn the line on the picture (quite possibly using Powerpoint). They go on to state that:
“It is thus implicit that each aromatic ring in a molecule has a solubility penalty equivalent to an extra log unit of hydrophobicity in addition to its intrinsic hydrophobicity value”
This is not exactly profound given how the authors have chosen to define SFI. It really couldn’t be any other way.
However, there is another issue that we highlighted when looking at the earlier study by colleagues of the authors. For pharmaceutical compound collections, the number of aromatic rings (#Ar) is likely to be correlated with molecular weight (MW) and a number of other measures of molecular size. What would have happened if the authors had done a similar analysis using MW or number of non-hydrogen atoms instead of #Ar? By focussing on #Ar and c log D pH7.4 the authors are saying that, provided that you control these properties, you don’t need to worry about molecular weight. We’re far from convinced that using #Ar rather than MW actually leads to a better assessment of solubility. Even if the authors had considered asking the question, answering it would have needed data-analytic capability far beyond what has been demonstrated in this article.
Well now it’s time to introduce you the Golden Triangle. In the context of this discussion this can be loosely described as druglike region which should not be confused with another druglike region where Thailand, Burma and Laos come into close proximity with each other.
The focus of the Golden Triangle is oral bioavailability (using in vitro permeability and metabolic stability as surrogates) and the properties used to define it are MW and logD. Readers will note the analogy between the line that defines SFI and what we will call the ‘right hand edge’ of triangle. However, the creators of the Golden Triangle also looked at the problems (low permeability) associated with low lipophilicity which gives the triangle its ‘left hand edge’. We must admit to being less than keen on the Golden Triangle but at least its creators scale their pie charts according to the number of data points that each summarises. We really would prefer that people just took their data and analysed it properly instead of fiddling around with pie chart matrices and waffling about clearer stepped differentiation within bands. However, if you really feel that you must powerpoint lines onto pictures and pass it off as data analysis then it’s probably a good idea to scale the pie charts first.
So that’s our take on SFI. While we believe that molecular size contributes adversely to solubility, we didn’t have much of a feel for the strength of the trend before reading the SFI article. We still don’t.
Wednesday, March 14, 2012
Thursday, February 9, 2012
By the pricking of my thumbs, something aromatic this way comes...
So we accepted an invitation to get physical in drug discovery although we are not sure that this was such a good idea. However, before we can return to the Solubility Forecast Index (SFI) we first need to look at an earlier study by colleagues of SFI’s creators to get a feel for the origins for the focus on aromatic rings. Following the relevant citation in ‘getting physical’ we aimed to find out whether too many aromatic rings are a liability in drug design. Why don’t you join us as we take a tour through the article.
One of the first things that you’ll notice is that one of the authors is described as ‘a medicinal chemistry design expert’ but don’t let this intimidate you because you’ve already learned how to deal with experts. The authors state in the abstract:
“On the basis of this analysis, it was concluded that the fewer aromatic rings contained in an oral drug candidate, the more developable that candidate is probably to be; in addition, more than three aromatic rings in a molecule correlates with poorer compound developability and, thus, an increased risk of attrition in development.”
It’s going to be interesting to see how they quantify ‘developable’ so let’s read on!
We’d like to start by looking at Figure 2 which shows a box plot of measured solubility and aromatic ring count and you might want to check out Figure 1 if you’re not clear about what all the funky graphics mean. We certainly agree that solubility decreases as the number of aromatic rings increases although it is difficult with a plot like this to tell exactly how strong the trend is. When presenting data using a graphic like this, it’s a good idea to arrange things so that the variance in each box is as constant as possible and plotting solubility logarithmically is likely to help in this regard. Transforming the data in this manner will also help with the skewness of the distributions in the boxes such as the one corresponding to 5 aromatic rings for which the mean is actually greater than Q3. However, we don’t want to dwell on this, or the analysis of albumin binding in Figure 7, too much because it is clear from a subsequent publication that these authors have already seen the error of their ways on this particular issue and we really want to move on.
Let’s take a quick look at Figure 3 which shows (yet another) another box plot and six pie charts that get redder (and less green) as the number of aromatic rings increases. The green bits represent the proportion of compounds with ClogP < 3 and the red bits the proportion of compounds with ClogP > 3 so the pie charts show what the box plot shows which is that ClogP increases with the number of aromatic rings. The authors assert an excellent correlation between lipophilicity and aromatic ring count although their reluctance to quantify this with a correlation coefficient should set some alarm bells ringing. They also note that “the addition of an aromatic ring usually results in a discrete and statistically significant jump in c log P” which certainly confused us since we are unaware of what makes some jumps discrete and others otherwise. We recalled the “clearer stepped differentiation within the bands” from SFI and wondered whether anybody where these chaps work ever uses correlation coefficients. A picture may indeed be worth a thousand words but surely one correlation coefficient is worth more than six pie charts.
So let’s move on. We’d like you to take a look at Figure 5. Looks familiar, doesn’t it? Well here’s where you’ve seen it before. Don’t correlations look so much better when you’ve hidden the variation! As loyal and cultured readers of this column, you will deafened by the cacophony of warning bells whenever you see data presented in this manner.
However, there is a more serious problem with all of this. The number of aromatic rings is a measure of molecular size and, for the compounds in pharmaceutical databases, it is likely to be correlated with other measures of molecular size such as molecular weight, number of non-hydrogen atoms, molecular volume and molecular surface area. Now you can see the problem. If the authors had selected one of these other properties and found a correlation, we would be discussing an article entitled, “The impact of molecular volume on compound developability...” and the Great Molecular Crapshoot would be haranguing them for not checking for the influence of aromatic ring count. If you’re going to assert that aromatic ring count is somehow special then you’re effectively saying that if we control the number of aromatic rings we can do whatever we want with molecular weight. There are things that one can do to investigate whether aromatic ring count is doing more than just contributing to molecular size but these would require a data-analytic capability beyond that which the authors have demonstrated here. It would all be less of a problem if the correlations between aromatic ring count and properties like solubility were very strong. However, if the correlations were indeed strong, we suspect that the authors would have been quoting some numbers rather than waffling about discrete and statistically significant jumps.
So where does this analysis lead? They authors suggest the following mnemonic for oral drug discovery programs:
“The fewer the number of aromatic rings contained in an oral drug candidate, the more developable that candidate is likely to be; specifically, more than three aromatic rings in a molecule correlates with poorer compound developability and, therefore, an increased risk of compound attrition.”
Reading this, we couldn’t help thinking that the tortured grammatical construction of this mnemonic appeared to be somewhat at odds with its being described as mnemonic. We asked ourselves how we might use this mnemonic in real life Drug Discovery. Should we still worry about tiresome details like lipophilicity and molecular weight if the compound has three or less aromatic rings? What should we do when the compound with three aromatic rings is actually less soluble than the one with four? Is the mnemonic relevant with target-related attrition? How is developability defined and how does it depend at all on the ability of the compound to hit the target? What were the reviewers of this manuscript smoking when they let it through?
Those of you still reading this piece are probably thinking that we’re being a bit harsh with all this criticism. Couldn’t you be a bit more constructive, we hear you cry and, Loyal Readers, we concede that you may have a point. We think this whole developability business needs to be more physical. In other words we need more equations and more physics and so we have devised a new descriptor to do precisely that. We propose that we use a count of the number of neutrons (which we propose calling Nn) in a molecule as a measure of its developability and eagerly await a run on the lighter isotopes as the Pharma companies dig themselves into their patent bunkers. What could be more physical than neutrons?
That’s where we’ll leave it for now but don’t go too far because we will soon be returning to the Solubility Forecast Index...
One of the first things that you’ll notice is that one of the authors is described as ‘a medicinal chemistry design expert’ but don’t let this intimidate you because you’ve already learned how to deal with experts. The authors state in the abstract:
“On the basis of this analysis, it was concluded that the fewer aromatic rings contained in an oral drug candidate, the more developable that candidate is probably to be; in addition, more than three aromatic rings in a molecule correlates with poorer compound developability and, thus, an increased risk of attrition in development.”
It’s going to be interesting to see how they quantify ‘developable’ so let’s read on!
We’d like to start by looking at Figure 2 which shows a box plot of measured solubility and aromatic ring count and you might want to check out Figure 1 if you’re not clear about what all the funky graphics mean. We certainly agree that solubility decreases as the number of aromatic rings increases although it is difficult with a plot like this to tell exactly how strong the trend is. When presenting data using a graphic like this, it’s a good idea to arrange things so that the variance in each box is as constant as possible and plotting solubility logarithmically is likely to help in this regard. Transforming the data in this manner will also help with the skewness of the distributions in the boxes such as the one corresponding to 5 aromatic rings for which the mean is actually greater than Q3. However, we don’t want to dwell on this, or the analysis of albumin binding in Figure 7, too much because it is clear from a subsequent publication that these authors have already seen the error of their ways on this particular issue and we really want to move on.
Let’s take a quick look at Figure 3 which shows (yet another) another box plot and six pie charts that get redder (and less green) as the number of aromatic rings increases. The green bits represent the proportion of compounds with ClogP < 3 and the red bits the proportion of compounds with ClogP > 3 so the pie charts show what the box plot shows which is that ClogP increases with the number of aromatic rings. The authors assert an excellent correlation between lipophilicity and aromatic ring count although their reluctance to quantify this with a correlation coefficient should set some alarm bells ringing. They also note that “the addition of an aromatic ring usually results in a discrete and statistically significant jump in c log P” which certainly confused us since we are unaware of what makes some jumps discrete and others otherwise. We recalled the “clearer stepped differentiation within the bands” from SFI and wondered whether anybody where these chaps work ever uses correlation coefficients. A picture may indeed be worth a thousand words but surely one correlation coefficient is worth more than six pie charts.
So let’s move on. We’d like you to take a look at Figure 5. Looks familiar, doesn’t it? Well here’s where you’ve seen it before. Don’t correlations look so much better when you’ve hidden the variation! As loyal and cultured readers of this column, you will deafened by the cacophony of warning bells whenever you see data presented in this manner.
However, there is a more serious problem with all of this. The number of aromatic rings is a measure of molecular size and, for the compounds in pharmaceutical databases, it is likely to be correlated with other measures of molecular size such as molecular weight, number of non-hydrogen atoms, molecular volume and molecular surface area. Now you can see the problem. If the authors had selected one of these other properties and found a correlation, we would be discussing an article entitled, “The impact of molecular volume on compound developability...” and the Great Molecular Crapshoot would be haranguing them for not checking for the influence of aromatic ring count. If you’re going to assert that aromatic ring count is somehow special then you’re effectively saying that if we control the number of aromatic rings we can do whatever we want with molecular weight. There are things that one can do to investigate whether aromatic ring count is doing more than just contributing to molecular size but these would require a data-analytic capability beyond that which the authors have demonstrated here. It would all be less of a problem if the correlations between aromatic ring count and properties like solubility were very strong. However, if the correlations were indeed strong, we suspect that the authors would have been quoting some numbers rather than waffling about discrete and statistically significant jumps.
So where does this analysis lead? They authors suggest the following mnemonic for oral drug discovery programs:
“The fewer the number of aromatic rings contained in an oral drug candidate, the more developable that candidate is likely to be; specifically, more than three aromatic rings in a molecule correlates with poorer compound developability and, therefore, an increased risk of compound attrition.”
Reading this, we couldn’t help thinking that the tortured grammatical construction of this mnemonic appeared to be somewhat at odds with its being described as mnemonic. We asked ourselves how we might use this mnemonic in real life Drug Discovery. Should we still worry about tiresome details like lipophilicity and molecular weight if the compound has three or less aromatic rings? What should we do when the compound with three aromatic rings is actually less soluble than the one with four? Is the mnemonic relevant with target-related attrition? How is developability defined and how does it depend at all on the ability of the compound to hit the target? What were the reviewers of this manuscript smoking when they let it through?
Those of you still reading this piece are probably thinking that we’re being a bit harsh with all this criticism. Couldn’t you be a bit more constructive, we hear you cry and, Loyal Readers, we concede that you may have a point. We think this whole developability business needs to be more physical. In other words we need more equations and more physics and so we have devised a new descriptor to do precisely that. We propose that we use a count of the number of neutrons (which we propose calling Nn) in a molecule as a measure of its developability and eagerly await a run on the lighter isotopes as the Pharma companies dig themselves into their patent bunkers. What could be more physical than neutrons?
That’s where we’ll leave it for now but don’t go too far because we will soon be returning to the Solubility Forecast Index...
Friday, January 27, 2012
Solubility forecast index awarded 5.7 for artistic expression...
We hope that you enjoyed the recent Primer on Lipophilicity and found reading it to be edifying and and educational. Although it really is an honour to write pieces like that one for such clever, cultured readers, we do need to return to the style with which our loyal readers associate us. The article that today blunders into the cross hairs bills itself as a contemporary perspective on solubility and hydrophobicity although we wonder if its authors truly get physical in drug discovery. The article, as you might guess from the title, explores relationships between solubility and logP or logD and it is instructive to read what the authors have to say about their data-analytic philosophy:
“Data plots with lines of best fit and unity gave a representation of the data, albeit with a statistical analysis, which did not adequately convey the distribution of data because of the large numbers. The distribution of values was better conveyed through normalized bar graphs and box plots using binned hydrophobicity and/or solubility values, which better represent the distribution of data in a more visually amenable manner.”
To paraphrase: We couldn’t find what we wanted to when we analysed the data so we drew some pictures instead.
OK, this assessment may seem harsh and we do admit that plotting data is certainly a good thing, especially as a precursor to analysis. However, we have shown you previously that weak trends can be made to look a whole heap stronger by hiding or masking variation and when you plot data enough you can end up seeing what you think should be there. Also, if you’ve got enough data then even the weakest trend becomes significant and we respectfully draw the attention of our readers to the tale (as opposed to the tail) of the 55% coin. When presenting trends, it’s really important to remember a trend’s strength is even more important than its mere existence.
So let’s get back to business and we’d like you to take a look at Figures 6a and 6b which illustrate the relationships between aqueous solubility and two different calculated lipophilicities, namely logP and logDpH7.4 that have been predicted using the ACD software. Solubility is ‘quantified’ as a series of bars that indicate the relative proportions of compounds in poor, intermediate and good categories. So hopefully, you’re still with us but please speak up if not. The lipophilicity values have been ordered into bins and as regular readers of the Crapshoot you’ll be wondering why they just don’t plot the data instead of putting it into all these bins. Now when you look at Figures 6a and 6b you might be thinking that the data is evenly distributed across the bins but if you look at the fine print on top of each bar, you’ll see this is most definitely not the case. Furthermore when you compare these numbers for corresponding bins in the two plots you’ll see that the distribution of the data across bins differs in the two plots. Not that you’d guess that from just looking at the plots and it does make meaningful visual comparison of the plots difficult.
So the authors would have us believe that ACD logDpH7.4 is a more effective than ACD clogP as a predictor of aqueous solubility. Let’s take a look at how they do this. Basically the ‘analysis’ consists of looking at the bar charts in Figure’s 6a and 6b and stating:
“The clearer stepped differentiation within the bands is apparent when log DpH7.4 rather than log P is used, which reflects the conisderable [sic] contribution of ionization to solubility.”
In other words, a beauty contest for charts.
However, we’re not quite done yet because we still need to take a look at the Solubility Forecast Index (SFI) although we have nasty feeling that we’re not going to like it when we do. SFI is defined as the sum of clogDpH7.4 and the number of aromatic rings (#Ar) and the equivalent bar chart to Figures 6a and 6b is shown in Figure 9. We are going to take a much, much closer look at SFI in another Crapshoot but for now let’s just see what the authors have to say about the bar charts:
“This graded bar graph (Figure 9) can be compared with that shown in Figure 6b to show an increase in resolution when considering binned SFI versus binned c log DpH7.4 alone.”
So I guess you’re all wondering what the difference is between “clearer stepped differentiation within the bands” and “an increase in resolution”. Please let us know if you do find out because we’d love to know as well. We’d also like to know exactly how the authors define resolution because to speak of an increase in resolution is to make a quantitative statement.We really don’t have any answer to this question so, as an instructive excercise, we suggest that our readers might attempt to describe the relationship between Figure 6a and Figure 9. Bonus points will be awarded for answers presented in Limerick format.
Of course the raison d'être of the Crapshoot is not just to seek the funny side of Drug Discovery and we also like to provide practical advice that will be seen as helpful and constructive. We advise the authors to seek the opinion of a professional statistician as to whether beauty contests for bar charts constitute a valid method for asserting that one parameter provides a quantitatively better description than another of solubility (or indeed any other property of interest). We also believe that editors of journals greatly value feedback from those who occasionally read those journals and so we offer the following advice. Find out who reviewed the manuscript for you and make sure that they don't do any more.
“Data plots with lines of best fit and unity gave a representation of the data, albeit with a statistical analysis, which did not adequately convey the distribution of data because of the large numbers. The distribution of values was better conveyed through normalized bar graphs and box plots using binned hydrophobicity and/or solubility values, which better represent the distribution of data in a more visually amenable manner.”
To paraphrase: We couldn’t find what we wanted to when we analysed the data so we drew some pictures instead.
OK, this assessment may seem harsh and we do admit that plotting data is certainly a good thing, especially as a precursor to analysis. However, we have shown you previously that weak trends can be made to look a whole heap stronger by hiding or masking variation and when you plot data enough you can end up seeing what you think should be there. Also, if you’ve got enough data then even the weakest trend becomes significant and we respectfully draw the attention of our readers to the tale (as opposed to the tail) of the 55% coin. When presenting trends, it’s really important to remember a trend’s strength is even more important than its mere existence.
So let’s get back to business and we’d like you to take a look at Figures 6a and 6b which illustrate the relationships between aqueous solubility and two different calculated lipophilicities, namely logP and logDpH7.4 that have been predicted using the ACD software. Solubility is ‘quantified’ as a series of bars that indicate the relative proportions of compounds in poor, intermediate and good categories. So hopefully, you’re still with us but please speak up if not. The lipophilicity values have been ordered into bins and as regular readers of the Crapshoot you’ll be wondering why they just don’t plot the data instead of putting it into all these bins. Now when you look at Figures 6a and 6b you might be thinking that the data is evenly distributed across the bins but if you look at the fine print on top of each bar, you’ll see this is most definitely not the case. Furthermore when you compare these numbers for corresponding bins in the two plots you’ll see that the distribution of the data across bins differs in the two plots. Not that you’d guess that from just looking at the plots and it does make meaningful visual comparison of the plots difficult.
So the authors would have us believe that ACD logDpH7.4 is a more effective than ACD clogP as a predictor of aqueous solubility. Let’s take a look at how they do this. Basically the ‘analysis’ consists of looking at the bar charts in Figure’s 6a and 6b and stating:
“The clearer stepped differentiation within the bands is apparent when log DpH7.4 rather than log P is used, which reflects the conisderable [sic] contribution of ionization to solubility.”
In other words, a beauty contest for charts.
However, we’re not quite done yet because we still need to take a look at the Solubility Forecast Index (SFI) although we have nasty feeling that we’re not going to like it when we do. SFI is defined as the sum of clogDpH7.4 and the number of aromatic rings (#Ar) and the equivalent bar chart to Figures 6a and 6b is shown in Figure 9. We are going to take a much, much closer look at SFI in another Crapshoot but for now let’s just see what the authors have to say about the bar charts:
“This graded bar graph (Figure 9) can be compared with that shown in Figure 6b to show an increase in resolution when considering binned SFI versus binned c log DpH7.4 alone.”
So I guess you’re all wondering what the difference is between “clearer stepped differentiation within the bands” and “an increase in resolution”. Please let us know if you do find out because we’d love to know as well. We’d also like to know exactly how the authors define resolution because to speak of an increase in resolution is to make a quantitative statement.We really don’t have any answer to this question so, as an instructive excercise, we suggest that our readers might attempt to describe the relationship between Figure 6a and Figure 9. Bonus points will be awarded for answers presented in Limerick format.
Of course the raison d'être of the Crapshoot is not just to seek the funny side of Drug Discovery and we also like to provide practical advice that will be seen as helpful and constructive. We advise the authors to seek the opinion of a professional statistician as to whether beauty contests for bar charts constitute a valid method for asserting that one parameter provides a quantitatively better description than another of solubility (or indeed any other property of interest). We also believe that editors of journals greatly value feedback from those who occasionally read those journals and so we offer the following advice. Find out who reviewed the manuscript for you and make sure that they don't do any more.
Thursday, January 19, 2012
A primer on lipophilicity
Well it has been a while since we last posted and it is not too far into 2012 for us to wish you a very happy new year that is free of categorical sin. In the next post, we’re going to take a look at relationships between aqueous solubility and lipophilicity so we thought that a lipophilicity primer would represent a noble public service. The most important quantity when discussing lipophilicity is the partition coefficient, P, which for a compound that lacks ionisable groups can be measured by shaking the compound with water and an organic solvent (usually 1-octanol ) that does not mix with it and we’ll only be discussing measurements in the octanol/water system in this post. When you’ve shaken everything long enough for compound to equilibrate properly between the solvents, you can stop shaking and wait for the octanol and water to separate into two layers. Once this has happened, we just need to measure the concentration of the compound in each solvent and calculate logP (we generally use the logarithm of P rather than P itself) using equation 1. The measurement of partition coefficients is pretty routine these days and there is even a piece of kit called a shake flask which means that you can go for well-earned coffee rather than having to shake a separation funnel. However, please remember that you can only use equation 1 to calculate logP for compounds that are predominantly neutral at the pH of the experiment.
When the compound has ionisable groups, the situation gets a bit more complicated. But only a bit, so please don’t worry because we’ve been through situations like these before together. First thing to remember is that when we measure lipophilicity of a compound we’re actually measuring something called the distribution coefficient, D. This quantity D is just like P except that we use the total concentration of both neutral and ionised forms of the compound in each solvent to determine D and the general situation can get rather messy. When we measure lipophilicity using octanol/water partitioning, we normally assume that only the neutral form of the compound goes into the octanol and that ionised forms of the compound will stay in the water. This assumption can break down but we’re not going to worry about that right now because it probably (hopefully?) doesn’t happen too much. The situation that we most frequently encounter when thinking about logD values is one in which the compound has a single ionisable group. In this case, we can calculate logD from measured concentrations using equation 2 which we’ve re-arranged to equation 3.
Let’s take a look at what equation 3 tells us. Firstly, when there is no ionisation, the fraction of compound in its neutral from is 1 so logP and logD are identical. Secondly, the fraction of compound in its neutral form cannot exceed 1 so its logarithm cannot exceed zero and logD cannot exceed logP. Thirdly, equation 3 tells us how to obtain logP from measured logD values. One approach is to measure logD at a pH at which the proportion of the compound in the ionised form is insignificant. For example, we could measure logD for a carboxylic acid at low pH (you’ll know if that it’s low enough if logD stops increasing as you lower pH). The alternative is to use pKa to calculate the fraction of compound in the neutral form and use equation 3 to calculate logP. Suppose we measure a logD value of 2.0 at a pH of 7.4 for an amine with a pKa of 10.4. In the buffer, the amine is 99.9% ionised and 0.1% neutral so the logP for this amine is 5.0.
An obvious question is which of logP and logD is more relevant to Drug Discovery and to be quite honest we’re not sure. The snap answer is that it depends on context. Both logP and logD are measures of how strongly the molecules of a compound interact with water and high values reflect weak interactions with water and a tendency for those molecules to head elsewhere should they find themselves in water. 'Elsewhere' can be any of a number of places including the hydrophobic core of a lipid bilayer membrane, inside a crystal lattice or bound to a protein. As a very rough rule of thumb, logD will be more relevant if the molecules have to 'de-ionise' to go ‘elsewhere’ while logP will be more relevant if the molecules go ‘elsewhere’ in their ionised states. This all sounds like pseudo-mystical psychobabble, we hear you cry and so we’ll try to make it a bit clearer with something a bit more specific. Let’s start with aqueous solubility and many of you will remember extracting carboxylic acids from organic solvents by shaking them some aqueous sodium hydroxide. This works because aqueous solubility of ionisable compounds tends to be limited by the solubility of the neutral form. Suppose a carboxylic acid with a pKa of 4.4 has a solubility of 1000µM at a pH of 7.4. The concentration of neutral acid under these conditions will only be 1µM and we can infer that the solubility of the neutral form of the acid is only 1µM. Let’s magically increase the pKa of the acid to 5.4 (which the observant amongst you will have observed may be a bit high for a typical carboxylic acid), leaving the solubility of the neutral form unchanged and see what happens. Our new acid is now only 99% ionised under assay conditions instead of 99.9% ionised which means that the neutral form will start precipitating (supersaturation permitting) once the total concentration of acid gets to 100µM. Increasing pKa which makes the acid less acidic results in a decrease in the measured solubility .
So we hope that this will give you a better idea of what ‘elsewhere’ means in this context. The next question is how should we use logP and logD as descriptors for analysing data. What you do depends a bit on what you have available and what sorts of compounds you’re dealing with. If you have measured logD values and the compounds lack ionisable groups then logD and logP are identical and these measured values will be more relevant than predicted values (provided of course you have a clear idea of the dynamic range of the logD measurement and quantification/detection limits). Life gets more complicated if you have to handle compounds with ionisable groups because you’re unlikely to have measured pKa values available for all the compounds that you’re interested in and access to logP will involve a predictive element if logD has only been measured at a single pH. You might decide that logD is more relevant than logP to your situation in which case you can use logD. However, when you use logD measured at a pH of 7.4 to model data you need to remember (equation 3) that you’ll be treating an amine with a pKa of 11.4 and a logP of 6.0 as equivalent to an amine with a pKa of 8.4 and a logP of 3.0.
The situation most frequently encountered when using lipophilicity as a descriptor is the one where both logP and logD are themselves predicted. Usually logD will be predicted from logP using an estimate for the pKa and, if this is the case, you really need to be asking yourself whether it really makes sense to bundle logP and fN together when they two quantities describe such different phenomena. If you use logD in a predictive model then that model will respond identically to the same change in logP and logfN and, if you’re really thinking about what you’re doing, you’ll be asking yourself if you really want your model to be doing this.
It’s getting to the point at which we should be wrapping up. We’ll leave you with links to a wikipedia page and an article that present some of the material we’ve discussed from a different angle and in more depth and we hope that you find them useful. In next Crapshoot we’ll returning to the critical review of literature that you’ve come to expect of us and you can expect some getting physical in drug discovery.
When the compound has ionisable groups, the situation gets a bit more complicated. But only a bit, so please don’t worry because we’ve been through situations like these before together. First thing to remember is that when we measure lipophilicity of a compound we’re actually measuring something called the distribution coefficient, D. This quantity D is just like P except that we use the total concentration of both neutral and ionised forms of the compound in each solvent to determine D and the general situation can get rather messy. When we measure lipophilicity using octanol/water partitioning, we normally assume that only the neutral form of the compound goes into the octanol and that ionised forms of the compound will stay in the water. This assumption can break down but we’re not going to worry about that right now because it probably (hopefully?) doesn’t happen too much. The situation that we most frequently encounter when thinking about logD values is one in which the compound has a single ionisable group. In this case, we can calculate logD from measured concentrations using equation 2 which we’ve re-arranged to equation 3.
Let’s take a look at what equation 3 tells us. Firstly, when there is no ionisation, the fraction of compound in its neutral from is 1 so logP and logD are identical. Secondly, the fraction of compound in its neutral form cannot exceed 1 so its logarithm cannot exceed zero and logD cannot exceed logP. Thirdly, equation 3 tells us how to obtain logP from measured logD values. One approach is to measure logD at a pH at which the proportion of the compound in the ionised form is insignificant. For example, we could measure logD for a carboxylic acid at low pH (you’ll know if that it’s low enough if logD stops increasing as you lower pH). The alternative is to use pKa to calculate the fraction of compound in the neutral form and use equation 3 to calculate logP. Suppose we measure a logD value of 2.0 at a pH of 7.4 for an amine with a pKa of 10.4. In the buffer, the amine is 99.9% ionised and 0.1% neutral so the logP for this amine is 5.0.
An obvious question is which of logP and logD is more relevant to Drug Discovery and to be quite honest we’re not sure. The snap answer is that it depends on context. Both logP and logD are measures of how strongly the molecules of a compound interact with water and high values reflect weak interactions with water and a tendency for those molecules to head elsewhere should they find themselves in water. 'Elsewhere' can be any of a number of places including the hydrophobic core of a lipid bilayer membrane, inside a crystal lattice or bound to a protein. As a very rough rule of thumb, logD will be more relevant if the molecules have to 'de-ionise' to go ‘elsewhere’ while logP will be more relevant if the molecules go ‘elsewhere’ in their ionised states. This all sounds like pseudo-mystical psychobabble, we hear you cry and so we’ll try to make it a bit clearer with something a bit more specific. Let’s start with aqueous solubility and many of you will remember extracting carboxylic acids from organic solvents by shaking them some aqueous sodium hydroxide. This works because aqueous solubility of ionisable compounds tends to be limited by the solubility of the neutral form. Suppose a carboxylic acid with a pKa of 4.4 has a solubility of 1000µM at a pH of 7.4. The concentration of neutral acid under these conditions will only be 1µM and we can infer that the solubility of the neutral form of the acid is only 1µM. Let’s magically increase the pKa of the acid to 5.4 (which the observant amongst you will have observed may be a bit high for a typical carboxylic acid), leaving the solubility of the neutral form unchanged and see what happens. Our new acid is now only 99% ionised under assay conditions instead of 99.9% ionised which means that the neutral form will start precipitating (supersaturation permitting) once the total concentration of acid gets to 100µM. Increasing pKa which makes the acid less acidic results in a decrease in the measured solubility .
So we hope that this will give you a better idea of what ‘elsewhere’ means in this context. The next question is how should we use logP and logD as descriptors for analysing data. What you do depends a bit on what you have available and what sorts of compounds you’re dealing with. If you have measured logD values and the compounds lack ionisable groups then logD and logP are identical and these measured values will be more relevant than predicted values (provided of course you have a clear idea of the dynamic range of the logD measurement and quantification/detection limits). Life gets more complicated if you have to handle compounds with ionisable groups because you’re unlikely to have measured pKa values available for all the compounds that you’re interested in and access to logP will involve a predictive element if logD has only been measured at a single pH. You might decide that logD is more relevant than logP to your situation in which case you can use logD. However, when you use logD measured at a pH of 7.4 to model data you need to remember (equation 3) that you’ll be treating an amine with a pKa of 11.4 and a logP of 6.0 as equivalent to an amine with a pKa of 8.4 and a logP of 3.0.
The situation most frequently encountered when using lipophilicity as a descriptor is the one where both logP and logD are themselves predicted. Usually logD will be predicted from logP using an estimate for the pKa and, if this is the case, you really need to be asking yourself whether it really makes sense to bundle logP and fN together when they two quantities describe such different phenomena. If you use logD in a predictive model then that model will respond identically to the same change in logP and logfN and, if you’re really thinking about what you’re doing, you’ll be asking yourself if you really want your model to be doing this.
It’s getting to the point at which we should be wrapping up. We’ll leave you with links to a wikipedia page and an article that present some of the material we’ve discussed from a different angle and in more depth and we hope that you find them useful. In next Crapshoot we’ll returning to the critical review of literature that you’ve come to expect of us and you can expect some getting physical in drug discovery.
Wednesday, April 20, 2011
The Transit Disagreement
Today is a special day since it marks four years since we started this blog. We thank Our Loyal Readers (all three of them) for their continued support.
Transport is very important. For example, after the Wehrmacht invaded Norway they wanted to reinforce the troops already there. This was not easy because Germany and Norway do not share a border and so they did a deal to allow a few troops to pass through Swedish territory. You should probably think about this transport as facilitated rather than active since the chaps from the Wehrmacht were simply goose-stepping down a concentration gradient rather than being carried in sedan chairs by their Swedish hosts.
Drug discovery is similar to the Norwegian Problem and we’re not talking about commercial whaling which is also a Japanese problem even if both claim that they do it for ‘scientific’ reasons. You need to equip your troops properly and then get enough of them there to do the job properly. The objective of drug design is to ensure that your creation actually hits its intended target(s) with minimal collateral damage. If you’re designing a drug for oral dosing then getting it into the blood stream is usually a good start because drug targets are usually in or on cells and these cells can’t get too far from the blood otherwise they die. Once you’ve got the drug into the circulation, you may or may not want it to get into cells and through other barriers such as the one that protects the brain, although achieving this degree of control is not trivial. The view from Pharma is that most drugs get to their targets by passive diffusion through cell membranes. However, this view has been challenged:
'In this article, we discuss the evidence supporting the idea that rather than being an exception, carrier-mediated and active uptake of drugs may be more common than is usually assumed — including a summary of specific cases in which drugs are known to be taken up into cells via defined carriers — and consider the implications for drug discovery and development.'
Let’s all think about how a protein can help get a drug into cells. The first way is by increasing the drug’s permeability through the membrane so that the drug can move faster down the concentration gradient. This mechanism, sometimes called facilitated diffusion, involves temporary binding of the drug to a protein in the cell membrane and, as is the case for passive diffusion, the drug moves down its concentration gradient. Diffusion whether facilitated or passive is still diffusion and it is hellishly difficult to quantify the relative contributions of the two processes for an arbitrary drug and cell. Facilitated diffusion can be saturated (just like an enzyme) but then again the lipid portion of the membrane may also have a finite capacity for drug.
Active transport can oppose diffusion and when this happens it may be easier to observe (in principle anyway). An input of energy is required when a drug is coerced up its concentration gradient (hence the term active). Let’s suppose that we have a barrier through which we believe active transport takes place and want to test this hypothesis. The usual approach is to measure the permeability from each side of the barrier and see how different the two measurements are. Another possible approach which doesn’t seem to get mentioned would be to allow the system to come to steady state and measure the concentrations on either side of the barrier.
Now we’ll take a closer look at the article. The authors state that, ‘There is abundant evidence for carrier mediated drug uptake in specific cases where it has been studied’. However, we were confused by much of the evidence for carrier mediated drug uptake that was presented by the authors. What does it mean for a drug to be a substrate for a transporter? Is it possible to quantify the relative contributions of passive diffusion through the lipid bilayer and carrier-mediated uptake? How predictive are these cell-based assays of the physiological reality of an in vivo situation? Is this carrier-mediated transport passive or active? Just how many drugs have been evaluated in these studies?
Another point made in support of the view that ‘rather than being an exception, carrier-mediated and active uptake of drugs may be more common than is usually assumed’ is that ‘Drugs can concentrate in specific tissues’. The demonstrate some awareness of the difficulties of defining a relevant intracellular concentration when they say, ‘Binding is probably not the major issue as intracellular concentrations can be significantly larger than any plausible stoicheiometric concentration of binding sites’. However, they do not seem to be aware that basic compounds can accumulate in the acidic interior of a lysosome and that it is not actually necessary to invoke active transport to provide a rationale for this observation.
Now we want you to take a look at Figure 3b which shows a plot of Caco-2 cell permeability against logK. Like us, you were probably wondering what K is and were as surprised as us to learn that K is the octanol-water partition coefficient which everybody else calls P. So logK is just our old friend logP although we don’t know if it is a predicted or measured quantity. Our advice to Systems Biologists is use logP rather than logK for this quantity if you want people in Drug Discovery to think that you know what you're talking about. You’ll notice that the correlation between permeability and logK (which is really logP) is not very good and there are a number of plausible explanations for this observation that have nothing to do with carrier-mediated transport. Firstly, if a compound contains ionisable groups there will be less neutral form available for partitioning into the membrane and perhaps you should be looking at logD or at least accounting for the ionisation. Secondly, octanol is not a good model for the membrane core because it has a polar hydroxyl group and gets pretty wet when in contact with water.
However, there is a third reason why you might not see a great correlation between Caco-2 cell permeability and logP that has nothing to do with logP. You’ll need to think a bit about this but please don’t worry because we’ll be right next you all the time. First, take a look at this helpful description of the Caco-2 permeability assay because it’ll give you an idea how these assays are run. When measuring permeability through a barrier you’ll typically introduce a known amount of compound on what we’ll call the ‘donor’ side of the barrier and then measure compound concentration on both sides (‘donor’ and ‘acceptor’) of the barrier as a function of time. Now consider the situation in which concentration is measured at a single time point. If the permeability is really low, the quantity of compound on the acceptor side of the membrane will be too small to measure and the concentration on the donor side will not differ detectably from the initial concentration. You can also have problems if the compound is too permeable because you can end up with similar concentrations on either side of the barrier, which can lead to significant uncertainty in the measured values of permeability. It also means that there is an upper limit to the permeability that can be measured and you can get an idea what this will be by taking a really close look at the assay protocol.
There’s other information that you can get from Caco-2 permeability assays. A Caco-2 cell monolayer is polarized, having apical (A) and basolateral (B) faces and intestinal absorption is in the A→B direction. However, you can also measure the permeability in the B→A direction and a significant difference from the A→B permeability is an indication that the compound is actively transported through the monolayer of cells. If, as the authors suggest is frequently the case, compounds require active transport in order to be absorbed from the gut then you’d expect that the A→B permeability will frequently exceed the B→A permeability in Caco-2 (and MDCK) cell permeability assays. It is rather curious that the authors do not discuss this point in connection with Figure 3b.
Our experience with Caco-2 assays is that it is actually more common for the B→A permeability to exceed the A→B permeability. This situation corresponds to efflux where the transporters pump the compound back into the gut and the ratio of B→A to A→B permeability is called the efflux ratio. If a compound can make it to the blood stream without the assistance of active transport then it’s difficult to argue a case for the same compound requiring active transport to get into other cells. In our view, the authors have not presented convincing evidence that supports a view that carrier-mediated and active uptake is particularly common. We were surprised that they did not look at measured ratios of B→A to A→B permeability in Caco-2 monolayers since these would have provided evidence with which they could have tested their hypothesis.
Despite our perceptions of the weakness of the case presented by the authors, we certainly accept that transporter-mediated uptake of drugs does occur in some cases. Our issue is more with the idea that it is the norm rather than the exception. We were a more than a little surprised to read in the abstract of the follow-up article that:
‘Drug entry into cells was previously thought to be via diffusion through the lipid bilayer of the cell membrane with the contribution to uptake by transporter proteins being of only marginal importance. Now, however drug uptake is understood to be mainly transporter-mediated’.
Understood? By whom?
Transport is very important. For example, after the Wehrmacht invaded Norway they wanted to reinforce the troops already there. This was not easy because Germany and Norway do not share a border and so they did a deal to allow a few troops to pass through Swedish territory. You should probably think about this transport as facilitated rather than active since the chaps from the Wehrmacht were simply goose-stepping down a concentration gradient rather than being carried in sedan chairs by their Swedish hosts.
Drug discovery is similar to the Norwegian Problem and we’re not talking about commercial whaling which is also a Japanese problem even if both claim that they do it for ‘scientific’ reasons. You need to equip your troops properly and then get enough of them there to do the job properly. The objective of drug design is to ensure that your creation actually hits its intended target(s) with minimal collateral damage. If you’re designing a drug for oral dosing then getting it into the blood stream is usually a good start because drug targets are usually in or on cells and these cells can’t get too far from the blood otherwise they die. Once you’ve got the drug into the circulation, you may or may not want it to get into cells and through other barriers such as the one that protects the brain, although achieving this degree of control is not trivial. The view from Pharma is that most drugs get to their targets by passive diffusion through cell membranes. However, this view has been challenged:
'In this article, we discuss the evidence supporting the idea that rather than being an exception, carrier-mediated and active uptake of drugs may be more common than is usually assumed — including a summary of specific cases in which drugs are known to be taken up into cells via defined carriers — and consider the implications for drug discovery and development.'
Let’s all think about how a protein can help get a drug into cells. The first way is by increasing the drug’s permeability through the membrane so that the drug can move faster down the concentration gradient. This mechanism, sometimes called facilitated diffusion, involves temporary binding of the drug to a protein in the cell membrane and, as is the case for passive diffusion, the drug moves down its concentration gradient. Diffusion whether facilitated or passive is still diffusion and it is hellishly difficult to quantify the relative contributions of the two processes for an arbitrary drug and cell. Facilitated diffusion can be saturated (just like an enzyme) but then again the lipid portion of the membrane may also have a finite capacity for drug.
Active transport can oppose diffusion and when this happens it may be easier to observe (in principle anyway). An input of energy is required when a drug is coerced up its concentration gradient (hence the term active). Let’s suppose that we have a barrier through which we believe active transport takes place and want to test this hypothesis. The usual approach is to measure the permeability from each side of the barrier and see how different the two measurements are. Another possible approach which doesn’t seem to get mentioned would be to allow the system to come to steady state and measure the concentrations on either side of the barrier.
Now we’ll take a closer look at the article. The authors state that, ‘There is abundant evidence for carrier mediated drug uptake in specific cases where it has been studied’. However, we were confused by much of the evidence for carrier mediated drug uptake that was presented by the authors. What does it mean for a drug to be a substrate for a transporter? Is it possible to quantify the relative contributions of passive diffusion through the lipid bilayer and carrier-mediated uptake? How predictive are these cell-based assays of the physiological reality of an in vivo situation? Is this carrier-mediated transport passive or active? Just how many drugs have been evaluated in these studies?
Another point made in support of the view that ‘rather than being an exception, carrier-mediated and active uptake of drugs may be more common than is usually assumed’ is that ‘Drugs can concentrate in specific tissues’. The demonstrate some awareness of the difficulties of defining a relevant intracellular concentration when they say, ‘Binding is probably not the major issue as intracellular concentrations can be significantly larger than any plausible stoicheiometric concentration of binding sites’. However, they do not seem to be aware that basic compounds can accumulate in the acidic interior of a lysosome and that it is not actually necessary to invoke active transport to provide a rationale for this observation.
Now we want you to take a look at Figure 3b which shows a plot of Caco-2 cell permeability against logK. Like us, you were probably wondering what K is and were as surprised as us to learn that K is the octanol-water partition coefficient which everybody else calls P. So logK is just our old friend logP although we don’t know if it is a predicted or measured quantity. Our advice to Systems Biologists is use logP rather than logK for this quantity if you want people in Drug Discovery to think that you know what you're talking about. You’ll notice that the correlation between permeability and logK (which is really logP) is not very good and there are a number of plausible explanations for this observation that have nothing to do with carrier-mediated transport. Firstly, if a compound contains ionisable groups there will be less neutral form available for partitioning into the membrane and perhaps you should be looking at logD or at least accounting for the ionisation. Secondly, octanol is not a good model for the membrane core because it has a polar hydroxyl group and gets pretty wet when in contact with water.
However, there is a third reason why you might not see a great correlation between Caco-2 cell permeability and logP that has nothing to do with logP. You’ll need to think a bit about this but please don’t worry because we’ll be right next you all the time. First, take a look at this helpful description of the Caco-2 permeability assay because it’ll give you an idea how these assays are run. When measuring permeability through a barrier you’ll typically introduce a known amount of compound on what we’ll call the ‘donor’ side of the barrier and then measure compound concentration on both sides (‘donor’ and ‘acceptor’) of the barrier as a function of time. Now consider the situation in which concentration is measured at a single time point. If the permeability is really low, the quantity of compound on the acceptor side of the membrane will be too small to measure and the concentration on the donor side will not differ detectably from the initial concentration. You can also have problems if the compound is too permeable because you can end up with similar concentrations on either side of the barrier, which can lead to significant uncertainty in the measured values of permeability. It also means that there is an upper limit to the permeability that can be measured and you can get an idea what this will be by taking a really close look at the assay protocol.
There’s other information that you can get from Caco-2 permeability assays. A Caco-2 cell monolayer is polarized, having apical (A) and basolateral (B) faces and intestinal absorption is in the A→B direction. However, you can also measure the permeability in the B→A direction and a significant difference from the A→B permeability is an indication that the compound is actively transported through the monolayer of cells. If, as the authors suggest is frequently the case, compounds require active transport in order to be absorbed from the gut then you’d expect that the A→B permeability will frequently exceed the B→A permeability in Caco-2 (and MDCK) cell permeability assays. It is rather curious that the authors do not discuss this point in connection with Figure 3b.
Our experience with Caco-2 assays is that it is actually more common for the B→A permeability to exceed the A→B permeability. This situation corresponds to efflux where the transporters pump the compound back into the gut and the ratio of B→A to A→B permeability is called the efflux ratio. If a compound can make it to the blood stream without the assistance of active transport then it’s difficult to argue a case for the same compound requiring active transport to get into other cells. In our view, the authors have not presented convincing evidence that supports a view that carrier-mediated and active uptake is particularly common. We were surprised that they did not look at measured ratios of B→A to A→B permeability in Caco-2 monolayers since these would have provided evidence with which they could have tested their hypothesis.
Despite our perceptions of the weakness of the case presented by the authors, we certainly accept that transporter-mediated uptake of drugs does occur in some cases. Our issue is more with the idea that it is the norm rather than the exception. We were a more than a little surprised to read in the abstract of the follow-up article that:
‘Drug entry into cells was previously thought to be via diffusion through the lipid bilayer of the cell membrane with the contribution to uptake by transporter proteins being of only marginal importance. Now, however drug uptake is understood to be mainly transporter-mediated’.
Understood? By whom?
Saturday, January 1, 2011
Happy New Year
Happy New Year from Pharma Fellow, Senior Pharma Fellow, The Blue Team, The Red Team and the rest of us here at The Great Molecular Crapshoot. Please make it your new year resolution to avoid all Categorical Sin.
Wednesday, December 29, 2010
The wisdom of herds
Well it has indeed been a while and we hope that many of you will have enjoyed a restful holiday season. Back in September we wrote about Experts and were so traumatised by the experience that it is only now that we can return to the theme. At least this time we promise to avoid all mention of Visionaries because the self-appointed Visionary is simultaneously one of the most pathetic and one of the most irritating people that you will ever encounter during a career in Pharma.
The article featured in this Crapshoot is a crowdsourcing evaluation of some chemical probes. What is crowdsourcing, we hear you cry and what place does it have in a journal that sees itself as associated with serious science? The honest answer is that we don’t know because the domain of applicability of our own expertise lies well outside the social ‘sciences’. So let’s go through this together.
The basis of crowdsourcing is the wisdom of crowds which is defined in Box 2 as:
‘This concept, popularized by Surowiecki, describes group decision making based on the aggregation of independent, individual decisions, where the average decision is more accurate than any individual decision. The four elements of a wise crowd are independence, diversity of opinion, decentralization and aggregation.’
Previously we have read about the behaviour of crowds of investors. In discussions about the behaviour of investors we frequently encounter terms such as ‘greed’ and ‘panic’ and rarely (if ever) hear groups of individuals described as ‘rational’ (except by the most theoretical of Economists). Crowds of investors and Pharma scientists share a herding instinct that can lead to group decision making that is neither rational nor accurate. Perhaps we should really be talking about herdsourcing.
So let’s take a closer look at the crowdsourcing study. A while back the NIH set up the Molecular Libraries and Imaging (MLI) initiative. The people in the Molecular Libraries Screening Centers Network (MLSCN) did some screening and, among other things, nominated 64 chemical probes. At this point the crowdsourcers dropped by and we really couldn’t help being reminded of the term Seagull Manager. The crowdsourced group (CSG) are described both as ‘a team of 11 scientists with diverse backgrounds in small molecule discovery’ and ‘well-known experts in preclinical drug discovery’. The team was invited to express their level of confidence in the probes and we were greatly amused to encounter the term ‘molecular confidence’. Can you imagine the answers to the question in Cheminformatics 101 asking you to use the term ‘molecular confidence’? (Brimming with molecular confidence, tetrafluromethane knew with certainty that she’d be able to handle anything that that the cytochrome P450s threw at her).
The ‘evaluation of the probes was performed on a qualitative ranking of 0 (high confidence, low dubiosity) to 10 (low confidence, high dubiosity)’. We are not sure that a scale of 0 to 10 can be described as ‘qualitative’. The ranking scheme may be inaccurate, imprecise and/or irrelevant but we just can’t see how something with eleven levels (we assume that non-integer scores were not allowed) can be described as qualitative rather than quantitative. It was not clear to us how the individual members of the CSG determined the level of dubiosity and we did not find that the James Joyce quote conveyed any information other than an impression of pretentiousness. We would have liked to know a bit more about how the CSG group assigned numbers to compounds. Did they perform detailed analysis or simply gaze expertly at structures? However, these details are not necessary for what we want to do next which is to take a closer look at the ‘Experts’.
The featured article could actually have been a very interesting study of ‘Experts’. Many interesting questions could have been addressed. Are some CSG members harsher on average than others in their assessment of probes? Is it possible to cluster the team members on the basis of their scores? Could the same information have been obtained with a smaller CSG? Who were the dissenters and who was most likely to regress to the mean? Perhaps a missed opportunity but we won’t dwell on this because we’re itching to get onto what really interests us about this work. How were the CSG members selected?
As we have already noted, the authors of the study describe the CSG members as ‘well-known experts in preclinical drug discovery’ and since the CSG members are also authors it is completely understandable that their expertise should be asserted in this manner. In our view some members of the CSG are not exactly household names and it is not clear that they are any more expert than the MLSCN personnel who nominated the probes in the first place. If we were assembling a group of Experts we’d want each one to have been corresponding author for some (non-review) articles in the last 3 years or so. We noticed (see Acknowledgements) that no fewer than six people contributed to the vote of one CSG member and we wondered if all six were Experts while speculating about the magnitude of their contribution. Another CSG member had only voted on a small minority of the 64 probes.
The assembly of the CSG raises some other issues that probably shouldn’t be talked about in polite company although we won’t let that inhibit us. Being described as ‘An Expert’ in a publication like this is beneficial both to the individual concerned and the organisation to which he or she belongs. As such there is a potential conflict of interest issue that at least needs to be acknowledged.
Crowdsourcing or herdsourcing? It is not for us to say for we are simple folk.
No Experts or ‘Experts’ were harmed in the production of this Crapshoot.
The article featured in this Crapshoot is a crowdsourcing evaluation of some chemical probes. What is crowdsourcing, we hear you cry and what place does it have in a journal that sees itself as associated with serious science? The honest answer is that we don’t know because the domain of applicability of our own expertise lies well outside the social ‘sciences’. So let’s go through this together.
The basis of crowdsourcing is the wisdom of crowds which is defined in Box 2 as:
‘This concept, popularized by Surowiecki, describes group decision making based on the aggregation of independent, individual decisions, where the average decision is more accurate than any individual decision. The four elements of a wise crowd are independence, diversity of opinion, decentralization and aggregation.’
Previously we have read about the behaviour of crowds of investors. In discussions about the behaviour of investors we frequently encounter terms such as ‘greed’ and ‘panic’ and rarely (if ever) hear groups of individuals described as ‘rational’ (except by the most theoretical of Economists). Crowds of investors and Pharma scientists share a herding instinct that can lead to group decision making that is neither rational nor accurate. Perhaps we should really be talking about herdsourcing.
So let’s take a closer look at the crowdsourcing study. A while back the NIH set up the Molecular Libraries and Imaging (MLI) initiative. The people in the Molecular Libraries Screening Centers Network (MLSCN) did some screening and, among other things, nominated 64 chemical probes. At this point the crowdsourcers dropped by and we really couldn’t help being reminded of the term Seagull Manager. The crowdsourced group (CSG) are described both as ‘a team of 11 scientists with diverse backgrounds in small molecule discovery’ and ‘well-known experts in preclinical drug discovery’. The team was invited to express their level of confidence in the probes and we were greatly amused to encounter the term ‘molecular confidence’. Can you imagine the answers to the question in Cheminformatics 101 asking you to use the term ‘molecular confidence’? (Brimming with molecular confidence, tetrafluromethane knew with certainty that she’d be able to handle anything that that the cytochrome P450s threw at her).
The ‘evaluation of the probes was performed on a qualitative ranking of 0 (high confidence, low dubiosity) to 10 (low confidence, high dubiosity)’. We are not sure that a scale of 0 to 10 can be described as ‘qualitative’. The ranking scheme may be inaccurate, imprecise and/or irrelevant but we just can’t see how something with eleven levels (we assume that non-integer scores were not allowed) can be described as qualitative rather than quantitative. It was not clear to us how the individual members of the CSG determined the level of dubiosity and we did not find that the James Joyce quote conveyed any information other than an impression of pretentiousness. We would have liked to know a bit more about how the CSG group assigned numbers to compounds. Did they perform detailed analysis or simply gaze expertly at structures? However, these details are not necessary for what we want to do next which is to take a closer look at the ‘Experts’.
The featured article could actually have been a very interesting study of ‘Experts’. Many interesting questions could have been addressed. Are some CSG members harsher on average than others in their assessment of probes? Is it possible to cluster the team members on the basis of their scores? Could the same information have been obtained with a smaller CSG? Who were the dissenters and who was most likely to regress to the mean? Perhaps a missed opportunity but we won’t dwell on this because we’re itching to get onto what really interests us about this work. How were the CSG members selected?
As we have already noted, the authors of the study describe the CSG members as ‘well-known experts in preclinical drug discovery’ and since the CSG members are also authors it is completely understandable that their expertise should be asserted in this manner. In our view some members of the CSG are not exactly household names and it is not clear that they are any more expert than the MLSCN personnel who nominated the probes in the first place. If we were assembling a group of Experts we’d want each one to have been corresponding author for some (non-review) articles in the last 3 years or so. We noticed (see Acknowledgements) that no fewer than six people contributed to the vote of one CSG member and we wondered if all six were Experts while speculating about the magnitude of their contribution. Another CSG member had only voted on a small minority of the 64 probes.
The assembly of the CSG raises some other issues that probably shouldn’t be talked about in polite company although we won’t let that inhibit us. Being described as ‘An Expert’ in a publication like this is beneficial both to the individual concerned and the organisation to which he or she belongs. As such there is a potential conflict of interest issue that at least needs to be acknowledged.
Crowdsourcing or herdsourcing? It is not for us to say for we are simple folk.
No Experts or ‘Experts’ were harmed in the production of this Crapshoot.
Subscribe to:
Posts (Atom)
