Can Stephen Get Even? Nicks and the Fallacy of Equal Opportunity
Roger Lyons on Nicks: Part II
As long as I’ve been associated with Werk Thoroughbred Consultants, Inc., the company has never claimed that its nick rating is a "true" reflection of the compatibility between a sire line and a broodmare sire line. The company’s advertising has always set a tone of moderation appropriate to statistically based information products, which, no matter how good they are, can only estimate how things work in the real world.
This contrasts with the approach taken by Blood-Horse Publications on behalf of its competing nick rating system, which they say reflects "true" compatibility, largely because it measures "true opportunity." That’s either the best statistical system I’ve ever heard of or the loosest use of the word "true" on record. Let’s see which.
Blood-Horse Publications constantly reminds us that their system exploits "massive database resources" to give their nick rating the "advantage" of measuring the stakes production of a cross against the number of times it’s been tried. To be sure, the amount of information their system encompasses is impressive. However, if the WTC system is based on all the pertinent information that is needed, as I believe, then, clearly, the Blood-Horse Publications system is using more information than is required.
This is not a virtue, as Sir William of Occam (you know, the guy with the razor) reminds us when he says, "The simpler the explanation, the more likely its correctness." He taught that, since the logic of an argument must ultimately answer for its assumptions, the argument requiring the fewest assumptions is probably the strongest. Mindful of the sting of Occam’s razor, let’s see if their system has any assumptions that bleed.
Back in 2005 I wrote a piece for WTC’s TeamWerk newsletter explaining, with uncharacteristic clarity, why reducing opportunity to the number of times a cross had been tried would NOT be a reliable way to evaluate crosses. It’s too blunt an instrument to work in a nick rating system without an adequate adjustment for the uneven distribution of quality. It’s like bringing boxing gloves to a knife fight.
So, you can imagine my surprise at reading that Blood-Horse Publications had found "a way around" the broad problem I had described. I read with interest the account of this at the Blood-Horse Publications website dedicated to its nick rating, anticipating an ingenious adjustment for variations in the quality of opportunity.
All I found was a formula whose reliability depends on the assumption that mares bred to a given stallion "have generally been bred to other stallions of similar quality." In attempting to put a lid on the broader problem (and I’m not saying they succeeded), the system opens a can of worms. It’s locked into what might be called the equal opportunity fallacy–the assumption that, while the quantity of opportunity varies, the quality is always the same.
Given how critical that assumption is to their system, as we shall see, it’s surprising how easy it is to think of circumstances under which a stallion’s quality might differ significantly from the quality of other stallions his former mates had been bred to. I’ll mention just one common scenario and point out the consequences.
It’s common knowledge that Stephen Got Even’s sire, A.P. Indy, has done exceptionally well when crossed with Mr. Prospector-line mares. Accordingly, three of Stephen Got Even’s 11 unrestricted stakes winners are from Mr. Prospector-line mares. Partially on that basis, the WTC nick rating system assigns an A rating to the Stephen Got Even-Mr. Prospector cross. Yet, it will be a challenge for Stephen Got Even to get even with the way the Blood-Horse Publications system treats opportunity as a variable.
To find out how well he did with Mr. Prospector-line mares, the Blood-Horse Publications system looks at the proportion of stakes winners among runners that are by Stephen Got Even and out of Mr. Prospector-line mares. It then compares that with how those same mares did with the other stallions to which they were bred.
Now, what would those "other stallions" be like relative to Stephen Got Even? The problem is that, although Stephen Got Even stands for a stud fee of $7,500 in 2009, he initially stood for $20,000. Given that those Mr. Prospector-line mares were bred to Stephen Got Even at that higher level, his former mates were undoubtedly bred largely to other stallions that were at or near that level.
Since stallions that find their level at $7,500 are generally inferior to stallions standing for $20,000, the odds are overwhelming that those other stallions are, on the whole, substantially better than Stephen Got Even. So, contrary to the assumption on which the Blood-Horse Publications nick rating is founded, Stephen Got Even is not of the same quality as those "other stallions."
Again, Stephen Got Even’s Blood-Horse Publications nick rating with Mr. Prospector-line mares is derived, in great part, by comparing his proportion of stakes winners out of those mares with that of the other stallions to which those mares were bred. That being the case, the cross is highly likely to have a very low Blood-Horse Publications nick rating, but not because it’s a bad cross. Those "other stallions" will have done better than Stephen Got Even with those mares because they are better stallions.
So, instead of measuring Stephen Got Even’s compatibility with Mr. Prospector-line mares, the Blood-Horse Publications nick rating actually measures the difference in quality between Stephen Got Even, a $7,500 stallion (if the market is to be believed), and a group of stallions at the $20,000 level. This is likely to happen even if only a minority of those stallions are substantially superior to Stephen Got Even.
For Stephen Got Even this is no small problem. His nick ratings with all broodmare sire lines will be adversely affected by that difference in quality. They will be artificially deflated in a system that Blood-Horse Publications claims has "a way around" the qualitative dimension of opportunity, but, in fact, does not have.
Moreover, it will be a problem for any stallion that has experienced an appreciable drop in market value. Mineshaft comes to mind since he is also by A.P. Indy. His stud fee dropped from a high of $100,000 to a 2009 stud fee of $30,000. In order for him to have favorable nick ratings, he’ll have to get better runners from his former mates than stallions standing for upwards from $60,000.
Mineshaft is still enjoying his birthright as a son of A.P. Indy, but just as soon as he has the requisite number of runners out of Mr. Prospector-line mares, his Blood-Horse Publications nick rating is going to plummet–and, quite possibly, just as unwarrantably as did that of Stephen Got Even.
Keep in mind that Stephen Got Even and Mineshaft represent an entire category of stallions–those whose market value declined appreciably. Strictly as an artifact of an over-reaching system, their Blood-Horse Publications nick ratings, like his, will be deflated. The opposite effect will apply to stallions whose market value increases appreciably since their performance with any given broodmare sire line will be compared with that of the lesser sires to which those mares were bred. Theirs will be inflated.
This might be acceptable to Blood-Horse Publications, but not to Sir William. He would go to work with his razor, and, after he’d pared away the offending parts of the system, what remained would resemble the WTC system, except that it would have a lot of work ahead of it.
Democratic societies can address the ideal of equality of opportunity through social policy. As we now know, certain things shouldn’t be entrusted to the market system. Thoroughbred breeding, however, is a game in which opportunity is systematically determined by practical arguments of the marketplace. The only way for a statistical system to deal with the inevitable inequalities of opportunity is to adjust quantitative measures for variations in quality. As far as I can tell, The Blood-Horse Publications system doesn’t do that.
Reducing the rich texture of actual opportunity to the number of times a cross has been tried plays better in a full-page ad in The Blood-Horse than it actually works in a nick rating system. If the Blood-Horse Publications system were able to yield a "true" measure of compatibility in the kinds of cases cited above, it wouldn’t be a typical result of the system’s logic. It would be a miracle.
Roger has more to say in future posts about the problem of estimating opportunity, but, in his next post, he will deal with how a nick rating system is affected by the standard of racing on which its evaluations are based.
Posted by Roger Lyons on Monday, February 16, 2009 at 9:07 am.
• Permalink • Comments Off on Can Stephen Get Even? Nicks and the Fallacy of Equal Opportunity