Post-diction effects: the top mass example December 17, 2006Posted by dorigo in personal, physics, politics, science.
My last post about deceiving details in the scientific presentation of data stimulated Tony Smith into suggesting that a similar effect to the one I described in the post was at work when the top quark mass was still unknown to most – but the first hints that it was sitting at around 180 GeV were starting to leak out of CDF.
I remember that time. It was the beginning of 1994, and I was attending my year of military duties as a civil servant in a cooperative where I worked with boys with psychiatric problems. I used to visit the Physics department about once a week, and I too started to hear from my colleagues in CDF the first news about the top being heavy – when most still believed it had to be at about 140 GeV of mass. The latter, at least, was the value preferred by most electroweak fits.
Ten years later I made an update to a plot I had done in 2002 for a Frontiers in Physics conference in Frascati; it is shown above. Beware, it is quite busy! In it, you see both experimental and theoretical information on the mass of the top quark as a function of time. The lower blue areas shade out top mas values excluded by direct searches, first at electron-positron colliders and then at proton-antiproton colliders. The upper green area is instead excluded by theoretical information. More precise fits to electroweak parameters allowed to “determine” the unknown value of the top quark mass as early as in the late eighties: that information is shown by the red line with yellow band. Finally, CDF and D0 determinations are shown by points with error bars.
Now, Tony’s point is that the red line had a jump towards 175 GeV or so just about at the time when the first information on the top mass was coming from kinematical fits to the first events collected by CDF in 1993-94. I think he is right, but let us take a more analytical standpoint, an d look at all the data at face value. You can get very detailed information by visiting the particle data group web site, http://pdg.lbl.gov . In it, you can browse the archives on past editions of the “Review of Particle Properties” just back to 1995. So let’s have a look at what they were publishing for the top quark back then.
There are a large number of indirect determinations of the top mass listed in the review. If we start from 1990, and first look at LEP ones only, here is what we get:
- OPAL 1991 – 100+70+24-52-11 GeV
- L3 1991 – 193+52-69 +-16 GeV
- ALEPH 1992 – 170+42+21-55-14 GeV
- ALEPH 1993 – 174+27+17-32-22 GeV
- L3 1993 – 152+36-46+-20 GeV
- OPAL 1993 – 91+-46+-9 GeV
- ALEPH 1994 – 184+25+17-29-18 GeV
- OPAL 1994 – 132+41+24-48-18 GeV
- L3 1994 – 158+32-40+-19 GeV
- DELPHI 1994 – 115+52+23-82-24 GeV
- DELPHI 1994 – 157+36+19-48-20 GeV
Taken at face value, these determination send a rather homogeneous picture – no real “bias” can be inferred from them. However, the devil is always hidden in the details. Indeed, each of these numbers comes with notes in the PDG review. From the notes, one learns about a few assumptions made in producing the estimates (such as the value of alpha_s used in the fits, the range of variability of the Higgs boson mass, and -most important- the kind of measurements used and those not used in the fits). One really starts thinking that to get a clearer picture it would be really necessary to read all the papers describing these results.
- Aleph seem to have gotten the top mass right from the start. However, they publsihed in 1993 a paper when they only gave an upper limit at 228 GeV to the top mass, while their central value is hidden in the paper: 50+-70 GeV! One thus learns that to get the limit they used experimental input on the CDF lower limit, then at 91 GeV… That analysis by Aleph was based on the width of the Z boson to b-quark pairs, a parameter that gave the LEP experiments some headache back then. I wonder if all other determinations by Aleph on the top mass used or not that particular measurement in their averages.
- Opal were always consistently low in their estimates of the top mass.
- L3 bounced around a bit, but one sees no real hint of the “bias” we are talking about here.
So that is the LEP panorama. But LEP was not alone in the business: others used LEP results as well as other determinations of electroweak parameters (SLD, but also neutrino scattering experiments) to make more “global” fits. Let’s look at these revue determinations:
- Ellis 1990 – 127+24-30 GeV
- Decamp 1990 – 120+-40+-20 GeV
- Langacker 1991 – 124+28+20-34-15 GeV
- Hioki 1991 – 145+47-48 GeV
- Gonzales 1991 – 119+39-45 GeV
- Schaile 1992 – 150+29+20-34-22 GeV
- Renton 1992 – 137+22+18-25-22 GeV
- PDG 1992 – 150+23-26+-16 GeV
- Ellis 1992 – 120+27-28 GeV
- Delaguila 1992 – 112+22-23 GeV
- Banerjee 1992 – 123+33-38+-19 GeV
- Quast 1993 – 147+22+17-26-22 GeV
- Passarino 1993 – 146+18-19+-17 GeV
- Montagna 1993 – 102+35+19-32-18 GeV
- Ellis 1993 – 132+20-22 GeV
- Blondel 1993 – 143+19-18 GeV
- Novikov 1994 – 161+15+16-16-22 GeV
- Montagna 1994 – 174+11+17-13-18 GeV
- Gurtu 1994 – 177+-9+16-20 GeV
- Ellis 1994 – 140+21-22 GeV
These other “global” fits give a more concrete picture of the situation in the years 1993-94. One must keep in mind that in 1993 the lower limit from experimental searches by CDF and D0 was growing almost by the day, and indeed some of the determinations above suffer from having incorporated that limit in their fits. Others were just influenced by the results, but until late 1993 nobody had real information on the “real” value of the top mass from direct determinations at the Tevatron (see plot above, where the light blue shading shows a consistent increase in the lower limit at 95%CL from CDF and D0 in those years). Indeed, before 1993 no determination was above 150 GeV, while from 1994 on all results seemed to align close to 170 GeV.
Can we cast doubt on the good faith of these scientists ? For sure not. They did their homework the best way they could. If some used the experimental lower limits in their fits, they did mention it in their papers, and that is perfectly legal. More subtle is the case when they chose not to use the lower limits: in that case, a involuntary bias might have been at work. The same can be said of the determinations published in early 1994, when the leaks from the Tevatron about the top mass were hard to avoid.
What is the bottom line ? I think there is nothing wrong in the way we do science in particle physics – the interplay between experimentalists and theoreticians is sane and fruitful. It is important, however, to keep a critic eye open when looking at results. That’s for sure!