Bumps part V – epilogue and a discussion of the publication of controversial results February 5, 2007Posted by dorigo in personal, physics, politics, science.
In the last part of this post, I discussed in detail the extraction of the true significance of the dimuon mass bump that Paolo Giromini had found in search for a bound state of a pair of scalar quarks. His search had not been a shot in the dark: indeed, it was motivated by a puzzling set of 13 events in the top quark sample, which could have no evident explanation in terms of known detector effects or physics; and by the observation of a large excess of jets with multiple leptons in a QCD sample used for calibrations of the b-tagging rate. A tentative explanation of these two observations -not the most economical, but still a plausible one- was the existence of a light scalar quark.
The long string of controversial results found by the Frascati group had started with the top pair cross section measurement in 1997, had continued with the impossible 13 “superjet” events, had become really hot with the additional excess of multi-lepton jets, and had reached its climax with the dimuon bump. It was simply too much meat on the fire, at a time when most of the efforts in CDF was not aimed at understanding the 5-years-old Run I data on which all those results were based, but rather at commissioning the CDF II detector in time for the Tevatron’s Run II.
Things started to deflate slowly, as the godparents of the top pair cross section admitted that the Frascati analysis had been extremely well done, and that the new, more precise result deserved publication. That paper was published only in 2001, after some additional barrage fire by a few collaborators who had become true opposers of Giromini – eventually, they had shot all their bullets, and had to admit defeat.
At the same time, the godparent committee I was part of (the “oversite” committee) had the difficult task to address the controversial new physics claims by Frascati. We decided to dismember the three analyses by dividing them in smaller chunks, allowing for an easier digestion by CDF, forcing the strong opposers of Giromini’s works to expose themselves.
First, the 13 events were presented in a draft of a Physical Review D paper which put forward just the facts, bargaining with Giromini that a separate paper would be allowed on the interpretation. This plan was fiercely opposers by the known few, and was the source of more endless debates, but in the end it won over. The paper called “Study of the heavy flavor content of jets produced in association with W bosons in ppbar collisions at sqrt(s)=1.8 TeV” was finally published as a PRD in 2002, with all but a few Collaborators’ names on it (CDF has a policy whereby anybody can remove their name from a paper in publication, if they feel like it). And a second paper, “Additional studies of the probability that the events with a superjet observed by CDF are consistent with the SM prediction“, still signed by “CDF Collaboration” but actually only containing the names of 20-or-so collaborators (me included) was published back to back with the first.
By that time, the godparent committee was itself dismembered, and the controversy slowly died out, as the focus of everybody became the analysis of Run II data.
Well, not the focus of everybody, actually: a third paper on the interpretation of the 13 events put forward by Giromini -the third part of the original article he had intended to write on the superjet events- only made it to publication on PRD after three more years of iterations with the referees of the journal. It was finally published as “Phenomenological study of the atypical heavy flavor production observed at the Fermilab Tevatron” in PRD 73 (2006), 014025, with only the names of the Frascati group members. And a fourth paper was published a few months earlier as PRD 72 (2005) 072002, as “Study of sequential semileptonic decays of b hadrons produced at the Tevatron“, again only with Frascati group member names.
And what about the dimuon bump ? Well, that one paper also made it to PRD in the end, as “Search for narrow resonances below the Upsilon mesons“, PRD 72 (2005) 092003. And I signed that one as well, although in the paper my evaluation of 3.2-sigma was made a bit happier by a more handwaving argument that estimated it at 3.5 sigma instead: I did not take my name off for just a small controversy with Paolo Giromini!
To conclude this saga, let me make a point about the diffusion of scientific results.
I have to insist here on a position I held back then, and I still subscribe to, which was a source of fierce debates in the internal meetings in CDF when the superjet events were discussed. My question, simply put, is the following. What is the point if a collaboration of 500 physicists scrutinizes a result, dissects it, finds nothing wrong with it, and still decides to avoid publishing it “because it is controversial” ? What exactly is the rationale of “hiding” this controversial piece of knowledge from the rest of, what, 5000?, interested readers in the world ?
The question is not pointless: what is our mission as scientists: the diffusion of knowledge, or the filtering of results which can be accepted by the community without putting the “reputation” of the collaboration at stake ?
Many in CDF felt those 13 event could not be published because their kinematical characteristics, which showed they were really non-Standard Model-like, was too inconclusive (no model fit the data, not SM nor anythink fancier, unless you really married weird models for which no phenomenological description held water) and yet too controversial. My point was the opposite: you are not convinced the analysis results are correct ? Well then, too bad: let the world know about them. What is it that we lose ? Our virginity, our name, the brand name of a collaboration that makes sound analyses and never publishes wrong results ? Should we protect the outside scientific community from claims that are not perfectly clear and understood ? Should we protect our name, our careers, our grants, because we feel our results could be criticized, or our scientific reputation questioned ?
When you get to the point of discussing whether to let out a result that you could not disprove, to the best of your efforts, but which you do not understand, then you are crossing the line. Whether that result is right or wrong is not the point. You are trying to do something which is not science any more, but rather politics.
I am for the advancement of science, much less for the advancement of scientific careers.