Discovering new physics at LHC: a paper worth a close look

I just finished reading an enlightening paper by Michelangelo Mangano, a theorist at CERN and longtime colleague in the CDF experiment. Michelangelo is one of the few theorists who have consistently helped CDF from the inside, falling just short of pulling cables during Run II commissioning but other than that doing his share of dirty work as a full collaborator.

Michelangelo’s insight in the problems and the pitfalls of extracting a signal of new physics in hadron collider data is well known to me, since the time when we were both members of the oversight committee charged with assessing the soundness of Paolo Giromini’s analyses of the “superjets” – 13 weird events isolated in a sample of b-tagged W+jet events collected by CDF during Run I (I described the saga of the superjets and the following signal of a scalar quark in a series of posts about a year ago). Working with him in that committee did teach me one thing or two back then.

The paper (titled “Understanding the Standard Model, as a bridge to the discovery of new phenomena at the LHC“) contains no formulas -everybody can read it and understand it. It is a refreshing read, which tries to put in perspective what it will mean to “discover new physics” with CMS and ATLAS.

Michelangelo in fact focuses on the very incremental process whereby data is analyzed, gradually better understood as the combination of known effects, to the point when discrepancies with the models arise, and an interpretation is required.

Hic sunt leones: a discrepancy is not a sign of new physics, regardless of whether there are indications that this or that model, cunningly tailored around the effect or unsuspectingly formulated years before, explains the discrepancy perfectly. Not until every possible mundane explanation has been ruled out. Michelangelo makes the point quite clearly:

“As we move away from the default Higgs scenarios, into the territory of new physics beyond the SM, life becomes more difficult. One should think of two phases for a discovery: establishing the deviation from the SM, and understanding what this deviation corresponds to. It is crucial to maintain these two phases separate. The fact that a given anomaly is consistent with one possible interpretation does not increase its significance as an indication of new physics. If we see something odd in a given final state, it is not by appealing to, or freshly concocting, a new physics model that gives rise to precisely this anomaly that makes the signal more likely or more credible. The process of discovery, namely the detection of a deviation from the SM by more than, say, 5 standard deviations of combined statistical and systematic uncertainties, should be based solely on the careful examination of whether indeed the signal violates the SM expectation. Assigning this discrepancy to a slot in the space of possible BSM scenarios is a subsequent step.”

In the paper Mangano considers different examples of where a discrepancy might build up. Mass peaks are of course one of the clearest indications of the creation of a new particle or final state, but anomalous shapes of kinematical distributions are no less intriguing to the hunting physicist. Indeed, one of the very first things that many will look in LHC data will be the distribution of missing transverse energy – a possible “smoking gun” for the production of supersymmetric particles. Here is a sobering cautionary remark:

“There is no question, therefore, that unless each of the background components can be separately tested and validated, it will not be possible to draw conclusions from the mere comparison of data against the theory predictions.

I am not saying this because I do not believe in the goodness of our predictions. But because claiming that supersymmetry exists is far too important a conclusion to make it follow from the straight comparison against a Monte Carlo. One should not forget relevant examples from the colliders’ history […]”.

Then a third “clear” signature of new physics is considered. The murkiest: counting experiments. In a counting experiment one relies solely on the numerical excess of the data in a given region of phase space, and in that, they are similar to searches for shape anomalies. Here a positive example is given, only for the purpose of showing just how complex it is to use with confidence a numerical excess as an indication of a discovery: the case of the top quark discovery. Here, a number of checks were necessary in order to make the excess rise from the status of a mere anomaly to that of a genuine new signal. But here too lies a caveat. Michelangelo compares the observation of the particle, which was based on just one, very striking, event, with an anomalous event collected by CDF in Run I, the infamous – one which has a probability of fractions of one part in a million to be due to known processes.

“In terms of pure statistics, the is (still today, after 30 times more luminosity has been collected by CDF and D0) even more significant as a deviation from the background (whether caused by physics or instrumental) than the first W obsertvations at UA1 and UA2. Why do we not consider it as evidence of new physics ? Because consensus built up in the community that, in spite of the ““, the evidence is not so compelling. On one side plausible BSM [Beyond the Standard Model] interpretations have been ruled out by the LEP experiments […]. On the other, doubts will always remain that some freaky and irreproducible detector effect may be at the origin of this event.”

Michelangelo is optimistic on the way the review process works and prevents incorrect claims from being accepted by the physics community in large experiments:

“All apparent instances of deviations from the SM emerged so far in hadronic or leptonic high-energy collisions have eventually been sorted out, thanks to intense tests, checks, and reevaluations of the experimental and theoretical systematics. This shows that the control mechanisms set in place by the commonly established practice are very robust.”

I agree with him… But if I think of the effort it took eight years ago to our oversight committee to perform those intense tests, checks, and reevaluations, in the case of the superjet affair… Well, I hope it will be somebody else’s job this time around. I believe Michelangelo concurs!