A few remarks on Matthew Strassler’s “Flesh and Blood with Multi-Muons”

[I know, I know… I had promised that today I would issue a fourth installment of my multi-threaded post on the multi-muon analysis, and instead this morning (well, that depends where you’re sitting) I am offering you something slightly different: instead than concrete details on the analysis, here is a review of a review of the same. I trust you understand that blogs, like newspapers or magazines, have their own priority lists…]

Last evening I read with a mixture of interest and surprise the paper recently appeared on the Arxiv by Matthew Strassler, a theorist from Rutgers University, and a supporter of so-called “hidden valley” models of physics beyond the Standard Model.

The interest stems from obvious reasons: after CDF published the study on multi-muon events, any discussion of the effect, as much as any tentative explanation -be it a mundane or an exotic one- is worth my undivided attention. And, mind you, let me say from the outset that I salute professor Strassler’s thoughts and considerations as useful and stimulating, and the mechanisms he suggests promising avenues for further research on the subject.

But there’s room for surprise, and not all of it is of pleasant nature.

Some of the surprise comes from a few of the remarks contained in the 20-pages document, and some comes from the way it is written. More on the remarks below, while about the way it is written I can say off-hand that I should probably be grateful to theorists these days, since they have started to make their papers free of complicated formulas, at the expense of a rather large rate of unnecessary adjectives: Strassler’s paper has indeed a remarkable formula count of zero.

In general I feel surprised by reading in an Arxiv paper something one usually finds in a blog: a list of ideas and questions concerning a paper published by a respectable scientific collaboration. It looks like prof. Strassler does not have a blog, and so he uses the Arxiv as a dump of his train of thoughts. Incidentally, this blog is of course open to him for a guest post, if he ever wants to try this kind of arena for his ideas.

I guess my criticism on the style boils down to this: it seems less productive to write an Arxiv paper containing a list of ideas and questions -and quite a bit of criticism-, than just picking up the phone and call the authors of the analysis, as I am told many other theorists are doing these days. No, he apparently has not made the phone call yet. That is quite unfortunate, because if he had he would maybe have learned a thing or two about the CDF analysis beyond what is published, and he would have had a chance to find an answer to some of his questions. Then, his ideas might have gotten some useful input and could have been refined. In his paper, instead, they sometimes read like a laundry list (check for instance pages 18-19, where he has seven bullets of plots he asks CDF to produce).

In his preprint Strassler mentions repeatedly that the multi-muon paper is written by “a subset of the CDF collaboration“. It appears that he stresses this fact on purpose, as if it is a datum of scientific importance. Fortunately he does not go as far as to claim that his observation casts doubt on the results, but his lingering on the issue appears strange, and to me, inappropriate. Calling our publication “a paper by a subset of the CDF collaboration” is plain wrong, because the paper is by the CDF collaboration, regardless of who signs it. The collaboration is one, and it is more than a collection of individuals: it admits no subset. I know theorists are much more promiscuous in the way they associate and disperse in different author lists; but a collaboration is a collaboration, and once a member, you only get to decide whether to sign or not a paper, but the collaboration publishes, not you.

This matter is important, so maybe I need to stress it once more. Let me remind everybody that the multi-muon analysis is a CDF publication, and that the CDF collaboration stands by this paper just as much as it stands by every other one of the half thousand it has published in its long, illustrious life. Signing a CDF paper is a great privilege, and since prof. Strassler does not know personally all of the people in CDF (I, for one, never had the pleasure to meet him), nor does he know about the internal discussions that have taken place concerning the publication, he should be expected to leave this issue aside, lest he gives the impression of discussing matters he is wholly unqualified to discuss. This impression is set from the very beginning in Strassler’s preprint, and remains in the background throughout its 20 pages, resonating in a few specific spots.

Let me now go into the contents of the “flesh and blood” paper very briefly. I cannot discuss all of it here today, but I will make an attempt at showing a couple of further examples of what I do not like in it, thereby creating a biased view of my overall opinion: the parts I liked will be left out of this post. Also, in the process of showing what I object to, I will be quoting out of context: a rather reproachable conduct, I must admit, but I have no real choice if I want to make this post shorter than the paper it deals with.

So here is the very incipit of the Introduction:

“Very recently, an unknown subset of the CDF collaboration has signed its name to one of particle physics’ most extraordinary papers”.

Well, after thanking prof. Strassler for the unnecessary, improbable adjective, one is left wondering whether he can compute the ratio of small integers, like 370/600. But, at least until we get to read about his cross section estimates, we prefer to grant that he can, and so we have to hypothesize that maybe, by “unknown subset” he means to say he does not know the 370 authors who signed the “extraordinary paper”. Paraphrasing Oscar Wilde, “To not know an experimentalist is an accident; to not know 370 is carelessness“. But Strassler does know at least two CDF members: these are two of his Rutgers colleagues, who in fact get thanked in the concluding lines of his paper. Unfortunately, they did not sign the CDF publication. From this one might be tempted to speculate that Strassler only got to hear comments and internal information biased in a particular direction…

Yet prof. Strassler is quite clear to state from the outset he is very interested in the CDF analysis:

“No one would be happier than the author of the present note if this “suggestion of evidence” were to hold up under scrutiny”.

I omit discussing whether I find acceptable or not the way he interprets as a “suggestion of evidence” the conclusions of the CDF study, but I cannot fail to explain that he should rather take a ticket and join the line of happy scientists cheering the discovery of new physics, than single out himself as the one. This is a small bit of immodesty which however, after having noted it, I think we should pardon, given that he has indeed worked on hidden valley models for a long time.

We can also pardon him for saying that the paper is “too short given its potential importance“, right in the next paragraph. On this one count, I think he really manages to stand out of the crowd head and shoulders: of all the comments I have heard about the CDF paper, none went so far as to say that the 70 pages were too few.

Then, a sentence I am still trying to decypher:

“No serious attempt is made to interpret the data. This exercise may well be helpful […] even if the specific results of [1] (and a related attempt at an intepretation by the experimentalists involved [11]) are eventually discredited.”

Does Strassler mean to say that the study in [11] (the interpretation of multi-muon events, by the original authors of the study) was unserious ? Or does he rather mean it is useful to put together interpretations of similar effects even if they end up straight in the waste bin ? That would justify the career of a lot of theorists…

After the above sentences, which are contained in the introduction, we find section II, which is called “Preliminary comments“. Here I am puzzled to find Strassler’s paper wrestling with the number of events quoted in the CDF publication, reaching odd conclusions. Strassler incorrectly quotes 75 picobarns as the cross-section for ghost events: a number which comes out of the blue, and for which my explanation is the following: he uses the number of ghost events, “153895” as he quotes (forgetting this number refers to the subset of “ghost events” passing loose SVX criteria, but of that I can pardon him, he has a thing with subsets), and he assumes this corresponds to 2.1 inverse femtobarns of data. Then, would do the trick: 150k divided by 2k inverse picobarns is indeed 75 picobarns . Is this what he computed ? Well, it is wrong, since the luminosity corresponding to the 153,895 events is 742 inverse picobarns, and not 2.1/fb. See, this is one of the many instances when one cannot help noticing that a phone call before submitting to the Arxiv would have been a good idea. Cross section estimates are best left with experimentalists, otherwise what will we do for a living ?

Also odd is his following remark:

“if the efficiency estimate were in error for a subclass of events, and the efficiency were only, say, 23.4 percent, then the number of ghost events would drop by 1/5”.

Now, please. CDF publishes a paper, it quotes an efficiency (24.4+-0.2%), and it estimates an excess. What do we get if a theorist, albeit a distinguished one, ventures to say that if the efficiency were wrong (by 5-sigma from the quoted value), the excess would be significantly different ? I miss the scientific value of that sentence. Wait, there is more: only a paragraph below he insists:

“For these two reasons, we must view the number of unexplained ghost events as highly uncertain”.

Excuse me: we own the data, we publish an estimate, we give a uncertainty. You may well question whether it is correct or not, but simply saying an estimate is “highly uncertain” without coming down to explain what mechanisms may have caused an error in the CDF determination of the efficiency, is not constructive criticism, and is rather annoying. Not to mention that the CDF publication where the ingredients for the determination of that efficiency were measured is not quoted in Strassler’s paper!

Ok, I think I have done enough commenting for today. To conclude this post, I will quote without commentary a few sentences which I find peculiar. I have to say it: while the CDF paper is not the clearest I have had the pleasure to sign, I feel the need to stand by it when I see it attacked by non-constructive criticism.

• “…the paper[…] is far too short given its potential importance, and many critical plots that could support the case are absent”.

• “No serious attempt is made to interpret the data”.

• “It is not clear why these checks were not performed”.

• “There are a number of other plots whose presence, or absence, in Appendix B of [1] is very surprising. In particular, though obviously presented so as to support the interpretation of [11], the plots in Appendix B do not actually appear to do so.”
• “…the challenges that this analysis faces are useful as a springboard for discussion. Clearly, if there were a signal of this type in the data, it would indeed by quite difficult to find it, and the approach used in [1] is far from optimal.”

After this list of less-than-constructive comments, let me quote Freeman Dyson for a change:

“The professional duty of a scientist confronted with a new and exciting theory (or data) is to try to prove it wrong. That is the way science works. This is the way science stay honest. Criticism is absolutely necessary to make room for better understanding.”

Am I the only one to think Dyson meant constructive criticism ?

UPDATE: version 2 of Strassler’s paper came out on November 17th, a week after version 1. This new version makes no mention at all of the “subset” of CDF authors. I thank Matthew Strassler for realizing this correction was useful.