jump to navigation

The typical tediousness vs interest chart June 6, 2007

Posted by dorigo in humor, physics, politics, science.
trackback

Large collaborations of high-energy physics experiments are sometimes slow in releasing their results for public consumption. Reputations are at stake, and physicists like to check and recheck their results before they sign them off. I have discussed in detail elsewhere on this site the review process of the CDF experiment, which is the only one I know for direct experience. There, I criticized it, because I think it is a bit too baroque. However, CDF has a great reputation of publishing only excellent and correct results, so I should keep quiet and live of the benefits the whole system has provided me: more than 200 publications with my name on them, of which I am really, really proud.

Anyway, coming to the topic of this post: Tony Smith in a comment on the former post asked how long it will take D-zero to publish their searches of MSSM Higgs bosons. Of course I have no clue. However,  my 15-years experience in CDF allows me to discuss the general trends of the tediousness of the review process when studied as a function of the importance of the result that is being published. My discussion should probably apply to any large particle physics experiment. 

There are, in my model, three different trends.

1) On the low interest side, as the relevance of the result approaches zero, there is an asymptote, called “Asymptotic irrelevance“. It is due to the fact that a really superfluous result will encounter such opposition that it will never make it to the public – which is to say, an infinitely long review process.

2) There is then a minimum for analyses done with a method already used in the past, which just use a little bit more statistics and shrink by a small amount the error bars of the previous determinations of the measured physical quantity. In CDF examples of such analyses are those measuring a cross section for a well-known phenomenon. Nobody wastes their time triple-checking things already checked in the past, when the analysis was new, so these results get public quite quickly. I dubbed this region “Routine minimum“.

3) As the interest of the result grows, so does the length of the review process. People want to be sure things are done the right way, techniques are optimized, and in general there is a lot of interest in the collaboration, which drives lots of feedback that the authors need to properly address. Also, there may be in a rare instances a growing envy for the authors by competing groups, which makes things slightly harder for them. One reaches a maximum of the tediousness for the “Flagship analyses”, those for which the experiment was really built: in CDF, it is the case of the top quark mass, for instance: for that reason, the peak of tediousness is called “Flagship pickiness“.

4) For exciting results which exceed the interest of flagship analyses, the trend reverses abruptly. We are here in a region where time is of the essence: a brand new result on a quantity never measured before, the observation of a new particle. In these cases, the collaboration works like a single man to get the result out of the door as quickly as possible. It was the case of the observation of oscillations of Bs mesons in CDF recently: despite the incredible complexity of that analysis, the review process was fast – not light, but extremely effective. The regime we approach here is that of a “Nobel urgency“, when it becomes more important to get there first rather than wearing the right tie.

So here is the big picture:

Comments

1. Markk - June 7, 2007

This actually looks quite reasonable. People spend their time where it is most useful. The best managers I ever saw (with high 8 digit budgets) often left a lot of projects on the table after they were started. Talking with one of them, he said it was a constant effort to keep people working on the highest value things. As values changed or the big picture became more clear, there were always things that were left behind. You want to spend resources on the thing you are there for. Your classification looks like a similar winnowing.

2. dorigo - June 7, 2007

Hi Markk,

I agree. I do not find the trend unreasonable, although I think the high points of torment one has to withstand to publish some new analysis are a bit too much at times. The qualitative trend I pictured above does not disturb me in the least.

Cheers,
T.

3. Chase - June 8, 2007

Great picture! I like the four categories of topics, and though you were focused on internal review, I think the same ideas apply for journal reviews also. But, contrary to Markk’s comment I think the graph doesn’t really speak to how well different topics are managed. If anything, it shows a pressure towards the safe results as opposed to the important ones! Especially for people needing to get something published. Something I will think about.

4. jeff - June 8, 2007

Ciao Tommaso. Where does your plot fall on your own plot?

5. dorigo - June 8, 2007

Hi Chase,

thanks… I am actually thinking some student of sociology could build a master thesis around this subject – because there is hundreds of papers from several giant collaborations to study, and each could be represented as a dot in that graph. I have no idea whether my depiction of reality is truthful or biased by my own experience, but what I drew seems likely to me.

Cheers,
T.

6. dorigo - June 8, 2007

Ha, Jeff, when I first read your comment it was in my “comments” column of my admin page, where it is not directly associated to the post – and I started thinking yours was a trick question. Plot in English is a small area in a garden (or cemetery) as well as a graph…

Anyway, my plot took very little to draw, thanks to my mastering of the niceties in xfig. I thought it over for about 10 minutes. So I would say it is at very little value of both x and y. Irrelevant but not too costly in terms of time spent.

Cheers,
T.

7. Markk - June 8, 2007

“But, contrary to Markk’s comment I think the graph doesn’t really speak to how well different topics are managed.”

It’s not the different projects that I was talking about – it’s the collaboration itself. At that level you are looking at a range of things always happening at once, and your job as a manager is to understand what you are really there for and try to apply as much of the effort as possible toward that goal. You will always have things going well, things not going well, surprises etc.

To me this chart shows that the effort is appropriate: tediousness, but that means more certainty for the big, what you’re there for, results; all out hands down quickness and effort for the surprising but important results, you are really just trying to make sure they are important and surprising at that point; less effort for the things you are already confident of; and no effort (so the time -> inf) for the things that don’t matter.

8. Tediousness vs. Interest » Ted Carnahan - June 14, 2007

[…] got done reading this post by a respected high-energy physicist at Fermilab. I find that the same behaviour applies in a lot […]


Sorry comments are closed for this entry

%d bloggers like this: