Several important discriminating variables have been introduced above. We complete the list by briefly mentioning several other kinematic variables of potential interest:
• the transverse momentum of the selected lepton,pT,`;
• various angular separations (∆R or ∆φ) between the lepton, the jets, and the missing transverse momentumETmiss;
• the scalar sum of the leading four selected jets’ transverse momenta, HT4 jets“ř4
i“1|p~Tjet,i|;
• theETmiss significanceETmiss{ b
HT4 jets; and
• the effective mass m4 jetseff “HT4 jets`ETmiss`p`T.
The two SRs targeting the mixed and three-body stop scenarios were defined in an iterative approach. The insights obtained from the study of individual variables as presented above was used as a starting point for the automated procedure defined in the following. The outcome of this procedure was in turn used as a starting point for further checks and adjustments before applying the automated procedure again; after a few iterations, the resulting SR definitions were adequate to proceed.
For each considered variable, a list of reasonable cut thresholds was defined, based on simulated distributions of the signal and background processes. The variables and their thresholds can be pictured on a grid as shown in figure 5.12. In this picture, a SR definition corresponds to the choice of up two points in each row, a lower and possibly an upper threshold: the preselection is shown with red markers as an example. Open line endings indicate that the corresponding cut is optional, a cross at a line ending indicates that introducing that cut is forbidden, e.g. while an upper limit on the lepton pT might be reasonable for the soft kinematics of a three-body decay, this would have lead to an overlap with a concurrent analysis effort expressly focusing on leptons with low momentum. Similarly, requiring theτ veto to fail (by introducing aă1requirement) would have let to selections enriched in dileptonic events with a hadronically decayingτ. Once the grid definition and a starting selection have been defined, the automated procedure performs the following steps:
1. evaluate the current selection on simulated signal and background samples to find expected yields S and B;
2. compute a figure of merit for the current selection:
• For looser selections, a signal significance estimate is used, defined as S{?
S`B`∆B2, assuming a flat relative systematic uncertainty on the background;
• OnceS{B exceeds0.2, orS drops below 15 events, theCLs method [179, 180]
is used2 is used, with a flat relative systematic uncertainty on the expected yields, and without the data-driven background normalisation described later.
3. All neighbouring selections are found and their figures of merit are computed;
2This choice of goal function implies the SR will be optimised for its exclusion reach, not for discovery potential.
3 4
100 120 140 160 180 200 220 240 260 280 300 320 340 360
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290
250 275 300 325 350 375 400 425 450 475 500 525
250 300 350 400 450 500 550 600 650 700 750 800
90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 130 150 170 190 210 230 250 270 290 310 330 350 370 390
0 1
Figure 5.12: Illustration of the optimisation grid underlying the final stages of the signal region definition. The red markers show how the preselection is represented on this grid.
4. the best performing neighbouring selection is kept; and
5. the procedure terminates if no improvement is found, otherwise it is repeated from step 1.
Neighbouring selections are those for which only a single cut is tightened or loosened by one unit on the grid, or an optional cut is introduced (or possibly removed) at its loosest setting. In this manner, at most 4 neighbours per variable need to be evaluated;
in practice this number is smaller, as not all possible lower or upper cuts are enabled.
Every five iterations and also after the final iteration, a summary plot is created, presenting the expected background composition and CLs exclusion reach; the two summary plots for the final SR definitions are shown in figure 5.13. This type of summary plot has been used as a basis for adjustment to the grid, and eventually also for selecting the most suitable SR definitions.
While working with it, the technique has been refined in several ways:
• If a promising selection is found, it is tested for stability: iteratively, one cut at a time is removed, and the optimisation is rerun to test whether it converges to the same selection; this mitigates some cases of over-tuning the selection to statistical fluctuations of the simulated samples.
• If the optimisation cannot find any improvements, it randomly changes several cuts by a few steps, and resumes; thismutation sometimes helps to escape local minima.
• Optimisations were run alternately on signal models with different stop and neu-tralino mass assumptions; this helped finding selections that perform well across extended stretches of the mass-plane.
• To improve the available signal statistics, signal models corresponding to adjacent points on the mass-plane are simply evaluated as “combined models”. Although this introduces a slight bias towards events from lower stop mass points (due to their higher production cross-section), the reduced statistical uncertainties result in an overall improvement.
• Attempts were made to reduce the bias towards very tight selections and the over-tuning this may cause: ifSfalls below 5, it is multiplied by a penalty termexppS{5´
1q. Similarly, ifB falls below 5, it is divided by exppB{5´1q.
• A significant speed-up was achieved bycaching andinterpolating CLsvalues where possible, always computing them accurately if the estimate is near the current best.
The definition of the step-sizes on the grid is crucial: for this thesis, they were defined manually and were adjusted several times to balance the risks of over-tuning and missing interesting selections. If they are not well balanced between different variables,
stop mass (GeV)
neutralino mass (GeV)
a)
stop mass (GeV)
neutralino mass (GeV)
b)
Figure 5.13: Visual representation of the SR performance for the definitions given in table 5.4. The upper pie chart shows the expected SM background yields, and the lower shows the dominant types of selected top-antitop events. TheCLsmap shows an estimate of the expected exclusion reach, points withCLs ă0.05 are within reach and are shown in green. a)SR for mixed stop decays,b)SR for three-body stop decays. For the latter, themT andamT2 cuts were loosened by 10 GeV each, as the SR is too tight to be used on its own; this will be discussed further in section 5.7.
some threshold changes may have relatively little impact and will be ignored by the optimisation, or picked up too late to guide the procedure in the best possible way. The grid definition was manually adapted whenever such an effect was observed, and the problem was mitigated to some extent by also considering “neighbouring” selections that are several steps away (e.g. exploring ETmiss ą 120,130,140, . . . GeV if the change from ą100to ą110resulted in an improvement). A possible improvement to be explored in the future is a dynamical definition of the step-sizes, based for example on quantiles of the current distributions.
The method has similarities to a simulated annealing optimisation: it should be explored if the convergence behaviour improves when not all neighbouring selections are evaluated, but instead a random neighbour is evaluated and selected with a probability that depends on its performance as well as the current “temperature” of the system. At high “temperature”, a worse performing selection may also sometimes be selected. The system is then slowly “cooled” as the optimisation progresses. This strategy avoids local minima more elegantly than randomly loosening cuts if no improvement is possible.
After several iterations, suitable SR definitions for the three-body and mixed stop decay scenarios were found. In a final clean-up iteration, cuts with low impact or unclear physical interpretation were removed from the grid, to keep the definitions as simple as possible. Table 5.4 shows the final definitions, and the expected background composition is shown in table 5.5 (see also figure 5.13). For the most part, the SR definitions are intuitively understandable, while the interpretation is less clear for some cuts, for example the mjjj requirement discussed above. The tight amT2 ă 90 GeV requirement for the three-body SR is a consequence of the amT2 definition (section 5.3.2): by construction we have mW ďamT2Àmt; an off-shell top quark reduces the available range and leads to an enhanced S{B near the strict lower bound. The fact that only a small part at the edge of the amT2distribution is selected requires special attention (section 5.7).