Difference: AnalysisVersion1405006TMVA (1 vs. 13)

Revision 13
16 Oct 2009 - Main.TomRockwell
Line: 1 to 1
Changed:
<
<
META TOPICPARENT name="Trash.Tier3WebHome"
>
>
META TOPICPARENT name="Trash.Trash/Tier3WebHome"
 

TMVA for Version 14.05.006

Ranking the Classifiers

Revision 12
13 Oct 2009 - Main.ChipBrock
Line: 1 to 1
Changed:
<
<
META TOPICPARENT name="WebHome"
>
>
META TOPICPARENT name="Trash.Tier3WebHome"
 

TMVA for Version 14.05.006

Ranking the Classifiers

Revision 11
04 May 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

TMVA for Version 14.05.006

Line: 75 to 75
  2 signal events were used for signal validation and 2 background event were used for background validation. This was achieved by the following line in TMVAnalysis.py:
Changed:
<
<
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=10000000000:NBkgTrain=100000000000::NSigTest=2:NBkgTest=2:SplitMode=Alternate:NormMode=NumEvents:!V" )
>
>
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=113:NBkgTrain=1121::NSigTest=2:NBkgTest=2:SplitMode=Alternate:NormMode=NumEvents:!V" )
  Using 0 events for NSigTest and NBkgTest causes the program to crash. It runs successfully with 1 event but this leads to a division by 0 in the significance calculation below.
Added:
>
>
Note that NSigTrain + NSigTest and NBkgTrain + NBkgTest must equal or less than the actual number of signal and background events, respectively, in the root file. If a number larger than the actual number of events is used the program dies. If an absurdly large number, such as 10000000, is used, the program uses an equal number of training and testing events.
 

TMVAnalysis outputs the root file TMVAout.1405006.root and information in the weights directory.
Revision 10
16 Apr 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<

Analysis Comparisons for Code Version 14.05.006

Run Over D3PDs using the MSU analysis code

The DPDPs are stored on the CERN cluster in the directory /rooms/living/ntuples/single_top/TopPhysDPDMaker/14.05.006/ The following samples were included in the analysis:

Signal: 5500, 5502 (note that there is no 5501 sample for v14.05.006)

Background: 5200, 5204, 6280, 6281, 6282, 6283, 7680, 7681, 7682, 7683, 7684, 7685, 7690, 7691, 7692, 7693, 7694, 7695, 7700, 7701, 7702, 7703, 7704, 7705

Unlike v13.0.30, all input files ran without crashing and it was not necessary to comment out any of them in the list files.

The output files from the MSU analysis code are stored in the CERN cluster in the directory /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/ The samples in the directory unweighted were processed without an event weight. The samples in the directory weighted were weighted so that the number of events corresponds to a luminosity of 100 pb-1. The weighted samples were used for this analysis.

Merge Signal and Background Files

This comparison was done for the electrons chain. Note that "Electron" is replaced by "Muon" in the file names for the muon chain.

The merging was done using the !MergeTrees.C routine located in the macros/TreeManipulation directory in the MSU analysis package and the merged files are in the directory /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/.

The 2 signal files were combined into a single signal file called Topology.!SingleTop.1405006.FDR2.Electron.Signal.root. The 24 background files were combined into a signal background file called Topology.!SingleTop.1405006.FDR2.Electron.Background.root. The merged files were then divided into training, validation, and yield samples, which was achieved by setting the Split flag to 1 in the config file. The events were split according to their order in the merged file. For example, the first event was categorized as training, the second as validation, the third as yield, the fourth as training, and so on. Functionality to randomly assign variables to the different categories will be added in the future.

Events with negative weights were excluded from the Training sample to retain compatibility with SPR, which cannot handle negative weights. Validation samples both with and without events with negative weights were made. The files without events having negative weights are signified by the label !NoNeg.

Put Signal and Background Trees into One File

In order to be compatible with the TMVA input requirements, the signal tree from the signal file and the background tree from the background file were placed in one file. This was accomplished by the CombineSigBkg.py routine located in the macros/TreeManipulation directory. This routine is run, for example, as
python CombineSigBkg.py \
   -S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.root \
   -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.root \
   -t TopTree \
   -o /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/CombineSigBkg/Topology.SingleTop.1405006.FDR2.Electron.root \

Variables Used

For comparison purposes, only the following variables were considered for training: HT, Jet1Pt =, =DeltaRJet1Jet2, !WTransverseMass.
>
>

TMVA for Version 14.05.006

 

Ranking the Classifiers

The effectiveness of a classifier is determined by both its Signal Efficiency and Background Rejection. From a plot of Background Rejection vs. Signal Efficiency,
Line: 168 to 123
 
significanceBDT.Hist.Right.png significanceBDT.Hist.Left.png

Comparisons with Jenny

Added:
>
>
The two analyses have complete agreement at this point.
 
compare.png
Line: 180 to 136
 
META FILEATTACHMENT attachment="significanceBDT.Hist.Left.eps" attr="" comment="" date="1236261569" name="significanceBDT.Hist.Left.eps" path="significanceBDT.Hist.Left.eps" size="6694" stream="significanceBDT.Hist.Left.eps" tmpFilename="/usr/tmp/CGItemp1745" user="PatRyan" version="1"
META FILEATTACHMENT attachment="significanceBDT.Hist.Right.png" attr="" comment="" date="1236261603" name="significanceBDT.Hist.Right.png" path="significanceBDT.Hist.Right.png" size="15388" stream="significanceBDT.Hist.Right.png" tmpFilename="/usr/tmp/CGItemp1597" user="PatRyan" version="1"
META FILEATTACHMENT attachment="significanceBDT.Hist.Left.png" attr="" comment="" date="1236261633" name="significanceBDT.Hist.Left.png" path="significanceBDT.Hist.Left.png" size="14451" stream="significanceBDT.Hist.Left.png" tmpFilename="/usr/tmp/CGItemp1704" user="PatRyan" version="1"
Added:
>
>
META FILEATTACHMENT attachment="compare.eps" attr="" comment="" date="1239889321" name="compare.eps" path="compare.eps" size="19845" stream="compare.eps" tmpFilename="/usr/tmp/CGItemp1542" user="PatRyan" version="1"
META FILEATTACHMENT attachment="compare.png" attr="" comment="" date="1239889377" name="compare.png" path="compare.png" size="23357" stream="compare.png" tmpFilename="/usr/tmp/CGItemp1574" user="PatRyan" version="1"
META TOPICMOVED by="PatRyan" date="1239890920" from="AtlasSingleTop.AnalysisVersion1405006" to="AtlasSingleTop.AnalysisVersion1405006TMVA"
Revision 9
13 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 30 to 30
  the fourth as training, and so on. Functionality to randomly assign variables to the different categories will be added in the future.

Events with negative weights were excluded from the Training sample to retain compatibility with SPR, which cannot handle negative weights.
Changed:
<
<
Validation samples both with and without events with negative weights were made. The files without events having negative weights are signified by the label NoNeg.
>
>
Validation samples both with and without events with negative weights were made. The files without events having negative weights are signified by the label !NoNeg.
 

Put Signal and Background Trees into One File

In order to be compatible with the TMVA input requirements, the signal tree from the signal file and the background tree from the background file were placed in one file. This was
Line: 45 to 45
 

Variables Used

For comparison purposes, only the following variables were considered for training:
Changed:
<
<
HT, Jet1Pt =, =DeltaRJet1Jet2, WTransverseMass.
>
>
HT, Jet1Pt =, =DeltaRJet1Jet2, !WTransverseMass.
 

Ranking the Classifiers

The effectiveness of a classifier is determined by both its Signal Efficiency and Background Rejection. From a plot of Background Rejection vs. Signal Efficiency,
Revision 8
13 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 122 to 122
 
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=10000000000:NBkgTrain=100000000000::NSigTest=2:NBkgTest=2:SplitMode=Alternate:NormMode=NumEvents:!V" )
Changed:
<
<
Using 0 or 1 validation events for NSigTest and NBkgTest, instead of 2 validation events, caused the program to crash. I'm not sure why this happens.
>
>
Using 0 events for NSigTest and NBkgTest causes the program to crash. It runs successfully with 1 event but this leads to a division by 0 in the significance calculation below.
 

TMVAnalysis outputs the root file TMVAout.1405006.root and information in the weights directory.
Line: 135 to 135
  Signal:
 python TMVApplication.py \
Changed:
<
<
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.Validation.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.Validation.NoNeg.root
  -o Signal.1405006.root

Background:
 python TMVApplication.py \
Changed:
<
<
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.Validation.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.Validation.NoNeg.root
  -o Background.1405006.root
Revision 7
10 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 73 to 73
  -o "TMVAout.ClassifierRanking.1405006.root"
Changed:
<
<
The program MaxEfficiency.py produces a plot showing Background Rejection vs. Signal Efficiency and ranks the classifiers. The plot and the rankings are shown below.
>
>
The program !MaxEfficiency.py produces a plot showing Background Rejection vs. Signal Efficiency and ranks the classifiers. The plot and the rankings are shown below.
  Both rankings have the methods PDERS, CutsGA, and Fisher as the top 3.

MaxEfficiency.png
Line: 113 to 113
  TMVAnalysis.py was run over the Training sample of the merged root files. The following command was used to execute the python script:
 
python TMVAnalysis.py \
Changed:
<
<
-S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.Training.NoNeg.root -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.Training.NoNeg.root -t "TopTree TopTree" -o TMVAout.1405006.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/CombineSigBkg/Topology.SingleTop.1405006.FDR2.Electron.Training.NoNeg.root -t "TopTreeSig TopTreeBkg" -o "TMVAout.1405006.root"
  2 signal events were used for signal validation and 2 background event were used for background validation. This was achieved by the following line in TMVAnalysis.py:
Revision 6
09 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 19 to 19
 

Merge Signal and Background Files

This comparison was done for the electrons chain. Note that "Electron" is replaced by "Muon" in the file names for the muon chain.
Changed:
<
<
The merging was done using the !MergeTrees.C routine located in the macros/TreeManiuplation in the MSU analysis package and
>
>
The merging was done using the !MergeTrees.C routine located in the macros/TreeManipulation directory in the MSU analysis package and
  the merged files are in the directory /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/.

The 2 signal files were combined into a single signal file called Topology.!SingleTop.1405006.FDR2.Electron.Signal.root.
Line: 32 to 32
  Events with negative weights were excluded from the Training sample to retain compatibility with SPR, which cannot handle negative weights. Validation samples both with and without events with negative weights were made. The files without events having negative weights are signified by the label NoNeg.
Added:
>
>

Put Signal and Background Trees into One File

In order to be compatible with the TMVA input requirements, the signal tree from the signal file and the background tree from the background file were placed in one file. This was accomplished by the CombineSigBkg.py routine located in the macros/TreeManipulation directory. This routine is run, for example, as
python CombineSigBkg.py \
   -S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.root \
   -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.root \
   -t TopTree \
   -o /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/CombineSigBkg/Topology.SingleTop.1405006.FDR2.Electron.root \
 

Variables Used

For comparison purposes, only the following variables were considered for training: HT, Jet1Pt =, =DeltaRJet1Jet2, WTransverseMass.
Line: 57 to 68
  The following command was used to execute TMVAnalysis.py
 
python TMVAnalysis.py \
Changed:
<
<
-S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.root -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.root -t "TopTree TopTree" -o ClassifierRanking.1405006.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/CombineSigBkg/Topology.SingleTop.1405006.FDR2.Electron.root -t "TopTreeSig TopTreeBkg" -o "TMVAout.ClassifierRanking.1405006.root"
 

The program MaxEfficiency.py produces a plot showing Background Rejection vs. Signal Efficiency and ranks the classifiers. The plot and the rankings are shown below.
Revision 5
05 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 143 to 143
  Significance.py was run in order to calculate the significance. For now, the significance was taken as Signal/sqrt(Background). A more accurate, and complicated, calculation of the significance will be performed in the future. Events were weighted by a factor of 3 to account for the splitting into training, validation, and yield samples.
Added:
>
>
Significance.py was run as
python Significance.py -S Signal.1405006.root -B Background.1405006.root -w 3 -o SignificanceOutput.1405006.root
  The classifier output distributions are shown below for signal and background.
BDT.png
Line: 161 to 166
 

-- PatRyan - 03 Mar 2009
Deleted:
<
<
  • Background Rejection vs. Signal Efficiency for various classifiers:
    MaxEfficiency.png
 
META FILEATTACHMENT attachment="MaxEfficiency.png" attr="" comment="Background Rejection vs. Signal Efficiency for various classifiers" date="1236183670" name="MaxEfficiency.png" path="MaxEfficiency.png" size="29318" stream="MaxEfficiency.png" tmpFilename="/usr/tmp/CGItemp1729" user="PatRyan" version="1"
Added:
>
>
META FILEATTACHMENT attachment="BDT.png" attr="" comment="" date="1236261517" name="BDT.png" path="BDT.png" size="15151" stream="BDT.png" tmpFilename="/usr/tmp/CGItemp1720" user="PatRyan" version="1"
META FILEATTACHMENT attachment="significanceBDT.Hist.Left.eps" attr="" comment="" date="1236261569" name="significanceBDT.Hist.Left.eps" path="significanceBDT.Hist.Left.eps" size="6694" stream="significanceBDT.Hist.Left.eps" tmpFilename="/usr/tmp/CGItemp1745" user="PatRyan" version="1"
META FILEATTACHMENT attachment="significanceBDT.Hist.Right.png" attr="" comment="" date="1236261603" name="significanceBDT.Hist.Right.png" path="significanceBDT.Hist.Right.png" size="15388" stream="significanceBDT.Hist.Right.png" tmpFilename="/usr/tmp/CGItemp1597" user="PatRyan" version="1"
META FILEATTACHMENT attachment="significanceBDT.Hist.Left.png" attr="" comment="" date="1236261633" name="significanceBDT.Hist.Left.png" path="significanceBDT.Hist.Left.png" size="14451" stream="significanceBDT.Hist.Left.png" tmpFilename="/usr/tmp/CGItemp1704" user="PatRyan" version="1"
Revision 4
05 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 63 to 63
  -o ClassifierRanking.1405006.root
Changed:
<
<
The Background Rejection vs. Signal Efficiency is shown in the plot below
>
>
The program MaxEfficiency.py produces a plot showing Background Rejection vs. Signal Efficiency and ranks the classifiers. The plot and the rankings are shown below. Both rankings have the methods PDERS, CutsGA, and Fisher as the top 3.
 
Changed:
<
<

The rankings of classifiers is shown in the tables below. Both have PDERS, CutsGA, and Fisher as the top 3.
>
>
MaxEfficiency.png
 

Rank Method
<-- -->
Sorted ascending
Area
Area Under the Curve
Line: 126 to 126
  Signal:
 python TMVApplication.py \
Changed:
<
<
-i /home/root_files/single_top/TopPhysDPDMaker/13.0.30/FDR2/merged/Topology.SingleTop.13030.FDR2.Electron.Signal.Validation.root -o Signal.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.Validation.root -o Signal.1405006.root
 

Background:
 python TMVApplication.py \
Changed:
<
<
-i /home/root_files/single_top/TopPhysDPDMaker/13.0.30/FDR2/merged/Topology.SingleTop.13030.FDR2.Electron.Background.Validation.root -o Background.root
>
>
-i /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.Validation.root -o Background.1405006.root
 

Note that the number of bins in the classifier outputs is 50. This is set by nbin = 50.
Revision 3
04 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 8 to 8
 

Signal: 5500, 5502 (note that there is no 5501 sample for v14.05.006)
Changed:
<
<
Background: 5200, 5204, 6280, 6281, 6282, 6283, 7680, 7681, 7682, 7683, 7684==, 7685, 7690, 7691, 7692, 7693, 7694, 7695, 7700, 7701, 7702, 7703, 7704, 7705
>
>
Background: 5200, 5204, 6280, 6281, 6282, 6283, 7680, 7681, 7682, 7683, 7684, 7685, 7690, 7691, 7692, 7693, 7694, 7695, 7700, 7701, 7702, 7703, 7704, 7705
 

Unlike v13.0.30, all input files ran without crashing and it was not necessary to comment out any of them in the list files.
Line: 32 to 32
  Events with negative weights were excluded from the Training sample to retain compatibility with SPR, which cannot handle negative weights. Validation samples both with and without events with negative weights were made. The files without events having negative weights are signified by the label NoNeg.
Changed:
<
<

Run TMVAnalysis over Training files

>
>

Variables Used

  For comparison purposes, only the following variables were considered for training: HT, Jet1Pt =, =DeltaRJet1Jet2, WTransverseMass.
Added:
>
>

Ranking the Classifiers

The effectiveness of a classifier is determined by both its Signal Efficiency and Background Rejection. From a plot of Background Rejection vs. Signal Efficiency, there are two methods by which to rank the classifiers. The first is by the length of a straight line connecting the upper-right point in the plot to the upper-right axis intersection of (1,1) and the second is by the area under the classifier curve. A shorter line or larger area infers a better classifier.

To produce the file used for classifier ranking, TMVAnalysis.py was run over the merged root files. Note that these are the merged files containing all events, and not those split into training, validation, and yield samples.

In TMVAnalysis.py, half the events were assigned to training and the other half to validation by the line
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=0:NBkgTrain=0::NSigTest=0:NBkgTest=0:SplitMode=Alternate:NormMode=NumEvents:!V" )

The kNN classifier was not used because it generated the following fatal error message
 <FATAL> KNN            : kNN result list is empty or has wrong size

The following command was used to execute TMVAnalysis.py
 
python TMVAnalysis.py \
   -S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.root \
   -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.root \
   -t "TopTree TopTree" \
   -o TMVAout.ClassifierRanking.1405006.root \

The Background Rejection vs. Signal Efficiency is shown in the plot below

The rankings of classifiers is shown in the tables below. Both have PDERS, CutsGA, and Fisher as the top 3.

Area Under the Curve
Rank Method Area
1 PDERS 0.668661
2 CutsGA 0.659969
3 Fisher 0.643790
4 LikelihoodPCA 0.535585
5 SVM_Gauss 0.528220
6 BDT 0.493489
7 Likelihood 0.491559
8 BDTD 0.490003
9 MLP 0.434129
10 HMatrix 0.413259
11 RuleFit 0.393389
12 FDA_MT 0.355496

Length of Line to Upper Right Corner
Rank Method Length
1 PDERS 0.475490
2 CutsGA 0.538328
3 Fisher 0.538659
4 BDT 0.609361
5 LikelihoodPCA 0.633015
6 Likelihood 0.633601
7 SVM_Gauss 0.675473
8 MLP 0.708466
9 BDTD 0.740976
10 HMatrix 0.744161
11 FDA_MT 0.783945
12 RuleFit 0.793519

Run TMVAnalysis over Training files

  TMVAnalysis.py was run over the Training sample of the merged root files. The following command was used to execute the python script:
 
python TMVAnalysis.py \
Line: 44 to 108
  -t "TopTree TopTree" -o TMVAout.1405006.root
Changed:
<
<
Other methods will be used in the future. Also, 1 signal event was used for signal validation and 1 background event was used for background validation. This was achieved by the following line in TMVAnalysis.py:
>
>
2 signal events were used for signal validation and 2 background event were used for background validation. This was achieved by the following line in TMVAnalysis.py:
 
Changed:
<
<
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=10000000000:NBkgTrain=100000000000::NSigTest=1:NBkgTest=1:SplitMode=Alternate:NormMode=NumEvents:!V" )
>
>
factory.PrepareTrainingAndTestTree( mycutSig, mycutBkg, "NSigTrain=10000000000:NBkgTrain=100000000000::NSigTest=2:NBkgTest=2:SplitMode=Alternate:NormMode=NumEvents:!V" )
 
Changed:
<
<
Using 0 or 1 NTest events instead of 2 events caused the program to crash. I'm not sure why using only 1 caused it to crash.
>
>
Using 0 or 1 validation events for NSigTest and NBkgTest, instead of 2 validation events, caused the program to crash. I'm not sure why this happens.
 

TMVAnalysis outputs the root file TMVAout.1405006.root and information in the weights directory.
Line: 96 to 160
 

-- PatRyan - 03 Mar 2009 \ No newline at end of file
Added:
>
>

  • Background Rejection vs. Signal Efficiency for various classifiers:
    MaxEfficiency.png

META FILEATTACHMENT attachment="MaxEfficiency.png" attr="" comment="Background Rejection vs. Signal Efficiency for various classifiers" date="1236183670" name="MaxEfficiency.png" path="MaxEfficiency.png" size="29318" stream="MaxEfficiency.png" tmpFilename="/usr/tmp/CGItemp1729" user="PatRyan" version="1"
Revision 2
04 Mar 2009 - PatRyan
Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Analysis Comparisons for Code Version 14.05.006

Line: 17 to 17
  number of events corresponds to a luminosity of 100 pb-1. The weighted samples were used for this analysis.

Merge Signal and Background Files

Changed:
<
<
The 3 signal files were combined into a single signal file called Topology.!SingleTop.13030.FDR2.Electron.Signal.root. The 3 background files were combined into a signal background file called Topology.!SingleTop.13030.FDR2.Electron.Background.root. Note that "Electron" is replaced by "Muon" for the muon chain in the file names.
>
>
This comparison was done for the electrons chain. Note that "Electron" is replaced by "Muon" in the file names for the muon chain.
  The merging was done using the !MergeTrees.C routine located in the macros/TreeManiuplation in the MSU analysis package and
Changed:
<
<
the merged files are in the directory /home/root_files/single_top/TopPhysDPDMaker/13.0.30/FDR2/merged/.
>
>
the merged files are in the directory /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/.
 
Changed:
<
<
The merged files were divided into training, validation, and yield samples, which was achieved
>
>
The 2 signal files were combined into a single signal file called Topology.!SingleTop.1405006.FDR2.Electron.Signal.root. The 24 background files were combined into a signal background file called Topology.!SingleTop.1405006.FDR2.Electron.Background.root. The merged files were then divided into training, validation, and yield samples, which was achieved
  by setting the Split flag to 1 in the config file. The events were split according to their order in the merged file. For example, the first event was categorized as training, the second as validation, the third as yield, the fourth as training, and so on. Functionality to randomly assign variables to the different categories will be added in the future.
Line: 38 to 39
  TMVAnalysis.py was run over the Training sample of the merged root files. The following command was used to execute the python script:
 
python TMVAnalysis.py \
Changed:
<
<
-S /home/root_files/single_top/TopPhysDPDMaker/13.0.30/FDR2/merged/Topology.SingleTop.13030.FDR2.Electron.Signal.Training.NoNeg.root -B /home/root_files/single_top/TopPhysDPDMaker/13.0.30/FDR2/merged/Topology.SingleTop.13030.FDR2.Electron.Background.Training.NoNeg.root
>
>
-S /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Signal.Training.NoNeg.root -B /home/root_files/single_top/TopPhysDPDMaker/14.05.006/EarlyData/merged/Topology.SingleTop.1405006.FDR2.Electron.Background.Training.NoNeg.root
  -t "TopTree TopTree"
Changed:
<
<
-o TMVAout.root
>
>
-o TMVAout.1405006.root
  Other methods will be used in the future. Also, 1 signal event was used for signal validation and 1 background event was used for background validation. This was achieved by the following line in TMVAnalysis.py:
Line: 50 to 51
  Using 0 or 1 NTest events instead of 2 events caused the program to crash. I'm not sure why using only 1 caused it to crash.
Changed:
<
<
TMVAnalysis outputs the root file TMVAout.root and information in the weights directory.
>
>
TMVAnalysis outputs the root file TMVAout.1405006.root and information in the weights directory.
 
Changed:
<
<
The text output from running the program can be found here: AnalysisTxt13030
>
>
The text output from running the program can be found here: AnalysisTxt1405006
 

Run TMVApplication over Validation files

TMVApplication.py was run over the Validation samples (with events having negative weights). Signal and background samples were run separately using the
 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback