MIPP Batch Processing

Introduction

Offline processing of MIPP data happens on the fngp-osg farm at Fermilab. The software is also capable of being run on other grid sites.
Processing is split into a number of passes.

Raw2root converts raw data to ROOT format.
Pass 1 performs basic calibrations:
- PMT pedestals
- Time offsets for Trigger PMT's (LRS3377 TDC's)
- Time offsets for T0 counters
- Time offsets for BC/DC wires
- Bad channel search in PMT's and chamber wires
Chamber alignment
- Performs run-by-run alignment of BC's, DC's, and PWC's
Pass 2 performs calibrations that depend on tracking and results of pass1:
- Beam Cherenkov time offsets and gains
- TPC drift velocity
Pass 3 performs calibrations that depend on Pass 2 results. The calibrations, track finding, and vertexing are refined in further passes.

More Information

A table Summary of past, present and future reconstruction passes gives more details on the passes.
While a pass is in progress, you can find the Batch Processing Progress on the Web! This page gets data from the database.
Go here to read about setup of FNAL farm and Enstore/dCache.
Read here about running MIPP software on the grid.
The check-list for starting a production pass gives details for the MIPP grid manager.
Other users can submit user jobs.

Data to be processed in Round 3

2985 runs, 40.9M events (16.8M on K-mass)

Selection criteria:

Runs after 11000
At least 500 events per run
Pulser turned off, non-zero trigger word setup
Trigger and BC readout turned on

PSQL criterion string is

WHERE where trigword&x'1000'::int=0 and detectorword&3!=0 and nevent>500 and trigword&x'FCFFF'::int!=0

The list of runs was selected into table runs_r3 in batchproc database for future reference.
Click here for summary of runs or here for a detailed run list.

Tracking progress

Summary of processing is written to the batchproc database, where raw2root, pass1, pass2, and pass3 are defined. All of these tables inherit from table bp whose structure is:


Column name	Type	Meaning
run	integer	Run number
subrun	integer	Subrun number
site	text	The site where file was processed (FNAL or LLNL)
host	text	Host name of machine that did the processing
blkin	integer	Input file size in number of blocks (1024 bytes)
blkout	integer	Output file size in number of blocks (1024 bytes)
histout	integer	Output histogram file size
datestart	timestamp with time zone	Time processing began
dateend	timestamp with time zone	Time processing was done
exitstatus	integer	Exit status of the job (0 if everything is ok)
round	integer	Processing round
release	text	Software release that was used

Andre Lebedev, Holger Meyer