QUINDI
a beam diagnostics simulator
QUINDI is a first principles beam diagnostics simulator which calculates the radiative spectrum from a relativistic electron bunch passing through a magnetic array.
*QUINDI is in beta, certain things might be buggy*
- Features -
- Postprocessing with Matlab
- Internal tracking or use external tracking file (TREDI)
When you run QUINDI, a 'Master' node is selected at random from the cluster, and the other nodes are designated as 'Slave' nodes.
The Master node transmits an individual particle's phasespace to a Slave node.
The Slave node is responsible for calculating everything for that particle over the entire detector grid, then it writes the data to an individual node HDF5 file in 'run/' and requests the next particle from the Master.
After all of the particles are processed, the Slave nodes terminate.
The Master node cycles through the node HDF5 files, adds up or concatinates the data, producing a single merged HDF5 file called 'mergedHDF5.h5'.
This file then must be postprocessed in some fashion. We've written a few Matlab scripts, but anything that can read HDF5 could be used.
QUINDI can do it's own tracking based on an input beam file, or it can use a TREDI HDF5 output file for the tracking.
The slave nodes operate in an asymmetrical fashion;
they receive one particle each, at a time, and will calculate the tracking, the fields, and the spectrum before moving on to the next particle.
QUINDI calculates the spectrum along a square detector, which is defined by it's physical size and by how much resolution you desire.
We run it at 15x15 gridpoints for a typical run, and we've run it as high as 28x28 before running into memory issues (this is highly dependent on hardware).
- Hardware/Software Requirements -
QUINDI requires a multiprocessor environment of at least 2 processors.
It was developed on our 24-node Opteron cluster with 24 gigs of memory running Gentoo, and it should work on any similar setup, or even with something as simple as a desktop dual-core CPU.
QUINDI was successfully run in OSX on a MacBook Pro with a Core Duo.
The following software must be installed on all nodes that QUINDI will be run on:
- Installation -
We want to make a fancy autoconf for QUINDI, but for now, all you have to do is 'make'. You might need to edit the Makefile if you installed MPI or HDF5 in a strange location. Everything will be compiled into a single executable called 'quindi'.
- How to Run -
The easiest way to run QUINDI is with 'mpirun', which is part of MPICH. Issue a command like:
mpirun -np 24 quindi main.in
The '-np 24' part is the number of nodes you are using. The master node needs to be included in this number. The 'quindi' part is what you want mpirun to run, and the 'main.in' part is the name of the main input file you want to use.
We also use Sun GridEngine for job queuing, but this is not required for QUINDI to work. The included script 'sge.sh' can be used if you change the paths, and this can be submitted via the 'qsub' command.
- Main Input File -
The main input file contains most of the input parameters you will want to adjust. The main input file can include some or all of the input parameters.
Lines beginning in '//' are ignored.
If a line is omitted, the default value shown in () will be used. The units are in [] where applicable:
r0(x,y,z),r1(x,y,z),r2(x,y,z): Location of the target, as 3D vectors with corresponding cartesian coordinates x, y, and z. Each of the 9 coordinates gets it's own line. [m]
roundtarget: Setting this to 1 causes the target to be elliptical with axes given as n1 and n2 (default: 0)
n1,n2: Number of detector points on target in two dimensions. Note: SpecGUI postprocessor requires n1=n2.
singletarget: Setting this to 1 forces the target to be a single point. (default: 0)
singleparticle: Setting this to 1 forces the input beam to be a single reference particle, i.e. beamfile=singleparticle.sdds (default: 0)
freq_cut: Spectrum calculation ranges from 0 Hz to this frequency. (default: 1e14) [Hz]
freq_sam: Number of frequency samples for spectrum calculation. (default: 100)
dz: Minimum distance step for calculation. When whotracks=tredi, this parameter is the interpolated stepsize. (default: 20e-6) [m]
timeToDump: Time when final phase space of bunch is dumped (default: 9e-9) [s]
magnetfile: Filename of magnetic lattice input file
magdimension: Dimensionality of the magnetic lattice, currently 1 or 2 (default: 2)
beamfile: Filename for the input beam distribution
beamcap: This will limit the total number of particles in the beam. Setting to 0 will process entire beam (default: 0)
columns: Number of columns in beamfile, either 6 or 8. (default: 6)
outputdir: Path for merged output file. Always include the trailing / and directory must already exist (default: ./)
whotracks: Who is doing the tracking, quindi or tredi (default: quindi)
trackfile: If whotracks is set to something other than quindi, this is the external file used for tracking
Here is an
example main input file.
- Beam File -
The parameter 'beamfile' describes the file that contains the input beam distribution (only used when 'whotracks=quindi'). This file should have 6 columns in the following format:
x-position y-position z-position x-momentum y-momentum z-momentum
The columns should be space delimited, with one line per particle. If you have one of those bastard files with two extra columns for time and charge, you can set the 'columns' parameter to 8 to ignore them.
Here is an
example 3-particle beamfile.
- Magnetic Lattice Input File -
The parameter 'magnetfile' describes the file with the information for the magnets the beam is sent through. Each line in this file represents a single magnet. QUINDI supports 1-dimensional and 2-dimensional magnetic lattices. A line in a 1D magnetic lattice file will look like this:
bend: L=0.5; Zpos=1.25; Bmax=0.8; Lfringe=0.025;
The first thing on each line should be the type of magnet, followed by a colon. Right now, the only choice is "bend". After this is four pieces of information, the units in []:
L: Length of magnet, not including the fringe [m]
Zpos: Absolute position of the magnet, from the origin to the closest edge [m]
Bmax: Peak field strength of magnet [T]
Lfringe: Length of the magnet's fringe [m]
Each parameter should have the parameter name, an equal sign, the value, and a semicolon at the end.
A line in a 2D magnetic lattice file looks like the following:
bend: L=0.41; Zpos=0.; Bmax=-0.2015; Lfringe=0.0185; Offset=0.; Angle1=21.; Angle2=0.; Height=0.2;
The 2D lattice has all the information of the 1D, plus the additional parameters:
Offset: Offset of the center of the magnet compared to the incoming beam [m]
Height: Height of the magnet [m]
Angle1: Angle of the incoming face of the magnet [deg]
Angle2: Angle of the outgoing face of the magnet [deg]
Angle1 and Angle2 are the angles formed by the faces compared to the line z=const (the beam moves primarily in the z-direction). A negative angle means the face slopes towards the incident beam, while a positive angle means the face slopes away from the incident beam. This is useful if you are modeling a magnet that is trapezoidal instead of rectangular.
- Who Tracks? -
If the parameter 'whotracks' is set to 'quindi', QUINDI behaves as normal and does it's own tracking. If set to 'tredi', QUINDI tracking will be bypassed, and the tracking data in the TREDI HDF5 output file named by the parameter 'trackfile' is used instead. Additionally, with 'whotracks=tredi', the TREDI trajectory will be smoothed out by cubic spline interpolation before being used for the proceeding calculations. In this case, the parameter 'dz' will be used for the target interpolated stepsize.
- QUINDI HDF5 Output File -
Each node will generate it's own HDF5 file, which are merged at the end of the run by the master node into a single HDF5 output file called 'mergedoutput.h5'. This file contains a ton of information, described here:
/info -> various metadata about the run
/info/detector -> coordinates of detector corners (on y-z plane)
/info/magnetic_lattice -> coordinates of magnets (on y-z plane)
/info/input_deck -> input deck of run
/phasespace/final/bx -> final x momentum of particle
/phasespace/final/by -> final y momentum of particle
/phasespace/final/bz -> final z momentum of particle
/phasespace/final/gamma -> gamma of particle
/phasespace/final/x -> final x position of particle
/phasespace/final/y -> final y position of particle
/phasespace/final/z -> final z position of particle
/phasespace/initial/bx -> initial x momentum of particle
/phasespace/initial/by -> initial y momentum of particle
/phasespace/initial/bz -> initial z momentum of particle
/phasespace/initial/gamma -> gamma of particle (same as final)
/phasespace/initial/x -> initial x position of particle
/phasespace/initial/y -> initial y position of particle
/phasespace/initial/z -> initial z position of particle
/target/Gridpoint#/spectralcuts -> frequency cuts for spectum
/target/Gridpoint#/spectrumX/magnitude -> magnitude of spectrum in X
/target/Gridpoint#/spectrumX/phase -> phase of spectrum in X
/target/Gridpoint#/spectrumY/magnitude -> magnitude of spectrum in Y
/target/Gridpoint#/spectrumY/phase -> phase of spectrum in Y
/target/Gridpoint#/spectrumZ/magnitude -> magnitude of spectrum in Z
/target/Gridpoint#/spectrumZ/phase -> phase of spectrum in Z
/target/Gridpoint#/xlocation -> x location of detector point
/target/Gridpoint#/ylocation -> y location of detector point
/target/Gridpoint#/zlocation -> z location of detector point
/trajectory/bdx -> x acceleration of particle
/trajectory/bdy -> y acceleration of particle
/trajectory/bdz -> z acceleration of particle
/trajectory/bx -> x velocity of particle
/trajectory/by -> y velocity of particle
/trajectory/bz -> z velocity of particle
/trajectory/t -> timesteps of particle
/trajectory/x -> x position of particle
/trajectory/y -> y position of particle
/trajectory/z -> z position of particle
If there are multiple particles in the beam, then they are stored serially as arrays in the various datasets. Trajectory only contains information about the very first particle the first node sees. There will be additional Gridpoint# subgroups if the detector has higher resolution.
- Postprocessing -
Postprocessing on the merged HDF5 file can be done with Matlab and the scripts in the "matlab/" directory.
These scripts dissect the merged HDF5 file and create variables for the appropriate things. The scripts have to be modified for the appropriate path to 'mergedHDF5.h5'.
hdf5_ps -- parses the phasespace group
hdf5_traj -- parses the trajectory group
hdf5_info -- parses the info group (mainly for plotting detector and magnets)
There is also a script called SpecGUI in the "matlab/specgui/" directory which is a handy way to view a ton of info about the spectrum using Matlab. In Matlab, the syntax goes like:
specgui('filename')
where filename is the location of the HDF5 merged file (in single quotes). This script only works for square targets!
See the SpecGUI page for more info.
- Questions/Comments/Bugs/Feature Requests -
Please send all questions, comments, and even some bug reports to
David Schiller. He will try to help get QUINDI up and running on your cluster!