TWR logo

As mentioned, tests of the the new SIS FPGA program to extract features directly in the TWR were successfully implemented by Wolfgang Wagner in Jan 04.  SIS reported readout of 90MB/sec from a specified VME crate.  Wolfgang measured the time to fill up the internal buffer in the SIS TWR.  For a TWR connected to 8 optical channels, and trigger rate of 180Hz (M=16), the time to fill the buffer was 4-5seconds, and for 8 electrical channels was 0.3 seconds, due to the much larger waveform for electrical channels.    The feature extraction routine implemented by
SIS was configured in the following way:  Threshold was ~150 digits (1.2mV/digit) for electrical and about 20-50 digits for the optical (these are nominal for the standard TWR04 DAQ), and the number of samples prior and following the threshold crossing is 10.  This generates 16 MB/s per fully loaded VME crate, assuming approximately half electrical and half optical channels.

Wolfgang also tested the the readout from SIS TWR through VME-VME bridge to VME to PCI bridge, as expected for the next system.  The readout speed did not change appreciably.  So this too is very encouraging.

So a rough outline of the data flow:
Each crate generates about 16-17MB/s.  VME bus is limited to about 30MB/s so crate traffic is well within specification.
The VME to VME link can transmit 90 MB/s to the master crate, easily keeping up with the data traffic from the VME crate
The DSP in each crate must repack the data format.  The TWR packs two 16 bit digitization from neighboring channels into a 32bit word. With feature extraction, only one of the two channels contains a fragment sample.  The repacking would reduce the data traffic from each crate to
approximately 8-9 MB/s.  This is a major action item since this reprogramming of the DSP has not been done, so no time information is available. Assuming the DSP programming is successful,  the data from three VME crate would be tranferred to one DL380 via the VME to PCI bridge.
For the current generation DL380, the PCI buss runs at only 30MB/s (hence the restriction to only 3 crates), but if we upgrade the computers
to the next generation PCI bus, it can handle more than 100MB/s.   In either case, a 10s block of time ordered  data is transported by 1GB/s ethernet to computers for trigger formation.  If we are restricted to using the 30MB/s bus, then two machines are required to collect data from all 6 crates.  Each of the two computers will then ship the same 10s block of data to another computer that will compute triggers.   The current idea is to ship 10s blocks of data to a particular computer, then another 10s to a second machine, and so on.  Hopefully, we will not need more than 3-4 machines for this task, but at the moment, this is completely unknown until we have a better understanding of trigger building routines.

New hardware for 04/05 to implement software trigger formation:

2 special VME crates with divided backplanes -          $10k
6 GPS interface card for each VME crate -DESY dollars
1+1 GPS distribution box  -DESY dollars
4-6 dual cpu DL380s with 100 MB PCI bus and 1 GB/s ethernet connections.  $30k
1 GB/s ethernet router for private data distribution  $3k
Probably a large 2-4 TB disk array for temporary storage for GRB events,  event building with data from IceCube strings  $10k
Possibly purchase 6-8 new PCI cards from SIS for the new 130MB/s PCI bus   $3k

Schedule for next season (04/05):
       1. Install requisite hardware and computer systems specified above
       2.  Develop software triggers that:
            a) can be implemented and executed with computation resources available at pole
            b) can be extended to include IceCube data from strings installed in Jan 05
            c) the efficiency and performance of each trigger can be evaluated while at pole

        3.  Write DSP code to repack the 32 bit data from neighboring channels so that every 16 bits has a sample from a feature. 
        4.  Investigate faster VME to PCI cards and purchase new computers with faster PCI bus

        5.  Determine if TWR05 capabilities allow integration of new data from IceCube strings

        6.  Optimize  SIS TWR FPGA code based on results of studies of data acquired at pole 03/04

        7.  Create appropriate data handler upgrades to merge and store data from TWR software trigger

        8.   Include option to install external triggers from SPASE and/or IceCube strings.  This involves several considerations.  We
                need to buy VME input register so we know which trigger corresponds to a particular GPS time.  We would need to
                latch a GPS time for each trigger that then gets readout by the trigger formation computer so that they know which time slice
                of waveform fragments to keep.

        9.  Write a new data reader for new raw binary data, and assess required changes to merger (or if merging with mu-DAQ is really required next year) and data handler.  We may want to include level 1 and level 2 filtering of the TWR data, but this is not obvious while the muon-DAQ processing is still being done.

Of course, this schedule must be coordinated with the complete upgrade plan for 04/05.  At the moment, we anticipate that

    1.  Install new SWAMPs to reduce the signal from the VLF array
    2.   Reduce the gain of the PMTs to extend the instantaneous dynamic range
    3.   Reduce the thresholds in the DMAD to be roughly comparable to the software thresholds in TWR
    4.   Re-calibrate TOs and prompt/delayed ADC response vs Npe in both muon-DAQ and TWR DAQ
            -note the response of the TWR DAQ should be independent of the software development activities
There are some looming questions with this plan
    1. Will IceCube drilling create so much noise that the AMANDA calibration process is not possible?  What contingency options are there?