HIGH-LEVEL SKA SIGNAL PROCESSING DESCRIPTION

 HIGH‐LEVEL SKA SIGNAL PROCESSING DESCRIPTION Document number .................................................................. WP2‐040.030.010‐TD‐001
Revision ........................................................................................................................... 1
Author ................................................................................. W.Turner, et. al. (see below)
Date ................................................................................................................. 2011‐03‐29
Status ............................................................................................... Approved for release
Name Designation Affiliation Date Signature Additional Authors A. Faulkner, B. Stappers, S. Ransom, R. Weber, R. Eatough,M.Kramer Submitted by: W. Turner Signal Processing Domain Specialist
SPDO 2011‐03‐29 Approved by: P. Dewdney Project Engineer SPDO 2011‐03‐29 WP2‐040.030.010‐TD‐001 Revision : 1 DOCUMENT HISTORY Revision Date Of Issue Engineering Change Comments Number A ‐ ‐ First draft release for internal review B ‐ ‐ C ‐ ‐ 1 29 March 2011 ‐ First release th
DOCUMENT SOFTWARE Package Version Filename Wordprocessor MsWord Word 2007 Block diagrams Other 01a‐WP2‐040.030.010‐TD‐001‐1_HighLevelDescr ORGANISATION DETAILS Name Physical/Postal Address SKA Program Development Office Jodrell Bank Centre for Astrophysics Alan Turing Building The University of Manchester Oxford Road Manchester, UK M13 9PL +44 (0)161 275 4049 www.skatelescope.org Fax. Website 2011‐03‐29 Page 2 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 TABLE OF CONTENTS 1 INTRODUCTION ............................................................................................. 7 1.1 Purpose of the document ....................................................................................................... 8 2 REFERENCES ................................................................................................ 9 3 HIERARCHY ................................................................................................ 11 3.1 Hierarchical Lifecycle ............................................................................................................ 11 4 ELEMENT LEVEL: SIGNAL PROCESSING .............................................................. 12 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 Environment .......................................................................................................................... 14 Simulator ............................................................................................................................... 14 Receptors .............................................................................................................................. 14 VLBI ....................................................................................................................................... 15 Power .................................................................................................................................... 15 Cooling .................................................................................................................................. 15 External Transient Triggers ................................................................................................... 15 Time Reference ..................................................................................................................... 15 Science Computing ................................................................................................................ 16 Monitoring and Control ........................................................................................................ 16 Stakeholders ......................................................................................................................... 16 5 SUBSYSTEM ............................................................................................... 17 6 RFI EXCISION ............................................................................................ 21 7 CORRELATOR ............................................................................................. 24 7.1 Delay Compensation Buffer .................................................................................................. 25 7.2 Channeliser ........................................................................................................................... 25 7.3 Corner Turn ........................................................................................................................... 28 7.4 Full Stokes Correlator ............................................................................................................ 29 7.4.1 Correlation Integration Period ...................................................................................... 29 7.4.2 Correlator Processing Load ........................................................................................... 30 8 CENTRAL BEAMFORMER ............................................................................... 32 8.1 8.2 8.3 Buffer .................................................................................................................................... 33 Voltage Storage ..................................................................................................................... 33 Beamforming ........................................................................................................................ 33 9 DE‐DISPERSION .......................................................................................... 35 9.1 9.2 9.3 Incoherent Dedispersion ....................................................................................................... 35 Delay and Sum Dedispersion ................................................................................................ 36 Pre‐summing channels for large dispersion measures ......................................................... 36 2011‐03‐29 Page 3 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 9.4 9.5 9.6 9.7 9.8 9.9 10 Accumulating and differencing algorithm ............................................................................ 37 De‐dispersion over multiple sample intervals ...................................................................... 37 Taylor tree based algorithms ................................................................................................ 37 Frequency Partitioning .......................................................................................................... 38 Coherent de‐dispersion ........................................................................................................ 39 Concept sizing ....................................................................................................................... 39 PULSAR SEARCH ...................................................................................... 44 10.1 Binary Search ........................................................................................................................ 44 10.1.1 Matched Filter ............................................................................................................... 44 10.1.2 Hough Transform .......................................................................................................... 46 10.1.3 Stack Search .................................................................................................................. 46 10.1.4 Phase Search ................................................................................................................. 46 10.1.5 Coherence Recovery ..................................................................................................... 46 10.1.6 Time Domain Resampling ............................................................................................. 47 10.2 Time Domain Re‐Sampling .................................................................................................... 47 10.3 FFT ......................................................................................................................................... 49 10.4 Whitening and Normalisation ............................................................................................... 50 10.5 Harmonic Sum ....................................................................................................................... 50 10.6 Threshold Detection .............................................................................................................. 52 10.7 Candidate Filtering ................................................................................................................ 52 10.7.1 Artifcial Neural Nets ...................................................................................................... 53 10.7.2 The Future ..................................................................................................................... 53 10.7.3 Application to the SKA .................................................................................................. 53 11 PULSAR TIMING ....................................................................................... 54 11.1 Basic Parameters ................................................................................................................... 54 11.2 Timing scenarios ................................................................................................................... 55 11.3 Monitoring and Cadence: ..................................................................................................... 55 11.4 Observing Frequency and Bandwidth ................................................................................... 56 11.5 Collecting area, beams and integration time: ....................................................................... 56 11.6 Forming the Beams: .............................................................................................................. 58 11.7 Time Resolution and Frequency Resolution. ........................................................................ 58 11.8 Data rates: ............................................................................................................................. 58 11.9 Processing the Beams ........................................................................................................... 59 11.9.1 (Coherent) De‐dispersion: ............................................................................................. 59 LIST OF FIGURES Figure 1 Heirarchical Development ...................................................................................................... 12 Figure 2: Signal Processing Functional Context Diagram ...................................................................... 13 Figure 3 Signal Processing Definition .................................................................................................... 17 Figure 4 A model of the u‐v plane for the SKA ...................................................................................... 18 Figure 5 Beamforming .......................................................................................................................... 18 2011‐03‐29 Page 4 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Figure 6 Internal Block Diagram of SKA1 Signal Processing .................................................................. 20 Figure 7 Correlator Definition ............................................................................................................... 24 Figure 8 Adjacent Channels Multiband Filter ....................................................................................... 28 Figure 9 4‐Channel Taylor Tree De‐disperser ....................................................................................... 38 Figure 10 Dispersion measure, DM, for pulsars at different galactic latitudes. ................................... 41 Figure 11 Binary Pulsar Search Algorithms ........................................................................................... 44 LIST OF TABLES Table 1 RFI Mitigation options, pro’s and con’s. ................................................................................... 22 Table 2 Technology Readiness Levels of RFI mitigation methods. ....................................................... 23 Table 3 DM Diagonal ............................................................................................................................. 40 Table 4 Dedispersion Processing loads per beam ................................................................................. 42 Table 5 Dedispersion Output Rate per beam ....................................................................................... 43 Table 6 Number of trial Accelerations .................................................................................................. 48 Table 7 Re‐sampling Processing Load per beam ................................................................................... 48 Table 8 Time Re‐sampling output rates ................................................................................................ 49 Table 9 FFT Processing Load per Beam ................................................................................................. 50 Table 10 Harmonic Sum Processing Load for acceleration Processing per Beam ................................ 51 Table 11 Harmonic Sum Output Rates per Beam ................................................................................. 52 2011‐03‐29 Page 5 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 LIST OF ABBREVIATIONS AA.................................. Aperture Array
Ant. ................................ Antenna
CoDR............................. Conceptual Design Review
DRM .............................. Design Reference Mission
FLOPS........................... Floating Point Operations per second
FoV ................................ Field of View
Ny .................................. Nyquist
OH ................................. Over Head
Ov .................................. Over sampling
PAF ............................... Phased Array Feed
PrepSKA........................ Preparatory Phase for the SKA
RFI................................. Radio Frequency Interference
rms ................................ root mean square
SEFD...........................System Equivalent Flux Density
SKA ............................... Square Kilometre Array
SKADS .......................... SKA Design Studies
SPDO ............................ SKA Program Development Office
SSFoM .......................... Survey Speed Figure of Merit
TBD ............................... To be decided
Wrt ................................. with respect to
2011‐03‐29 Page 6 of 59 1
WP2‐040.030.010‐TD‐001 Revision : 1 Introduction The aim of this document is to present a high level functional breakdown of the Signal Processing aspects of the SKA telescope primarily for Phase 1, SKA1, but with consideration of scalability to Phase 2, SKA2, of the project. SKA Memo 125 [27] defines the main scientific goals and baseline technical concept for the SKA phase 1. This definition identifies the major science goals for SKA1: 
Study the history and role of neutral Hydrogen in the Universe from the dark ages to the present‐day 
Employ the detection and timing of binary pulsars and spin‐stable millisecond pulsars as probes of fundamental physics including o
testing theories of gravity (including General Relativity and quantum gravity) o
to discover gravitational waves from cosmological sources o
to determine the equation of state of nuclear matter In addition, Memo 125 provides a baseline technical concept of SKA1 receptors including: 
A low‐frequency sparse aperture array with an A/Tsys of up to 2000 m2/K operating at frequencies between 70 and 450 MHz. The array will be centrally condensed but some of the collecting area will be in stations located out to a maximum baseline length of 100 km from the core 
A dish array with Aeff/Tsys of up to 1000 m2/K using approximately two hundred and fifty 15‐metre antennas, employing an instrumentation package that will use single‐pixel feeds to provide high sensitivity and excellent polarisation characteristics over a frequency range of 0.45‐3 GHz. The array will be centrally condensed but some of the elements will be co‐
located with the sparse aperture array stations out to a maximum baseline length of 100 km from the core. This Signal Processing High Level Description document is part of a document series generated to provide a top down and bottom up approach in support of the Signal Processing CoDR. This document set includes includes the following: 
Signal Processing High Level Description 
Technology Roadmap 
Design Concept Descriptions 
Signal Processing Requirements 
Signal Processing Costs 
Signal Processing Risk Register 
Signal Processing Strategy to Proceed to the Next Phase 
Signal Processing Co DR Review Plan 
Software & Firmware Strategy 2011‐03‐29 Page 7 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 The focus of this document is providing the high level logical architecture of the SKA signal processing. This means identifying the structure and behaviour of the key functional blocks and their interfaces within the context of the SKA as a whole. It should be pointed out that the term ‘interface’ is used as an abstract term as the aim of this document is not to specifically identify physical solutions but the logical functionality. However it is recognised that some aspects of the physical architecture need to be considered. For example, some physical aspects of the receptor technologies are likely to be imposed as constraints as part of the systems requirements. This document also flows down and develops the strategies identified in the Systems Engineering Management Plan (SEMP) as presented within the context of the signal processing domain. This includes the hierarchical approach and the iterated requirements driven design process. 1.1 Purpose of the document The purposes of this document are as follows: • 
• To present the context of Signal Processing within the SKA system hierarchy. To provide an overview of the Signal Processing life‐cycle. To provide a system break down of the Signal processing capability in terms of identifying: o
Where Interface Control Documents, ICD, are required. o
Function blocks o
To present an overview of the various algorithmic schemes that are being put forward for SKA signal processing 2011‐03‐29 Page 8 of 59 2
WP2‐040.030.010‐TD‐001 Revision : 1 References [1] SKA Science Case [2] The Square Kilometre Array Design Reference Mission: SKA‐mid and SKA‐Lo v 0.4 [3] Science Operations Plan [4] System Interfaces [5] Environmental requirements (natural and induced) [6] SKA strategies and philosophies [7] Risk Register [8] Requirements Traceability [9] Logistic Engineering Management Plan (LEMP) [10] Risk Management Plan (RMP) [11] Document Handling Procedure [12] Project Dictionary [13] Strategy to proceed to the next phase [14] WP3 SKA array configuration report [15] WP3 SKA site RFI environment report [16] WP3 Troposphere measurement campaign report [17] SKA Science‐Technology Trade‐off Process (WP2‐005.010.030‐MP‐004) [18]A. Faulkner, et al., Aperture Arrays for the SKA: the SKADS White Paper, January 2010. [19] E. de Lera‐Acedo et al., System Noise Analysis of an Ultra Wide Band Aperture Array: SKADS Memo T28. [20] SKA Monitoring and Control Strategy WP2‐005.065.000‐R‐001 Issue Draft E [21]“The Square Kilometre Array”, Peter E. Dewdney, Peter J. Hall, Richard T. Schilizzi, and T. Joseph L. W. Lazio, Proceedings of the IEEE Vol. 97,No. 8, August 2009 [22] Thompson, A. R., Moran, J. M., and Swenson, G. W. “Interferometry and Aperture Synthesis in Radio Astronomy” (second edition), Wiley, 1986. [23] System Engineering Management Plan (SEMP) WP2‐005.010.030‐MP‐001Reference 3 [24] SKA System Requirement Specification (SRS) [25] SKA IP Policy Document [26] International Technology Roadmap for Semiconductors (ITRS), available at www.itrs.net. [27] A Concept Design for SKA Phase 1 (SKA1) SSEC SKA Phase 1 Sub‐committee, http://www.skatelescope.org/PDF/memos/125_Memo_Garrett.pdf [28] RFI Mitigation Implementation for Pulsar Radio Astronomy D. Ait‐Allal, R. Weber, C. Dumez‐
Viou, I. Cognard, and G. Theureau [29] E.Serpedin, F. Panduru, I. Sari, and G.B. Giannakis, “Bibliography on cyclostationarity ” Signal Processing, vol. 85, pp. 2233‐2303, Dec. 2005. [30]R. Weber, P. Zarka, V. Ryabov, R. Feliachi, J. Grießmeier, L. Denis,V. Kozhyn, V. Vinogradov, and P. Ravier, “Data preprocessing for decametre wavelength exoplanet detection: an example of cyclostationary rfi detector,” Eusipco, Poznan, Poland, 2007. [31] L. D'Addario, “Searching For Dispersed Transient Pulses With ASKAP”, SKA Memo 124, March 10, 2010. [32] R. Navarro, “Efficient Summing of ASKAP Beamformer Power Spectra over Multiple Dispersion Measures”, CRAFT memo, July 6, 2010. 2011‐03‐29 Page 9 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 [33] J. H. Taylor, “A Sensitive Method for Detecting Dispersed Radio Emission”, Astron. Astrophys. Suppl., issue 15, pp. 367‐369, 1974. [34] R. N. Manchester, A. G. Lyne, F. Camil, J. F. Bell, V. M. Kaspi, N. D'Amico, N. P. F. McKay, F. Crawford, I. H. Stairs, A. Possenti, M. Kramer, D. C. Sheppard, “The Parkes Multi‐Beam Pulsar Survey – I. Observing and Data Analysis Systems, Discovery and Timing of 100 Pulsars”, Mon. Not. R. Astron. Soc., issue 328, pp. 17‐35, 2001. [35] J.M. Cordes, M.A. McLaughlin, Searches for fast radio transients, The Astrophysical Journal (2003), pp. 1142. [36] G. M. Nita, D. E. Gary, Z. Liu, G. J. Hurford, & S. M. White, 2007, Radio Frequency Interference Excision Using Spectral‐Domain Statistics, PASP, 119, 805. [37]Burke‐Spolaor et al., Peryton Event, submitted . [38] Kramer et al., 2004, New Astr. Rev., 48, 993 [39] Cordes et al., 2004, New Astr. Rev., 48, 1413 [40] Lorimer & Kramer, 2005, Handbook of Pulsar Astronomy, CUP [41] Smits et al., 2009, A&A. 493. 1161 [42] Smits et al., 2011, SKA Phase I Memo [43]R P Eatough, A Search for Relativistic Binary Pulsars in The Galactic Plane (PhD Thesis) [44]T Colgate, N Clarke, Searching for Fast Transients with SKA Phase 1 WP2‐040.030.010‐TD‐004 Rev B [45] SKA Science Working Group, The Square Kilometre Array Design Reference Mission: SKA Phase 1 Rev 1.3 2011.01.17 2011‐03‐29 Page 10 of 59 3
WP2‐040.030.010‐TD‐001 Revision : 1 Hierarchy The SKA subsystem is of sufficient scale and complexity that the Systems Engineering Management plan has defined multiple layers of hierarchy: L8: SKA User L7: System L6: Element L5: Sub‐System L4: Assembly L3: Sub‐Assembly L2: Component L1: Part Although not explicitly stated in the SEMP, the hierarchical approach has the advantage of breaking down the complexity of the system. Each layer is only concerned about its own functionality and its interface to the immediately adjacent layers. Within the hierarchical scheme, Signal Processing is defined at the element level deriving its requirements directly from a subset of System level requirements. In turn, the sub‐system level allows the Signal Processing element to be partitioned further into Level 5 functionality. Introducing these layers of hierarchy ensures that the complexity of the system is broken down such that individual layers only have to deal with their relevant perspective of the system. 3.1 Hierarchical Lifecycle Figure 1 shows how the hierarchical nature of the system translates into its development life cycle. Each level in the hierarchy has its own level of requirements that drive the architectural design for that level. These requirements are derived as a result of partitioning the architecture from the next higher level in the hierarchical structure. There is also a feedback to the requirements at the next level up to allow any potential issues with the requirements to be identified. This feedback scheme may ultimately ripple up the hierarchy to the SKA User level. In this case, the scope of the SKA telescope may need to be renegotiated. As part of the life‐cycle, the hierarchy is also imposed on the installation and setting to work of the system. The requirements at each level are to be verifiable allowing integration to be performed from the Part level upwards ultimately resulting in a fully validated system. 2011‐03‐29 Page 11 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 SKA User
Operations
Support
Program
Engineering
Installation
& Validation
Product
Engineering
Installation
&
Verification
Integrated
System
Product
Engineering
Installation
&
Verification
Integrated
Element
Product
Engineering
Installation
&
Verification
Integrated
Sub-System
Product
Engineering
Installation
&
Verification
Integrated
Assembly
Sub-Assy
Architectural
Design
Product
Engineering
Installation
&
Verification
Integrated
SubAssembly
Component
Specification
Component
Design Build
& Test
User
Requirement
Telescope
Feedback
System
System
Architectural
Design
System
Requirement
Feedback
Partition
Element
Element
Architectural
Design
Element
Requirement
Feedback
Partition
Sub-system
Sub-System
Architectural
Design
Sub-system
Requirement
Feedback
Partition
Assembly
Assembly
Requirement
Assembly
Architectural
Design
Feedback
Sub-Assembly
Component
Part
SubAssembly
Requirement
Partition
Part
Specification
Components
Figure 1 Heirarchical Development 4
Element Level: Signal Processing This document presents a functional model of the signal processing domain based on the Structure and Behaviour diagrams defined by the SysML general‐purpose graphical Modelling language supplemented by supporting text. At this stage, the language is being used purely as a means of providing formalised diagrams for the document and these have not been entered into a modelling tool. The language allows a model of the system to be presented in a hierarchical manner with the ability to drill down through the hierarchy whilst keeping the complexity of individual diagrams to a reasonable level. This document treats the Signal Processing as a standalone model which can be integrated into a larger system model. Consequently the signal processing description starts by providing the context at the element (see Figure 1) level prior to moving down through the system hierarchy layers. For this SysML block definition diagram the SYSMOD profile notations for actors has been used. 2011‐03‐29 Page 12 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 bdd [package] Context [Signal processing context]
0..*
Environment
Engineer Maintainer
Operator
Monitoring &
Control
Simulator
Scientist
0..*
«System»
Signal Processing
Digitised
RF + RFI
Processed Data
Science
Computing
Receptors
0..*
1..*
External
Transient
Triggers
Time
Reference
VLBI
Power
Cooling
Figure 2: Signal Processing Functional Context Diagram The aim of the diagram is to identify the complete set of external and user systems that interface to the Signal Processing domain at both phase 1 and phase 2 of the project. External systems are treated as black boxes and are represented by a 3‐D box in the diagram. User systems provide a mechanism for user interaction and typically include keyboards displays etc. The User System is also presented as a 3‐D box in the diagram but is in association with a ‘stick‐man’ symbol representing the actor. The lines connecting blocks within the diagram represent associations between the blocks. Within the Figure 2 context, these associations are largely based on flows between the blocks. Flows are not limited to data exchange but can include physical entities such as fluids or electrical current. The flows don’t have to be atomic: for example the receptors provide a flow of digitised RF data combined with RFI Data. The multiplicity of an item is provided at the ends of the association lines. For example there are zero to any number of External Transient Triggers or Simulators and 1 to any number of Time References. Where a multiplicity isn’t provided it is to be assumed to be unity. Each interface will require an Interface Control Documentation set; this will include one or more: o
Data Exchange Specifications o
Physical Interface Specifications 2011‐03‐29 Page 13 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 This document limits its coverage to logical interfaces. However, physical implementation options for the key interfaces are provided within the Design Concept Description documents. A brief description of each interface is provided below with an indication of whether applicable to phase 1, phase 2 or both. Detail of the interfaces and the first drafts of their Interface Control Documents will be developed as part of the development phase of the Signal Processing leading up to the Sub‐System Requirement Review, SRR. These will present the initial requirements for both the physical aspects, data flow and meta data flow across each interface. 4.1 Environment Overall environmental conditions for the telescope including temperature, humidity, shock, vibration, particle and wildlife ingress. For the Signal Processing the environment is to a greater extent controlled by the equipment housing. For the Phase 1 of the project this is provided by the Correlator Room in the Central Processing facility. At Phase 2, dish station beamforming is that is in close proximity to the Stations. Proposed Correlator Room requirements are provided within the Signal Processing Requirements document. The interface to the environment is applicable at both phase 1 and phase 2 of the project. 4.2 Simulator Stimulators may be required to support development of Signal Processing equipment and provide preliminary testing prior to shipment to the Signal Processing facility. The interface to simulators is applicable at both phase 1 and phase 2 of the project. 4.3 Receptors The RF signal is the wanted signal from the astronomical source being observed. The Design Reference Mission [2] defines the performance envelope for the telescope. RF Interference represents any external contaminating RF signal. This is site dependent and is detailed in WP3 SKA site RFI environment report [15]. Interface to 50 Sparse Aperture Arrays and 250 Dishes equipped with Single Pixel Feeds will be implemented at Phase 1 as detailed in SKA memo 125. Phase 2 will extend the capability of the telescope by increasing the number of Sparse Aperture Arrays and dishes to nominally 250 and 3000 respectively. In addition, there is the potential inclusion of o
Wide Band single pixel Feeds o
Dense Aperture Arrays o
Phased array Single Pixel Feeds 2011‐03‐29 Page 14 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 4.4 VLBI The VLBI Data Interface Specification Release 1.0 ratified 26th June 2009 specifies the standardized transport‐independent VLBI data‐interchange format that is suitable for all types of VLBI data transfer, including real‐time and near‐real‐time e‐VLBI well as disk‐file storage. http://www.vlbi.org/vsi/docs/VDIF%20specification%20Release%201.0%20ratified.pdf The complementary physical interface specification is currently being written. Although the VDIF specification makes no mention of data‐transport protocol, it has been developed with an awareness of expected methods of data transport, including network transport using various standard protocols, as well as physical or electronic transport of standard disk files. VLBI interface is not applicable to phase 1 of the project. 4.5 Power External power to the system is dealt with in the Power section of the Strategies and Philosophies document [6]. The interface to the power distribution is applicable at both phase 1 and phase2 of the project 4.6 Cooling The strategy for dealing with cooling for the SKA telescope is detailed in the Cooling section of the Strategies and Philosophies document [6]. The interface to the cooling is applicable at both phase 1 and phase2 of the project 4.7 External Transient Triggers The SKA telescope is to provide the facility for receiving external transient triggers. The interface is to utilise the SkyAlert service (http://www.skyalert.org/ ) (TBC) which collects and distributes astronomical events in near‐real time and distributes the resultant data in accordance to the provisional standard VOevent (http://www.ivoa.net/Documents/REC/VOE/VOEvent‐20061101.html ). The transient events include but are not limited to supernovae, gamma‐ray bursts, micro‐lensing . Transient detection triggers may also be generated internal to the SKA telescope if these are external to the Signal Processing, they are to be included as part of the External Transient Trigger interface. The interface for transient triggering isn’t part of phase1 of the project 4.8 Time Reference It is anticipated that this will be satellite GPS and is detailed in the Timing and Synchronisation section of the SKA Strategies and Philosophies document [6]. 2011‐03‐29 Page 15 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 The interface to the Time Reference is applicable at both phase 1 and phase2 of the project 4.9 Science Computing The Science Computing provides data and meta‐data reduction to a format desired by the end user scientist from raw u‐v plane or non imaging data. The interface to the Science Computing is applicable at both phase 1 and phase2 of the project 4.10 Monitoring and Control Monitoring includes output from the telescope to the operator. This will provide information on the health status and the configuration of the telescope and is detailed in the Monitoring and Control Strategies and Philosophies document [20]. External operator control of the telescope is detailed in the Science Operations Plan [3] and the Monitoring and Control Strategies and Philosophies document [20] . The interface to the Science Computing is applicable at both phase 1 and phase2 of the project 4.11 Stakeholders Within the context of Signal Processing the Stakeholders are the external systems or humans that interact with Signal Processing equipment. The term ‘interact’ is used to indicate an associated change of state or the behavioural aspects of the system. The interactions of the Stakeholders are to be captured using Use Cases which are to be captured as part of the Requirements set. The Signal Processing Stakeholders include (but may not be limited to): • Scientists The Scientist defines and then submits a plan that details the high level usage of the telescope required for performing observations that support science experiments. • Operators The Operator is normally a Staff Astronomer or an Engineer that controls the SKA Telescope during science experiments or engineering experiments. • Maintainers The Maintainer is a technical person that is skilled and qualified prior to receiving SKA Telescope Technical Training and is responsible for Corrective and Preventive Maintenance and for the telescope. The Maintainer is also involved during telescope task execution. The maintainer monitors the system health displays regularly during task execution and could, when required, takes manual control of resources for the purposes of testing and diagnosis. (This authorisation needs to be delegated by the operator). 2011‐03‐29 Page 16 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 • Engineer The Engineer is part of the multi‐disciplined SKA design team and is responsible for design commissioning, verification and incremental upgrade of the telescope. The interface to the Signal Processing stakeholders is applicable at both phase 1 and phase2 of the project. 5
Subsystem Having presented the Signal processing as a black box with a definition of its external interfaces in section 4, a definition of the Signal Processing internal functionality is now presented. bdd [block] system [Signal processing definitions]
«block»
Signal Processing
1..*
1..*
«block»
Correlator
«block»
Non-Imaging
Computing
1..*
«block»
Beamforming
1..*
«block»
RFI Excision
Figure 3 Signal Processing Definition Figure 3 provides a SysML graphical representation of the logical subsystem types that make up the Signal Processing element 
RFI Mitigation 
Correlator 
Beamforming 
Non‐Imaging Computing RFI Mitigation functionality is cross cutting across the whole of the SKA telescope with different strategies applied at different points in the system for different observation modes. It’s functionality within the Signal Processing cross‐cuts that of Correlation, Non‐Imaging and Beamforming. Section 6 provides an overview of the RFI Mitigation strategies applicable to the Signal Processing. 2011‐03‐29 Page 17 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Correlation is the first step in combining the receptors as part of imaging. The correlator provides cross correlation between each pair of receptors (known as a baseline) sharing the same frequency range and pointing. The correlator performs what is known as Full Stokes correlation which provides cross correlation for the four combinations of the two polarisations from receptors. The distance between the receptors (baseline length) determines the resolution of the image and the number of baselines the number of points within the image. The correlator produces what is known as the u‐v plane which is the Fourier Transform of the final image. The Science Computing provides the inverse transform to create the final image. A snap shot of the u‐v plane for the SKA is shown in Figure 4 Figure 4 A model of the u‐v plane for the SKA Due to the rotation of the Earth with respect to the point being imaged, each point within the u‐v plane moves in an elipse as a function of time. This is used to fill in the gaps between the individual points. However, there are a couple of aspects that need to be considered for correlation that potentially impact on the image quality. These relate to the effect of smearing as a result of sampling theorem in terms of bandwidth and the amount of integration that can be implemented on the cross correlation products [22]. Section 7 provides a functional breakdown and sizing of the processing associated with Correlation including details of the bandwidth and integration rate limits required. Beamforming allows the field of view available to the telescope to be expanded by combining receptors into arrays to allow directional reception of signals. Incoming signal
Elements
+ + + + + + + + + + + + + + + + + + + + + + + +
Beam
C0I0+ C 1I1+ C2I2+ C3I3+ C 4I4+ C 5I5 + C6I6+ C 7I7+ C 8I8+ C 9I9+ C10I10 + C11I11+ C12I12+ C13I13+ C14I14+ C15I15+ C16I16+C17I17+ C18I18+ C19I19+ C20I20+ C21I21+ C22I22+ C23I23
C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5+......
Figure 5 Beamforming 2011‐03‐29 Page 18 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 The formation of beams is achieved by introducing geometric delays for each receptor and summing across all receptors in the array Figure 1. By the manipulation of the individual delays the direction of the beam can be steered and even multiple beams generated. A useful approximation to introducing time delays is phase delay; however, this is only applicable if the bandwidth time product is much less than unity. The use of phase delay techniques reduces the beamforming to multiplying each data stream from a receptor by a complex coefficient prior to adding the data. For the SKA1 beamforming within the Signal processing domain is limited to Central beamforming. This forms beams across dishes and/or AA_low beams within the central 5km diameter core. Further details are provided in section 8 Non‐Imaging Processing is a term used for: 
Pulsar search 
Pulsar Timing 
Transients detection At present only Pulsar Search and Timing is part of the SKA with transient detection being SKA2. However aspects of transient detection require consideration as part of the extensibility of SKA1. Common to all three of these is De‐dispersion which refers to the process of correcting frequency dependent time delays introduced as a result of the properties of the Inter‐Galactic medium through which the received signals are likely to have propagated through. Several techniques have been developed to provide this processing and these are detailed in section 9. Pulsar searching and more specifically the techniques developed for the detection of binary pulsar systems is detailed in section 10. Pulsar Timing is detailed in section 11 Having identified the types of processing included as part of the signal processing domain, it is informative to provide a diagram illustrating some lower level detail and how individual blocks logically relate to each other. Figure 6 provides an Internal Block Diagram the Signal Processing. This is a representative logical implementation and may develop with the definition phase of the project as lower architectural issues are identified. Details of the lower level blocks will be covered in the remaining sections of this document. 2011‐03‐29 Page 19 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 ibd [block] system [Signal Processing IBD]]
ibd [block] system [Correlator CentralBeamformer IBD]]
RF + RFI Dish
RF + RFI AA_low
C&M
«block»
:Delay
Compensation
Buffer Dish
«block»
:Delay
Compensation
Buffer SAA
«block»
:Coefficient
Generation
«block»
:Coarse
Channelisation
«block»
:Fine
Channelisation
& Fractional Bit
Rotation
0..1
«block»
:Stokes
Correlation
«block»
:Corner Turn
Science
Computing
«block»
:RFI Mitigation
«block»
:Integration
«block»
:Central
Beamforming
RFI
Database
Time Reference
«block»
: I I2
ibd [block] system [Non Imaging Processing IBD]]
ibd [block] system [Pulsar Survey IBD]]
«block»
:DeDispersion
«block»
:Binary Search
«block»
:Harmonic Sum
«block»
:Whitening&
Normalisation
«block»
:Candidate
Selection
Science
Computing
ibd [block] system [Pulsar Timing]]
«block»
:Coherent
DeDispersion
«block»
:Folding
«block»
:Pulse arrival
Time Prediction
Science
Computing
Ephemeris
& PolyCo.
Figure 6 Internal Block Diagram of SKA1 Signal Processing
2011‐03‐29 Page 20 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 6
RFI Excision The effectiveness of mitigation is limited by the estimation and detection accuracies of the signals involved. Different astronomical observing modes may require different interference mitigation techniques and approaches. Examples of these modes are spectral line observations, polarisation measurements, synthesis imaging, and pulsar research. There are many ways to define categories for interference, such as narrow band or wide band, fixed or moving sources, categories based on statistical properties (e.g. spatial and temporal coherence) or based on modulation type, distinctions based on the amount of a‐priori information of the transmitter or on differences in spatial properties or polarisation, categories based on field strength, power and temporal spectral occupancy, and categories of overlapping signal parameter domains. Clearly, a great diversity of approaches is possible, and in choosing an optimal approach the following should be considered: 
Depending on the interference properties, the architecture of the radio telescope and the type of observation, the same RFI mitigation technique can be useless or very efficient. 
Efficiency is generally linked with specificity. The more a priori information on the RFI can be exploited, the better will be the RFI mitigation algorithm. In other words, it is impossible to define one single approach which will cover all current and future scenarios. The consequence is that several (as far as possible “orthogonal”) methods have to be implemented such that they can be used in conjunction. For exotic or unexpected scenarios, the radio telescope architecture should be flexible enough to allow reallocation of signal processing resources to RFI mitigation. The RFI challenge in the SKA candidate sites in Australia and South Africa may not be so great. Under this assumption, one basic or recurrent scenario could be to carefully design the analogue parts, taking RFI threats into consideration, but to limit the digital measures to “flagging". In that case, the digital signal processing resources could be fully dedicated to regular signal processing tasks most of the time and could be partially re‐used (scheduled) for observations facing specific RFI issues. In particular, it would be worthwhile to continuously monitor the quality of the data. Given the extreme sensitivity of the SKA telescope, this task has to be a by product of the radio telescope it‐
self (i.e. an auxiliary antenna will not be sensitive enough). So, it would be interesting to implement some detection methods (to be defined) as regular signal processing tasks at station level and core level. The results could be linked to a kind of RFI statistics database or could be attached to the data for flagging. Table 1 shows a table describing what class of RFI mitigation techniques could be applied at the different levels of the SKA signal flow, from antenna level to core level. In addition, this table provides some pro's and con's, assuming that the corresponding implementation will be done in the digital domain. However, it appears that their impact on both the image residual and the calibration effectiveness is not fully understood yet, especially in the case of spatial filtering techniques and many of the paetric techniques. Besides, none of the techniques have been applied in very large scale telescope arrays. 2011‐03‐29 Page 21 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Signal Path Method Pro’s
Con’s Antenna beam‐formers (e.g. PAF) varying spatial filtering, including sidelobe canceller reduce strong RFI enables the use of less ADC bits / lessens LNA req. fluctuating beam may impair calibration fixed spatial filtering reduce strong RFI enables the use of less ADC bits /lessens LNA req. difficult; needs careful calibration [excision] lower SP load at output station beamformers ‐
Station beamformers fixed spatial filter very cheap; reduce data
more complex operation; connection with central transport rate to central site systems varying spatial filters, sidelobe canceller somewhat better suppression than fixed; tracking possibilities may be costly; changing sidelobes may impair calibration excision (assuming no subband filtering is done yet) low SP load unless booking is done on excised samples; fast transients bookkeeping very costly; impairing gain estimate otherwise parametric techniques (assuming wide bands) can be used in combination with other methods may be costly Precorrelation Interstation sidelobe cancelling/ spatial filtering, moving sources may be applicable at shorter influences UVW data timescales than at location points; may impair of correlator output calibration Correlation excision can be done at short timescales and short bandwidths; common practice may be complex; may be time consuming Table 1 RFI Mitigation options, pro’s and con’s. In Table 2, an estimate of the degree of maturity of the different RFI mitigation approaches is presented. Two evaluation scales are proposed, one based on current experimentations within existing radio telescopes (i.e. small/medium size radio telescopes) and another one based on the requirement for a large scale radio telescope such as envisioned in the SKA project. In that case, the different levels have been associated to some fundamental steps in the SKA design process, which are: 2011‐03‐29 Page 22 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 RFI Mitigation Algorithms a.
Excision TRL Justification 6 Post Correlation Narrowband Flagging –TRL 9 Pre Correlation Narrowband Excision –TRL 8 Pre Correlation Broadband Excision –TRL 8 b.
Detection 6 Power Detection – TRL9
Analysis based on analysis of the pdf‐ TRL 7 Higher Order Statistics detection e.g. Kurtosis – TRL 9 Cyclo Stationary Detection – TRL7 Multiple Antenna Detection subspace techniques – TRL7 c.
Spectral Filtering 5 Band Selection – TRL9
Notch Filtering out of band – TRL9 Notch Filtering in band TRL 1 Cyclostationary Spectral Filtering – TRL3 ‐7 d.
Spatial Filtering
5 Spatial filtering at (phased array) station level – TRL7 Pre‐correlation spatial filtering – TRL7 Post Correlation filtering using closure phases – TRL3‐8 Spatial filtering using reference antennas – TRL7 Spatial Filtering using cyclostationary – TRL3 e.
f.
Single Channel Filtering 5 Miscellaneous Techniques 3 Subtraction of estimated RFI waveform – TRL7 Parametric RFI estimation and subtraction – TRL3‐7 Polarisation based RFI Mitigation – TRL1
Fringe rotation Techniques – TRL3 RFI suppression by delay smearing – TRL3 Imaging and post‐correlationRFI removal using clean and beamforming techniques – TRL3 Estimating RFI correlation matrix using cyclo‐stationary techniques – TRL3 Table 2 Technology Readiness Levels of RFI mitigation methods. 2011‐03‐29 Page 23 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 As part of forming the coherently de‐dispersed data, methods of RFI excision can be applied. There are a number of possible mechanisms by which RFI rejection might occur and it may be that until data from the pathfinders are in place that it isn’t known what will be necessary. At present it is assumed that RFI mitigation is based on the flagging of frequency channels that are suspected of being contaminated. The flagging is based on two strategies: 
The use of historic data provided via an RFI data base 
Threshold detection based on the auto correlation of individual frequency channels 
Detections within the Signal Processing chain can potentially in feed to the RFI data base. A basic algorithm would be that a mask of known frequencies which correspond to RFI, such a list might be made up from previous observations. Such an algorithm would be at a relatively low computational load and may be built into the channelization or de‐dispersion step by blanking channels. 7
Correlator bdd [block] system [Correlator & Central Beamformer definitions]
«block»
Correlator
«block»
RFI Mitigation
«block»
Full Stokes
Correlator
«block»
Channeliser
«block»
Delay
Compensation
Buffer
«block»
Channeliser
Coarse
«block»
CornerTurn
«block»
Central
Beamformer
«block»
Monitor&
Control
«block»
Channeliser
Fine
Figure 7 Correlator Definition This section provides details on the functional breakdown of the correlator in accordance with Figure 7 and provides processing sizing and bandwidth estimates. 2011‐03‐29 Page 24 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 7.1 Delay Compensation Buffer Digital data received at the channeliser is compensated with respect to signal propagation delay by use of a data buffer. This buffer may also form a storage area for test data in off line test diagnostics. This delay compensation is only time sample accurate for the received signal. The propagation delay is a function of the communication path length which as a first approximation is taken as the base‐line length which is nominally 200km and over 3000km for SKA1 and SKA2 respectively. In reality the communication path will not be point to point and as a result will be greater than the baseline length. Assuming the signal propagates at the speed of light in optical cable (2 x 108 m/s‐1) , the buffer depth will need to compensate for 1ms and 15ms delay for baselines of 200km and 3000km respectively. For SKA phase 1 dishes: _
250
200 10
2 10
~ 2 1 10
2
4 For SKA phase 1 Sparse Aperture Arrays: _
50
200 10
2 10
480
~ 67 0.35 10
2
4 This assumes that all baselines require the same amount of storage as the maximum leading to an over estimate in the memory requirement. Only storing data applicable to the delay for each baseline can reduce the memory requirements but at the expense of complexity of the memory management. A potential compromise could be to use a few block ranges of baseline length with associated delays. The effectiveness is quite high due the high percentage of antenna within the core. 7.2 Channeliser Channelization refers to the process of splitting the received RF base banded signal into a contiguous set of narrow frequency channels. This section specifically describes the process in association with imaging. There are four reasons why channelization is required: 



To facilitate the approximation of phase shift to time delay for digital domain beam‐forming To provide the frequency resolution appropriate to the time resolution for Non‐Imaging computing To minimise the radial smearing of U‐V data. To minimise the impact of RFI 2011‐03‐29 Page 25 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 The first of these requires that the worst case bandwidth time product of the incident wave‐front across the collection (station) of receptors used to digitally form beams using phase shifting techniques is much less than unity. It should be noted that the requirements of channelization for Non‐Imaging Processing differ from those associated with imaging processing. The frequency resolution, ∆ν, of the channeliser corresponds to the required time resolution of 50us for pulsar sear and .2us to 1 us pulsar timing: ∆ 1
20 , 1 5 For Sparse Aperture Arrays some channelization will be implemented at the array as part of the station beamforming process. When considering the overall SKA, the radial smearing of U‐V data is detailed in Thompson, Moran and Swenson’s Interferometry and Synthesis in Radio Astronomy, the relative amplitude, Ra, produced for a bandwidth ∆ν at an observation frequency νo, is approximated by the expression: The radius, , of the Field of View is proportional to where is the wavelength and d the antenna diameter in metres. 2011‐03‐29 Page 26 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 And the synthetic beam radius, , is proportional to where Dmax is the maximum baseline. ∆
1
0.939
1
2
An accepted simplification is: ∆
~ 1
10
Consequently the frequency cell width required is inversely proportional to the maximum baseline for a given observation frequency and amplitude smearing. For dishes with less than 2 percent smearing over a 200 km baseline: ∆ 1
3 10
. 98
0.939
∆
49 1 15
200000
For Sparse AAs with less than 2 percent smearing over a 200 km baseline: ∆ 1
3 10
. 98
0.939
∆
590 1 180
200000
In both case this is less stringent than the frequency resolution requirement of 2kHz identified for the DRM Chapter 11: Tracking Galaxy Evolution over Cosmic Time using H1 Absorption. Recently, channelisation is typically implemented using the Multiband Filtering techniques based on an FFT architecture, though hierarchical FIR filtering has also been used on the WIDAR correlator. The disadvantage of the Multiband filter technique is the leakage between frequency channels. However this can be resolved by oversampling techniques as demonstrated on the ASKAP path finder project. In this case, the channelization is split into two stages: coarse channels before beam‐
forming and fine channels after beam‐forming. A description of the technique is provided in ALMA Memo 447. The processing load per antenna feed per polarisation is dependent on the quality of the multiband filter response, the FFT size and the amount of up‐sampling applied. It can be shown the number of taps in the form of a FIR filter is proportional to the ratio of the original sample frequency over the filter transition band (Crochiere, R E and Rabiner, L R: Optimum FIR Digital Implementations for Decimation , Interpolation and Narrowband Filtering. Ballanger, M G: Computation Rate and Storage Estimation in Multirate Digital Filtering with Half‐Band Filters) 2011‐03‐29 Page 27 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 .
∆
where 2 < k < 4 depending on the amount of pass‐band and stop‐band ripple that is acceptable and fold is the pre‐decimated sample rate. Figure 8 shows two adjacent channels of the multiband filter that has been up‐sampled to provide separation between frequency channels. Figure 8 Adjacent Channels Multiband Filter The up‐sampling is achieved by overlapping the incoming data to provide a channel frequency separation of fs . The channel width is f and the width of the frequency channel at half the rejection amplitude is fso. The up‐sampling is achieved by overlapping the incoming data stream by a factor that is a ratio of two integer values p and q such that: Where Noverlap is the number of samples that are overlapped. The processing load for the channelization is therefore: 1
2
2
Typically the channelizer will be implemented in two or more sequential stages: course through to fine. The finer resolution channelizer is implemented on each of the time series emerging from the up‐stream up‐sampled coarser channelizer . 7.3 Corner Turn The data produced by the channeliser is a streamed set of frequency channels for each receptor. However, the cross correlation process used in the full Stokes Correlator may require (depending on 2011‐03‐29 Page 28 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 architecture) the data be reordered such that a set of data from all receptors for each frequency channel is provided. 7.4 Full Stokes Correlator Full Stokes Correlation is provided by the correlator to provide a set of U‐V plane points for each frequency channel and beam direction. The number of correlations 1
2
Where Nant is the number of stations, NBeams is the number of beams generated per station and Nchan is the number of frequency channels generated by the channelizer. The correlator processing load, C, is independent of the number of frequency channels, NChan as the correlation rate is inversely proportional to the number of channels. 2
Ω
180
16
1.2
2
The resultant correlation data can be integrated to reduce bit rate. The maximum integration period is determined by the acceptable level of smearing of U‐V data due to the rotation period of the Earth against the sky. 7.4.1 Correlation Integration Period From [22], the relative amplitude produced for an integration period, τa, is approximated by the expression: 1
1 0.8326
3
2
2
1
2
1
2
0
Identifying the term (l12+ m12 sin2 δ0) as the radius squared of the maximum field of view the equation can be rearranged to provide the maximum integration time, τa, in terms of a desired smearing constraint on Ra 2011‐03‐29 Page 29 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 31 0.8326
1
The radius, , of the Field of View is proportional to And the synthetic beam radius, . , is proportional to . The Earth’s rotation, ωe = 7.92 x 10‐5 radians per second 3 1
0.8326
3 1 0.98
0.8326 7.92 10
15
200 10
The output rate of the correlator is to be minimised to limit the load on the Imaging Computing. This is achieved by integrating the Correlation results. However, integration has the effect of smearing the data and reducing the amplitude from the peak response to a point source due to the effects of the Earth’s rotation through the fringes. Currently the DRM calls for less than 2 percent smearing which corresponds to 32 > 0.98 ∆ The upper frequency taken to calculate the integration time is either the maximum capability of the receptor or the upper frequency of the Science requirement depending on which is the lower. On this basis, more than one receptor technology may be required to provide the required frequency coverage. In the more general case, the frequency coverage of individual technologies may differ from the values quoted and may even provide overlaps in frequency across technologies. This provides the option of correlation across receptor technologies which are likely to have differing sample rates. In this case, interpolation of the data streams is required to provide sample level time alignment for the correlation process. Further work is required to identify the associated processing load and evaluate the merits of correlating across receptor types. 7.4.2 Correlator Processing Load The correlator is full Stokes with a processing load proportional to the square of the number of antennas, Na and signal bandwidth, BW. Nominally, the load is independent of the number of frequency channels 2011‐03‐29 Page 30 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 1
2
4
/
For dishes SKA phase 1 250 250
2
1
4 2.55 10 2 1440 4 8 ~ 8 /
For Sparse Aperture Array 50 50
2
1
8 ~ 42 380 /
10 2
The memory requirements for the Correlator output are potentially significant with the compounded effect of the number of baselines, the number of channels Nchan and number of beams Nbeams 1
2
For SKA phase 1 dishes 250 250
2
1
1 1 10
10
~ 125 1
4 For SKA phase 1 Aperture Arrays 50 50
2
1
380 10
1 10
~ 1 480
4 As mentioned previously the number of channels increases with baseline length. Only storing the required number of channels for each baseline can reduce the memory requirements but at the expense of complexity of the memory management. A potential compromise could be to use a few block ranges of baseline length with associated numbers of channels. The effectiveness is quite high due the high percentage of antenna within the core. 2011‐03‐29 Page 31 of 59 8
WP2‐040.030.010‐TD‐001 Revision : 1 Central Beamformer At present it is assumed that there will be thirty five 180 metre diameter AA_Low arrays in the central 5 km core and 175 15 metre diameter dishes and these will be used for phase 1 Pulsar Searching . The following frequency ranges are being considered for SKA1 AA_Low: 350 – 450 MHz. Dish1: 450MHz – 1 GHz Dish2: 1GHz – 2 GHz Dish3: 2GHz – 3 GHz In general, the Field of View of diameter D at wavelength, λ, is: 4
1.2 180
The SKA1 DRM [45] requires a 36,000 deg2 surveys to be completed within two years. Assuming the survey is made up of individual 600 second observations and that only 200 days of the 2 years are used for different observations(to allow time for repeat observations and calibration), then 1.25 deg2 are required per observation. Consequently, the number of beams from a 180m diameter station to fill the required 1.25 square degrees for pulsar processing at the upper frequency of 450MHz is: 1.25
4
1.2 180
1.25
4
1.2 180
450 3 10
10 180 25 This does not include any over‐lapping of beams that may be required. The total data rate from NSAA aperture arrays with bandwidth BSAA is: 35 25 450
350 10 2
2
2 2 4
1 / The total data rate from Ndish dishes with instantaneous bandwidth Bdish is: 2011‐03‐29 Page 32 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 2
2 175 1000
450 10 2
_
175 2000
1000 10 2
2 4
_
175 3000
2000 10 2
2 4
_
2 4
1.5 / 2.8 / 2.8 / 8.1 Buffer A buffer is provided to compensate for the time of arrival of data arriving from the Sparse Aperture Arrays. The maximum difference in time of arrival for phase 1, ttr, is the time for optical signals to travel 2.5 km which is of the order of 8.3 us. The minimum memory requirement for AA_low is then: 480
2
225 2
For dishes: 2
25 2
8.2 Voltage Storage The incoming data rate from the AA_Low Arrays for 1.25 square degrees FoV has been shown to be of the order of 1 T bits per second for phase 1 and dishes up to 2.8 T bits/s. The observation time Tobs is of the order of 600 seconds. Consequently, to store an observation’s worth of receptor data requires at least 210 T Bytes of disk storage. This does not include the overhead for metadata which is assumed to be of the order of 10% 8.3 Beamforming Beam‐forming allows individual receptor elements to be combined in such a way that the resultant beam can be steered. To maintain optimum sensitivity beam‐forming should ideally be performed coherently either by introducing finely controlled time delays or – under narrowband conditions – phase delays. Central beam‐forming for SKA1 is used to form beams across dishes or AA_Low station beam sets to meet the requirements of the pulsar survey and timing chapters of the DRM including the survey “on‐sky” time. 2011‐03‐29 Page 33 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Smits et al [41] show that beam‐forming is more efficient if implemented hierarchically using sub‐
arrays and the number of operations for the beam‐forming is Nosb: 2
2
_
where B is the Bandwidth and ξ is a factor to allow for the possible extension of the FoV by the use of PAFs which takes the value of one for single pixel feeds. For aperture arrays the processing load is given by: 1.25
2
180
2
2
Receptor Bandwidth G Hz Nosb N beams Dish 2GHz to 3GHz Dish 1 GHz to 2GHz 6 x 1015 operations 111,111* Dish 0.45 to 1GHz 3 x 1015 operations 111,111** AA_low 0.35 to 0.45 GHz 8 x 1013 operations 18,981 * Dish FoV less than 1.25 deg2 2 **Dish FoV more than 1.25 deg
Smits also suggests that the second stage of hierarchical beam‐forming might be incoherent to reduce the number of beams required. However this reduces sensitivity by a factor of Beam‐former Output rate: ∆ 2
_
The number of bits at the beam‐former output to ensure there is no clipping is 14 which assumes for each beam a single 4 bit multiply (array data and coefficient) followed by 35 accumulates into a 14 bit accumulator. . 2011‐03‐29 Page 34 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 9
De‐Dispersion bdd [block] system [DeDispersion definitions]
«block»
DeDispersion
«block»
Incoherent
«block»
Coherent
«block»
Delay & Sum
«block»
Accumulation
& Difference
«block»
PreSumming
«block»
Taylor Tree
«block»
Multiple
Sample
Period
«block»
Frequency
Partitioning
9.1 Incoherent Dedispersion Incoherent de‐dispersion involves processing the received signal after it has been detected, that is after it has been channelized into its spectral components (via a filterbank) and after the signals from each of its channels have been converted to intensity‐like quantities via a square‐law detector. At this point, processing is deemed to be incoherent since the resulting signals do not contain any phase information. Incoherent de‐dispersion corrects for dispersion by advancing the spectral components of the signal by the dispersive delays predicted for an assumed dispersion measure. As lower frequencies are dispersed more than higher frequencies, this is achieved by delaying the higher frequency components so that they coincide with the expected arrival time of the lowest frequency component. The re‐aligned signal components are then summed together to produce a de‐dispersed version of the input signal for the assumed DM. Delays and sums are common digital signal processing operations. In digital de‐dispersion systems, where each signal channel consists of a digitised stream of samples, significant delays can be applied to individual channels by storing samples within memory. The storage forms a frequency time array from which samples can be retrieved and summed using digital accumulators. 2011‐03‐29 Page 35 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 9.2 Delay and Sum Dedispersion A delay‐and‐sum approach to de‐dispersion, described in [31], involves summing the frequency‐time samples for each of the assumed dispersion measures in turn, completing the sums for all dispersion measures within one sample interval, then advancing the frequency‐time array by a sample interval and commencing the next round of additions. This procedure suffers from the following memory bandwidth inefficiencies: a. Some samples are needed in the sums for different dispersion measures and are read from memory multiple times within a sample interval. b. The samples needed for de‐dispersion are distributed in small clumps through‐out the frequency‐time array. This inefficiency is particular to SDRAM technology which relies on burst accesses to contiguous memory locations in order to achieve high access bandwidths. By itself the delay‐and‐sum algorithm scales poorly with the number of signals being de‐dispersed and with finer time and frequency resolutions. For more scalable systems, additional and/or alternative techniques are necessary to overcome these inefficiencies. 9.3 Pre‐summing channels for large dispersion measures With delay‐and‐sum de‐dispersion, summations for large DMs consume a greater proportion of memory bandwidth than those for small DMs, because they are dispersed across a greater number of samples in the memory's frequency‐time array. For large DMs, most of the array samples that need to be summed occupy successive time intervals within common frequency channels. This latter fact can be exploited by summing the samples before storing them to memory so that fewer samples need to be read back from memory for de‐dispersion. One such scheme involves several levels of sample integrations, with separate frequency‐time arrays for each integration level. The lowest integration level, level 0, is used to de‐disperse the lowest group of DMs using the delay‐and‐sum algorithm described earlier. For level 1, every pair of samples are summed and the summed samples are stored to the frequency‐time array for level 1. The level 1 frequency‐time array is used to de‐disperse the next group of larger DMs. Likewise, every pair of level 1 samples are summed and stored to the frequency‐time array for level 2, which is used to de‐
disperse the next group of larger DMs; and so it goes on for the higher levels. Note that the sample times of the level i samples become twice the sample times of the level i‐1 samples, thus de‐dispersions at higher levels are performed at courser and courser time resolutions. Generally, this reduces the signal‐to‐noise ratio of the de‐dispersed signals, but courser time resolutions have less of an impact on the SNR of more highly dispersed signals, because they are more temporally dispersed and scattered [35], so the reduction in SNR can be controlled by careful selection of the DMs for each level. Using this pre‐summing technique with only 4 levels, while maintaining more than 90% of the SNR, the memory bandwidth can be reduced by a factor of 5. 2011‐03‐29 Page 36 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 9.4 Accumulating and differencing algorithm Another scheme devised to reduce the memory bandwidth by pre‐summing the samples is documented in [39]. In this scheme, the de‐disperser maintains an accumulator at its input for each frequency channel. The accumulators continuously sum the samples that the de‐disperser receives every sample interval for the corresponding channels, and the accumulators are allowed to wraparound when they overflow. Rather than storing the received samples to memory, the de‐
disperser captures the state of each accumulator to memory once every sample interval. To determine the sum of the samples for a given DM within a frequency channel, the de‐disperser need only read two values from memory: the accumulated value one sample before the start of the pulse dispersed within that channel, and the accumulated value at the end of the pulse dispersed within that channel. The de‐disperser reads these two values and differences them, thus reducing the sum of potentially many samples to a difference of just two accumulator values per channel. The number of bits per accumulator is necessarily larger than the number of bits per sample and therefore more memory is required to store the accumulations, but depending on the range of DMs searched, significant memory bandwidth can be saved using this approach. For low DMs where pulses are dispersed across less than two samples per frequency channel, this method actually increases the amount of processing and memory bandwidth needed, since it always retrieves and differences two accumulations per channel, whereas only one or two samples per channel would need to be retrieved from memory using the delay‐and‐sum approach. Greater optimization is therefore achievable by reserving this method for higher DM values. 9.5 De‐dispersion over multiple sample intervals The schemes described so far perform de‐dispersion operations for each DM independently, one after another, without taking advantage of the fact that most samples are needed in the de‐
dispersion calculations for other DMs. Also, for a given DM, they calculate each successive value of the dedispersed signal independently, without taking advantage of the fact that most samples are needed in the calculations for several successive de‐dispersion values. Within a period of multiple sample intervals, each sample is retrieved from memory only once and is reused across an array of parallel accumulators – one accumulator per DM per sample interval within the period. This technique improves memory efficiency by using larger period sizes, but at the expense of larger arrays of accumulators and greater latency (proportional to the size of the period). 9.6 Taylor tree based algorithms The de‐dispersion algorithms described to this point involve many redundant operations in that they add the same samples multiple times for different DMs. Taylor tree de‐dispersion [33] reduces processing by avoiding redundant additions performed within a sample interval across all DMs. A Taylor tree consists of a network of delay and sum elements inter‐connecting N inputs with N outputs. Figure 9 illustrates a four‐channel Taylor tree (N = 4) with delay elements represented by their Z‐transform. Each input, in, represents a channel of the dispersed signal, with iN‐1 being the channel of highest frequency. Each output, on, represents a de‐dispersed version of the signal, with 2011‐03‐29 Page 37 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 o0 having a DM of 0, and oN‐1 having the largest DM. The appealing feature of this structure is that the number of addition operations is equal to Nlog2N (whereas N2 additions are required when redundant additions are not avoided). Figure 9 4‐Channel Taylor Tree De‐disperser One of the shortcomings of Taylor trees is that they implement linear approximations to dispersion (not proportional to the inverse‐square of the frequency) which are less accurate for lower frequencies and wider bandwidths. However, [34] suggests that the input signal channels can be “linearized” by inserting redundant null channels between the existing channels. In effect, this spreads the signal out in frequency, with more spreading at lower frequencies so that the dispersion is linear before it is de‐dispersed through the Taylor tree. Another shortcoming of Taylor trees is that they search linear ranges of DMs from zero to the “diagonal” DM, i.e. the DM at which the dispersion gradient is one channel per sample interval. [34] suggests ways of extending the range of DMs searched by using an array of Taylor trees of equal size. The first tree in the array operates on the input channel samples to give a range of N dedispersions from a DM of zero to the diagonal DM. The second tree operates on a linearly delayed version of the input, where each channel is delayed in proportion to its channel number: channel 0 has no delay, and channel N‐1 is delayed by (N‐1) sample intervals. Thus the second tree produces another N DMs from the diagonal DM to twice the diagonal DM. For the third tree, the channel samples are summed in pairs and delayed as described above to give another N DMs from twice the diagonal DM to four times the diagonal DM; and so on. In this way the DM step size becomes exponentially larger in steps of N DMs. 9.7 Frequency Partitioning Frequency partitioning is another technique that can be used to reduce the overall processing required for de‐dispersion. The technique has been used in the Taylor tree de‐dispersion system described in [34] and Section 7.5.2 describes how it can be used with the delay‐and‐sum algorithm. The concept is simply to partition the channels into sub‐bands and to de‐disperse each frequency sub‐band individually. Better processing efficiency can be realised by virtue of the smaller frequency sub‐bands being less (and more linearly) dispersed. A second, coarser stage of dedispersion is needed to combine the de‐dispersed streams for each frequency sub‐band. 2011‐03‐29 Page 38 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 9.8 Coherent de‐dispersion An analysis in [35] shows that incoherent de‐dispersion can never completely remove the effects of dispersion from a signal, even if the dispersion measure is precisely known, because it only removes the inter‐channel effects of dispersion; it does not remove the dispersion that occurs within each finite width frequency channel. Coherent de‐dispersion on the other hand can (in theory) completely remove the effects of dispersion from a signal, given that the dispersion measure is known. Coherent de‐dispersion involves processing the received signal before it has been detected such that the signal maintains its phase information. Dispersion represents a rotation of the signal's phase in proportion to the inverse‐square of the frequency, and coherent de‐dispersion therefore involves rewinding the dispersive phase rotation. Essentially, this is a de‐convolution procedure in which the convolution function (the impulse response of the dispersive medium) has the form of a “chirp” pulse. Algorithms for performing coherent de‐dispersion using dedicated hardware require further investigation. 9.9 Concept sizing The SKA beams need to be buffered in channelized form for the length of an observation in order to be de‐dispersed at various dispersion measures and to resample for alternative accelerations. A typical search observation time Tobs would be of the order 10 minutes long with the number of samples accumulating to twice this value for optimising the subsequent FFTs. The corresponding DMmax for the frequency channel width, ∆ν, to restrict smearing to one time sample: 1
∆
20
3
8.3
103
∆
Larger DMs will temporally smear over a larger time frame which leads to the concept of the diagonal DM. This involves dropping the time resolution used in de‐dispersion in quantum factors of 2 as a function of DM whilst maintain the same frequency resolution. This relationship is correct if only interstellar dispersion is relevant, but interstellar scattering alters (reduces) the number of trial values needed. For large DMs the pulse broadening from scattering dominates the time resolution and so a coarser grid of DM values can be used as DM gets larger. The number of dispersion measures for a frequency band ranging from fmin to fmax in GHz and a sample time, tsamp, in microseconds: 2011‐03‐29 Page 39 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 4150 max
max
1
1
2
1
2
Table 3 provides details of the number of dispersion measures for both frequency ranges available from the dish and for the top 100 MHz of the sparse array for phase 1 of the SKA. The table assumes a DM diagonal where time resolution is traded against the maximum dispersion measure. To obtain high dispersion measure and high time resolution requires higher frequencies. This relationship is correct if only interstellar dispersion is relevant, but interstellar scattering alters (reduces) the number of trial values needed. For large DMs the pulse broadening from scattering dominates the time resolution and so a coarser grid of DM values can be used as DM gets larger. tsamp DMmax us Dish NDM DMmax NDM DMmax NDM DMmax NDM Dish Dish Dish Dish Dish Sparse AA Sparse AA 2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 GHz GHz GHz GHz GHz GHz GHz .35 ‐ .45 GHz 50 2400 27,778 300 18,700 27 8,972 13 3,457 100 4800 13,889 600 9,350 54 4,486 26 1,728 200 1,200 9,350 108 4,486 52 1,728 400 2,400 9,350 216 4,486 103 1,728 800 4,800 9,350 432 4,486 207 1,728 1,600 9,375 864 4,486 413 1,728 Table 3 DM Diagonal 2011‐03‐29 Page 40 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Figure 10 Dispersion measure, DM, for pulsars at different galactic latitudes.1 The Dispersion measure as a function of galactic Latitude is shown in Figure 1 which provides a graphical indication of the amount of sky that requires high dispersion measure searching. It can be envisaged that the Sparse AAs could be used to search off the galactic centre where the dispersion measures are not so high and the dishes used (possibly simultaneously) for the galactic centre. To estimate roughly the processing load, a hybrid version of Taylor tree dispersion is considered within the limitations detailed in section 9.6. The basic Taylor Tree structure is shown in Figure 9 for a 4 channel implementation. This structure is adapted to take advantage of the diagonal DM methodology where time resolution is traded against Dispersion Measure. Assuming zero padding of the number of frequency channels to the number of dispersion measures, the processing load per second for each element along the DM diagonal using a Taylor tree is Nd_ops: _
1
1
(adapted from B. Klein (MPIfR) unpublished] and taken from Tools of Radio Astronomy: Wilson,
Rohlfs and Huttemeister)
2011‐03‐29 Page 41 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 tsamp NDM us ND_ops NDM ND_ops NDM ND_ops NDM ND_ops (G MACs) (G MACs) (G MACs) (G MACs) Dish Dish Dish Dish Dish Dish Sparse AA Sparse AA
2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 .35 ‐ .45 GHz GHz GHz GHz GHz GHz GHz GHz 50 27,778 8.2 18,700 5.3 8,972 2.4 3,457 0.8 100 13,889 1.9 9,350 1.2 4,486 0.5 1,728 0.2 200 9,350 0.6 4,486 0.3 1,728 0.09 400 9,350 0.3 4,486 0.1 1,728 0.05 800 9,350 0.2 4,486 0.07 1,728 0.02 1,600 4,486 0.03 1,728 0.01 Table 4 Dedispersion Processing loads per beam The total processing load : _
2011‐03‐29 _
~ 21 Page 42 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 In addition there are also delay elements per diagonal DM element, Ndly which for large DM approximates to: ~ 4
4
Ignoring any zero padded channels, the output rate, Gdps is: 1
tsamp NDM us Gdps NDM Gdps NDM Gdps NDM Gdps (M bit/s) (M bit/s) (M bit/s) (M bit/s) Dish Dish Dish Dish Dish Dish Sparse AA Sparse AA
2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 .35 ‐ .45 GHz GHz GHz GHz GHz GHz GHz GHz 50 27,778 556 18,700 375 8,972 179 3,457 69 100 13,889 139 9,350 94 4,486 45 1,728 17 200 9,350 47 4,486 22 1,728 9 400 9,350 23 4,486 11 1,728 4 800 9,350 12 4,486 6 1,728 2 1,600 4,486 3 1,728 1 Table 5 Dedispersion Output Rate per beam2 2
Assumes data is truncated to 4 bits
2011‐03‐29 Page 43 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 10 Pulsar Search SKA Memo 125 provides has identified two major science goals that are to drive the technical specifications for the SKA1. One of these is: ‘Detecting and timing binary pulsars and spin‐stable millisecond pulsars in order to test theories of gravity (including General Relativity and quantum gravity), to discover gravitational waves from cosmological sources, and to determine the equation of state of nuclear matter.’ 10.1 Binary Search The detection of binary pulsar systems as part of a pulsar search requires algorithms that are capable of compensating for loss of sensitivity caused by the pulsar’s elliptical orbital motion as a result of the Doppler component. Ralph Eatough’s 2009 PhD thesis [43] provides an overview of the common time and frequency domain techniques that have been developed to compensate for the effects of pulsar orbital motion and are represented in Figure 11. Non binary systems can be considered as a special case where the Doppler component is zero. bdd [block] system [Binary search definitions]
«block»
Binary Search
«block»
Matched
Filter
«block»
Coherence
Recovery
«block»
Stack Search
«block»
Hough
Transform
«block»
Phase
Search
«block»
Time Domain
Resampling
Figure 11 Binary Pulsar Search Algorithms A brief overview of each technique is provided in the following sections. 10.1.1 Matched Filter An alternative method of conducting "constant acceleration" searches uses complex matched filtering in the Fourier domain as opposed to re‐sampling of the de‐dispersed time series. The local (meaning only those near the Fourier frequency of interest) complex Fourier amplitudes from the 2011‐03‐29 Page 44 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 FFT of a time series are convolved with analytically computed templates to generate optimally sampled two‐dimensional portions of the frequency‐frequency derivative (or f‐fdot) plane. The templates may be thought of as digital filters whose lengths represent how many Fourier bins a signal linearly drifts during an observation. The number of bins drifted is a parameter typically called 'z', which can be directly related to acceleration 'a' via: where T is the observation duration, f is the pulsar spin frequency (or harmonic of its spin frequency), and c is the speed of light. Since the templates only depend on 'z' (and not on the spin frequency) they may be pre‐computed and stored. A range of them will efficiently generate horizontal slices in the f‐fdot plane via the FFT convolution theorem. N independent fdot (or 'z') slices in an f/fdot plane of length 'M' Fourier bins (where M is typically << the number of bins in the input time series) can be computed with roughly 2 5 (for a forward and inverse FFTof the M Fourier amplitudes and N * M multiplications to apply the filters). This process is repeated as many times as is necessary such that the full Nyquist frequency range of the input time‐series is covered. There are three significant advantages to the Frequency‐domain method of acceleration searching over time‐domain re‐sampling. 1. The correlations used to create blocks of the f‐fdot plane are memory local and can be computed very rapidly as the input Fourier amplitudes and templates stay in cache. This memory locality and the fact that many templates are applied to a single set of input Fourier amplitudes, means that each slice in the f‐fdot plane is independent and can be easily computed via GPUs or threads in a CPU. In fact, a GPU version of the most popular Fourier‐
domain acceleration search code ('accelsearch' from PRESTO) is currently in development. The fact that all of the acceleration "trials" are computed en masse, without the need to re‐
sample and re‐FFT each long input time series can result in factors of several computational speed‐up. 2. The f‐fdot plane can be uniformly and optimally sampled (including "interbinning" to reduce the effects of the "scalloped" Fourier response with frequency). This is in stark contrast to time‐domain acceleration searches which are only optimally spaced at a single frequency. At lower frequencies, the time‐domain method oversamples the f‐fdot plane, while at higher frequencies the f‐fdot plane is under sampled and Fourier power for higher harmonics can be lost. 3. Related to #2, computing an f‐fdot plane up to +/‐ some value of 'z' gives different ranges of acceleration based on the spin frequency of the pulsar (see the acceleration ‐ 'z' relation given above). For slow‐ish pulsars (as one might expect to detect around a young‐ish black hole), much larger acceleration ranges would be probed than an equivalent time‐domain acceleration search that took the same amount of computer time. 2011‐03‐29 Page 45 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 One disadvantage to Fourier‐domain acceleration searching is that harmonic summing is signicantly more complicated than for time‐domain re‐sampling. This is because two‐dimensional portions of the f‐fdot plane must be summed as opposed to one‐dimensional portions of the simple power‐
spectrum. If the full f‐fdot plane can be computed and kept in memory, which may very well be possible for large‐scale surveys with the SKA (where time series are likely to be 5‐20 min in duration), the harmonic summing calculations and book‐keeping will be dramatically simplified. The reference for Fourier domain acceleration searching is Ransom, Eikenberry & Middleditch 2002, AJ, 124, 1788. 10.1.2 Hough Transform The original Hough algorithm developed by Paul Hough in the 1960s maps a straight line y = ax + b in the (x, y) plane into (a, b) parameter space where it becomes a point. Consequently a point in (x, y) space can be represented as a line of form b = y − ax in parameter space. Many points arranged in a preferred direction in the (x, y) plane would appear as lines in the (a, b) plane that converge at a particular a and b that parameterize the line. The principle can be extended to almost any functional form in the (x, y) plane although this results in higher dimensional parameter spaces. The method has been used in the PhD thesis of Aulbert (2005) to search for sinusoidal tracks left by binary pulsars in dynamic power spectra. 10.1.3 Stack Search A stack search works by simply chopping the time series up into a number of smaller segments (Wood et al., 1991). Each segment is then Fourier transformed and the segments are summed together with various offsets corresponding to different frequency drift rates i.e. different acceleration trials. Typically, only linear offsets (constant accelerations) are applied but, since the algorithm is efficient there is no reason why quadratic and even cubic frequency offsets could not be searched. The configuration of summed spectra with the binary candidate showing highest SNR should be given by the correct acceleration trial. Unfortunately, by splitting the time series into segments and operating on them separately the phase information of the observation is lost and the spectra are summed incoherently. This results in a 30% reduction in SNR compared to the equivalent spectral SNR of the solitary pulsar (Faulkner, 2004). 10.1.4 Phase Search The phase search implemented in Scott Ransom’s software Presto2 essentially performs a number of short DFTs over different parts of the fluctuation spectrum. For very short period binaries where the observation length covers at least one orbit, the phase modulation search can be applied (Jouteux et al., 2002, Ransom et al., 2003). The effect of these very short orbital periods is to create sidebands in the Fourier power spectrum. DFTs are used to sum any sidebands (collecting the power). Both the orbital and pulsar periods can be found using this method. 10.1.5 Coherence Recovery This frequency domain technique (Ransom et al., 2002) involves taking an FFT of the time series to generate the power spectrum with the pulse smeared over a number of spectral bins. The functional 2011‐03‐29 Page 46 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 form of the smearing is known so it is possible to collect the power back into one frequency bin by using a filter that is the complex conjugate of the smearing function. The filter is applied over all frequencies in the spectrum to look for periodicities corresponding to particular constant accelerations. The method is in principle computationally efficient as an FFT of the entire time series only needs to be performed once. 10.1.6 Time Domain Resampling In the time domain the equivalent technique is time domain resampling. Here the time samples are transformed into a frame inertial with respect to the pulsar, but instead of a search to find the exact form of v(t) a constant acceleration is assumed, i.e. v(t) = a0t. A time interval in the observers frame t can then be transformed into the pulsar frame τ by simple application of the Doppler formula, 1
1
The constant τ0 is chosen such that τ = tsamp at the midpoint of the observation (e.g. Camilo et al., 2000). New samples are computed from a linear interpolation over the original time series (Middleditch & Kristian, 1984). Following resampling the time series is then searched with the standard FFT techniques. 10.2 Time Domain Re‐Sampling This section takes the Time Domain Re‐Sampling binary search case identified as one of the options in section 10.1.6 and provides a ball park estimation of the processing loads and data rates. This is not meant to represent a preference for the algorithm. The development phase will investigate and model each of the algorithms in more detail to determine which is optimal for the SKA. Assuming constant acceleration for the re‐sampling and that the maximum pulse smearing is tsamp. Then, for samples lying exactly between acceleration trials, 2
Substituting t = T/2 and letting allows the acceleration step size to be calculated 8 8 3 10
600
2011‐03‐29 Page 47 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Assuming that the time series for the lowest DM is decimated by a factor of 2, the following table details the number of trail accelerations as a function of sample time: tsamp us δa ms‐2 Nacc 100 (50) 0.66 (0.33) 303 100 0.66 303 200 1.3 152 400 2.7 76 800 5.3 38 1,600 10.6 19 Table 6 Number of trial Accelerations3 The processing load at each time resolution: _
1
_
Assuming Nops_dm = 2 tsamp us NDM ND_ops NDM ND_ops NDM ND_ops NDM ND_ops (G Macs) (G Macs) (G Macs) (G Macs) Dish Dish Dish Dish Dish Dish Sparse AA Sparse AA
2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 .35 ‐ .45 GHz GHz GHz GHz GHz GHz GHz GHz 100 (50) 27,778 337 18,700 227 8,972 109 3,457 42 100 13,889 84 9,350 57 4,486 27 1,728 11 200 9,350 14 4,486 7 1,728 3 400 9,350 4 4,486 2 1,728 0.7 800 9,350 0.9 4,486 0.4 1,728 0.2 1,600 0.2 4,486 0.1 1,728 0.04 Table 7 Re‐sampling Processing Load per beam4 3
4
Assumes + 100ms-2 acceleration range
Assumes Nacc of Table 6
2011‐03‐29 Page 48 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 The output rate, Grsps is: 1
tsamp NDM us Grsps NDM Grsps NDM Grsps NDM Grsps (G bit/s) (G bit/s) (G bit/s) (G bit/s) Dish Dish Dish Dish Dish Dish Sparse AA Sparse AA
2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 .35 ‐ .45 GHz GHz GHz GHz GHz GHz GHz GHz 50 27,778 168 18,700 114 8,972 54 3,457 21 100 13,889 42 9,350 28 4,486 14 1,728 5 200 9,350 7 4,486 3 1,728 1 400 9,350 2 4,486 0.9 1,728 0.3 800 9,350 0.4 4,486 0.2 1,728 0.08 1,600 4,486 0.05 1,728 0.02 Table 8 Time Re‐sampling output rates 10.3 FFT The processing load at each time resolution: _
tsamp us 1
5 Nrs_ops Nrs_ops Nrs_ops Nrs_ops (G Macs) (G Macs) (G Macs) (G Macs) Sparse AA Dish Dish Dish SparseAA 2 ‐ 3 1 ‐ 2 0.45‐1 GHz GHz GHz .35‐.45 GHz 2 – 3 GHz 1‐2 GHz 100 27,778 18,700 8,972 3,457 303 600 3960 2670 1280 493 100 13,889 9,350 4,486 1,728 303 600 948 640 306 118 200 9,350 4,486 1,728 152 600 153 73 28 400 9,350 4,486 1,728 76 37 18 7 Ndm Ndm Ndm Ndm Dish Dish Dish 2011‐03‐29 Nacc Tobs 600 .45 ‐ 1 .35 ‐ .45 GHz GHz Page 49 of 59 tsamp us 800 WP2‐040.030.010‐TD‐001 Revision : 1 Ndm Ndm Ndm Ndm Dish Dish Dish Nrs_ops Nrs_ops Nrs_ops Nrs_ops (G Macs) (G Macs) (G Macs) (G Macs) Sparse AA Dish Dish Dish SparseAA 2 ‐ 3 1 ‐ 2 0.45‐1 GHz GHz GHz .35‐.45 GHz 2 – 3 GHz 1‐2 GHz 1600 Nacc Tobs .45 ‐ 1 .35 ‐ .45 GHz GHz 9,350 4,486 1,728 38 600 9 4 2 4,486 1,728 19 600 1 0.4 Table 9 FFT Processing Load per Beam The output rate, Gfftps is: 1
This is the same as the Input rate 10.4 Whitening and Normalisation According to Kramer and Lorimer [40]‘fluctuations in the receiver and/or data acquisition systems often manifest themselves via a significant low‐frequency or red noise component when viewed in the Fourier domain’. It is standard practice to whiten and normalise the spectrum prior to any detection processing. 10.5 Harmonic Sum Harmonic summing provides a gain in sensitivity over that provided by a single harmonic analysis. The energy in the harmonics is a function of the duty cycle of the pulsar pulse. 32 harmonics is required for slow and un‐accelerated pulsars and 8 harmonics are optimal for milli second pulsars and acceleration searches. This is because the accuracy of the "linear" acceleration approximation is proportional to 1/f. So the higher harmonics see more and more non‐ linear effects and therefore contribute less and less to the accumulated signal to noise ratio. In fact, historically, in most cases, binary pulsars have been detected in only 3 harmonics. The exact boundary between the use of 32 and 8 harmonics is still to be determined. For estimating the processing load it is assumed that summing of up to 8 harmonics for acceleration processing and is provided by the processing chain for each beam, dispersion measure and acceleration trial. This is implemented by stretching the power spectrum, in the frequency dimension, across an observation by factors of two. To cover 8 harmonics requires this process to occur 3 times. Consequently, the processing load, Ghs : 2011‐03‐29 Page 50 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 1
tsamp us Ndm Ndm Ndm Ndm Dish Dish Dish Nhs Nhs Nhs Nhs (G Macs) (G Macs) (G Macs) (G Macs) Sparse AA Dish Dish Dish SparseAA 2 ‐ 3 1 ‐ 2 0.45‐1 GHz GHz GHz .35‐.45 GHz 2 – 3 GHz 1‐2 GHz 100 27,778 18,700 8,972 3,457 303 600 3234 2184 1044 402 100 13,889 9,350 4,486 1,728 303 600 810 545 261 101 200 9,350 4,486 1,728 152 600 137 65 25 400 9,350 4,486 1,728 76 600 34 16 7 800 9,350 4,486 1,728 38 600 8 4 2 4,486 1,728 19 600 1.2 0.4 1600 Nacc Tobs .45 ‐ 1 .35 ‐ .45 GHz GHz Table 10 Harmonic Sum Processing Load for acceleration Processing per Beam 2011‐03‐29 1
4 Page 51 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 tsamp NDM us Grsps NDM Grsps NDM Grsps NDM Grsps (G bit/s) (G bit/s) (G bit/s) (G bit/s) Dish Dish Dish Dish Dish Dish Sparse AA Sparse AA
2 – 3 2 – 3 1 – 2 1 – 2 .45 – 1 .45 – 1 .35 ‐ 45 .35 ‐ .45 GHz GHz GHz GHz GHz GHz GHz GHz 50 27,778 168 18,700 114 8,972 54 3,457 21 100 13,889 42 9,350 28 4,486 14 1,728 5 200 9,350 7 4,486 3 1,728 1 400 9,350 2 4,486 0.9 1,728 0.3 800 9,350 0.4 4,486 0.2 1,728 0.08 1,600 4,486 0.05 1,728 0.02 Table 11 Harmonic Sum Output Rates per Beam 15 / 10.6 Threshold Detection The resultant data from the harmonic sum is searched for power components that exceed a threshold determined by the acceptable false alarm rate. The threshold level is determined by: /
1
4
4
Where Nsamp is the number of samples of the spectrum 10.7 Candidate Filtering Large all sky surveys for radio pulsars produce extremely large numbers of candidate pulsars. As discussed in Eatough et al (2010)[43] the most recent Parkes Mutli‐beam Survey reprocessing resulted in more than 8 million candidates. While tools have been established which allow for more efficient selection of which of these candidates to view have been established these typically only reduce the number of candidates by about an order of magnitude. This still leaves a significant problem as even 1 million candidates requires years of effort to view. The location of both phases of the SKA in predominantly radio quiet regions will decrease the influence of interfering signals on the number of candidates. Moreover the multi‐beam nature of the telescopes will also provide a very effective anti‐coincidence filter, which should also reduce 2011‐03‐29 Page 52 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 spurious detections. However, the number of candidates produced in surveys with the SKA will still greatly out strip those produced by any previous survey. This is mainly due to the extreme sensitivity, the amount of sky that will be surveyed, the high time resolution, large number of dispersion measures and acceleration trials. It is also more than likely that the processing of the pulsar survey data will be happening in real time and so further processing of data to try and confirm candidates may not be possible. So robust methods to identify candidates in the data sets will be required to ensure that confirmation observations are effient and if possible only the data sets from only the best candidates are kept for further inspection. 10.7.1 Artifcial Neural Nets Recently Eatough et al (2010) and Bates et al (2011) have introduced the idea of using Artificial Neural Nets for the selection of candidates for subsequent viewing. These algorithms were applied to a reprocessing of the Parkes Multi‐beam Survey and the new High Time Resolution Survey with Parkes. The basis of the neural net approach to sorting pulsar candidates is to try to describe the plots which are viewed by eye, and use the natural pattern recognition of humans, with a set of numbers, or scores, which can be used to identify common traits of pulsars. A set of known pulsars and a set of "not‐pulsars" are used to train the neutral net, the resultant "net" is then applied to a validation set to determine how well it is doing. Eatough et al established that about 92% of all known pulsars in a sample of some 2.5 million candidates and Bates et al extended this work and applied it to a larger set of known pulsars and added more scores but found a total recovery rate of about 85% for pulsars with periods longer than about 100 milliseconds. They find that there is a clear relationship with the pulse duty cycle and the ability to recover the pulsar and so further improvement of these techniques is possible. It should be pointed out that applying the neural net reduces the number of candidates that need to be viewed by more than an order of magnitude and has already helped discover a few 10's of pulsars. 10.7.2 The Future Further investigation of the appropriate scoring scheme for identifying pulsars is currently underway. In particular improving the performance of the nets for the detection of millisecond pulsars will be essential. It is important to note that the higher time resolution of the HTRU survey over the PMB surveys has already improved the response to MSPs. Another area of active research is which neural net algorithms provide the best performance. At present relatively simple and old algorithms have been tried and there are efforts on going to improve this. It will also be important to investigate the performance of these nets to finding radio transients. We have started on applying them to the RRAT sources, but so far with less success than for radio pulsars, but this likely reflects the usefulness of the scores being used. 10.7.3 Application to the SKA The large number of candidate sources that will be revealed in the SKA pulsar and fast transient surveys will be a vital aspect of the whole processing and observing effort. To minimise follow up 2011‐03‐29 Page 53 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 observations robust candidates need to be identified, but it is also vital that the number of candidates to be viewed manually be greatly reduced. While the application of Neural Nets to pulsar searches is still in it's infancy it has shown great promise. Moreover it doesn't add a significant processing load to the overall pulsar processing signal chain. On existing 2.6 GHz CPUs it takes just a few seconds to produce the necessary scores for each candidate. If there are less than a few tens of candidates per beam this means that it always takes much less than the processing time to find the candidate, than to generate scores. Once these scores have been accumulated, running the Neural Net on a few thousand potential candidates also takes just a few minutes on a single core 2.6 GHz CPU. The use of some kind of system for reducing the number of pulsar and fast transient candidates that need to be viewed manually will be essential in the SKA era. These algorithms need not add significantly to the overall processing load and will greatly improve the observing efficiency by reducing the number of false positives. They will also significantly decrease any data products that need to be archived. Development work is still required in determining the best possible set of scores, algorithms and training methods for the pulsars and especially for the transients. 11 Pulsar Timing One of the key goals of the SKA in both phase 1 and 2 will be to perform high precision timing of known pulsars in order to test theories of gravity and to detect and study a gravitational wave background. An important aspect of this pulsar timing is the initial, post‐discovery, timing of pulsars discovered with the SKA in the Galactic Census observations of phase 1 and 2. It is not until this initial timing solution is hand that we can make an initial assessment of whether a pulsar is interesting or not. For the majority of the pulsars discovered in the survey, modest timing precision is required, while for the high precision timing of a much smaller number of pulsars, such as the millisecond pulsars and those in binaries, the highest demands in time and frequency resolution, calibration, collecting area and cadence will be set. This section of the document describes the technical and operational requirements for achieving the stated headline and key science of the SKA in Phase I and II. It does not address the issue of polarization calibration which is addressed elsewhere. 11.1 Basic Parameters The basic functional requirements for a timing programme with the SKA is described in this section of the document. It consists of three possible parts, all of which required for the headline science: 
Phase 1 
Phase 2 
Galactic Centre The optimal timing parameters are highly source dependent, i.e. they depend on whether the pulsar is a young, normal pulsar (compared to an old millisecond pulsar), the flux density and pulse jitter of 2011‐03‐29 Page 54 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 the source and whether the pulsar is in a binary orbit. Moreover, timing observations of newly discovered pulsars have to be done with a different cadence than known pulsars timed for specific experiments, such as gravitational wave detection or tests of theories of gravity. Indeed, the specifics of pulsar timing make the requirements different from most other pulsar timing projects; in particular as a large Field‐of‐View will not compensate for a reduced sensitivity. This is in particular the case for studying the fast binary pulsars, where sufficient sensitivity must be available to ``resolve'' the orbit with appropriate time‐of‐arrival (TOA) measurements that cover the orbit in small enough intervals. Hence, it is important to collect enough flux density for having a precisely measured TOA with an integration time as short as possible. On the other hand, while raw sensitivity (collecting area and bandwidth) is extremely important, it is not the only criterion that determines the observing requirements. See further below 11.2 Timing scenarios Three different scenarios are considered PhaseI/PhaseII/Galactic Centre Pulsars to be timed (point sources): 5000/25000/50 Single Obs Duration: > 120 s / many hours Integration/Pointing from monthly (1000's of sources), biweekly (~100 sources), daily for few hours (for fast binary systems)/daily Number of Stations: variable (multi‐beaming, sub‐arraying, full array)/full array Collecting area, A/T: >1000 m2K ‐1 / >10,000 m2K‐1 / 5,000 ‐ 10,000 m2 K‐1 Diameter of Stations: non‐critical, array assumed to be phased up/whole array Size of Core: non‐critical, array assumed to be phased up Frequency: 500 ‐ 3000 MHz / 10 ‐ 15 GHz Bandwidth: >20%, ideally 500‐3000 instantaneously / 4 GHz Sampling Time: 0.2 us /1us 11.3 Monitoring and Cadence: Pulsar timing requires the regular observations of pulse arrival times with high precision. The latter scales, to first order, with signal‐to‐noise ratio which is larger at lower frequencies due to the steep spectrum of pulsars. For normal timing observations of known pulsars monthly or bi‐weekly observations are sufficient. However, if the pulsar is in a binary orbit, dense coverage of all orbital phases is required. For orbits of a few hours, this can be achieved in a single session while pulsars with orbital periods of days, weeks or months need to be covered in appropriate intervals. For new pulsars, a 'timing solution' needs to be obtained first. This usually requires dense observations at the 2011‐03‐29 Page 55 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 start with increasing intervals between the observations (i.e. from minutes to hours, days and weeks). Obtaining a full timing solution also requires coverage over a year (in order to break a correlation between the pulsar position and the pulsar spin down). Solving a pulsar and obtaining a full solution can be shortened if the position of the pulsar can be measured at the beginning using imaging over large baselines to arcsec precision. 11.4 Observing Frequency and Bandwidth The optimal observing frequency for high precision timing is usually between 1‐3 GHz, as a compromise that takes into account the steep spectrum of pulsars and the effects of the interstellar weather which become reduced at high frequencies. The ionized interstellar medium (ISM) disperses the signal and the ISM's inhomogeneities lead to a scattering of the signal via multi‐path propagation. While the former can be accounted for with optimal de‐dispersion techniques, the latter usually cannot but the effects scale inversely with frequency with about the fourth power. Moreover, the turbulent properties of the ISM mean that the effective dispersion and scattering properties vary on a number of timescales, so that they should be determined with quasi‐
simultaneous multi‐frequency or wide bandwidth observations. Large bandwidth also reduces the impact of interstellar scintillation which can cause the brightness of a point source like pulsars to vary significantly. The ISM effects are in particular severe for pulsars in the Galactic plane. For the Galactic Centre region, timing (and searching) will have to be conducted at 10 GHz or even higher. Here, however, the size of the region to be studied and monitored is small and localized. In contrast, pulsar signals from sources outside the Galactic plane suffer less from ISM effects, so that for timing observations between 500‐1000 MHz may be sufficient. Memo 130 assumes for Phase I frequency coverage of 0.45‐1 GHz and 1‐2 GHz for dishes with Single Pixel Feeds and 70‐450 MHz with aperture arrays. In this case, high precision timing for pulsars in the plane would be conducted with dishes between 1‐2 GHz, while the bulk of the sources will be timed between 0.45‐1 GHz with dishes or at 70‐450 MHz with aperture arrays. Typically, a bandwidth of 20% of the centre frequency is demanded for timing observations. For high precision measurements, a larger fractional bandwidth is highly desirable, in particular to combat ISM effects. A simultaneous coverage of frequencies between 0.4 and 3 GHz would also eliminate the need to re‐observe the same source at multiple frequencies to account for interstellar weather. Given Memo 130, a larger fraction bandwidth (~500 MHz at low frequencies, ~1 GHz above 1 GHz) would be available, although it would split the SKA in sub‐arrays to achieve simultaneous coverage of larger frequency ranges. 11.5 Collecting area, beams and integration time: The time needed to achieve a precise TOA depends on two main factors: minimum signal‐to‐noise‐ratio (S/N~10 for normal pulsar timing, S/N >~100‐1000 for precision timing) and pulse jitter. The integration time for weak pulsars will be limited by the radiometer equation, while for strong pulsars a sufficient number of pulses need to be added before a stable pulse profile is reached, regardless of the S/N (e.g. Liu et al. 2011). In the latter case, a few thousand pulses should be added. For typical periods of millisecond pulsars, a few minutes of observing time will be sufficient. For high precision timing of millisecond pulsars, the maximum time (for achieving a fixed S/N and sufficient number of added pulses) is to be used. 2011‐03‐29 Page 56 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 Smits et al (2009) [41] demonstrated that the number of pulses located in a single FoV (and hence could be timed simultaneously) is small and presented an algorithm to optimize the observing time. They concluded that the larger number of beams with full sensitivity available with aperture arrays would reduce the needed observing time dramatically compared with a solution that uses sub‐
arraying of dishes. This is caused by the much denser concentration of pulsars in the Galactic plane which would have to be observed at higher frequencies and hence dishes with smaller FoV. It is important to note that for many of the pulsars it is not possible to trade FoV for sensitivity, as a large instantaneous sensitivity is required to obtain a TOA for pulsars in short binary orbits. Only then can a TOA be measured in sufficiently short intervals that allow us to resolve the orbit. As these systems are usually more isotropically distributed they will also not greatly benefit from wider FoVs. Estimating the required collecting area is difficult, as it strongly depends on the flux density of the most exciting pulsars to be discovered. However, we can use our known population to give an estimate. In general, there we will be two types of experiment that require the highest precision, namely the test of theories of gravity by monitoring the motion of a relativistic binary in a fast orbit with finely sampled TOAs, and the monitoring for pulsars in a pulsar timing array to detect gravitational waves. As some of us have shown in Liu et al. (2011), given sufficient calibration, this translates into sensitivity of the telescope. Using the best relativistic laboratory to date as a guideline, the double pulsar, we currently need a 100‐m telescope with about 30% fractional bandwidth to obtain a precise TOA in 30 sec to resolve the 147‐min binary orbit. Placing the 1.6‐mJy (1400 MHz) at a distance of the Galactic Centre (rather than in the true distance of only 1 kpc) we need an A/T of 26,000 m2K‐1 to do a similar experiment for a new source half‐way through the Galaxy. An increase in bandwidth will help reducing this requirement, and the hope that 1‐2 min TOAs may be sufficient to resolve the orbit, we estimate that we need a minimum of about an A/T of 10,0000 m2K‐1(1‐2 GHz) to time a newly discovered pulsar with the full SKA. Performing the GR‐test Phase I headline science as outlined in Kramer & Stappers (2010) requires at least 1,000 m2K‐1. We note that finding and timing a pulsar orbiting SGR A* to study its space‐time will need to be done at a frequency of 10 GHz or higher, which requires significant A/T to compensate for the steep spectrum of pulsars. In general, however, the timing precision can be less than needed for the other experiment, so that we estimate that a collecting area of about 5,000‐10,000 m2K‐1 is needed to extract the science. The other experiment, the detection of gravitational waves (GWs) with Phase I and the study of the GWs with Phase II, requires high‐sensitivity observations of known millisecond pulsars. The goal is to increase the sample of pulsars timed with a precision better than 100 ns to about 100 or more (compared to the ~5 today). Following the results and simulations presented by Smits et al. (2011), this seems possible with a similar sensitivity as needed for the gravity tests described above. The collecting area of the SKA is distributed sparsely. Here we assume that the full SKA can be phased up to form multiple tied array beams. The usefulness of a wide FoV and many beams for pulsar timing has been demonstrated by Smits et al. (2009). Here, we mostly gain for timing pulsars 2011‐03‐29 Page 57 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 in the Galactic plane where the density is large. Increasing the FoV to 200 deg^2 or more, also makes timing the pulsars off the plane vastly more efficient. For timing pulsars in the Galactic plane, it is highly beneficial that the upper frequency range of the Aperture Arrays is about 1 GHz and that more than 50% of the total collecting SKA area can be phased up. Using the strategy of Smits et al., a maximum FoV of 250 deg2 for AA and 20 deg2 for dishes (with PFA), as well as assuming that sufficient beams can be produced to pixelize these FoVs, it would take up to 6 days to obtain a single timing point for 14 000 pulsars to be discovered in the surveys. Obtaining one high‐precision timing point it will take up to 3 days with dishes and only 14 hours for timing with AA. In SKA Phase 1 the smaller field of view will influence how many pulsars will be in each field of view. There will of course also be less pulsars but not in the same proportion. Simulations for SKA phase 1 show for a FoV of 2.1 deg2, we will get a maximum of 15 pulsars in the FoV. For a FoV of 30 deg2, there will be 200 pulsars at most. So in order to be as efficient as possible we would need of the order of 50 beams for the timing, as these will be needed for the AAs, and will greatly improve the efficiency of the timing. 11.6 Forming the Beams: In order to achieve the maximum sensitivity the station/dish beams from each of the dishes/stations in as much of the array as possible need to be added coherently. To form the coherent sum requires that the phase relationship between the signals for each beam from each dish station be known precisely. It is important to note that phase calibration also requires that there is a known phase relationship between the time and frequency references at each of the stations. To obtain phase corrections will require the regular observation of calibration point sources and a decomposition analysis. These phases will then need to be applied to the data from each of the stations along with the geometric corrections to point each of the tied‐array beams in the correct direction. The output product for all beams should be complex channelized data so that it is possible to perform coherent de‐dispersion on all beams. While this is strictly necessary only for the high precision timing objects, it simplifies the pipeline, should be technically achievable for a sufficient number of beams and will also improve timing precision at lower frequencies. Moreover it will have application to other areas of pulsar science such as single pulse and polarization studies. 11.7 Time Resolution and Frequency Resolution. We assume a maximum time resolution of 0.2 us for the final data product. The ability to fully record the complex channelized data at high time resolution over the entire available bandwidths for a reduced number of beams will also be required. 11.8 Data rates: The data rate can be defined in terms of the Nyquist sampling of the baseband data as there will be no averaging of the data before it is transported back to the main computing centre even if there is channelisation. Assuming that the data is sampled at 4 bits, and that we need only transport data of about 50 beams (we obtain a data then we have a rate of 30 Gbytes/s. The exact number of required 2011‐03‐29 Page 58 of 59 WP2‐040.030.010‐TD‐001 Revision : 1 beams can be traded against observing time, as it will simply take longer to go through the list. However, that is only possible up to a point where other observations or sufficient sampling is not possible anymore. We consider 50 beams at the lowest possible limit but recommend more to reduce the impact on other science areas. We note that it is at the beamforming stage where it will be crucial that information about the quality of the station data, either through station calibration, or checks done once the data arrive a central processing facility, to determine that the data from all the station are providing data of sufficient quality. The exact algorithm for determining this quality and how to adjust data when stations drop in or out will need to be developed especially if changes are occurring on relatively short timescales, that is shorter than a typical observation duration. 11.9 Processing the Beams When all of the beams (one per pulsar to be timed) have been formed it is necessary to perform a number of steps which are common to all of them. These include to coherently de‐dispersing the data (thereby also forming all Stokes parameters), interference rejection, polarisation calibration and folding. 11.9.1 (Coherent) De‐dispersion: To correct for the dispersive delay due to the interstellar medium requires that the frequency dependent phase shift of the signal applied by the ISM is unwrapped again in a process known as coherent de‐dispersion (e.g. Lorimer & Kramer 2005). This process requires complex channelized data for each beam. In contrast to search observations, the dispersion measure for pulsars to be timed is known and the result of the discovery process. Improvements to the dispersion measure precision will be obtained in an off‐line analysis process. This process includes the Fourier‐Transform of the complex data into the frequency domain, this process may not be necessary if the data are already delivered as complex channelized data. The data is then multiplied with the inverse ISM filter function plus tapering and the re‐transformation into the time domain. 2011‐03‐29 Page 59 of 59