--- title: "R021: Cospas-Sarsat Meosar System Demonstration And Evaluation Phase I Report C" description: "Official Cospas-Sarsat R-series document R021" sidebar: badge: text: "R" variant: "note" # Extended Cospas-Sarsat metadata documentId: "R021" series: "R" seriesName: "Reports" documentType: "report" isLatest: true issue: 2 revision: 1 documentDate: "December 2015" originalTitle: "Cospas-Sarsat Meosar System" --- > **📋 Document Information** > > **Series:** R-Series (Reports) > **Version:** Issue 2 - Revision 1 > **Date:** December 2015 > **Source:** [Cospas-Sarsat Official Documents](https://www.cospas-sarsat.int/en/documents-pro/system-documents) --- COSPAS-SARSAT MEOSAR SYSTEM DEMONSTRATION AND EVALUATION PHASE I REPORT C/S R.021 Issue 1 COSPAS-SARSAT MEOSAR SYSTEM DEMONSTRATION AND EVALUATION PHASE I REPORT HISTORY Issue Revision Date Comments Approved by the Cospas-Sarsat Council (CSC-55) TABLE OF CONTENTS Page History..................................................................................................................................................... i Table of Contents ................................................................................................................................... ii List of Figures ....................................................................................................................................... iv List of Tables ........................................................................................................................................ iv Document Summary .............................................................................................................................. v 1. BACKGROUND ................................................................................................................1-6 1.1 MEOSAR System Description ...................................................................................1-6 1.2 The Cospas-Sarsat MEOSAR Demonstration and Evaluation Plan ...........................1-6 1.3 The Phase I of the Cospas-Sarsat MEOSAR D&E ....................................................1-7 1.3.1 Objective of the D&E Phase I ........................................................................1-7 1.3.2 Report of the D&E Phase I ............................................................................1-7 2. CONDUCT OF PHASE I AND MEOSAR SYSTEM CONFIGURATION .................2-1 2.1 Tests Conducted During the Phase I ..........................................................................2-1 2.2 Participants in the D&E Phase I .................................................................................2-3 2.3 Configuration of the D&E Phase I .............................................................................2-5 2.3.1 Experimental Space Segment ........................................................................2-5 2.3.2 Experimental Ground Segment ......................................................................2-5 2.3.3 Beacon Simulators and Test Beacons ............................................................2-6 2.4 Test Coordination .......................................................................................................2-7 2.5 Operational Issues Encountered during the Phase I Testing ......................................2-7 2.6 Data Collection ...........................................................................................................2-8 3. RESULTS OF THE TECHNICAL TESTS AND DISCUSSION ..................................3-1 3.1 Test T-1 (Processing Threshold and System Margin) ................................................3-1 3.1.1 Analysis .........................................................................................................3-1 3.1.2 Interpretation ..................................................................................................3-1 3.2 Test T-2 (Impact of Interference) ...............................................................................3-4 3.2.1 Analysis .........................................................................................................3-4 3.2.2 Interpretation ..................................................................................................3-4 3.3 Test T-3 (Valid/Complete Message Acquisition).....................................................3-10 3.3.1 Analysis .......................................................................................................3-10 3.3.2 Interpretation ................................................................................................3-10 3.4 Test T-4 (Independent 2D Location Capability) ......................................................3-14 3.4.1 Analysis .......................................................................................................3-14 3.4.2 Interpretation ................................................................................................3-14 3.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) ..............3-20 3.5.1 Analysis .......................................................................................................3-20 3.5.2 Interpretation ................................................................................................3-21 3.6 Test T-6 (MEOSAR System Capacity) ....................................................................3-29 3.6.1 Analysis .......................................................................................................3-29 3.6.2 Interpretation ................................................................................................3-30 3.7 Test T-7 (Networked MEOLUT Advantage) ...........................................................3-33 3.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) .....................3-33 4. CONCLUSIONS AND RECOMMENDATIONS ...........................................................4-1 4.1 Conclusion ..................................................................................................................4-1 4.1.1 Test T-1 (Processing Threshold and System Margin) ...................................4-1 4.1.2 Test T-2 (Impact of Interference) ..................................................................4-1 4.1.3 Test T-3 (MEOLUT Valid/Complete Message Acquisition) ........................4-1 4.1.4 Test T-4 (Independent Location Capability)..................................................4-2 4.1.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) ....4-3 4.1.6 Test T-6 (MEOSAR System Capacity) .........................................................4-5 4.1.7 Test T-7 (Networked MEOLUT Advantage) ................................................4-5 4.1.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) ..........4-5 4.2 Recommendations for the Conduct of Subsequent D&E Phases ...............................4-5 4.2.1 Test T-1 (Processing Threshold and System Margin) ...................................4-5 4.2.2 Test T-2 (Impact of Interference) ..................................................................4-6 4.2.3 Test T-3 (MEOLUT Valid/Complete Message Acquisition) ........................4-6 4.2.4 Test T-4 (Independent Location Capability)..................................................4-6 4.2.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) ....4-6 4.2.6 Test T-6 (MEOSAR System Capacity) .........................................................4-6 4.2.7 Test T-7 (Networked MEOLUT Advantage) ................................................4-6 4.2.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) ..........4-7 4.3 Recommendations for the Implementation of the MEOSAR System ........................4-7 LIST OF ANNEXES ANNEX A DETAILED LOG OF PHASE I TESTS LIST OF FIGURES Figure 1: The MEOSAR System Concept ..........................................................................................1-6 Figure 2: Phase I Test Planning (as Run) ............................................................................................2-3 Figure 3: MEOLUTs Involved in Phase I Testing with 3,000 km Radius Circles .............................2-6 Figure 4: Beacon Simulators Used in the MEOSAR D&E Phase I ....................................................2-7 Figure 5: Spectrum Graphic Sample of T-1 Run 2 .............................................................................3-5 Figure 6: Test T-4 Activations in Red, Some Other Transmission Overlapping Boxed in Green .....3-6 Figure 7: Interference for Test T-5 as Seen by Galileo .......................................................................3-7 Figure 8: Average Throughput for Each Antenna for Test T-5 Observed by the Ottawa MEOLUT .3-8 Figure 9: Test T-6 Beacon Bursts and Overlapping Interference Seen by DASS Satellites. ..............3-9 Figure 10: Location of the 33 Operational Beacons Deployed for Test T-5 ....................................3-21 LIST OF TABLES Table 1: List of Technical Tests, Test Coordinators and Test Reports ...............................................2-2 Table 2: Participation in MEOSAR D&E Phase I Tests .....................................................................2-4 Table 3: List of Experimental MEOSAR Satellites Used During the MEOSAR D&E Phase I .........2-5 Table 4: MEOLUTs Participating in MEOSAR D&E Phase I Tests..................................................2-6 DOCUMENT SUMMARY This document provides the report of Phase I of the Cospas-Sarsat MEOSAR Demonstration and Evaluation (D&E), tests which were conducted from February 2013 to March 2014. Section 1 provides background on the MEOSAR system and reference material. Section 2 reviews the planning and conduct of the tests, noting the list of participants, MEOSAR space and ground assets configuration used during the tests and information of interest on the coordination of the tests. Section 3 details, for each D&E test the key results and interpretations as provided by each test participant that contributed to this Report (Canada, France, Russia, Turkey and USA). The underlying sub-sections were provided under the responsibility of these administrations and, therefore, were not reviewed nor commonly agreed by the Correspondence Working Group on the Phase I Report. Section 4 provides, for each test, the conclusions and recommendations agreed by the Correspondence Working Group on the Phase I Report, as well as general recommendation regarding the implementation of the MEOSAR system. 1-6 1. BACKGROUND 1.1 MEOSAR System Description Figure 1 provides a graphical summary of the MEOSAR concept. This picture shows the relay of beacon signals, via multiple satellites, to the MEOLUT. Beacon data is processed by the MEOLUT that derive the beacon locations, and passed onto the MCC, which in turn notifies the RCC. Figure 1: The MEOSAR System Concept 1.2 The Cospas-Sarsat MEOSAR Demonstration and Evaluation Plan The Cospas-Sarsat Council has directed that a demonstration and evaluation (D&E) be performed to confirm the expected capabilities and benefits of a satellite system in medium-altitude Earth orbit (MEO) that uses onboard repeater instruments to relay distress alert signals emanating from 406 MHz distress radiobeacons. The CSC further directed that the D&E should establish the technical and operational performance characteristics of the MEOSAR system. The framework for the D&E of the MEOSAR system is provided in document C/S R.018 “Cospas- Sarsat Demonstration and Evaluation Plan for the 406 MHz MEOSAR System”. In particular, documents provide guidelines for: 1-7  conducting the D&E of the MEOSAR system in a standard manner among the participants,  collecting a set of results from individual participants, using compatible formats, that can be consolidated into a final report for review by Cospas-Sarsat participants and other interested parties,  analysing and translating the results into a set of recommendations for a decision by the Cospas-Sarsat Council to enter the Initial Operational Capability Phase. Additional resources regarding the MEOSAR system (e.g, space segment information) are available in document C/S R.012 “Cospas-Sarsat 406 MHz MEOSAR Implementation Plan”. CSC-49 agreed to divide the MEOSAR D&E Phase into three phases:  Phase I, during which the participants perform only technical tests,  Phase II, during which the participants perform technical and operational tests,  Phase III, during which the participants replicate the tests of the Phases I and II, when satellites with L-band downlinks are widely available. 1.3 The Phase I of the Cospas-Sarsat MEOSAR D&E 1.3.1 Objective of the D&E Phase I In MEOSAR D&E Phase I, participants performed only technical tests (see the detailed definition in document C/S R.018) to characterise the technical performance of the MEOSAR system. Due to the limited space segment available, some tests had to be coordinated and the processing be tuned accordingly, in particular for location accuracy tests. 1.3.2 Report of the D&E Phase I The D&E Phase I report was produced by a Correspondence Working Group with the support of the Secretariat, based on:  the reports on the conduct of the tests provided by the test coordinators (see Table 1),  contributions from the test participants, which provided their interpretation of the test results (see section 3),  agreement among the participants on common conclusions and recommendation for the D&E Phase I (see section 4). - END OF SECTION 1 - 2-1 2. CONDUCT OF PHASE I AND MEOSAR SYSTEM CONFIGURATION 2.1 Tests Conducted During the Phase I Table 1 provides the list of technical tests planned for the Phase I, their completeness status, the participants undertaking the role of test coordinator and the reference to the test reports written by the test coordinators. The detailed conduct of each test can be found in the test coordinator’s reports. Planning of the D&E Phase I. The initial planning of the MEOSAR D&E tests, as proposed at Annex L of document C/S R.018, showed an estimated Phase I duration of 20 weeks assuming a beginning in January 2013, leading to completion in May 2013. Due to unexpected delays in the January 2013 commencement, and to the unavailability of some participants during the EWG-1/2013 meeting, this planning was re-evaluated at the end of February 2013 and the end of Phase I was delayed until the end of June 2013. However the test campaign did not progress as anticipated and several additional delays were encountered during the tests performed by the participants. At the JC-27 Meeting in June 2013, the D&E participants agreed upon the tests to be conducted during the summer 2013 period, which included the second run of tests T-1 (Processing Threshold and System Margin) and T-4 (Independent 2D Location Capability). At the TG-1/2013 Meeting in September 2013, the D&E participants agreed upon the tests remaining for the conclusion of the Phase I with a test T-5 conducted in November 2013 and a test T-6 conducted in December 2013 and beginning of March 2014. The test participants also agreed to not conduct the runs of test T-7 (Networked MEOLUT Advantage) because the network configuration was not available and optional test T-8 (Combined MEO/GEO Operation Performance), due to the time constraints. Consequently, the technical tests of the MEOSAR D&E Phase I can be considered as completed (see Annex A that provides the detailed log of the Phase I tests as conducted). As agreed by the Council (see section 5.2.14 of the CSC-51 report), Phase I will be concluded once the Phase I report is reviewed by the Joint Committee at its JC-28 session in June 2014. The production of the Phase I report, for which a draft version was anticipated to be provided at the TG-2/2014 Meeting, was delayed due to the late provision of the Test Coordinators’ reports. Table 1 and Figure 2 provide the schedule of the tests conducted during the Phase I testing as run. 2-2 Test Definition Run Status Test Coordinator Test Report Reference T-1 Processing Threshold and System Margin Completed in February - March 2013 USA Dated 28 February 2014 Completed in July – August 2013 T-2 Impact of Interference Records available only for tests T-1 Run 2, T-4 Run 2, T-5 and T-6 Run 2 Canada No report available T-3 Valid/Complete Message Acquisition France SAR-RE-DEMEO-783-CNES Iss 1 Rev 3, dated 14 May 2013 Replaced by a test run at lower transmission rate conducted in April T-4 Independent 2D Location Capability Completed in April 2013 USA Dated 28 February 2014 Completed in June – August 2013 T-5 Independent 2D Location Capability for Operational Beacons - Completed in November 2013 Turkey T-5 Run1 Beacon Deployment Report - consolidated v2 - 26.02.2014 T-5 Run1 Test Coordinator Report v1 - 6.05.2014 T-6 MEOSAR System Capacity Completed in May 2013 France SAR-RE-DEMEO-788-CNES Iss 1 Rev 1, dated 23 April 2014 Completed in December 2013 and T-7 Networked MEOLUT Advantage Cancelled (network not ready) Canada Not applicable Cancelled (network not ready) T-8 Combined MEO/GEO Operation Performance - Cancelled (optional test) Turkey Not applicable Table 1: List of Technical Tests, Test Coordinators and Test Reports 2-3 Figure 2: Phase I Test Planning (as Run) 2.2 Participants in the D&E Phase I Table 2 provides the participants in each run of test, which provided at least raw data as per Table J.1 of document C/S R.018 or a test report. Some participants did not provide test results and/or test report. Table 2 also provides the test during which spectrum of the 406 MHz band was recorded. For test T-5, the participation in test T-5 is identified either in supplying test beacons or in involving MEOLUTs. 2-4 Test Definition Run T-2: Impact of Interference (by Canada) Australia Brazil Canada France Russia Turkey UK USA Hawaii Maryland T-1 Processing Threshold and System Margin X (1- channel) X X (1-channel) X X X X X X X X X X Under repair Under upgrade X X X T-3 Valid/Complete Message Acquisition X (1-channel) X X (2-channel) X X X X X X Modified script Under Upgrade X Under Upgrade X X X X T-4 Independent 2D Location Capability Under Upgrade Under Upgrade X X (2-channel) X X X X X Under upgrade X X T-5 Independent 2D Location Capability for Operational Beacons Test beacon X X X X X X X MEOLUT X X X X X X T-6 MEOSAR System Capacity X X X X X X (2 channels) X X X (Toulouse run only) X X X X X Table 2: Participation in MEOSAR D&E Phase I Tests 2-5 2.3 Configuration of the D&E Phase I 2.3.1 Experimental Space Segment Table 3 provides the list of experimental MEOSAR satellites available for testing during MEOSAR D&E Phase I. MEOSAR Constellation Satellite (C/S ID) Satellite availability status for Phase I or launch date DASS (GPS-II) Available Available Available Available Available Available Available Available Available Available Available subsequent to launch on 4 October 2012 Available subsequent to launch d on 15 May 2013 Galileo Available for testing from March 2013 Available for testing from March 2013 Glonass Available with limitations (no ephemeris data available) Table 3: List of Experimental MEOSAR Satellites Used During the MEOSAR D&E Phase I 2.3.2 Experimental Ground Segment The ground segment equipment in place for the Phase I of the MEOSAR D&E consisted of experimental MEOLUTs located in Brazil, Canada, France, Russia, Turkey, the UK and the USA. Table 4 provides the MEOLUTs available for testing, their number of antennas, their software configuration and their availability (note that some participants may have experienced unexpected down periods for some channels, thus limiting their participation in particular tests, see the Test Coordinators reports for more detail). 2-6 Country/ Organisation Location Number of Antennas Configuration Available for D&E testing since Brazil Brasilia HGT MEOLUT 600 LP v 1.6 / SP v 1.4 / FP v 1.4 Canada Ottawa HGT MEOLUT 600 LP v1.5 / SP v1.3 / FP v1.3 France Toulouse HGT MEOLUT 600 LP v1.6 / SP v1.4 / FP v1.4 Russia Moscow Not provided Turkey Ankara HGT MEOLUT600 LP v1.6 / SP v1.4 / FP v1.4 UK Kinloss HGT MEOLUT 600 (S-band only) LP v1.7 / SP v1.5 / FP v1.5 USA Hawaii Not provided Maryland Not provided Table 4: MEOLUTs Participating in MEOSAR D&E Phase I Tests Figure 3: MEOLUTs Involved in Phase I Testing with 3,000 km Radius Circles 2.3.3 Beacon Simulators and Test Beacons Three beacon simulators were used during the Phase I testing, located in Hawaii and Maryland, USA and Toulouse, France. After each test, the beacon log files were provided by each administration providing beacon simulators. 2-7 Figure 4: Beacon Simulators Used in the MEOSAR D&E Phase I (MEOSAR Visibility Circles at Five Degree Elevation) 2.4 Test Coordination A smooth progression of the D&E planning and tests has been observed thanks to the active participation of the Test Coordinators and Test Participants. No formal D&E test had to be postponed or re-scheduled due to a coordination issue. However, a dry-run test encountered some difficulty because of the work simultaneously being performed by the Galileo Programme. In addition to this particular case, other tests had been planned by Test Participants at times similar to those of SAR/Galileo commissioning tests, requiring an active coordination between France and EC/ESA in order to avoid the simultaneous transmission of beacon signals. 2.5 Operational Issues Encountered during the Phase I Testing The Cospas-Sarsat operational community was informed of upcoming D&E tests by a SIT605 message prior to each test. So far, no major operational issue was encountered during the MESOAR D&E tests, and at no time did it become necessary to terminate beacon simulator transmissions. In order to prevent any issues related to the unexpected behaviour of the SARP-3 processors aboard Sarsat 11, Sarsat 12 and Sarsat 13, as detailed in Attachment 2 of document JC-26/Inf.14, each beacon test script was verified and cleared in coordination with France. Nonetheless, during the T-3 test run 2-8 using the Hawaii beacon simulator, the memory of two SARP-3 instruments recorded a limited number of erroneous messages sent by the simulator, and some LEOLUTs produced a few erroneous location solutions. These solutions were filtered out by the MCCs as the messages were invalid and clearly related to the on-going tests. Further investigation confirmed that erroneous messages were recorded in the SARP-3 memory and that LEOLUT processing should not have produced any location solution given that each erroneous message was unique in the SARP-3 memory. The number of erroneous messages recorded remains acceptable in comparison to the size of the SARP-3 memory and no specific mitigation action had to be undertaken for the particular test T-3. However, additional investigations were conducted and, taking the most cautious approach, this matter will be carefully monitored during upcoming future tests to prevent any filling of the SARP-3 memory with erroneous messages. As an additional risk mitigation measure, not beacon message transmissions were conducted during test T-6 (System Capacity), for which the number of transmitted bursts is larger than for the other technical tests. 2.6 Data Collection During the tests, the participants collected the following data:  beacon log data to collect the beacon IDs transmitted (if applicable),  MEOLUT raw data as per csv format defined in Table J.1 of document C/S R.018,  MEOLUT location data as per csv format defined in Table J.2 of document C/S R.018,  MEOLUT pass schedule data as per csv format defined in Table J.3 of document C/S R.018. All the data provided by the test participants were saved on the MEOSAR D&E FTP server. - END OF SECTION 2 - 3-1 3. RESULTS OF THE TECHNICAL TESTS AND DISCUSSION The following sections provide, for each test:  references to the test participant’s reports presenting the results of the MEOSAR D&E tests conducted during the Phase I testing,  a summary of the interpretation of the test analyses, as provided by each administration. 3.1 Test T-1 (Processing Threshold and System Margin) 3.1.1 Analysis The following test reports were provided by the participants: Administration Test report reference France “C/S D&E T-1 Test Report Processing Threshold And System Margin” SAR-RE-DEMEO-762-CNES, v1.3 Russia Test report available on FTP server Turkey TRMEO T-1 Report, dated 10 April 2013 USA USA Hawaii MEOLUT Report Revision 1.0, dated 15 April 2013 Technical Test T1 Run 1 - USA Maryland Test Report, dated 21 May 2013 Technical Test T1 Run 02 - USA Maryland Test Report, dated 12 September 2013 3.1.2 Interpretation 3.1.2.1 Canada Data was collected for the first run of test T-1 for the Maryland simulator run only and from one of the original antennas used during the MEOSAR Proof of Concept. Average C/No calculated at the ground station was 33.7 dB.Hz. The lowest signal power level received by the old antenna was a 29 dBm burst. For the second Maryland run, a new four-antenna set was available. The average C/No increased to 35.7 dB.Hz (calculated over all signals received) and all four antennas received signals as lows as 2 dBm. The results showed that the new antennas performed much better than the older ones. However, for the second run, while detection rates did reach the 70% threshold for all antennas individually for power levels above 35 dBm, below this power level, detection rates varied greatly, and these rates were much less than the 70% threshold. Overall detection rate was much lower than 70% for each antenna. However, all antennas did detect signals as low as 22 dBm. This fact was a positive outcome considering the interference present in the entire band, as noted in test T-2 spectrum graphic collections. Interference impacts the automatic gain control of the repeaters and therefore can impair 3-2 the detection of weaker signals. As well, weaker signals fade out often for certain passes due to the elevation angle of the beacon simulator from the repeaters view point, which would be one more reason for the variability and reduction of the detection threshold for signals below 35 dBm. Canada cannot provide further interpretations at this time, but would consider analysing its data in the future to help with comparisons with and insights when comparing with the results of Phase II testing. 3.1.2.2 France Best results were obtained during run 2 with the Maryland transmission. At 37 dBm of transmitted power, up to 71% of single satellite channels have a valid message throughput higher than 70% (target value). It was noted that the results in term of message throughput are significantly dependent on the elevation angle between the beacon and the satellite due to the beacon antenna pattern. An issue was observed with the French MEOLUT regarding confirmed messages detection, whose throughput is too low compared to valid message throughput. Further work is required to investigate and resolve the issue. Also to be investigated, low performances with the Toulouse transmission were observed, for which only 25% of single satellite channels have a valid message throughput higher than 70%. 3.1.2.3 Russia In rare occasions, the throughput for beacons emitting at 37 dBm was higher than 80%, with lower throughput values for 35 dBm and further down to 22 dBm. The possible reasons for this behaviour were investigated and it was found out that the receiver message integration capability was left unchanged after test T-3 and was operating in such a way that the period deviation of the emitting beacon was restrained to 50±0.1 seconds to discard other beacon messages emitting at the same frequency. Unfortunately, the period of the messages in was 48 seconds and that hindered the receiver from getting bursts integrated. It was believed that this might led to the poor results, especially in the lower power range. The results obtained at MEOLUT for run 1 have shown that the throughput performance might have been better. Considering the burst integration issue at Russian MEOLUT, a retest may be required to correctly and comprehensively assess the throughput performance. Unfortunately, the Moscow MEOLUT was unable to participate in run 2 of test T-1 due to an antenna damage. 3.1.2.4 Turkey For the Toulouse transmission of 13-14 March 2013, the 2-channel Ankara MEOLUT had a maximum valid (resp. complete) system message throughput of 60% (resp. 55%), not reaching the 70% threshold aimed by test T-1 in the transmitted 22 dBm - 37 dBm power range. It took the addition of the GEO channel for the 3-channel Ankara MEOLUT (2 MEOSAR + 1 GEOSAR channels) to reach a valid 3-3 (resp. complete) system message throughput of 70% at 31 dBm (resp. 35 dBm) with corresponding average C/No values of 37 dB-Hz (resp. 39 dB-Hz). For the Maryland transmission of 20-21 February 2013, even the GEO-complemented 3-channel Ankara MEOLUT had a maximum valid (resp. complete) message throughput of 57% (resp. 49%), and for the Hawaii transmission of 7-8 March 2013, a maximum valid (resp. complete) message throughput of 46% (resp. 38%), not reaching the 70% threshold aimed by T-1 in the transmitted 22 dBm - 37 dBm power range. The results seemed to indicate that the processing threshold would be improved (i.e., get lower) and consequently the system margin would increase by the addition of more MEOSAR channels as well as the proximity of the transmission source (i.e., the beacon simulator). Clearly, a 2-channel MEOLUT was not adequate to have a system throughput above the targeted 70% threshold. 3.1.2.5 USA-Hawaii Following the analysis of data collected during the test T-1, run 1 the following key observations were noticed:  Erroneous messages were received by different LEOLUTs tracking SARP-3-equipped satellites at various geographical locations during the testing period, and this behaviour was investigated further both by the USA and France,  Single channel throughput statistics for satellites being tracked in the northwest quadrant were lower by around 10%,  Antenna \#2 had significantly lower overall detections than others,  Antenna \#6 got no detections at all,  The two agreed upon methodologies for obtaining the average C/No values appeared to generate very similar numbers. As a conclusion to this run, the data was too inconsistent to obtain a reliable estimate of system margin, either for the single satellite or the multiple antenna scenarios. In particular, the required detection threshold of 70% detection rate was generally not achieved for the single channel case. Following the analysis of data collected during the test T-1, run 2 the following key observations were noticed:  The rerun of technical test T-1 provided one of the first opportunities at the Hawaii MEOLUT to successfully compare the performance of DASS satellites to Galileo satellites, which proved to be a very productive exercise as the detection rates or the Galileo satellites were notably better,  Overall the Hawaii MEOLUT results for run 2 showed a marked improvement in comparison to run 1 (even when removing antennas \#2 and \#6 from the run 1 analysis),  As seen before for the Hawaii MEOLUT, detection statistics for satellites being tracked in the northwest quadrant were lower by around 10% (although not as pronounced with the Galileo data collected). Compared to run1, there was considerable improvement in performance, both in consistency and in general. However, there was still not enough consistency to arrive at any single value for system margin either from throughput of single satellite/antenna pairings or multiple antenna cases. 3-4 3.1.2.6 USA-Maryland Single channel results demonstrate variability caused by the effect the beacon antenna pattern, ground blockage, and differences in the amount of earth noise in the uplink, have on the results. As the elevation angle to the satellite exceeds 50 degrees, the transmitted EIRP of the beacon decreases and the probability of reception, therefore, decreases. Similarly, when the footprint of the satellite receive antenna is over different geographical areas, the earth noise increases and the C/No of the received beacon messages decreases and the probability of reception, therefore, decreases. More examples of these effects were contained in document EWG-1/2008/3/9. The multi antenna results showed that, for four antennas, the message throughput ranges from 97% to 88% for values of beacon output power down to 32 dBm. It is useful to note that the throughput is 96% at the minimum allowed beacon output power of 35 dBm as specified in document C/S T.001 (see the table below). Power (dBm) Number of Transmitted Bursts Detection rate of Valid Message received Detection rate of Valid Complete received 0.98 0.97 0.98 0.97 0.96 0.96 0.91 0.90 0.91 0.90 0.89 0.88 3.2 Test T-2 (Impact of Interference) 3.2.1 Analysis The following test reports were provided by the participants: Administration Test report reference Canada Documents EWG-1/2014/2/2 and TG-1/2013/Inf. 15 3.2.2 Interpretation 3.2.2.1 Canada All spectrum graphics recorded during the MEOSAR D&E tests are available on the FTP D&E server. Spectrum graphics can be viewed either using the GIMP tool provided to all participants for download within the T-2 test folder, or by using any common picture viewer. A very small sample of certain time periods to enforce an observation and help with the interpretation of the results is provided below, but to fully appreciate and explore the spectrum graphics it is strongly recommended to view the sample here in their native image file using GIMP. 3-5 Spectrums recorded during test T-1 Data was collected for test T-1 for the first run from one of the original antennas from the MEOSAR Proof of Concept. For the second run, all four antennas were available. The results showed that the new antennas performed better than the older ones. However, while detection rates did reach the 70% throughput threshold for all antennas individually for power levels above 35 dBm, below that, detection rates varied greatly, dropping below threshold quickly. Looking at the spectrum graphics produced for test T-1 (see Figure 5), immediate and directly overlapping interference was not present as often enough, so as to attribute the low detection rate to interference alone. Looking at the RF levels received, in many instances the ground station LUT processors should have been able to decode the beacon simulator signals, but no burst was detected, and hence no beacon message was decoded. Clearly, there were occasions were the actual signal was received at the ground station and downconverted properly to the LUT signal processors, but for whatever reason, the signal was not decoded. This lead to the observation that the signal processors are burdened with dealing with the interference and the multitude of signals that could be considered as possible beacon bursts. This observation was important to note, as anything from more reference beacons to more interference or increased number of users just above or below the distress band would negatively impact detection rates, with the current processing techniques used and capabilities of, heritage/conventional processors. Figure 5: Spectrum Graphic Sample of T-1 Run 2 (as presented in document TG-1/2013/Inf.15) Spectrums recorded during test T-3 Data was collected for test T-3 for the first run from two of the original antennas used during the MEOSAR Proof of Concept. For the second run only one LUT channel was available and, as such, 3-6 only spectrum graphics for test T-2 was collected. As such, Canada can only comment that detection rate issues would have been impacted by the interference present at the times of the tests. Spectrums recorded during test T-4 Data was collected from all beacon simulators active in test T-4, from all four MEOLUT antennas. Canada did not have time to analyse the complete data sets. From partial analysis of its data, Canada noticed interference and fade outs (possible scintillation events) when comparing to the spectrum graphics of test T-2 that would have impacted some of the test burst transmissions, and which could explain various missed bursts seen by other LUTs as well. Clearly, as seen in Figure 6, there were occasions were the actual signal was received at the ground station and downconverted properly to the LUT signal processors, but for whatever reason, the signal was not decoded, just as in test T-1. This lead to the observation that the signal processors are burdened with dealing with the interference and the multitude of signals that could be considered possible beacon bursts. Figure 6: Test T-4 Activations in Red, Some Other Transmission Overlapping Boxed in Green Spectrums recorded during test T-5 As seen in Figure 7 (this figure was presented and detailed in document EWG-1/2014/2/2), channels used by test beacons operating from 406.037 to 406.040 MHz were in the clear for most of the time for all three test dates, although some interference were observed around 406.045 MHz. Beacons in the 406.020 to 406.030 MHz channels suffered from interferers most of the time, especially around 406.024 MHz. This would mean that detection rates of the 406.028 MHz beacons would present lower detection or throughput rates than those in the 406.037 and 406.040 MHz channels. 3-7 Figure 7: Interference for Test T-5 as Seen by Galileo Note the interferers throughout the band Ottawa’s LUT detection rates from test T-5 are provided in Figure 8 as a point of comparison. Antenna 15 had an intermittently active signal processing card, and thus had lower detection rates at certain times. Note that it was not an antenna or RF path activity issue, just a hardware problem that would not have been seen and easily determined without the help of the spectrum graphics. 3-8 Figure 8: Average Throughput for Each Antenna for Test T-5 Observed by the Ottawa MEOLUT. At this stage, the results do not provide clear guidance for the specifications and parameters regarding the exact coverage areas in which the locations can meet the accuracy requirements within 5 km, 95% of the time, as proposed currently in documents C/S R.012 and C/S R.018. Country Beacon ID Antenna 14 Antenna 15 Antenna 16 Antenna 17 1 In 4 Antennas 4 In 4 Antennas Average Antenna Australia BEFC0 00000 000E3 0.0 0.0 0.2 0.4 0.6 0.0 0.2 Austratlia Average 0.0 0.0 0.2 0.4 0.6 0.0 0.2 279C6 32662 FFBFF 33.2 56.0 38.8 49.1 87.2 4.7 44.3 279C6 360D0 FFBFF 37.0 2.3 24.7 28.9 64.9 0.5 23.3 279C6 7A164 FFBFF 76.6 32.5 51.8 70.1 91.9 12.3 57.7 279C7 4DCFE FFBFF 0.0 0.0 0.0 0.1 0.0 0.0 0.02 279C7 53BAE FFBFF 69.8 62.8 51.8 51.8 92.2 12.6 59.0 279C7 53CA0 FFBFF 45.8 63.2 52.3 57.8 89.2 11.7 54.8 Canada Average Within Canada 56.4 53.6 48.7 57.2 90.1 10.3 54.0 1C7C0 84B5A FFBFF 21.6 30.9 16.7 21.4 65.8 0.0 22.7 1C7C0 84B5C FFBFF 40.9 4.9 26.2 28.1 70.7 0.0 25.0 9C7E4 3316C 0028C 2.6 7.4 1.5 9.9 19.6 0.0 5.3 9C7E4 333E4 0028C 7.9 2.4 0.6 2.9 12.9 0.0 3.5 9C7E4 C37BA 73590 0.2 0.2 0.1 0.1 0.3 0.0 0.1 9C7E4 C37BA 735D0 7.9 9.7 2.6 7.6 24.9 0.0 6.9 9C7FE 28D29 90CA0 8.9 15.8 4.7 27.5 48.8 0.0 14.2 9C7FE 28D41 52900 3.7 0.2 0.0 0.8 4.2 0.0 1.2 France Average 11.7 9.0 6.5 12.3 30.9 0.0 9.9 21FD0 F9502 FFBFF 5.8 3.7 2.1 4.4 13.8 0.0 4.0 21FD0 F9514 FFBFF 27.4 20.5 24.1 12.7 64.8 0.1 21.2 21FD0 F9520 FFBFF 27.4 3.2 10.6 24.7 56.4 0.0 16.5 21FD0 F9532 FFBFF 9.9 3.9 2.8 7.5 22.0 0.0 6.0 21FD0 F953A FFBFF 11.3 7.1 7.5 3.2 27.2 0.0 7.3 Turkey Average 16.4 7.7 9.4 10.5 36.8 0.0 11.0 1D1C0 007D2 FFBFF 18.5 16.0 8.2 8.4 44.7 0.0 12.8 1D1C0 007D4 FFBFF 32.6 25.8 16.4 15.6 66.7 0.0 22.6 1D1C0 007D6 FFBFF 26.5 21.2 13.9 11.3 57.9 0.0 18.2 1D1D6 10002 FFBFF 28.4 2.6 17.7 18.5 49.8 0.1 16.8 1D1D6 28018 FFBFF 42.7 29.8 21.6 23.8 78.2 0.6 29.5 1D1E0 6C6BF 81FE0 51.0 8.3 35.3 37.6 81.5 0.1 33.1 1D1E1 21ABF 81FE0 34.3 5.9 18.9 26.2 63.1 0.1 21.3 1D1E4 F1BBF 81FE0 47.6 3.7 25.6 33.9 76.1 0.1 27.7 UK Average 35.2 14.2 19.7 21.9 64.8 0.1 22.7 2DDC6 7A0B4 FFBFF 64.0 46.8 49.2 35.3 87.7 5.7 48.8 2DDC6 7A0C4 FFBFF 29.7 32.2 26.3 30.4 73.6 0.5 29.7 2DDC7 52E20 FFBFF 69.3 41.7 50.9 57.3 89.8 9.8 54.8 2DDC7 52E2A FFBFF 2.7 10.2 14.1 21.6 44.9 0.0 12.1 ADDE4 11528 00330 14.5 10.9 9.4 7.4 21.1 1.0 10.6 USA Average 36.0 28.4 30.0 30.4 63.4 3.4 31.2 USA Canada France Turkey UK 3-9 Spectrums recorded during test T-6 Canada collected data from the run 2 of test T-6 with the Toulouse beacon simulator in December 2013 (see Figure 9). From its analysis the detection rates did not significantly drop with the increase of the number of transmitted bursts. The 406.070 MHz interferer directly overlapped one to three frequency channels (runs) in which the test T-6 Run 2 tests were executed for the Toulouse beacon. The interferer was present more than half the time of the test beacon simulator sequence. Figure 9: Test T-6 Beacon Bursts and Overlapping Interference Seen by DASS Satellites. Note fading of bursts in purple box. As well, the 406.060 MHz interferer was also present in many portions of the test run. This would mean that a reduction anywhere from 10% to 30% in the number of received bursts would be expected, due the interference seen during the runs of test T-6. Despite the interference, overall the detection rates and location accuracies were slightly better than those seen in test T-5. 3-10 Overall summary of test T-2 From the analysis and interpretation provided above for the technical tests, it was noted that persistent interference was seen by all MEOSAR satellites around 406.070 MHz, as well as intermittent interference around 406.060 MHz, when in view of north and eastern quadrature of the globe. This interference was not typically seen in the upper half of the 406 MHz band when satellites do not see Western Africa and Eurasia. Similar interference and noise behaviour was observed in all spectrum graphics for all tests. Canada underlines the importance of spectrum monitoring and recommends to continue monitoring the 406 MHz band spectrum and analysing the detection rate per channel, or probability of detection per channel, for all technical tests. This parameter had been assumed to be very high (i.e., 85%), but was not met yet by the Ottawa ground station. Current tests showed average detection rate to be no better than 70% per single satellite channel in distances less than 1,000 km away from the MEOLUT, and to be around 40% for beacons up to 5,000 km away. This would mean that a 6-channel MEOLUT would be needed to achieve single burst locations better than 95% of the time for distances up to 5,000 km away from the beacon. 3.3 Test T-3 (Valid/Complete Message Acquisition) 3.3.1 Analysis The following test reports were provided by the participants: Administration Test report reference France “C/S D&E T-3 Test Report MEOLUT Valid/Complete Message Acquisition” SAR-RE-DEMEO-750-CNES v1.1 Russia Test report is available on FTP D&E server Turkey TRMEO T-3 Report v3 – dated 2 June 2013 USA Technical Test T-3 Run1 – USA Hawaii MEOLUT Report, Revision 1.0 dated 3 April 2013 Technical Test T3 Run1 - USA Maryland Test Report, dated 15 April 2013 Technical Test T3 ReRun - USA Maryland Test Report, dated 31 May 2013 3.3.2 Interpretation 3.3.2.1 Canada Data was collected for test T-3 for the first run from two of the original antennas from the MEOSAR Proof of Concept. For the second run only one MEOLUT channel was available and the signal processing at the MEOLUT was intermittent due to a hardware issue. As such, only spectrum graphics for test T-2 were collected. As such Canada can only comment that detection rates could have been impacted by the interference present at the times of the tests, which would have increased the time to obtain a valid message. 3-11 3.3.2.2 France At 37 dBm, for the Toulouse transmission with the modified script (i.e., at a lower beacon transmission rate), the probability to obtain a valid message after one transmitted burst with the French MEOLUT was 90% and reached 100% after seven transmitted bursts, and the transfer time was about 10 s. When the distance of the beacon increased to 6,000 km, the probability to receive a valid message after seven transmitted bursts decreased to 90% or 80%. France noted that the Ankara and French MEOLUTs demonstrated improved results with this run of test T-3 using a modified script, thus showing some limitations on the receiving capacity. This observation remained to be further investigated. 3.3.2.3 Russia No consistent results were obtained due to MEOLUT inability to properly integrate beacon bursts emitted as per T-3 script. However, in order to prepare for run 1 of test T-3, the receiver firmware was upgraded to retain signal integration capability and ensure compliance with test scenario that required receiver to process 50 beacons transmitting at the same frequency with a one second transmission rate. In the view of the above there might have been occasions when beacon message from various beacons were mixed in one integration sequence. This occurrences being unlikely have not been investigated, however a certain “cushion” was foreseen to eliminate the probability of taking them into account while processing the results. Generally, the results have shown the deterioration of the valid/complete message detection performance as the emissions locations moved from Toulouse towards Maryland and further to Hawaii, with the decrease of C/No ratio accordingly. Nonetheless, taking into account the aggressive scenarios of the tests the results proved the capability of the Russian MEOLUT to process the beacon messages relayed through MEOSAR satellites with certain level of quality that may be sufficient to meet expected performance requirements. 3.3.2.4 Turkey For the run using the Toulouse beacon simulator with the modified script (at a lower beacon transmission rate), the valid message average detection probability of the Ankara MEOLUT increased from 88% (resp. 86%) for 1 burst to 99% (resp. 98%) for 7 bursts for a beacon transmission power of 37 dBm (resp. 33 dBm). The Ankara MEOLUT complete message average detection probability increased from 87% (resp. 84%) for 1 burst to 99% (resp. 97%) for 7 bursts for a beacon transmission power of 37 dBm (resp. 33 dBm). Valid (resp. complete) message transfer times of 8 seconds (resp. 10 seconds) were obtained at 37 dBm, and valid (resp. complete) message transfer times of 13 seconds (resp. 18 seconds) were obtained at 33 dBm. For the run using the Maryland beacon simulator with the modified script, the valid message average detection probability of the Ankara MEOLUT increased from 68% (resp. 51%) for 1 burst to 99% (resp. 78%) for 7 bursts for a beacon transmission power of 37 dBm (resp. 33 dBm). The complete message average detection probability of the Ankara MEOLUT increased from 63% (resp. 40%) for 1 burst to 98% (resp. 78%) for 7 bursts for a beacon transmission power of 37 dBm (resp. 33 dBm). Valid (resp. complete) message transfer times of 22 seconds (resp. 26 seconds) were obtained at 3-12 37 dBm, and valid (resp. complete) message transfer times of 77 seconds (resp. 92 seconds) were obtained at 33 dBm. The results seemed to indicate that average detection probabilities improved, as expected, with the number of transmitted bursts as well as with the beacon transmission power. Similarly, message transfer times improved (i.e., got shorter) with the beacon transmission power. On the other hand, for the test run using the Toulouse beacon simulator with the modified script, 1-burst detection probabilities well exceeded 70% for both 37 dBm and 33 dBm, in contrast with the lower system throughputs obtained in test T-1 at the highest beacon transmission power of 37 dBm – a point that needs to be further investigated. 3.3.2.5 USA-Hawaii Following the analysis of data collected during the test T-3, run 1 the following key observations:  Antennas \#2 and \#6 had significantly lower performance (for detection rate, in particular),  Degradation in the performance to the north and northwest of the MEOLUT was noted and was more pronounced at the lower transmit power (33 dBm vs. 37 dBm)  In comparing the performance of the Hawaii MEOLUT between the first and the additional modified run, the detection percentage on 3 of the 4 good antennas showed improvement, but overall the differences were not significant. Due to unexplained inconsistencies in the data, there was no definitive conclusion with regard to the test objectives as a result of the testing. However, it is clear that MEOSAR in general is a very good system for beacon message detection (both valid and complete), with minimal delays in transfer times as well. 3.3.2.6 USA-Maryland Results for 37 dBm are somewhat limited by satellite visibility at the time of the test but are still excellent. 3-13 Single Burst Valid Message Probability (37 dBm) and Number of Satellites for the Maryland MEOLUT with the Maryland Simulator Seven Burst Valid Message Probability (37 dBm) and Number of Satellites for the Maryland MEOLUT with the Maryland Simulator Gaps in performance can also be caused by ground based interference degrading satellite performance. 3-14 3.4 Test T-4 (Independent 2D Location Capability) 3.4.1 Analysis The following test reports were provided by the participants: Administration Test report reference France “C/S D&E T4 Test Report: Independent 2D Location Capability” SAR-RE-DEMEO-765-CNES 0100 v2.0 Turkey TRMEO T-4 Report, dated 29 April 2013 USA Technical Test T-4 Run1 - USA Hawaii MEOLUT Report, Revision 1.0 dated 24 May 2013 Technical Test T4 - USA Maryland Test Report, dated 31 May 2013 3.4.2 Interpretation 3.4.2.1 Canada From partial analysis of its data, Canada noticed interference and fade outs (possible scintillation events), when comparing to the spectrum graphics of test T-2, that would have impacted some of the beacon bursts transmitted and which could explain various missed bursts seen by other participants’ MEOLUTs. Some interference was noted as well on the upper transmission channel from the Toulouse beacon simulator. Canada could not provide further conclusions at this time, but would consider analysing this data in the future to help with comparisons with, and insights for, the results of Phase II testing. 3.4.2.2 France Run 2 performances improved with respect to run 1 results, thanks to the script modification (i.e., lower beacon transmission rate), the pass schedule optimization and the technical issue of antenna \#3 fixed. For the run 2, the computed independent location probability was respectively 82 % for the Toulouse transmission and 64% for the Maryland transmission (both after 4 bursts), which seemed to be encouraging, even if the results cannot be directly compared to the requirement of 95% after 12 transmitted bursts. The probability to obtain an independent location accuracy better than 5 km was respectively 72% for the Toulouse transmission and 56% for the Maryland transmission. The location error was reduced if only locations derived from four-satellite measurements were considered. The probability to obtain single burst independent location accuracy better than 5 km did not meet the requirement of 95%. A complementary analysis showed that this could be due to inaccurate TOA calibration of the French MEOLUT. However, the location accuracy of the standalone MEOLUT could be improved by increasing the number of antennas in order to obtain locations with four or more tracked satellites. 3-15 The time to obtain the first independent location within 5 km from the actual beacon position was of about two minutes (in the range from 100s to 150s). 3.4.2.3 Turkey Whereas the 2-channel Ankara MEOLUT generated a number of locations for the run 2 with the Toulouse beacon simulator, some of them relatively accurate, we consider that a 2-channel MEOLUT was not really suitable to be involved in location-related tests such as test T-4. 3.4.2.4 USA-Hawaii Following the analysis of data collected during the test T-4, run 2 the following key observations were noticed:  An aspect of performance not directly captured in the required results for test T-4 was location probability. The following table indicates the percentage of the time that the MEOLUT computed a single burst location relative to when the mutual visibility of three or more satellites presented the opportunity to compute one, and while much better at higher power, the overall numbers are somewhat low. Transmitted Power (dBm) Single Burst Locations Received Single Burst Locations Expected Percentage 71.5% 29.3% Both Powers 49.9% Single Burst Location Probability  The data in the three main result tables required for test T-4 did not appear to have a definitive pattern which may have been a product of the limited data set, specifically due to the low number of beacon IDs per slot (which also appeared to skew results for the 95th percentile),  The expected pattern of improvement as more bursts are used to compute a location was successfully demonstrated,  Recording the number of mutually visible satellites and channels provided useful additional information in the result tables. As a conclusion, the percentage of locations generated was low, and location accuracy suffered as well due to many of available locations having poor geometries. The space segment of 11 satellites used by the Hawaii MEOLUT during this testing did however provide a limited capability, and matters were compounded by issues with data collection for the Maryland and Toulouse beacon simulator runs. The Hawaii MEOLUT had very limited participation in test T-4, run 2. The Hawaii reference beacon (simulator) did not successfully run due to an unexpected issue with the newest script that had been agreed at JC-27. Due to the lack of mutual visibility between the space segment, the Hawaii MEOLUT and the Maryland and Toulouse simulators, the number of locations generated was very limited and no data analysis was possible. 3-16 3.4.2.5 USA-Maryland The following three charts plot the probability of calculating a location:  The first chart shows how the average probability of location increases as the number of bursts (NB) increases.  The second chart shows more detail about how the probability of single burst location varies across time slots and how it may be affected by the number of satellites that were being tracked. The probability of single burst location varies quite a bit and there is not as strong a correlation with the number of satellites being tracked as expected.  The third chart shows more detail about how the probability of multi-burst locations (for cases of NB = 6 and 7) varies across time slots and how it may be affected by the number of satellites that were being tracked. The probability of multi-burst locations (for cases of NB = 6 and 7) is more consistent and the correlation with the number of satellites being tracked is more evident. 3-17 The following three charts plot the accuracy of the locations calculated:  The first chart shows how the average location errors decrease as the number of bursts (NB) increases. For values of NB greater than 2 the average location error is less than 5 km for the case of nominal beacon power.  The second chart shows more detail about how the accuracy of single burst location varies across time slots and how it may be affected by the DOP of the satellites that were being 3-18 tracked. The accuracy of single burst location varies quite a bit and the expectation that as DOP increases, the location accuracy decreases is not consistent.  The third chart shows more detail about how the accuracy of multi-burst locations (for cases of NB = 6 and 7) varies across time slots and how it may be affected by the number of satellites that were being tracked. The probability of multi-burst locations (for cases of NB = 6 and 7) is more consistent and the correlation with the DOP of satellites being tracked is more evident. 3-19 3-20 3.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) 3.5.1 Analysis The following test reports were provided by the participants: Administration Test report reference Australia Beacon deployment report (see annex of the Beacon Deployment Report, Rev. 1, dated 26 February 2014, consolidated by the test coordinator) Canada Beacon deployment report (see annex of the Beacon Deployment Report, Rev. 1, dated 26 February 2014, consolidated by the test coordinator) France “Operational Beacons Deployment For D&E Test T-5 France Participation” SAR-RE-DEMEO-833-CN v1.0 “D&E T-5 Test Report: Independent 2d Location Capability For Operational Beacons” SAR-RE-DEMEO-811-CN v1.0 Turkey T-5 Run1 Turkey Beacon Deployment Report - 24.02.2014 T-5 Run1 TRMEO Report v1 - 23.02.2014 UK Beacon deployment report (see annex of the Beacon Deployment Report, Rev. 1, dated 26 February 2014, consolidated by the test coordinator) USA Participant Report T5 Run01-Maryland, dated 27 February 2014 US Beacon Deployment Report Figure 10 below provides the locations of the test beacons used for test T-5. More details on the beacon models, beacon features and their 24-hour activation periods are available in the Beacon Deployment Report (Rev. 1, dated 26 February 2014) consolidated by the test coordinator from test participants’ reports. 3-21 Figure 10: Location of the 33 Operational Beacons Deployed for Test T-5 3.5.2 Interpretation 3.5.2.1 Canada The frequency channels of test beacons activated in the range 406.037 to 406.040 MHz, were clear of interference for most of the time for all three days of testing, except for some interference around 406.045 MHz. Beacons in the 406.020 to 406.030 MHz frequency channels had interferers most of the time especially around 406.024 MHz. This would mean that detection rates of the 406.028 MHz beacons would present lower detection or throughput rates than those in the range 406.037 and 406.040 MHz. Detection rates for beacons within 800 km of the Ottawa station was about 54%, and just under 40% for activations within 5,000 km. Overall detection rate for each antenna of the Ottawa ground station for T-5 tests. Comparing the spectrum graphics with the MEOLUT results from Ottawa, which showed average detection rates to be no better than 60% per single channel in distances less than 1,000 km away from the MEOLUT; and to be around 0.25 or 25% for beacons up to 5,000 km away, interference is clearly impacting the detection rates. With this lower rate, it would mean a ten-channel MEOLUT or a network of ten antennas would be needed to achieve single burst locations better than 95% of the time within Avg Ottawa LUT Within 800 km 59.0 51.2 49.1 57.2 90.1 10.2 54.1 3000- 5000 km 34.7 15.6 22.7 23.7 59.0 0.9 24.2 5000- 8000 km 24.5 14.8 14.5 15.5 52.8 0.1 17.3 8000- 12500 km 12.5 10.1 9.5 16.8 41.4 0.0 12.2 Beyond 12500 km 4.3 1.3 0.7 2.3 8.0 0.0 2.2 Average Within 5000km 46.8 33.4 35.9 40.4 74.5 5.6 39.1 Beyond 5000km 15.4 9.8 9.3 12.5 37.4 0.0 11.7 3-22 an equivalent coverage area with radius or distance up to 5,000 km, or about 15% of the earth’s surface. For around 1,000 km (about 1% earth’s surface), a five-channel MEOLUT would give single burst locations better than 95% of the time. At this stage, however, the results did not provide clear guidance for the specifications and parameters regarding the exact coverage areas in which the locations could meet the accuracy requirements of 5 km, 95% of the time, as proposed currently in documents C/S R.012 and C/S R.018. Regarding location accuracy, analysis of the Canadian data showed that for unique single burst locations, where location solutions converged within 2,000 iterations, the error was better than 5 km more than 75% of the time. Solutions that needed more than 2,000 iterations to converge, or solutions using multiple bursts, from multiple antennas, showed a greater variance in errors. In any case, the errors were not less than 5 km, at least or better than, 95% of the time. One reason for this is the interference which impacts the C/No of the beacon bursts seen at the satellite. At this stage, the results did not provide clear enough guidance for specifications and parameters regarding the location accuracy and coverage areas in which the locations can meet the requirements within 5 km, 95% of the time as proposed currently. More beacons need to be activated in ranges between 1,000 to 3,000 km range. All in all though, considering the in-band interference, MEOSAR has good detection capability and the results are promising regarding the location accuracy, but further testing is needed with more antennas “seeing” the satellites so more statistically meaningful results regarding the error and coverage areas requirements can be deduced. 3.5.2.2 France The methodology to be followed for the post-processing of test T-5 should be reviewed for future runs. In fact, the use of a fix window for the locations computation was not fully adapted to the current processing of the French MEOLUT and could affect the interpretation of the results. The French MEOLUT was configured in automatic antenna tracking mode and collected data with only three channels most of the time due to a hardware failure on antenna \#4. As a consequence, location probability and accuracy were strongly impacted by this failure. On top of that, the number of locations was too low for some beacons, thus preventing the generation of reliable statistics. The single channel throughput did not exceed 70% even for beacons near the MEOLUT. Beyond a certain distance between the beacons and the MEOLUT (~ 13,000 km) the detection probability is sensibly degraded. The probability to obtain a location with an error lower than 5 km is higher for multi-burst locations than for single burst locations. The overall accuracy results showed that about 70% of the locations errors are below 5 km. The 95th percentile of the location error is above 22 km for the Toulouse1 France beacon and equals to 15 km for the Maine-United States beacon signals (single and multi-burst locations included). Independent location probability with an error less than 5 km after seven transmitted bursts seems to be non-compliant with the requirement of 95% after 12 transmitted bursts. Better results are expected for future runs of test T-5 with a MEOLUT nominal configuration (i,e., with four channels available) and with the development of the MEOSAR constellations. 3-23 3.5.2.3 Turkey Prior to test T-5, the Ankara MEOLUT was upgraded from two channels to six channels. Regarding the detection of activated beacons, 30 out of the 33 beacons deployed were detected by the Ankara MEOLUT during testT-5, the remaining three beacons either having transmission issues or being located too far away from the MEOLUT. Five of the 30 beacons detected were located more than 10,000 km away from the TRMEO, thus confirming the detection benefit of the MEOSAR system, even with a limited MEOSAR space segment. Regarding the System Throughput (i.e. probability of burst detection with at least one satellite), on average around 60% of the transmitted bursts were detected, increasing to 75% for beacons in the immediate vicinity of the MEOLUT, thus confirming the “low detection rate” issue observed during earlier tests. Concerning the detection of bursts by multiple channels, only 7.6% of the transmitted bursts were detected over three days through at least four satellites, and 24.3% through at least three satellites, which can be attributed to the aforementioned “low detection rate” issue. Regarding the location probability (i.e., the ratio of the number of n-burst locations to the expected number of n-burst locations during at least four-satellite covisibility periods), no correlation was observed between location probability and the distance between the beacons and the MEOLUT. Location accuracy was, as expected, better within the geographic region of the MEOLUT (a circle centered at the MEOLUT with a radius of 3,500 km), with a 50th percentile of 2 km and a 75th percentile of 5 km. However, at its 95th percentile, the location accuracy went up to the 10-30 km range and sometimes beyond that range. In addition, the following observations were made:  No significant improvement was noticed due to the integraton of up to 7 bursts. In general, single-burst locations were almost as accurate as multi-burst locations.  The number of satellites used in the calculation of a location seemed to be the most significant factor determining location accuracy. Consequently, Turkey anticipated the definition of “nominal locations” as those locations calculated with four or more satellites, and “marginal locations” as those locations calculated with three or fewer satellites, possibly with the use, as well, of the DOP value in those definitions, pointing out the potential necessity for MEOLUT networking in the real world. If the current “low detection rate” issue was not significantly improved by the future L-band satellites, four-channel MEOLUTs might have difficulties in systematically generating locations derived from four satellites. Turkey recommended that the following parameters be noted and taken into account in the ongoing work on MEOLUT Specifications and Design Guidelines:  Burst detection rates (MEOLUT System Throughput),  Impact on location accuracy of the number of satellites used to calculate a location,  Concept of “nominal” and “marginal” locations. 3-24 3.5.2.4 USA-Hawaii Following the analysis of data collected during the test T-5, the following key observations were noticed  Two beacons supplied by France provided the closest activations to the Hawaii MEOLUT during test T-5 test, but both were still quite far away (about 4,440 km). There was significantly less data received from Papeete1 (75 single burst locations) compared to Papeete2 (538 single burst locations). This large gap from both systems implied either a problem with the Papeete1 beacon, or a very significant difference in the environment in which it was deployed. For those locations generated, the accuracy was reasonably good, in particular with the limited constellation of 12 DASS satellites applied, coming in at 70.7% of single burst location produced within 5 km, and 100% within 5 km at seven received bursts.  While most other statistics were significantly impacted by distance, the range of the MEOSAR system was soundly demonstrated by a number of cases.  A fair amount of correlation between DOP and location accuracy can be seen, and although there are areas that do not coincide, DOP values appear to provide useful information and a good basis for a potential quality factor for MEOSAR data. As a conclusion, it is emphasized that this test was performed with the limited space segment of 12 DASS S-Band satellites. In addition, all of the beacon activations were a significant distance from the Hawaii MEOLUT. Overall the location accuracy could be better, but under these less than ideal circumstances, many good locations were still generated. 3.5.2.5 USA-Maryland The performance measured during this test was affected by several factors: 1. the distance between the MEOLUT and the beacon, 2. the amount of mutual visibility of the satellites to the beacon and the MEOLUT, and 3. the resulting satellite geometries, or DOP, used to create the single burst solutions. As mentioned previously, the Maryland MEOLUT used its automatically generated pass schedule each day, which included the 12 DASS S-Band satellites only being tracked by four antennas. Therefore, there was no attempt to optimize the pass schedule for any particular beacon location. However, the number of available satellites was limited to half the number that will be available in the operational system, which yielded values of DOP that are worse than what would be achievable with a full satellite constellation, and, therefore, limited the performance of the system. In order to illustrate these effects, graphs are presented that show the single burst location accuracy along with the DOP of the satellites used to generate the location as a function of time. On the same graph the number of satellites with mutual visibility to both the beacon and the MEOLUT is shown. These graphs were produced for four beacons on Day 3 of testing that were located within 3,000 miles of the MEOLUT. It can be seen that location accuracy improves as the DOP value is reduced. It can also be seen that there are significant portions of the 24-hour period that contained fewer than three satellites with mutual visibility between the beacon and the MEOLUT. Since it takes three or more 3-25 satellites to produce a single burst location, the amount of available data was limited by satellite visibility. Of course, this situation will greatly improve as more satellites are added to the MEOSAR constellation, but the current situation must be taken into account when interpreting the current results. Admin Beacon-id (15 Hex) Beacon Type Freq (MHz) GPS Location Alt (m) Approx distance (km) from MEOLUT Received raw packets Location results Canada 279C753BAEFFBFF EPIRB 406.037 Yes Leland, Frontenac, Ont., Canada   USA 2DDC752E20FFBFF EPIRB 406.037 Yes Maine, United States   USA ADDE41152800330 EPIRB 406.028 Yes San Juan, Puerto Rico 6.1   USA 2DDC67A0B4FFBFF PLB 406.028 Yes Honduras   3-26 3-27 Method 1 Results The goal for this method was to see how well the MEOLUT could locate the beacon after 1,2,3,…,7 transmitted bursts. Every merged location reported as “7 burst” location was produced by combining as many single burst locations that occurred within a 7 transmitted burst window. The following graph includes all beacons for which the Maryland MEOLUT received data. It, therefore, includes data from beacons that are as far as 8,600 km away from the MEOLUT. Nevertheless, there is performance improvement as more opportunity, i.e., larger window size, is made available. 3-28 The counts are given below: 1 Burst 2 Bursts 3 Bursts 4 Bursts 5 Bursts 6 Bursts 7 Bursts Error 1 km Error 5 km Error 10 km Total Method 2 Results The goal for this method was to determine how the merged locations improve as more single burst locations are used to produce them. In this method, merged locations were reported using the actual number of single burst locations used to produce the merged locations, not the number of transmitted bursts. The following graph includes all beacons for which the Maryland MEOLUT received data. It, therefore, includes data from beacons that are as far as 8,600 km away from the MEOLUT. Nevertheless, there is significant improvement as more data is used to produce the merged location. The slight dip from 6 to 7 bursts is attributed to the relatively small amount of data available for the 7 burst results. 3-29 The following lists the number of data points for each burst count. 1 Burst 2 Bursts 3 Bursts 4 Bursts 5 Bursts 6 Bursts 7 Bursts Error 1 km Error 5 km Error 10 km Total 3.6 Test T-6 (MEOSAR System Capacity) 3.6.1 Analysis The following test reports were provided by the participants: Administration Test report reference France “C/S D&E T6 Test Report MEOSAR System Capacity” SAR-RE-DEMEO-788-CNES v1.0 Russia Test report is available on FTP server Turkey TRMEO T-6 Report, dated 30 September 2013 USA Technical Test T-6 Run2 – USA Hawaii MEOLUT Report, Revision 1.0, dated 11 April 2014 Technical Test T-6 Run1 - USA Hawaii MEOLUT Report, Revision 1.0, dated 18 July 2013 Technical Test T6 Run 01 - USA Maryland Test Report, dated 6 September 2013 Technical Test T6 Run 02 - USA Maryland Test Report, dated 21 August 2014 3-30 3.6.2 Interpretation 3.6.2.1 Canada The 406.070 MHz interferer directly overlapped one to three frequency channels in which the run 2 of test T-6 was executed for the Toulouse beacon. The interferer was present more than half the time of the test beacon simulator sequence. As well, the 406.060 MHz interferer was also present in many portions of the test run. This would mean that a reduction anywhere from 10% to 30% in the number of received bursts would be expected due the interference seen during the T-6 tests. Despite the interference, overall the detection rates were better, and location accuracies were slightly better, than those seen in test T-5. T-6 detection rate or throughput for Toulouse beacon simulator. Note that the throughput here was about twice the rate than the average throughput for the T-5 tests from equivalent distances. While the overall system detection capability results were very good, the location accuracy was not as good as initially hypothesised would be achieved with a four-antenna system. One reason for this was the interference which impacted the C/No level of bursts signals as received by the satellite, which negatively affected frequency and time of arrival estimation accuracy, and thus, the location accuracy. Based on the current detection rates seen, by increasing the number of antennas, the time to locate and the location accuracy would improve. From the analysis, the results did not provide clear guidance on the system capacity threshold or limit. Canada would recommend that, in Phase II testing, the number of transmitted bursts (NB) be increased and two more NB levels of 150 and 200 be added, if the beacon simulators could handle this level of transmissions for the testing duration. As well, analysis might need to be done in smaller time period ranges or “chunks”, so one could better compare results when specific interferers were not present, or when beacon view elevation angles were approximately in the same range. All in all, considering the in-band interference, the MEOSAR system still has very good capacity and the results from Phase I were promising. 3.6.2.2 France The system capacity in test T-6 was evaluated in the range of 25 to 100 simultaneous active beacons and assessed for two system parameters:  throughput performance (detection probability and time to first Valid/Complete Messages),  location performance. For both parameters, no drop-off value was observed and the system capacity could not be assessed by any participants. Antenna Antenna Antenna Antenna 1 In 4 Antenna Average 4 In 4 Antennas Antenna Antenna Antenna Antenna 1 In 4 Antenna Average 4 In 4 Antennas 46.1 43.0 72.2 46.9 91.3 52.0 10.0 41.4 40.5 69.5 45.0 90.2 49.1 7.0 42.9 34.4 60.9 44.7 83.1 45.7 6.5 39.6 32.4 58.4 43.1 81.9 43.4 4.9 35.7 34.9 56.9 53.9 86.2 45.4 6.5 32.9 32.8 54.7 51.8 85.1 43.1 4.9 29.1 31.6 52.6 58.4 81.8 42.9 6.1 26.7 29.8 50.6 56.4 80.8 40.9 4.7 NB Valid Messages Detection Probability(%) Complete Messages Detection Probability(%) 3-31 The computation of the multi-burst locations was specific to each MEOLUT due to the diversity in manufacturer, software and parameter settings. Participants were then invited to provide as much information as possible about MEOLUTs features and parameters setting in order to consolidate the analysis of multi-burst location performances. In order to quantify the system capacity, France suggested conducting future runs of test T-6 with a larger range of simultaneous active beacons (from 50 to 200 beacons for example). 3.6.2.3 Russia No observations were available from Run 1 of test T-6 and only data pertaining to a first portion of run 2 of test T-6 was analysed. Note also that only the system capacity using the MEOLUT throughput performance was assessed as the Russian MEOLUT featured only one antenna. Results from two slots were slightly inferior to others. MEOSAR detection probability requirement is defined in document C/S R.012, Annex E, as the probability of detecting the transmission of a 406 MHz beacon and recovering at the MEOLUT a valid beacon message, within ten minutes from the first beacon message transmission shall be a minimum of 99%. After 350 s, which is a little less than six minutes, the probabilities of producing first complete and valid message when 100 beacons were transmitting were slightly lower than 80% and slightly above 90% if two specific slots were discarded from consideration. Assuming that the requirement did not take into account the number of channels of the MEOLUT, a multichannel MEOLUT would doubtlessly demonstrate better probabilities, drawing nearer to 100%. T-6 run 1 results had shown almost the same probabilities with Toulouse transmission approaching 100% and Maryland transmission being around 80%. It was, therefore, assumed that the system capacity based on the MEOLUT throughput performance was 100 beacons or higher. 3.6.2.4 Turkey Valid/complete message detection probabilities and times to first valid/complete messages would normally be expected to worsen as the number of simultaneously active beacons (NB = 25, 50, 75, 100) increased. This was not always observed in the results as the second-best performance for the Toulouse transmission, run 1 was obtained for NB=100 and the best performance for the Maryland transmission, run 1 was obtained for NB=50. Valid/complete message detection probabilities around 80% to 90% for the Toulouse transmission were achieved regardless of the number of simultaneously active beacons (NB = 25, 50, 75, 100) whereas the performance was down to around 40% for the Maryland Tx - a direct impact of the large distance between the MEOLUT and the Maryland beacon simulator. No significant drop in valid/complete message detection probabilities was observed when NB increased from 25 to 100. After 100 seconds, almost a 100% message detection rate was achieved to obtain a valid/complete message in the case of the Toulouse transmission regardless of the number of simultaneously active 3-32 beacons (NB = 25, 50, 75, 100). This performance was around 70% in the case of the Maryland transmission after 350 seconds - again, a direct impact of the large distance between the MEOLUT and the Maryland beacon simulator. As it could be expected, a reliable determination of system capacity in terms of location capability (i.e., the NB value which met the 95% probability of a multi-burst location error lower than 5 km) was not possible with the results obtained by a two-channel MEOLUT (in other words, a 2-channel MEOLUT was not really suitable to be involved in location-related tests ). Note that the Ankara MEOLUT did not participate in run 2 of test T-6. 3.6.2.5 USA-Hawaii Following the analysis of data collected during the test T-6, run 1 the following key observations were noticed:  Overall, the detection percentages were low, and the expected behaviour was not demonstrated,  For the Maryland transmission, the results remained low even when taking actual mutual visibility into account, but for Toulouse the detection rates within mutually visible time periods were high albeit this was a limited time frame. As a conclusion, the results did not provide the desired outcome. Specifically, a degradation in performance as the number of beacons increases would be expected, and was not observed. While this is could be interpreted as a positive result indicating a high system capacity, the distance between the Hawaii MEOLUT and both simulators, did not produce enough data to drive the test. Data from test T-6, run 2 with the Toulouse beacon simulator was collected and minimally analysed, but no results were uploaded to the MEOSAR D&E FTP server. The reference beacon collocated with the MEOLUT in Hawaii could not provide transmissions at the short intervals required for this test due to its inherent design. As the purpose of test T-6 was to determine the system capacity of the MEOSAR system, the more test data that was processed through the MEOLUT the more likely it would be that the capacity where performance falls off can be determined. The distances between the Hawaii MEOLUT and both the Maryland and Toulouse beacon simulators severely limited the usefulness of T-6 results, and with no co-located simulator, no consistent or identifiable pattern from the results could be achieved. 3.6.2.6 USA-Maryland The probability of obtaining a Valid or Complete message from a single beacon burst ranged from 95% to 64% for NB equal 25 to 100, respectively. The range of probabilities change from 96% to 75% if we compensated for the collisions of beacon bursts resulting from the design of the test script. As expected, the improvement was greater as NB increased. The probability of obtaining a Valid or Complete message rapidly increased as more beacon bursts were used until it reached nearly 100% within 350 seconds, or seven transmitted bursts, for all values of NB. 3-33 The probability of obtaining a location was affected by the number of satellites being tracked by the MEOLUT. Of course, this was a function of the number of antennas tracking satellites with mutual visibility to the beacon. The probability of obtaining a location from a single beacon burst when four satellites were in clear view of the beacon ranged from 62% to 39% for NB equal 25 to 100, respectively. The average probability for time slots when fewer satellites were in clear view was significantly smaller, demonstrating the significant effect that the number of satellites within view has on location probability. The probability of obtaining a multi-burst location when four satellites were in clear view of the beacon ranged from 1% to 93% for NB equal 25 to 100, respectively. Once again, there was degradation, that was more pronounced as NB increases, when fewer satellites were used. Location accuracy was clearly affected by the DOP used to calculate the location. It was also clear that the typical DOP values used to generate this data was not the same as the typical DOP for a full constellation of 24 GPS satellites. However, the data showed that, when we approached that value, location performance improved significantly. Therefore, an overall average for location accuracy did not provide enough insight into the underlying conditions affecting location accuracy to be useful by itself. In an attempt to normalize DOP so that a comparison can be made as NB increases, the location accuracy for the time slot that had the lowest average DOP for each NB was considered. The probability that the single burst location error was less than 5 km range from 67% to 56% as NB increased. The probability that the multi-burst location error was less than 5 km ranged from 92% to 65% as NB increases. To determine system capacity, a required level of performance needed to be determined for each parameter. However, at this time the partial satellite constellation available added some uncertainty as to how to use the data measured during this test. The good news was that this test would be run again during Phase 2 and those results would add clarity to the determination of system capacity. 3.7 Test T-7 (Networked MEOLUT Advantage) The D&E participants decided to not conduct test T-7 because the network configuration was not available. 3.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) The D&E participants decided to not conduct test T-8 (optional test) due to time constraints. - END OF SECTION 3 - 4-1 4. CONCLUSIONS AND RECOMMENDATIONS This section provides the conclusions commonly agreed by participants in the MEOSAR D&E tests and their recommendations for future conduct of the tests in the MEOSAR D&E Phase II. 4.1 Conclusion 4.1.1 Test T-1 (Processing Threshold and System Margin) The test participants agreed on the following conclusions regarding test T-1.  System margin for single-burst throughput using single-channel results The detection percentage produced from the single-channel testing varied enough and did not consistently surpass the 70% threshold defined in document C/S R.018, and therefore it was not possible to arrive at a system margin for single-burst throughput using single-channel results.  System margin for single-burst throughput using multi-antenna results Results were improved using multi antennas and a margin above the 70% threshold was achieved when at least four antennas were used. However, it was the view of the participants that the 70% threshold value was not the right number to use for the multi-antenna results. Reference Participants’ reports for description of underlying causes of the variability of the results. 4.1.2 Test T-2 (Impact of Interference) The test participants agreed on the following conclusions regarding test T-2. From Canada’s analysis, in its T-2 report, it was noted that persistent interference is seen by all MEOAR satellites around 406.070 MHz, as well as intermittent interference around 406.060 MHz, when in view of north and eastern quadrature of the globe. This interference is not typically seen in the upper half of the 406 MHz band when satellites do not see Western Africa and Eurasia. Similar interference and noise behaviour was observed in all spectrum graphics for all tests. Locations of these interferes are provided in the T-2 report. These interferers directly impact the transmissions of the beacon simulators used for all such tests, and indirectly on all tests due to the effect of the interference energy on the AGCs of the satellite repeaters; and other interferers directly impacted T-5. This is one reason for the difficulty in determining and reaching capacity and system thresholds, and for the negative impact on the detection rates seen in tests T-1, T-3 and T-6 of Phase I. 4.1.3 Test T-3 (MEOLUT Valid/Complete Message Acquisition) Two runs of test T-3 were conducted. While not all MEOLUTs participated in a second run referred as a modified test, whose script was characterized by a lower transmission rate, compared to run 1. 4-2 Most of the participating MEOLUT got improved results with the second run using the modified test script. For nominal power of 37 dBm, the results of the test T-3 has shown that the probability of detection of a valid message is about 90% after 1 burst and higher than 99% after 7 bursts, which is compatible with expectation for minimum performance at full operational capability (FOC) contained in Annex E of document C/S R.012 (99% after 10 minutes) for many MEOLUTs. The expected performances were not reached for some MEOLUTs, and while all causes could not be explained, many might be explained by:  limited co-visibility conditions (in term of number of channels) with the beacon simulators,  the test script restricted beacon transmission to 7 consecutive bursts per beacon ID (350 sec vs. 600 sec, which is equivalent to the 10 minutes allowed for 99 % probability of detection as per the MIP),  limited functionality and/or limited participation of some MEOLUTs to some test runs,  interference The main conclusions drawn from the test were the following:  the results are compatible with the expectation for minimum performance at full operational capability (FOC) contained in Annex E of document C/S R.012 (MIP).  the results have shown the deterioration of the valid/complete message production performance as the distance between MEOLUT and the beacon simulator increases.  the average detection probabilities improved with an increase of the number of transmitted bursts as well as with an increase of the beacon transmission power.  the transfer time measured for valid messages was around 5 to 10 seconds. Message transfer times improved (i.e, got shorter) with the beacon transmission power.  the difference between the first run and the modified script may be explained by some limitations in the processing capacity of the participating MEOLUTs. It was decided to use the modified test script for the subsequent runs of tests T-3 and T-4, which was agreed by Participants and included in document C/S R.018 Issue 2 –Revision 1 at CSC-51. It is expected that the results will improve as the MEOSAR L-band space segment is expanded in the future. 4.1.4 Test T-4 (Independent Location Capability) The initial run of test T-4 highlighted issues that necessitated a revision of the test script and a second run of the test. Run 2 of test T-4 was conducted with a modified transmission script as agreed during JC-27 in order to reduce the beacon transmission rate (one beacon every two seconds with only one frequency, instead of one beacon every 0.5 second alternating between two frequencies). The results summarized here, therefore, result from the analysis of run 2 of test T-4 only. Independent Location Probability The probability that a MEOLUT provides an independent 2D location with a location error less than X km (X = 1, 5 or 10 km) did not reach desired values. Performance for X = 5 km varied but was always less than 95% but improved as the number of transmitted bursts used was increased. 4-3 Independent Location Accuracy The 50th percentile, the 75th percentile, and the 95th percentile of the location error of 2D locations did not reach desired values. Performance for 95th percentile varied but was always greater than 5 km but improved as the number of transmitted bursts used was increased Time to First Independent Location The time elapsed between the first burst transmitted and the first 2D independent location with an error less than X km (X = 1, 5 or 10 km) was not more than 2 to 3 minutes. Conclusions While the results were not as good as expected, the tests results showed that:  The probability of calculating a location was affected by: o the single channel throughput, o the number of satellites being tracked with mutual visibility to the beacon, o the amount of time, or number of beacon bursts transmitted.  The accuracy of calculated locations were impacted by many reasons including: o the number of satellites used to derive the location (locations derived with additional satellites were statistically more accurate), o the geometry of the satellites tracked by MEOLUT antennas (characterized by DOP for example), o the signal to noise ratio (C/No) of each channel, o the accuracy and the calibration of TOA/FOA measurements produced by the MEOLUTs. It is expected that the results will improve in the future as:  the probability of location and the location accuracy of the standalone MEOLUT will increase as the number of available antennas-satellite pairing increases, thus allowing the MEOLUT to obtain locations with more tracked satellites.  the availability of more satellites on orbit will allow for the typical DOP to be improved which will increase location accuracy. 4.1.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) Detection benefit of the MEOSAR system The tests carried out over 3 days with 33 operational beacons deployed worldwide soundly demonstrated the vast geographic range of individual MEOLUTs and confirmed the detection benefit of the MEOSAR system, even with a limited MEOSAR space segment that consisted, at the time of the tests, of 12 DASS, 2 Galileo and 1 Glonass satellites. Some participants only tracked the DASS satellites whereas other participants tracked all available satellites. 4-4 Detection rates The detection probability was gradually degraded as the distance between the beacons and the MEOLUTs increased. These results, which were corroborated by the detection rates observed during earlier tests, led the participants to the conclusion that an increase in the number of available satellites the number of antennas per MEOLUT and an improvement of the single channel detection rate hopefully brought by the advent of operational L-band satellites would be needed to improve performance on system detection and meet the expectation on location probability and accuracy. Independent Location Probability The location probability did not meet the expectation for minimum performance of 98% at full operational capability (FOC) contained in Annex E of document C/S R.012. However, some accurate locations were generated for beacons at extreme distances (e.g., greater than 7,000 km) from the MEOLUT, which is indicative of the range of MEOSAR. Independent Location Accuracy The location accuracy did not meet the expectation for minimum performance of 5 km accuracy, 95% of the time at full operational capability (FOC) contained in Annex E of document C/S R.012. However, the location accuracy was, as expected, better within the geographic region of the MEOLUTs (a circle centered at the MEOLUT with a radius of some 3,000 km), location error was frequenctly below 5 km wihin that geographic region. Composite locations calculated by the integration of up to 7 bursts offered a higher probability to obtain a location accuracy better than 5 km. T-5 processing methodology Run 1 of the T-5 tests has been an opportunity to identify certain missing aspects of the original T-5 processing methodology, and to amend the “windowing methodology” and include MEOLUT System Throughput and n-Burst Independent Location Probability into the list of data to be analyzed and reported by Participants. Conclusion The test results showed that:  the MEOSAR system’s capability to detect beacons is very good, sometimes beyond expectations,  the location accuracy was not as good during this first run of test T-5 as initially hypothesized,  the results did not provide clear guidance for the specifications and parameters regarding the exact coverage areas in which the calculated locations could meet the accuracy minimum performance expectation at full operational capability (FOC) contained in Annex E of document C/S R.012 (MIP) (5 km, 95% of the time),  one reason for the limitation in the location accuracy performance was the negative impact of interference on the channel detection rates,  increasing the number of MEOLUT antenna-satellites pairings would improve the location probability and accuracy, as well as the time to locate,  all in all, MEOSAR has very good detection capability and the location results were promising. 4-5 4.1.6 Test T-6 (MEOSAR System Capacity) The system capacity in test T-6 (standalone MEOLUT configuration) was evaluated in the range of 25 to 100 simultaneous active beacons and assessed for two system parameters:  throughput performance (detection probability and time to first Valid/Complete Messages),  location performance. The test did not lead to clear conclusion regarding throughput and location performances and it was agreed to increase the number of transmitted beacon to try reaching a clear decrease of performance. The expected outcome in term of location probability and accuracy were not always observed by the participants even for NB = [25,50,75,100] simultaneous beacons. Participants at EGW-1/2014 agreed to modify the definition of test T-6 for future runs, which was subsequently approved in document C/S R.018 Issue 2 – Revision 2 at CSC-53. The modifications to the test T-6 definition were:  an increase of the number of NB transmitted beacons from NB = [25,50,75,100] to NB = [25,50,75,100,150,200],  modification of the analysis methodology to introduce the computation of the probability to produce single burst locations. The computation of the composite locations is specific to each MEOLUT (manufacturer, software and parameters setting). Therefore, participants are invited to provide as much information as possible about MEOLUTs features and parameter settings to assist the analyses. 4.1.7 Test T-7 (Networked MEOLUT Advantage) No conclusions were drawn on test T-7 as this test was not conducted during the MEOSAR D&E Phase I. 4.1.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) No conclusions were drawn on test T-8 as this optional test was not conducted during the MEOSAR D&E Phase I. 4.2 Recommendations for the Conduct of Subsequent D&E Phases 4.2.1 Test T-1 (Processing Threshold and System Margin) It is recommended in the analysis of future runs of test T-1 to:  attempt to correlate lower performance cases with occurrence of channel interferers,  attempt to determine a more suitable threshold value for the assessment of the system margin for single-burst throughput using multi-antenna results. 4-6 4.2.2 Test T-2 (Impact of Interference) It is recommended to continue to monitor the 406 MHz spectrum and prepare spectrum plots so that the single-channel throughput can be analysed for any technical tests, to allow the evaluation of the negative impact interference has on the MEOSAR, and by extension the entire Cospas-Sarsat System. 4.2.3 Test T-3 (MEOLUT Valid/Complete Message Acquisition) It is recommended that the test be modified to be able to compare the test results with the expectation for minimum performance at full operational capability (FOC) contained in Annex E of document C/S R.012 (i.e., 99% probability of valid beacon message detection within ten minutes). Specifically, beacon transmission could be extended to 13 transmitted bursts (i.e., ten minute beacon transmission) or the test analysis could aggregate results from multiple beacon IDs to generate the desired results without changing the test script. The final methodology shall be proposed by the Test Coordinator and agreed with other participants. 4.2.4 Test T-4 (Independent Location Capability) It is recommended that further analyses be conducted to evaluate the relationship between location accuracy and various parameters (e.g., DOP, number of satellites used in location determination, C/No measurements, etc.). In order to achieve expected results when T-4 is run as part of Phase II, more satellites than were available during this Phase I test runs are needed. It is desirable to have as many L-band satellites as possible. 4.2.5 Test T-5 (Independent 2D Location Capability for Operational Beacons) It is recommended to continue monitoring the 406 MHz spectrum and it is suggested that participants optionally analyze the detection rates per channel and per satellite type (L-band vs. S-band). 4.2.6 Test T-6 (MEOSAR System Capacity) Modifications to the test T-6 definition regarding the number of simultaneous active beacons and the analysis methodology were agreed at EWG-1/2014 and it is recommended that test participants conduct future runs of test T-6 accordingly. Participants are invited:  to provide as much information as possible about MEOLUTs features and parameter settings to assist the analyses,  to provide details about the methodology applied to compute statistics regarding detection and location performances and in particular about the mutual visibility conditions of the test periods selected for the test. 4.2.7 Test T-7 (Networked MEOLUT Advantage) No recommendations were provided on test T-7 as this test was not conducted during the MEOSAR D&E Phase I. 4-7 4.2.8 Test T-8 (Combined MEO/GEO Operation Performance (Optional)) No recommendations were provided on test T-8 as this test was not conducted during the MEOSAR D&E Phase I. 4.3 Recommendations for the Implementation of the MEOSAR System It has been noted over many MEOSAR-related meetings and tests to date that TOA and FOA measurement accuracies were amongst crucial factors affecting location accuracy. It is therefore recommended that Participants revisit and fine tune their MEOLUT TOA/FOA measurements techniques in order to improve location accuracy It is recommended that further analyses be conducted to evaluate the relationship between location accuracy and various parameters (e.g., DOP, number of satellites used in location determination, C/No measurements, etc.) to provide useful information and a basis for a potential Quality Factor for MEOSAR data. - END OF SECTION 4 - A-1 ANNEX A DETAILED LOG OF PHASE I TESTS Week Nb Date Start Test Test Run Time 1st Tx (yyyy-mm-dd UTC) Time last Tx (yyyy-mm-dd UTC) Beacon location Comments 2014-03-05 T-6 2014-03-05 14:45:00 2014-03-07 23:58:00 Maryland 2014-02-05 T-6 2014-02-05 16:05:00 2014-02-07 19:13:00 Maryland CANCELLED due to weather conditions 2014-01-03 T-6 2014-01-03 18:00:00 2014-01-03 18:41:00 Maryland Dry run (one sequence) 2013-12-16 T-6 2013-12-16 17:53:00 2013-12-18 22:45:00 Maryland Retest - DISREGARD due to simulator issue 2013-12-11 T-6 2013-12-11 21:40 2013-12-13 16:00:00 Toulouse 2013-12-09 T-6 2013-12-09 2013-12-11 Maryland CANCELLED due to weather conditions 2013-11- 14/19/21 T-5 2013-11-14 14:00:00 2013-11-19 14:00:00 2013-11-21 14:00:00 2013-11-15 14:00:00 2013-11-20 14:00:00 2013-11-22 14:00:00 Many locations Beacons of various types but in any case test coded and homer device deactivated 2013-08-29 T-4 2013-08-29 14:00:00 2013-08-30 14:00:00 Toulouse Retest of the test conducted on 2013-07-25 2013-08-13 T-1 2013-08-13 17:00:00 2013-08-14 17:00:00 Hawaii With the script agreed at JC-27, except used country code 367 instead of 338. Transmissions were on whole second intervals (3 & 4 seconds apart) 2013-08-01 T-1 2013-08-01 14:00:00 2013-08-02 14:00:00 Maryland With the script agreed at JC-27 2013-07-31 T-1 2013-07-31 14:00:00 2013-08-01 14:00:00 Toulouse With the script agreed at JC-27 2013-07-25 T-4 2013-07-25 14:00:00 2013-07-26 14:00:00 Toulouse With the script agreed at JC-27 2013-07-24 T-4 2013-07-24 14:00:00 2013-07-25 14:00:00 Hawaii TRANSMISSION CANCELED due to a simulator issue. 2013-07-23 T-4 2013-07-23 14:00:00 2013-07-24 14:00:00 Maryland With the script agreed at JC-27 2013-05-23 T-6 2013-05-23 14:00:00 2013-05-24 14:00:00 Maryland The 24 minute script will run once each hour 2013-05-16 T-6 2013-05-16 12:00:00 2013-05-16 14:00:00 Toulouse Transmission slot calculated to maximize the satellite visibility for all the MEOLUTs (except Hawaii). Beginning of each sequence: 12:00, 12:30, 13:00 and 13:30 2013-05-14 T-6 Dry run 2013-05-14 13:00:00 2013-05-14 14:30:00 Toulouse Each transmission slot with NB=25,50,75,100 beacons 2013-04-25 T3 modified - 2013-04-25 14:00:00 2013-04-26 14:00:00 Maryland T-3 at a lower beacon transmission rate (24 beacons in 48 seconds) 2013-04-23 T3 modified - 2013-04-23 13:00:00 2013-04-24 13:00:00 Toulouse T-3 at a lower beacon transmission rate (50 beacons in 48 seconds) 2013-04-04 T-4 2013-04-04 13:00:00 2013-04-05 13:00:00 Hawaii 2013-04-02 T-4 2013-04-02 13:00:00 2013-04-03 13:00:00 Toulouse Transmission stop between April 2, 21:00 UTC and April 3, 07:00 UTC. Transmission extended until April 4, 01:00 UTC 2013-04-01 T-4 Dry Run 2013-04-01 16:00:00 2013-04-01 18:00:00 Hawaii Dry run 2013-03-28 T-4 Dry Run 2013-03-28 15:00:00 2013-03-28 17:00:00 Toulouse Dry run 2013-03-25 T-4 2013-03-25 14:00:00 2013-03-26 14:00:00 Maryland 2013-03-20 T-4 Dry run 2013-03-20 14:00:00 2013-03-20 20:00:00 Maryland Dry run 2013-03-13 T-1 2013-03-13 13:00:00 2013-03-14 13:00:00 Maryland Toulouse A-2 2013-03-07 T-1 2013-03-07 13:00:00 2013-03-08 13:00:00 Hawaii 2013-03-06 T-1 Dry run 2013-03-06 18:00:00 2013-03-06 19:00:00 Hawaii 1 hour dry run 2013-02-25 T-1 Dry run 2013-02-25 11:00:00 2013-02-25 13:00:00 Toulouse 2013-02-20 T-1 2013-02-20 14:00:00 2013-02-21 14:00:00 Maryland 2013-02-14 T-3 2013-02-14 14:00:00 2013-02-15 14:00:00 Maryland 2013-02-13 T-3 2013-02-13 13:00:00 2013-02-14 13:00:00 Hawaii 24 beacons transmitted per power level (instead of 100) 2013-02-08 T-3 Dry run 2013-02-08 16:00:00 2013-02-08 18:00:00 Hawaii Subset of test T-3 (24 beacons transmitted per power level, instead of 100). Beacon log file available on the D&E server. 2013-02-05 T-3 2013-02-05 13:00:00 2013-02-06 13:00:00 Toulouse NOTICE: the test started on 2013-02-05 at 14:00:00 and ended on 2013-02-06 at 14:00:00. No beacon messages transmitted between 15:00:00 and 15:15:00 on 2013-02-05 2013-01-29 T-3 Dry run 2013-01-29 21:30:00 2013-01-30 21:30:00 Maryland Subset of test T-3 2013-01-10 T-3 Dry run 2013-01-10 08:30:00 2013-01-11 08:30:00 Toulouse Beacon log file available on the D&E server 2012-12-20 T-3 Dry run 07:30:00 10:11:00 Toulouse Subset of test T-3 2012-12-06 T-3 Dry run 09:00:00 10:11:40 Toulouse Subset of test T-3. Back-up window 12:55:00 – 14:16:40 - END OF ANNEX A - - END OF DOCUMENT - Cospas-Sarsat Secretariat 1250 Boul. René-Lévesque West, Suite 4215, Montreal (Quebec) H3B 4W8 Canada Telephone: +1 514 500 7999 / Fax: +1 514 500 7996 Email: mail@cospas-sarsat.int Website: www.cospas-sarsat.int