Public safety requires broadband networks that work everywhere because an emergency incident can occur anywhere. States and territories have now opted into the First Responder Network Authority (FirstNet) network, but state, local and tribal agencies now have an important decision about whether to adopt the service. Coverage and network performance are key attributes that agencies will consider when comparing the AT&T-based FirstNet solution with those from other commercial providers.
Related Stories
A Proposal for the 4.9 GHz Band
Several Organizations Support FirstNet Spectrum Renewal, NSA Expresses Concerns
To further investigate this challenge, Televate, an engineering consultancy, conducted a head-to-head comparison between two nationwide cellular carriers throughout a suburban county, where the majority of the population is concentrated in one area of the county. The analysis was conducted with two off-the-shelf smartphones of the same model, operating in the same location inside a vehicle, and tested in the same manner driving throughout the county. The test devices used standard commercial accounts and did not have priority service, an important feature to public safety.
Data Quality Indicators
Ultimately, public safety wants sufficient data bandwidth to satisfy the demands of the applications in use and when public-safety operations require reliable service. FirstNet required 768 kilobits per second (kbps) downlink and 256 kbps uplink in its request for proposals (RFP). The tests measured data bandwith parameters, as well as signal levels to determine the degree to which a particular network would provide indoor service. They also measured the data speeds, or throughput, the networks could deliver throughout the county. The signal level, radio signal noise levels and the spectrum resources available on a cell site can affect throughput, so these measurements were of particular interest. Finally, the amount of LTE spectrum employed by the two carriers was studied. Employed spectrum affects the total capacity available for each cell site, and it can impact the peak speeds available to a single user when the spectrum resources of multiple LTE channels are combined, called carrier aggregation.
Test Results
Perhaps what is most important to the user is where the various levels of service are available. In other words, can a user obtain a usable connection through the network, even at slow speeds, throughout the county? Certainly, a device displaying “out of service” is a prime example of no service availability, but if a device shows a connection, yet the connection quality is so poor that it cannot reliably pass messages through the network, public safety would not consider that location “covered.”
Especially for Carrier 1, a substantial portion of the area tested used 3G, and the performance on the 3G networks was lower than on LTE. Carrier 1 served only 82 percent of the area with LTE, while Carrier 2 served 96 percent of the area with LTE. Despite these results, both carriers’ published coverage maps indicate they serve virtually the entire county with 4G LTE service. Both carriers also depict 100-percent coverage of the county on their published maps. However, these maps may depict outdoor coverage, while our tests were conducted inside a moving vehicle. While the phones were connected for more than 98 percent of the measured area during the test, the test revealed that a far smaller area achieved reliable broadband service.
Coverage Levels
Signal level has a major impact on the type of service available at any given location. At higher signal levels, data speeds are likely to be higher, and high signal levels will penetrate more walls and deliver more reliable indoor service.
Carrier 2 has stronger signal levels overall throughout the county, enabling the carrier to provide greater in-building service levels — 15 percent more of the area tested — and in-vehicle service levels — 3 percent more of the area tested.
Carrier 1 covers 99 percent of the areas measured with outdoor service, but this would require vehicles installed with modems and roof-mounted antennas — adding substantial cost for more expensive equipment and installation to achieve a comparable level of service. Carrier 2 provided 98 percent of the LTE service area with signal levels sufficiently strong enough to achieve reliable in-vehicle service. These percentages are based only on the areas where LTE service is available from both carriers.
Data Speeds and Throughput
High signal levels do not guarantee good broadband performance. Network congestion, total spectrum available and carrier interference are factors in the amount of capacity and data speeds available to an individual user at a given location. If a major incident occurs at one of these locations, hundreds of users and local consumers could seek resources. The tests used as many resources as the network would allow to help gain greater perspective on the total throughput available to users throughout the county in case of an emergency incident impacting network capacity.
Carrier 1 provided service speeds of at least 768 kbps to the user — a speed capable of rich web browsing, CAD, AVL and most applications except video — to 95 percent of the measured area. Surprisingly, although Carrier 2 had larger LTE service availability and stronger signal levels, it had decreased downlink throughput performance. Carrier 2 provided download broadband levels of service to only 79 percent of the measured area. Carriers 1 and 2 provided data rates capable of high-definition — speeds higher than 2 Megabits per second (Mbps) — real-time streaming video in 85 percent and 68 percent of the area, respectively. Clearly, signal levels alone do not predict the performance of a broadband network.
The uplink performance is similar for the carriers, but both provided “broadband” speeds far less than the typical 95-percent coverage objective of mission-critical public-safety systems. Both carriers provided broadband upload speeds — at least 256 kbps — to only 80 percent of the service area. And speeds capable of high-resolution streaming video — more than 1 Mbps — were possible in roughly half of the area tested.
These speeds would accommodate, for example, video from an ambulance or a police cruiser in the field. Speeds of 2 Mbps needed for reliable high-definition video from the field were achieved in less than 20 percent of the area tested. Clearly, while the network did provide some service to most of the county, the uplink speeds did not achieve broadband speeds at the reliable levels of service typically required of public-safety networks.
The tests were conducted during the weekend, and there were no special events identified in any part of the county during the tests. So, the demand experienced during these test periods is likely similar to ordinary traffic periods. Throughput measurements were conducted to and from the same internet-based server for both carriers, so the results should isolate the performance differences of the two carriers. Given that Carrier 2’s signal levels were 4 dB stronger and the interference levels were lower (2 dB lower interference) in areas where both carriers provide LTE, the primary drivers of performance suggest that Carrier 2’s performance is because of issues outside of the quality of the radio channel.
One potential differentiator could be the amount of spectrum allocated to LTE in this area. A spectrum scan for LTE services in the populated area of the county found that Carrier 1 used 20 megahertz of downlink spectrum, and Carrier 2 used 30 megahertz of downlink spectrum. This additional spectrum could affect data speeds in two ways. First, more spectrum results in greater overall capacity. Second, higher downlink speeds can occur as a result of carrier aggregation. This feature allows the LTE system to allocate more than one LTE channel to individual subscribers. However, the test revealed that despite Carrier 2’s 50 percent advantage in LTE spectrum availability, it allocated less downlink spectrum resources to users than Carrier 1, especially at lower signal levels. The quantity of spectrum resources, over time, is fundamental to providing high levels of throughput to an individual user. As a result, Carrier 2’s downlink performance suffered. There is no evidence that congestion (lack of available resources) occurred to cause this effect, but if so, priority and pre-emption may alleviate these resource allocation issues for Carrier 2.
These tests underscored the importance of measuring system performance, not only service availability. The analysis demonstrated substantial differences between two carriers in regards to service availability and coverage levels, as well as a substantial difference between the availability of any service versus the availability of broadband services. Clearly, Carrier 2 was superior in terms of service availability and coverage penetration. Carrier 2 was also superior in terms of LTE service availability. However, the testing showed that Carrier 2 did not translate that advantage to downlink performance. Given Carrier 2’s advantage in signal levels and LTE service availability, it was surprising to see Carrier 1’s better downlink throughput results. And, while uplink performance was similar between the two carriers, both carriers did not provide broadband service levels at the coverage reliability levels expected by public safety.
Looking Forward
As agencies move forward and consider adoption of the FirstNet network, they need to carefully look at the performance of the various options available to them. Carrier coverage maps are not useful when it comes to conducting a realistic comparison. Tests such as these will help agencies fully understand the real-world performance of the networks within their respective service area. Uncovering issues, such as Carrier 1’s service area, frequent transitions to 3G technologies and signal levels, as well as Carrier 2’s downlink throughput, will help begin a dialog on the important enhancements necessary for providers. Which carrier is best in this county depends on the most important attribute for the service — is it the largest footprint or the higher download data speeds?
The tests underscore the importance of an agency fully understanding its operational requirements. In this case, a substantial amount of the county does not achieve 768 kbps downlink and 256 kbps uplink speeds for both carriers. An agency would need to assess its applications and determine what data throughput speeds are required to fulfill its mission. Perhaps, far lower speeds are acceptable. If an agency expects to stream video applications countywide, that raises the bar. In this case, only half the area for both carriers achieved uplink data speeds of 1 Mbps or higher.
The coverage measurements tool used to conduct these tests is out of reach for most public-safety agencies. It is expensive and requires cellular network engineering training to use. Most agencies can’t afford to have engineers drive around their service area to test in this fashion. In addition, a carrier’s performance can change dramatically over time. Carriers are adding new sites, migrating to LTE, adding new bands and otherwise optimizing the network.
An ideal coverage testing solution is an “agency-sourced” collection method whereby an agency’s own users collect data as needed. This approach enables agencies to test performance in the locations most important to them, in the configuration that best suits their environment, and when and as often as they’d like.
A new mobile application, Pinpoint, collects coverage and performance information using off-the-shelf smartphones with no formal training required. The tool delivers expensive engineering software to a public-safety agency. Agency personnel can travel around their normal or otherwise defined areas, by vehicle or foot, to automatically collect data at intervals frequent enough to qualify network performance. Armed with this information, an agency can engage in a more robust discussion with a carrier to address important coverage and performance gaps as well as to understand the seasonal factors, such as the effects foliage has on signal levels and data throughput.
As data becomes more mission critical, the broadband networks that deliver information needed to provide lifesaving services to the public become essential. It is important for agencies to understand their broadband data needs, now and in the future, and how the coverage and performance of commercial carriers fulfill these requirements.
Editor’s Note: The full version of this article with charts and a map ran in the June/July 2018 issue of MissionCritical Communications magazine.
Afeite Dadja is a certified project management professional and consultant at Televate with more than 20 years of experience in commercial cellular and public-safety LMR engineering and project management. He has designed more than a dozen public-safety Long Term Evolution (LTE) systems comprising more than 1,000 cell sites.
Joe Ross, a senior partner at Televate, has nearly 25 years of leadership in designing and operating LMR and commercial cellular systems. He chaired the Public Safety Spectrum Requirements Working Group for the National Public Safety Telecommunications Council (NPSTC) Technology Committee and is an editorial advisor to MissionCritical Communications magazine.
Editor's Note: We have been told that pre-emption affects all calls except 9-1-1 calls. So 9-1-1 should not be negatively affected.