MIT Scientists Conduct First Third-Party Tests at FirstNet’s Innovation Lab
Monday, February 10, 2020 | Comments

Scientists from the Massachusetts Institute of Technology (MIT) Lincoln Lab (LL) conducted video testing at the First Responder Network Authority (FirstNet) Innovation and Test Lab in Boulder, Colorado, in May 2019. The tests demonstrated that data pre-emption and priority were dependent on a variety of factors.

The tests were the first external-party testing conducted in the FirstNet Lab and aimed to evaluate the effects of different Long Term Evolution (LTE) and nationwide public-safety broadband network (NPSBN) parameters on the quality of service (QoS) of streaming videos including the Public Safety Innovation Accelerator Program (PSIAP) dataset. The MIT team developed software to enable evaluation of video streaming performance.

Video applications are projected as a stressing and significant service of the NPSBN. The scientists transmitted video from the dataset across public-safety and commercial broadband networks under different network conditions.

The New Jersey Office of Homeland Security and MIT LL developed a PSIAP video and imagery dataset of representative and operational public-safety scenarios to enable NPSBN testing with public-safety data. The federal Public Communications Research Communications (PSCR) program funds the PSIAP research.

Initial tests included two use cases for radio access network (RAN) loading and same video. The test parameters included attenuation and signal to interference noise ratio (SINR), spectrum, quality of service class identifier (QCI), allocation and retention priority and subscriber identity modules (SIMs).

The scientists ran tests with both high user equipment (UE) simulated load and high bandwidth simulated load. For the high UE scenario, they ran both uplink (UE to server) and downlink (server to UE) tests. The testers only considered FirstNet and commercial band 14 SIM configurations to compare the performance between the NPSBN and commercial AT&T configurations.

During both the high UE load uplink and downlink tests, each UE loaded the same PSIAP test video and initiated a stream to the media server. Once the simulated load reached several hundred UEs, the device with the commercial SIM lost connection to the network, and the stream terminated. However, the device with the NPSBN SIM remained connected, and the stream continued. There were some dropped frames in the received stream but not significantly more than in the control runs.

A separate downlink test with high-bandwidth simulated load to model UEs using significant downlink bandwidth was set up identically to the high UE load downlink test, except the group ran the high-bandwidth RAN configuration, which simulates UEs downloading large files from a server.

“We found that both devices experienced comparable degradation in streaming video quality in this test,” the report said. “The streams on both devices exhibited significant packet loss, resulting in dropped frames and choppy playback; both streams also struggled to maintain the original video resolution. These effects were observed for 10 to 30 simulated UEs, with increasing load resulting in more dramatic degradation. At 40 and 50 simulated UEs, both streams were rendered unresponsive and eventually lost connection to the server.”

Because of time constraints, the MIT LL officials were unable to conduct an uplink test with high bandwidth simulated load.

“The results of the high-bandwidth tests illustrate a bottleneck in the testing architecture,” the report said. “Since the high-bandwidth load is transferred over the internet, outside of the FirstNet network, the data priority policies of FirstNet do not apply. Thus, internet traffic of NPSBN devices could be subject to congestive effects of a small number of commercial UEs if they are using significant bandwidth. This also highlights that the FirstNet network alone is not a comprehensive solution for prioritizing and managing public-safety communications.”

The group suggested future evaluations should further explore the effects of network resources and bandwidth usage on connection quality; additional PCI and PVI configurations; develop additional RAN loading configurations to explore different loading scenarios; and explore the utility of streaming output of analytics, which likely will require less bandwidth rather than high bandwidth video.

The tests “demonstrated that devices using FirstNet NPSBN SIM cards were not pre-empted due to a large number of UE connections, unlike devices using the commercial SIM,” the report said. “We also found that downlink transmission for both NPSBN and commercial devices were degraded by high-bandwidth usage from a small number of devices, due to the internet traffic traveling outside of the FirstNet core network, where data priority cannot be enforced.”

The researchers released the software under a permissive license here. A specific subset of the dataset, the Low Altitude Disaster Imagery (LADI) dataset, will be used in the National Institute of Standards and Technology (NIST) TREC Video Retrieval Evaluation (TRECVID 2020), specifically the disaster scene description and indexing task. The capabilities developed under this federally funded challenge could be tested at FirstNet in the future.

In a partnership with Wayne State, MIT researchers prototyped an architecture for in-vehicle testing. This in-vehicle prototype was intended to help envision the police cruiser or fire truck of the future and additional work on developing in-vehicle architectures continues.

The MIT Lincoln Lab project was established via a licensing agreement. No additional confirmed external-party testing is scheduled for the lab, a FirstNet spokesperson said.

The full paper is here.

Would you like to comment on this story? Find our comments system below.

Post a comment
Name: *
Email: *
Title: *
Comment: *


No Comments Submitted Yet

Be the first by using the form above to submit a comment!


March 2023

27 - 30
International Wireless Communications Expo (IWCE) 2023
Las Vegas

May 2023

23 - 25
Critical Communications World (CCW)
Helsinki, Finland

More Events >
White Papers
More White Papers >

Site Navigation