Tom Hunt, CEO and President
It might seem like the Internet of Things (IoT) is all anyone is talking about, but in fact it’s still an emerging market. In many ways, it resembles the Wild West more than a mature, well-organized market.
Because of both its “Internet” aspect and the multitude of “Things” it encompasses, on a trajectory to top 50 billion connected things by 2020, the IoT is awash in protocols. Just as the Wild West presented settlers with a largely lawless opportunity to claim some land for themselves, most makers of connected products see the IoT as a huge land grab. Individual IoT vendors are trying to stake their claims as quickly as possible, using the tools and protocols best suited to their particular goals. They don’t have time to worry about coordinating with everything else joining the IoT.
Consider the huge diversity of actual or potential IoT products in markets ranging from industrial automation, smart cities, smart buildings, and smart utilities to smart homes and smart health. The list goes on and on, from thermostats to tractors, coffee makers to street lighting, door locks to vending machines, pet feeders to environmental monitoring systems, and fitness trackers to trains.
These IoT products have vastly different data characteristics and needs. Some generate considerable amounts of data continuously, while others might send data to the cloud infrequently or unpredictably. Some require active control through a web or mobile app; others operate autonomously.
Think about all the wireless networks in use to transport data to and from the various IoT product types: Wi-Fi, Bluetooth, Zigbee, LoRaWAN, Cat 1 LTE and other cellular systems, SIGFOX, Zwave, Thread, IEEE 802.15.14, WiMax, and more. Then consider the server- or gateway-level communications protocols, such as HTTP, AllJoyn, Apple HomeKit, Nest Weave, Google Brillo, Artik, ZeroMQ (0MQ), Constrained Application Protocol (CoAP), and MQTT.
Each of these protocols exists for a reason and serves a particular purpose. They have different bandwidth capacities, operate at different ranges, have different latencies (i.e., delays), offer different cost structures, and come in both proprietary and standards-based formats. Each IoT product maker chooses protocols based on what’s best for their particular products.
The idea of developing a single set of standards that would work for this disparate range of products, and the data they generate, seems almost laughable. But the IoT market has made grand promises about convenience, comfort, safety, and productivity improvements. For the IoT to reach its potential as it scales to many billions of connected products, and to deliver on its promises, the issue of IoT data optimization must be solved. And that means dealing with the suitability of protocols based on specific use cases, as well.
IoT Optimization Must Encompass Protocol Considerations
Optimizing data compression for IoT devices is crucial, especially as the IoT continues to expand and scale toward its predicted tens of billions of devices.
That’s because traditional data compression technologies are challenged by many fundamental characteristics of IoT sensors and devices, including their constrained (i.e., limited) computing, memory, and storage resources; batteries that might need to stay on in the field for months or years at a time; and the need to communicate over slower and/or low-power wireless networks.
As an example, traditional compression works best with large amounts of data, but constrained IoT devices typically generate frequent, bursty messages. And although IoT products typically don’t require the network capacity or extremely low latency of security video streaming, for instance, they still need to transmit actionable data fast enough to perform basic functions: turning something on or off; closing a gate or door; sending an alert upon detecting a particular threshold of temperature or pressure; or signaling when a health or environmental condition being monitored is outside of a safe range.
What’s more, networking protocols themselves can exacerbate the challenges facing IoT data communications.
For one thing, networking protocols add headers and other transmission instructions to IoT data, which creates overhead that causes message sizes to balloon. A 100-byte message could carry a 200-byte header, expanding the message to three times the expected size. Multiplied over many transmissions a day, these super-sized IoT data messages can send wireless costs spiraling out of control.
Additionally, some protocols are more “chatty” than others, meaning they require a lot of extra bits to do their jobs. This chattiness extends transmission times, which causes battery radios to remain on longer, thus consuming more battery power and decreasing the IoT devices battery life.
Low-power wireless network protocols impose additional compression-related constraints on IoT communications. Some limit message size, frequency, or both. For instance, the SIGFOX wireless protocol limits each message to no more than 12 bytes of data, and it limits each device to a maximum of 140 messages per day. Other emerging wireless networks also require careful balancing of traffic over their narrow bandwidth, which is almost impossible to achieve without compressing the data.
The Way Forward for IoT Data Optimization
Data optimization in the IoT entails thinking differently about how data is compressed and transmitted. It’s important to understand:
- The data and its needs before compressing
- What networking protocols are being used to transmit the data, any limitations a particular protocol might impose, and if there are options for switching to a better protocol not on a message-by-message basis, but at the device level
- Any network traffic restrictions so they can be handled without changing software
To achieve IoT data optimization, a three-stage methodology has emerged. This approach was originally created and patented for automobile navigation systems, which have size, power, memory, and bandwidth constraints very similar to that of IoT products. Together, these three steps can achieve up to a 20:1 overall compression ratio.
The three stages are:
- Compaction -The IoT data message is pre-processed to compact it. Before encoding takes place, the data payload is analyzed to determine the data type, which elements can be reduced in size, and how to maximize compression.
- Compression -Based on the data type identified during the compaction phase, an appropriate algorithm is selected to compress the data optimally.
- Conversion -If a more optimal networking protocol is available, the message is converted to operate over that protocol. Then, the compression API identifies any traffic restrictions (e.g., to restrict message size)imposed by the network, the application, or protocol policies and determines if a less-chatty protocol exists. The API then parses the message to meet those requirements.
This three-stage IoT data optimization process works in conjunction with protocol connector technology that enables communication between dozens of different protocols. It also enables IoT messages to switch to a less-chatty protocol that might be available.
Residing on the front end of a server in the cloud or data center, this protocol connector technology offers optimized protocols that can reduce the amount of data being sent, without disrupting network back-end connectivity requirements. It receives IoT data in any protocol; translates it on the fly into the required network protocol; then sends it to the server or any client device in real time. Such “protocol swapping” to choose a more suitable option is a brand new capability for the IoT, and it adds a new level of interoperability for this emerging market.
Working together, optimized IoT data compression and advanced protocol connector technology offer a solution to looming challenges facing the IoT marketplace. If implemented widely, this approach can bring some much-needed order and cohesiveness to the IoT Wild West. More importantly, it can solve serious problems that many IoT providers haven’t even recognized yet well before they threaten to stop the momentum of the IoT industry as a whole.
Tom Hunt is CEO and president of WindSpring, Inc., San Jose, Calif. The company website, www.windspring.com, has more information about optimized IoT data compression and protocol connector technologies.