IoT data quality challenges and how to solve them

Al Sisto

Blog by: Al Sisto - 15 / Feb / 2019

The value of the IoT lies in data – which means, in turn, that the value of the IoT lies in the quality of the data an IoT ecosystem can collect, analyse and interpret. This is not without its challenges.

Here are five of the key challenges your organisations might experience in terms of the quality of its IoT data – and how to overcome them.

Validation: identifying the irrelevant

Dirty data is one of the most obvious – and potentially costly – IoT quality issues. Imagine you have deployed connected sensors to automatically identify when a part on your heavy machinery needs servicing or replacing, and the information they transmit is incorrect in some way? You could end up ordering an unnecessary replacement part, or dispatching a costly engineer to fix a problem that doesn’t exist. Solving this challenge requires a multifaceted approach. You need tools and processes in places that can identify and flag potentially incorrect data, because it lies outside of normal or expected ranges.

Translation: turning data into a useful format

Often, IoT data is available but its format is unhelpful, perhaps because it cannot be usefully interpreted as it is, or because it needs to be enriched in some way. Videos captured by connected cameras, for example, may need to be broken down frame by frame so that an analytics platform can properly ‘read’ them and identify useful objects or patterns. Data on the performance or status of different elements on a production line may need to analysed as a unified whole, rather than in disparate parts.

Integration: combining with non-IoT data

It is important to remember that the information captured by your IoT ecosystem is just one part of a much broader data picture. Before the IoT came along, other enterprise IT systems were still capturing vast quantities of potentially useful information, from messaging systems, log files, transaction records and so on.

Failing to integrate and analyse that information in combination with the IoT can mean that IoT data is viewed out of context or in an incomplete way. This means that companies need to discover their other enterprise data, wherever it is being stored throughout the architecture, and combine it with their IoT ecosystem, so that they have a holistic view of all enterprise information. Solving this challenge may involve working with a systems integrator or interoperability specialist, to ensure that new and legacy systems are drawn together comprehensively.

Management at the edge

Even the smallest and simplest IoT ecosystems generate huge volumes of data at extraordinary speeds – and that data generation is ongoing and dynamic. If the data your IoT devices are collecting and transmitting is not stored and managed effectively, then you will quickly find your organisation failing to harness it properly, or worse, dealing with incomplete, corrupted or misleading information.

The IoT is driving a tendency for data to be processed closer to devices, at the network edge. Whilst this brings great benefits in terms of reduced latency and increased efficiency, it also means that IT staff need to ensure that data is properly managed outside of the core datacentre.

Security: protecting data from threats

As in all aspects of enterprise IT, data in the IoT needs to protected from malicious threats, accidental breaches, technology failures and human error. However, the very scale and dynamism of the data generated in the IoT – as well as the number of devices involved in a typical IoT ecosystem – make this a very different proposition from enterprise IT security just a few years ago.

Solving this challenge begins by ensuring that your organisation has a clear and comprehensive approach to ownership of IoT data security, avoiding overlapping or worse, neglected responsibilities. Then, you need to think about protection of your IoT data from the point of generation, through transmission and at rest, and how to verify and protect each individual connected device on your IoT ecosystem.

The IoT can allow you access to business intelligence you were never previously able to capture. It can generate new insights into your operations, and help you innovative and optimise your processes like never before. The foundation of all of these processes is top-quality data.

 

 

Topics: IoT, data, IIoT

Example CTA

Subscribe to Email Updates

Posts by Topic

see all