Group 23

Big Data Quality and Integrity, Fundamental Elements of the Industrial Internet of Things

In recent years, there has been an ever-increasing interest in developing solutions for Industrial Internet of Things (IIoT) applications. The interest spans many industries, with a large amount of the effort focused on predictive maintenance solutions. Predictive maintenance can be described as “extracting valuable business information from big data.” This is accomplished by applying complex analyses to vast amounts of data acquired from many thousands of sensors.

Automated analysis can sometimes lead to surprising results, in which case systems engineers would be justified to question whether or not the big data, and the automated analysis, is accurate. Everyone has heard the old adage, “garbage in, garbage out,” but the reality is that ensuring the quality and integrity of big data is not particularly easy. In fact, the sheer quantity of data churned out by thousands and thousands of sensors can put a tremendous load on legacy data acquisition methods. In addition, with wireless technology quickly becoming the connection option of choice for IIoT applications, mainly due to its convenience and mobility, the stability of wireless communications is a critical issue. A cause for concern is the inevitable unexpected connection interruptions that plague any wireless network, and which could result in data loss and expensive shutdowns of important business processes. Moxa’s smart data acquisition method helps to shrink the amount of data that needs to be transmitted and ensure data completeness. In short, smart data acquisition enhances the quality and integrity of big data, resulting in more accurate analyses.

Active Data Acquisition and Automatic Data Completeness

Active Data Acquisition

For many years now “updating data by polling,” which is the norm for SCADA software, has been the industry standard for communication between the server and clients. In the IIoT era, legacy polling methods do not mix well with big data, particularly since by its very nature, polling results in the collection of tons of worthless data from thousands of sensors, leading to high data storage costs and time consuming data analytics. When your sensors spit out data at a frequency much smaller than the polling interval, such as is the case when monitoring a machine’s ON/OFF status, “update by exception” can reduce the amount of data storage and increase the efficiency of data analytics. Moxa smart I/O supports Moxa’s patented Active Tag function, which actively updates digital or analog sensor data to Moxa’s MX-AOPC UA Server by exception or per configured percentage of changes. Not only will you save network bandwidth, you will also reduce the amount of data storage required.

Active Data Acquisition

Automatic Data Completeness

Most SCADA systems support saving data to a database in real time. However, data loss will occur when the network connecting the SCADA software with the remote I/O devices goes offline, or when the SCADA software crashes. A common solution to this problem is to back up data in local storage devices located near remote I/O devices. However, extra programming effort is required to collect offline data logs from the local storage devices and parse it to a database, since most SCADA software does not provide this kind of solution. The critical drawbacks are:

  • Third-party or in-house software engineers need to create additional software to handle data completeness. Developing this kind of software is costly and difficult to integrate and maintain.
  • An operations engineer needs to define the start and end time of the data loss duration for each remote I/O device, and the engineer needs to manually trigger the data completeness process, increasing the chance of collecting duplicate data or neglecting the data loss duration.

Moxa’s ioLogik 2500 series, MX-AOPC UA Server, and MX-AOPC UA Logger together form a turnkey solution for real-time data acquisition, data buffering in local storage devices, and automatic data completeness after network failures. MX-AOPC UA Logger is used to import data from MX-AOPC UA Server into a database in real time. After the network fails and then recovers, the logger automatically retrieves data logs, with timestamp matching the duration of the disconnection, from the data buffers of specific ioLogik 2500 devices, and then pushes the supplementary data into the database.