The dirty secret of digital transformation in manufacturing is that it is not really a technology problem; rather, it is a data problem. Many off-the-shelf technology solutions that work in other fields are wonderful when all the data is delivered in one format or one structure, but they struggle to add the proper context to messy data. Manufacturing data is among the messiest data around. It comes in a wide variety of formats (e.g. SQL, CSV, XML, and proprietary formats) by a wide variety of systems with different metric measurements (e.g. PLC, MES, QMS, ERP, and historians). Sometimes the data is a streaming process, measured as a time series. Other times it is discrete data (e.g. code, alarm, and defect reports) and measured at the part level.
Sight Machine’s Pipeline as a Service: Transforming Manufacturing Data Diversity Into A Data Foundation With Rich Context
Sight Machine’s Pipeline as a Service takes this alphabet soup of data and transforms it using data pipelines that enable manufacturing companies to create stable yet agile data foundations on which you can build out analytics and manufacturing productivity improvement programs.
This is our third version of our data pipeline technology and we have learned much along the way, working with dozens of plants and manufacturing teams to improve their productivity. Our Pipeline as a Service is the summation of years of learning how to create common data models that abstract away much of the complexity of data integration and mathematically map to digital representations of commonly used manufacturing processes and outcomes. Beyond these common use cases, Sight Machine’s Pipeline as a Service works on any unique process that can be defined by data collected from manufacturing assets, inputs, outputs, and outcomes. In fact, part of the value of Sight Machine’s Pipeline as a Service is the ease with which manufacturers can extend data pipelines to incorporate either new or external data sources, and can create an infinite number of permutations of what they want to examine with minimal toil and reconfigurations.
By creating these common data models, Sight Machine’s Pipeline as a Service significantly shrinks time-to-value for manufacturers and enables measurable and meaningful outcomes for manufacturing teams within weeks or months. We frequently see clients identify ways to improve quality, availability or performance (core components of OEE) by 5% or more on specific metrics or processes even as we are building the data pipelines and running first-cut analysis of their manufacturing process data.
What Sight Machine’s Pipeline as a Service Does
Our Pipeline as a Service does the following:
- Takes in raw data.
- Transforms that data using a library of pre-built operators.
- Empowers the rapid and iterative construction of Manufacturing Applications that are inexpensive to maintain and update (because of the stability of the data foundation).
Typically, Sight Machine’s Pipeline as a Service acquires data from Sight Machine’s FactoryTX intelligent data acquisition platform. FactoryTX can also ingest, read, and transform data from other manufacturing data platforms and parse it for digestion in Sight Machine’s Pipeline as a Service. To give you an idea, our Pipeline as a Service can consume and transform to add context to nearly all relevant types of process data including:
- Machine process data
- Downtime information codes
- Quality information (including time-delayed or manually entered data)
- Shift schedule data
On the other side of the Pipeline emerges useful, actionable data that maps back to the time a batch or part was produced. Operators and engineers can then explore hypotheses about the data and pipe data outputs into dashboards, visualization tools, and analytics packages, among other endpoints. A partial list of potential outputs of Sight Machine’s Pipeline as a Service includes the:
- Production model
- Part model
- Downtime model
- Defect model
From Pipeline transforms, manufacturing operations teams can quickly and objectively analyze and measure OEE and its three key components (Performance, Availability, Quality). Our Pipeline as a Service can do this while reducing the need to write complex code and simplifying programming workflows. By minimizing the need for data rework and cleaning, data scientists and data engineers can focus more of their time on building and testing models and applications that deliver value.
Manufacturing teams only need to ensure that they have come to an agreement on a Data Dictionary (See our article on this topic in Industry Week). Based on their Data Dictionary, they will have modified their tags and fields to make sure that there is a unified and agreed upon schematic for data types and metrics. These adjustments are crucial to ensuring that the transform operators we have written into our Pipeline as a Service work as designed and transform the various data types and sources accurately and to the right timescale.
How Sight Machine’s Pipeline as a Service Works
Two key concepts for understanding our Pipeline as a Service are “operators” and “pipelines.” Operators are algorithms or functions that transform raw manufacturing data into normalized formats. Pipelines are a series of data operations that take place in sequence. Each pipeline is a set of instructions – operators – that tells the platform how to process the incoming data. Each operator functions as a block in the pipeline that performs a specific action or calculation on the data. Think of the blocks as LEGOs that can be stacked and rearranged as needed. Pipelines are extremely configurable with few limitations. This gives manufacturing teams the flexibility to configure a pipeline and operators to capture the reality of their specific production process, without having to do significant amounts of development.
Sight Machine has already built out a number of operators that are commonly used to transform manufacturing data from raw to more useful analytics-ready formats. Manufacturing teams can add their own custom operators via extensions to the Pipeline as a Service platform. It does take a bit of work to properly configure and test Pipelines, but the work does not involve heavy coding and usually can be done in a matter of weeks or a couple of months, depending on the organization. What’s more, the organization’s internal team can usually do this on their own with some basic guidance.
Sight Machine’s Pipeline as a Service is a core element of why we at Sight Machine believe that the only way to improve productivity in plants is to build a solid data foundation upon which all analytics and continuous improvement efforts can be scaffolded up. Pipelines can be cloned, shared, and easily modified to reflect new developments or incorporate new data sources. This reflects the real world in manufacturing, where data is always changing and the environment is incredibly dynamic.
