A guide for improving data management, driving IT/OT collaboration and enabling sustainable and resilient manufacturing operations
The foundation for effective data management must include three levels of technology:
- Pipes to collect and move data out of its source. With the rise of cloud and now edge computing, innovation in this area continues, especially as 5G comes to the market.
- Mathematic output and visualizations, where significant time, money, and skills have been devoted (think business intelligence (BI), data science, AI/ML) but scale has proven to be a challenge. It is now possible to deploy a tool that collects and analyzes point sources of data for specific, local problems in plants. But this approach does not work beyond point solutions. Efforts stall because plant data is so varied and difficult to draw insights from holistically.
- Sense-making is the increasingly necessary but often missing middle step between pipes and insights. Technologies here help turn data into usable information. For some fields, horizontal solutions and DIY tools are feasible, and IT and operations teams should do the work. In others with especially complex data environments, domain-oriented products can provide significant acceleration.
From years of hard lessons, manufacturers know they must address the middle gap — and when they do, there is an invaluable outcome: a common set of information generated by all plant data. This single foundation is immediately useful for operations, data science, and business teams — it is accurate, transparent, consistent, trusted, and delivered in real-time.
This kind of broad, standardized foundation is ambitious, and so difficult to achieve with existing data engineering and data science methods that many companies do not yet even seek it.
Let’s take a closer look at the current challenges of plant data before diving into how Sight Machine enables sense-making, and lastly, how the results translate to more productive, sustainable, and resilient factories.
Common Data Challenges in Smart Manufacturing
Let’s take a closer look at the current challenges of plant data before diving into how Sight Machine enables sensemaking, and lastly, how the results translate to more productive, sustainable, and resilient factories.
Successful sense-making hinges on the ability to translate data into actionable insights, but this is easier said than done. Manufacturing IT teams, engineers, and data scientists are currently grappling with the following common data challenges:
- Extraordinary heterogeneity. Today’s increasingly connected factories generate a high volume of data from different places, at different rates, in different formats. Data is often dirty, out of order, late, or missing. Productivity measures depend on data from many separate sources — such as sensors, operators, business information — which is exceptionally difficult to knit together in a meaningful way. Industrial data differs greatly from the well-structured data sets seen in other technology fields and as such, many industrial organizations have implemented digital technologies and automation with wildly unrealistic expectations when it comes to getting the most of their data.
- Data is “raw” and does not flow through mature IT infrastructure. Data is raw, gathered by different means, and housed in different planes. While business process information has historically been managed by IT leaders, factory data is generated by physical assets managed by the OT team. Solutions must go to the front end of the pipes to acquire and move data effectively. For data to be useful, you need pipes that bridge IT and OT effectively.
- Data takes days or months to gather, model, and analyze. In order to use data for productivity improvements, information must be generated automatically and presented in real-time in a way that enables workers to take action. Retrospective analysis is not enough.
- Coming from so many sources, critical data is often late, missing, or out of order. This calls for a variety of capabilities not usually found in generalpurpose stream processing tools. Technologies like Apache Flink have evolved to address issues with event processing and windowing, but manufacturing data still stretches these tools beyond their focus.
- Data has to do many jobs. Data scientists use AI/ML. Operators use dozens of plant floor software point solutions, which are widespread, idiosyncratic, and often decades old. Sometimes, plant IT or OT builds models, and, in larger companies, so does corporate IT. So how do you arrive at a comprehensive view of information that is universally useful to data science groups, operations, finance, and supply chain?
- Both the analysis and the underlying data must be transparent and trusted. Especially in manufacturing, stakeholders are used to working with raw data. They need to be able to see, access, and modify data sources. If not, they won’t trust the resulting insights.
The Path to Meaningful Insights
By continuously converting plant data into a useful foundation, sense-making serves every stakeholder of the manufacturing enterprise:
Sense-making provides one common Data Foundation for all models — both operational and analytical.
Save time and money and drive consensus.
Sense-making handles changing data environments
Get the most out of your data models and technology investments with continuous, meaningful insights.
Stream-processing technologies provide continuous, real-time understanding.
Move from reactive to proactive decisions and improve strategy at every level of activity, from asset to enterprise
Build the Data Foundation and Enhance Sense-Making with Sight Machine
Sight Machine’s unique end-to-end solution is designed to fill the gap between pipes and output, seamlessly plugging into a plant’s existing systems and technologies. The heart of the offering is the critical sense-making layer — a streaming pipeline that continuously analyzes all plant data.
The product transforms and analyzes data from any manufacturing environment in process and discrete industries, and plants small and large. It complements and accelerates existing investments and provides OT and IT leaders with a single layer of trusted information.
This is achieved through two key product features:
- Common Data Models that are applied to all manufacturing activity regardless of the product made or assets used.
- Pipeline as a Service that enables sense-making. Many plant engineering teams are in the habit of building individual pipelines, each bespoke and each targeting a specific problem. However, this method falls short in its sensemaking capabilities.
Sight Machine enables a universal, configurable pipeline applied to all manufacturing activity that effectively streams, models, and maps plant data from Common Data Models. Watch the video to learn more and see it in action.
Define Common Data Models with the Unit of Work at the Core
A first step is to define the Common Data Models. Getting this right is far from intuitive.
Say we are trying to understand a vehicle’s production. A car has 30,000 parts, and if we are going to build data models for all those parts being made, we would then have to model tens of thousands of machines, too. There is no way all these models can be organized and understood — but that has not stopped companies from trying. Machines have generally been the building block of choice, but this is obviously problematic. Sight Machine uses only about ten Common Data Models. One of the most important of these is the Production Mode, which allows customers to continuously understand production activity at different levels: machines, lines, and plants. But effective modeling should not start with machines, as seen with traditional industry approaches; it begins by modeling units of work.
Think about all those machines needed to make a car. A unit of work can be the repetitive act of die casting an engine block. Applying paint or a coating. Injecting plastic into a mold for a dashboard piece. It can represent work done in a discrete process or in continuous production, and it can take a second or hours. Types of work are limitless, but the idea of the unit of work is common, almost elemental. A unit of work is just the repeated cycle of activity by a machine.
From there, unit of work input is translated to Common Data Models that allow real-time insights into machines, lines, plants, the enterprise, and the supply chain.
Sight Machine’s unique approach reduces the manufacturing process to its most basic element, the unit of work, and provides answers for these critical questions:
What happened as the work was being done?
How long did each unit of work take?
What were the characteristics of the material being worked on at the beginning of the unit of work?
What were the characteristics of the material being worked on at the end?
How do Common Data Models organize data?
Every time a machine does a unit of work, a row of information is generated from all the data associated with that work. The unit of work is described with data from sensors on the machines, quality systems, manufacturing execution systems (MSP), historians, enterprise resource planning (ERP), and even ambient data like temperature and humidity, or other data about raw material characteristics.
There is no minimum level of data required to characterize a unit of work, and no maximum. Information about the unit of work can “flex” with the level of ingested data. In data table terms, the row of information can be modified to add parameters in new columns as new data becomes available. Information about the unit of work also changes as data sources change or late-arriving data or missing data is incorporated. This kind of flexible recalculation, not possible until the advent of robust stream processing tools, is essential for factory data.
Once you have a common data model built on units of work, it becomes more straightforward to apply the many kinds of analytical techniques engineers and data scientists know well: both within Sight Machine, which has rich analytic and visualization layers, and with other products and internal tools and systems.
With access to multi-dimensional, real-time information, teams can go beyond analysis. They can move, for example, into much more active process management, wherein experimentation and validation become feasible with previously unimaginable speed. Teams can also use information derived from units of work to manage maintenance or capital investment and combine production data to gain supply chain insight. These and other initiatives become feasible when teams have access to contextualized information that consistently represents all manufacturing processes.
Create a Universal Data Pipeline
While Common Data Models are core elements of the modern data stack, defining the models themselves is far from sufficient. What’s needed next is a robust, configurable pipeline to stream data, map it, and handle high complexity. Sight Machine provides this through our platform’s Pipeline as a service feature.
Conceptually, we can all imagine translating a selection of data, perhaps quite voluminous, into manufacturing units of work. Now imagine doing this continuously, converting billions of data points per day into accurate information about millions of units of work. This is where Sight Machine’s second core innovation — Pipeline as a Service — becomes critical.
From almost a decade of streaming and analyzing data from plant environments, Sight Machine has developed a universal data pipeline for manufacturing. The pipeline includes hundreds of techniques to map, model, and distill data so raw plant data is continuously mapped accurately into Common Data Models, and in turn elevated into analytics. The Universal Data Pipeline employs as many as 50 automated transformations. It is browser-based, configurable, and transparent: data engineers and data scientists can see both raw and transformed data throughout, and they can use it themselves to generate information. Once configured, the universal data pipeline continuously generates useful information, and adjusts in the background to changing data.
Sight Machine’s Pipeline as a Service is a core element of why we at Sight Machine believe that the only way to improve productivity in plants is to build a solid data foundation upon which all analytics and continuous improvement efforts can be scaffolded up. Pipelines can be cloned, shared, and easily modified to reflect new developments or incorporate new data sources. This adapts to the real world of manufacturing, where data is always changing and the environment is incredibly dynamic.
Let’s go back to the car’s engine block example in the previous section on Common Data Models.
Late in the production process for one of the engine blocks described above, the engine block is assigned a serial number. To do quality analysis on the engine’s production, ideally we would connect quality results to process data at various checkpoints; engines are often made across multiple plants with multiple serial numbers. As a matter of data mapping, this entails a difficult step: we want to associate data and trace it to a part long before the part actually got its number, or, in complicated scenarios, its first number. As a mechanical matter, how do we connect that latearising serial number back to the earlier steps?
Sight Machine’s Universal Data Pipeline makes this connection automatically using built-in, automated data transformations. The Universal Data Pipeline can also manage time stamp variation, incorporate late data retrospectively and automatically (sometimes up to 30 days late), and apply countless contextual rules. For instance, a packaging machine might send a signal that it has shut off. If you’re the machine builder, there is no way to know, absent context, if the machine has shut off because of an internal problem or because of an external, upstream cause. Sight Machine enables root cause analysis to accurately determine the cause of the shut off.
Sight Machine’s Microsoft Partnership Provides an Integrated Solution for Manufacturing
With a strong Data Foundation built on Sight Machine, companies can now accelerate and unify their efforts for all work streams — operations, IT, data science — around established technologies. Sight Machine supports and integrates with technologies from all major edge, cloud, and analytics providers.
Our partnership with Microsoft’s Agile Factory enables Sight Machine customers to harness the power of all of their production data through Microsoft’s powerful cloud computing capabilities, delivering the system-wide visibility and robust insights required to efficiently and effectively manage plant operations anytime, anywhere. The result? Faster insights at a greater scale that make a tangible impact on the bottom line. Download the ebook to learn more about our partnership with Microsoft.
Reference architecture of Sight Machine in a full Azure environment, including ADS, Data Bricks for Azure, Synapse, Azure ML, Azure Purview, Azure Edge, Azure Cloud, and all of Microsoft’s manufacturing and supply chain partners.
IT/OT Data Convergence Breeds Bottom Line Improvements
Sight Machine brings equal value to OT and IT, delivering data driven insights that facilitate collaboration and drive bottom-line results. Plants can be up and running on the solution and pay back their investment with hard, quantifiable results in well under a year.
The next moves are in the hands of technology and operations leaders. Companies may want to develop their own algorithms or combine contextualized information about production with other information in unique, proprietary ways. This is the domain of tools and the numerous visualization, analytic, and AI frameworks that are available. But only with the right Data Foundation in place will both OT and IT teams be able to work together to make tangible progress on business goals.
How the four levers of manufacturing performance — throughput, quality, cost and flexibility — drive productivity improvements at different levels of data analysis:
Improvement use cases and potential outcomes include:
- Finding and fixing the root causes of quality issues
- Identifying the causes of micro-stops in high-speed automated production
- Developing best practices across sites
- Minute-by-minute visibility of key production KPI’s and changes across the manufacturing enterprise
- Visibility of production across the supply chain
- Dynamic recipes for optimizing process control. Recipes take account of variations in raw materials, production conditions, goals to be optimized
- Reducing energy use in heat-intensive industries through improved quality and use of assets
- Optimizing asset productivity and scheduling of process steps in processes involving varying demand and supply
Traceability and Supply Chain
- Joining insight about upstream and downstream production processes
- Sharing quality and throughput data between tiers
- Visibility and analysis of distributed assets: useful to providers of assets, plants, and enterprises who seek to manage production facilities as an integrated fleet
Sight Machine is committed to solving our customers’ challenges and enabling resilient, sustainable factories of the future. To that end, Sight Machine is one of 50 companies participating in the World Economic Forum’s (WEF’s) invite-only Global Innovators Community, which is dedicated to accelerating the growth of advanced manufacturing while helping stakeholders fulfill their social responsibility.
As part of this partnership, Sight Machine is engaging with WEF’s Advanced Manufacturing and Production Platform across various working groups, including:
- New Business Models by Advanced Manufacturing
- Carbon Reduction in Manufacturing
- Unlocking Value in Manufacturing through Data Sharing and Open Data
In addition, we will also share our learnings from working with customers across all manufacturing industries with the Forum’s Global Lighthouse Network, a network of the world’s leading 44 manufacturing companies that are setting an example through digital transformation(5). Sight Machine is contributing to the network through our insights into scaling advanced manufacturing technologies to dozens of plants in an enterprise. We will be working in the Global Lighthouse Network alongside our trusted partner, Microsoft. Moving forward, Microsoft and Sight Machine will help the World Economic Forum identify new plants and manufacturing companies to add to this network and we would love the opportunity to consider your organization.