From a Time-Series Database to a Key Operational Technology for the Enterprise: Part II

This three-part series expands on a talk that I gave at a conference focused on time-series data in 2018.

The first article in this series explored challenges related to time when elevating a time-series database from simply an information technology, to being an operational technology, used through the enterprise. In this article, part two in the series, I will explore integrating time-series data with asset metadata and asset models, to provide context for the underlying telemetry.

Assets

Probably the most vital aspect for elevating a time-series database from being just another data-store, to an infrastructure that is operationally critical to the entire enterprise, is augmenting the telemetry stored in the database with asset information, and the asset models of the business. Instead of being just a collection of time-series, an asset-centric view expresses the underlying telemetry in the language of the business.

At first glance, it makes sense to store the asset metadata alongside the telemetry, as a sequence of immutable facts, following the Event Sourcing model. This works really well when monitoring services or applications in a data center. Telemetry like query latencies, or the usage of resources like memory, processor, network, and disk, can be associated with metadata like the service name, software version, data center, availability zone, business cost-center, and so on. Because these configurations are generally declarative and the deployments automated, the metadata do not change, and the associations are reliable. If changes are made, it is generally through a new deployment, with a new, unique set of metadata. In an operational setting, integrating asset information is a lot more complex.

First, what is an asset? The answer to this question usually depends on your perspective. For example, if you are scheduling or operating an airplane, you may view the entire airplane as the asset. However, if you are an aircraft engine manufacturer providing a contract for the maintenance and performance of the engine, you may view the engine alone as the asset, independent of the aircraft that it is installed on.

Your perspective also influences the time-series that define an asset, requiring polymorphic, role-based views of the same asset. Consider a steam turbine. If you are an operator, you might care about the flow rate, temperature, pressure, power, and the current set-point. A condition-based maintenance application might also incorporate the flow rate, temperature, pressure, and power, but, in addition, the vibration, faults, and run-time of the turbine since the last maintenance was performed, not considering the current set-point at all. Many industries also require role-based security to view the different series that define an asset. A time-series database that only provides database security, or security through a single definition of the series that compose an asset, is usually too inflexible in an operational setting.

The underlying series that characterize an asset can be different, depending on your perspective, requiring polymorphic views of the asset and the series that define it. Role-based security is often applied through the same polymorphic model.

When managing a large number of assets, in addition to understanding how a single asset is performing, we often need to understand how a group of assets are performing in the aggregate. It is common for assets to have parent-child or hierarchical relationships. These hierarchical arrangements may be logical, based on the customer or the asset owner, or physical, based on proximal relationships. For example, when managing distributed, renewable energy assets, it is common to want to know how much power a group of assets, like a collection of wind turbines, is producing in real-time. A single asset can also be a part of multiple hierarchies, perhaps included in a hierarchy described by geography, as well as one defined by the firmware version, the asset owner, or the capabilities of the hardware. It is common to apply role-based security in relation to the asset hierarchy as well. Some operators may be able to view telemetry and issue control commands to assets in a certain jurisdiction, while not being permitted to view telemetry or control assets in a neighboring jurisdiction.

Assets are often arranged in hierarchies representing physical or virtual relationships. Role-based access is applied through the asset model.

Most time-series databases support a group-by query that can group telemetry based on common asset-metadata. If used judiciously, asset metadata can be used to describe parent-child relationships, or even hierarchies. Most group-by queries, however, only group by the data that are present in the database for the relevant time-frame. This type of query works great for monitoring assets within the data center—queries like the mean number of requests-per-second, grouped by all API servers, without having to consider how many servers there are, or even which servers are handling the requests. In an industrial setting, it is often just as important to know which assets are not reporting telemetry. We need to understand the uncertainty in aggregations, as a representation of the data quality. For example, the total power output for a wind farm is currently 57 MW, but with only 19 of the 20 wind turbines reporting telemetry. Is the 20^th wind turbine off-line, or has the telemetry from it been lost, or delayed in transit? We need to factor this uncertainty into our operational decisions, especially when performing closed-loop control.

As the relationships between and among assets become increasingly important in the time-series system, capturing how the asset model itself is changing over time becomes critical. Tomorrow, more assets might come on-line. Next week, or next month, the asset hierarchy might change, as assets are commissioned, retired, or restructured. For querying historical data, it is important to query the telemetry with the asset model that was governing at the time, not the current asset model. This starts to give the asset model itself a time-series component, so that we can see when it changed, how it changed, and what it looked like in the past.

Asset models will change over time. It can be important to understand when, how, and why they changed. We need to represent historical time-series data with the asset model that was governing at the time, not the current asset model.

Similarly, assets themselves will change and evolve. Mechanical parts wear, or fail, and they are replaced. For an aircraft, it might have the same airframe and other structures, but with a new engine. Understanding when the engine was replaced, what the current engine is, what the previous engine was, what the associated performance-curves are, can be essential for providing context to the telemetry. The owner of an asset can also change over time as assets are bought, sold, or acquired. Changes in assets themselves, or in the metadata associated with an asset, like customer information, can make it difficult to associate all of the metadata at the time of ingest, in an immutable fashion. Most often, it is necessary to change, adapt, and revise the metadata independently of the the telemetry that was stored in the time-series database. If the only means of associating asset metadata with telemetry is by tagging measurements on ingress, it becomes impractical, or even impossible, to re-ingest data, just to evolve or revise the metadata. A more flexible model is required. I will revisit this idea in the next article in this series.

In many physical processes, especially the ones common in chemical, mechanical, or electrical engineering, it can be valuable to express directional relationships between assets. In an oil refinery, the outlet mass flow-rate of a pump is the inlet mass flow-rate to the hot-side of an adjacent heat exchanger. These two flow-rates must be equivalent in terms of the product, the direction, the units, and so on. Describing assets in this manner essentially establishes a declarative strong-tying system, and supports automatically creating calculations and visualizations derived from the asset model, which can be very powerful, especially when managing systems at scale.

A piping and instrumentation diagram expresses directional relationships between assets. The mass flow-rate out of the pump is equal to the mass flow-rate into the hot-side of the adjacent heat exchanger. Directional relationships can provide strong-typing to the asset model and support automatically deriving calculations and visualizations.

In summary, to describe the relationships between and among assets, we need a lot more than group-by queries. We need to be able to describe hierarchical, proximal, and directional relationships. These relationships can change over time, along with the assets themselves, giving a time-series component to the asset models as well. Not all asset metadata will be available at the time of data ingest, so rather than simply tagging time-series data on ingress, we need flexible asset-models that can evolve over time and be associated with the underlying, immutable telemetry. Asset models are also important for supporting different views of the same assets based on the security context and through role-based access.

In the third and final article in this series, I will explore how to integrate a time-series database with the many other systems that surround it. I will focus on how to do this in a reliable and scalable manner.