9. Verification

This section describes how to formally verify whether the control sequence is implemented according to specification. This process would be done as part of the commissioning, as indicated in step 9 in the process diagram Fig. 3.1. For the requirements, see Section 5.3.

For clarity, we remind that verification tests whether the implementation of the control sequence conforms with its specification. In contrast, validation would test whether the control sequence, together with the building system, is such that it meets the building owner’s need. Hence, validation would be done in step 2 in Fig. 3.1.

As this step only verifies that the control logic is implemented correctly, it should be conducted in addition to other functional tests, such as tests that verify that sensor and actuators are connected to the correct inputs and outputs, that sensors are installed properly and that the installed mechanical system meets the specification.

9.1. Scope of the verification

For OpenBuildingControl, we currently only verify the implementation of the control sequence. Outside the scope of our verification are tests that verify whether the I/O points are connected properly, whether the mechanical equipment is installed and functions correctly, and whether the building envelop is meeting its specification. Therefore, with our tests, we aim to verify that the control provider implemented the sequence as specified, and that it executes correctly.

9.2. Methodology

A typical usage would be as follows: A commissioning agent exports trended control inputs and outputs and stores them in a CSV file. The commissioning agent then executes the CDL specification for the trended inputs, and compares the following:

  1. Whether the trended outputs and the outputs computed by the CDL specification are close to each other.
  2. Whether the trended inputs and outputs lead to the right sequence diagrams, for example, whether an airhandler’s economizer outdoor air damper is fully open when the system is in free cooling mode.

Technically, step 2 is not needed if step 1 succeeds. However, feedback from mechanical designers indicate the desire to confirm during commissioning that the sequence diagrams are indeed correct (and hence the original control specification is correct for the given system).

Fig. 9.1 shows the flow diagram for the verification. Rather than using real-time data through BACnet or other protocols, set points, inputs and outputs of the actual controller are stored in an archive, here a CSV file. This allows to reproduce the verification tests, and it does not require the verification tool to have access to the actual building control system. During the verification, the archived data are read into a Modelica model that conducts the verification. The verification will use three blocks. The block labeled input file reader reads the archived data, which may typically be in CSV format. As this data may be directly written by a building automation system, its units will differ from the units used in CDL. Therefore, the block called unit conversion converts the data to the units used in the CDL control specification. Next, the block labeled control specification is the control sequence specification in CDL format. This is the specification that was exported during design and sent to the control provider. Given the set points and measurement signals, it outputs the control signals according to the specification. The block labeled time series verification compares this output with trended control signals, and indicates where the signals differ by more than a prescribed tolerance in time and in signal value. The block labeled sequence chart creates x-y or scatter plots. These can be used to verify for example that an economizer outdoor air damper has the expected position as a function of the outside air temperature.

Below, we will further describe the blocks in the box labeled verification.


Fig. 9.1 Overview of the verification that tests whether the installed control sequence meets the specification.


We also considered testing for criteria such as “whether room temperatures are satisfactory” or “a damper control signal is not oscillating”. However, discussions with design engineers and commissioning providers showed that there is currently no accepted method for turning such questions into hard requirements. We implemented software that tests criteria such as “Room air temperature shall be within the setpoint \(\pm 0.5\) Kelvin for at least 45 min within each \(60\) minute window.” and “Damper signal shall not oscillate more than \(4\) times per hour between a change of \(\pm 0.025\) (for a \(2\) minute sample period)”. Software implementations of such tests are available on the Modelica Buildings Library github repository, commit 454cc75.

Besides these tests, we also considered automatic fault detection and diagnostics methods that were proposed for inclusion in ASHRAE RP-1455 and Guideline 36, and we considered using methods such as in [Ver13] that automatically detect faulty regulation, including excessively oscillatory behavior. However, as it is not yet clear how sensitive these methods are to site-specific tuning, and because field tests are ongoing in a NIST project, we did not implement them.

9.3. Modules of the verification test

To conduct the verification, the following models and tools are used.

9.3.1. CSV file reader

To read CSV files, the data reader Modelica.Blocks.Sources.CombiTimeTable from the Modelica Standard Library can be used. It requires the CSV file to have the following structure:

# comment line
double tab1(6,2)
# time in seconds, column 1
  0   0
  1   0
  1   1
  2   4
  3   9
  4  16

Note, that the first two characters in the file need to be #1 (a line comment defining the version number of the file format). Afterwards, the corresponding matrix has to be declared with type double, name and dimensions. Finally, in successive rows of the file, the elements of the matrix have to be given. The elements have to be provided as a sequence of numbers in row-wise order (therefore a matrix row can span several lines in the file and need not start at the beginning of a line). Numbers have to be given according to C syntax (such as 2.3, -2, +2.e4). Number separators are spaces, tab, comma, or semicolon. Line comments start with the hash symbol (#) and can appear everywhere.

9.3.2. Unit conversion

Building automation systems store physical quantities in various units. To convert them to the units used by Modelica and hence also by CDL, we developed the package Buildings.Controls.OBC.UnitConversions. This package provides blocks that convert between SI units and units that are commonly used in the HVAC industry.

9.3.3. Comparison of time series data

We have been developing a tool called funnel to conduct time series comparison. The tool imports two CSV files, one containing the reference data set and the other the test data set. Both CSV files contain time series that need to be compared against each other. The comparison is conducted by computing a funnel around the reference curve. For this funnel, users can specify the tolerances with respect to time and with respect to the trended quantity. The tool then checks whether the time series of the test data set is within the funnel and computes the corresponding exceeding error curve.

The tool is available from https://github.com/lbl-srg/funnel.

It is primarily intended to be used by means of a Python binding. This can be done in two ways:

  • Import the module pyfunnel and use the compareAndReport and plot_funnel functions. Fig. 9.2 shows a typical plot generated by use of these functions.
  • Run directly the Python script from terminal. For usage information, run python pyfunnel.py --help.

For the full documentation of the funnel software, visit https://github.com/lbl-srg/funnel


Fig. 9.2 Typical plot generated by pyfunnel.plot_funnel for comparing test and reference time series.

9.3.4. Verification of sequence diagrams

To verify sequence diagrams we developed the Modelica package Buildings.Utilities.IO.Plotters. Fig. 9.3 shows an example in which this block is used to produce the sequence diagram shown in Fig. 9.4. While in this example, we used the control output of the CDL implementation, during commissioning, one would use the control signal from the building automation system. The model is available from the Modelica Buildings Library, see the model Buildings.Utilities.Plotters.Examples.SingleZoneVAVSupply_u.


Fig. 9.3 Modelica model that verifies the sequence diagram. On the left are the blocks that generate the control input. In a real verification, these would be replaced with a file reader that reads data that have been archived by the building automation system. In the center is the control sequence implementation. Some of its output is converted to degree Celsius, and then fed to the plotters on the right that generate a scatter plot for the temperatures and a scatter plot for the fan control signal. The block labeled plotConfiguration configures the file name for the plots and the sampling interval.


Fig. 9.4 Control sequence diagram for the VAV single zone control sequence from ASHRAE Guideline 36.

Simulating the model shown in Fig. 9.3 generates an html file that contains the scatter plots shown in Fig. 9.5.


Fig. 9.5 Scatter plots that show the control sequence diagram generated from the simulated sequence.

9.4. Example

In this example we validated a trended output of a control sequence that defines the cooling coil valve position. The cooling coil valve sequence is a part of the ALC EIKON control logic implemented in building 33 on the main LBNL campus in Berkeley, CA. The subsequence is shown in Fig. 9.6. It comprises a PI controller that tracks the supply air temperature, an upstream subsequence that enables the controller and a downstream output limiter that is active in case of low supply air temperatures.


Fig. 9.6 ALC EIKON specification of the cooling coil valve position control sequence.


Fig. 9.7 CDL specification of the cooling coil valve position control sequence.

We created a CDL specification of the same cooling coil valve position control sequence, see Fig. 9.7, to validate the recorded output. We recorded trend data in 5 second intervals for

  • Supply air temperature in [F]
  • Supply air temperature setpoint in [F]
  • Outdoor air temperature in [F]
  • VFD fan enable status in [0/1]
  • VFD fan feedback in [%]
  • Cooling coil valve position, which is the output of the controller, in [%].

The input and output trends were processed with a script that converts them to the format required by the data readers. The data used in the example begins at midnight on June 7 2018. In addition to the input and output trends, we recorded all parameters, such as the hysteresis offset (see Fig. 9.8) and the controller gains (see Fig. 9.9), to use them in the CDL implementation.


Fig. 9.8 ALC EIKON outdoor air temperature hysteresis to enable/disable the controller


Fig. 9.9 ALC EIKON PI controller parameters

We configured the CDL PID controller parameters such that they correspond to the parameters of the ALC PI controller. The ALC PID controller implementation is described in the ALC EIKON software help section, while the CDL PID controller is described in the info section of the model Buildings.Controls.OBC.CDL.Continuous.LimPID. The ALC controller tracks the temperature in degree Fahrenheit, while CDL uses SI units. An additional implementation difference is that for cooling applications, the ALC controller uses direct control action, whereas the CDL controller needs to be configured to use reverse control action, which can be done by setting its parameter reverseAction=true. Furthermore, the ALC controller outputs the control action in percentages, while the CDL controller outputs a signal between \(0\) and \(1\). To reconcile the differences, the ALC controller gains were converted for CDL as follows: The proportional gain \(k_{p,cdl}\) was set to \(k_{p,cdl} = u \, k_{p,alc}\), where \(u=9/5\) is a ratio of one degree Celsius (or Kelvin) to one degree Fahrenheit of temperature difference. The integrator time constant was converted as \(T_{i,cdl} = k_{p,cdl} \, I_{alc}/(u \, k_{i,alc})\). Both controllers were enabled throughout the whole validation time.

Fig. 9.10 shows the Modelica model that was used to conduct the verification. On the left hand side are the data readers that read the input and output trends from files. Next are unit converters, and a conversion for the fan status between a real value and a boolean value. These data are fed into the instance labeled cooValSta, which contains the control sequence as shown in Fig. 9.7. The plotters on the right hand side then compare the simulated cooling coil valve position with the recorded data.


Fig. 9.10 Modelica model that conducts the verification.

Fig. 9.11, which was produced by the Modelica model using blocks from the Buildings.Utilities.Plotters package, shows the trended input temperatures for the control sequence, the trended and simulated cooling valve control signal for the same time period, which are practically on top of each other, and a correlation error between the trended and simulated cooling valve control signal.


Fig. 9.11 Verification of the cooling valve control signal between ALC EIKON computed signal and simulated signal.

The difference in modeled vs. trended results is due to the following factors:

  • ALC EIKON uses a discrete time step for the time integration with a user-defined time step length, whereas CDL uses a continuous time integrator that adjusts the time step based on the integration error.
  • ALC EIKON uses a proprietary algorithm for the anti-windup, which differs from the one used in the CDL implementation.

Fig. 9.12 Verification of the cooling valve control signal with the funnel software (error computed with an absolute tolerance in time of 1 s and a relative tolerance in y of 1%).

Despite these differences, the computed and the simulated control signals show good agreement, which is also demonstrated by verifying the time series with the funnel software, whose output is shown in Fig. 9.12.