Testing

This article is part of the GoldSim Style Guide. For an introduction, please start here.

You should include tests in the model that ensure it is producing expected outputs. Typically, these tests are run during and after construction of the model. You can store the test logic and results in a top-level Container called, “Testing” or include this within the component Containers of the system. The benefit to storing these calculations in a top-level container is that you can remove them when you put the model into production mode. Some examples of tests are presented in this section to provide you with some ideas, but you should determine what tests are most suitable for you.

Verification

Verification testing is the process of evaluating specific functionality of the model to determine whether it operates as designed. Typically, the test will compare the output the model produces to the output from an external source, such as documentation or output from a 3^rd party tool. It is recommended that these tests focus on a small portion of the model logic. Like the other tests, these can be organized in a top-level Container.

The Script element can log messages and warnings to the run log of the model. This is a good way to document test results during the testing phase of the model. There are three types of Log statements.  All three can be inserted by selecting the appropriate type from the Script element’s Insert menu (or pressing the appropriate hot-key when in the Script dialog):

When a Log statement is inserted in a Script, the following dialog will be displayed:

Note that at the bottom of the dialog you can modify the type of Log statement (Message, Warning or Error).  If you check the Show Message checkbox, the message will also appear in a pop-up window during the simulation whenever the statement is executed.

One of the most powerful features of the Log statement is that you can write the value of a variable, output or expression in the message. To include a scalar value in the log, type {=VALUE}, where VALUE is the output, variable or expression whose value you wish to display.

For example, assume you had a script variable named X, and an external output (an output from outside the Script that was referenced inside the Script) named Y.  A Log message statement like this:

would look like this in the script:

The message that would be displayed in a pop-up would look like this:

In the Run Log, it would look like this:

If the variables for which you are writing values have units, the units will be displayed.  For example, if X was a length, this would be indicated in the message:

However, you cannot control the units in which they are displayed.  They are always displayed in the base units for the dimension (e.g., meters for lengths, seconds for time).

The Script element is a good tool to help with testing of the model. Should you want to disable the testing, you could put the statements inside an IF statement that is controlled from a Data element at the top level of the model. For example, you could have a Data element that is set to True or False called “Testing_ON”. If true, then all the tests will be evaluated in the Script element. Otherwise, they are skipped.

Mass Balance

In GoldSim models that represent a system of material flows, it is critical that mass balance testing be performed for the system. This will provide a way to verify that the model accounts for gains and losses in the system. We recommend that you add a Container at the top level of the model specifically to hold the mass balance test calculations and results. An example model that calculates closure of mass balance can be found in our library, here.

It is also recommended that you include a mass balance check for smaller portions of the overall system. The reason for this is because the mass balance check over the entire system will not tell you where the problem is, should one occur. A test on small portions of the model will point you to the specific portion that has the problem.

If your model is organized in the manner described in previous sections, then linking the necessary flows and amounts for a mass balance test is straightforward.

Calibration

Model calibration is very important, especially for models that are intended to simulate existing conditions. It is a way to validate the logic in the model and provide a higher level of confidence that it will provide realistic forecasts. It is recommended that a top-level container be added to the model, which includes the calibration information, such as measured outputs that the model is expected to match along with a result plot to show how well the simulated vs measured output match.

You can rely on GoldSim’s optimization functionality to test how close a simulated time history matches measured time series data. Examples of this approach can be found in our library here:

• Model Calibration Using Optimization
• Webinar Archive: Calibrating Your Model
• Root Mean Square Error (RMSE)
• Calibrate a Snowmelt Runoff Model

NEXT -->