As I mentioned in my previous post, I really like using Windows Performance Counters in automated tests to gain insights into applications. It is not difficult to expose custom Performance Counters from your own application to report various metrics. But what about testing these custom counters? Like any other component of the application, custom counters should have automated tests to ensure they are reporting the correct values and to prevent regressions during further development of the application.

Some Performance Counter types can be quite challenging to test, yet others can be quite simple. In this post I’ll describe how to write automated tests for some of the more common counter types used for custom Performance Counters. All of the tests assume that they are running against the application in isolation in order to make the results deterministic.

NumberOfItems64

This counter is the most basic. It reports the most recently observed raw value and no calculation is performed. It can be used for maintaining a simple count, like the number of records in a table or the number of operations performed.

This counter can be tested by sampling the initial value at the start of the test, performing a deterministic number of operations, and then asserting that the counter incremented by the number of operations. Suppose that we are developing a database application that has a custom counter that reports the number of records in the database (e.g., Record Count). We can write the following C# test.

[TestMethod]
public void RecordCountPerformanceCounterReportsCorrectValue()
{
    var counter = new PerformanceCounter("OurCustomDatabase", "Record Count");

    // NextValue will get the current value
    var initialRecordCount = counter.NextValue();

    // Insert a deterministic number of records into the database
    const int recordCount = 1000;
    InsertRecords(recordCount);

    // NextValue will get the current value
    var finalRecordCount = counter.NextValue();

    // Assert that the counter incremented by the correct count
    Assert.AreEqual(recordCount, finalRecordCount - initialRecordCount);
}

RateOfCountsPerSecond64

This counter reports the average rate between two samples. Consider that our database application has a custom counter that reports the number of records inserted per second (e.g., Record Insertions/sec). How can we write an automated test to verify the correct behavior of this custom counter?

One way would be to insert records into the database and use the technique demonstrated previously to measure the calculated value for this counter across the test. But what should the calculated value of this counter be? The only thing that we can assert reliably is that the calculated value is greater than 0. We could write the test such that we attempt to insert the records at some target rate and then assert that the calculated value for the counter matches this rate plus or minus some tolerance. This, however, makes the test nondeterministic and a lot more complicated.

Performance Counters that report a rate are implemented by simply incrementing a value in shared memory. Recall that the rate is calculated by the client only when the counter is sampled. This makes counters very lightweight. The client uses the value at the start of the interval, the value at the end of the interval, and the length of the interval, to compute the rate. Therefore, we can write a completely deterministic test by ignoring the calculated value of the counter, using the raw value instead.

[TestMethod]
public void DatabaseInsertionsPerSecondPerformanceCounterReportsCorrectValue()
{
   var counter = new PerformanceCounter("OurCustomDatabase", "Record Insertions/sec");

   // Get the raw value of the counter before the test
   var initialInsertCount = counter.RawValue;

   // Insert a deterministic number of records into the database
   const int recordCount = 1000;
   InsertDatabaseRecords(recordCount);

   // Get the raw value of the counter after the test
   var finalInsertCount = counter.RawValue;

   // Assert that the counter was incremented by the number of records inserted
   Assert.AreEqual(recordCount, finalInsertCount - initialInsertCount);
}

This is not only a robust test for the implementation of the custom counter itself (i.e., if I insert 1000 records, the raw counter value should increment by 1000 and I'll rely on Windows to calculate the correct rate), the same technique could also be used in other tests to assert that the system has been arranged in the expected state before initiating the test (i.e., to assert that all the records have been inserted rather than residing in a queue waiting to be processed).

AverageCount64

This counter reports an average across the sample interval. It is calculated by dividing the difference in the count by the number of operations performed during the sample interval. Consider that our database application has a custom counter that reports the average record size in bytes for newly inserted records (e.g., Average Record Size). In this case, we can independently calculate the expected average and compare it to the sampled average.

[TestMethod]
public void AverageRecordSizeInBytesPerformanceCounterReportsCorrectValue()
{
    var counter = new PerformanceCounter("OurCustomDatabase", "Average Record Size");

    // Call NextValue once to initialize the counter
    counter.NextValue();

    // Insert a deterministic number of randomly-sized records into the database
    const int recordCount = 1000;
    int bytesInserted = InsertRandomSizedRecords(recordCount);

    // Calculate the expected average
    var expectedAvgRecordSize = (float)(bytesInserted / recordCount);

    // Sampling the performance counter again will calculate the average across the test
    var sampledAvgRecordSize = counter.NextValue();

    // Assert that the counter reports the correct average
    Assert.AreEqual(expectedAvgRecordSize, sampledAvgRecordSize);
}

Given what we know about how counters are implemented, however, we could also choose to test this counter by sampling the raw counter values as we did above for the rate counter.

[TestMethod]
public void AverageRecordSizeInBytesPerformanceCounterReportsCorrectValue()
{
    var counter = new PerformanceCounter("OurCustomDatabase", "Average Record Size");

    // Call NextValue once to initialize the counter
    var initialSample = counter.NextSample();

    // Insert a deterministic number of randomly-sized records into the database
    const int recordCount = 1000;
    float bytesInserted = InsertRandomSizedRecords(recordCount);

    // Sampling the performance counter again will calculate the average across the test
    var finalSample = counter.NextSample();

    // Assert that the expected number operations were performed
    Assert.AreEqual(recordCount, finalSample.BaseValue - initialSample.BaseValue);

    // Assert that the counter incremented by the expected number of bytes
    Assert.AreEqual(bytesInserted, finalSample.RawValue - initialSample.RawValue);
}

Conclusion

In this post I’ve given examples of deterministic automated tests for the most common Windows Performance Counters types. Testing some of the more complex counter types can be more challenging because they are less deterministic. I may cover some of these in a future post.