12. Testing Framework

NetSpyGlass has simple built-in testing framework that can be used to write unit tests for Python hooks or simply run test scripts. Here is how to do this.

  • First, create directory somewhere on the file system where you are going to store your test scripts. This directory does not have to be (and probably should not be) inside of the NetSpyGlass directories. Lets assume this directory is /home/vadim/src/nsg/tests.

  • in this directory create test file. This file is basically standard Python unit test script, that is, it should have the name that starts with test_ and should define a class that subclasses class TestCase. We are going to use the following test case as an example, saved in the file test_alert_4.py:

    __author__ = 'vadim'
    
    import unittest
    
    from nw2functions import *
    import utils
    
    
    class TestAlert4(unittest.TestCase):
    
        def setUp(self):
            super(TestAlert4, self).setUp()
            self.ctx = utils.setup_test(60)
    
        def make_mvar(self):
            mvar1 = utils.make_monitoring_variable(
                self.ctx, var_name='testVar', dev_name='test1', dev_id=1, component_name='comp1', index=1)
            mvar1.addTag('BGP4PeerAddress.10.0.0.1')
            mvar1.addTag('Explicit.foo')
            mvar1.addTag('Vendor.Juniper')
            mvar1.addTag('Role.Router')
            mvar1.addTag('BGP4Peer.AS1000')
            return mvar1
    
        def test_1(self):
    
            mvar1 = self.make_mvar()
    
            # add observation to the variable with value 0, try alert
            current_time = 1418269740000
            utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 0)])
    
            res = alert(
                name='test_alert',
                input=[mvar1],
                condition=lambda x: x > 1,
                description='value is greater than 1',
                details={},
                notification_time=300,
                fan_out=True
            )
    
            # alert did not fire
            self.assertEqual(res, [])
    
            # add another observation and try alart again
            current_time += 60000
            utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 2)])
    
            res = alert(
                name='test_alert',
                input=[mvar1],
                condition=lambda x: x > 1,
                description='value is greater than 1',
                details={},
                notification_time=300,
                fan_out=True
            )
    
            # this time alert fires
            self.assertEqual(len(res), 1)
    
  • you need to include NetSpyGlass functions modules nw2functions and utils in your test.

  • Test case SetUp function should call utils.setup_test() with polling interval value (seconds) as a parameter. This function returns context object that we’ll need later. Context has many fields, one of which is a reference to the data pool object. All monitoring variables should be added to the data pool by calling self.ctx.data_pool.addVariable(“name”, mvar), this makes it possible to call import_var() and export_var() somewhere in the test.

    Note

    Polling interval value passed as a parameter to utils.setup_test() must match time stamp step used in the test data in the unit test.

  • function utils.make_monitoring_variable() from module utils that comes with NetSpyGlass wraps several steps necessary to create new monitoring variable object.

  • We can use other methods of the net.happygears.nw2.py.MonitoringVariable object to set up tags and other attributes needed for the test. In the example above we add some tags.

  • actual test consists of sequences where we add an observation to the variable and try to call alert(). This function returns list of active alerts it creates, if any. This means if returned list is empty, we have no active alerts and if the list is not empty, its items are active alerts. You can always call str(alert) to “print” the alert and then compare result with expected strings.

    Note

    Every time we add an observation to the time series, we should increment its time stamp and also set current time in the global context object by calling ctx.setCurrentTimeStamp(). Some operations with alerts and monitoring variables compare time stamps to current time but to make unit tests predictable, they take current time from the context object rather than from the system clock.

  • function utils.add_to_timeseries() can be used to add observations to the variable time series. Typical operation of adding test data to a variable looks like this:

    utils.add_to_timeseries(
        mvar1.timeseries,
        [
            '1418269740000,1',
            '1418269800000,2',
            '1418269860000,3',
            '1418269920000,4',
            '1418269980000,5',
            '1418270040000,6',
            '1418270100000,7',
            '1418270160000,8'
        ])
    

12.1. Running tests

Once the test file has been created and saved, simply enter the directory where it was saved and run script tester.sh that comes with NetSpyGlass package:

$ cd src/nsg/tests
$ /opt/netspyglass/current/bin/tester.sh

2015-06-13 21:12:31,459 INFO [Tester                    ] Making new Python interpreter
2015-06-13 21:12:34,461 INFO [Tester                    ] Running tests in /Users/vadim/src/nsg/tests
test_big_change_alert_1 (test_1.TestAlert4) ... ok

----------------------------------------------------------------------
Ran 1 tests in 0.235s

OK

12.2. Utility functions

Module utils provides several functions useful in unit tests

Copyright (C) 2014 HappyGears - All Rights Reserved

utils.setup_test(polling_interval_sec)

Test case set up function

Parameters:polling_interval_sec – polling interval, sec
Returns:context object
utils.make_monitoring_variable(ctx, var_name, dev_name, dev_id, component_name, index)

Make test MonitoringVariable object with given set of parameters

Parameters:
  • ctx – RuleRunnerContext
  • var_name – variable name
  • dev_name – device name
  • dev_id – device id
  • component_name – component name
  • index – component index
Returns:

MonitoringVariable object

utils.make_monitoring_variable_2(ctx, var_name, dev_name, dev_id, index, component_name, if_speed)

Make test MonitoringVariable object with given set of parameters

Parameters:
  • ctx – RuleRunnerContext
  • var_name – variable name
  • dev_name – device name
  • dev_id – device id
  • index – component index (interface ifIndex)
  • component_name – component name (interface name)
  • if_speed – interface speed
Returns:

MonitoringVariable object

utils.add_to_timeseries(timeseries, str_observations)

take list of strings, convert it to list of Observations and add them to the time series of given monitoring variable

Parameters:
  • timeseries – TimeSeriesBuffer object
  • str_observations – list of strings
utils.ts_to_strings(timeseries)

convert list of Observation objects to list of strings

Parameters:timeseries – list of Observation objects
Returns:list of strings

12.3. Context object

Context object returned by call to utils.setup_test() exposes several attributes that are useful in building tests.

utils.devices

Device repository. You can add devices to it by calling add() and then retrieve them by their device id using []:

node = NetworkNode()
node.setHostName('test4')
node.setId(4)
node.addTag('Vendor.Juniper')
ctx.devices.add(node)

# somewhere else
node4 = ctx.devices[4]
utils.current_time

Simply “current system time” for the test. The value is set automatically when you call utils.add_to_timeseries() but you can set it explicitly by calling self.ctx.setCurrentTimeStamp() and then retrieve using attribute current_time. The time is in milliseconds.

utils.polling_interval

polling interval (seconds). This is the value that was passed as an argument to the call utils.setup_test()

12.4. Alert object

Function nw2functions.alert() returns a list of instances of class net.happygears.nw2.alerts.Alert that represent triggered alerts (if any). Normally, you dont need to do anything with these objects and can just ignore the returned value. However this is useful in the tests because it allows you to verify not only that the call to nw2functions.alert() actually has activated alerts, but also to check fields of the created alerts, including expanded macros. To be able to do this, you’ll need to include Java package that provides class net.happygears.nw2.alerts.Alert into your unit test module

from net.happygears.nw2.alerts import Alert

12.5. Examples of Unit Tests

12.5.1. Test Alert and its Fields

The following test creates an alert and verifies that it activates given specific input variable and condition function. The test also verifies all the fields in the created Alert object:

import unittest

from net.happygears.nw2.py import MonitoringDataType
from net.happygears.nw2.alerts import Alert
from nw2functions import *
import utils


class TestAlert(unittest.TestCase):

    def setUp(self):
        super(TestAlert, self).setUp()
        self.ctx = utils.setup_test(60)

    def make_mvar(self):
        mvar1 = utils.make_monitoring_variable(
            self.ctx, var_name='testVar', dev_name='test1', dev_id=1, index=1, component_name='comp1')
        mvar1.getDS().setDataFormat(MonitoringDataType.Counter64)
        return mvar1

    def test_alert_fan_out_1(self):

        mvar1 = self.make_mvar()

        data = ['1418269740000,1', '1418269800000,2']
        utils.add_to_timeseries(mvar1.timeseries, data)

        res = alert(
            name='test_alert',
            input=[mvar1],
            condition=lambda x: x > 1,
            description='value is greater than 1',
            details={},
            fan_out=True
        )

        # this time alert fires
        self.assertEqual(len(res), 1)

        alert_obj = res[0]
        assert isinstance(alert_obj, Alert)

        actual_json = alert_obj.toJson(True)
        expected_json = '''{
  "key" : "057e4b749f31c1c5baea83a580a872c7",
  "name" : "test_alert",
  "deviceName" : "test1",
  "deviceId" : 1,
  "componentName" : "comp1",
  "componentIndex" : 1,
  "value" : "2.0",
  "description" : "value is greater than 1",
  "tags" : [ ],
  "variable" : "test_alert.1.1",
  "inputVariable" : "testVar.1.1",
  "activeSince" : 1418269800000,
  "active" : true,
  "silenced" : false,
  "matchingSilenceId" : 0,
  "timeLastNotificationSent" : 0,
  "updatedAt" : 1418269800000,
  "duration" : 0.0,
  "percentage" : 100.0,
  "notificationTimeMs" : 0.0,
  "fanout" : true
}'''
        self.assertEqual(actual_json, expected_json, actual_json)