12. Testing Framework¶
NetSpyGlass has simple built-in testing framework that can be used to write unit tests for Python hooks or simply run test scripts. Here is how to do this.
First, create directory somewhere on the file system where you are going to store your test scripts. This directory does not have to be (and probably should not be) inside of the NetSpyGlass directories. Lets assume this directory is /home/vadim/src/nsg/tests.
in this directory create test file. This file is basically standard Python unit test script, that is, it should have the name that starts with test_ and should define a class that subclasses class TestCase. We are going to use the following test case as an example, saved in the file test_alert_4.py:
__author__ = 'vadim' import unittest from nw2functions import * import utils class TestAlert4(unittest.TestCase): def setUp(self): super(TestAlert4, self).setUp() self.ctx = utils.setup_test(60) def make_mvar(self): mvar1 = utils.make_monitoring_variable( self.ctx, var_name='testVar', dev_name='test1', dev_id=1, component_name='comp1', index=1) mvar1.addTag('BGP4PeerAddress.10.0.0.1') mvar1.addTag('Explicit.foo') mvar1.addTag('Vendor.Juniper') mvar1.addTag('Role.Router') mvar1.addTag('BGP4Peer.AS1000') return mvar1 def test_1(self): mvar1 = self.make_mvar() # add observation to the variable with value 0, try alert current_time = 1418269740000 utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 0)]) res = alert( name='test_alert', input=[mvar1], condition=lambda x: x > 1, description='value is greater than 1', details={}, notification_time=300, fan_out=True ) # alert did not fire self.assertEqual(res, []) # add another observation and try alart again current_time += 60000 utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 2)]) res = alert( name='test_alert', input=[mvar1], condition=lambda x: x > 1, description='value is greater than 1', details={}, notification_time=300, fan_out=True ) # this time alert fires self.assertEqual(len(res), 1)you need to include NetSpyGlass functions modules nw2functions and utils in your test.
Test case SetUp function should call utils.setup_test() with polling interval value (seconds) as a parameter. This function returns context object that we’ll need later. Context has many fields, one of which is a reference to the data pool object. All monitoring variables should be added to the data pool by calling self.ctx.data_pool.addVariable(“name”, mvar), this makes it possible to call import_var() and export_var() somewhere in the test.
Note
Polling interval value passed as a parameter to
utils.setup_test()
must match time stamp step used in the test data in the unit test.function utils.make_monitoring_variable() from module utils that comes with NetSpyGlass wraps several steps necessary to create new monitoring variable object.
We can use other methods of the net.happygears.nw2.py.MonitoringVariable object to set up tags and other attributes needed for the test. In the example above we add some tags.
actual test consists of sequences where we add an observation to the variable and try to call alert(). This function returns list of active alerts it creates, if any. This means if returned list is empty, we have no active alerts and if the list is not empty, its items are active alerts. You can always call str(alert) to “print” the alert and then compare result with expected strings.
Note
Every time we add an observation to the time series, we should increment its time stamp and also set current time in the global context object by calling ctx.setCurrentTimeStamp(). Some operations with alerts and monitoring variables compare time stamps to current time but to make unit tests predictable, they take current time from the context object rather than from the system clock.
function utils.add_to_timeseries() can be used to add observations to the variable time series. Typical operation of adding test data to a variable looks like this:
utils.add_to_timeseries( mvar1.timeseries, [ '1418269740000,1', '1418269800000,2', '1418269860000,3', '1418269920000,4', '1418269980000,5', '1418270040000,6', '1418270100000,7', '1418270160000,8' ])
12.1. Running tests¶
Once the test file has been created and saved, simply enter the directory where it was saved and run script tester.sh that comes with NetSpyGlass package:
$ cd src/nsg/tests
$ /opt/netspyglass/current/bin/tester.sh
2015-06-13 21:12:31,459 INFO [Tester ] Making new Python interpreter
2015-06-13 21:12:34,461 INFO [Tester ] Running tests in /Users/vadim/src/nsg/tests
test_big_change_alert_1 (test_1.TestAlert4) ... ok
----------------------------------------------------------------------
Ran 1 tests in 0.235s
OK
12.2. Utility functions¶
Module utils provides several functions useful in unit tests
Copyright (C) 2014 HappyGears - All Rights Reserved
-
utils.
setup_test
(polling_interval_sec)¶ Test case set up function
Parameters: polling_interval_sec – polling interval, sec Returns: context object
-
utils.
make_monitoring_variable
(ctx, var_name, dev_name, dev_id, component_name, index)¶ Make test MonitoringVariable object with given set of parameters
Parameters: - ctx – RuleRunnerContext
- var_name – variable name
- dev_name – device name
- dev_id – device id
- component_name – component name
- index – component index
Returns: MonitoringVariable object
-
utils.
make_monitoring_variable_2
(ctx, var_name, dev_name, dev_id, index, component_name, if_speed)¶ Make test MonitoringVariable object with given set of parameters
Parameters: - ctx – RuleRunnerContext
- var_name – variable name
- dev_name – device name
- dev_id – device id
- index – component index (interface ifIndex)
- component_name – component name (interface name)
- if_speed – interface speed
Returns: MonitoringVariable object
-
utils.
add_to_timeseries
(timeseries, str_observations)¶ take list of strings, convert it to list of Observations and add them to the time series of given monitoring variable
Parameters: - timeseries – TimeSeriesBuffer object
- str_observations – list of strings
-
utils.
ts_to_strings
(timeseries)¶ convert list of Observation objects to list of strings
Parameters: timeseries – list of Observation objects Returns: list of strings
12.3. Context object¶
Context object returned by call to utils.setup_test()
exposes several attributes that are useful in
building tests.
utils.
devices
¶Device repository. You can add devices to it by calling
add()
and then retrieve them by their device id using []:node = NetworkNode() node.setHostName('test4') node.setId(4) node.addTag('Vendor.Juniper') ctx.devices.add(node) # somewhere else node4 = ctx.devices[4]
utils.
current_time
¶Simply “current system time” for the test. The value is set automatically when you call utils.add_to_timeseries() but you can set it explicitly by calling self.ctx.setCurrentTimeStamp() and then retrieve using attribute current_time. The time is in milliseconds.
utils.
polling_interval
¶polling interval (seconds). This is the value that was passed as an argument to the call utils.setup_test()
12.4. Alert object¶
Function nw2functions.alert()
returns a list of instances of class
net.happygears.nw2.alerts.Alert
that represent triggered alerts (if any). Normally, you dont need
to do anything with these objects and can just ignore the returned value. However this is useful in the tests
because it allows you to verify not only that the call to nw2functions.alert()
actually has activated
alerts, but also to check fields of the created alerts, including expanded macros. To be able to do this, you’ll
need to include Java package that provides class net.happygears.nw2.alerts.Alert
into your unit
test module
from net.happygears.nw2.alerts import Alert
12.5. Examples of Unit Tests¶
12.5.1. Test Alert and its Fields¶
The following test creates an alert and verifies that it activates given specific input variable and condition function. The test also verifies all the fields in the created Alert object:
import unittest
from net.happygears.nw2.py import MonitoringDataType
from net.happygears.nw2.alerts import Alert
from nw2functions import *
import utils
class TestAlert(unittest.TestCase):
def setUp(self):
super(TestAlert, self).setUp()
self.ctx = utils.setup_test(60)
def make_mvar(self):
mvar1 = utils.make_monitoring_variable(
self.ctx, var_name='testVar', dev_name='test1', dev_id=1, index=1, component_name='comp1')
mvar1.getDS().setDataFormat(MonitoringDataType.Counter64)
return mvar1
def test_alert_fan_out_1(self):
mvar1 = self.make_mvar()
data = ['1418269740000,1', '1418269800000,2']
utils.add_to_timeseries(mvar1.timeseries, data)
res = alert(
name='test_alert',
input=[mvar1],
condition=lambda x: x > 1,
description='value is greater than 1',
details={},
fan_out=True
)
# this time alert fires
self.assertEqual(len(res), 1)
alert_obj = res[0]
assert isinstance(alert_obj, Alert)
actual_json = alert_obj.toJson(True)
expected_json = '''{
"key" : "057e4b749f31c1c5baea83a580a872c7",
"name" : "test_alert",
"deviceName" : "test1",
"deviceId" : 1,
"componentName" : "comp1",
"componentIndex" : 1,
"value" : "2.0",
"description" : "value is greater than 1",
"tags" : [ ],
"variable" : "test_alert.1.1",
"inputVariable" : "testVar.1.1",
"activeSince" : 1418269800000,
"active" : true,
"silenced" : false,
"matchingSilenceId" : 0,
"timeLastNotificationSent" : 0,
"updatedAt" : 1418269800000,
"duration" : 0.0,
"percentage" : 100.0,
"notificationTimeMs" : 0.0,
"fanout" : true
}'''
self.assertEqual(actual_json, expected_json, actual_json)