.. _testing: Testing Framework ***************** NetSpyGlass has simple built-in testing framework that can be used to write unit tests for Python hooks or simply run test scripts. Here is how to do this. - First, create directory somewhere on the file system where you are going to store your test scripts. This directory does not have to be (and probably should not be) inside of the NetSpyGlass directories. Lets assume this directory is `/home/vadim/src/nsg/tests`. - in this directory create test file. This file is basically standard Python unit test script, that is, it should have the name that starts with `test_` and should define a class that subclasses class `TestCase`. We are going to use the following test case as an example, saved in the file `test_alert_4.py`:: __author__ = 'vadim' import unittest from nw2functions import * import utils class TestAlert4(unittest.TestCase): def setUp(self): super(TestAlert4, self).setUp() self.ctx = utils.setup_test(60) def make_mvar(self): mvar1 = utils.make_monitoring_variable( self.ctx, var_name='testVar', dev_name='test1', dev_id=1, component_name='comp1', index=1) mvar1.addTag('BGP4PeerAddress.10.0.0.1') mvar1.addTag('Explicit.foo') mvar1.addTag('Vendor.Juniper') mvar1.addTag('Role.Router') mvar1.addTag('BGP4Peer.AS1000') return mvar1 def test_1(self): mvar1 = self.make_mvar() # add observation to the variable with value 0, try alert current_time = 1418269740000 utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 0)]) res = alert( name='test_alert', input=[mvar1], condition=lambda x: x > 1, description='value is greater than 1', details={}, notification_time=300, fan_out=True ) # alert did not fire self.assertEqual(res, []) # add another observation and try alart again current_time += 60000 utils.add_to_timeseries(mvar1.timeseries, ['{0},{1}'.format(current_time, 2)]) res = alert( name='test_alert', input=[mvar1], condition=lambda x: x > 1, description='value is greater than 1', details={}, notification_time=300, fan_out=True ) # this time alert fires self.assertEqual(len(res), 1) - you need to include NetSpyGlass functions modules `nw2functions` and `utils` in your test. - Test case `SetUp` function should call `utils.setup_test()` with polling interval value (seconds) as a parameter. This function returns `context` object that we'll need later. Context has many fields, one of which is a reference to the data pool object. All monitoring variables should be added to the data pool by calling `self.ctx.data_pool.addVariable("name", mvar)`, this makes it possible to call `import_var()` and `export_var()` somewhere in the test. .. note:: Polling interval value passed as a parameter to :py:func:`utils.setup_test()` must match time stamp step used in the test data in the unit test. - function `utils.make_monitoring_variable()` from module `utils` that comes with NetSpyGlass wraps several steps necessary to create new monitoring variable object. - We can use other methods of the `net.happygears.nw2.py.MonitoringVariable` object to set up tags and other attributes needed for the test. In the example above we add some tags. - actual test consists of sequences where we add an observation to the variable and try to call `alert()`. This function returns list of active alerts it creates, if any. This means if returned list is empty, we have no active alerts and if the list is not empty, its items are active alerts. You can always call `str(alert)` to "print" the alert and then compare result with expected strings. .. note:: Every time we add an observation to the time series, we should increment its time stamp and also set current time in the global context object by calling `ctx.setCurrentTimeStamp()`. Some operations with alerts and monitoring variables compare time stamps to current time but to make unit tests predictable, they take current time from the context object rather than from the system clock. - function `utils.add_to_timeseries()` can be used to add observations to the variable time series. Typical operation of adding test data to a variable looks like this:: utils.add_to_timeseries( mvar1.timeseries, [ '1418269740000,1', '1418269800000,2', '1418269860000,3', '1418269920000,4', '1418269980000,5', '1418270040000,6', '1418270100000,7', '1418270160000,8' ]) Running tests ============= Once the test file has been created and saved, simply enter the directory where it was saved and run script `tester.sh` that comes with NetSpyGlass package:: $ cd src/nsg/tests $ /opt/netspyglass/current/bin/tester.sh 2015-06-13 21:12:31,459 INFO [Tester ] Making new Python interpreter 2015-06-13 21:12:34,461 INFO [Tester ] Running tests in /Users/vadim/src/nsg/tests test_big_change_alert_1 (test_1.TestAlert4) ... ok ---------------------------------------------------------------------- Ran 1 tests in 0.235s OK Utility functions ================= Module `utils` provides several functions useful in unit tests .. automodule:: utils :members: setup_test, make_monitoring_variable, make_monitoring_variable_2, add_to_timeseries, ts_to_strings Context object ============== Context object returned by call to :py:func:`utils.setup_test()` exposes several attributes that are useful in building tests. .. py:attribute:: devices Device repository. You can add devices to it by calling :py:func:`add()` and then retrieve them by their device id using `[]`:: node = NetworkNode() node.setHostName('test4') node.setId(4) node.addTag('Vendor.Juniper') ctx.devices.add(node) # somewhere else node4 = ctx.devices[4] .. py:attribute:: current_time Simply "current system time" for the test. The value is set automatically when you call `utils.add_to_timeseries()` but you can set it explicitly by calling `self.ctx.setCurrentTimeStamp()` and then retrieve using attribute `current_time`. The time is in milliseconds. .. py:attribute:: polling_interval polling interval (seconds). This is the value that was passed as an argument to the call `utils.setup_test()` Alert object ============ Function :func:`nw2functions.alert()` returns a list of instances of class :class:`net.happygears.nw2.alerts.Alert` that represent triggered alerts (if any). Normally, you dont need to do anything with these objects and can just ignore the returned value. However this is useful in the tests because it allows you to verify not only that the call to :func:`nw2functions.alert()` actually has activated alerts, but also to check fields of the created alerts, including expanded macros. To be able to do this, you'll need to include Java package that provides class :class:`net.happygears.nw2.alerts.Alert` into your unit test module :: from net.happygears.nw2.alerts import Alert .. _examples_of_tests: Examples of Unit Tests ====================== .. _unit_test_alert_1: Test Alert and its Fields ------------------------- The following test creates an alert and verifies that it activates given specific input variable and condition function. The test also verifies all the fields in the created Alert object:: import unittest from net.happygears.nw2.py import MonitoringDataType from net.happygears.nw2.alerts import Alert from nw2functions import * import utils class TestAlert(unittest.TestCase): def setUp(self): super(TestAlert, self).setUp() self.ctx = utils.setup_test(60) def make_mvar(self): mvar1 = utils.make_monitoring_variable( self.ctx, var_name='testVar', dev_name='test1', dev_id=1, index=1, component_name='comp1') mvar1.getDS().setDataFormat(MonitoringDataType.Counter64) return mvar1 def test_alert_fan_out_1(self): mvar1 = self.make_mvar() data = ['1418269740000,1', '1418269800000,2'] utils.add_to_timeseries(mvar1.timeseries, data) res = alert( name='test_alert', input=[mvar1], condition=lambda x: x > 1, description='value is greater than 1', details={}, fan_out=True ) # this time alert fires self.assertEqual(len(res), 1) alert_obj = res[0] assert isinstance(alert_obj, Alert) actual_json = alert_obj.toJson(True) expected_json = '''{ "key" : "057e4b749f31c1c5baea83a580a872c7", "name" : "test_alert", "deviceName" : "test1", "deviceId" : 1, "componentName" : "comp1", "componentIndex" : 1, "value" : "2.0", "description" : "value is greater than 1", "tags" : [ ], "variable" : "test_alert.1.1", "inputVariable" : "testVar.1.1", "activeSince" : 1418269800000, "active" : true, "silenced" : false, "matchingSilenceId" : 0, "timeLastNotificationSent" : 0, "updatedAt" : 1418269800000, "duration" : 0.0, "percentage" : 100.0, "notificationTimeMs" : 0.0, "fanout" : true }''' self.assertEqual(actual_json, expected_json, actual_json)