4. Views

Views is a mechanism that allows the user to organize maps of a large network. Instead of always looking at a map that has 1000 devices, you can have multiple maps that show subset of the devices, grouped by some criteria that make sense to you. You can easily build maps to show OSPF areas or BGP peering or put devices in each data center into their own map and build an overview map to show how data centers are connected to each other.

Each view appears in the UI as a tile in the home page. If views are organized into a hierarchy, you can “dig down” by clicking objects that represent lower level views in maps. In any case, there is exact one-to-one correspondence between views and maps.

NetspyGlass offers programmatic mechanism you can use to match devices to views. This is done using Python script referenced by the parameter network.viewBuilder in the config file nw2.conf. Here is how this is done.

4.1. Building Your Own Views

To start, find file doc/views_example.py (this file is part of the distribution tar package) and copy it under different name to the same directory where you keep your configuration file nw2.conf. The name of the file can be anything; we recommend views.py and will use it in this documentation.

Note

Python hook scripts, such as views.py, should be readable by the user account NetSpyGlass server is running under. When you install NetSpyGlass using distributed rpm or deb package, this is user nw2.

Open the new file you just created in your favorite editor. The script consists of the following parts:

  • few global functions used to illustrate different ways to match device attributes
  • list VIEWS that actually defines views
  • class UserViewBuilder that serves as an API between this script and NetSpyGlass

Comments in the script views_example.py document its functions.

To activate your own view builder script, add configuration parameter viewBuilder inside of the network section in the nw2.conf configuration file as follows:

1
2
3
network {

    viewBuilder = "views.UserViewBuilder"

The value consists of the script name without extension “.py” and the name of the class defined inside.

When NetSpyGlass wants to create views, it loads this script, creates an instance of class UserViewBuilder defined in this script and calls its function execute(), which should return list of instances of the Python class View.

Default implementation of class UserViewBuilder creates View objects from Python dictionaries defined as items in the list VIEWS. This is just a reference implementation, you can change it if you prefer different way of doing it. The only thing that really matters is that there should be class with method execute() that returns list of View objects, everything elase is up to you. Function execute() will be called with one argument, a list of PyDevice objects.

Each View object has the following attributes:

  • name (string)
    the name of the view
  • parent (string)
    The name of the parent view. This is optional, views that have no parent parameter are the top-level views
  • match_function a function of one argument.
    This function is called with PyDevice object representing a device. This function should return True if the device should be a member of the view. This attribute is optional, if it is missing, the view matches all devices.
  • intf_match_function a function of two arguments.
    It is called with PyDevice object representing a device and PyNetworkInterface object representing one of its interfaces. This function should return True if the interface should be a member of the view, that is, link that terminates on the interface should appear on the map. This attribute is optional, if it is missing, the view includes all interfaces of matching devices.
  • adjacent_devices (boolean)
    if True, the view will also include devices one hop away from the devices that match this view. If this parameter is missing, its value is set to False
  • connecting_devices (boolean)
    if True, the view will also include devices that connect devices that match this view. Default value is False.

The match_function parameter should be a function that is going to be called with an instance of Python class net.happygears.nw2.py.py_wrappers.PyDevice.PyDevice. This function can analyse device name, address or tags to decide if it should be a member of the view.

Parameter intf_match_function is a function that is passed two arguments: an object net.happygears.nw2.py.py_wrappers.PyDevice that represents device and object net.happygears.nw2.py.py_wrappers.PyNetworkInterface that represents one of its interfaces. A network link appears in the map only if this function returned True for device/interface pairs on both ends of the link.

NetSpyGlass uses parameters of the View object as follows:

  1. all devices are passed to the match_function that decides which ones should appear in the map. If View does not have parameter match_function, then all devices match.
  2. if parameters adjacent_devices or connecting_devices have value True, corresponding adjacent and connecting devices are added too.
  3. next, links between already selected devices are checked using function referred to by the parameter intf_match_function. For each side of the link, NetSpyGlass calls function intf_match_function with two arguments: object that represents device and object that represents its interface on the corresponding side of the link. The link is added to the map if at least one end satisfies criteria defined by function intf_match_function. If View object does not have parameter intf_match_function, all links between devices selected by steps 1 and 2 are added to the map.

Examples in Examples of Views and in the script doc/views_example.py demonstrate different ways to match host attributes name, address and tags.

You can use regular expressions (via Python module re) to match host name and tags. You can also use Python module ipaddr to match IP addresses instead of operating with addresses as strings. Embedded Python interpreter comes with almost full set of standard Python modules, all of them are available to you in the view builder script.

To build your own views, you should start with editing list VIEWS. Remove or edit example views you do not need and add your own. We recommend that you create a global function for each view, to be used as its match_function parameter.

When view builder script is ready and saved to the file, edit your configuration file nw2.conf and add parameter viewBuilder inside of the network block. It should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
network {
    name = "Test Network"

    devices = [
        { address = "10.0.14.1",        snmpCommunity = public,  },
        { address = "10.0.14.15",       snmpCommunity = public   },
        { address = "10.0.14.20",       snmpCommunity = public,  },

        # ... records for more devices go here ...
    ]

    viewBuilder = "views.UserViewBuilder"

Note that this parameter refers to the Python class defined in the script views.py. Restart NetSpyGlass when you add or change viewBuilder parameter.

NetSpyGlass monitors view builder script defined by the parameter viewBuilder and reloads views whenever it changes. You do not need to restart the system every time you edit this script. It usually takes a couple of seconds to rebuild views and they should appear in your browser right away.

When the system loads the script and creates views, it writes lots of information to the info and error logs. Watch logs logs/info.log and logs/error.log for errors. Python exceptions appear in both log files and are annotated with the file name and line number.

4.2. Examples of Views

the following script creates three views: “all” (includes all devices), “Backbone” (includes devices that are members of OSPF area 0) and “Peering” (includes only devices that have BGP peering with other ASes than their own, that is, run eBGP):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import view

def match_ospf_area_0(host):
    return 'OSPFArea.0' in host.tags


def match_bgp_speaker(host):
    return 'Role.eBgpPeer' in host.tags


VIEWS = [
    view.View(name='all'),

    view.View(
        name='backbone',
        match_function=match_ospf_area_0,
    ),

    view.View(
        name='Peering',
        match_function=match_bgp_speaker,
    ),

]


class UserViewBuilder(object):
    def __init__(self, log):
        self.log = log

    def execute(self):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :return:  list of View objects
            """
            return VIEWS

Here is an example that shows how to build a map for specific vlan (for example vlan 22). Network discovery process assigns tags with vlan IDs and names to devices and interfaces, I am using these tags in this example to build the map. First, I use match_function to pick devices that have tag VlanId.22. Only devices that have at least one interface in this vlan are going to have this tag. Then I add adjacent devices to pick up those that may not have been discovered because they do not answer to SNMP. I want these devices in my map, so I set adjacent_devices to True. Finally, I define parameter intf_match_function as a function that matches interface tags to pass only those that have tag ifVlan.22. If a device was not discovered, it does not get any of these tags but the logic used with parameters match_function and intf_match_function described above accepts this device anyway. Here is the script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import view


VIEWS = [
    view.View(name='all'),

    view.View(
        name='vlan22',
        match_function=lambda host: 'VlanId.22' in host.tags,
        intf_match_function=lambda host, intf: 'ifVlan.22' in intf.tags,
        adjacent_devices=True
    ),
]

class UserViewBuilder(object):
    def __init__(self, log):
        self.log = log

    def execute(self):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :return:  list of View objects
            """
            return VIEWS

4.2.1. Matching Rules for Hierarchical Views

Consider the following views forming three-level hierarchy:

_images/aafig-46583ffccf8b2145b4107316eae24ffc576e1af7.svg

The general rule is that the matching function of each view is called with devices that were determined to belong to the view of the upper level. In this example, matching function of the view “clusters” would be called with all devices known to the system. The function may pick some of them or may be allow all of them to be part of the view “clusters”. Next, the system will call matching function of view “Cluster1”, but this time, it will check only devices that are members of “clusters”. On the next level, while building view “Cluster1Core”, the system will only examine devices that are members of “Cluster1”.

Once devices have been allocated to views, NetSpyGlass builds maps. This is done “bottom-up”, that is, it builds lower level views first. For each view, it adds devices to the map and checks if it has lower level view. If this is the case, it then removes devices that are members of the lower level view from the map and replaces them with a symbol that will represent that view. Finally, it reconnects network links that used to be terminated on the devices but should now connect to the symbol representing lower level view.

4.2.2. Class View

Copyright (C) 2014 HappyGears - All Rights Reserved

Unauthorized copying of this file, via any medium is strictly prohibited. Proprietary and confidential

DO NOT MODIFY ! This file is part of the distribution and may change in the future versions.

class view.View(*args, **kwargs)

Bases: object

This class describes a view.

Parameters:view_dict – a dictionary that describes the view. Allowed keys are: ‘name’, ‘parent’, ‘match_function’, ‘connecting_devices’, ‘adjacent_devices’

Alternatively, all parameters that describe the view can be provided as arguments:

Parameters:
  • name – view name
  • parent – the name of the parent view
  • match_function – python function that will be called with one parameter (PyDevice object) to determine if a device should be a member of the view. The function should return True or False
  • intf_match_function – python function that will be called with two parameters: PyDevice object and PyNetworkInterface object. It should decide if the interface should be a member of the view. The function should return True or False
  • connecting_devices – (boolean) if True, the view will also include devices that connect devices that match this view. Default value is False.
  • adjacent_devices – (boolean) if True, the view will also include devices one hop away from the devices that match this view. If this parameter is missing, its value is set to False

Examples:

1
2
3
4
5
6
v = View({
    'name': 'view_name',
    'matching_function': lambda host: host in ['host_foo', 'host_bar']
})

v = View(name='view_name', matching_function=lambda host: host in ['host_foo', 'host_bar'])
match(host)

This function calls match_function to determine if given host should belong to this view

Parameters:host – an instance of class Host (see above)
Returns:True if the host belongs here

4.3. Examples of Hierarchical Views

There are many uses for nested views, here are a few

4.3.1. Clusters

this is probably the most typical use for the nested views. The idea is to build at least two levels of views: the top level shows clusters (or data centers) and links that connect them. Clicking any cluster opens up lower level map that shows only devices located in this cluster and possibly adjacent devices, too. Example file views_example.py (located in the directory doc of the distribution package) demonstrates how this is done. Assuming we have three clusters “SJ”, “EQ” and “RD”, we can build views as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
SJ_SUBNETS = [
    ipaddr.IPNetwork('10.3.1.0/24'),
    ipaddr.IPNetwork('10.3.2.0/24'),
    ipaddr.IPNetwork('10.3.3.0/24'),
    ipaddr.IPNetwork('10.3.4.0/24')
]


def match_rd_cluster(dev):
    return re.search('(\.rd$)|(^rd.*)', dev.name, re.IGNORECASE) is not None


def match_eq_cluster(dev):
    return re.search('(\.eq$)|(^eq.*)', dev.name, re.IGNORECASE) is not None


def match_sj_cluster(dev):
    if re.search('(\.sj$)|(^sj.*)', dev.name, re.IGNORECASE):
        return True
    addr = ipaddr.IPAddress(dev.address)
    return any(addr in net for net in SJ_SUBNETS)


VIEWS = [
    {
        'name': 'clusters',
    },

    {
        'name': 'RD',
        'parent': 'clusters',
        'match_function': match_rd_cluster,
        'adjacent_devices': True
    },

    {
        'name': 'EQ',
        'parent': 'clusters',
        'match_function': match_eq_cluster,
        'adjacent_devices': True
    },

    {
        'name': 'SJ',
        'parent': 'clusters',
        'match_function': match_sj_cluster,
        'adjacent_devices': True
    },

]

class UserViewBuilder(object):
    """
    NetSpyGlass uses this class and its functions as an API to the view
    configuration managed by the user.

    :param log:   system logger object
    """

    def __init__(self, log):
        self.log = log

    def execute(self, devices):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass)
        :rtype : list of strings
        :return:  list of View objects
        """
        return [view.View(x) for x in VIEWS]

Here, we use combination of criteria to decide which cluster given device should belong to. For RD and EQ we rely on the naming convention (match device names using some regular expressions), but for SJ we match both names and ip addresses.

The result is view “clusters” that shows just three objects “RD”, “EQ” and “SJ”, connected with links that display combined values of monitoring variables, such as traffic level. Clicking any of these objects opens map of the lower level with devices located in the corresponding cluster.

4.3.2. Virtualization

NetSpyGlass can discover internal network configuration in VMWare ESX virtualization hosts and determine virtual network topology. It associates devices to virtual machines and corresponding virtualization hosts and determines how they are connected together and to the outside network via virtual switches and port groups. This means you can build maps showing VMs in the context of both virtual and physical networks.

No special configuration is required to build composite map showing both virtual and physical switches, this is the default. Virtual switches will appear in maps as objects with the name composed from the name of the virtualization host and the name of the virtual server, such as “esxi55-1[Public]”. Here “Public” is the name of the virtual switch.

NetSpyGlass assigns tags in facet “VmHost” to devices it determines to be virtual machines and objects it creates to describe virtual switches. The value of this tag is the name of the corresponding virtualization host. For example, if virtualization host “esxi55-1” has VM “linux-test-1” and virtual switch “Public”, device “linux-test-1” gets tag “VmHost.esxi55-1”. Object created to descripe virtual switch “Public” gets the same tag as well. This allows us to build nested views to put all VMs and virtual switches inside of one view. The following configuration defines top level view “virtualization” that has two lower level views “esx1” and “esx2” that contain all VMs and virtual switches hosted by vm servers “esxi55-1” and “esxi55-2”:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def match_esxi1(host):
    return 'VmHost.esxi55-1' in host.tags or host.name == 'esxi55-1'

def match_esxi2(host):
    return 'VmHost.esxi55-2' in host.tags or host.name == 'esxi55-2'


VIEWS = [
    {
        'name': 'virtualization',
        'match_function': match_virt
    },

    {
        'name': 'esxi1',
        'match_function': match_esxi1,
        'parent': 'virtualization',
        'adjacent_devices': True
    },

    {
        'name': 'esxi2',
        'match_function': match_esxi2,
        'parent': 'virtualization',
        'adjacent_devices': True
    },
}

class UserViewBuilder(object):
    """
    NetSpyGlass uses this class and its functions as an API to the view
    configuration managed by the user.

    :param log:   system logger object
    """

    def __init__(self, log):
        self.log = log

    def execute(self, devices):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass)
        :rtype : list of strings
        :return:  list of View objects
        """
        return [view.View(x) for x in VIEWS]

Views “esxi1” and “esxi2” match tag “VmHost” or device name and add adjacent devices. This way, each view contains all VMs, virtual switches and the ESXi host itself, as well as adjacent devices such as switch the host is connected to.