3. Views

Views is a mechanism that allows the user to organize maps of a large network. Instead of always looking at a map that has 1000 devices, you can have multiple maps that show subset of the devices, grouped by some criteria that make sense to you. You can easily build maps to show OSPF areas or BGP peering or put devices in each data center into their own map and build an overview map to show how data centers are connected to each other.

Each view appears in the UI as a tile in the home page. If views are organized into a hierarchy, you can “drill down” by clicking objects that represent lower level views in maps. In any case, there is exact one-to-one correspondence between views and maps.

NetspyGlass offers programmatic mechanism you can use to match devices to views. This is done using Python script views.py that you can find in the directory scripts in the integrated Git repository

3.1. Building Your Own Views

Script views.py consists of the following parts:

• few global functions used to illustrate different ways to match device attributes
• list VIEWS that actually defines views
• class UserViewBuilder that serves as an API between this script and NetSpyGlass

When NetSpyGlass wants to create views, it loads this script, creates an instance of class UserViewBuilder defined in this script and calls its function execute(), which should return list of instances of the Python class View.

Default implementation of class UserViewBuilder creates View objects from Python dictionaries defined as items in the list VIEWS. This is just a reference implementation, you can change it if you prefer different way of doing it. The only thing that really matters is that there should be class with method execute() that returns list of View objects, everything elase is up to you. Function execute() will be called with one argument, a list of PyDevice objects.

Each View object has the following attributes:

• name (string)

  the name of the view

• parent (string)

  The name of the parent view. This is optional, views that have no parent parameter are the top-level views

• match_function a function of one argument.

  This function is called with PyDevice object representing a device. This function should return True if the device should be a member of the view. This attribute is optional, if it is missing, the view matches all devices.

• adjacent_devices (boolean)

  if True, the view will also include devices one hop away from the devices that match this view. If this parameter is missing, its value is set to False

• connecting_devices (boolean)

  if True, the view will also include devices that connect devices that match this view. Default value is False.

The match_function parameter should be a function that is going to be called with an instance of Python class net.happygears.nw2.py.py_wrappers.PyDevice.PyDevice. This function can analyse device name, address or tags to decide if it should be a member of the view.

NetSpyGlass uses parameters of the View object as follows:

1. all devices are passed to the match_function that decides which ones should appear in the map. If View does not have parameter match_function, then all devices match.
2. if parameters adjacent_devices or connecting_devices have value True, corresponding adjacent and connecting devices are added too.

Examples in Examples of Views demonstrate different ways to match host attributes name, address and tags.

You can use regular expressions (via Python module re) to match host name and tags. You can also use Python module ipaddr to match IP addresses instead of operating with addresses as strings. Embedded Python interpreter comes with almost full set of standard Python modules, all of them are available to you in the view builder script.

To build your own views, start with editing list VIEWS. Remove or edit example views you do not need and add your own.

NetSpyGlass monitors Python scripts and reloads then when they change. To upload your changes, commit and push to git repository. NetSpyGlass pulls from the repository on schedule with interval 2 min so your script should appear there in a couple of minutes.

NetSpyGlass rebuilds views after it finishes discovery of each device or on command. If you want to rebuild views manually to test your script, use command nsgcli make views on command line, or navigate to System / Actions in the UI and click “Update Views”.

3.2. Examples of Views

the following script creates three views: “all” (includes all devices), “Backbone” (includes devices that are members of OSPF area 0) and “Peering” (includes only devices that have BGP peering with other ASes than their own, that is, run eBGP):

import view

def match_ospf_area_0(host):
    return 'OSPFArea.0' in host.tags


def match_bgp_speaker(host):
    return 'Role.eBgpPeer' in host.tags


VIEWS = [
    view.View(name='all'),

    view.View(
        name='backbone',
        match_function=match_ospf_area_0,
    ),

    view.View(
        name='Peering',
        match_function=match_bgp_speaker,
    ),

]


class UserViewBuilder(object):
    def __init__(self, log):
        self.log = log

    def execute(self):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :return:  list of View objects
            """
            return VIEWS

3.2.1. Matching Rules for Hierarchical Views

Consider the following views forming three-level hierarchy:

The general rule is that the matching function of each view is called with devices that were determined to belong to the view of the upper level. In this example, matching function of the view “clusters” would be called with all devices known to the system. The function may pick some of them or may be allow all of them to be part of the view “clusters”. Next, the system will call matching function of view “Cluster1”, but this time, it will check only devices that are members of “clusters”. On the next level, while building view “Cluster1Core”, the system will only examine devices that are members of “Cluster1”.

Once devices have been allocated to views, NetSpyGlass builds maps. This is done “bottom-up”, that is, it builds lower level views first. For each view, it adds devices to the map and checks if it has lower level view. If this is the case, it then removes devices that are members of the lower level view from the map and replaces them with a symbol that will represent that view. Finally, it reconnects network links that used to be terminated on the devices but should now connect to the symbol representing lower level view.

3.2.2. Class View

class view.View(*args, **kwargs)

Bases: object

This class describes a view.

Parameters:view_dict – a dictionary that describes the view. Allowed keys are: ‘name’, ‘parent’, ‘match_function’, ‘connecting_devices’, ‘adjacent_devices’

Alternatively, all parameters that describe the view can be provided as arguments:

Parameters:

• name – view name
• parent – the name of the parent view
• match_function – python function that will be called with one parameter (PyDevice object) to determine if a device should be a member of the view. The function should return True or False
• intf_match_function – python function that will be called with two parameters: PyDevice object and PyNetworkInterface object. It should decide if the interface should be a member of the view. The function should return True or False
• connecting_devices – (boolean) if True, the view will also include devices that connect devices that match this view. Default value is False.
• adjacent_devices – (boolean) if True, the view will also include devices one hop away from the devices that match this view. If this parameter is missing, its value is set to False

Examples:

v = View({
    'name': 'view_name',
    'matching_function': lambda host: host in ['host_foo', 'host_bar']
})

v = View(name='view_name', matching_function=lambda host: host in ['host_foo', 'host_bar'])

match(host)

This function calls match_function to determine if given host should belong to this view

Parameters:host – an instance of class Host (see above) Returns:True if the host belongs here

3.3. Examples of Hierarchical Views

There are many uses for nested views, here are a few

3.3.1. Clusters

this is probably the most typical use for the nested views. The idea is to build at least two levels of views: the top level shows clusters (or data centers) and links that connect them. Clicking any cluster opens up lower level map that shows only devices located in this cluster and possibly adjacent devices, too. Example file views_example.py (located in the directory doc of the distribution package) demonstrates how this is done. Assuming we have three clusters “SJ”, “EQ” and “RD”, we can build views as follows:

SJ_SUBNETS = [
    ipaddr.IPNetwork('10.3.1.0/24'),
    ipaddr.IPNetwork('10.3.2.0/24'),
    ipaddr.IPNetwork('10.3.3.0/24'),
    ipaddr.IPNetwork('10.3.4.0/24')
]


def match_rd_cluster(dev):
    return re.search('(\.rd$)|(^rd.*)', dev.name, re.IGNORECASE) is not None


def match_eq_cluster(dev):
    return re.search('(\.eq$)|(^eq.*)', dev.name, re.IGNORECASE) is not None


def match_sj_cluster(dev):
    if re.search('(\.sj$)|(^sj.*)', dev.name, re.IGNORECASE):
        return True
    addr = ipaddr.IPAddress(dev.address)
    return any(addr in net for net in SJ_SUBNETS)


VIEWS = [
    {
        'name': 'clusters',
    },

    {
        'name': 'RD',
        'parent': 'clusters',
        'match_function': match_rd_cluster,
        'adjacent_devices': True
    },

    {
        'name': 'EQ',
        'parent': 'clusters',
        'match_function': match_eq_cluster,
        'adjacent_devices': True
    },

    {
        'name': 'SJ',
        'parent': 'clusters',
        'match_function': match_sj_cluster,
        'adjacent_devices': True
    },

]

class UserViewBuilder(object):
    """
    NetSpyGlass uses this class and its functions as an API to the view
    configuration managed by the user.

    :param log:   system logger object
    """

    def __init__(self, log):
        self.log = log

    def execute(self, devices):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass)
        :rtype : list of strings
        :return:  list of View objects
        """
        return [view.View(x) for x in VIEWS]

Here, we use combination of criteria to decide which cluster given device should belong to. For RD and EQ we rely on the naming convention (match device names using some regular expressions), but for SJ we match both names and ip addresses.

The result is view “clusters” that shows just three objects “RD”, “EQ” and “SJ”, connected with links that display combined values of monitoring variables, such as traffic level. Clicking any of these objects opens map of the lower level with devices located in the corresponding cluster.

3.3.2. Virtualization

NetSpyGlass can discover internal network configuration in VMWare ESX virtualization hosts and determine virtual network topology. It associates devices to virtual machines and corresponding virtualization hosts and determines how they are connected together and to the outside network via virtual switches and port groups. This means you can build maps showing VMs in the context of both virtual and physical networks.

No special configuration is required to build composite map showing both virtual and physical switches, this is the default. Virtual switches will appear in maps as objects with the name composed from the name of the virtualization host and the name of the virtual server, such as “esxi55-1[Public]”. Here “Public” is the name of the virtual switch.

NetSpyGlass assigns tags in facet “VmHost” to devices it determines to be virtual machines and objects it creates to describe virtual switches. The value of this tag is the name of the corresponding virtualization host. For example, if virtualization host “esxi55-1” has VM “linux-test-1” and virtual switch “Public”, device “linux-test-1” gets tag “VmHost.esxi55-1”. Object created to descripe virtual switch “Public” gets the same tag as well. This allows us to build nested views to put all VMs and virtual switches inside of one view. The following configuration defines top level view “virtualization” that has two lower level views “esx1” and “esx2” that contain all VMs and virtual switches hosted by vm servers “esxi55-1” and “esxi55-2”:

def match_esxi1(host):
    return 'VmHost.esxi55-1' in host.tags or host.name == 'esxi55-1'

def match_esxi2(host):
    return 'VmHost.esxi55-2' in host.tags or host.name == 'esxi55-2'


VIEWS = [
    {
        'name': 'virtualization',
        'match_function': match_virt
    },

    {
        'name': 'esxi1',
        'match_function': match_esxi1,
        'parent': 'virtualization',
        'adjacent_devices': True
    },

    {
        'name': 'esxi2',
        'match_function': match_esxi2,
        'parent': 'virtualization',
        'adjacent_devices': True
    },
}

class UserViewBuilder(object):
    """
    NetSpyGlass uses this class and its functions as an API to the view
    configuration managed by the user.

    :param log:   system logger object
    """

    def __init__(self, log):
        self.log = log

    def execute(self, devices):
        """
        Generate list of View objects to describe views. Views can
        refer to each other by name using attribute "parent" to establish
        hirarchy. This function only creates "blank" views, it does not add
        devices to them.

        :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass)
        :rtype : list of strings
        :return:  list of View objects
        """
        return [view.View(x) for x in VIEWS]

Views “esxi1” and “esxi2” match tag “VmHost” or device name and add adjacent devices. This way, each view contains all VMs, virtual switches and the ESXi host itself, as well as adjacent devices such as switch the host is connected to.