5. Tags¶

Tags in NetSpyGlass are organized in “facets”. Facets represent certain characteristics or aspects of devices, interfaces or hardware components. For example, devices can be classified by their role, such as “Switch”, “Router” or “Load Balancer”. Tags that belong to facet “Role” describe this characterisation. Other tag facets can be used to classify devices and components in many different ways. It is possible to add many tags that belong to the same facet.

NetSpyGlass groups facets internally to separate tags that belong to a device from tags that belong to its interfaces. This way, the system can display only essential tags in different parts of the UI to avoid clutter.

NetSpyGlass server defines set of tag facets it uses to attach information about devices, network interfaces, hardware components, protocols and firewall counters to objects created during network discovery. Some of these facets are explained below. You can add your own tags using tag selector Python hook script (see User Defined Device Tags).

5.2. How Tags are Assigned¶

Many tags are assigned by the server at the end of the discovery run to attach information about devices and their components to internal objects that represent them in NetSpyGlass. At this point NetSpyGlass have already learned everything it can about devices: vendor, model, inventory of interfaces and hardware components and network topology. This informaion is used to assing tags to objects that represent devices, interfaces, hardware components, protocols, firewall counters etc. We call tags assigned this way “implicit”, thet are immutable and are reassigned again are the end of the next discovery run.

The next step after implicit tags are assigned is to call Python hook script described by the configuration parameter tagSelector in the configuration file nw2.conf. If this parameter is absent, the system uses default tag selector script that ships as an internal Java resource. We provide a copy of this script in the file tag_selector.py in the directory doc in the distribution tar archive. This copy is not actually used by the program, it is provided so you can inspect it and use it as a reference. The default tag_selector script does not assign any tags but defines the class and its functions and provides additional documentation in the doc strings.

As with other Python hooks, configuration parameter tagSelector describes Python module and class name:

network {
   tagSelector = "tags.UserTagSelector"

In this example, NetSpyGlass will try to load class UserTagSelector from tags.py. Class UserTagSelector must be derived from the class TagSelector described in the next section Tag Selector Class.

Once tags are assigned, they can be used in many ways. First, we can use tags to analyse devices and components and decide if they should be monitored or not. See Selection of components to monitor for an explanation and example of how to use tags in monitoring variable builder hook script. There you will see an example of the Python function that decides which interfaces should be monitored by checkig tags of the PyNetworkInterface object passed to it as a parameter.

Once monitoring variables are created, they carry copy of the set of tags taken from the device and the component the variable corresponds to. You can use these tags to find, filter and group monitoring variables in your data processing rules script (see Data Processing Rules).

This high level diagram illustrates how tags are assigned and then propagate through the system:

5.3. Tag Selector Class¶

class tag_selector.TagSelector(log, graph)¶

Bases: object

Tag selector Python hook. This class is created when NetSpyGlass reconfigures devices and is used to assign user-defined tags to devices and interfaces. This happens at the following times:

on startup if the server determines that some devices have been updated in the config
when user modifes tag selector hook script that defines class based on TagSelector
always after network discovery

Constructor is passed two arguments:

Parameters:	log – Java logger object. Use it like this: self.log.info(‘some text’) graph – Topology graph. This can be None when the server starts for the first time and there is no topology information yet. This script should handle the case when graph is None to avoid crashing

The base class does not assign any tags. To assign user-defined tags, create your own class based on this and override functions assign_device_tags(), assign_interface_tags() and assign_hw_component_tags(). There is no need to override any of these functions if you do not want to add corresponding tags. See User Defined Device Tags for more information and examples.

Functions defined in this class add, remove or modify tags of device, interface or hardware component objects using their attribute tags (a set of strings). Each added tag must conform to the format TagFacet.word. If the format of the new tag does not match “TagFacet.word” (two words, separated by a dot) or tag facet is not one of the standard facets, the tag can not be added and the error will be logged in the log “error.log”.

assign_device_tags(device)¶

Analyze the device and add or remove tags. Device passed as argument already has implicit (automatic) tags set by the system. This function can modify them or add new ones by adding strings to the attribute ‘tags’ (a set of strings). Each added tag must conform to the format “TagFacet.word”. For example, to add a new tag to the device, just add string to device.tags:

device.tags.add('MyTagFacet.Word')

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:	device – an `net.happygears.nw2.py.py_wrappers.PyDevice` object

assign_hw_component_tags(device, components)¶

Analyze objects in the list “components” (parent device object is also provided as argument “device”) and add or remove tags. Both device and components already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:	device – instance of class PyDevice components – list of `net.happygears.nw2.py.py_wrappers.PyHardwareComponent` objects

assign_interface_tags(device, interfaces)¶

Analyze interface objects in list “interfaces” (parent device object is also provided as argument “device”) and add or remove tags. Both device and interfaces already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

One of the intended purposes of this function is to parse interface description and generate tags.

Parameters:	device – instance of class PyDevice interfaces – list of `net.happygears.nw2.py.py_wrappers.PyNetworkInterface` objects

assign_protocol_descriptor_tags(device, components)¶

Analyze and manipulate tags on objects in the list “components” where each item is an object of class net.happygears.nw2.py.py_wrappers.PyProtocolDescriptor. This class is an abstraction that is used to describe a parameter or a counter of a protocol. NetSpyGlass automatically adds tags to these objects when they are created; this function can analyse and manipulate those tags.

Both device and components already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”.

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:	device – instance of class PyDevice components – list of `net.happygears.nw2.py.py_wrappers.PyProtocolDescriptor` objects

5.4. User Defined Device Tags¶

User can create their own tags using Python script tags.py located in the directory scripts in the git repository co-located with the cluster. All clusters ship with this script as a prototype, the copy we include with new cluster simply defines required class and methods but does not do anything.

NetSpyGlass watches this script for changes and reloads its whenever the script is modified. There is no need to restart the system, jsut edit, commit and push, then watch System/Logs/Python panel for errors and warnings.

A class should be defined in this script, using tag_selector.TagSelector as its base class. The class should have the following methods:

assign_device_tags() : this method is used to add tags to the device object
assign_interface_tags() : this method is used to add tags to interface objects

A copy of the script included with new cluster already has these methods and provides some usage examples in comments.

Example:

import tag_selector


class UserTagSelector(tag_selector.TagSelector):
    def __init__(self, log, graph):
        super(UserTagSelector, self).__init__(log, graph)

    def assign_device_tags(self, device):
        if device.name in ['router1', 'router2']:
            device.tags.add('Explicit.core')
            device.tags.add('Explicit.test1')

In this simplified example, we add tags Explicit.core and Explicit.test to devices “router1” and “router2” simply by matching their name. You can use regular expressions via Python module re to implement complex pattern matches or module ipaddr to match device addresses. In the end, to add a tag just add a string in the format “TagFacet.word” to the set device.tags.

5.4.1. User Defined Interface Tags: Parsing Interface Descriptions¶

The same python hook script can be used to add interface tags. To do this, add function assign_interface_tags to the class as follows:

import re

from net.happygears.nw2.py.py_wrappers import PyDevice
from net.happygears.nw2.py.py_wrappers import PyNetworkGraph
import tag_selector


class UserTagSelector(tag_selector.TagSelector):
    def __init__(self, log, graph):
        super(UserTagSelector, self).__init__(log, graph)

    def assign_interface_tags(self, device, interfaces):
        for intf in interfaces:
            m = re.search('T=peerAS(\d+)', intf.description)
            if m:
                asn = m.group(1)
                intf.tags.add('ifDescription.AS' + str(asn))

In this example we use regular expression to match pattern “T=peerAS(d+)” in the interface description and add tag in facet ifDescription. The tag looks like ifDescription.AS174 (i.e. it has peer AS number encoded in the description). You can add any number of tags to the same interface this way, however they all must belong to the facet ifDescription to be processed correctly by the rest of the system. The meaning of these tags is completely up to you.

Example above demonstrates how we can parse interface description and assign tags to the network interface objects. We can take this one step further and assign tags extracted from interface description to devices on either side of the link. To do this, we’ll use object PyNetworkGraph passed to the constructor of the tag selector class:

import re

from net.happygears.nw2.py.py_wrappers import PyDevice
from net.happygears.nw2.py.py_wrappers import PyNetworkGraph


class UserTagSelector(object):
    def __init__(self, log, graph):
        self.log = log
        self.graph = graph

    def assign_interface_tags(self, device, interfaces):
        for intf in interfaces:
            if 'T=transit' in intf.description:
                intf.tags.add('ifDescription.transit')
                opposite = self.graph.get_connected_device(device, intf)
                if opposite:
                    opposite_dev, opposite_intf = opposite
                    assert isinstance(opposite_dev, PyDevice)
                    self.log.info('Device ' + device.name + ': adding tag "transit" to ' + opposite_dev.name)
                    opposite_dev.tags.add('Explicit.transit')

We use simple check to see if string “T=transit” appears somewhere in the interface description and set interface tag ‘ifDescription.transit”. Then we call get_connected_device() on the network graph object to get device and interface on the opposite side of the link. It is possible that such device does not exist, if NetSpyGlass could not determine where this interface is connected. If this is the case, this function returns None. Otherwise it returns a tuple (opp_dev, opp_intf) where opp_dev is the device on the opposite side of the link (a PyDevice object) and opp_intf is corresponding network interface (a PyNetworkInterface object). We can use these two objects to add tags just like we did before, using their tags attribute.

5.6. External Tags¶

NetSpyGlass supports special category of device and component tags that we call “External tags”. These tags can be updated using API call that writes them to the database. Every time NetSpyGlass updates tags of a device and its components, it does this in three steps:

run built-in algorithms that assign standard tags using information collected about the device during discovery process
apply external tags stored in the database at that moment
run user’s Python app to assign user-defined tags

Note

The Python app has access to all tags created previously, including external ones.

The API call used to update external tags only manipulates records in the database but does not update actual tags attached to the “live” device objects.

We assume that the user is going to build an automation to use this API call, for example by integrating with Netbox or another external database that acts as a source of meta data about devices, interfaces and components. The automation can run on schedule and can compare tags present in the external source with those stored in NetSpyGlass, then use the API call to synchronize them. It may be convenient to do this in batches to reduce amount of data that needs to be stored and compared at a time. To avoid possible race conditions and necessary internal queueing, the API call used to update external tags only updates database records. Once all external tags have been updated, the automation should make another API call to initiate tag update in NetSpyGlass.

5.6.1. API call used to query and update external tags¶

5.6.1.1. GET /v2/tags/net:netid/external[?id=:devID|name=:name|address=:address]¶

This call will return external tags for all devices or a device identified by the query parameters. Only one device match parameter can be provided at a time, but if the query parameter is missing, the call returns external tags for all devices and components.

The response format is as follows:

[
  {
    "category": "device",
    "device": {
      "id" : 12345,
      "name" : "device_name",
      "address" : "10.1.2.3"
    },
    "tags": [
        "Location.SJC",
        "AcmeRole.BB"
    ]
  },
  {
    "category": "interface",
    "device": {
      "id" : 12345,
      "name" : "device_name",
      "address" : "10.1.2.3"
    },
    "component": {
        "index": 1010,
        "name": "ge-0/0/0"
    },
    "tags": [
        "NsgControl.noLink"
    ]
  },

]

See the next section for the description of the “category”.

Information about the device includes its id, name and address. This information is provided to make it easier to match the device against external sources, such as Netbox, where NetSpyGlass device ID is unlikely to be present, however the name and address can be used. The address returned by this call is the one NetSpyGlass uses to communicate with the device.

We also provide both the index and name for the interface and component. If the value of “category” is “device”, the tag is meant to be attached to the device rather than interface or a component and section “component” may be skipped.

Note that the call does not support filtering by the component and always returns external tags attached to all components (if any) of the matching device.

5.6.1.2. POST /v2/tags/net:netid/external¶

The body should have the following format:

[
  {
    "category": <category>,
    "device": {
      <device match clause>
    },
    "component" : {
        <component match clause>
    }
    "tags": [
        "Facet.Word"
    ]
  }
]

The category can be one of the following:

“device” - the tag will be assigned to the device; key “index” is ignored
“interface” - the tag will be assigned to the interface with ifIndex given by key “index”
“component” - the tag will be assigned to the hardware component with index given by the key “index”

Device match clause can be one of:

“id” - the value should be numeric device ID
“name” - the value should be device name
“address” - the value should be device IP address

only one device match clause is supported at a time.

Component match can be one of:

“index” - interface or h/w component is matched by its index
“name” - interface or h/w component is matched by the name

Each dictionary inside of the envelope array assigns one tag, but the call can pass any number of these in any combination. If the call tries to update multiple tags for many devices and some of the device or component match clauses do not match, the response will mention them under “errors” but the server will continue to process the rest and return HTTP response 200. The response of this call has the following format:

{
  "errors": [
    "Device not found: name=invalid-name",
    "Device not found: address=10.99.32.1"
  ],
  "status": "tags updated for 10 devices"
}

5.6.1.3. Example¶

The following example demonstrates different combinations of device and component match clause formats.

[
  {
    "category": "device",
    "device": {
      "id" : 12345,
    },
    "tags": [
        "Location.SJC"
    ]
  },
  {
    "category": "device",
    "device": {
      "id" : 98765,
    },
    "tags": [
        "Location.IAD"
    ]
  },
  {
    "category": "interface",
    "device": {
      "id" : 1001,
    },
    "component" : {
      "index": 1010
    }
    "tags": [
        "NsgControl.noLink"
    ]
  },
  {
    "category": "interface",
    "device": {
      "id" : 1001,
    },
    "component" : {
      "name": "ge-0/0/0"
    }
    "tags": [
        "NsgControl.noLink"
    ]
  },
  {
    "category": "component",
    "device": {
      "name": "jnpr-router"
    },
    "component": {
      "name" : "Routing Engine 0"
    },
    "tags": [
        "NsgControl.SomeTag1",
        "NsgControl.Another",
    ]
  }
]

Note

The input data format for this API call supports matching of device and interface/component by their numeric id, name or address. Since internal object models in NetSpyGlass rely on the ids, the server uses provided name or address to find the device or interface and records their id when it writes information about external tag to the database. The match of the name or address happens once when the API call is executed. If the name or address changes later, the tag will remain attached to the same device or interface until you make this API call again.

5.6.1.4. Removing External Tags¶

The POST API call described above always replaces all external tags for the matching device and component. If you need to remove all external tags for a device or a component, execute the same API call with body data that does not have “tags” key. The scope of this call is determined by the category, device and component match clause. If the call has category “device”, then all external tags attached to the matching device will be removed but any external tags attached to its components will remain intact.

The following would remove all external tags for the device with id=56:

[
  {
    "category": "device",
    "device": {
      "id" : 56
    }
  }
]

The following would remove all external tags for the component and device identified by name:

[
  {
    "category": "device",
    "device": {
      "name" : "router1"
    },
    "component": {
      "name" : "Routing Engine 0"
    }
  }
]

5. Tags¶

5.1. Standard Tag Facets¶

5.1.1. Tags that describe device¶

5.1.2. Tags that describe interfaces¶

5.2. How Tags are Assigned¶

5.3. Tag Selector Class¶

5.4. User Defined Device Tags¶

5.4.1. User Defined Interface Tags: Parsing Interface Descriptions¶

5.5. Tags in facet NsgControl¶

5.6. External Tags¶

5.6.1. API call used to query and update external tags¶

5.6.1.1. GET /v2/tags/net:netid/external[?id=:devID|name=:name|address=:address]¶

5.6.1.2. POST /v2/tags/net:netid/external¶

5.6.1.3. Example¶

5.6.1.4. Removing External Tags¶