7. Tags

Tags in NetSpyGlass are organized in “facets”. Facets represent certain characteristics or aspects of devices, interfaces or hardware components. For example, devices can be classified by their role, such as “Switch”, “Router” or “Load Balancer”. Tags that belong to facet “Role” describe this characterisation. Other tag facets can be used to classify devices and components in many different ways. It is possible to add many tags that belong to the same facet.

NetSpyGlass groups facets internally to separate tags that belong to a device from tags that belong to its interfaces. This way, the system can display only essential tags in different parts of the UI to avoid clutter.

NetSpyGlass server defines set of tag facets it uses to attach information about devices, network interfaces, hardware components, protocols and firewall counters to objects created during network discovery. Some of these facets are explained below. You can add your own tags using tag selector Python hook script (see User Defined Device Tags).

7.1. Tag Facets

Explicit: this facet holds tags assigned by the user in the application. Each tag is just a word and tags in this facet can be assigned to devices and interfaces.

7.1.1. Tags that describe device

Vendor : vendor name, ‘Cisco’, ‘Juniper’ etc

Role : device role, one of the following:

  • Unknown
  • SimulatedNode
  • SimulatedBridge
  • Router
  • Switch
  • Server
  • LoadBalancer
  • WirelessClient
  • WirelessAP
  • Firewall
  • eBgpPeer
  • iBgpPeer

Protocol : protocols this device runs. Possible values are:

  • Unknown
  • Stp_Off
  • UnknownStp
  • Stp
  • Sstp ( 802.1q )
  • Rstp ( 802.1W )
  • PVST
  • PvstPlus
  • RapidPvst
  • Mistp
  • MistpPvstPlus
  • Mst ( IEEE 802.1s Multiple Spanning Tree (MST)
  • VPLS
  • OSPF
  • BGP4

OSPFArea : OSPF areas this device is a member of, as numbers

BGP4LocalAS : Local AS

BGP4Peer : this facet holds two kinds of tags: AS numbers (strings such as “AS174”) and BGP peer name (e.g. “myrouter1”)

7.1.2. Tags that describe interfaces

ifRole : interface role. Possible values:

  • Unknown
  • RouterInterface
  • UntaggedSwtichPort
  • TaggedSwtichPort
  • OutOfBandManagement
  • VlanInterface
  • VirtualInterface
  • BroadcastTypeInterface
  • LoopbackInterface
  • SimulatedInterface
  • WLan
  • Internal (e.g. unit 32767 on Juniper. We ignore these while building L2 topology)
  • Tunnel
  • PeeringInterface
  • OspfInterface

ifSpeed : interface speed as a string, e.g. “1G”, “10G”

ifType : the value of ifType from RFC1213 MIB

ifAdminStatus : administrative status (Up, Down, Testing)

ifOperStatus : operational status (Up, Down, Testing)

ifDescription : this facet holds tags generated from interface descriptions

Link : tags in this facet describe the opposite end of the network link terminated on the interface

ifBGP4Peer : tags in this facet describe BGP4 peer connected to the inteface. Tag words are in the form “ASxxxx” (where ‘xxxx’ is the AS number) or the host name of the iBGP peer.

OSPFState : tags in this facet describe OSPF state of the interface. Values can be as follows:

  • unknown
  • down
  • loopback
  • waiting
  • pointToPoint
  • designatedRouter
  • backupDesignatedRouter
  • otherDesignatedRouter

ifVlan : vlan number the interfave is a member of. Tagged ports have multiple tags in this facet

Aggregator : tags in this facet are assigned to 802.3ad aggregation ports and describe aggregator port they belong to. For example, if 802.3ad aggregator port Port-Channel1 consists of aggregation ports Gi0/10, Gi0/11, then interface Port-channel1 will gets tag “ifRole.Aggregator” and interfaces Gi0/10 and Gi0/11 get tags “ifRole.AggregationPort” and “Aggregator.Po1”

7.2. How Tags are Assigned

Many tags are assigned by the server at the end of the discovery run to attach information about devices and their components to internal objects that represent them in NetSpyGlass. At this point NetSpyGlass have already learned everything it can about devices: vendor, model, inventory of interfaces and hardware components and network topology. This informaion is used to assing tags to objects that represent devices, interfaces, hardware components, protocols, firewall counters etc. We call tags assigned this way “implicit”, thet are immutable and are reassigned again are the end of the next discovery run.

The next step after implicit tags are assigned is to call Python hook script described by the configuration parameter tagSelector in the configuration file nw2.conf. If this parameter is absent, the system uses default tag selector script that ships as an internal Java resource. We provide a copy of this script in the file tag_selector.py in the directory doc in the distribution tar archive. This copy is not actually used by the program, it is provided so you can inspect it and use it as a reference. The default tag_selector script does not assign any tags but defines the class and its functions and provides additional documentation in the doc strings.

As with other Python hooks, configuration parameter tagSelector describes Python module and class name:

network {
   tagSelector = "tags.UserTagSelector"

In this example, NetSpyGlass will try to load class UserTagSelector from tags.py. Class UserTagSelector must be derived from the class TagSelector described in the next section Tag Selector Class.

Once tags are assigned, they can be used in many ways. First, we can use tags to analyse devices and components and decide if they should be monitored or not. See Selection of components to monitor for an explanation and example of how to use tags in monitoring variable builder hook script. There you will see an example of the Python function that decides which interfaces should be monitored by checkig tags of the PyNetworkInterface object passed to it as a parameter.

Once monitoring variables are created, they carry copy of the set of tags taken from the device and the component the variable corresponds to. You can use these tags to find, filter and group monitoring variables in your data processing rules script (see Data Processing Rules).

This high level diagram illustrates how tags are assigned and then propagate through the system:

_images/tags_assignment_flow.png

7.3. Tag Selector Class

class tag_selector.TagSelector(log, graph)

Bases: object

Tag selector Python hook. This class is created when NetSpyGlass reconfigures devices and is used to assign user-defined tags to devices and interfaces. This happens at the following times:

  • on startup if the server determines that some devices have been updated in the config
  • when user modifes tag selector hook script that defines class based on TagSelector
  • always after network discovery

Constructor is passed two arguments:

Parameters:
  • log – Java logger object. Use it like this: self.log.info(‘some text’)
  • graph – Topology graph. This can be None when the server starts for the first time and there is no topology information yet. This script should handle the case when graph is None to avoid crashing

The base class does not assign any tags. To assign user-defined tags, create your own class based on this and override functions assign_device_tags(), assign_interface_tags() and assign_hw_component_tags(). There is no need to override any of these functions if you do not want to add corresponding tags. See User Defined Device Tags for more information and examples.

Functions defined in this class add, remove or modify tags of device, interface or hardware component objects using their attribute tags (a set of strings). Each added tag must conform to the format TagFacet.word. If the format of the new tag does not match “TagFacet.word” (two words, separated by a dot) or tag facet is not one of the standard facets, the tag can not be added and the error will be logged in the log “error.log”.

assign_device_tags(device)

Analyze the device and add or remove tags. Device passed as argument already has implicit (automatic) tags set by the system. This function can modify them or add new ones by adding strings to the attribute ‘tags’ (a set of strings). Each added tag must conform to the format “TagFacet.word”. For example, to add a new tag to the device, just add string to device.tags:

device.tags.add('MyTagFacet.Word')

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:device – an net.happygears.nw2.py.py_wrappers.PyDevice object
assign_hw_component_tags(device, components)

Analyze objects in the list “components” (parent device object is also provided as argument “device”) and add or remove tags. Both device and components already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:
assign_interface_tags(device, interfaces)

Analyze interface objects in list “interfaces” (parent device object is also provided as argument “device”) and add or remove tags. Both device and interfaces already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

One of the intended purposes of this function is to parse interface description and generate tags.

Parameters:
assign_protocol_descriptor_tags(device, components)

Analyze and manipulate tags on objects in the list “components” where each item is an object of class net.happygears.nw2.py.py_wrappers.PyProtocolDescriptor. This class is an abstraction that is used to describe a parameter or a counter of a protocol. NetSpyGlass automatically adds tags to these objects when they are created; this function can analyse and manipulate those tags.

Both device and components already have implicit (automatic) tags added by the system when they are passed to this function. Function “assign_device_tags()” has already been called on the device, too. Tags are copied to monitoring variables later when the server creates them.

This function can add, remove or modify tags using attribute “tags” (a set of strings). Each added tag must conform to the format “TagFacet.word”.

Default implementation of this function does nothing. To add user-defined tags, create your own class derived from TagSelector and override this function.

Parameters:

7.4. User Defined Device Tags

Python hook tagSelector can be used to assign tags to devices and interfaces. Here is how to do this. First, we create Python module tags.py in the same directory where configuration file nw2.conf is located. In this module we define class UserTagSelector as follows:

import tag_selector


class UserTagSelector(tag_selector.TagSelector):
    def __init__(self, log, graph):
        super(UserTagSelector, self).__init__(log, graph)

    def assign_device_tags(self, device):
        if device.name in ['router1', 'router2']:
            device.tags.add('Explicit.core')
            device.tags.add('Explicit.test1')

In this simplified example, we add tags Explicit.core and Explicit.test to devices “router1” and “router2” simply by matching their name. You can use regular expressions via Python module re to implement complex pattern matches or module ipaddr to match device addresses. In the end, to add a tag just add a string in the format “TagFacet.word” to the set device.tags.

NetSpyGlass watches tag selector hook script and reloads configuration whenever the script is modified. There is no need to restart the system.

7.4.1. User Defined Interface Tags: Parsing Interface Descriptions

The same python hook script can be used to add interface tags. To do this, add function assign_interface_tags to the class as follows:

import re

from net.happygears.nw2.py.py_wrappers import PyDevice
from net.happygears.nw2.py.py_wrappers import PyNetworkGraph
import tag_selector


class UserTagSelector(tag_selector.TagSelector):
    def __init__(self, log, graph):
        super(UserTagSelector, self).__init__(log, graph)

    def assign_interface_tags(self, device, interfaces):
        for intf in interfaces:
            m = re.search('T=peerAS(\d+)', intf.description)
            if m:
                asn = m.group(1)
                intf.tags.add('ifDescription.AS' + str(asn))

In this example we use regular expression to match pattern “T=peerAS(d+)” in the interface description and add tag in facet ifDescription. The tag looks like ifDescription.AS174 (i.e. it has peer AS number encoded in the description). You can add any number of tags to the same interface this way, however they all must belong to the facet ifDescription to be processed correctly by the rest of the system. The meaning of these tags is completely up to you.

When a tag is added to an interface by the call to assign_interface_tags, the following messages appear in the log logs/info.log:

2014-06-20 16:27:59,461 INFO  Thread-3 [.nw2.tags.TagSelectorHook]: Device 10.0.14.228  c3560g-1: Calling tag_selector Python hook
2014-06-20 16:27:59,613 INFO  Thread-3 [.nw2.tags.TagSelectorHook]:     Gi0/31: Adding interface tags [Explicit.test, ifDescription.AS6941]

Example above demonstrates how we can parse interface description and assign tags to the network interface objects. We can take this one step further and assign tags extracted from interface description to devices on either side of the link. To do this, we’ll use object PyNetworkGraph passed to the constructor of the tag selector class:

import re

from net.happygears.nw2.py.py_wrappers import PyDevice
from net.happygears.nw2.py.py_wrappers import PyNetworkGraph


class UserTagSelector(object):
    def __init__(self, log, graph):
        self.log = log
        self.graph = graph

    def assign_interface_tags(self, device, interfaces):
        for intf in interfaces:
            if 'T=transit' in intf.description:
                intf.tags.add('ifDescription.transit')
                opposite = self.graph.get_connected_device(device, intf)
                if opposite:
                    opposite_dev, opposite_intf = opposite
                    assert isinstance(opposite_dev, PyDevice)
                    self.log.info('Device ' + device.name + ': adding tag "transit" to ' + opposite_dev.name)
                    opposite_dev.tags.add('Explicit.transit')

We use simple check to see if string “T=transit” appears somewhere in the interface description and set interface tag ‘ifDescription.transit”. Then we call get_connected_device() on the network graph object to get device and interface on the opposite side of the link. It is possible that such device does not exist, if NetSpyGlass could not determine where this interface is connected. If this is the case, this function returns None. Otherwise it returns a tuple (opp_dev, opp_intf) where opp_dev is the device on the opposite side of the link (a PyDevice object) and opp_intf is corresponding network interface (a PyNetworkInterface object). We can use these two objects to add tags just like we did before, using their tags attribute.