.. _views: Views ***** .. highlight:: python :linenothreshold: 2 Views is a mechanism that allows the user to organize maps of a large network. Instead of always looking at a map that has 1000 devices, you can have multiple maps that show subset of the devices, grouped by some criteria that make sense to you. You can easily build maps to show OSPF areas or BGP peering or put devices in each data center into their own map and build an overview map to show how data centers are connected to each other. Each view appears in the UI as a tile in the home page. If views are organized into a hierarchy, you can "drill down" by clicking objects that represent lower level views in maps. In any case, there is exact one-to-one correspondence between views and maps. NetspyGlass offers programmatic mechanism you can use to match devices to views. This is done using Python script `views.py` that you can find in the directory `scripts` in the integrated Git repository Building Your Own Views ======================= Script `views.py` consists of the following parts: - few global functions used to illustrate different ways to match device attributes - list `VIEWS` that actually defines views - class `UserViewBuilder` that serves as an API between this script and NetSpyGlass When NetSpyGlass wants to create views, it loads this script, creates an instance of class :class:`UserViewBuilder` defined in this script and calls its function :func:`execute()`, which should return list of instances of the Python class :class:`View`. Default implementation of class :class:`UserViewBuilder` creates View objects from Python dictionaries defined as items in the list `VIEWS`. This is just a reference implementation, you can change it if you prefer different way of doing it. The only thing that really matters is that there should be class with method :func:`execute()` that returns list of `View` objects, everything elase is up to you. Function :func:`execute()` will be called with one argument, a list of :class:`PyDevice` objects. Each View object has the following attributes: - name (string) the name of the view - parent (string) The name of the parent view. This is optional, views that have no `parent` parameter are the top-level views - match_function a function of one argument. This function is called with PyDevice object representing a device. This function should return True if the device should be a member of the view. This attribute is optional, if it is missing, the view matches all devices. - adjacent_devices (boolean) if True, the view will also include devices one hop away from the devices that match this view. If this parameter is missing, its value is set to False - connecting_devices (boolean) if True, the view will also include devices that connect devices that match this view. Default value is False. The `match_function` parameter should be a function that is going to be called with an instance of Python class `net.happygears.nw2.py.py_wrappers.PyDevice.PyDevice`. This function can analyse device name, address or tags to decide if it should be a member of the view. NetSpyGlass uses parameters of the `View` object as follows: 1. all devices are passed to the `match_function` that decides which ones should appear in the map. If `View` does not have parameter `match_function`, then all devices match. 2. if parameters `adjacent_devices` or `connecting_devices` have value `True`, corresponding adjacent and connecting devices are added too. Examples in :ref:`view_examples` demonstrate different ways to match host attributes `name`, `address` and `tags`. You can use regular expressions (via Python module `re`) to match host name and tags. You can also use Python module `ipaddr` to match IP addresses instead of operating with addresses as strings. Embedded Python interpreter comes with almost full set of standard Python modules, all of them are available to you in the view builder script. To build your own views, start with editing list `VIEWS`. Remove or edit example views you do not need and add your own. NetSpyGlass monitors Python scripts and reloads then when they change. To upload your changes, commit and push to git repository. NetSpyGlass pulls from the repository on schedule with interval 2 min so your script should appear there in a couple of minutes. NetSpyGlass rebuilds views after it finishes discovery of each device or on command. If you want to rebuild views manually to test your script, use command `nsgcli make views` on command line, or navigate to System / Actions in the UI and click "Update Views". .. _view_examples: Examples of Views ================= the following script creates three views: "all" (includes all devices), "Backbone" (includes devices that are members of OSPF area 0) and "Peering" (includes only devices that have BGP peering with other ASes than their own, that is, run eBGP):: import view def match_ospf_area_0(host): return 'OSPFArea.0' in host.tags def match_bgp_speaker(host): return 'Role.eBgpPeer' in host.tags VIEWS = [ view.View(name='all'), view.View( name='backbone', match_function=match_ospf_area_0, ), view.View( name='Peering', match_function=match_bgp_speaker, ), ] class UserViewBuilder(object): def __init__(self, log): self.log = log def execute(self): """ Generate list of View objects to describe views. Views can refer to each other by name using attribute "parent" to establish hirarchy. This function only creates "blank" views, it does not add devices to them. :return: list of View objects """ return VIEWS Matching Rules for Hierarchical Views ------------------------------------- Consider the following views forming three-level hierarchy: .. aafig:: :aspect: 60 :scale: 100 :proportional: :textual: "clusters" | +---- "Cluster1" | | | +---- "Cluster1Core" | +---- "Cluster2" The general rule is that the matching function of each view is called with devices that were determined to belong to the view of the upper level. In this example, matching function of the view "clusters" would be called with all devices known to the system. The function may pick some of them or may be allow all of them to be part of the view "clusters". Next, the system will call matching function of view "Cluster1", but this time, it will check only devices that are members of "clusters". On the next level, while building view "Cluster1Core", the system will only examine devices that are members of "Cluster1". Once devices have been allocated to views, NetSpyGlass builds maps. This is done "bottom-up", that is, it builds lower level views first. For each view, it adds devices to the map and checks if it has lower level view. If this is the case, it then removes devices that are members of the lower level view from the map and replaces them with a symbol that will represent that view. Finally, it reconnects network links that used to be terminated on the devices but should now connect to the symbol representing lower level view. Class View ---------- .. automodule:: view :members: :undoc-members: :show-inheritance: Examples of Hierarchical Views ============================== There are many uses for nested views, here are a few Clusters -------- this is probably the most typical use for the nested views. The idea is to build at least two levels of views: the top level shows clusters (or data centers) and links that connect them. Clicking any cluster opens up lower level map that shows only devices located in this cluster and possibly adjacent devices, too. Example file `views_example.py` (located in the directory `doc` of the distribution package) demonstrates how this is done. Assuming we have three clusters "SJ", "EQ" and "RD", we can build views as follows:: SJ_SUBNETS = [ ipaddr.IPNetwork('10.3.1.0/24'), ipaddr.IPNetwork('10.3.2.0/24'), ipaddr.IPNetwork('10.3.3.0/24'), ipaddr.IPNetwork('10.3.4.0/24') ] def match_rd_cluster(dev): return re.search('(\.rd$)|(^rd.*)', dev.name, re.IGNORECASE) is not None def match_eq_cluster(dev): return re.search('(\.eq$)|(^eq.*)', dev.name, re.IGNORECASE) is not None def match_sj_cluster(dev): if re.search('(\.sj$)|(^sj.*)', dev.name, re.IGNORECASE): return True addr = ipaddr.IPAddress(dev.address) return any(addr in net for net in SJ_SUBNETS) VIEWS = [ { 'name': 'clusters', }, { 'name': 'RD', 'parent': 'clusters', 'match_function': match_rd_cluster, 'adjacent_devices': True }, { 'name': 'EQ', 'parent': 'clusters', 'match_function': match_eq_cluster, 'adjacent_devices': True }, { 'name': 'SJ', 'parent': 'clusters', 'match_function': match_sj_cluster, 'adjacent_devices': True }, ] class UserViewBuilder(object): """ NetSpyGlass uses this class and its functions as an API to the view configuration managed by the user. :param log: system logger object """ def __init__(self, log): self.log = log def execute(self, devices): """ Generate list of View objects to describe views. Views can refer to each other by name using attribute "parent" to establish hirarchy. This function only creates "blank" views, it does not add devices to them. :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass) :rtype : list of strings :return: list of View objects """ return [view.View(x) for x in VIEWS] Here, we use combination of criteria to decide which cluster given device should belong to. For RD and EQ we rely on the naming convention (match device names using some regular expressions), but for SJ we match both names and ip addresses. The result is view "clusters" that shows just three objects "RD", "EQ" and "SJ", connected with links that display combined values of monitoring variables, such as traffic level. Clicking any of these objects opens map of the lower level with devices located in the corresponding cluster. Virtualization -------------- NetSpyGlass can discover internal network configuration in VMWare ESX virtualization hosts and determine virtual network topology. It associates devices to virtual machines and corresponding virtualization hosts and determines how they are connected together and to the outside network via virtual switches and port groups. This means you can build maps showing VMs in the context of both virtual and physical networks. No special configuration is required to build composite map showing both virtual and physical switches, this is the default. Virtual switches will appear in maps as objects with the name composed from the name of the virtualization host and the name of the virtual server, such as "esxi55-1[Public]". Here "Public" is the name of the virtual switch. NetSpyGlass assigns tags in facet "VmHost" to devices it determines to be virtual machines and objects it creates to describe virtual switches. The value of this tag is the name of the corresponding virtualization host. For example, if virtualization host "esxi55-1" has VM "linux-test-1" and virtual switch "Public", device "linux-test-1" gets tag "VmHost.esxi55-1". Object created to descripe virtual switch "Public" gets the same tag as well. This allows us to build nested views to put all VMs and virtual switches inside of one view. The following configuration defines top level view "virtualization" that has two lower level views "esx1" and "esx2" that contain all VMs and virtual switches hosted by vm servers "esxi55-1" and "esxi55-2":: def match_esxi1(host): return 'VmHost.esxi55-1' in host.tags or host.name == 'esxi55-1' def match_esxi2(host): return 'VmHost.esxi55-2' in host.tags or host.name == 'esxi55-2' VIEWS = [ { 'name': 'virtualization', 'match_function': match_virt }, { 'name': 'esxi1', 'match_function': match_esxi1, 'parent': 'virtualization', 'adjacent_devices': True }, { 'name': 'esxi2', 'match_function': match_esxi2, 'parent': 'virtualization', 'adjacent_devices': True }, } class UserViewBuilder(object): """ NetSpyGlass uses this class and its functions as an API to the view configuration managed by the user. :param log: system logger object """ def __init__(self, log): self.log = log def execute(self, devices): """ Generate list of View objects to describe views. Views can refer to each other by name using attribute "parent" to establish hirarchy. This function only creates "blank" views, it does not add devices to them. :param devices: list of :class:`PyDevice` objects (all devices monitored by NetSpyGlass) :rtype : list of strings :return: list of View objects """ return [view.View(x) for x in VIEWS] Views "esxi1" and "esxi2" match tag "VmHost" or device name and add adjacent devices. This way, each view contains all VMs, virtual switches and the ESXi host itself, as well as adjacent devices such as switch the host is connected to.