Java classes that can be accessed from Python scripts ***************************************************** .. highlight:: python :linenothreshold: 2 Summary ======= Java classes that can be accessed from Python scripts that build monitoring variables, process monitoring data or build reports. Even though these are Java classes, they have method `__getattr__()` that provides standard "Pythonic" way to access their properties as class attributes. That is, you can call both Java "getter" method: .. code-block:: none device.getName() and access the same property as an attribute: .. code-block:: none device.name We also provide method `get()` that works exactly like `__getattr__()` and is intended for Velocity templates used to build reports. This means you can write: .. code-block:: none $device.name in the report template instead of: .. code-block:: none $device.getName() both methods will work though. Since these wrapper classes are intended to be used with Python scripts, some attributes were given alternative names to conform with PEP8. For example, attribute :py:attr:`PyDevice.sys_name` is mapped to the Java getter method :py:meth:`PyDevice.getSytsName()`. This document lists only attribute names. Classes and Functions ===================== MonitoringVariable ------------------ .. class:: net.happygears.nw2.py.MonitoringVariable This class represents single instance of a monitoring variable. An object of this class holds information about one interface, hardware component, a protocol counter or any other entity that we monitor. Attributes ^^^^^^^^^^ .. py:attribute:: device The name of the device this variable instance collects monitoring data for .. py:attribute:: component The name if the interface or hardware component this variable collects monitoring data for. .. py:attribute:: iface This attribute returns the same value as *MonitoringVariable.component*. This attribute is deprecated and will be removed in future versions, please use *MonitoringVariable.component* instead. .. py:attribute:: ds Reference to an instance of class :py:class:`DataSource` (see below) .. py:attribute:: statistics :py:class:`net.happygears.nw2.py.Statistics` object (see below) .. py:attribute:: timeseries :py:class:`net.happygears.nw2.time_series_buffer.TimeSeries` object (see below) .. py:attribute:: tags :noindex: Tags: unmodifiable set of strings, each string is in the form "TagFacet.TagWord" Use this attribute to check if the variable has a tag using Python `in` operator:: if 'Vendor.Cisco' in mvar.tags: # do something if this is Cisco device To add new tag to the variable use method `addTag()` Instance Methods ^^^^^^^^^^^^^^^^ Not all methods of the Java class MonitoringVariable are accessible from Python. Here are few that can be useful: .. py:method:: addTag(tag_string) Add tag to the variable. Tag should be in the form of a two-component string, speparated with a dot: `Facet.Word`. Examples:: Vendor.Cisco ColorLevel.1 ifOperStatus.Up For example:: mvar.addTag('Aggregate.PaidPeering') .. py:method:: removeTag(tag_string) Remove given tag from the set of tags of the variable. Note that this does not nothing if the tag belongs to the device or its component (those are read-only). .. py:method:: getTags(facet) return unmodifiable set of tags that belong to the specified tag facet. This is different from the attribute :py:attr:`tags`: or method :py:func:`getTags()` (with no arguments) that returns full set of tags of the variable, call to :py:func:`getTags()` returns only subset of tags that belong to the specified tag facet. For example, you can get ifIndex of the parent interface this way:: assert isinstance(var, MonitoringVariable) for tag in var.getTags('ifParent'): # tag looks like 'ifPArent.5', where the number is ifIndex of parent interface parent_if_index = int(tag.split('.')[1]) .. py:method:: add(observation) Append new observation to the end of the list `MonitoringVariable.timeseries` .. py:method:: add(observations) Merge observations from the list `observations` with observations in the list `MonitoringVariable.timeseries`. This method sorts observations by the time stamp and makes sure new ones are inserted in the right position. Old and new observations with identical timestamp are deduplicated. .. py:method:: double getMinValue() calculates and returns minimum value by comparing values of all observations in the time series of this monitoring variable stored in the memory buffer. Returns result as a number .. py:method:: double getMaxValue() calculates and returns maximum value by comparing values of all observations in the time series of this monitoring variable stored in the memory buffer. Returns result as a number .. py:method:: __cmp__(other) compare values of the latest data points in time series of `self` and `other`. This makes it possible to sort lists of MonitorngVariable instances or use built-in Python functions `max()` and `min()`. .. py:method:: __cmp__(value) compare value of the latest data point in the time series of `self` with constant. .. py:method:: __add__(other) .. py:method:: __sub__(other) .. py:method:: __mul__(other) .. py:method:: __div__(other) Arithmetic operations between latest values of the time series of two :class:`MonitoringVariable` objects `this` and `other`. All these functions return copy of `this` with time series modified so it has just one data point with calculated value. `tags` in the variable retain their values only if they are the same in `this` and `other` (in other words, after the call to these functions the set of tags is an intersection of sets of tags of two operands before the call). These functions enable calls to Python built-ins `sum()`, `reduce()` and others. For example, call to `sum()` can be used to to calculate sum of values of monitoring variables instances in a list. .. py:method:: __add__(k) .. py:method:: __sub__(k) .. py:method:: __mul__(k) .. py:method:: __div__(k) The same as the above, but operates with constant `k` as a second argument. .. py:method:: __radd__(k) .. py:method:: __rsub__(k) .. py:method:: __rmul__(k) .. py:method:: __rdiv__(k) These versions of the arichmetics functions operate with reversed arguments and enable Python expressions such as:: k + mvar k * mvar where `k` is a constant and `mvar` is an instance of :class:`MonitoringVariable`. These magic functions make it possible to write in Python:: aggregate_var = aggregate_var + input_traffic_rate * 60 where both `aggregate_var` and `input_traffic_rate` are both instances of :class:`MonitoringVariable`. Expression used in this example can be used to calculate total amount of input or outputtraffic (in bytes) through interface. This formula can be used in combination with call to :func:`reduce()`. Statistics ---------- .. class:: net.happygears.nw2.py.Statistics Object of this class provides functions that can be used to perform basic statistical calculations over the time series of the parent monitoring variable. Example:: def is_outlier(mvar, value, n_sigmas): assert isinstance(mvar, MonitoringVariable) mean = mvar.statistics.mean() sigma = mvar.statistics.sigma() return value != mean and abs(value - mean) >= (n_sigmas * sigma) Methods ^^^^^^^ .. py:method:: double mean() returns a number equal to the mean value of observations in the whole time series stored in memory. Observations with value `NaN` are skipped. Note that this operates only on the data in memory. The length of the time series in memmory is determined by the value of configuration parameter `monitor.storage.retentionHrs` .. py:method:: double mean(begin, length) returns a number equal to the mean value of :py:obj:`length` non-NaN observations in the time series beginning with observation with index :py:obj:`begin`. Observations with value `NaN` are skipped and do not count towards :py:obj:`length`. If `begin + length` extends beyond the end of the time series buffer, this function stops at the last element. Note that this operates only on the data in memory. .. py:method:: double median() returns a number equal to the median value of observations in the whole time series stored in memory. Observations with value `NaN` are skipped. Note that this operates only on the data in memory. The length of the time series in memmory is determined by the value of configuration parameter `monitor.storage.retentionHrs` .. py:method:: double median(begin, length) returns a number equal to the median value of :py:obj:`length` non-NaN observations in the time series beginning with observation with index :py:obj:`begin`. Observations with value `NaN` are skipped and do not count towards :py:obj:`length`. If `begin + length` extends beyond the end of the time series buffer, this function stops at the last element. Note that this operates only on the data in memory. .. py:method:: double sigma() returns a number equal to the standard deviation of values of observations in the whole time series stored in memory. Observations with value `NaN` are skipped. Note that this operates only on the data in memory. The length of the time series in memmory is determined by the value of configuration parameter `monitor.storage.retentionHrs` .. py:method:: double sigma(begin, length) returns a number equal to the standard deviation of :py:obj:`length` non-NaN observations in the time series beginning with observation with index :py:obj:`begin`. Observations with value `NaN` are skipped and do not count towards :py:obj:`length`. If `begin + length` extends beyond the end of the time series buffer, this function stops at the last element. Note that this operates only on the data in memory. TimeSeries ---------- .. class:: net.happygears.nw2.time_series_buffer.TimeSeriesBuffer Object of this class represents single time series, that is, a set of timestamp-value pairs that represents monitoring data collected for single :py:class:`net.happygears.nw2.py.MonitoringVariable` instance. This behaves like a list where items are objects of the class :py:class:`net.happygears.nw2.py.Observation`, so you can iterate over it and access items by index. Unlike in case of the standard Python lists, functions that access observations return a copy of the data stored in the time series. You do not need to create objects of this class manually, instead, you always access them as attribute `timeseries` of :py:class:`net.happygears.nw2.py.MonitoringVariable`:: assert isinstance(mvar, MonitoringVariable) for observation in mvar.timeseries: print observation Methods ^^^^^^^ .. py:method:: long getLastTimeStamp() returns time stamp of the last observation in the time series .. py:method:: double getLastValue() returns value of the last observation in the time series. This can be a NaN. .. py:method:: getLastNonNaNValue() Return value of the latest observation that is not a NaN regardless of its timestamp and how far back in time that was .. py:method:: boolean isLastNaN() checks if the last observation in the time series is a NaN. .. py:method:: void put(long timestamp, double value) Puts new observation into the time series buffer .. py:method:: void clear() clears time series buffer. .. py:method:: Observation first() returns first observation as :class:`Observation` object .. py:method:: Observation last() returns last observation as :class:`Observation` object .. py:method:: long getTimeStamp(int idx) returns time stamp of the observation identified by its index :py:obj:`idx` .. py:method:: double getValue(int idx) returns value of the observation identified by its index :py:obj:`idx` .. py:method:: double getLastValue() returns the value of the last observation in the time series. If the time series holds numeric values, this function returns the last one. If the time series holds strings, this function returns `NaN`. .. py:method:: String getLastValueAsString() returns the value of the last observation in the time series. This function returns string representation of the last value regardless of its type, that is, if the time series holds numeric values, this function returns formatted value as a string. .. py:method:: Observation get(int idx) returns observation identified by its index :py:obj:`idx` as :class:`Observation` object .. py:method:: void updateLastValue(final double value) update value of the last observation in the buffer .. py:method:: double getMinValue() calculates and returns minimum value by comparing values of all observations in the time series buffer. Returns result as a number .. py:method:: double getMaxValue() calculates and returns maximum value by comparing values of all observations in the time series buffer. Returns result as a number Python "magic" methods ^^^^^^^^^^^^^^^^^^^^^^ .. py:method:: __len__() returns the size of the time series .. py:method:: __nonzero__() returns true if the time series is not empty. This function enables the following Python expression:: if mvar.timeseries: # do something when time series is not empty .. py:method:: Observation __getitem__(int key) returns :class:`Observation` object identified by its index :py:obj:`key`. With this function we can access observations in the time series as if it was a list:: mvar.timeseries[1] negative index is also supported:: mvar.timeseries[-1] slices are supported too:: mvar.timeseries[1:10] Observation ----------- .. class:: net.happygears.nw2.py.Observation() Attributes ^^^^^^^^^^ .. py:attribute:: timestamp a time stamp in milliseconds (long) .. py:attribute:: value :noindex: value of the observation. This is stored as an instance of Java class Number and can be used in Python directly. This can be NaN. Methods ^^^^^^^ .. py:method:: Observation(timestamp) :param timestamp: observation time stamp (time in milliseconds) Create an Observation object with given timestamp and value of NaN .. py:method:: Observation(timestamp, value) :param timestamp: observation time stamp (time in milliseconds) :param value: observation value (a number) Create an Observation object with given timestamp and value .. py:method:: isNaN() Returns True of the value of this Observation is NaN. Python "magic" methods ^^^^^^^^^^^^^^^^^^^^^^ .. py:method:: __cmp__(other) .. py:method:: __eq__(value) .. py:method:: __ne__(value) .. py:method:: __lt__(value) .. py:method:: __le__(value) .. py:method:: __gt__(value) .. py:method:: __ge__(value) "rich" comparison methods are supported and compare Observation value, ignoring its timestamp. .. py:method:: __str__(other) Return string representation of the Observation, useful for logging and reports. Timestamp is printed in seconds. DataSource ---------- .. class:: net.happygears.nw2.py.DataSource Instance of this class are creaeted internally when the system prepares monitoring variables used for device polling. `MonitoringVariable` instances hold reference to the instance of `DataSource` class. You should not need to create objects of this class in the Python script. Attributes ^^^^^^^^^^ .. py:attribute:: name :noindex: The name of the data source. This matches the name of the interface or hardware component this data source describes. .. py:attribute:: index :noindex: index is equal to ifIndex if Data Source corresponds to an interface or some kind of unique index of hw component that we have learned while discovering device components. .. py:attribute:: container Container describes hw component that contains the component named by attribute "name". Not all components have containers, this depends on the device vendor. .. py:attribute:: description :noindex: Component description. In case of an interface, this is interface description. .. py:attribute:: unit Units in which collected data is measured. For example, for the variables that collect data from temperature sensors, this could be 'C'. DataScaler ---------- .. class:: net.happygears.nw2.py.DataScaler .. py:method:: DataScaler(DataSource ds) :param DataSource ds: input DataSource object this scaler takes into account pre-set data range it takes from the data source. Since it does not have access to the timeseries, it can not take into account actual values. Methods ^^^^^^^ .. py:method:: DataScaler(MonitoringVariable mvar) :param MonitoringVariable mvar: monitoring variable to take values from this constructor has access to both DataSource object (a member of MonitoringVariable) and timeseries. Scaler will compute the range using actual values of observations in the time series .. py:method:: DataScaler.scaleValue(Observation o) :param Observation o: input Observation object Takes an Observation object and returns a number equal to the scaled value of this observation .. py:method:: DataScaler.getPrefix() returns calculated prefix. .. py:method:: DataScaler.getUnit() returns the same basic unit it copies form the DataSource object Example:: assert isinstance(mvar, MonitoringVariable) scaler = DataScaler(mvar) value = mvar.last() print scaler.scaleValue(value) + ' ' + scaler.getPrefix() + scaler.getUnit() Wrappers ======== The following classes are Python wrappers for the Java classes used to hold information about devices, network interfaces, hardware components etc. These classes are used in variable and view builder scripts. PyDevice -------- .. class:: net.happygears.nw2.py.py_wrappers.PyDevice Attributes ^^^^^^^^^^ .. py:attribute:: name :noindex: Device name. This can be one of the following: - if config file `nw2.conf` mentions this device by name, then the value of this attribute is taken from the config. - if the device is configured by its ip address, then the program tries to use the value returned by sysName OID when we discover the device. This is usually the "hostname" configured on the device. - if we could not get the value of sysName, then the program runs reverse DNS lookup for the address of the device and assigns the result to this attribute. - if reverse DNS lookup query fails, the value of this attribute is equal to the ip address of the device. .. py:attribute:: sys_name Device name returned by the System MIB `sysName` OID. This is usually host name configured on the device .. py:attribute:: reverse_dns DNS PTR record configured for this device's IP address .. py:attribute:: hostname Device name with all domain parts stripped. If device is configured with short host name, the value of this attribute is equal to it. If the host name configured on the device is really FQDN, then this attribute is equal to its leftmost component. .. py:attribute:: location Device location configured in the System MIB `sysLocation` OID .. py:attribute:: contact Contact name configured in the System MIB `sysContact` OID .. py:attribute:: sys_descr Device description returned by the System MIB `sysDescr` OID. .. py:attribute:: box_descr Device's hardware description .. py:attribute:: sw_rev Software revision running on the device .. py:attribute:: address IP address used to communicate with the device .. py:attribute:: tags :noindex: Tags of this device as a set of strings. You can manipulate tags by adding or removing items of this set. Setting new value to this attribute is not supported. Instance Methods ^^^^^^^^^^^^^^^^ .. py:method:: getTagsInFacet(facet) returns list of tags that belong to the specified tag facet. This is different from the attribute :py:attr:`tags`: where the attribute returns full set of tags of the variable, call to :py:func:`getTagsInFacet()` returns only subset of tags that belong to the specified tag facet. .. py:method:: getNode() returns reference to the underlying :py:class:`NetworkNode` object. .. py:method:: forwardingClasses() returns set of formwaridng class numbers configured on the device .. py:method:: getQueueNum(int fcNumber) :param fcNumber: forwarding class number returns Cos queue number for the given forwarding class number .. py:method:: getFcName(int fcNumber) :param fcNumber: forwarding class number returns forwarding class name for the given forwarding class number .. py:method:: getFcNumber(int queueNumber) :param queueNumber: queue number returns forwarding class number for the given queue number .. py:method:: getVmHostName() returns VM host name if this device is virtual machine, otherwise returns empty string .. py:method:: getPyInterfaces() returns list of :py:class:`net.happygears.nw2.py.py_wrappers.PyNetworkInterface` objects that represent network interfaces of this device .. py:method:: getInterface(int ifIndex) return :py:class:`net.happygears.nw2.py.py_wrappers.PyNetworkInterface` object that represents network interface with given `ifIndex`. If such interface does not exist, this method returns `None`. PyNetworkInterface ------------------ .. class:: net.happygears.nw2.py.py_wrappers.PyNetworkInterface Attributes ^^^^^^^^^^ .. py:attribute:: name :noindex: Interface name .. py:attribute:: description :noindex: Interface description .. py:attribute:: abbreviated_name Abbreviated name of the interface. NetSpyGlass uses internal translation table to generate abbreviated names. For example, "GigabitEthernet" translates to "ge" and "TenGigabitEthernet" translates to "te". .. py:attribute:: address First address and netmask of the interface as a string in "address/prefixlen" notation .. py:attribute:: addresses List of all addresses of the interface as strings in in "address/prefixlen" notation .. py:attribute:: tags :noindex: Tags of this interface as a set of strings. You can manipulate tags by adding or removing items of this set. Setting new value to this attribute is not supported. .. py:attribute:: if_index Interface ifIndex .. py:attribute:: if_speed Value returned by ifSpeed OID .. py:attribute:: if_high_speed Value returned by ifHighSpeed OID if it is supported .. py:attribute:: isis_circuit Id of ISIS circuit terminating on this interface .. py:attribute:: cos_queues all QoS queues configured on the interface as a set of numbers PyHardwareComponent ------------------- .. class:: net.happygears.nw2.py.py_wrappers.PyHardwareComponent Attributes ^^^^^^^^^^ .. py:attribute:: name :noindex: Component name .. py:attribute:: index :noindex: Component index. This index is guaranteed to be unique within parent device. Hardware components and interfaces use separate indexes so h/w component index may conisidentally be the same as ifIndex of some interface. In most cases h/w component index is derived from the OID of the component but some times can be computed (this depends on the vendor and SNMP MIB used to discover the component) .. py:attribute:: description :noindex: Component description if available .. py:attribute:: model Component model if available .. py:attribute:: oid SNMP OID we use to query to get component status or current value if it is a sensor .. py:attribute:: sensor_scale Values returned by queries to the oid are multiplied by this coefficient to normalize them. For example, if SNMP MIB says that some sensor reports value in units of 0.01A, then sensor_scale may be set to 100 to normalize values and record them in units of 1A. This makes it possible to maintain common database of values for devices from different vendoes that might report values in different units. .. py:attribute:: sensor_data_type (a string) this attribute describes the unit this sensor uses and has one of the following values: - other - unknown - voltsAC - voltsDC - amperes - watts - hertz - celsius - percentRH - rpm - cmm - truthvalue - specialEnum - dBm this list will grow over time as we add support for new sensors. If hardware component is not a sensor, the value of this attribute is "other" .. py:attribute:: component_class (a string) this attribute describes high level class the component belongs to and can be one of the following: - other - unknown - chassis - backplane - container - powerSupply - fan - sensor - module - port - stack - cpu - opticalTransceiver - chassisAlarm - pdu - disk - memory this list will grow over time as we add support for new component classes. .. py:attribute:: tags :noindex: Tags as a set of strings PyChassisAlarm -------------- .. class:: net.happygears.nw2.py.py_wrappers.PyChassisAlarm :py:class:`net.happygears.nw2.py.py_wrappers.PyChassisAlarm` is a special case of :py:class:`net.happygears.nw2.py.py_wrappers.PyHardwareComponent` that has attribute :py:attr:`component_class` with value `chassisAlarm`. This object represents a counter of minor or major chassis alarms raised by the device. This class has all attributes of :py:class:`net.happygears.nw2.py.py_wrappers.PyHardwareComponent` and one additional attribute: Attributes ^^^^^^^^^^ .. py:attribute:: type (a string) alarm type, can have values 'major' or 'minor' PyProtocolDescriptor -------------------- .. class:: net.happygears.nw2.py.py_wrappers.PyProtocolDescriptor This class is an abstraction that describes parameter or a counter of a protocol. For example, this can be BGP peer state or number of BGP updates received from a peer and so on. Attributes ^^^^^^^^^^ .. py:attribute:: name :noindex: Component name .. py:attribute:: index :noindex: Object index. This index is guaranteed to be unique within parent device. In most cases index is derived from the OID of the component but some times can be computed (this depends on the vendor and SNMP MIB used to discover the component) .. py:attribute:: description :noindex: Component description if available .. py:attribute:: oid SNMP OID we use to query to get current value of the counter .. py:attribute:: sensor_scale Values returned by queries to the oid are multiplied by this coefficient to normalize them. .. py:attribute:: protocol (a string) the name of the network protocol, such as 'OSPF' or 'BGP4' PyNetworkGraph -------------- .. class:: net.happygears.nw2.py.py_wrappers.PyNetworkGraph Objects of this class wraps internal class that represents network topology graph. Instance Methods ^^^^^^^^^^^^^^^^ .. py:method:: get_connected_device(device, intf) :param device: :py:class:`net.happygears.nw2.py.py_wrappers.PyDevice` object that represents device :param intf: :py:class:`net.happygears.nw2.py.py_wrappers.PyNetworkInterface` object that represents network interface of the same device :return: a tuple `(opp_dev, opp_intf)` where `opp_dev` is :py:class:`net.happygears.nw2.py.py_wrappers.PyDevice` object that represents device on the opposite end of the network link terminated on `dev:intf` and `opp_intf` is an :py:class:`net.happygears.nw2.py.py_wrappers.PyNetworkInterface` object that represents corresponding interface of `opp_dev`. If interface `intf` is not connected anywhere, this function returns `None`. You can find example of this method usage in section :ref:`tags_examples`. Examples ======== The following examples are taken from the interface variables builder script, you can find a copy of this script in `python/variable_builders/interface.py` in your NetSpyGlass installation. Here is an example of a function that accesses network interface tags to check some conditions:: def is_interesting_interface(self, intf): """ This function implements some basic checks to decide if we want to monitor this interface :type intf: PyNetworkInterface :param intf: PyNetworkInterface wrapper object :return: True if we want to monitor this interface """ return ('ifAdminStatus.Up' in intf.tags and not 'ifRole.LoopbackInterface' in intf.tags and not 'ifRole.OutOfBandManagement' in intf.tags and not 'ifRole.SimulatedInterface' in intf.tags and not 'ifRole.Internal' in intf.tags) Next example demonstrates access to attributes `if_index` and `if_speed` of network interface object:: def make_basic_vars(self, intf): """ Generate configuration for the basic set of monitoring variables for the interface: interface utilization, errors, discards :type intf: PyNetworkInterface :param intf: PyNetworkInterface wrapper object :return: a dictionary where the key is variable name and value is another dictionary """ if_index = intf.if_index in_octets_oid = 'IF-MIB:ifInOctets.{0}'.format(if_index) out_octets_oid = 'IF-MIB:ifOutOctets.{0}'.format(if_index) if_speed_oid = 'IF-MIB:ifSpeed.{0}'.format(if_index) if_speed_scale = 1.0 if intf.if_high_speed > 0: in_octets_oid = 'IF-MIB:ifHCInOctets.{0}'.format(if_index) out_octets_oid = 'IF-MIB:ifHCOutOctets.{0}'.format(if_index) if_speed_oid = 'IF-MIB:ifHighSpeed.{0}'.format(if_index) if_speed_scale = 1E6 # ifHighSpeed is in 1,000,000 bits/sec return { # --------------------------------------------------------------- 'ifSpeed': { 'component': intf, 'snmp_oid': if_speed_oid, 'scale': if_speed_scale }, 'ifHCInOctets': { 'component': intf, 'snmp_oid': in_octets_oid }, 'ifHCOutOctets': { 'component': intf, 'snmp_oid': out_octets_oid } }