2.4. Configuration

When you install NetSpyglass from rpm or deb package, the package also installs prototype configuration file /opt/netspyglass/home/nw2.conf. This file has comments to document all parameters, however it does not include all possible configuration parameters. The server uses “default” configuration that it loads from built-in configuration file and then merges it with configuration defined in the file /opt/netspyglass/home/nw2.conf. Parameters defined in file /opt/netspyglass/home/nw2.conf override those from the default built-in configuration. You can find a copy of the built-in default configuration file in the file /opt/netspyglass/current/doc/default_config/netspyglass.conf. This file has all recognized configuration parameters with comments, however it is provided for reference and is just a copy; editing it is not going to have any effect. If you want to change some of the parameters you find in it, copy them to your configuration file /opt/netspyglass/home/nw2.conf and make the change there.

Configuration file has hierarchical structure and consists of individual variables, dictionaries and lists. Its syntax is losely based on JSON with some differences:

  • strings can be “bare” (unquoted) if they do not contain white space or special characters
  • list items do not need to be separated by comma when written one item per line
  • when value of a parameter is a dictionary, you can skip = or :
  • You can find more syntax rules and reference to the online documentation in Configuration file syntax

The syntax of the configuration file is described below in Configuration file syntax.

In the documentation we refer to the configuration parameters by combining enclosing dictionary name and the key name, separated by a dot. For example:

ui {
    url = "http://somehost:9100"
}

Parameter url can be referred to as ui.url in other parts of the configuration file. We also use this notation in the documentation.

In order to start using the application, edit at least the following parameters:

  • home: this should point to the directory where all files are located

  • ui.url: this is the url UI backend is going to listen on. The default binds it to localhost but you probably need to change this to make it accessible from outside of the machine. We recommend using machine’s IP address in ui.url. Port number 9100 is used by the UI is just the default and can be changed in case of conflicts. For example:

    ui {
        url = "http://192.168.1.100:9500/"
    }
    
  • network.name: the name of your network. The name should be in double quotes if it includes white space

  • network.polling: list of polling configurations. Each item in this list describes one SNMP polling configuration you have on your network. See below for more details on this.

  • network.devices: list of addresses and snmp communities of devices. See more on this below.

By default server uses embedded SQL database. We recommend to keep it for evaluations, but if you intend to use your own MySQL database, change the following parameter:

  • ui.storage.connectionUrl: this parameter tells UI backend how to connect to the database. Currently we support two databases: embedded HSQL database and MySQL. Prototype config file provides examples for both, including additional parameters where needed. The default is to use embedded database and place its files in the subdirectory “db” inside of the application home directory.

MySQL connection string includes database name and authentication credentials in the following format:

ui.storage.connectionUrl = "mysql://sqlserver.domain.com:3306/nw2mon?user=nw2&password=nw2pass"

here nw2mon is database name, nw2 is user name and nw2pass is the password.

Note

User nw2 must have priviliges allowing it full access to the database. Database must exist at the time of server startup.

See Databases for more information.

In addition to the paramerers listed above, here is a short list of the most basic configuration parameters you may need to change when you start using NetSpyGlass:

  • api.address: This parameter is useful when machine where NetSpyGlass is running is multi-homed and has several ip addresses and you want to use one address for the UI web server access and another for the intra-cluster communication. Note that parameter ui.url should specify address 0.0.0.0 in this case. Other NetSpyGlass cluster servers will use address specified in the parameter api.address to connect to the server.
  • monitor.pollingIntervalSec: polling interval in seconds. Default is 1 min
  • monitor.storage: dictionary of parameters that describe Time Series Database you want to use to store monitoring data. We support the following databases: embedded RRD-like storage, Graphite, InfluxDb and hbase. Prototype config file provides examples for all of these, with embedded RRD storage enabled by default. It is also possible to run without any permanent storage at all if you comment out all storage parameters. Parameter memoryPoolDurationHrs is required though.
  • network.name: the name of the network. This is going to appear in the UI and should be meaningful to you. Currently, NetSpyGlass supports only one network definition per application but this may change in the future.
  • network.devices: list of device addresses and snmp community strings. See below Adding Network Devices

Note

Since by default the server uses embedded SQL database and RRD storage, you can start minimal instance of the application by just editing home directory, connection urls and adding addresses of your devices in the network.devices section.

2.4.1. Polling Configurations

Device configuration refers to the polling configuration by name; actual polling configurations are defined in the section network.polling. Each entry in this section is a dictionary with the following items:

  • protocol: must be “snmp”
  • version: SNMP version (1, 2, 3). If this parameter is missing, the version is assumed to be 2.
  • community: used with versions 1 and 2. If the value of this parameter is a blank string (e.g. community = “”), devices using this SNMP configuration are not going to be discovered or polled but will appear in maps if L3 link to it could be constructed using information collected from another device
  • user: used with SNMP v3
  • password: corresponding password
  • secLevel: This is a string that defines security level for SNMPv3. The following levels are recognized: noAuthNoPriv, authNoPriv, authPriv
  • auth: authentication. MD5 and SHA are supported.

Default configuration file defines the following polling configurations:

polling = {

    v1public : {
        protocol : snmp,
        version : 1,
        community : public
    },

    v2public : {
        protocol : snmp,
        version : 2,
        community : public
    },

    v3md5public : {
        protocol : snmp,
        version : 3,
        secLevel : authNoPriv,
        auth : MD5,
        user : snmpuser,
        password : public
    },

}

To extend this section, define your own dictionary network.polling in your configuration file nw2.conf. Information from this section defined in the default configuration file and your own will be merged and used together by the server. Here is an example:

polling = {

    datacenter1 : {
        protocol : snmp,
        version : 2,
        community : communityForDC1
    },

    datacenter2 : {
        protocol : snmp,
        version : 2,
        community : communityForDC2
    },

    datacenter3 : {
        protocol : snmp,
        version : 3,
        secLevel : authNoPriv,
        auth : MD5,
        user : snmpuser,
        password : scretPwdForDC3
    },

}

Note

Limitations in SNMPv3 support in the current version of NetSpyGlass:

  • only USM security model is supported
  • in case security model authPriv is used (both authentication and privacy are turned on), the same password is used for both, and privacy is always set to DES (des56)

2.4.2. Adding Network Devices

By default, NetSpyGlass only polls devices that have been added to the configuration by the operator. It does not try to find devices on the network, unless explicitly configured to do so.

Configuration of the network devices that NetSpyGlass will monitor is located in the section network.devices. This section is a list of dictionaries where each dictionary describes one device.

Each device record consists of the following items:

  • address: this parameter defines ip address we use to talk the device. This can be either bare ip address or device name. If the latter, then the server will try to resolve the name into ip address using DNS. If the name does not resolve, device can not be used (the server logs error and skips this configuration entry). When device is identified by the ip address, the name for the corresponding device object in NetSpyGlass is assigned using the following sources: if the device responds to SNMP queries, then the value of sysName OID is used. If the device does not respond to SNMP queries, then the server uses reverse DNS lookup to find the name that corresponds to the specified ip address. If both methods fail, device name is going to be its ip address. Optional configuration parameter name can be used to assign the name to the device object when it is identified by its ip address in the parameter address and DNS PTR record does not exist for this address.
  • name: this optional parameter can be used to assign a name to the device object when it is identified by ip address, does not respond to SNMP queries (so its host name can not be determined) and when there is no DNS PTR record for its ip address
  • polling: a name of one of the polling configurations from the section network.polling or a string that consists of a comma-separated list of such names. If the value is just a single configuration name, then corresponding polling configuration is used. If the value of this parameter is a list of words, the server tries them in turn, assuming each word is a name of the polling configuration in the section network.polling. The server tries them at the beginning of the discovery process and uses the one that works.
  • snmpCommunity: This is a legacy parameter, it is ignored if parameter polling (see above) is used. The value of this parameter is R/O snmp community. Note that devices can have different community strings. If the value of this parameter is a blank string (e.g. snmpCommunity = “”“), then device is not going to be discovered or polled but will appear in maps if L3 link to it could be constructed from another device
  • snmpVersion: This is a legacy parameter, it is ignored if parameter polling (see above) is used. The value of this parameter is SNMP version that should be used for this device. Supported versions are 1 and 2. The value of this parameter should be a number as shown in the example below. If this parameter is missing, NetSpyglass uses SNMP version 2. Use parameter snmp if you need SNMPv3.

Here is an example:

devices = [

    { address = "10.0.14.122",                    polling = v2public },
    { address = "10.0.14.123",  name = "server1", polling = v2public },
]

2.4.2.1. Using Scripts to Add Devices

To automate the process of adding network devices to NetSpyGlass configuration, put all device definitions to a separate configuration file which can be generated by a script and then import it into the main configuration file nw2.conf. Suppose we want to keep device definitions in the file devices.conf. The contents of this file is going to look like this:

network {

    devices = [
        { address = "10.0.14.1", polling = v2public, },
        { address = "capsule",   polling = v2public, },

    ]
}

We include this file into the main configuration file:

network {

    name = test_network

    # Other configuration parameters for the network go here

}

include "devices.conf"

Operator include finds configuration files in the same directory where the file that includes them is located.

In this example, the dictionary network is present in both files nw2.conf and devices.conf and the include statement is located outside of this dictionary. This works because the two copies of the dictionary are merged when configuration is loaded and resolved and the server ends up using dictionary network with both keys name and devices.

2.4.3. Network Discovery

Once UI backend and monitor are running, connect to the url you configured in the ui.url configuration parameter with your browser. You should see the home page of the application with logo, network name and empty tiles. Start network discovery by clicking button “Discoverable”. The program will start network discovery and you’ll see a message “Discovery is in progress, started at … ” in the UI. It may take a few minutes to complete but once it is done, tiles should get populated with network maps and you should see discovery status (“Success” or “Failure”) next to the “Discoverable” button.

You can watch discovery progress in the log file logs/info.log

2.4.3.1. Schedule

the program will run network discovery on schedule described by parameters discovery.schedule or discovery.interval in the configuration file. Only one of these can be present. Parameter discovery.interval simply defines interval in hours, beginning from the moment when the program has started. Parameter discvoery.schedule is a string in the cron-like format. Here is an example:

discovery {
    # scheduled discovery, interval is in hours
    # interval = 24

    # Run discovery every day at 7am
    schedule = "0 0 7 * * ?"
}

The format is explained here:

http://quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-06

2.4.4. Configuration file syntax

Parsing of the configuration files is implemented using library “HOCON”: https://github.com/typesafehub/config

2.4.4.1. Basic rules of the config file format

  • Comments, with # or //
  • Allow omitting the {} around a root object
  • Allow = as a synonym for :
  • Allow omitting the = or : before a { so you can use foo { a : 42 }
  • Allow omitting commas as long as there’s a newline
  • Allow trailing commas after last element in objects and arrays
  • Allow unquoted strings for keys and values
  • Unquoted keys can use dot-notation for nested objects, foo.bar=42 means foo { bar : 42 }
  • Duplicate keys are allowed; later values override earlier, except for object-valued keys where the two objects are merged recursively
  • include feature merges root object in another file into current object, so foo { include "bar.json" } merges keys in bar.json into the object foo
  • include with no file extension includes any of .conf, .json, .properties
  • you can include files, URLs, or classpath resources; use include url("http://example.com") or file() or classpath() syntax to force the type, or use just include "whatever" to have the library do what you probably mean (Note: url()/file()/classpath() syntax is not supported in Play/Akka 2.0.)
  • substitutions foo : ${a.b} sets key foo to the same value as the b field in the a object
  • substitutions concatenate into unquoted strings, foo : the quick ${colors.fox} jumped
  • substitutions fall back to environment variables if they don’t resolve in the config itself, so ${HOME} would work as you expect. Also, most configs have system properties merged in so you could use ${user.home}.
  • substitutions normally cause an error if unresolved, but there is a syntax ${?a.b} to permit them to be missing.
  • += syntax to append elements to arrays, path += "/bin"
  • multi-line strings with triple quotes as in Python or Scala