NetSpyGlass Server JSON API *************************** .. highlight:: json :linenothreshold: 2 .. role:: python(code) :language: python .. toctree:: :maxdepth: 1 API v2.0 ======== UI -- GET /v2/ui/net/:id/status ^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve status (defined network, discovery status and views) GET /v2/nsg/discovery/net/:id/start ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Starts network discovery GET /v2/ui/net/:id/dashboards/device/:id ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call serves the first ("Summary") page of the device panel GET /v2/ui/net/:id/devices ^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information about monitored devices (discovery data) GET /v2/ui/net/:id/devices/:devid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information about device identified by its id (discovery data) GET /v2/ui/net/:id/data/device/:devid?[format=(brief|full)][category=:name] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information about device identified by its id (discovery data) with ability to choose format. GET /v2/ui/net/:id/popup/device/:devid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call returns information used to build popup graph that opens when user clicks device label in maps. This popup displays graph built with monitoring data. GET /v2/ui/net/:id/popup/device/:devid/intf/:ifIndex ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call returns information used to build popup graph that opens when user clicks link label in maps. This popup displays graph built with monitoring data. GET /v2/ui/net/:id/graph/:triplet,:triplet ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call returns information used to build monitoring graphs GET /v2/ui/net/:id/data/:triplet,:triplet ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call returns collected monitoring information without formatting and scaling needed to build graphs GET /v2/ui/net/:id/views/:viewid/map ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information used to build network map in the UI for the view identified by its id GET /v2/ui/net/:id/views/:viewid/map?layout=auto ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ runs automatic map layout algorithm and returns newly built map GET /v2/ui/net/:id/views/:viewid/map?layout=reset ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reset coordinates of all devices in this map, this makes devices "float" GET /v2/ui/net/:id/views/:viewid/map?layout=fit ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ try to fit the map into browser window by scaling coordinates of all devices linearly GET /v2/ui/net/:id/views/:viewid/linkGraphVars ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of variables defined in the configuration file with parameter `deviceDetailsComponents.Interface` GET /v2/ui/net/:id/views/:viewid/linkVar/:lv/nodeVar/:nv ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns monitoring data for the map defined by view id for the link variable `lv` and device variable `nv` PUT /v2/ui/net/:id/views/:viewid/coordinates ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ updates coordinates of one or several devices in the map defined by the view id. Input data is expected in the request body in JSON format:: [ { 'id': nodeId1, 'x': x1, 'y': y1 }, ... ] POST /v2/ui/net/:id/keyvalue[/:handle] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If `handle` is not specified, takes data passed in the body of the request and creates new record in the database, then returns handle that can be used later to retrieve it. If record with the same data already exists, its handle is returned. The data is treated as an upaque text block (it does not have to be JSON but typically probably will be). The handle is returned in JSON format as follows: .. code-block:: none { "handle" : "1" } If `handle` is specified, updates data associated with handle :handle. If handle with this id does not exist, returns 404. Data is passed in the body of the request just like in case of POST call described above. GET /v2/ui/net/:id/keyvalue/:handle ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ given handle, returns associated data. In case of invalid handle, returns 404. GET /v2/ui/net/:id/gw/variables?rule_spec ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ see below: :ref:`rule_spec` returns list of items that describe monitoring variables that match matching rule `rule_spec`. GET /v2/ui/net/:id/gw/catalog/categories/list ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of variable categories GET /v2/ui/net/:id/gw/catalog/categories/:category ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of variable names in given category GET /v2/ui/net/:id/gw/catalog/variables/:v1,:v2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information about monitoring variables with names `v1`, `v2`, etc. GET /v2/ui/net/:id/gw/catalog/variables/:v1,:v2/devices/:dev1,:dev2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information about monitoring variables with names `v1`, `v2` limited to devices `dev1`, `dev2` etc. GET /v2/ui/net/:id/gw/catalog/tags/facets/list ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of tag facets not filtered by any variables or devices Actions ------- GET/POST /v2/ui/net/:id/actions/reload/devices GET/POST /v2/ui/net/:id/actions/make/views GET/POST /v2/ui/net/:id/actions/reload/config GET/POST /v2/ui/net/:id/actions/make/variables GET/POST /v2/ui/net/:id/actions/expire/variables GET/POST /v2/ui/net/:id/actions/alerts/purge/queue GET/POST /v2/ui/net/:id/actions/make/maps GET/POST /v2/ui/net/:id/actions/dump GET/POST /v2/ui/net/:id/actions/rebalance GET/POST /v2/ui/net/:id/actions/tsdb/reconnect Grafana ------- POST /v2/query/net/:id/categories/ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of known variable categories POST /v2/query/net/:id/variables/:category ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of variable names in the category POST /v2/query/net/:id/data/ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information selected by the parameters of the query sent via request body in json format body for the request "POST /v2/query/net/:id/data" :: { "targets": [ { "variable": "cpuUtil", "sortByEl": "none", "selector": "last", "format": "time_series", "limit": -1, "columns": "time,variable,device,component,metric", "tags": "Expliti.core,!Vendor.Cisco" }, { "variable": "ifInRate", "device": "ex2200", "sortByEl": "none", "selector": "choose selector", "format": "time_series", "limit": -1, "columns": "time,variable,device,component,metric", "tags": "Vendor.Juniper" } ], "from": "now-30m", "until": "now", "groupByTime": "1m", } Example of the full set of parameters recognized inside of the array `targets`:: { "variable": $variableName, "device": $deviceName, "component": $componentName, "description": $description, "tags": "Expliti.core,!Vendor.Cisco", # list of tag matching rules: comma separated list of "Facet.Word" or "!Facet.Word" "sortByEl": $sortingOrder, # "columnName:order", order can be "ascending", "descending" or "none". If empty string, server does not sort "selector": $selector, # can be "min", "max", "last" "limit": -1, # a number; -1 means unlimited "unique": $columnName, # if present and not empty, this is the name of the column that should have unique values in the response "format": "time_series", # "time_series","table","list" "columns": "time,variable,device,component,metric", # comma separated list of column names. Can include "time","variable","device","component","metric" or any tag facet name "id": "A" # query target id (from Grafana query dialog) } Fields "variable" and "device" are always matched literally (not a regex). Fields "component" and "description" can match as regex if the value is enclosed in "/ /" Tag matching expression is a comma separated list of tags in the form "Facet.Word" or "Facet". If tag is prepended with a "!", the match is negative, that is, rule matches if variable does not have this tag. If tag is in the form "Facet" (no tag word), then the rule matches if variable has any tags in this facet. If facet name is prepended with "!", as in "!Facet", then the rule matches variables that do not have any tags in this facet. Expression "Facet.*" is equivalent to "Facet" and "!Facet.*" is equivalent to "!Facet". Legacy API (v1.0) ================= Network-related commands ------------------------ GET /api/ui/status ^^^^^^^^^^^^^^^^^^ retrieve status (all areas with their discovery status and views) GET /api/ui/networks ^^^^^^^^^^^^^^^^^^^^ retrieve list of areas GET /api/ui/networks/:netid retrieve specific network GET /api/ui/networks/:netid/full ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve full area dump, including all nodes GET /api/ui/networks/:netid/generations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of discovery generations for given area GET /api/ui/networks/:netid/generations/:generation_num ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve discovery data for the specified generation number GET /api/ui/networks/:netid/generations/:timestamp ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve discovery generation that corresponds to the discovery process that completed at the time given by the timestamp (milliseconds) GET /api/ui/networks/:netid/discover ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ start network discovery GET /api/ui/networks/:netid/tiles ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ force regeneration of maps for all tiles and views GET /api/ui/networks/:netid/topo ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of nodes and links for the given network in detailed format for the latest generation GET /api/ui/networks/:netid/topo/7 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of nodes and links for the given generation number GET /api/ui/networks/:netid/topo/1382411525350 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of nodes and links for the given network for the given moment in time (milliseconds) GET /api/ui/networks/:netid/nodes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of devices for given area GET /api/ui/networks/:netid/nodes[?generation=:gen] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve list of devices for given area. This can be optionally restricted by the generation :gen (generation number or a timestamp in ms) GET /api/ui/networks/:netid/nodes/:nodeid[?generation=:gen] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve detailed information for the given node. The output includes complete discovery information but does not include monitoring data. Optional query parameter "generation" can be used to query for specific generation of the device. GET /api/ui/networks/:netid/nodes/:nodeid/summary[?generation=:gen] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve summary report for the given node. The output is generated using Velocity template specified by the configuration clause `network.monitor.device_summary_report` GET /api/ui/networks/:netid/views/:viewid/map ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve network map for the given view for the current area geenration (this is the main API call used by the frontend to display maps because view ID does not change, whereas map ID changes after each discovery since we generate new maps) GET /api/ui/networks/:netid/views/:viewid/map?layout ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ force layout calculation of the map GET /api/ui/networks/:netid/views/:viewid/map?reset ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ reset layout of the map (all nodes become not "fixed" and will float) GET /api/ui/networks/:netid/views/:viewid/map?fit&zoom=:zm&viewBox=0,0,1000,1000 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ "zoom to fit": rearrange nodes on the map to fit "viewBox" at zoom level "zoom" **TODO: implement this** GET /api/ui/networks/:netid/views/:viewid/map?generation=:gen ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve network map for the given view for the given area geenration GET /api/ui/networks/:netid/icons ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ retrieve catalog of icons in the following format: .. code-block:: none { 'router': { '0': { color: '#234523', width: 32, height: 32}, '1': { color: '#567465', width: 64, height: 64}, .... }, 'switch': { '0': { color: '#234523', width: 18, height: 18}, '1': { color: '#567465', width: 64, height: 64}, ... } ... } first level key is device role (the same role is returned in the respnse to the API call /api/ui/networks/:netid/views/:viewid/map ). Second level key is "Color Level". Color levels for device icons will be used to visualize monitoring data for devices in the future. PUT /api/ui/networks/:netid/views/:viewid/nodes/coordinates ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ update graph node coordinates (map static data object). Data is passed in the request body in JSON as a list of dictionaries: .. code-block:: none [ { 'id': nodeId1, 'x': x1, 'y': y1 }, ... ] View Manipulation (Not Implemented) ----------------------------------- POST /api/networks/:netid/views ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ add new view (Not Implemented at the moment) PUT /api/networks/:netid/views/:viewid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ update view (Not Implemented at the moment) PUT /api/networks/:netid/views/:viewid?validate ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ validate view (Not Implemented at the moment) Tries to execute matching rule and returns error messages or list of matching devices in the following JSON format: .. code-block:: none { "error" : "", "devices" : [ { "address" : "10.1.1.2", "name" : "router1.sj", "tags" : "BGP4Peer.AS7473, OSPFArea.0, Protocol.BGP4, Protocol.OSPF, Role.Router, Role.Switch, Role.eBgpPeer, Role.iBgpPeer, Vendor.Cisco" }, . . . . ] } Both commands expect the following parameters in the body of the request: - name="name", # text string - parent_type="network", # string, must be "network" or "view" - parent_id=NN, # a number, network ID for top level views (tiles) or parent view id - rule="rule script goes here", # text string - addConnectingDevices="true", # boolean, true/false - addAdjacentDevices="true" # boolean, true/false How to test:: curl http://localhost:9100/api/ui/networks/1/views \ -d name="test3" \ -d parent_type="network" \ -d parent_id=1 \ -d rule="'0' in device.tags.OSPFArea" \ -d addConnectingDevices=true \ -d addAdjacentDevices=true DELETE /api/networks/:netid/views/:viewid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ delete view (Not Implemented at the moment) UI state -------- the following API calls are currently used to store UI state to make it possible to make meaningful bookmarks and to simplify perma-links for embeddable graphs. POST /api/ui/networks/:netid/keyvalue ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Takes data passed in the body of the request and creates new record in the database, then returns handle that can be used later to retrieve it. If record with the same data already exists, its handle is returned. The data is treated as an upaque text block (it does not have to be JSON but typically probably will be). The handle is returned in JSON format as follows: .. code-block:: none { "handle" : "1" } POST /api/ui/networks/:netid/keyvalue/:handle ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ updates data associated with handle :handle. If handle with this id does not exist, returns 404. Data is passed in the body of the request just like in case of POST call described above. GET /api/ui/networks/:netid/keyvalue/:handle ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ given handle, returns associated data. In case of invalid handle, returns 404. Reports ------- GET /api/ui/networks/:netid/reports ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of available reports in JSON format as follows: .. code-block:: none { "network" : "test_network", "reports" : [ { "fileName" : "report-2014-11-07", "createdAt" : 1415410920000 } ] } createdAt time stamp is in milliseconds GET /reports ^^^^^^^^^^^^ returns list of available reports in HTML format (possibly will be deprecated in the future). GET /reports/report_file ^^^^^^^^^^^^^^^^^^^^^^^^ retrieves report file. File name can be obtained by calling /api/ui/networks/:netid/reports or /reports Monitoring Data --------------- The following API calls retrieve monitoring data for a device specified by its node id. GET /api/monitor/networks/:netid/nodes/:nodeid[?sortBy=var1&sortBy=var2...] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns information necessary to build device details page. This includes summary of basic parameters for the node such as the name, vendor, model, device tags and component tags, as well as last value for all monitoring variables for the give node (JSON) Optional parameters: - sortBy the value indicates which column to sort data by and optionally sort order. This has been extended to include words "name" and "description" (which are not monitoring variables but are added as columns by the UI). The value of this parameter should be composed of the tab name, variable name (or "name" or "description") and sort order, spearated by dots. For example: .. code-block:: none Interface.name Interface.ifInRate.desc CPU.cpuUtil.asc Note that this uses category attribute "name" rather than "display" because "name" is guaranteed to have no white spaces and is not going to be localized. Both attributes can be found in the list "categories" that looks like this: .. code-block:: none "categories" : [ { "name" : "Interface", "display" : "Interfaces", "headers" : [ { "name" : "ifOperStatus", "varGroup" : "Op", "varGroupColSpan" : 1, "columnTitle" : "Status", "unit" : "", "description" : "Interface operational status, 1=up, 2=down" }, { The variable name in the value of parameter "sortBy" matches value of attribute "name" in the corresponding category header. In the example above, this would be "ifOperStatus". Sort order is described by the last component in the value of the parameter "sortBy". It can be "asc" for ascenfing order or "desc" for descending. If the last component is missing, components are sorted in the ascending order by default. You can specify several sorting rules at the same time. Adding multiple `sortBy` parameters could be used to set up sort order in different tabs of the Device Details page. You can have different sort order for variables in different tabs. Examples:: GET /api/monitor/networks/1/nodes/1?sortBy=Interface.name.asc GET /api/monitor/networks/1/nodes/1?sortBy=Interface.ifInRate.desc&sortBy=CPU.name GET /api/monitor/networks/:netid/nodes/:nodeid/brief ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns minimal block of monitoring information about the device. This includes only its name, address, box description, OS revision, uptime and the time of last contact. GET /api/monitor/networks/:netid/nodes/:nodeid/interfaces/:ifIndex/linkSummary ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (deprecated) get summary of the monitoring data for given node and interface (used for the popup in the UI that appears when user clicks link label in a map) GET /api/monitor/networks/:netid/nodes/:nodeid/interfaces/:ifIndex/popupGraph ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get top headers and list of variables for the popup graph that opens when user clicks link end label in a map. The variables are taken from the configuration parameter `mapLegend.interfaceVars` GET /api/monitor/networks/:netid/nodes/:nodeid/popupGraph ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get top headers and list of variables for the popup graph that opens when user clicks device label in a map. The variables are taken from the configuration parameter `mapLegend.deviceVars` Monitoring Data requests used to build labels in maps ----------------------------------------------------- The following requests are used to retrieve monitoring data that can be used to build link and device labels in maps. GET /api/monitor/networks/:netid/views/:viewid/linkVar/:intfvarName/nodeVar/:nodevarName ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this request retrieves latest values of two monitoring variables, `intfvarName` and `nodevarName`, for devices and links in the latest generation of the map that corresponds to the view identified by `viewid` Graphs ------ GET /api/monitor/networks/:netid/graph/:varid1,:varid2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get information neessary to build a graph of values of variables :varid1, :varid2 and so on. Variables are defined by their identifiers which are constructed from variable name, nodeId and index, separated by a dot: "ifInRate.4.123". The index is equal to the interface ifIndex value for interface-related variable. This call supports no more than 10 variables. Returned JSON consists of a list of dictionaries describing graph legend, axis and other parameters for each variable and the list of valies in comma-separated format: .. code-block:: none { "info" : [ { "varName" : "ifOutRate", "legend" : "Te3/1 TISCALI-01", "legendEmbedded" : "router1.eq : Te3/1 TISCALI-01", "index" : 21, "varId" : "ifOutRate.21", "unit" : "Gbit/sec", "range" : [ 0.0, 1.3612093977468167 ], "significantFigures" : 3, "scale" : 1.0E-9, "axisSide" : "left", "timeseries" : [ "timestamp,ifOutRate.21", "1397519310000,1.237", "1397519340000,1.195", ] }, { "varName" : "ifOutRate", "legend" : "Gi2/2 gsju2-eth0", "legendEmbedded" : "router1.eq : Gi2/2 gsju2-eth0", "index" : 6, "varId" : "ifOutRate.6", "unit" : "Mbit/sec", "significantFigures" : 3, "range" : [ 0.0, 43.443740035332155 ], "scale" : 1.0E-6, "axisSide" : "right" "timeseries" : [ "timestamp,ifOutRate.6", "1397519310000,8.945", "1397519340000,15.233", ] } ], "warnings" : "", "errors" : "" } This call supports the following optional query parameters: - intervalHr=:intervalValue returns data for the specified interval of time (in hours). The interval is assumed to end "now". If this parameter is missing, interval is assumed to be equal to 24h. - start=:startTimestamp[&end=:endTimestamp] returns data for the interval of time defined by the start and end time. If the end time stamp is missing, it is assumed to be "now". Both start and end time are expected to be timestamp (long int) in milliseconds or ISO8601 datetime - length=:number trim time series to this size. First, we by divide original time series onto chunks by the timestamp. The total number of these chunks is equal to the half of :number. Then we find minimum and maximum observation values inside of each interval and add two new synthesized observations to the output list, one with the value equal to the minimum, and the other the vlaue equal to the maximum. This way, we reduce number of returned data points for the graph but do not lose local extremums. GET /api/monitor/networks/:netid/graph/?rule_spec ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is a variant of the previous call where monitoring variables to be graphed are identified by matching filter. See description of the rule format below in the section "Format of the variable matching rule" This call returns data in the same JSON format as the call "/api/monitor/networks/:netid/graph/:varid1,:varid2" described above. You can combine parameters of the filter with parameters "intervalHr", "start", "end" and "length". See description of the rule format below in the section :ref:`rule_spec` Generic monitoring data request ------------------------------- GET /api/monitor/networks/:netid/data/:varid1,:varid2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get monitoring data (time series) for variables :varid1, :varid2 and so on. Variables are defined by their identifiers which are constructed from variable name, nodeId and index, separated by a dot: "ifInRate.4.123". The index is equal to the interface ifIndex value for interface-related variable. This call supports no more than 10 variables. Returned JSON consists of a list of dictionaries describing each variable and list of valies in comma-separated format: .. code-block:: none { "info" : [ { "varName" : "ifOutRate", "deviceId" : 24, "index" : 1, "device" : "router2.sj", "component" : "Te1/1", "unit" : "bit/sec", "timeseries" : [ "timestamp,ifOutRate.1", "1404592860000,1502192154.133333", "1404592920001,1376278236.800000", ] }, { "varName" : "ifOutRate", "deviceId" : 24, "index" : 6, "device" : "router2.sj", "component" : "Te3/1", "unit" : "bit/sec", "timeseries" : [ "timestamp,ifOutRate.6", "1404592860000,2307981008.133333", "1404592920001,2067110022.333333", ] } ], "warnings" : "", "errors" : "" } Normally, returned observations (timestamp : value pairs) are taken from the Data Processor which collects them from individual device monitors at regular intervals. Time stamps correspond to the moments of time when Data Processor collected observations from the monitors. This call supports the following optional query parameters: - intervalHr=:intervalValue returns data for the specified interval of time (in hours). The interval is assumed to end "now". If this parameter is missing, interval is assumed to be equal to 24h. - start=:startTimestamp[&end=:endTimestamp] returns data for the interval of time defined by the start and end time. If the end time stamp is missing, it is assumed to be "now". Both start and end time are expected to be timestamp (long int) in milliseconds or ISO8601 datetime - forceDb forces data load from the database and ignores any data stored in memory - forceMem returns only data stored in memory in DataProcessor, does not try to load any from the database. Time interval of the returned time series may not match the request if amount of data in memory does not match. - monitor return only observations kept in the memory buffer of device monitor. Actual data collection is done by device monitors at regular intervals, they poll devices and store data in cyclical memory buffer that holds only 4 observations. This process is independent of the data processor which collects data on its own schedule. The polling interval in device monitors and data collection interval in the data processor are the same, but can be shifted in time. Data processor uses interpolation when it collects data from monitors. As the result, time stamps of observations retrieved with parameter "monitor" are different from the time stamps of the observations retrieved without it because the former come from device montors while the latter come from the data processor. GET /api/monitor/networks/:netid/data/?rule_spec ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is a variant of the previous call where monitoring variables are identified by matching "rule" that has the following format: .. code-block:: none name=varName&devices=dev1,dev2&components=comp1,comp2&tags=tag1,tag2&tagMatch=OP This call returns data in the same JSON format as the call "/api/monitor/networks/:netid/data/:varid1,:varid2" described above. You can combine parameter "rule" with other parameters of that call. See description of the rule format below in the section :ref:`rule_spec`. Graphing Workbench ------------------ GET /api/monitor/networks/:netid/gw/ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get list of all available monitoring variables. This call returns the following JSON data: .. code-block:: none [ { "name" : "variable_name", "devices" : [ "dev1", "dev2" ], "components" : [ "intf1", "intf2" ], "tags" : ["TagFacet.tag", "TagFacet.tag" ] } , . . . ] For example: .. code-block:: none [ { "name" : "bgpPeerInUpdates", "devices" : [ "router1.sj", "router1.eq", "router1.rd", "router2.eq", "router2.rd", "router2.sj" ], "components" : [ ], "tags" : [ "BGP4LocalAS.AS36323", "BGP4Peer.AS13768", "BGP4Peer.AS174", "BGP4Peer.AS1784", "BGP4Peer.AS19151", "BGP4Peer.AS19817", "BGP4Peer.AS24115", "BGP4Peer.AS3257", "BGP4Peer.AS32934", }, { "name" : "bgpPeerInUpdatesRate", "devices" : [ "router1.sj", "router1.eq", "router1.rd", "router2.eq", "router2.rd", "router2.sj" ], "components" : [ "bgpPeerInUpdates" ], "tags" : [ "BGP4LocalAS.AS36323", "BGP4Peer.AS13768", "BGP4Peer.AS174", "BGP4Peer.AS1784", "BGP4Peer.AS19151", "BGP4Peer.AS19817", "BGP4Peer.AS24115", "BGP4Peer.AS3257", "BGP4Peer.AS32934", }, { GET /api/monitor/networks/:netid/gw?rule_spec ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Get list of monitoring variable instances that match given filter. The filter uses the same matching rule format as call "/api/monitor/networks/:netid/graph/?rule_spec" See description of the rule format below in the section :ref:`rule_spec` This call returns variables in the following JSON format: .. code-block:: none [ { "name" : "ifHCInOctets", "component" : "Gi8/31", "device" : "router1.eq", "devId" : 1, "index" : 63, "description" : "router1.eq:Gi8/31 " }, { "name" : "ifHCInOctets", "component" : "Vl202", "device" : "router1.eq", "devId" : 1, "index" : 92, "description" : "router1.eq:Vl202 subnet-208.94.2.64-gsab" }, { GET /api/monitor/networks/:netid/vars/:varname ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get list of last values of all instances of one monitoring variable. Parameter "varname" is variable name (no device Id and no index). This call returns the following JSON data: .. code-block:: none [ { "device" : "c3560g-1", "address" : "10.0.14.228", "component" : "Gi0/17", "color" : "#dddddd", "colorLevel" : "0", "value" : "172846.93333333332", "scaledValue" : "0.173", "unit" : "bit/sec", "scaledUnit" : "Mbit/sec", "tags" : "Explicit.core, Link.vlan1_1, ifOperStatus.Up, ifRole.PhysicalPort, ifAdminStatus.Up, ifSpeed.1G, ifRole.UntaggedSwtichPort, ifRole.BroadcastTypeInterface" }, { "device" : "c3560g-1", "address" : "10.0.14.228", "component" : "Gi0/18", "color" : "#ffdead", "colorLevel" : "1", "value" : "146676.53333333333", "scaledValue" : "146.677", "unit" : "bit/sec", "scaledUnit" : "kbit/sec", "tags" : "Explicit.core, Link.10.0.14.90, ifOperStatus.Up, ifRole.PhysicalPort, ifSpeed.10M, ifAdminStatus.Up, ifRole.UntaggedSwtichPort, ifRole.BroadcastTypeInterface" }, { . . . ] Note the other format of this call: **GET /api/monitor/networks/:netid/vars?rule_spec** See description of the rule format below in the section :ref:`rule_spec` GET /api/monitor/networks/:netid/catalog/[triplet1,triplet2...] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ get "catalog" of monitoring variables. The set of variables can be limited by providing comma-separated list of "triplets", otherwise all known monitoring variables are reported. A "triplet" consists of variable name, device id and component index, in this order, separated by a dot. For example: .. code-block:: none ifInRate.34.7 This triplet describes variable that measures inbound interface traffic for device with id=34 and interface with ifIndex=7. See :ref:`Monitoring Variable reference ` in section :ref:`monitoring_variables` for more details. This call returns data in the following JSON format: .. code-block:: none { "ifInErrors.34.7" : { "category" : "Interfaces", "varDescription" : "Inbound packet Errors", "device" : "router1.rd", "component" : "Gi2/3", "componentDescription" : "rdsw3 (vlan203)-rdac", "unit" : "pkt/s" }, "ifInRate.34.7" : { "category" : "Interfaces", "varDescription" : "Inbound traffic on the interface", "device" : "router1.rd", "component" : "Gi2/3", "componentDescription" : "rdsw3 (vlan203)-rdac", "unit" : "bit/sec" } The key in the first level dictionary is variable triplet. Note that the set of variables changes over time for various reasons. When NetSpyGlass starts, it only has set of variables that were defined in the configuration or hardcoded. New variables are created by the Python hook script that processes collected monitoring data. This script runs on schedule, this means new variables are created after the first polling cycle, when the system runs the script for the first time. Later on, new variables can be created when user adds new device to the configuration we NetSpyGlass discovers it. Variables can also disappear, for example when user removes device from the configuration. Monitor JSON API ---------------- GET /api/metrics/healthcheck ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This call returns JSON dictionary that reflects status of several internal components in the monitor, as well as its summary health status. This can be used for monitoring. GET /api/metrics/ping ^^^^^^^^^^^^^^^^^^^^^ This call simply returns string "pong" if the monitor is alive. GET /api/metrics/metrics?pretty=true ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this call returns various internal metrics of the monitor in JSON format GET /api/quit ^^^^^^^^^^^^^ this signals monitor to quit. Note that this call only works when made by connecting to the monitor directly rather than via UI backend proxy. .. _rule_spec: Format of the variable matching rule ------------------------------------ This rule is used in several API calls to match monitoring variables. These calls are: .. code-block:: none GET /api/monitor/networks/:netid/graph?rule_spec GET /api/monitor/networks/:netid/data?rule_spec GET /api/monitor/networks/:netid/vars?rule_spec GET /api/alerts/networks/:netid/alerts[?active=true|false][rule_spec] The rule consists of several matching parameters: .. code-block:: none name=var1,var2&devices=dev1,dev2&components=comp1,comp2&tags=tag1,tag2&tagMatch=OP&sortBy=column.direction&limit=N&start=N1&count=N2 Values of all parameters should be url-encoded. The value of the parameter `name` is a comma separated list of variable names, e.g. name=jvmMemTotal,jvmMemFree Tags should be in the form "TagFacet.Word" or "!TagFacet.Word". Using "!" in front of the tag means the rule matches when this tag is not present in the set of tags attached to the monitoring variable. Tag matching operation OP can be one of the following (assuming `tags` is the set of tags attached to the monitoring variable): .. _tag_matching_ops: .. table:: Tag matching operators and their presentation in the Graphing Workbench +-----------+----------------------------------------------+-------------------------------------------------------------+ | operation | equivalent logic | how does this appear in the Graphing Workbench tags filter | +===========+==============================================+=============================================================+ | "AND" | :python:`tag1 in tags and tag2 in tags` | Contains all of... | +-----------+----------------------------------------------+-------------------------------------------------------------+ | "OR" | :python:`tag1 in tags or tag2 in tags` | Contains any of ... | +-----------+----------------------------------------------+-------------------------------------------------------------+ | "NOT_AND" | :python:`not(tag1 in tags and tag2 in tags)` | Does not contain all of ... | +-----------+----------------------------------------------+-------------------------------------------------------------+ | "NOT_OR" | :python:`not(tag1 in tags or tag2 in tags)` | Does not contain any of ... | +-----------+----------------------------------------------+-------------------------------------------------------------+ Here is how query translates into logic operations: +-------------------------------------------------+--------------------------------------------------------------------+ | Query | equivalent logic | +=================================================+====================================================================+ | tags=tag1,tag2,tag3&tagMatch=AND | :python:`tag1 in tags and tag2 in tags and tag3 in tags` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,!tag3&tagMatch=AND | :python:`tag1 in tags and tag2 in tags and tag3 not in tags` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,tag3&tagMatch=OR | :python:`tag1 in tags or tag2 in tags or tag3 in tags` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,!tag3&tagMatch=OR | :python:`tag1 in tags or tag2 in tags or tag3 not in tags` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,tag3&tagMatch=NOT_AND | :python:`not(tag1 in tags and tag2 in tags and tag3 in tags)` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,!tag3&tagMatch=NOT_AND | :python:`not(tag1 in tags and tag2 in tags and tag3 not in tags)`| +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,tag3&tagMatch=NOT_OR | :python:`not(tag1 in tags or tag2 in tags or tag3 in tags)` | +-------------------------------------------------+--------------------------------------------------------------------+ | tags=tag1,tag2,!tag3&tagMatch=NOT_OR | :python:`not(tag1 in tags or tag2 in tags or tag3 not in tags)` | +-------------------------------------------------+--------------------------------------------------------------------+ Perhaps not all of these combinations are practically useful, however they are supported for completenness. Graphing Workbench does not support negated tag match (i.e. `!Facet.Word`) but supports `NOT_AND` and `NOT_OR` operations as shown in :ref:`tag_matching_ops` The value of the parameter "sortBy" is a name of the column and sorting direction, separated by a dot, e.g. "devices.asc", "component.desc". Following column names are recognized: - devices - component - description - curr - min - max Sorting direction can be "asc" or "desc". If sorting direction is missing or is not recognized, the system uses ascending sorting order. Parameter "limit" limits number of variables in the returned data block. Use "limit" in combination with "sortBy" to get top N variables. For example, combination "sortBy=curr.asc&limit=10" returns top 10 variables by current value. Parameters "start" and "count" are used for pagination (for example, with data table in Graphing Workbench) .. note:: All parameters are optional. For example, if parameter `name` is missing but `tags` is present, then the rule matches any monitoring variables with given combination of tags regardless of their name. If parameters `name`, `devices`, `components` and `tags` (that is, those that actually match various fields of monitoring variable object) are missing, then the rule is considered to be "empty" and does not match any variables. Alerts and Alert silences JSON API ---------------------------------- .. _json_api_list_alerts: GET /api/alerts/networks/:netid/alerts[?active=true|false][rule_spec] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns list of alerts that exist in the system, possible filtered in one of ways described below. Note that alert can be in the state "Active" or "Cleared" and this call considers both before applying the filter. If you want to get only active alerts, supply parameter `active=true`: .. code-block:: none GET /api/alerts/networks/:netid/alerts?active=true If you want to get only cleared alerts, supply parameter `active=false`: .. code-block:: none GET /api/alerts/networks/:netid/alerts?active=false You can also apply filter defined by the matching rule that follows the same format used in other API calls, see section :ref:`rule_spec`. API call in the following example matches any alerts that have tags `Explicit.alert_group_2` and `Explicit.LOCATION_SJC`: .. code-block:: none GET /api/alerts/networks/:netid/alerts?tags=Explicit.alert_group_2,Explicit.LOCATION_SJC&tagMatch=AND You can combine parameter `active=true` with rule match: .. code-block:: none GET /api/alerts/networks/:netid/alerts?active=true&tags=Explicit.alert_group_2,Explicit.LOCATION_SJC&tagMatch=AND This call returns list of dictionaries where each dictionary describes one alert. At this time it looks like this: .. code-block:: none { "key" : "a1b779e08514b5e7e76fb58fa2b2a294", "name" : "busyCpuAlert", "deviceName" : "rdac3", "deviceId" : 131, "componentName" : "GenuineIntel: Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz", "componentIndex" : 4294968065, "value" : "80.0", "values" : { "cpuUtil.131.4294968065" : "80.0" }, "description" : "CPU utilization is over 75% for 20% of time for the last 10 min", "tags" : [ "Explicit.datanodes_hbase0", "Explicit.hbase0", "Model.linux", "Role.Server", "Role.datanodes_hbase0", "Role.hbase0", "Vendor.NetSnmp" ], "variable" : "busyCpuAlert.131.4294968065", "inputVariable" : "cpuUtil.131.4294968065", "activeSince" : 1433965140000, "active" : true, "silenced" : true, "matchingSilenceId" : 20, "timeLastNotificationSent" : 0, "duration" : 600.0, "percentage" : 20.0, "notificationTimeMs" : 300000.0 } - key : unique identifier of this alert; this key can be used for deduplication - name : the name of the alert defined in the call to Python function :py:func:`nw2functions.alert()` - deviceName: name of the device - deviceId: id of the device - componentName: name of the component - componentIndex: index of the component - value : the latest value of the input monitoring variable that triggered the alert. Note that if the alert configuration uses parameter `percentage`, the alert is actually triggered by combination of several values in the time series of the input variable rather than this one latest value - description : alert description taken from the parameter of the call to :py:func:`nw2functions.alert()` with all macros expanded - tags : tags of this alert. Tags are copied from the input monitoring variable - variable : the triplet for the monitoring variable created for this alert. - inputVariable : the triplet for the input monitoring variable used to create this alert - activeSince : time stamp (in ms) when this alert entered state "active" last time if it is currently active - active : true if alert is active, false otherwise - silenced : true if there is silence that matches this alert, otherwise false - matchingSilenceId : the id of the matching silence. If there is no matching silence, this is equal to 0 - timeLastNotificationSent : time (in ms) when we sent notification for this alert last time - duration : configuration parameter taken from the call to :py:func:`nw2functions.alert()` that created this alert - percentage : configuration parameter taken from the call to :py:func:`nw2functions.alert()` that created this alert - notificationTimeMs : configuration parameter taken from the call to :py:func:`nw2functions.alert()` that created this alert GET /api/alerts/networks/:netid/alerts/:alertKey ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Returns alert with given key if it exists. .. note:: Unlike with other objects, such as networks, devices, vilews, maps and even alert silences (see blow), which use numbers for their Ids, alert key is a string of random characters .. _json_api_list_silences: POST /api/alerts/networks/:netid/silences ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Add new silence to the system. Takes data from the body of the POST call and creates new alert silcnce. The data should follow this format: .. code-block:: none { "expirationTimeMs": 3600000, "match" : { "key": "alertkey", "varNme": "varName", "deviceId": deviceId, "deviceName": "deviceName", "index": index, "tags": "Role.Router, !Explicit.test" } } Expiration time interval is in milliseconds. All items in the `match` directory are optional, you are only required to provide attributes you actually want to use to match alerts. .. note:: Even though an empty `match` is permitted, the silence created with empty match rule never matches any alerts and is useless. This call returns small JSON dictionary: .. code-block:: none { "id": 1000, "status": "success", "message": "" } - `id`: id of newly created silence - `status`: can be "success" or "error" - `message`: verbose error message returned when `status` has value `error` POST /api/alerts/networks/:netid/silences/:id ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Updates existing silence with id `id`. Takes data from the body of the POST call and creates new alert silcnce. The data should follow this format (the same as with the call that creates the silence): .. code-block:: none { "expirationTimeMs": 3600000, "user": "vadim", "reason": "doing maitenance for the next hour", "match" : { "key": "alertKey", "varName": "varName", "deviceId": deviceId, "deviceName": deviceName, "index": index, "tags": "Role.Router, !Explicit.test" } } This call returns small JSON dictionary with the same items as the call that creates new silence: .. code-block:: none { "id": 1000, "status": "success", "message": "" } - `id`: id of newly created silence - `status`: can be "success" or "error" - `message`: verbose error message returned when `status` has value `error` GET /api/alerts/networks/:netid/silences/ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns all active silences in JSON format: .. code-block:: none [ { "id": 1000, "createdAt": 1418269740000, "updatedAt": 1418269740000, "expirationTimeMs": 3600000, "user": "vadim", "reason": "doing maitenance for the next hour", "match" : { "key": "", "varName": "cpuBusy", "deviceId": 9, "deviceName": "router1.sj", "index": 11, "tags": "Role.Router, !Explicit.test" } }, ] note that this only returns list of active silences, that is, those that have not expired yet. All time values (createdAt, updatedAt and expirationTimeMs) are in milliseconds. GET /api/alerts/networks/:netid/silences/:id ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returns silence with id `:id` in JSON format: .. code-block:: none [ { "id": 1000, "createdAt": 1418269740000, "updatedAt": 1418269740000, "expirationTimeMs": 3600000, "user": "vadim", "reason": "doing maitenance for the next hour", "match" : { "key": "", "varName": "cpuBusy", "deviceId": 9, "deviceName": "", "index": 0, "tags": "Role.Router, !Explicit.test" } }, ] note that this only returns silence if it is active, that is, it has not expired yet. All time values (createdAt, updatedAt and expirationTimeMs) are in milliseconds. Matching attributes have default names if they were omitted when silence has been created. These values are: #. key, varName, deviceName, tags: if these attributes are empty string, they are ignored and do not match anything #. deviceId, index: if these attributes have value 0, they are ignored