9.3. Configuration¶
Syslog Server configuration is part of the NSG Agent configuration and can be found at syslogServer section in the file agent.conf stored with the NSG servers in the cloud. This file is located in directory conf in the git repository where all server configuration files and Python apps are kept. To change something in the Syslog Server configuration, such as server port or read timeout, just edit file conf/agent.conf, commit and push. Servers will pick up the change and push it to all agents automatically.
Syslog Server configuration contains two kinds of parameters, one defines network settings like interface and port server binds to and receives syslog messages from. Another one defines Grok parser settings. Grok parser is the process inside NSG Agent responsible for real-time conversion of incoming syslog messages into structured data (Json documents) that eventually be published to ElasticSearch by NetSpyGlass server.
Below is an example of syslogServer
configuration
syslogServer {
# Transport layer
host = "0.0.0.0"
port = 5514
timeoutReadMillis = 20000
timeoutConnectMillis = 20000
messageMaxLength = 65535
# Grok parser
include "grok.conf"
timeAuthority = agent
# Internal
# Changes to parameters below affect the robustness of the Syslog Server
# and expected to be done by NetSpyGlass personal only
# The content of processor parameter is omitted
processors = []
messageFieldNameToKafkaKey = "logSource"
}
Configuration begins with definition of networking transport layer parameters
host
: network interface server binds to, if 0.0.0.0 is specified, server listens all interfacesport
: TCP/UDP port server listens totimeoutReadMillis
: server read timeout in millisecondstimeoutConnectMillis
: server connection timeout in millisecondsmessageMaxLength
: maximum length of syslog message allowed, server drops messages longer than messageMaxLength
and ends with Grok parser configuration.
include "grok.conf"
: instruction to load custom (user-defined) Grok patterns and expressions from configuration file grok.conftimeAuthority
: source of timestamp for ES document containing the syslog message. There are two options available, agent - the system time of the machine running the agent and syslog - time extracted from the syslog message headermessageFieldNameToKafkaKey
: (internal) ES document field name that contains identification of the device issued the syslog messageprocessors
: (internal) Grok parser configuration
Important
The processors
section of Grok parser configuration must be considered as unchangeable and
may be updated only along with the newer versions of NSG Agent or NetSpyGlass server.
Grok parser is configured with built-in patterns defined in processors
section and custom patterns that can be
modified or extended by users. Custom or user-defined patterns are stored in grok.conf
configuration file that
can be found next to agent.conf
in directory conf in the git repository where all server configuration files and
Python apps are kept.
Below is the initial content of grok.conf
grok.regex = [
{
name = "INTERFACENAME"
expression = "[a-zA-Z][a-zA-Z0-9\\-./]+"
}
{
name = "INTERFACESTATUS"
expression = "(up|down)"
}
]
grok.patterns = [
"Native VLAN mismatch discovered on %{INTERFACENAME:logIfLocalName} \\(%{NONNEGINT}\\), with %{HOSTNAME} %{INTERFACENAME:logIfRemoteName} \\(%{NONNEGINT}\\)"
"Line protocol on Interface %{INTERFACENAME:logIfLocalName}, changed state to %{INTERFACESTATUS:logIfLocalOperStatus}"
"Interface %{INTERFACENAME:logIfLocalName}, changed state to (%{WORD} )?%{INTERFACESTATUS:logIfLocalOperStatus}"
]
There are only two sections in this file. One is grok.regex
, this is a collection of custom Grok expressions that
can be defined in addition to the built-in Grok expressions. Every object of this collection must have
expression’s name name
and the regular expression in the field expression
. Those custom expressions along with
built-in can be used in user-defined Grok patterns that are collected in grok.patterns
list.