.. _syslog_configuration:

Configuration
=============

Syslog Server configuration is part of the :ref:`NSG Agent configuration<nsg-agent-configuration>` and can be found
at `syslogServer` section in the file `agent.conf` stored with the *NSG servers in the cloud*.
This file is located in directory `conf` in the git repository where all server configuration files and Python apps are kept.
To change something in the Syslog Server configuration, such as server port or read timeout,
just edit file `conf/agent.conf`, commit and push. Servers will pick up the change and push it to all agents automatically.

Syslog Server configuration contains two kinds of parameters, one defines network settings like interface and port server
binds to and receives syslog messages from. Another one defines Grok parser settings. Grok parser is the process inside
NSG Agent responsible for real-time conversion of incoming syslog messages into structured data (Json documents) that
eventually be published to ElasticSearch by NetSpyGlass server.

Below is an example of ``syslogServer`` configuration

.. code-block:: none

    syslogServer {
        # Transport layer
        host = "0.0.0.0"
        port = 5514
        timeoutReadMillis = 20000
        timeoutConnectMillis = 20000
        messageMaxLength = 65535

        # Grok parser
        include "grok.conf"
        timeAuthority = agent

        # Internal
        # Changes to parameters below affect the robustness of the Syslog Server
        # and expected to be done by NetSpyGlass personal only
        # The content of processor parameter is omitted
        processors = []
        messageFieldNameToKafkaKey = "logSource"
    }

Configuration begins with definition of networking transport layer parameters

 - ``host``: network interface server binds to, if **0.0.0.0** is specified, server listens all interfaces
 - ``port``: TCP/UDP port server listens to
 - ``timeoutReadMillis``: server read timeout in milliseconds
 - ``timeoutConnectMillis``: server connection timeout in milliseconds
 - ``messageMaxLength``: maximum length of syslog message allowed, server drops messages longer than **messageMaxLength**

and ends with Grok parser configuration.

 - ``include "grok.conf"``: instruction to load custom (user-defined) Grok patterns and expressions from configuration file **grok.conf**
 - ``timeAuthority``: source of timestamp for ES document containing the syslog message. There are two options available,
   **agent** - the system time of the machine running the agent and **syslog** - time extracted from the syslog message header
 - ``messageFieldNameToKafkaKey``: (internal) ES document field name that contains identification of the device issued the syslog message
 - ``processors``: (internal) Grok parser configuration

.. important::
    The ``processors`` section of Grok parser configuration  must be considered as unchangeable and
    may be updated only along with the newer versions of :ref:`NSG Agent<nsg-agent>` or NetSpyGlass server.

Grok parser is configured with built-in patterns defined in ``processors`` section and custom patterns that can be
modified or extended by users. Custom or user-defined patterns are stored in ``grok.conf`` configuration file that
can be found next to ``agent.conf`` in directory `conf` in the git repository where all server configuration files and
Python apps are kept.

Below is the initial content of ``grok.conf``

.. code-block:: none

    grok.regex = [
        {
            name = "INTERFACENAME"
            expression = "[a-zA-Z][a-zA-Z0-9\\-./]+"
        }
        {
            name = "INTERFACESTATUS"
            expression = "(up|down)"
        }
    ]

    grok.patterns = [
        "Native VLAN mismatch discovered on %{INTERFACENAME:logIfLocalName} \\(%{NONNEGINT}\\), with %{HOSTNAME} %{INTERFACENAME:logIfRemoteName} \\(%{NONNEGINT}\\)"
        "Line protocol on Interface %{INTERFACENAME:logIfLocalName}, changed state to %{INTERFACESTATUS:logIfLocalOperStatus}"
        "Interface %{INTERFACENAME:logIfLocalName}, changed state to (%{WORD} )?%{INTERFACESTATUS:logIfLocalOperStatus}"
    ]

There are only two sections in this file. One is ``grok.regex``, this is a collection of custom Grok expressions that
can be defined in addition to the built-in :ref:`Grok expressions<grok_exp>`. Every object of this collection must have
expression's name ``name`` and the regular expression in the field ``expression``. Those custom expressions along with
built-in can be used in user-defined Grok patterns that are collected in ``grok.patterns`` list.