15. Using the Greenbone Management Protocol

The vulnerability management functionality of the Greenbone Security Manager (GSM) is also available via the Greenbone Management Protocol (GMP).

Greenbone Networks provides the Greenbone Vulnerability Management Tools (gvm-tools) to access the functionality made available by GMP (see Chapter 15.3). This user manual covers gvm-tools up to version 2.0.0 beta.

The newest version of GMP is documented at https://docs.greenbone.net/API/GMP/gmp-9.0.html.

15.1. Changes to GMP

GMP is regularly updated to apply changes in the functionality provided by the underlying service and to provide a consistent and comprehensive interface.

Updates result in a new version of GMP. Each new version includes a list of added, modified and removed protocol elements, e.g., commands or attributes. The most recent version of the list is available at https://docs.greenbone.net/API/GMP/gmp-9.0.html#changes.

Depending on the changes, the old version may be available for some time. During this transitional phase, the new and the old version are available at the same time.

This list helps to prepare for upcoming changes as soon as possible. It does not represent the complete list of upcoming changes.

15.2. Activating GMP

Before GMP can be used, it has to be activated on the GSM.

While the web interface uses GMP locally on the appliance, GMP is not remotely accessible via the network by default.

The remote GMP service can be activated using the GOS administration menu (see Chapter 7.2.3.2).

In general, the access to GMP is authenticated and encrypted with SSL/TLS. The same users as for the web interface are used. The users are subject to the same restrictions and have the same permissions.

15.3. Using gvm-tools

The Greenbone Vulnerability Management Tools (gvm-tools) are a collection of tools that provide access to the functionalities of the Greenbone Management Protocol (GMP).

gvm-tools are available as a command line interface (CLI) and as a Python shell for Microsoft Windows and any operating system that supports Python, including Linux.

gvm-tools can be downloaded at the project’s official GitHub repository. Python 3.5 or later is required. To install gvm-tools, follow the instructions provided at https://github.com/greenbone/gvm-tools#installation.

In addition, gvm-tools are available as statically linked EXE files for all currently supported versions of Microsoft Windows (see https://support.microsoft.com/en-us/help/13853/windows-lifecycle-fact-sheet).

The EXE versions of gvm-tools do not require Python and may be downloaded directly from Greenbone Networks at:

Important

External links to the Greenbone download website are case-sensitive.

Note that upper cases, lower cases and special characters have to be entered exactly as they are written in the footnotes.

Note

gvm-tools are licensed under the GNU General Public License v3.0 and may be adapted and built for other uses cases, based on the source code.

15.3.1. Accessing with gvm-cli.exe

GMP is XML based. Every command and every response is a GMP object.

The command line tool gvm-cli.exe supplied by Greenbone Networks offers direct sending and receiving of XML commands and responses.

gvm-cli.exe supports the following connections:

  • SSH
  • TLS
  • Unix Domain Socket

gvm-cli.exe supports several command line switches which can be displayed using:

$ gvm-cli -h
usage: gvm-cli [-h] [-c [CONFIG]]
             [--log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
             [--timeout TIMEOUT] [--gmp-username GMP_USERNAME]
             [--gmp-password GMP_PASSWORD] [-V] [--protocol {GMP,OSP}]
             CONNECTION_TYPE ...

optional arguments:
  -h, --help            show this help message and exit
  -c [CONFIG], --config [CONFIG]
                        Configuration file path (default: ~/.config/gvm-
                        tools.conf)
  --log [{DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                        Activate logging (default level: None)
  --timeout TIMEOUT     Response timeout in seconds, or -1 to wait
                        indefinitely (default: 60)
  --gmp-username GMP_USERNAME
                        Username for GMP service (default: '')
  --gmp-password GMP_PASSWORD
                        Password for GMP service (default: '')
  -V, --version         Show version information and exit
  --protocol {GMP,OSP}  Service protocol to use (default: GMP)

connections:
  valid connection types

  CONNECTION_TYPE       Connection type to use
    ssh                 Use SSH to connect to service
    tls                 Use TLS secured connection to connect to service
    socket              Use UNIX Domain socket to connect to service

While gvm-cli.exe supports more command line switches the additional options are only displayed when the connection type is specified:

$ gvm-cli ssh -h
usage: gvm-cli ssh [-h] --hostname HOSTNAME [--port PORT]
                 [--ssh-username SSH_USERNAME]
                 [--ssh-password SSH_PASSWORD] [-X XML] [-r] [--pretty]
                 [--duration]
                 [infile]

positional arguments:
  infile                File to read XML commands from.

optional arguments:
  -h, --help            show this help message and exit
  --hostname HOSTNAME   Hostname or IP address
  --port PORT           SSH port (default: 22)
  --ssh-username SSH_USERNAME
                        SSH username (default: 'gmp')
  --ssh-password SSH_PASSWORD
                        SSH password (default: 'gmp')
  -X XML, --xml XML     XML request to send
  -r, --raw             Return raw XML
  --pretty              Pretty format the returned xml
  --duration            Measure command execution time

All current GSM appliances supporting GOS 6 use SSH to encrypt GMP. The use of TLS is deprecated, not officially supported and may be removed in a future version.

The gvm-tools are mostly helpful for batch mode (batch processing, scripting).

With gvm-cli.exe GMP can be used in a simple way:

gvm-cli --xml "<get_version/>"
gvm-cli --xml "<get_tasks/>"
gvm-cli < file

15.3.1.1. Configuring the Client

For using command gvm-cli logging into the appliance is required.

The needed information is supplied either using command line switches or a configuration file (~/.config/gvm-tools.conf).

To provide the GMP user with command line switches use:

  • --gmp-username
  • --gmp-password

Alternatively a configuration file ~/.config/gvm-tools.conf containing the information can be created:

[Auth]
gmp_username=webadmin
gmp_password=kennwort

This configuration file is not read by default. The command line switch --config or -c has to be added to read the configuration file.

15.3.1.2. Starting a Scan Using the Command gvm-cli

A typical example for using GMP is the automatic scan of a new system.

In the example it is assumed that an Intrusion Detection System (IDS) is used that monitors the systems in the Demilitarized Zone (DMZ) and immediately discovers new systems and unusual TCP ports that are not used already. If such an event is discovered, the IDS should automatically initiate a scan of the new system with the help of a script.

For this, the command gvm-cli can be used although the command gvm-pyshell or using self written python scripts may be more suitable (see Chapter 15.3.2.1). The processing of the XML output is better supported by python than by using the shell.

Starting point is the IP address of the new suspected system. A target needs to be created for this IP address on the GSM.

The command create_target is described at:

https://docs.greenbone.net/API/GMP/gmp-9.0.html#command_create_target.

  1. If the IP address is saved in the variable IPADDRESS, create the respective target as follows:
$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 \
--xml "<create_target><name>Suspect Host</name>\
<hosts>$IPADDRESS</hosts></create_target>"

<create_target_response status="201" status_text="OK, resource
created" id="4574473f-a5d0-494c-be6f-3205be487793"/>
  1. Create the task as follows:
$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 \
--xml "<create_task><name>Scan Suspect Host</name> \
<target id=\"4574473f-a5d0-494c-be6f-3205be487793\"></target> \
<config id=\"daba56c8-73ec-11df-a475-002264764cea\"></config></create_task>"


<create_task_response status="201" status_text="OK, resource
created" id="ce225181-c836-4ec1-b83f-a6fcba70e17d"/>

→ The output is the ID of the task. It is required to start and monitor the task.

The other IDs used by the command can be retrieved using the following commands which display the available targets and scan configs:

$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 --xml "<get_targets/>"

$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 --xml "<get_configs/>"

Note

The output of the commands above is XML.

  1. Start the task as follows:
$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 \
--xml '<start_task task_id="ce225181-c836-4ec1-b83f-a6fcba70e17d"/>'

→ The connection will be closed by the GSM. The task is running.

  1. Display the status of the task as follows:
$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 \
--xml '<get_tasks task_id="ce225181-c836-4ec1-b83f-a6fcba70e17d"/>'

<get_tasks_response status="200" status_text="OK"><apply_overrides>
...<status>Running</status><progress>98<host_progress>
<host>192.168.255.254</host>98</host_progress></progress>.../>

→ As soon as the scan is completed, the report can be downloaded.

For this the ID that was output when the task was created is required and a meaningful report format has to be entered.

  1. Display the IDs for the report formats as follows:
$ $ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 --xml '<get_report_formats/>'
  1. Load the report as follows:
$ gvm-cli --gmp-username webadmin --gmp-password kennwort ssh \
--hostname 192.168.222.115 \
--xml '<get_reports report_id="23a335d6-65bd-4be2-a83e-be330289eef7" \
format_id="35ba7077-dc85-42ef-87c9-b0eda7e903b6"/>'

Tip

To fully and automatically process the data, the task can be combined with an alert that forwards the report based on a given condition.

15.3.2. Accessing with gvm-pyshell.exe

The command line tool gvm-pyshell.exe supplied by Greenbone Networks offers the direct sending and receiving of XML commands and XML responses using python commands. The commands take care of the generation and parsing of the XML data.

The tool supports the following connections:

  • TLS
  • SSH
  • Socket

While the current GSM appliances (GOS 6) use SSH to protect GMP, older appliances used TLS and Port 9390 to transport GMP. The gvm-tools can be used with both the older and the current GOS.

The gvm-tools are mostly helpful for batch mode (batch processing, scripting).

The authentication configuration of the command gvm-pyshell can be stored in a file in the home directory of the user. The syntax is explained in Chapter 15.3.1.1.

The Python implementation follows the GMP API (https://docs.greenbone.net/API/GMP/gmp-9.0.html). Optional arguments in the API are identified by a ?. The following example explains the usage of the Python functions:

gmp.create_task("Name","Config","Scanner","Target",comment="comment")

Tip

While mandatory arguments can be supplied in the correct order and are identified automatically they can also be specified using their identifier:

gmp.create_task(name="Name",config_id="Config",scanner_id="Scanner",
target_id="Target",comment="comment")

15.3.2.1. Starting a Scan Using the Command gvm-pyshell

A typical example for using GMP is the automatic scan of a new system.

In the example it is assumed that an Intrusion Detection System (IDS) is used that monitors the systems in the Demilitarized Zone (DMZ) and immediately discovers new systems and unusual TCP ports that are not used already. If such an event is discovered, the IDS should automatically initiate a scan of the new system with the help of a script.

For this, the command gvm-pyshell is very suitable. The processing of the XML output is better supported by python than by using the shell.

Starting point is the IP address of the new suspected system. A target needs to be created for this IP address on the GSM.

The command create_target is described at:

https://docs.greenbone.net/API/GMP/gmp-9.0.html#command_create_target.

  1. The following lines illustrate the commands required when using gvm-pyshell:
$ gvm-pyshell \
--gmp-username webadmin --gmp-password kennwort \
ssh --hostname 192.168.222.115
GVM Interactive Console 2.0.0 API 1.1.0. Type "help" to get information about
  functionality.
>>> res=gmp.create_target("Suspect Host", make_unique=True, \
  hosts=['192.168.255.254'])
>>> target_id = res.xpath('@id')[0]

The variable target_id contains the ID of the created target. This ID can be used to create the corresponding task.

Note

The task creation requires the following input:

  • target_id
  • config_id
  • scanner_id
  • task_name
  • task_comment
  1. All existing scan configurations can be displayed as follows:
>>> res = gmp.get_configs()
>>> for i, conf in enumerate(res.xpath('config')):
...    id = conf.xpath('@id')[0]
...    name = conf.xpath('name/text()')[0]
...    print('\n({0}) {1}: ({2})'.format(i, name, id))
  1. All existing scanners can be displayed using the same technique. If only the built-in scanners are used the following IDs are hard coded:
    • OpenVAS scanner: 08b69003-5fc2-4037-a479-93b440211c73
    • CVE scanner: 6acd0832-df90-11e4-b9d5-28d24461215b
  2. Create the task as follows:
>>> res=gmp.create_task(name="Scan Suspect Host",
... config_id="daba56c8-73ec-11df-a475-002264764cea",
... scanner_id="08b69003-5fc2-4037-a479-93b440211c73",
... target_id=target_id)
>>> task_id = res.xpath('@id')[0]
  1. Start the task as follows:
>>> gmp.start_task(task_id)

→ The current connection is closed immediately. Further commands are not required.

All commands can be put in a Python script which may be invoked by the Python shell:

len_args = len(args.script) - 1
if len_args is not 2:
    message = """
    This script creates a new task with specific host and nvt!
    It needs two parameters after the script name.
    First one is name of the target and the second one is the
    chosen host. The task is called target-task

    Example:
        $ gvm-pyshell ssh newtask target host
    """
    print(message)
    quit()

target = args.script[1]
host = args.script[2]
task = target + " Task"

# Full and Fast
myconfig_id = "daba56c8-73ec-11df-a475-002264764cea"

# OpenVAS Scanner
myscanner_id = "08b69003-5fc2-4037-a479-93b440211c73"

res=gmp.create_target(target, True, hosts=host)
mytarget_id = res.xpath('@id')[0]

res=gmp.create_task(name=task,
                    config_id=myconfig_id,
                        scanner_id=myscanner_id,
                            target_id=mytarget_id)
mytask_id = res.xpath('@id')[0]

gmp.start_task(mytask_id)

15.3.3. Example Scripts

The gvm-tools come with a collection of example scripts which can be used by the command gvm-script.

Currently the following scripts are available for gvm-tools version 2.0.0 (https://github.com/greenbone/gvm-tools/tree/master/scripts):

  • application-detection.gmp.py: this script displays all hosts with the searched application.
  • cfg-gen-for-certs.gmp.py: this script creates a new scan configuration with NVTs based on a given CERT-Bund advisory.
  • clean-sensor.gmp.py: this script removes all resources from a sensor except active tasks.
  • create-dummy-data.gmp.py: this script generates dummy data.
  • DeleteOverridesByFilter.gmp.py: this script deletes overrides using a filter.
  • monthly-report2.gmp.py: this script displays all vulnerabilities based on the reports of a given months. Made for GOS 4.x.
  • monthly-report.gmp.py: this script will display all vulnerabilities based on the reports of a given months. Made for GOS 3.1.
  • nvt-scan.gmp.py: this script creates a new task with a specific host and NVT using a hardcoded base configuration.
  • startNVTScan.gmp.py: this script creates a new task with a specific host and NVT interactively.
  • SyncAssets.gmp.py: this script uploads assets to the asset database.
  • SyncReports.gmp.py: this script pulls reports and uploads them to a second GSM using container tasks.

Tip

These scripts can serve as a starting point for the development of custom scripts.

15.4. Status Codes

GMP uses status codes similar to HTTP status codes. The following codes are used:

2xx:

The command was sent, understood and accepted successfully.

200: OK
201: Resource created
202: Request submitted
4xx:

A user error occurred.

400: Syntax error

This includes different syntax errors. Often elements or attributes in the GMP command are missing. The status text shows additional information.

Currently this status code is also used for missing or wrong authentication.

401: Authenticate First
This is the error code that is used for missing or wrong authentication. Currently the value 400 is still used.
403: Access to resource forbidden
This is the error code that is used for not having enough permissions. Often 400: Permission denied is displayed instead as well.
404: Resource missing
The resource could not be found. The resource ID was empty or wrong.
409: Resource busy
This error code happens, for example, if the feed synchronization is started while it is already in progress.
5xx:

A server error occurred

500: Internal Error
This can be caused by entries that exceed an internal buffer size.
503: Scanner loading NVTs
The scanner is currently busy loading the NVTs from its cache. The request should be made again at a later time.
503: Service temporarily down
Possibly the scanner daemon is not running. Often the problem are expired certificates.
503: Service unavailable
The GMP command is blocked on the GSM.