23. OpenVAS Scanner Protocol

The OpenVAS Scanner Protocol (OSP) is a XML-based stateless request-response API that offers a unified abstraction for vulnerability scanners. The Greenbone Security Manager is able to seamless integrate OSP scanners into the vulnerability management. OSP scanners are all controlled in the same way and the results are stored in the database all in the same structures. An arbitrary number of different or same OSP scanners can be connected.

The term “Vulnerability Scanner” is a to be interpreted pretty far-reaching. It also could be about queries into a patch management system or a plain asset query. As a result of a scanner it is only expected that it is vulnerability details or asset information. The latter could be installed software packages, open ports, running services, TLS certificates or similar.

Some OSP scanner are integrated on the GSM appliance and can be enabled via GOS-Admin menu. But it is also possible to add remote OSP scanners.

23.1. Enabling additional OSP Scanners

Additional Scanners may be enabled through the GOS-Admin-Menu. The GSM comes with the OpenVAS scanner configured by default. No other scanner is enabled.

Note

Starting with GOS 3.1.17 pilot users can choose additional scanners. This feature will be available for all users through a later update. To become a pilot user, contact the Greenbone Support.

To enable the additional scanners enter the GOS-Admin-Menu and select Advanced/Scanner Management. The following menu will be presented:

_images/enable-scanner.png

Enabling additional scanners

To enable the w3af scanner select Add OSP w3af Scanner. Depending on the scanner chosen, some information might be displayed for acceptance:

_images/accept-scanner.png

Accept the terms and conditions

Then the scanner is created. The scanner will be available after an additional reboot.

_images/reboot-scanner.png

A reboot is required after enabling the scanner

Using the same menu you can update and disable the scanner as well.

23.2. How to write your own OSP wrapper

The OSP protocol documentation is available from the Greenbone website at

http://docs.greenbone.net/API/OSP/osp.html

23.2.1. The challenge: debsecan as OSP scanner

In the following we will work on the challenge to create a OSP wrapper for the tool “debsecan”. This tool is available for Debian GNU/Linux systems and can create a list of vulnerabilities the present system is subject to. The tool goes beyond a simple match about which package updates are missing. It rather does an online query to learn about software packages where security problems are identified, but not yet been finally processed to become a security update.

Further details about the tool are available on the debsecan homepage:

http://www.enyo.de/fw/software/debsecan/

For executing debsecan a low privileged user account is sufficient:

$ debsecan
CVE-2015-3333 chromium (remotely exploitable, high urgency)
CVE-2015-3334 chromium (remotely exploitable, medium urgency)
CVE-2015-3336 chromium (remotely exploitable, medium urgency)
TEMP-0000000-EA424A libbluray1
CVE-2014-9447 libelf1 (remotely exploitable, medium urgency)
CVE-2014-8354 libmagickcore5
CVE-2014-8355 libmagickcore5
CVE-2014-8562 libmagickcore5
CVE-2014-8716 libmagickcore5
TEMP-0000000-2FC21E libmagickcore5 (low urgency)
TEMP-0000000-7C079F libmagickcore5
TEMP-0000000-EEF23C libmagickcore5 (low urgency)
TEMP-0000000-FDAC72 libmagickcore5
TEMP-0773834-5EB6CF libmagickcore5
CVE-2013-4288 libpolkit-gobject-1-0 (low urgency)
CVE-2002-2439 libstdc++6-4.7-dev (low urgency)
CVE-2014-5044 libstdc++6-4.7-dev
...

As you can see, many vulnerabilities are already linked to a CVE. Now we want to make these information available to the Greenbone Security Manager (GSM).

23.2.2. The basis: ospd

After all, OSP is only a specification. So, you could implement an OSP wrapper with an arbitrary programming language.

However, to start immediately with the actual coupling you can use the readily available OpenVAS module “ospd”. It is written in Python and offers a base class as well as supporting functions. The entire server functionality including TLS encryption is already done.

You can download the current ospd package from this site:

http://www.openvas.org/install-source-de.html

We will work with version 1.0.0:

https://wald.intevation.org/frs/download.php/1999/ospd-1.0.0.tar.gz

And of course we will check the signature:

https://wald.intevation.org/frs/download.php/1999/ospd-1.0.0.tar.gz.sig

$ gpg --verify ospd-1.0.0.tar.gz.sig

$ tar xzf ospd-1.0.0.tar.gz
$ cd ospd-1.0.0/

We will now install the package at a temporary place:

$ mkdir /tmp/osptest
$ export PYTHONPATH=/tmp/osptest/lib/python2.7/site-packages/

Finally ospd is installed into that temporary directory:

$ python setup.py install --prefix=/tmp/osptest

Of course many roads lead to Rome. You could install the package to arbitrary other places and in other ways. If you are familiar with Python you may choose your preferred way.

23.2.3. The skeleton for the new OSP scanner

As a first step we create a new directory and the central file “wrapper.py”.

One important element is an inheritance of the “OSPDaemon” class with a very simple self information for the time being, which we will call “OSPDdebsecan”. And the other important element is the main routine of the service itself (“main”).

$ mkdir -p debsecan/ospd_debsecan
$ cd debsecan/ospd_debsecan
$ gvim wrapper.py
from ospd.ospd import OSPDaemon
from ospd.misc import main as daemon_main
from ospd_debsecan import __version__

class OSPDdebsecan(OSPDaemon):

    """ Class for ospd-debsecan daemon. """

    def __init__(self, certfile, keyfile, cafile):
        """ Initializes the ospd-debsecan daemon's internal data. """
        super(OSPDdebsecan, self).__init__(certfile=certfile, keyfile=keyfile,
                                    cafile=cafile)
        self.server_version = __version__
        self.scanner_info['name'] = 'debsecan'

    def check(self):
        """ Checks that debsecan command line tool is found and is executable. """
        return True

def main():
    """ OSP debsecan main function. """
    daemon_main('OSPD - debsecan wrapper', OSPDdebsecan)

Next we will complete the skeleton to obtain an operational, yet pretty stupid service. For this we create “__init_.py” in the same directory where “wrapper.py” is located:

__version__ = '1.0b1'

And finally the control module for the package: debsecan/setup.py:

from setuptools import setup

from ospd_debsecan import __version__

setup(
    name='ospd-debsecan',
    version=__version__,

    packages=['ospd_debsecan'],

    url='http://www.openvas.org',
    author='OpenVAS Development Team',
    author_email='info@openvas.org',

    license='GPLV2+',
    install_requires=['ospd==1.0.0'],

    entry_points={
        'console_scripts': ['ospd-debsecan=ospd_debsecan.wrapper:main'],
    },
)

With this structure the new server can be installed and started:

$ python setup.py install --prefix=/tmp/osptest

If not yet happened, you might need to adjust the search path for the new server:

$ export PATH=$PATH:/tmp/osptest/bin

This given, the server can be called directly:

$ ospd-debsecan --version
OSP Server for debsecan version 1.0b1
OSP Version: 1.0
Using: OSPd 1.0.0
Copyright (C) 2014, 2015 Greenbone Networks GmbH
License GPLv2+: GNU GPL version 2 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Lets next test the server functionality and send the server into the background so that it waits for request at port 2346 of our local system:

$ ospd-debsecan -b 127.0.0.1 -p 2346 \
  -k /tmp/osptest/var/lib/openvas/private/CA/clientkey.pem \
  -c /tmp/osptest/var/lib/openvas/CA/clientcert.pem \
  --ca-file /tmp/osptest/var/lib/openvas/CA/cacert.pem  &

The applied keys and certificates are those which were actually created for the OpenVAS scanner by the tool “openvas-mkcert”. For simplicity here we use use them instead of properly creating our own key pair. If you have installed OpenVAS scanner with a different prefix, then use that one instead of “/tmp/osptest/”.

As you can already suspect, the authentication for our OSP server works via client certificate. An authentication with a username and password is not supported.

With the “omp” command line tool of the “openvas-cli” package we can contact the server:

$ omp -h 127.0.0.1 -p 2346 --use-certs -X "<get_version/>"
<get_version_response status_text="OK" status="200"><protocol><version>1.0</versi...

As a default, the “omp” option “–use-certs” will use the standard paths for keys and certificates. Of course it works this way only if executed in the same environment as the server.

The response can be made nicer and easier to read with the additional option “–pretty-print”:

$ omp -h 127.0.0.1 -p 2346 --use-certs --pretty-print  -X "<get_version/>"
    <get_version_response status_text="OK" status="200">
      <protocol>
        <version>1.0</version>
        <name>OSP</name>
      </protocol>
      <daemon>
        <version>1.0.0</version>
        <name>OSPd</name>
      </daemon>
      <scanner>
        <version>No version</version>
        <name>debsecan</name>
      </scanner>
    </get_version_response>

Lets look at this result in the graphical user interface.

Via menu Configuration/Scanners we create a new scanner, see figure Creating a new OSP scanner. The certificates are the same as used for starting the scanner.

_images/create-osp-scanner.png

Creating a new OSP scanner

Directly after the creation of the new scanner, the details about the scanner are displayed, see figure Online response test for the OSP scanner. This “Online Response” shows the same version identification as above plus a parameter: “debug_mode” is a standard parameter of the base class “OSPDaemon”.

_images/osp-online-test.png

Online response test for the OSP scanner

So far well done: We have a working server. Unfortunately it can’t do anything of the actual aim. This will change now.

23.2.4. Establish connection between debsecan and OSP

For this we need to define another class for our OSPDaemon, “exec_scan”:

def exec_scan(self, scan_id, target):
    """ Starts the debsecan scanner for scan_id scan. """

    # run the debsecan command
    result = subprocess.check_output(["debsecan"])

    # parse the output of the debsecan command and create
    # respective alarms
    for line in result.split("\n"):
        words = line.split()
        if words.__len__() > 2 and words[0].split("-")[0] == "CVE":
            self.add_scan_alarm(scan_id, host=target, name=words[0],
                                value=line)

Like before we install and start the new version of the server and test the functionality at command line:

$ omp -h 127.0.0.1 -p 2346 --use-certs --pretty-print \
   -X "<start_scan target='localhost'><scanner_params/></start_scan>"

<start_scan_response status_text="OK" status="200">
  <id>8f48d691-c136-488f-8f31-d8761e1c75e1</id>
</start_scan_response>

This gives us the ID of our scan. Lets have a look at the scan details:

$ omp -h 127.0.0.1 -p 2346 --use-certs --pretty-print \
   -X "<get_scans scan_id='8f48d691-c136-488f-8f31-d8761e1c75e1'/>"

<get_scans_response status_text="OK" status="200">
  <scan id="8f48d691-c136-488f-8f31-d8761e1c75e1" target="localhost"
    end_time="1428004156" progress="100" start_time="1428004156">
    <results>
      <result host="localhost" severity="" test_id="" name="CVE-2015-3333"
       type="Alarm">CVE-2015-3333 chromium (remotely exploitable, high urgency)
      </result>
      <result host="localhost" severity="" test_id="" name="CVE-2015-3334"
       type="Alarm">CVE-2015-3334 chromium (remotely exploitable, medium urgency)
      </result>
      <result host="localhost" severity="" test_id="" name="CVE-2015-3336"
       type="Alarm">CVE-2015-3336 chromium (remotely exploitable, medium urgency)
      </result>
      <result host="localhost" severity="" test_id="" name="CVE-2014-9447"
       type="Alarm">CVE-2014-9447 libelf1 (remotely exploitable, medium urgency)
      </result>
      <result host="localhost" severity="" test_id="" name="CVE-2013-4288"
       type="Alarm">CVE-2013-4288 libpolkit-gobject-1-0 (low urgency)</result>
      <result host="localhost" severity="" test_id="" name="CVE-2002-2439"
       type="Alarm">CVE-2002-2439 libstdc++6-4.7-dev (low urgency)</result>
      <result host="localhost" severity="" test_id="" name="CVE-2013-2074"
       type="Alarm">CVE-2013-2074 kdelibs5-plugins (remotely exploitable, low urgency)
      </result>
    </results>
  </scan>
</get_scans_response>

This looks good indeed. Now for a scan via the GUI. For this we need a San Configuration for debsecan, even though it is actually empty. The creation is done via the menu Configuration/Scan Configs, see figure Creating a new Scan Config for OSP scanner debsecan.

_images/osp-scanconfig.png

Creating a new Scan Config for OSP scanner debsecan

Next lets create the task as shown in figure Creating a task for OSP scanner debsecan.

_images/osp-newtask.png

Creating a task for OSP scanner debsecan

Finally we start the task in the usual way and get the results illustrated in figure Scan results of OSP debsecan.

_images/osp-scanresults.png

Scan results of OSP debsecan

The severity of the results is automatically determined by the GSM based on the internal CVE database.

23.2.5. Result

With 40 lines of Python source code we established a OSP scanner connection that transfers the CVE vulnerability information of the tool debsecan into the Greenbone Security Manager.

When used as a regular task, we can now combine debsecan-based checks with scheduled execution, we can add notes or overrides and actually any other functionality that makes a full vulnerability management.

However, the OSP wrapper for debsecan is still a coarse one. It lacks error handling and by a better processing of the original output of debsecan we could achieve quite more information for the GSM.

Apart from such improvements, there is another desirable goal: The transformation of OSP-debsecan into a remote scanner. So far, the target system is ignored and instead always the local system is checked. With configured credentials on GSM-side, we could extend the OSP wrapper to log into the given target system, execute debsecan on that remote host and process the results in the usual way. This would allow to check entire networks without the need to install ospd-debsecan on each host. The OSP scanner “ospd-ovaldi” is an example for such a remote scanner.

The current state of OSP-debsecan of course is Open Source licensed under GPLv2+ and available here:

https://wald.intevation.org/scm/viewvc.php/trunk/osp-servers/debsecan/?root=openvas

23.2.6. Adding some error handling

OSP offers to send error messages in case some problem occurred during the scan. Such messages will occur in the “Errors” section in the user interface.

The method to be used here is “add_scan_error” and it works analog to the “add_scan_alarm” method. Launching the external tool “debsecan” could fail, for example if it is not installed, so lets handle this in the method “exec_scan”:

# run the debsecan command
try:
    result = subprocess.check_output(["debsecan"])
except:
    self.add_scan_error(scan_id, host=target,
      value="A problem occurred trying to execute 'debsecan'.")
    return