Integrating E2E and Application Security Testing: HOW To with NightwatchJS and OWASP ZAP

Team we45
April 18, 2018
Integrating E2E and Application Security Testing: HOW To with NightwatchJS and OWASP ZAP

The Problem

Ever since I started my journey in DevSecOps and Application Security Automation, one of the key areas of my work has been “Parameterized Scanning”. “Parameterized Scanning” started off when we were attempting to automate an application security test for one of our largest clients, a World’s Top 10 Travel Portal. Before Parameterized Scanning, the pen tester had to rely either on manually crawling through the application to identify parameters, consequently testing them to identify security vulnerabilities, or use the “spider” feature of the Web Application Scanner. The spider feature of most Web Application Vulnerability Scanners has become a relic that we should be associating with the dark ages. Why? Simply because traditional spidering simply does not work. Traditional spidering works on the principle of crawling a particular HTML page, discovering new links, forms, etc on the HTML page and attempting to identify more such pages with the same technique, recursively. This largely has been rendered obsolete today.

This creates a significant problem for DAST Scanners. Some of them provide “Recorded Authentication” settings. They figure that since “Authenticated Scans” (scans where you need to be authenticated as the user) are the only thing that’s important, they allow a user to record an authentication sequence in their scan tool and replay this scan sequence to recreate the authentication and subsequently, the scanner still resorts to spidering the application. This still doesn’t solve the problem, because spiders don't find anything anymore.


The Solution — Parameterized Scanning

Parameterized Scanning is the practice of creating/leveraging software test automation to use as an input for Application Security Testing. Let’s assume that your organization has a QA department. Let’s assume that this QA department creates test automation (End to End Tests, e2e test) in Selenium or similar framework. The idea is to leverage that existing test automation to serve as an input for the DAST Scanner. Essentially:

  1. The Test runs against the application, with the DAST scanner as the proxy
  2. Once the test completes the scan process, the DAST Scan starts scanning for security vulnerabilities
  3. Once the scan is done, the DAST scanner generates/publishes reports/artifacts that can be used by engineering teams to triage and fix the identified vulnerabilities

As you can see, this model has some non-trivial benefits (only a few are captured here):

We at we45, have created a host of solutions/tools that facilitate Parameterized Scanning, right from our product Orchestron Client to RoboZap, RoboArachni (Github link) etc that can be used by our customers and the community at large to perform Parameterized, Automated Scanning, leveraging existing or new e2e tests. However, thus far, my examples and work had largely been with Python based libraries like Python-Selenium, Python-requests among others.

I have written this blogpost to showcase an experiment in a JavaScript End-to-End Testing Framework. I have been working extensively with NodeJS recently, so I thought that I would give this a shot and write about some of my experiences for everyone’s benefit.

For this example, I have used the following:

The Good

The Bad:

Steps in the Example

The first thing we want to do is set up Nightwatch to proxy the browser’s traffic through our ZAP’s proxy port. In my example, this port is 8090 (ZAP). Here’s a gist of the Proxy Settings in nightwatch.conf.js. I have also setup a bunch of other settings in this file, including the ZAP executable path, the report path, etc.

const CHROME_CONFIGURATION = { browserName: 'chrome', javascriptEnabled: true, acceptSslCerts: true, chromeOptions: { args: [ '--proxy-server=' ] } };

//These are ZAP Specific Settings const OTHER_SETTINGS = { zap_jrpc_server: "http://localhost:4000/jsonrpc", zap_report_path: "/Users/abhaybhargav/Documents/Code/node/nightwatch_zap/report.json", zap_report_format: "json", test_report_title: "ZAP Test for weCare Application", test_report_author: "Abhay Bhargav", zap_policy_name: "Light" }; module.exports = { src_folders: ['tests'], selenium: SELENIUM_CONFIGURATION, test_settings: ENVIRONMENTS, other_settings: OTHER_SETTINGS //ZAP Settings }

The next thing I did, was setup a “before” test hook. True to the name, this is a test hook that is fired, BEFORE the test commences. What I like to do is to start the ZAP executable in this hook and get it ready to receive requests from the E2E test. I have executed ZAP in the GUI mode, but adding a headless flag when you are invoking the executable runs it without the GUI (needed in CI-style environments)

before: function(client, done) { ZapManager.startZap(done); setTimeout(() => { done(); }, 10000); },

Next, the E2E test executes and runs all the tests. Here, I have two very simple tests setup. This is your actual test.

"Login to weCare App": function(client) { client .url(client.launchUrl + "/login/") .waitForElementVisible("body",1000) .assert.visible("input[type=email]") .assert.visible("input[type=password]") .setValue("input[type=email]", '') .setValue("input[type=password]", 'secdevops') .click("button[id=submit]") .waitForElementVisible("body",1000) .pause(2000) }, "Search For Tests": function(client) { client .url(client.launchUrl + "/tests/") .waitForElementVisible("body",1500) .setValue("input[name=search]", "Liver") .click("input[name=look]") .waitForElementVisible("body",1000) },

Once the test executes, we want the browser to close. Subsequently, I signal to ZAP (through the ZAP JSON RPC service) to commence an “Active Scan” against the application. Once the Active Scan is triggered, its status (in percentage) is queried every 10 seconds. Be advised that with large apps, scans can take the proverbial “several hours”. Once the scan completes (100%), the result is written to the the report path (from nightwatch.conf.js) as a JSON document. Once all this is done, ZAP shuts down and all is well with the world All of this magic happens in the “after” test hook.

after: function(client, done) { //Browser closes here client.end(function() { done(); }); let scan_id; let scan_status = 0; //ZAP Scan Starts here setTimeout(() => {, { method: "start_zap_active_scan", params: { baseUrl: client.launchUrl, scan_policy: configs.other_settings.zap_policy_name, }, jsonrpc: "2.0", id: 1 }) .then(res => { scan_id = parseInt(; console.log("[+] ZAP Active Scan Started successfully with ID: " +; done(); }) .catch(err => { console.log(; }) },1500); //ZAP Active Scan Status is queried every 10 seconds until scan status reaches 100% let timerId = setInterval(() => {, { method: "get_ascan_status", params: [scan_id], jsonrpc: "2.0", id: 2 }) .then(res => { scan_status = parseInt(; console.log("[+] Scan Status: " + + "%")); }) .catch(err => { console.error(err); }); if (scan_status === 100) { clearInterval(timerId); //ZAP Report is written here setTimeout(() => {, { method: "write_json_report", params: { fullpath: configs.other_settings.zap_report_path, export_format: configs.other_settings.zap_report_format, report_title: configs.other_settings.test_report_title, report_author: configs.other_settings.test_report_author, }, jsonrpc: "2.0", id: 1 }) .then(res => { console.log("[+] " +; setTimeout(() => { ZapManager.stopZap(done); },400); done(); }) .catch(err => { console.error(err); }) },1000); done(); } }, 10000); done(); }

Tips and Tricks

This entire example can be downloaded from this repo: