Allow passing tag_filter through test_settings

[bhagatali]

I'm submitting a ...

[ ] bug report
[X] feature request
[ ] support request => Please do not submit support request here, instead use [Stack Overflow](https://stackoverflow.com/questions/tagged/nightwatch.js+cucumber)

Current behavior

I am trying out using nightwatch-cucumber for our e2e testing. The website I work for is served in multiple languages. I want to have the capability to run all my e2e scenarios in parallel for multiple languages, so my nightwatch.conf.js is setup with multiple test_settings that run all my e2e scenarios in parallel.

I am using the programmatical execution to start the tests and I am able to pass on additional cucumber options to my tests. Having this setup allows me to pass my required tags to run my scenarios. But as best as I understand, the tags apply to all test_settings I have in my nightwatch.conf.js.

Expected/desired behavior

I am hoping to have tags passed for each test_settings through my nightwatch config?

In summary, if I have a desiredCapability that is running intended tests for English, I want to be able to pass a cucumber tag of @english. And a desiredCapability that is running intended tests for French, I want to be able to pass a cucumber tag of @french to it.

Reproduction of the problem

If I was only using nightwatch I could do something like this (please notice the tag_filter mentioned as yahoo in the test_settings):

const seleniumServer = require('selenium-server')
const phantomjs = require('phantomjs-prebuilt')
const chromedriver = require('chromedriver')

module.exports = {
  src_folders : ["features"],
  output_folder: 'reports',
  custom_assertions_path: '',
  live_output: false,
  disable_colors: false,
  test_workers: true,
  selenium: {
    start_process: true,
    server_path: seleniumServer.path,
    log_path: '',
    host: '127.0.0.1',
    port: 4444
  },
  test_settings: {
    default: {
      launch_url: 'http://localhost:8087',
      selenium_port: 4444,
      selenium_host: '127.0.0.1',
      desiredCapabilities: {
        browserName: 'chrome',
        javascriptEnabled: true,
        acceptSslCerts: true,
        'phantomjs.binary.path': phantomjs.path
      },
      tag_filter: "yahoo",
      screenshots : {
        enabled : true,
        on_failure : true,
        path: 'reports/screenshots/default'
      }
    },
    chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        javascriptEnabled: true,
        acceptSslCerts: true
      },
      selenium: {
        cli_args: {
          'webdriver.chrome.driver': chromedriver.path
        }
      }
    },
    firefox: {
      desiredCapabilities: {
        browserName: 'firefox',
        javascriptEnabled: true,
        acceptSslCerts: true
      }
    }
  }
}

Passing in the tags as below works in nightwatch-cucumber:

const nightwatch = require('nightwatch');

const argv = {
  config: 'nightwatch.conf.js',
  env: 'chrome',
  filter: '',
  tag: 'yahoo',
  skiptags: 'yahoo-search',
  group: 'search-engine'
};

nightwatch.runner(argv, () => {console.log('Complete!')})

But if I was to pass in tag_filter in my nightwatch.conf.js config when I use nightwatch-cucumber the code strips out any tags passed in through test_settings in favor of tags passed as arguments into the nightwatch runner.

What is the motivation / use case for changing the behavior?

Our use case to change this behavior is to allow my tests running in parallel to filter out scenarios based on tags passed in. As I mentioned above, we have around 500 scenarios we run for testing our website. Some of these scenarios are language specific and driven by tags. I want to be able to run test cases for different languages in parallel based on the tags passed in.

I want to be able to do something like this:

const seleniumServer = require('selenium-server')
const phantomjs = require('phantomjs-prebuilt')
const chromedriver = require('chromedriver')

module.exports = {
  src_folders : ["features"],
  output_folder: 'reports',
  custom_assertions_path: '',
  live_output: false,
  disable_colors: false,
  test_workers: true,
  selenium: {
    start_process: true,
    server_path: seleniumServer.path,
    log_path: '',
    host: '127.0.0.1',
    port: 4444
  },
  test_settings: {
    default: {
      launch_url: 'http://localhost:8087',
      selenium_port: 4444,
      selenium_host: '127.0.0.1',
      desiredCapabilities: {
        browserName: 'chrome',
        javascriptEnabled: true,
        acceptSslCerts: true,
        'phantomjs.binary.path': phantomjs.path
      },
      screenshots : {
        enabled : true,
        on_failure : true,
        path: 'reports/screenshots/default'
      }
    },
    english_chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        javascriptEnabled: true,
        acceptSslCerts: true
      },
      tag_filter: "english",
      selenium: {
        cli_args: {
          'webdriver.chrome.driver': chromedriver.path
        }
      }
    },
    french_chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        javascriptEnabled: true,
        acceptSslCerts: true
      },
      tag_filter: "french",
      selenium: {
        cli_args: {
          'webdriver.chrome.driver': chromedriver.path
        }
      }
    }
  }
}

Please tell us about your environment:

npm ls nightwatch nightwatch-cucumber && node --version
[email protected] /Users/abt236/abtesting-atdd-nightwatch
├── [email protected]
└── [email protected]

v8.11.1