Parallel Execution by passing --workers=2 (any number greater than 1) not working.

[anupamtomar]

Parallel execution not working even though the workers' configuration has been done in the nightwatch.conf.js file.

Responding like below-

Note: but on the other hand, it works fine if executed sequentially using any of the below commands-
npx nightwatch -g Test\UI_Cases\SomeFolder\JWT --dev --workers=1
npx nightwatch -g Test\UI_Cases\SomeFolder\JWT --dev

Please find below the complete configuration of nightwatch.conf.js file we have-

module.exports = {
  src_folders: ['Test/UI_Cases'],
  custom_commands_path: ['./nightwatch/commands'],
  output_folder: 'reports',
  globals_path: '',

  test_workers: {
    enabled: true,
    workers: 1
  },
  detailed_output: false,
  live_output: true,

  screenshots: {
    enabled: true,
    path: 'reports/screenshots',
    on_error: true,
    on_failure: true
  },

  skip_testcases_on_fail: false,

  test_settings: {
    default: {
      desiredCapabilities: {
        browserName: 'chrome'
      },

      webdriver: {
        start_process: true,
      },
    },

  },

};

Please someone help me to identify if I am missing something.

[anupamtomar]

Hi @gravityvi,
it seems a bug since tried the same execution on auto-generated tests by Nightwatch using npm init nightwatch in a new and fresh setup.
could you please help me out here?

[anupamtomar]

Hi @gravityvi,
I am really struggling with this and did not get any leads till now, my same framework is working, fine with 1 worker but whenever I try parallel, it just exits with this coloured response.
moreover, I have observed during parallel execution debugging that I am not able to see my custom arguments as well as the nightwatch's arguments after execution reads nightwatch's conf file initially and figured out that it is a parallel run and it removes all the arguments given from terminal command and adds a new argument as below for each concurrent workers process,
below are all the process.argv I can see after parallel identification by Nightwatch-

0:
'/snap/node/7382/bin/node'
1:
'/home/myusername/somefolder/repos/project_name/node_modules/piscina/dist/src/worker.js'
2:
'--test-worker'
3:
'--parallel-mode'

when my framework does not find mandatory custom arguments then it exits the process

[anupamtomar]

Is there anyone in the Nightwatch Community who can help me out here??

[gravityvi]

Hey @anupamtomar can we connect on Nightwatch Discord server. I will try to help you over there. would need some details regarding your setup.

[gravityvi]

Hey @anupamtomar 👋 could you share verbose logs for the same?

[anupamtomar]

Hey @aakanksha88,
Nope, Still facing the same issue, and we have upgraded to Nightwatch 3.1.3 too.

Hi @gravityvi, Could you please help us out here?

[aakanksha88]

Thanks for the response @anupamtomar, will keep on monitoring this thread and also please let us know in case you get any resolution

[aakanksha88]

Hey @anupamtomar Could you please check and confirm whether you are also observing the same behavior - no. of test suites executed in parallel depends on the no. of CPU cores ?
No. of test suites which can be executed in parallel = No. of CPU cores of the system

[anupamtomar]

Hey @aakanksha88,
My Local system has 4 Cores but I am not even able to execute with 2 workers, Currently, I am able to run sequentially by passing a single worker. Moreover, if I try passing workers any number greater than 1 I get a colorful response as I have shared above in ScreenShot, and no execution happens. shared my nightwatch.conf.js file's configuration above too.

[aakanksha88]

Hey @anupamtomar thanks for sharing details, yes that is hard if you are not able to execute with more than 1 worker.

[aakanksha88]

Hi @gravityvi we are also facing similar issue when we are trying to upgrade Nightwatch Version to 3.1 from 2.6.19

During parallel execution on selenium grid or local, we see at the max 4 workers are only getting utilized for test execution whereas the number of workers we are passing through command line or even through nightwatch.json is more than 4 - we tried passing workers as 10,20, 30, more than 90 as well

Could you please help on this as we are not able to figure out what could be the issue

Thanks

[gravityvi]

@aakanksha88 Thanks for giving a detailed explanation for this. I will create a new issue and fix it.

[aakanksha88]

Hi @gravityvi Thank you for considering a fix for the issue.

Need some clarifications , could you please help providing same

1. As you replied earlier "After a certain point increase in the number of worker threads would lead to a decreased performance" , what could be the maximum number of threads that can be used in order to not see decreased performance
2. As per the release documentation for NightwatchJs Version Upgrade 3, as part of new features it is mentioned as

"Performance improvements
Nightwatch v3 has implemented worker threads for parallel runs and we have observed improvements of up to 25% between Nightwatch v2 & v3"

Will the fix have any impact on this ? Will we still see performance improvement in v3 using same number of workers threads as that of v2?

[gravityvi]

@aakanksha88 The number of threads depends more on the test workflow. Having more threads than the number of cores will cause each thread to steal time from the other threads. But at the same time rarely a thread uses 100% of CPU. I think a good point to start would be to use 1.5*number_of_cores.

Nightwatch v3 would have more performance than nightwatch v2 in any case. Nightwatch v3 uses worker threads as compared to Child processes in Nightwatch v2.

Also, join our discord server in case you have further queries. We would be happy to help you 🙂

[aakanksha88]

@gravityvi Good for now :) , thank you so much for all the clarifications

We will keep monitoring the issue opened by you for fixing this but still If could help with what would be the tentative timeframe for fix to be released so that we can plan our version upgrade accordingly.

Thank you once again.

[aakanksha88]

Hi @gravityvi Could you please let us know by when the fix for above issue will be available, thanks.