WEBDEV-6884 update scripts execution

.github/workflows/call-run-tests.yml

@@ -7,25 +7,32 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
-      - uses: actions/setup-node@v3
+      - uses: actions/setup-node@v4
         with:
           node-version: 20
           cache: 'npm'
 
+      - name: Check npm version installed
+        run: |
+          npm -v
+          npx -v
+
       - name: Install dependencies
-        run: npm ci
+        run: |
+          npm ci
+          npm install -D @playwright/test@latest
 
       - name: Install Playwright Browsers
         run: npx playwright install --with-deps
 
       - name: Run Playwright tests
-        run: ./run-tests.sh
-
-      - uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: playwright-report
-          path: playwright-report/
-          retention-days: 1
+        run: npm run test 
+
+      # - uses: actions/upload-artifact@v4
+      #   if: always()
+      #   with:
+      #     name: playwright-report
+      #     path: playwright-report/
+      #     retention-days: 1

.github/workflows/donation-tests.yml

@@ -26,4 +26,4 @@ jobs:
         run: npx playwright install --with-deps
 
       - name: Run Donation Tests
-        run: npm run test:donation
+        run: npm run execute -- --test=donation

.gitignore

@@ -3,6 +3,7 @@ node_modules/
 /playwright-report
 /playwright/.cache
 /playwright/.auth
+/playwright-summary
 
 .env
 *.err

README.md

@@ -2,62 +2,123 @@
 
 # End to end tests for Archive.org using [Playwright](https://playwright.dev/)
 
+
 ## (Optional) BrowserStack Local Setup
 
-- login to BrowserStack and retrieve the account username and access key, see documentation [here](https://www.browserstack.com/docs/automate/playwright/getting-started/nodejs/test-runner)
+- Login to BrowserStack and retrieve the account username and access key, see documentation [here](https://www.browserstack.com/docs/automate/playwright/getting-started/nodejs/test-runner)
 
 
-## Running tests locally
+## Local Setup
 
-- install dependencies:
+- Install dependencies:
 
     `npm i`
 
-- install Playwright browser libs:
+- Install Playwright browser libs:
 
     `npx playwright install`
 
-- run all the tests in headless mode and generate 1 whole test report:
+- Create a `.env` file by copying the contents from `.env.sample` and add the respective values you want to use for testing
+
+    - this is required to run tests with loggedIn flows
+
+
+## Running tests
+
+- Run all tests: `npm run test`
+
+- Run tests using `generate` command:
+
+    - See `scripts/generateCommand.js`, update the arguments that starts at `line:36`
+
+    - Run the command: `npm run generate:command`, run the generated by the command
+
+- Run tests using custom execute script:
+
+    > Note:
+    >   - This will run a custom script that can take different shell script arguments/parameters for the following:
+    >   - The arrangement/order for passing of arguments can be jumbled for flexibility
+    >   - It's important to add `--` convention in CLI after the `npm command`
+    >       - It will separate npm-specific arguments from the arguments that are intended for the nodejs script
+    >       - Without `--`, it will not pass the arguments properly
+
+
+    - __Test category (by folder structure)__:
+        - It will run all tests if you didn't specify the test category you want to test
+        - This argument is annotated as `--test`
+        - This can only execute 1 category in 1 command
+        - Refer to the folder names in `tests` directory (`tests/*`), except for `tests/page-objects`
+        - Sample parameters: `about, av, books, collection, details, donation, home, login, music, profile, search`
+        - Sample execution:
+
+            `npm run execute -- --test=about`
 
-    `npm run test`
+            `npm run execute -- --test=books`
 
-- run all the tests in headless mode by each category and generate test report by category:
+            `npm run execute -- --test=donation`
 
-    `./run-tests.sh`
+            `npm run execute -- --test=profile`
 
-- run all the tests in headed mode (this will load multiple browsers):
+    - __Test execution mode__:
+        - The tests will run in headless mode by __default__ which will execute the tests in terminal only
+        - This argument is annotated as `--headed`
+        - Headed mode will spawn a browser window in the screen and execute the tests
+        - Sample execution:
 
-    `npm run test:headed`
+            `npm run execute -- --test=about --headed`
 
-- create a `.env` file by copying the contents from `.env.sample` and add the respective values you want to use for testing
+            `npm run execute -- --headed --test=about`
 
+    - __Test execution - [debug mode](https://playwright.dev/docs/running-tests#debug-tests-with-the-playwright-inspector)__:
+        - This will spawn a browser window and Playwright Inspector window in the screen which will help on inspecting the whole test execution flow step by step
+        - This argument is annotated as `--debug`
+        - Sample execution:
 
-## Running individual tests by category (headless mode)
+            `npm run execute -- --test=about --debug`
 
-- run about tests: `npm run test:about`
-- run av tests: `npm run test:av`
-- run books tests: `npm run test:books`
-- run collection tests: `npm run test:collection`
-- run details tests: `npm run test:details`
-- run home tests: `npm run test:home`
-- run login tests: `npm run test:login`
-- run music tests: `npm run test:music`
-- run search tests: `npm run test:search`
-- run profile tests: `npm run test:profile`
+            `npm run execute -- --debug --test=books`
 
+    - __Test execution - [UI mode](https://playwright.dev/docs/running-tests#debug-tests-in-ui-mode)__:
+        - This will spawn a browser window and Playwright Inspector window in the screen which will help on inspecting the whole test execution flow step by step
+        - This argument is annotated as `--ui`
+        - Sample execution:
 
-## Running individual tests by category (headed mode)
+            `npm run execute -- --test=about --ui`
 
-- run about tests: `npm run test:about:headed`
-- run av tests: `npm run test:av:headed`
-- run books tests: `npm run test:books:headed`
-- run collection tests: `npm run test:collection:headed`
-- run details tests: `npm run test:details:headed`
-- run home tests: `npm run test:home:headed`
-- run login tests: `npm run test:login:headed`
-- run music tests: `npm run test:music:headed`
-- run search tests: `npm run test:search:headed`
-- run profile tests: `npm run test:profile:headed`
+    - __Test execution with [Trace viewer](https://playwright.dev/docs/trace-viewer) mode__:
+        - Trace viewer will record the test run which will be included in the Playwright report
+        - This argument is annotated as `--trace`
+        - It is disabled by default, you can enable it with the command below
+        - Accepted parameters: `trace`
+        - Sample execution:
+
+            `npm run execute -- --test=about --trace`
+
+    - __Test execution by browser__:
+        - Tests will run in all browsers by default if you didn't specify the browser
+        - This argument is annotated as `--browser`
+        - Accepted parameters: `chromium, firefox, webkit`
+        - Sample execution:
+
+            `npm run execute -- --test=about --browser=chromium`
+
+            `npm run execute -- --test=about --browser=firefox --debug`
+
+            `npm run execute -- --test=about --browser=webkit --headed`
+
+    - __Test execution by test title__:
+        - This command will grep for test title specified in `.spec.ts` test description
+        - This argument is annotated as `--title`
+        - The custom script will treat it as the `-g` parameter or run as a `grep` command like how Playwright does it
+        - Tests will run in all browsers by default if you didn't specify the browser
+        - This can be combined with other commands if needed
+        - Sample execution:
+
+            `npm run execute -- --title="TV has borrow button"`
+
+            `npm run execute -- --title="Canonical About page has correct title and text" --browser=chromium`
+
+            `npm run execute -- --title="Canonical About page has correct title and text" --debug`
 
 
 ## Running tests using VSCode Playwright plugin
@@ -77,12 +138,16 @@
 - run command format: `npx playwright test <test-file-path> --debug`
 - sample: `npx playwright test tests/search/search-layout.spec.ts --debug`
 
+## Running specific test case title:
+
+- run command format: `npx playwright test -g "<test-case-title>"`
+- sample: `npx playwright test -g "TV has borrow button"`
+
 
 ## View tests execution result
 
 - run: `npm run show:report`
 
-
 ## Reference guide for writing tests
 
 - https://playwright.dev/docs/pom