WEBDEV-6884 update scripts execution

.github/workflows/call-run-tests.yml

@@ -7,25 +7,32 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
-      - uses: actions/setup-node@v3
+      - uses: actions/setup-node@v4
         with:
           node-version: 20
           cache: 'npm'
 
+      - name: Check npm version installed
+        run: |
+          npm -v
+          npx -v
+
       - name: Install dependencies
-        run: npm ci
+        run: |
+          npm ci
+          npm install -D @playwright/test@latest
 
       - name: Install Playwright Browsers
         run: npx playwright install --with-deps
 
       - name: Run Playwright tests
-        run: ./run-tests.sh
-
-      - uses: actions/upload-artifact@v3
-        if: always()
-        with:
-          name: playwright-report
-          path: playwright-report/
-          retention-days: 1
+        run: npm run test 
+
+      # - uses: actions/upload-artifact@v4
+      #   if: always()
+      #   with:
+      #     name: playwright-report
+      #     path: playwright-report/
+      #     retention-days: 1

.github/workflows/donation-tests.yml

@@ -26,4 +26,4 @@ jobs:
         run: npx playwright install --with-deps
 
       - name: Run Donation Tests
-        run: npm run test:donation
+        run: npm run execute -- --test=donation

.gitignore

@@ -3,6 +3,7 @@ node_modules/
 /playwright-report
 /playwright/.cache
 /playwright/.auth
+/playwright-summary
 
 .env
 *.err

README.md

@@ -2,62 +2,167 @@
 
 # End to end tests for Archive.org using [Playwright](https://playwright.dev/)
 
+
 ## (Optional) BrowserStack Local Setup
 
-- login to BrowserStack and retrieve the account username and access key, see documentation [here](https://www.browserstack.com/docs/automate/playwright/getting-started/nodejs/test-runner)
+- Login to BrowserStack and retrieve the account username and access key, see documentation [here](https://www.browserstack.com/docs/automate/playwright/getting-started/nodejs/test-runner)
+
+
+## Local Setup
+
+- Install dependencies:
+
+    ```bash
+    npm i
+    ```
+
+- Install Playwright browser libs:
+
+    ```bash
+    npx playwright install
+    ```
+
+- Create a `.env` file by copying the contents from `.env.sample` and add the respective values you want to use for testing. This is required to run tests with loggedIn flows
+
 
+## Run all tests
 
-## Running tests locally
+- Run command:
 
-- install dependencies:
+    ```bash
+    npm run test
+    ```
 
-    `npm i`
 
-- install Playwright browser libs:
+## Running Tests with Custom Parameters
 
-    `npx playwright install`
+- To run a custom script with flexible shell script arguments/parameters, follow these steps:
 
-- run all the tests in headless mode and generate 1 whole test report:
+    1. **Command Structure:**
+        - Ensure to use the `--` convention after the npm command to separate npm-specific arguments from those intended for the Node.js script
+        - Without `--`, arguments may not pass correctly to the script
+
+    2. **Flexibility in Argument Order:**
+        - You can pass script arguments in any order to suit your needs for flexibility and customization
 
-    `npm run test`
 
-- run all the tests in headless mode by each category and generate test report by category:
+    ### Test Category (by Folder Structure)
 
-    `./run-tests.sh`
+    - Running tests from specific categories in the `tests` directory (except `tests/page-objects`):
+    - If no category is specified, all tests will run by default.
+    - Parameter: `--test`
+    - Example usage:
+        ```bash
+        npm run test -- --test=about
+        npm run test -- --test=books
+        npm run test -- --test=donation
+        npm run test -- --test=profile
+        ```
 
-- run all the tests in headed mode (this will load multiple browsers):
+    ### Test Execution Modes
 
-    `npm run test:headed`
+    - **Headed Mode:**
+        - Executes tests with a browser window visible on screen.
+        - Parameter: `--headed`
+        - Example usage:
+            ```bash
+            npm run test -- --test=about --headed
+            npm run test -- --headed --test=about
+            ```
 
-- create a `.env` file by copying the contents from `.env.sample` and add the respective values you want to use for testing
+    - **Debug Mode:**
+        - Executes tests with Playwright Inspector for step-by-step debugging.
+        - Parameter: `--debug`
+        - Example usage:
+            ```bash
+            npm run test -- --test=about --debug
+            npm run test -- --debug --test=books
+            ```
 
+    - **UI Mode:**
+        - Executes tests with Playwright Inspector for UI-based debugging.
+        - Parameter: `--ui`
+        - Example usage:
+            ```bash
+            npm run test -- --test=about --ui
+            ```
 
-## Running individual tests by category (headless mode)
+    - **Trace Viewer Mode:**
+        - Records test runs for inclusion in the Playwright report.
+        - Parameter: `--trace`
+        - Example usage:
+            ```bash
+            npm run test -- --test=about --trace
+            ```
 
-- run about tests: `npm run test:about`
-- run av tests: `npm run test:av`
-- run books tests: `npm run test:books`
-- run collection tests: `npm run test:collection`
-- run details tests: `npm run test:details`
-- run home tests: `npm run test:home`
-- run login tests: `npm run test:login`
-- run music tests: `npm run test:music`
-- run search tests: `npm run test:search`
-- run profile tests: `npm run test:profile`
+    ### Test Execution by Browser
 
+    - Running tests in specific browsers:
+    - By default, tests run in all available browsers.
+    - Parameter: `--browser`
+    - Accepted parameters: `chromium`, `firefox`, `webkit`
+    - Example usage:
+        ```bash
+        npm run test -- --test=about --browser=chromium
+        npm run test -- --test=about --browser=firefox --debug
+        npm run test -- --test=about --browser=webkit --headed
+        ```
 
-## Running individual tests by category (headed mode)
+    ### Test Execution by Test Title
 
-- run about tests: `npm run test:about:headed`
-- run av tests: `npm run test:av:headed`
-- run books tests: `npm run test:books:headed`
-- run collection tests: `npm run test:collection:headed`
-- run details tests: `npm run test:details:headed`
-- run home tests: `npm run test:home:headed`
-- run login tests: `npm run test:login:headed`
-- run music tests: `npm run test:music:headed`
-- run search tests: `npm run test:search:headed`
-- run profile tests: `npm run test:profile:headed`
+    - Running tests matching specific test case titles:
+    - Searches for test titles within `.spec.ts` files.
+    - Parameter: `--title`
+    - Example usage:
+        ```bash
+        npm run test -- --title="TV has borrow button"
+        npm run test -- --title="Canonical About page has correct title and text" --browser=chromium
+        npm run test -- --title="Canonical About page has correct title and text" --debug
+        ```
+
+    ### Use helper function to generate custom test commands
+
+    - To generate and execute custom test commands using the `generateCommand.js` script, follow these steps:
+
+    - **Setup script configuration:**
+        - Ensure you have `generateCommand.js` configured with the desired parameters (`test`, `browser`, `title`, `headed`, `trace`, `debug`, `ui`, `device`)
+
+        - Adjust these parameters as needed for your specific testing scenarios
+
+        Example configuration:
+        ```javascript
+        const sampleArgs = {
+            test: 'books',
+            browser: 'chromium', // Options: 'chromium', 'firefox', 'webkit'; default: all browsers
+            title: '', // Specify a test case title within ""
+            headed: false, // Set to true for headed mode
+            trace: false, // Set to true to enable trace mode
+            debug: false, // Set to true to enable debug mode
+            ui: true, // Set to true to enable UI mode
+            device: '' // Options: 'mobile', 'desktop'; default: desktop
+        };
+        ```
+
+    - **Generating custom test command:**
+
+        - Run the script to generate a command string that includes the specified parameters:
+
+            ```bash
+            npm run generate-command
+            ```
+
+        - This will output a generated command that can be used to execute specific tests based on your configuration:
+
+            ```bash
+            ➜ npm run generate:command
+
+            > [email protected] generate:command
+            > node scripts/generateCommand.js
+
+            Generated command: npm run test -- --test=books --browser=chromium --ui
+            ```
+
+        - Copy the generated command and execute it in your terminal to run tests with the specified configurations
 
 
 ## Running tests using VSCode Playwright plugin
@@ -67,20 +172,45 @@
 
 ## Running specific test spec by file:
 
-- run command format: `npx playwright test <test-file-path>`
-- sample: `npx playwright test tests/search/search-layout.spec.ts`
-- headed: `npx playwright test tests/search/search-layout.spec.ts --headed`
+- Command format: `npx playwright test <test-file-path>`
 
+- Sample (headless): 
+    ```bash
+    npx playwright test tests/search/search-layout.spec.ts
+    ```
+
+- Sample (headed): 
+    ```bash
+    npx playwright test tests/search/search-layout.spec.ts --headed
+    ```
 
 ## Running specific test spec by file in debug mode:
 
-- run command format: `npx playwright test <test-file-path> --debug`
-- sample: `npx playwright test tests/search/search-layout.spec.ts --debug`
+- Command format: `npx playwright test <test-file-path> --debug`
+
+- Sample: 
+    ```bash
+    npx playwright test tests/search/search-layout.spec.ts --debug
+    ```
+
+
+## Running specific test case title:
+
+- Command format: `npx playwright test -g "<test-case-title>"`
+
+- Sample: 
+    ```bash
+    npx playwright test -g "TV has borrow button"
+    ```
 
 
 ## View tests execution result
 
-- run: `npm run show:report`
+- Run command: 
+
+    ```bash
+    npm run show:report
+    ```
 
 
 ## Reference guide for writing tests