Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: migrate build script to typescript #3423

Open
wants to merge 245 commits into
base: master
Choose a base branch
from

Conversation

JeelRajodiya
Copy link
Contributor

@JeelRajodiya JeelRajodiya commented Nov 22, 2024

Description

  • The PR migrates the build scripts to Typescript

Deploy Preview for asyncapi-website ready!

Built without sensitive environment variables

Name Link
🔍 Latest deploy log https://app.netlify.com/sites/asyncapi-website/deploys/678b5e3ee3013d00087ed64b
😎 Deploy Preview https://deploy-preview-3423--asyncapi-website.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Related issue(s)
Related to #3187

Summary by CodeRabbit

Release Notes

🚀 New Features

  • Enhanced internationalization configuration with next-i18next.config.cjs.
  • Introduced new functions for building structured lists of posts and case studies.
  • Added a utility function for reading and writing JSON files with error handling.
  • Introduced a new TypeScript declaration file for handling JSON modules.
  • Added TypeScript type definitions for RSS feeds and dashboard functionality.
  • New configuration for ESLint to allow more flexibility in coding practices.
  • New TypeScript interfaces and types for tools categorization and processing.
  • Introduced a new function for fetching and processing YouTube video data.
  • Added a logging utility using Winston for improved error reporting.

🔧 Improvements

  • Transitioned from CommonJS to ES module syntax in multiple scripts.
  • Improved error handling and type safety across various modules.
  • Updated script configurations for better TypeScript support.
  • Enhanced navigation and document processing logic with robust error handling.
  • Improved the structure of the Jest configuration for TypeScript compatibility.
  • Enhanced functionality for merging automated and manual tool data.
  • Improved handling of financial information processing.
  • Centralized error logging through a dedicated logger utility.

🧹 Chores

  • Refactored import/export statements for consistency.
  • Standardized code formatting across the project.
  • Updated Jest and Babel configurations for TypeScript compatibility.
  • Cleaned up test files to reflect updated import styles.

🔒 Security & Performance

  • Implemented more robust error handling in utility and build scripts.
  • Enhanced type safety throughout the project.

💻 Developer Experience

  • Improved TypeScript type definitions for better clarity and usability.
  • Simplified module imports and exports for easier maintenance.
  • Enhanced script modularity for improved organization and readability.

Copy link
Contributor

coderabbitai bot commented Nov 22, 2024

Walkthrough

This pull request introduces a migration of the project's scripts and utilities from JavaScript to TypeScript, enhancing type safety and modernizing the codebase. The changes span multiple files across the project, focusing on transitioning to ES module syntax, adding TypeScript type definitions, and improving error handling. Key modifications include updating import/export statements, introducing type interfaces, and refactoring utility functions to utilize TypeScript's features. Additionally, new logging mechanisms are implemented to replace console logging for better control over error reporting.

Changes

File Path Change Summary
.eslintrc Added new override for scripts/**/* to disable import/no-extraneous-dependencies. Modified existing override for components/logos/* to disable max-len.
next-i18next.config.cjs Introduced new configuration file for next-i18next, specifying locales, default locale, and namespaces for internationalization.
next-i18next.config.js Deleted old configuration file for next-i18next.
package.json Added "type": "module", updated script commands to use tsx, replaced typescript dependency with tsx, updated typescript version, and added new devDependencies.
pages/_document.tsx Updated import path for i18nextConfig from .config to .config.cjs.
scripts/adopters/index.js Deleted file containing buildAdoptersList function.
scripts/adopters/index.ts Introduced new buildAdoptersList function to convert YAML to JSON.
scripts/build-docs.ts Transitioned to ES module syntax, added type safety, and enhanced navigation processing functions with error handling.
scripts/build-meetings.ts Transitioned to ES module syntax, improved error handling, and added Google Calendar API integration.
scripts/build-newsroom-videos.js Deleted old file that fetched video data from YouTube API.
scripts/build-newsroom-videos.ts Introduced new function to fetch videos from YouTube, with error handling and data processing.
scripts/build-pages.ts Transitioned to ES module syntax with updated function signatures and type annotations.
scripts/build-post-list.js Deleted old file that generated structured post lists.
scripts/build-post-list.ts Introduced enhanced functionality for generating structured post lists with robust error handling.
scripts/build-rss.js Deleted old file for generating RSS feeds.
scripts/build-rss.ts Introduced new function for generating RSS feeds with detailed type definitions and error handling.
scripts/tools/categorylist.js Deleted old file defining tool categories.
scripts/tools/categorylist.ts Introduced new TypeScript file defining tool categories with type safety.
scripts/utils.js Deleted old utility file.
scripts/utils.ts Introduced new utility functions with enhanced error handling and type definitions.
tsconfig.json Updated TypeScript configuration for improved readability and added target option.
types/packages/jgexml__json2xml.d.ts Introduced TypeScript declaration file for jgexml/json2xml.
types/packages/markdown-toc.d.ts Introduced TypeScript type declarations for markdown-toc module.
types/scripts/build-posts-list.ts Added new TypeScript interfaces and types related to table of contents and navigation.
types/scripts/build-rss.ts Added TypeScript type definitions related to RSS feeds.
types/scripts/dashboard.ts Introduced comprehensive TypeScript interfaces for dashboard functionality.
types/scripts/tools.ts Added new TypeScript interfaces and types related to tools and categorization.
utils/getStatic.ts Updated import statement for i18nextConfig.
utils/languageDetector.ts Updated import statement for i18nextConfig.
utils/staticHelpers.ts Minor string formatting change in generateCaseStudyContent.
tests/adopters/index.test.js Updated import statements to destructure imports.
tests/build-rss.test.js Updated import statement for rssFeed function to reflect change in export style.
tests/casestudies/index.test.js Updated import statement for buildCaseStudiesList function to reflect change in export style.
tests/finance/index.test.js Updated import statement for buildFinanceInfoList function to reflect change in export style.
tests/index.test.js Updated import statements for several functions to use destructuring syntax.
tests/readAndWriteJson.test.js Updated import statement for fs module to directly import promises API.
tests/dashboard/build-dashboard.test.js Introduced logging mechanism using logger utility, replacing console logging.
tests/markdown/check-markdown.test.js Updated to use logger utility for error logging instead of console.
tests/tools/combine-tools.test.js Introduced logger utility for error logging, replacing console.error.
tests/tools/tools-object.test.js Introduced logger utility for warning and error logging, replacing console.error.
tests/build-meetings.test.js Updated import path for meetingsData and added new test cases for error handling.
tests/build-newsroom-videos.test.js Updated import path for buildNewsroomVideos and added new test cases for error handling.
tests/fixtures/combineToolsData.js Introduced new object for tools with missing data, along with formatting changes.
tests/fixtures/dashboardData.js Reformatted object properties for improved readability without altering content.
tests/build-docs/addDocButtons.test.js Updated import statement for addDocButtons to reference TypeScript file.
tests/build-docs/buildNavTree.test.js Updated import path for buildNavTree and changed property access from bracket to dot notation.
tests/build-docs/convertDocPosts.test.js Updated import statement for convertDocPosts to reference TypeScript file.
tests/build-post-list.test.js Updated import statement for build-post-list to reference TypeScript file.
tests/build-tools.test.js Updated import statement for buildTools to reference TypeScript file.
tests/utils.test.js Updated import statement for convertToJson to reference TypeScript file.
scripts/markdown/check-edit-links.ts Transitioned to ES module syntax, added TypeScript interfaces, and improved error handling.
tests/markdown/check-edit-links.test.js Updated import statement for check-edit-links to reference TypeScript file and introduced logger utility for error logging.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant App
    participant Logger
    participant API

    User->>App: Initiate action
    App->>API: Request data
    API-->>App: Return data
    App->>Logger: Log success
    Logger-->>App: Log entry
    App-->>User: Display result
Loading

🐇 "In the code, we hop and play,
New types and scripts lead the way.
With logs to guide us, clear and bright,
Our TypeScript journey takes flight!
From JavaScript's past, we now depart,
With every change, we craft our art!" 🐇

Possibly related PRs

  • feat: added test for build-rss.js #3101: The main PR modifies ESLint configuration related to file handling in the scripts/**/* directory, which is relevant to the changes made in the build-rss.js script that also deals with file operations and error handling.
  • feat: add test for combine tools script #3136: The changes in the combine tools script involve error handling and file operations, which relate to the ESLint modifications that aim to improve coding practices in the scripts/**/* directory.
  • feat: add tests for build post list script #3284: The updates to the build-post-list.js script include error handling and file operations, which align with the ESLint configuration changes that affect how scripts are processed.

Suggested labels

ready-to-merge, gsoc

Suggested reviewers

  • derberg
  • akshatnema
  • magicmatatjahu
  • anshgoyalevil
  • Mayaleeeee
  • devilkiller-ag
  • sambhavgupta0705
  • asyncapi-bot-eve

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (10)
tests/build-meetings.test.js (2)

73-87: Improve performance by avoiding the delete operator.

Replace the delete operator with an undefined assignment for better performance:

- delete process.env.CALENDAR_SERVICE_ACCOUNT;
+ process.env.CALENDAR_SERVICE_ACCOUNT = undefined;
🧰 Tools
🪛 Biome (1.9.4)

[error] 74-74: Avoid the delete operator which can impact performance.

Unsafe fix: Use an undefined assignment instead.

(lint/performance/noDelete)


63-71: Standardize error assertion pattern across test cases.

For consistency, consider using the same error assertion pattern across all error test cases. The expect().rejects.toThrow() pattern used in the newer tests is more concise and recommended:

- try {
-   await buildMeetings(outputFilePath);
- } catch (err) {
-   expect(err.message).toContain('Google API error');
- }
+ await expect(buildMeetings(outputFilePath)).rejects.toThrow('Google API error');

Also applies to: 89-99, 101-115

tests/build-newsroom-videos.test.js (3)

14-26: Enhance test isolation and cleanup.

Consider these improvements to the test setup:

  1. Clean up environment variables in afterAll
  2. Reset all mocks, not just fetch
 afterAll(() => {
   removeSync(testDir);
+  delete process.env.YOUTUBE_TOKEN;
 });

 beforeEach(() => {
-  fetch.mockClear();
+  jest.clearAllMocks();
 });

36-43: Extract URL parameters as constants.

Consider extracting the URL parameters into named constants at the top of the file for better maintainability and reusability across tests.

+const YOUTUBE_API_CONFIG = {
+  BASE_URL: 'https://youtube.googleapis.com/youtube/v3/search',
+  CHANNEL_ID: 'UCIz9zGwDLbrYQcDKVXdOstQ',
+  MAX_RESULTS: '5',
+};

 const expectedUrl = new URL('https://youtube.googleapis.com/youtube/v3/search');
-expectedUrl.searchParams.set('channelId', 'UCIz9zGwDLbrYQcDKVXdOstQ');
+expectedUrl.searchParams.set('channelId', YOUTUBE_API_CONFIG.CHANNEL_ID);

103-109: Improve environment variable handling in tests.

The current implementation has two minor issues:

  1. Using delete operator can impact performance
  2. Environment variable cleanup should be handled in afterEach for better test isolation
+beforeEach(() => {
+  fetch.mockClear();
+  process.env.YOUTUBE_TOKEN = 'testkey';
+});

+afterEach(() => {
+  process.env.YOUTUBE_TOKEN = undefined;
+});

 it('should throw an error if YOUTUBE_TOKEN environment variable is not set', async () => {
-  delete process.env.YOUTUBE_TOKEN;
+  process.env.YOUTUBE_TOKEN = undefined;
   await expect(buildNewsroomVideos('/path/to/write')).rejects.toThrow(
     'YOUTUBE_TOKEN environment variable is required'
   );
-  process.env.YOUTUBE_TOKEN = 'testkey';
 });
🧰 Tools
🪛 Biome (1.9.4)

[error] 104-104: Avoid the delete operator which can impact performance.

Unsafe fix: Use an undefined assignment instead.

(lint/performance/noDelete)

tests/markdown/check-markdown.test.js (2)

120-120: Remove commented out code.

Remove the commented out expectation as it adds unnecessary noise to the test file.

-    // expect(logger.error).toHaveBeenCalledWith('Error in directory', expect.any(Error));

34-166: Consider adding more edge cases to improve test coverage.

While the current test coverage is good, consider adding these test cases:

  1. Test with empty frontmatter object ({})
  2. Test with malformed YAML frontmatter
  3. Test with non-string values for URL fields
  4. Test with empty strings in required fields

Would you like me to help generate these additional test cases?

tests/build-docs/buildNavTree.test.js (3)

Line range hint 20-52: Maintain consistent property access syntax.

The code mixes dot notation (result.welcome, result.reference) with bracket notation (result['getting-started']). Choose one style for consistency:

  • Dot notation for better readability when properties are valid identifiers
  • Bracket notation when properties contain special characters or spaces
-    expect(result['getting-started'].item).toEqual(
+    expect(result.gettingStarted.item).toEqual(

65-66: Consider renaming properties to avoid spaces.

The property name 'Item without sectionId' contains spaces, requiring bracket notation. Consider using camelCase naming in the test fixtures for consistency and better TypeScript compatibility.

-    expect(result.root.children).toHaveProperty('Item without sectionId');
-    expect(result.root.children['Item without sectionId'].item).toEqual(
+    expect(result.root.children).toHaveProperty('itemWithoutSectionId');
+    expect(result.root.children.itemWithoutSectionId.item).toEqual(

113-118: Consider making sorting tests more robust.

The current tests rely on array indices which could be fragile. Consider:

  1. Using find() or filter() to locate items by title
  2. Adding explicit assertions for the presence of all expected items
  3. Verifying the array length to ensure no missing items
-    const apiChildren = result.reference.children.api.children;
-    expect(apiChildren[0].title).toBe('Authentication');
-    expect(apiChildren[1].title).toBe('Endpoints');
-    expect(apiChildren[2].title).toBe('Rate Limiting');
+    const apiChildren = result.reference.children.api.children;
+    expect(apiChildren).toHaveLength(3);
+    const [auth, endpoints, rateLimit] = apiChildren;
+    expect(auth.title).toBe('Authentication');
+    expect(endpoints.title).toBe('Endpoints');
+    expect(rateLimit.title).toBe('Rate Limiting');
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d365f5d and a1a9c86.

📒 Files selected for processing (11)
  • tests/adopters/index.test.js (1 hunks)
  • tests/build-docs/addDocButtons.test.js (7 hunks)
  • tests/build-docs/buildNavTree.test.js (4 hunks)
  • tests/build-docs/convertDocPosts.test.js (1 hunks)
  • tests/build-meetings.test.js (1 hunks)
  • tests/build-newsroom-videos.test.js (1 hunks)
  • tests/build-pages.test.js (2 hunks)
  • tests/build-post-list.test.js (10 hunks)
  • tests/build-tools.test.js (1 hunks)
  • tests/markdown/check-markdown.test.js (1 hunks)
  • tests/tools/tools-object.test.js (5 hunks)
✅ Files skipped from review due to trivial changes (4)
  • tests/build-docs/addDocButtons.test.js
  • tests/build-docs/convertDocPosts.test.js
  • tests/build-post-list.test.js
  • tests/build-tools.test.js
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/tools/tools-object.test.js
  • tests/build-pages.test.js
  • tests/adopters/index.test.js
🧰 Additional context used
🪛 Biome (1.9.4)
tests/build-meetings.test.js

[error] 74-74: Avoid the delete operator which can impact performance.

Unsafe fix: Use an undefined assignment instead.

(lint/performance/noDelete)

tests/build-newsroom-videos.test.js

[error] 104-104: Avoid the delete operator which can impact performance.

Unsafe fix: Use an undefined assignment instead.

(lint/performance/noDelete)

🔇 Additional comments (6)
tests/build-meetings.test.js (2)

29-39: LGTM! Well-structured test setup and teardown.

The test environment setup and cleanup are properly implemented with appropriate handling of directories and environment variables.


117-146: LGTM! Well-implemented error handling tests.

The new test cases for invalid data structure and missing dateTime property are well-implemented and provide good coverage for edge cases. They use the recommended expect().rejects.toThrow() pattern and have clear error messages.

tests/build-newsroom-videos.test.js (2)

5-5: Add .ts extension to the import statement.

Update the import statement to include the .ts extension for better module resolution.


1-109: Overall implementation looks good!

The test suite is well-structured with comprehensive coverage of success and error scenarios. The suggested improvements are minor and focus on maintainability rather than correctness.

🧰 Tools
🪛 Biome (1.9.4)

[error] 104-104: Avoid the delete operator which can impact performance.

Unsafe fix: Use an undefined assignment instead.

(lint/performance/noDelete)

tests/markdown/check-markdown.test.js (1)

18-32: Well-structured test setup and teardown!

Good practices observed:

  • Proper temporary directory management
  • Comprehensive mock setup and cleanup
  • Proper error handling
tests/build-docs/buildNavTree.test.js (1)

83-83: Well-structured error handling test!

Good addition of the error object existence check, ensuring both the error message and the error object itself are verified.

const { readFileSync, mkdirSync, rmSync } = require('fs');
const { buildMeetings } = require('../scripts/build-meetings');
const { mockEvents, expectedContent } = require('../tests/fixtures/meetingsData');
const { buildMeetings } = require('../scripts/build-meetings.ts');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update import statement to use TypeScript module syntax.

Since we're importing from a TypeScript file, consider updating the import statement to use ES modules syntax for consistency with TypeScript:

- const { buildMeetings } = require('../scripts/build-meetings.ts');
+ import { buildMeetings } from '../scripts/build-meetings';
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const { buildMeetings } = require('../scripts/build-meetings.ts');
import { buildMeetings } from '../scripts/build-meetings';

Comment on lines +150 to +157
it('should return true or false for URLs', () => {
expect(isValidURL('http://example.com')).toBe(true);
expect(isValidURL('https://www.example.com')).toBe(true);
expect(isValidURL('ftp://ftp.example.com')).toBe(true);
expect(isValidURL('invalid-url')).toBe(false);
expect(isValidURL('/path/to/file')).toBe(false);
expect(isValidURL('www.example.com')).toBe(false);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Remove duplicate test cases.

The following test cases are duplicated:

  1. URL validation test (lines 150-157 duplicates lines 141-148)
  2. Missing frontmatter test (lines 163-166 duplicates lines 159-162)

Remove the duplicate test cases to maintain clean and maintainable tests.

Also applies to: 163-166

Comment on lines +10 to +11
} = require('../../scripts/markdown/check-markdown.ts');
const { logger } = require('../../scripts/utils/logger.ts');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix incorrect TypeScript import extensions.

Node.js require statements cannot directly import TypeScript files. The .ts extension should be removed or changed to .js as the TypeScript files will be compiled to JavaScript before execution.

-} = require('../../scripts/markdown/check-markdown.ts');
-const { logger } = require('../../scripts/utils/logger.ts');
+} = require('../../scripts/markdown/check-markdown');
+const { logger } = require('../../scripts/utils/logger');
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
} = require('../../scripts/markdown/check-markdown.ts');
const { logger } = require('../../scripts/utils/logger.ts');
} = require('../../scripts/markdown/check-markdown');
const { logger } = require('../../scripts/utils/logger');

@@ -1,24 +1,23 @@
const { buildNavTree } = require('../../scripts/build-docs');
const { buildNavTree } = require('../../scripts/build-docs.ts');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix the import path for the TypeScript module.

The import statement with .ts extension won't work directly with Node.js/Jest. You should:

  1. Import the compiled JavaScript file instead of the TypeScript source
  2. Remove the file extension from the import path (let the module resolution handle it)
-const { buildNavTree } = require('../../scripts/build-docs.ts');
+const { buildNavTree } = require('../../scripts/build-docs');
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const { buildNavTree } = require('../../scripts/build-docs.ts');
const { buildNavTree } = require('../../scripts/build-docs');

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
tests/markdown/check-markdown.test.js (4)

13-15: Consider using jest.spyOn for logger mocks.

Instead of manually creating mock functions, consider using jest.spyOn for better type safety and automatic cleanup:

-jest.mock('../../scripts/utils/logger', () => ({
-  logger: { error: jest.fn(), warn: jest.fn() }
-}));
+jest.mock('../../scripts/utils/logger');
+
+beforeEach(() => {
+  jest.spyOn(logger, 'error').mockImplementation();
+  jest.spyOn(logger, 'warn').mockImplementation();
+});
+
+afterEach(() => {
+  jest.clearAllMocks();
+});

28-32: Improve error handling in cleanup.

The cleanup could fail silently. Consider adding error handling:

   afterEach(async () => {
     mockConsoleError.mockRestore();
     mockProcessExit.mockRestore();
-    await fs.rm(tempDir, { recursive: true, force: true });
+    try {
+      await fs.rm(tempDir, { recursive: true, force: true });
+    } catch (error) {
+      console.warn(`Failed to clean up temp directory: ${error.message}`);
+    }
   });

62-70: Improve test assertion specificity.

The test checks for errors in markdown files but doesn't verify the specific error message content. Consider adding more specific assertions:

   await checkMarkdownFiles(tempDir, validateBlogs);

-  expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('Errors in file invalid.md:'));
+  expect(logger.warn).toHaveBeenCalledWith(
+    expect.stringContaining('Errors in file invalid.md:'),
+    expect.arrayContaining([
+      'Date is required',
+      'Tags are required',
+      'Cover is required',
+      'Authors are required'
+    ])
+  );
   mockConsoleLog.mockRestore();

91-97: Add test coverage for specific error types.

The test verifies that an error is thrown but doesn't check specific error handling for different scenarios. Consider adding test cases for:

  • Permission denied errors
  • Invalid file format errors
  • Network errors when reading remote files
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a1a9c86 and a0d6ff8.

📒 Files selected for processing (2)
  • tests/markdown/check-markdown.test.js (1 hunks)
  • tests/utils.test.js (2 hunks)
✅ Files skipped from review due to trivial changes (1)
  • tests/utils.test.js
🔇 Additional comments (2)
tests/markdown/check-markdown.test.js (2)

10-11: Fix incorrect TypeScript import extensions.

Node.js require statements cannot directly import TypeScript files. The .ts extension should be removed or changed to .js as the TypeScript files will be compiled to JavaScript before execution.


150-157: Remove duplicate test cases.

The following test cases are duplicated:

  1. URL validation test (lines 150-157 duplicates lines 141-148)
  2. Missing frontmatter test (lines 163-166 duplicates lines 159-162)

Remove the duplicate test cases to maintain clean and maintainable tests.

Also applies to: 163-166

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
scripts/dashboard/build-dashboard.ts (2)

29-35: Consider extracting magic number to a named constant

The magic number 2592000 would be more maintainable as a named constant with clear units.

+const SECONDS_PER_MONTH = 30 * 24 * 60 * 60; // 30 days * 24 hours * 60 minutes * 60 seconds
+
 function monthsSince(date: string) {
   const seconds = Math.floor((new Date().valueOf() - new Date(date).valueOf()) / 1000);
-  // 2592000 = number of seconds in a month = 30 * 24 * 60 * 60
-  const months = seconds / 2592000;
+  const months = seconds / SECONDS_PER_MONTH;

259-262: Improve error handling in start function

Consider propagating the error to allow proper handling by callers.

   } catch (e) {
     logger.error('There were some issues parsing data from github.');
     logger.error(e);
+    throw e; // Propagate error to allow proper handling by callers
   }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a0d6ff8 and 1dfa3c7.

📒 Files selected for processing (1)
  • scripts/dashboard/build-dashboard.ts (1 hunks)
🔇 Additional comments (4)
scripts/dashboard/build-dashboard.ts (4)

1-20: Well-structured imports and type definitions!

The separation of type imports and the organization of dependencies is clean and follows TypeScript best practices.


21-23: Correct ESM path resolution implementation!

The use of fileURLToPath and dirname ensures proper path resolution in ESM context.


143-147: Remove non-null assertion for better type safety

The non-null assertion on reviews.nodes could lead to runtime errors.

-            discussion.reviews.nodes!.reduce((acc, curr) => acc + curr.comments.totalCount, 0)
+            (discussion.reviews.nodes?.reduce((acc, curr) => acc + curr.comments.totalCount, 0) ?? 0)

249-251: Type assertions are intentional

As confirmed in previous discussions, the type assertions are intentional and the types are well-understood.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (3)
scripts/build-rss.ts (3)

11-15: ⚠️ Potential issue

Refactor getAllPosts to prevent build system failures.

Based on previous learnings from PR #3101, making getAllPosts() asynchronous can cause build system failures. Additionally:

  1. Using any type assertion reduces TypeScript's type safety benefits
  2. Direct JSON import at build time might be problematic

Consider this alternative implementation:

-async function getAllPosts() {
-  const posts = ((await import('../config/posts.json')) as any).default;
-
-  return posts;
-}
+import type { BlogPost } from '@/types/scripts/build-rss';
+import posts from '../config/posts.json';
+
+interface Posts {
+  [key: string]: BlogPost[];
+}
+
+function getAllPosts(): Posts {
+  return posts;
+}

23-34: 🛠️ Refactor suggestion

Consider using a robust HTML sanitization library.

The current manual string replacements could miss edge cases or unhandled HTML entities, potentially leading to XSS vulnerabilities.

Consider using a dedicated HTML sanitization library like html-entities:

-function clean(s: string) {
-  let cleanS = s;
-
-  cleanS = cleanS.split('&ltspan&gt').join('');
-  cleanS = cleanS.split('&amp').join('&');
-  cleanS = cleanS.split('&#39;').join("'");
-  cleanS = cleanS.split('&lt;').join('<');
-  cleanS = cleanS.split('&gt;').join('>');
-  cleanS = cleanS.split('&quot;').join('"');
-
-  return cleanS;
-}
+import { decode } from 'html-entities';
+
+function clean(s: string): string {
+  return decode(s);
+}

129-129: 🛠️ Refactor suggestion

Replace dummy enclosure length.

Using a dummy value for the enclosure length could cause RSS reader compatibility issues.

Consider calculating the actual file size:

async function getImageSize(url: string): Promise<number> {
  try {
    const response = await fetch(url);
    const contentLength = response.headers.get('content-length');
    return contentLength ? parseInt(contentLength, 10) : 0;
  } catch (error) {
    console.warn(`Failed to get image size for ${url}:`, error);
    return 0;
  }
}
🧹 Nitpick comments (13)
scripts/build-docs.ts (3)

1-6: Optimize lodash imports for better tree-shaking.

Instead of importing the entire lodash library, use the modular import for better tree-shaking:

-import lodash from 'lodash';
-const { sortBy } = lodash;
+import { sortBy } from 'lodash';

115-117: Improve error message construction in buildNavTree.

The error message could be more descriptive by properly handling different error types:

-    throw new Error(`Failed to build navigation tree: ${err}`);
+    throw new Error(`Failed to build navigation tree: ${err instanceof Error ? err.message : String(err)}`);

184-239: Consider extracting navigation logic into smaller functions.

The map operation is quite complex with multiple nested conditions. Consider breaking it down into smaller, more focused functions for better maintainability:

  • Extract next page logic into getNextPage(post, index, structuredPosts)
  • Extract previous page logic into getPrevPage(post, index, structuredPosts, rootSections)

Example:

function getNextPage(index: number, structuredPosts: Details[]): NavigationPage {
  if (index + 1 >= structuredPosts.length) return {};
  
  if (!structuredPosts[index + 1].isRootElement && !structuredPosts[index + 1].isSection) {
    return {
      title: structuredPosts[index + 1].title,
      href: structuredPosts[index + 1].slug
    };
  }
  
  return {
    title: `${structuredPosts[index + 1].title} - ${structuredPosts[index + 2].title}`,
    href: structuredPosts[index + 2].slug
  };
}
scripts/build-rss.ts (1)

45-144: Consider breaking down the rssFeed function.

The function is quite long and handles multiple responsibilities. Consider splitting it into smaller, more focused functions for better maintainability.

Suggested breakdown:

  1. validatePosts: Handle post validation (lines 48-63, 88-92)
  2. createRSSChannel: Set up the RSS channel metadata (lines 68-86)
  3. createRSSItem: Create individual RSS items (lines 107-134)
  4. generateAndSaveXML: Handle XML generation and file writing (lines 136-140)
scripts/build-post-list.ts (3)

1-2: Consider addressing ESLint warnings instead of disabling them.

The ESLint rules no-await-in-loop and max-depth are disabled globally. These rules typically indicate potential performance issues and complex code structure.

Consider:

  1. Refactoring loops to use Promise.all() where possible
  2. Breaking down deeply nested code into smaller functions

35-47: Enhance the slugifyToC function implementation.

The function could be improved for better type safety and performance.

-export function slugifyToC(str: string) {
+export function slugifyToC(str: string): string {
+  // Pre-compile regex for better performance
+  static readonly HEADING_ID_REGEX = /[\s]*(?:\{#([a-zA-Z0-9\-_]+)\}|<a[\s]+name="([a-zA-Z0-9\-_]+)")/;
+
-  if (typeof str !== 'string') return '';
-  if (!str.trim()) return '';
+  if (typeof str !== 'string' || !str.trim()) return '';
   
   let slug = '';
   const idMatch = str.match(HEADING_ID_REGEX);
   const [, headingId, anchorId] = idMatch || [];
   
   slug = (headingId || anchorId || '').trim();
   
   return slug;
 }

258-284: Improve error handling in buildPostList.

The error handling could be enhanced with better typing and more specific error messages.

+type BuildPostListError = Error & {
+  code?: 'INVALID_BASE_PATH' | 'INVALID_WRITE_PATH' | 'EMPTY_DIRECTORIES' | 'PROCESSING_ERROR';
+};

 export async function buildPostList(
   postDirectories: string[][],
   basePath: string,
   writeFilePath: string
 ): Promise<void> {
   try {
     if (!basePath) {
-      throw new Error('Error while building post list: basePath is required');
+      const error: BuildPostListError = new Error('basePath is required');
+      error.code = 'INVALID_BASE_PATH';
+      throw error;
     }
     // ... rest of the validation
   } catch (error) {
-    throw new Error(`Error while building post list: ${(error as Error).message}`, { cause: error });
+    const buildError = error as BuildPostListError;
+    throw new Error(
+      `Build failed: ${buildError.code ? `[${buildError.code}] ` : ''}${buildError.message}`,
+      { cause: error }
+    );
   }
 }
scripts/markdown/check-markdown.ts (6)

16-25: Consider restricting URL protocols for better security.

The current URL validation accepts any valid URL format. Consider restricting to specific protocols (http/https) for better security.

 function isValidURL(str: string) {
   try {
     // eslint-disable-next-line no-new
-    new URL(str);
+    const url = new URL(str);
+    return ['http:', 'https:'].includes(url.protocol);
 
-    return true;
   } catch (err) {
     return false;
   }
 }

30-38: Enhance type safety with more specific types.

Consider using string literals for type and adding validation constraints through TypeScript.

 interface FrontMatter {
   title: string;
-  date: string;
+  date: `${number}-${number}-${number}`; // YYYY-MM-DD format
-  type: string;
+  type: 'blog' | 'doc';
   tags: string[];
   cover: string;
   weight?: number;
   authors: {
     name: string;
     link: string;
     photo: string;
   }[];
 }

63-65: Strengthen date validation.

The current date validation accepts any valid date format. Consider enforcing a specific format (e.g., YYYY-MM-DD).

-  if (frontmatter.date && Number.isNaN(Date.parse(frontmatter.date))) {
+  const dateRegex = /^\d{4}-\d{2}-\d{2}$/;
+  if (!frontmatter.date || !dateRegex.test(frontmatter.date) || Number.isNaN(Date.parse(frontmatter.date))) {
     errors.push(`Invalid date format: ${frontmatter.date}`);
   }

97-99: Remove unnecessary istanbul ignore.

The istanbul ignore next seems unnecessary here as this line should be covered by tests.

-  /* istanbul ignore next */
   return errors.length ? errors : null;

131-174: Consider batch processing and explicit error handling.

For large directories, processing all files in parallel might consume too much memory. Consider implementing batch processing and adding explicit error handling for file operations.

 async function checkMarkdownFiles(
   folderPath: string,
   validateFunction: (frontmatter: FrontMatter) => string[] | null,
   relativePath = ''
 ) {
   try {
     const files = await fs.readdir(folderPath);
-    const filePromises = files.map(async (file) => {
+    // Process files in batches of 50
+    const BATCH_SIZE = 50;
+    for (let i = 0; i < files.length; i += BATCH_SIZE) {
+      const batch = files.slice(i, i + BATCH_SIZE);
+      const filePromises = batch.map(async (file) => {
         const filePath = path.join(folderPath, file);
         const relativeFilePath = path.join(relativePath, file);

182-192: Implement graceful shutdown handling.

Consider implementing signal handlers for graceful shutdown, especially important for large directories.

 async function main() {
+  const shutdown = async (signal: string) => {
+    logger.warn(`Received ${signal}. Cleaning up...`);
+    // Allow ongoing validations to complete
+    await Promise.allSettled([/* ongoing promises */]);
+    process.exit(0);
+  };
+
+  process.on('SIGTERM', () => shutdown('SIGTERM'));
+  process.on('SIGINT', () => shutdown('SIGINT'));
+
   try {
     await Promise.all([
       checkMarkdownFiles(docsFolderPath, validateDocs),
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 29a71e7 and cfad022.

📒 Files selected for processing (7)
  • package.json (5 hunks)
  • scripts/build-docs.ts (1 hunks)
  • scripts/build-post-list.ts (1 hunks)
  • scripts/build-rss.ts (1 hunks)
  • scripts/markdown/check-markdown.ts (1 hunks)
  • scripts/tools/tools-object.ts (1 hunks)
  • tests/dashboard/build-dashboard.test.js (6 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/dashboard/build-dashboard.test.js
  • scripts/tools/tools-object.ts
  • package.json
🧰 Additional context used
📓 Learnings (1)
scripts/build-rss.ts (1)
Learnt from: akshatnema
PR: asyncapi/website#3101
File: scripts/build-rss.js:4-6
Timestamp: 2024-11-12T06:59:33.852Z
Learning: Converting `getAllPosts()` to asynchronous in `scripts/build-rss.js` causes the existing build system to fail.
🔇 Additional comments (8)
scripts/build-docs.ts (3)

133-134: Previous review comment about guarding against undefined access is still valid.


146-147: Previous review comment about error message construction is still valid.


247-247: LGTM!

The exports are clean and follow best practices.

scripts/build-rss.ts (1)

1-4: LGTM! Clean imports following TypeScript best practices.

The imports are well-organized, using ES module syntax and properly importing types from a dedicated types file.

scripts/build-post-list.ts (2)

16-23: Consider encapsulating global state.

The global mutable variables specWeight, finalResult, and releaseNotes should be encapsulated within a class or function scope to prevent potential issues in concurrent scenarios.


147-247: Refactor walkDirectories for better maintainability.

The function is complex and should be split into smaller, more focused functions with better error handling.

scripts/markdown/check-markdown.ts (2)

1-10: LGTM! Well-structured imports and setup.

The code uses modern ES modules, async file operations, and a custom logger utility instead of console.log.


110-119: Remove unnecessary istanbul ignores and improve test coverage.

The istanbul ignore else directives suggest incomplete test coverage. Consider adding test cases for these conditions instead of ignoring them.

*/
export async function rssFeed(type: BlogPostTypes, rssTitle: string, desc: string, outputPath: string) {
try {
let posts = (await getAllPosts())[`${type}`] as any[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove type assertion to improve type safety.

The type assertion to any[] reduces TypeScript's type safety benefits.

-  let posts = (await getAllPosts())[`${type}`] as any[];
+  let posts = (await getAllPosts())[type];

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines +156 to +246
for (const dir of directories) {
const directory = posix.normalize(dir[0]);
/* istanbul ignore next */
const sectionSlug = dir[1] || '';
const files = await readdir(directory);

for (const file of files) {
let details: Details;
const fileName = normalize(join(directory, file));
const fileNameWithSection = normalize(join(fileName, '_section.mdx'));
const slug = `/${normalize(relative(basePath, fileName)).replace(/\\/g, '/')}`;
const slugElements = slug.split('/');

if (await isDirectory(fileName)) {
if (await pathExists(fileNameWithSection)) {
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
details = frontMatter(await readFile(fileNameWithSection, 'utf-8'), {}).data as Details;
/* istanbul ignore next */
details.title = details.title || capitalize(basename(fileName));
} else {
details = {
title: capitalize(basename(fileName))
};
}
details.isSection = true;
if (slugElements.length > 3) {
details.parent = slugElements[slugElements.length - 2];
details.sectionId = slugElements[slugElements.length - 1];
}
if (!details.parent) {
details.isRootSection = true;
details.rootSectionId = slugElements[slugElements.length - 1];
}
details.sectionWeight = sectionWeight;
details.slug = slug;
addItem(details);
const rootId = details.parent || details.rootSectionId;

await walkDirectories(
[[fileName, slug]],
resultObj,
basePath,
details.title,
details.sectionId,
rootId,
details.sectionWeight
);
} else if (file.endsWith('.mdx') && !fileName.endsWith(`${sep}_section.mdx`)) {
const fileContent = await readFile(fileName, 'utf-8');
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
const { data, content } = frontMatter(fileContent, {});

details = data as Details;
details.toc = toc(content, { slugify: slugifyToC }).json;
details.readingTime = Math.ceil(readingTime(content).minutes);
details.excerpt = details.excerpt || markdownToTxt(content).substr(0, 200);
/* istanbul ignore next */
details.sectionSlug = sectionSlug || slug.replace(/\.mdx$/, '');
details.sectionWeight = sectionWeight;
details.sectionTitle = sectionTitle;
details.sectionId = sectionId;
details.rootSectionId = rootSectionId;
details.id = fileName.replace(/\\/g, '/');
details.isIndex = fileName.endsWith(join('index.mdx'));
details.slug = details.isIndex ? sectionSlug : slug.replace(/\.mdx$/, '');
if (details.slug.includes('/reference/specification/') && !details.title) {
const fileBaseName = basename(details.slug);
const versionDetails = getVersionDetails(details.slug, specWeight--);

details.title = versionDetails.title;
details.weight = versionDetails.weight;

if (releaseNotes.includes(details.title)) {
details.releaseNoteLink = `/blog/release-notes-${details.title}`;
}

details = handleSpecificationVersion(details, fileBaseName);
}

// To create a list of available ReleaseNotes list, which will be used to add details.releaseNoteLink attribute.
if (file.startsWith('release-notes') && dir[1] === '/blog') {
const { name } = parse(file);
const version = name.split('-').pop();

releaseNotes.push(version);
}

addItem(details);
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add proper error handling for file operations.

The walkDirectories function performs multiple file system operations without proper error handling.

Add try-catch blocks around file operations:

 for (const dir of directories) {
+  try {
     const directory = posix.normalize(dir[0]);
     const sectionSlug = dir[1] || '';
     const files = await readdir(directory);
 
     for (const file of files) {
       // ... rest of the code
     }
+  } catch (error) {
+    throw new Error(`Failed to process directory: ${error instanceof Error ? error.message : 'Unknown error'}`);
+  }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for (const dir of directories) {
const directory = posix.normalize(dir[0]);
/* istanbul ignore next */
const sectionSlug = dir[1] || '';
const files = await readdir(directory);
for (const file of files) {
let details: Details;
const fileName = normalize(join(directory, file));
const fileNameWithSection = normalize(join(fileName, '_section.mdx'));
const slug = `/${normalize(relative(basePath, fileName)).replace(/\\/g, '/')}`;
const slugElements = slug.split('/');
if (await isDirectory(fileName)) {
if (await pathExists(fileNameWithSection)) {
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
details = frontMatter(await readFile(fileNameWithSection, 'utf-8'), {}).data as Details;
/* istanbul ignore next */
details.title = details.title || capitalize(basename(fileName));
} else {
details = {
title: capitalize(basename(fileName))
};
}
details.isSection = true;
if (slugElements.length > 3) {
details.parent = slugElements[slugElements.length - 2];
details.sectionId = slugElements[slugElements.length - 1];
}
if (!details.parent) {
details.isRootSection = true;
details.rootSectionId = slugElements[slugElements.length - 1];
}
details.sectionWeight = sectionWeight;
details.slug = slug;
addItem(details);
const rootId = details.parent || details.rootSectionId;
await walkDirectories(
[[fileName, slug]],
resultObj,
basePath,
details.title,
details.sectionId,
rootId,
details.sectionWeight
);
} else if (file.endsWith('.mdx') && !fileName.endsWith(`${sep}_section.mdx`)) {
const fileContent = await readFile(fileName, 'utf-8');
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
const { data, content } = frontMatter(fileContent, {});
details = data as Details;
details.toc = toc(content, { slugify: slugifyToC }).json;
details.readingTime = Math.ceil(readingTime(content).minutes);
details.excerpt = details.excerpt || markdownToTxt(content).substr(0, 200);
/* istanbul ignore next */
details.sectionSlug = sectionSlug || slug.replace(/\.mdx$/, '');
details.sectionWeight = sectionWeight;
details.sectionTitle = sectionTitle;
details.sectionId = sectionId;
details.rootSectionId = rootSectionId;
details.id = fileName.replace(/\\/g, '/');
details.isIndex = fileName.endsWith(join('index.mdx'));
details.slug = details.isIndex ? sectionSlug : slug.replace(/\.mdx$/, '');
if (details.slug.includes('/reference/specification/') && !details.title) {
const fileBaseName = basename(details.slug);
const versionDetails = getVersionDetails(details.slug, specWeight--);
details.title = versionDetails.title;
details.weight = versionDetails.weight;
if (releaseNotes.includes(details.title)) {
details.releaseNoteLink = `/blog/release-notes-${details.title}`;
}
details = handleSpecificationVersion(details, fileBaseName);
}
// To create a list of available ReleaseNotes list, which will be used to add details.releaseNoteLink attribute.
if (file.startsWith('release-notes') && dir[1] === '/blog') {
const { name } = parse(file);
const version = name.split('-').pop();
releaseNotes.push(version);
}
addItem(details);
}
}
}
for (const dir of directories) {
try {
const directory = posix.normalize(dir[0]);
/* istanbul ignore next */
const sectionSlug = dir[1] || '';
const files = await readdir(directory);
for (const file of files) {
let details: Details;
const fileName = normalize(join(directory, file));
const fileNameWithSection = normalize(join(fileName, '_section.mdx'));
const slug = `/${normalize(relative(basePath, fileName)).replace(/\\/g, '/')}`;
const slugElements = slug.split('/');
if (await isDirectory(fileName)) {
if (await pathExists(fileNameWithSection)) {
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
details = frontMatter(await readFile(fileNameWithSection, 'utf-8'), {}).data as Details;
/* istanbul ignore next */
details.title = details.title || capitalize(basename(fileName));
} else {
details = {
title: capitalize(basename(fileName))
};
}
details.isSection = true;
if (slugElements.length > 3) {
details.parent = slugElements[slugElements.length - 2];
details.sectionId = slugElements[slugElements.length - 1];
}
if (!details.parent) {
details.isRootSection = true;
details.rootSectionId = slugElements[slugElements.length - 1];
}
details.sectionWeight = sectionWeight;
details.slug = slug;
addItem(details);
const rootId = details.parent || details.rootSectionId;
await walkDirectories(
[[fileName, slug]],
resultObj,
basePath,
details.title,
details.sectionId,
rootId,
details.sectionWeight
);
} else if (file.endsWith('.mdx') && !fileName.endsWith(`${sep}_section.mdx`)) {
const fileContent = await readFile(fileName, 'utf-8');
// Passing a second argument to frontMatter disables cache. See https://github.com/asyncapi/website/issues/1057
const { data, content } = frontMatter(fileContent, {});
details = data as Details;
details.toc = toc(content, { slugify: slugifyToC }).json;
details.readingTime = Math.ceil(readingTime(content).minutes);
details.excerpt = details.excerpt || markdownToTxt(content).substr(0, 200);
/* istanbul ignore next */
details.sectionSlug = sectionSlug || slug.replace(/\.mdx$/, '');
details.sectionWeight = sectionWeight;
details.sectionTitle = sectionTitle;
details.sectionId = sectionId;
details.rootSectionId = rootSectionId;
details.id = fileName.replace(/\\/g, '/');
details.isIndex = fileName.endsWith(join('index.mdx'));
details.slug = details.isIndex ? sectionSlug : slug.replace(/\.mdx$/, '');
if (details.slug.includes('/reference/specification/') && !details.title) {
const fileBaseName = basename(details.slug);
const versionDetails = getVersionDetails(details.slug, specWeight--);
details.title = versionDetails.title;
details.weight = versionDetails.weight;
if (releaseNotes.includes(details.title)) {
details.releaseNoteLink = `/blog/release-notes-${details.title}`;
}
details = handleSpecificationVersion(details, fileBaseName);
}
// To create a list of available ReleaseNotes list, which will be used to add details.releaseNoteLink attribute.
if (file.startsWith('release-notes') && dir[1] === '/blog') {
const { name } = parse(file);
const version = name.split('-').pop();
releaseNotes.push(version);
}
addItem(details);
}
}
} catch (error) {
throw new Error(`Failed to process directory: ${error instanceof Error ? error.message : 'Unknown error'}`);
}
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
scripts/markdown/check-edit-links.ts (1)

186-187: Log errors using logger.error before throwing in the main function

In the main function, if an error occurs, you throw a new error without logging it. It's a good practice to log errors before throwing them to aid in debugging and monitoring.

Apply this diff to improve error logging:

} catch (error) {
-    throw new Error(`Failed to check edit links: ${error}`);
+    logger.error(`Failed to check edit links: ${error}`);
+    throw new Error(`Failed to check edit links: ${error}`);
}
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cfad022 and c5928da.

📒 Files selected for processing (3)
  • package.json (5 hunks)
  • scripts/markdown/check-edit-links.ts (5 hunks)
  • tests/markdown/check-edit-links.test.js (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • package.json
🔇 Additional comments (3)
scripts/markdown/check-edit-links.ts (1)

2-2: ⚠️ Potential issue

Correct the module name for 'node-fetch' import

On line 2, the module 'node-fetch-2' is being imported. The standard package name is 'node-fetch'. Please verify the module name and ensure it matches the installed dependency in your project.

Apply this diff to correct the import statement:

-import fetch from 'node-fetch-2';
+import fetch from 'node-fetch';

If you intended to use a different package named 'node-fetch-2', please ensure it's properly installed and compatible with your codebase.

tests/markdown/check-edit-links.test.js (2)

10-12: Ensure compatibility when requiring TypeScript files in JavaScript tests

You're requiring TypeScript files directly in your JavaScript test file. This can cause runtime errors if your testing environment doesn't support importing TypeScript modules directly. Consider one of the following:

  • Compile the TypeScript files before running tests.
  • Convert your test file to TypeScript for consistency.
  • Adjust your test environment to handle TypeScript modules.

Please ensure that your test environment is configured to handle TypeScript modules appropriately.


Line range hint 2-2: Correct the module name for 'node-fetch' import

On line 2, the module 'node-fetch-2' is being required. The correct package name is 'node-fetch'. Ensure that the package name matches the installed dependency to prevent runtime errors.

Apply this diff to correct the require statement:

-const fetch = require('node-fetch-2');
+const fetch = require('node-fetch');

If a different package named 'node-fetch-2' is intended, verify that it's installed and compatible with your test environment.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (7)
scripts/markdown/check-edit-links.ts (7)

1-8: Consider using a more modern HTTP client.

The node-fetch-2 package is outdated. Consider using undici (Node.js built-in fetch) or node-fetch v3 for better maintenance and security.

-import fetch from 'node-fetch-2';
+import { fetch } from 'undici';

16-20: Enhance interface documentation.

The PathObject interface could benefit from JSDoc comments describing each property's purpose.

+/**
+ * Represents a markdown file's paths and its corresponding edit link
+ */
 interface PathObject {
+  /** The file system path to the markdown file */
   filePath: string;
+  /** The URL path where the file will be served */
   urlPath: string;
+  /** The GitHub edit link for the file, null if not applicable */
   editLink: string | null;
 }

Line range hint 27-52: Improve error handling in processBatch.

The error handling could be improved in several ways:

  1. The error is rejected without being logged
  2. The timeout logic could be extracted into a reusable utility

Consider this refactor:

+const checkWithTimeout = async (url: string, timeout: number) => {
+  const controller = new AbortController();
+  const timeoutId = setTimeout(() => controller.abort(), timeout);
+  try {
+    const response = await fetch(url, {
+      method: 'HEAD',
+      signal: controller.signal
+    });
+    return response;
+  } finally {
+    clearTimeout(timeoutId);
+  }
+};

 async function processBatch(batch: PathObject[]): Promise<(PathObject | null)[]> {
   const TIMEOUT_MS = Number(process.env.DOCS_LINK_CHECK_TIMEOUT) || 5000;

   return Promise.all(
     batch.map(async ({ filePath, urlPath, editLink }) => {
       try {
         if (!editLink || ignoreFiles.some((ignorePath) => filePath.endsWith(ignorePath))) return null;

-        const controller = new AbortController();
-        const timeout = setTimeout(() => controller.abort(), TIMEOUT_MS);
-        const response = await fetch(editLink, {
-          method: 'HEAD',
-          signal: controller.signal
-        });
-
-        clearTimeout(timeout);
+        const response = await checkWithTimeout(editLink, TIMEOUT_MS);
         if (response.status === 404) {
           return { filePath, urlPath, editLink };
         }

         return null;
       } catch (error) {
+        logger.error(`Error checking ${editLink}:`, error);
         return Promise.reject(new Error(`Error checking ${editLink}: ${error}`));
       }
     })
   );
 }

60-86: Improve type safety in checkUrls.

The function could benefit from stronger typing and better error handling:

  1. The batch size should be validated
  2. The pause duration should be configurable

Consider this enhancement:

+const DEFAULT_BATCH_SIZE = 5;
+const DEFAULT_PAUSE_DURATION = 1000;

-async function checkUrls(paths: PathObject[]): Promise<PathObject[]> {
+async function checkUrls(
+  paths: PathObject[],
+  options?: {
+    batchSize?: number;
+    pauseDuration?: number;
+  }
+): Promise<PathObject[]> {
   const result: PathObject[] = [];
-  const batchSize = Number(process.env.DOCS_LINK_CHECK_BATCH_SIZE) || 5;
+  const batchSize = Math.max(1, Number(process.env.DOCS_LINK_CHECK_BATCH_SIZE) || options?.batchSize || DEFAULT_BATCH_SIZE);
+  const pauseDuration = Math.max(0, Number(process.env.DOCS_LINK_CHECK_PAUSE_DURATION) || options?.pauseDuration || DEFAULT_PAUSE_DURATION);

   // ... rest of the function ...

-      await pause(1000);
+      await pause(pauseDuration);

95-113: Improve type safety in determineEditLink.

The function could benefit from a custom type for edit options and better null handling:

+interface EditOption {
+  /** The path segment to match against */
+  value: string;
+  /** The base URL for edit links */
+  href: string;
+}

 function determineEditLink(
   urlPath: string,
   filePath: string,
-  editOptions: { value: string; href: string }[]
+  editOptions: EditOption[]
 ): string | null {
   // Remove leading 'docs/' if present for matching
   const pathForMatching = urlPath.startsWith('docs/') ? urlPath.slice(5) : urlPath;

-  const target = editOptions.find((edit) => pathForMatching.includes(edit.value));
+  const target = editOptions.find((edit) => 
+    edit.value === '' || pathForMatching.includes(edit.value)
+  );

-  // Handle the empty value case (fallback)
-  if (target?.value === '') {
+  if (!target) {
+    return null;
+  }
+
+  if (target.value === '') {
     return `${target.href}/docs/${urlPath}.md`;
   }

-  // For other cases with specific targets
-  /* istanbul ignore next */
-  return target ? `${target.href}/${path.basename(filePath)}` : null;
+  return `${target.href}/${path.basename(filePath)}`;
 }

144-145: Remove unnecessary istanbul ignore.

The /* istanbul ignore else */ comment seems unnecessary as the else branch is covered by the subsequent else if condition.

-        /* istanbul ignore else */
-
         if (stats.isDirectory()) {

166-192: Improve main function configuration and error handling.

The main function could benefit from:

  1. Configuration options for batch size and pause duration
  2. Better error logging before throwing
+interface CheckLinksOptions {
+  batchSize?: number;
+  pauseDuration?: number;
+  timeout?: number;
+}

-async function main() {
+async function main(options?: CheckLinksOptions) {
   const editOptions = editUrls;

   try {
     const currentFilePath = fileURLToPath(import.meta.url);
     const currentDirPath = dirname(currentFilePath);
     const docsFolderPath = path.resolve(currentDirPath, '../../markdown/docs');
     const paths = await generatePaths(docsFolderPath, editOptions);

     logger.info('Starting URL checks...');
-    const invalidUrls = await checkUrls(paths);
+    const invalidUrls = await checkUrls(paths, {
+      batchSize: options?.batchSize,
+      pauseDuration: options?.pauseDuration
+    });

     if (invalidUrls.length > 0) {
       logger.info('\nURLs returning 404:\n');
       invalidUrls.forEach((url) => logger.info(`- ${url.editLink} generated from ${url.filePath}\n`));
       logger.info(`\nTotal invalid URLs found: ${invalidUrls.length}`);
     } else {
       logger.info('All URLs are valid.');
     }
   } catch (error) {
+    logger.error('Failed to check edit links:', error);
     throw new Error(`Failed to check edit links: ${error}`);
   }
 }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c5928da and 1cb5862.

📒 Files selected for processing (1)
  • scripts/markdown/check-edit-links.ts (5 hunks)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
.github/workflows/validate-case-studies-structures.yaml (2)

4-4: Fix trailing spaces.

Remove trailing spaces after push: to maintain consistent code style.

-  push: 
+  push:
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 4-4: trailing spaces

(trailing-spaces)


4-4: Consider adding path filters to the push trigger.

The current configuration will run on all pushes, which could be inefficient. Consider adding path filters to only run when relevant files are changed:

push:
  paths:
    - 'config/casestudies/*.yml'
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 4-4: trailing spaces

(trailing-spaces)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1cb5862 and cc194a6.

📒 Files selected for processing (1)
  • .github/workflows/validate-case-studies-structures.yaml (1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/validate-case-studies-structures.yaml

[error] 4-4: trailing spaces

(trailing-spaces)

.github/workflows/validate-case-studies-structures.yaml Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants