Skip to content

Commit

Permalink
bug: Fix excl docs, switch to Buffer.byteLength
Browse files Browse the repository at this point in the history
  • Loading branch information
russellsteadman committed Jan 30, 2022
1 parent aac0d17 commit 00f8bd5
Show file tree
Hide file tree
Showing 4 changed files with 95 additions and 94 deletions.
4 changes: 2 additions & 2 deletions packages/exclusion/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ for most simple bot use-cases.
## Usage

```js
const Exclusion = require('exclusion');
const { RobotsTxt } = require('exclusion');

// Fetch a robots.txt file...

// Pass a robots.txt string to initialize the parser
const robotsTxt = new Exclusion(myRobotsTxtString);
const robotsTxt = new RobotsTxt(myRobotsTxtString);

// Check a path
robotsTxt.isPathAllowed('/certain/path', 'MyUserAgent');
Expand Down
178 changes: 89 additions & 89 deletions packages/exclusion/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions packages/exclusion/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "exclusion",
"version": "0.1.0",
"version": "0.1.1",
"description": "A simple, RFC-compliant robots.txt parser",
"author": "Russell Steadman",
"license": "MIT",
Expand All @@ -18,7 +18,8 @@
},
"repository": {
"type": "git",
"url": "git+https://github.com/russellsteadman/netscrape.git"
"url": "https://github.com/russellsteadman/netscrape.git",
"directory": "packages/exclusion"
},
"keywords": [
"robotstxt",
Expand Down
2 changes: 1 addition & 1 deletion packages/exclusion/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ export class RobotsTxtLine {
// Normalize the path
this.value = normalizePath(this.value);
this.prematch = prematch(this.value, { isStrictEOL: this.strictEOL });
this.priority = Buffer.from(this.value).length;
this.priority = Buffer.byteLength(this.value);
}
}

Expand Down

0 comments on commit 00f8bd5

Please sign in to comment.