Skip to content

Commit

Permalink
docs: add pip section to README
Browse files Browse the repository at this point in the history
style: format code
  • Loading branch information
slashtechno committed Jun 17, 2024
1 parent d8dda06 commit f26359d
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 7 deletions.
20 changes: 16 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ An extremely customizable web scraper with a modular notification system and per
## Usage
### Prerequisites
- A browser
- Preferably a Firefox-based browser or some chromium-based browsers
- Most Chromium-based browsers and Firefox-based browsers should work
- Edge is not recommended
- Selenium should also be able to download and cache the appropriate browser if necessary
### Basic Configuration
- Configuration for the web scraper is handled through a TOML file
- To see an example configuration, see `config.example.toml`
Expand All @@ -35,11 +36,23 @@ An extremely customizable web scraper with a modular notification system and per
2. Change directory into the repository
- `cd scrape-and-ntfy`
3. Configure via `config.toml`
- Optionally, you can configure some other options via environment variables or command-line arguments
- Optionally, you can configure some other options via environment variables or command-line arguments in the `docker-compose.yml` file
4. `docker compose up -d`
- The `-d` flag runs the containers in the background
- If you want, you can run [`sqlite-web`](https://github.com/coleifer/sqlite-web) by uncommenting the appropriate lines in `docker-compose.yml` to view the database in a browser on [localhost:5050](http://localhost:5050)

### `pip`
#### Specific perquisites
- Python (3.11+)
#### Installation and usage
1. Install with `pip`
- `pip install scrape-and-ntfy`
- Depending on your system, you may need to use `pip3` instead of `pip` or `python3 -m pip`/`python -m pip`.
2. Configure
3. Run `scrape-and-ntfy`
- This assumes `pip`-installed scripts are in your `PATH`


### PDM
#### Specific perquisites
- Python (3.11+)
Expand All @@ -52,7 +65,6 @@ An extremely customizable web scraper with a modular notification system and per
3. Run `pdm install`
- This will install the dependencies in a virtual environment
- You may need to specify an interpreter with `pdm use`
4. Configure via `config.toml`
- Optionally, you can configure some other options via environment variables or command-line arguments
4. Configure
5. `pdm run python -m scrape_and_ntfy`
- This will run the bot with the configuration in `config.toml`
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "scrape-and-ntfy"
version = "0.1.1"
version = "0.1.2"
description = "An extremely customizable web scraper with a modular notification system and persistent storage via SQLite."
authors = [
{name = "slashtechno", email = "[email protected]"},
Expand All @@ -21,6 +21,7 @@ classifiers = [
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
]
keywords = ["web", "scraper", "ntfy", "webhooks", "docker"]

[build-system]
requires = ["pdm-backend"]
Expand Down
2 changes: 1 addition & 1 deletion src/scrape_and_ntfy/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ def main():
url=s["url"],
css_selector=s["css_selector"],
interval=s.get("interval", 60),
name = s.get("name", None),
name=s.get("name", None),
pause_time=s.get("pause_time", 0),
notifiers=notifiers,
scroll_to_bottom=s.get("scroll_to_bottom", False),
Expand Down
4 changes: 3 additions & 1 deletion src/scrape_and_ntfy/scraping/scraper.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,9 @@ def scrape_all_urls(cls):
scraper, message, notification_event
)
else:
message = f"Data unchanged for {scraper["name"]} with data \"{data}\""
message = (
f"Data unchanged for {scraper["name"]} with data \"{data}\""
)
cls.send_to_all_notifiers(
scraper, message, Notifier.NotifyOn.NO_CHANGE
)
Expand Down

0 comments on commit f26359d

Please sign in to comment.