Skip to content
This repository has been archived by the owner on Mar 22, 2023. It is now read-only.

Commit

Permalink
refactor:file download (#16)
Browse files Browse the repository at this point in the history 
* refactor:file download

* refactor:provide file save path option

Co-authored-by: tangyoha <[email protected]>
  • Loading branch information
@tangyoha
tangyoha and tangyoha authored Dec 7, 2022
1 parent 68636da commit 5a584cf
Show file tree
Hide file tree
Showing 17 changed files with 941 additions and 264 deletions.
2 changes: 0 additions & 2 deletions .github/FUNDING.yml
View file

This file was deleted.

2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/documentation-improvement.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ assignees: ''

#### Location of the documentation

[this should provide the location of the documentation, e.g. "CONTRIBUTION.md" or the URL of the documentation, e.g. "https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/CONTRIBUTING.md"]
[this should provide the location of the documentation, e.g. "CONTRIBUTION.md" or the URL of the documentation, e.g. "https://github.com/tangyoha/telegram_media_downloader/blob/master/CONTRIBUTING.md"]

#### Documentation problem

Expand Down
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ Please take a moment to review this document in order to make the contribution p

### Where do I go from here?

If you've noticed a bug or have a feature request, [make one](https://github.com/Dineshkarthik/telegram_media_downloader/issues)! It's generally best if you get confirmation of your bug or approval for your feature request this way before starting to code.
If you've noticed a bug or have a feature request, [make one](https://github.com/tangyoha/telegram_media_downloader/issues)! It's generally best if you get confirmation of your bug or approval for your feature request this way before starting to code.

If you have a general question about telegram-media-downloader, you can ask it on [Discussion](https://github.com/Dineshkarthik/telegram_media_downloader/discussions) under `Q&A` category and any ideas/suggestions goes under `Ideas` category, the issue tracker is only for bugs and feature requests.
If you have a general question about telegram-media-downloader, you can ask it on [Discussion](https://github.com/tangyoha/telegram_media_downloader/discussions) under `Q&A` category and any ideas/suggestions goes under `Ideas` category, the issue tracker is only for bugs and feature requests.

### Fork & create a branch

Expand Down Expand Up @@ -66,13 +66,13 @@ Please follow these coding standards when writing code for inclusion in telegram

Telegram-media-downloader follows the [PEP8](https://www.python.org/dev/peps/pep-0008/) standard and uses [Black](https://black.readthedocs.io/en/stable/) and [Pylint](https://pylint.pycqa.org/en/latest/) to ensure a consistent code format throughout the project.

[Continuous Integration](https://github.com/Dineshkarthik/telegram_media_downloader/actions) using GitHub Actions will run those tools and report any stylistic errors in your code. Therefore, it is helpful before submitting code to run the check yourself:
[Continuous Integration](https://github.com/tangyoha/telegram_media_downloader/actions) using GitHub Actions will run those tools and report any stylistic errors in your code. Therefore, it is helpful before submitting code to run the check yourself:
```sh
black media_downloader.py utils
```
to auto-format your code. Additionally, many editors have plugins that will apply `black` as you edit files.

Writing good code is not just about what you write. It is also about _how_ you write it. During [Continuous Integration](https://github.com/Dineshkarthik/telegram_media_downloader/actions) testing, several tools will be run to check your code for stylistic errors. Generating any warnings will cause the test to fail. Thus, good style is a requirement for submitting code to telegram-media-downloader.
Writing good code is not just about what you write. It is also about _how_ you write it. During [Continuous Integration](https://github.com/tangyoha/telegram_media_downloader/actions) testing, several tools will be run to check your code for stylistic errors. Generating any warnings will cause the test to fail. Thus, good style is a requirement for submitting code to telegram-media-downloader.

This is already added in the repo to help contributors verify their changes before contributing them to the project:
```sh
Expand Down Expand Up @@ -196,4 +196,4 @@ Explain the motivation for the change in the commit message body. This commit me

### Code of Conduct

As a contributor, you can help us keep the community open and inclusive. Please read and follow our [Code of Conduct](https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/CODE_OF_CONDUCT.md).
As a contributor, you can help us keep the community open and inclusive. Please read and follow our [Code of Conduct](https://github.com/tangyoha/telegram_media_downloader/blob/master/CODE_OF_CONDUCT.md).
5 changes: 3 additions & 2 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,17 @@ dev_install: install
python3 -m pip install -r dev-requirements.txt

static_type_check:
mypy media_downloader.py utils --ignore-missing-imports
mypy media_downloader.py utils module --ignore-missing-imports

pylint:
pylint media_downloader.py utils -r y
pylint media_downloader.py utils module -r y

style_check: static_type_check pylint

test:
py.test --cov media_downloader --doctest-modules \
--cov utils \
--cov module \
--cov-report term-missing \
--cov-report html:${TEST_ARTIFACTS} \
--junit-xml=${TEST_ARTIFACTS}/media-downloader.xml \
Expand Down
121 changes: 73 additions & 48 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,55 @@
<h1 align="center">Telegram Media Downloader</h1>

<p align="center">
<a href="https://github.com/Dineshkarthik/telegram_media_downloader/actions"><img alt="Unittest" src="https://github.com/Dineshkarthik/telegram_media_downloader/workflows/Unittest/badge.svg"></a>
<a href="https://github.com/tangyoha/telegram_media_downloader/actions"><img alt="Unittest" src="https://github.com/tangyoha/telegram_media_downloader/workflows/Unittest/badge.svg"></a>
<a href="https://codecov.io/gh/Dineshkarthik/telegram_media_downloader"><img alt="Coverage Status" src="https://codecov.io/gh/Dineshkarthik/telegram_media_downloader/branch/master/graph/badge.svg"></a>
<a href="https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/LICENSE"><img alt="License: MIT" src="https://black.readthedocs.io/en/stable/_static/license.svg"></a>
<a href="https://github.com/tangyoha/telegram_media_downloader/blob/master/LICENSE"><img alt="License: MIT" src="https://black.readthedocs.io/en/stable/_static/license.svg"></a>
<a href="https://github.com/python/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
</p>

<h3 align="center">
<a href="https://github.com/Dineshkarthik/telegram_media_downloader/discussions/categories/ideas">Feature request</a>
<a href="./README_CN.md">中文</a><span> · </span>
<a href="https://github.com/tangyoha/telegram_media_downloader/discussions/categories/ideas">Feature request</a>
<span> · </span>
<a href="https://github.com/Dineshkarthik/telegram_media_downloader/issues">Report a bug</a>
<a href="https://github.com/tangyoha/telegram_media_downloader/issues">Report a bug</a>
<span> · </span>
Support: <a href="https://github.com/Dineshkarthik/telegram_media_downloader/discussions">Discussions</a>
Support: <a href="https://github.com/tangyoha/telegram_media_downloader/discussions">Discussions</a>
<span> & </span>
<a href="https://t.me/tgmdnews">Telegram Community</a>
</h3>

### Overview:
### Overview

Download all media files from a conversation or a channel that you are a part of from telegram.
A meta of last read/downloaded message is stored in the config file so that in such a way it won't download the same media file again.

### Support:
### Support

| Category | Support |
|--|--|
|Language | `Python 3.7 ` and above|
|Language | `Python 3.7` and above|
|Download media types| audio, document, photo, video, video_note, voice|

### ToDo:
### ToDo

- Add support for multiple channels/chats.

### Installation

For *nix os distributions with `make` availability

```sh
$ git clone https://github.com/Dineshkarthik/telegram_media_downloader.git
$ cd telegram_media_downloader
$ make install
git clone https://github.com/tangyoha/telegram_media_downloader.git
cd telegram_media_downloader
make install
```

For Windows which doesn't have `make` inbuilt

```sh
$ git clone https://github.com/Dineshkarthik/telegram_media_downloader.git
$ cd telegram_media_downloader
$ pip3 install -r requirements.txt
git clone https://github.com/tangyoha/telegram_media_downloader.git
cd telegram_media_downloader
pip3 install -r requirements.txt
```

## Configuration
Expand All @@ -52,31 +59,33 @@ All the configurations are passed to the Telegram Media Downloader via `config.

**Getting your API Keys:**
The very first step requires you to obtain a valid Telegram API key (API id/hash pair):
1. Visit [https://my.telegram.org/apps](https://my.telegram.org/apps) and log in with your Telegram Account.
2. Fill out the form to register a new Telegram application.
3. Done! The API key consists of two parts: **api_id** and **api_hash**.

1. Visit [https://my.telegram.org/apps](https://my.telegram.org/apps) and log in with your Telegram Account.
2. Fill out the form to register a new Telegram application.
3. Done! The API key consists of two parts: **api_id** and **api_hash**.

**Getting chat id:**

**1. Using web telegram:**
1. Open https://web.telegram.org/?legacy=1#/im
2. Now go to the chat/channel and you will see the URL as something like
- `https://web.telegram.org/?legacy=1#/im?p=u853521067_2449618633394` here `853521067` is the chat id.
- `https://web.telegram.org/?legacy=1#/im?p=@somename` here `somename` is the chat id.
- `https://web.telegram.org/?legacy=1#/im?p=s1301254321_6925449697188775560` here take `1301254321` and add `-100` to the start of the id => `-1001301254321`.
- `https://web.telegram.org/?legacy=1#/im?p=c1301254321_6925449697188775560` here take `1301254321` and add `-100` to the start of the id => `-1001301254321`.

1. Open <https://web.telegram.org/?legacy=1#/im>

2. Now go to the chat/channel and you will see the URL as something like
- `https://web.telegram.org/?legacy=1#/im?p=u853521067_2449618633394` here `853521067` is the chat id.
- `https://web.telegram.org/?legacy=1#/im?p=@somename` here `somename` is the chat id.
- `https://web.telegram.org/?legacy=1#/im?p=s1301254321_6925449697188775560` here take `1301254321` and add `-100` to the start of the id => `-1001301254321`.
- `https://web.telegram.org/?legacy=1#/im?p=c1301254321_6925449697188775560` here take `1301254321` and add `-100` to the start of the id => `-1001301254321`.

**2. Using bot:**

1. Use [@username_to_id_bot](https://t.me/username_to_id_bot) to get the chat_id of
- almost any telegram user: send username to the bot or just forward their message to the bot
- any chat: send chat username or copy and send its joinchat link to the bot
- public or private channel: same as chats, just copy and send to the bot
- id of any telegram bot


### config.yaml

```yaml
api_hash: your_api_hash
api_id: your_api_id
Expand All @@ -97,50 +106,66 @@ file_formats:
- epub
video:
- mp4
save_path: D:\telegram_media_downloader
file_path_prefix:
- chat_title
- media_datetime
disable_syslog:
- INFO
```
- api_hash - The api_hash you got from telegram apps
- api_id - The api_id you got from telegram apps
- chat_id - The id of the chat/channel you want to download media. Which you get from the above-mentioned steps.
- last_read_message_id - If it is the first time you are going to read the channel let it be `0` or if you have already used this script to download media it will have some numbers which are auto-updated after the scripts successful execution. Don't change it.
- ids_to_retry - `Leave it as it is.` This is used by the downloader script to keep track of all skipped downloads so that it can be downloaded during the next execution of the script.
- media_types - Type of media to download, you can update which type of media you want to download it can be one or any of the available types.
- file_formats - File types to download for supported media types which are `audio`, `document` and `video`. Default format is `all`, downloads all files.
- **api_hash** - The api_hash you got from telegram apps
- **api_id** - The api_id you got from telegram apps
- **chat_id** - The id of the chat/channel you want to download media. Which you get from the above-mentioned steps.
- **last_read_message_id** - If it is the first time you are going to read the channel let it be `0` or if you have already used this script to download media it will have some numbers which are auto-updated after the scripts successful execution. Don't change it.
- **ids_to_retry** - `Leave it as it is.` This is used by the downloader script to keep track of all skipped downloads so that it can be downloaded during the next execution of the script.
- **media_types** - Type of media to download, you can update which type of media you want to download it can be one or any of the available types.
- **file_formats** - File types to download for supported media types which are `audio`, `document` and `video`. Default format is `all`, downloads all files.
- **save_path** - The root directory where you want to store downloaded files.
- **file_path_prefix** - Store file subfolders, the order of the list is not fixed, can be randomly combined.
- `chat_title` - channel or group title, it will be chat id if not exist title.
- `media_datetime` - media date, also see pyrogram.types.Message.date.strftime("%Y_%m").
- `meida_type` - meida type, also see `media_types`.
- **disable_syslog** - You can choose which types of logs to disable,see `logging._nameToLevel`.

## Execution

```sh
$ python3 media_downloader.py
python3 media_downloader.py
```
All the downloaded media will be stored inside respective direcotry named in the same path as the python script.

| Media type | Download directory |
|--|--|
| audio | path/to/project/audio |
| document | path/to/project/document |
| photo | path/to/project/photo |
| video | path/to/project/video |
| voice | path/to/project/voice |
| voice_note | path/to/project/voice_note |
All downloaded media will be stored at the root of `save_path`.
The specific location reference is as follows:

The complete directory of video download is: `save_path`/`chat_title`/`media_datetime`/`meida_type`.
The order of the list is not fixed and can be randomly combined.
If the configuration is empty, all files are saved under `save_path`.

## Proxy

`socks4, socks5, http` proxies are supported in this project currently. To use it, add the following to the bottom of your `config.yaml` file

```yaml
proxy:
scheme: socks5
hostname: 11.22.33.44
hostname: 127.0.0.1
port: 1234
username: your_username
password: your_password
username: your_username(option)
password: your_password(option)
```

If your proxy doesn’t require authorization you can omit username and password. Then the proxy will automatically be enabled.

## Contributing

### Contributing Guidelines
Read through our [contributing guidelines](https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/CONTRIBUTING.md) to learn about our submission process, coding rules and more.

Read through our [contributing guidelines](https://github.com/tangyoha/telegram_media_downloader/blob/master/CONTRIBUTING.md) to learn about our submission process, coding rules and more.

### Want to Help?
Want to file a bug, contribute some code, or improve documentation? Excellent! Read up on our guidelines for [contributing](https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/CONTRIBUTING.md).

Want to file a bug, contribute some code, or improve documentation? Excellent! Read up on our guidelines for [contributing](https://github.com/tangyoha/telegram_media_downloader/blob/master/CONTRIBUTING.md).

### Code of Conduct
Help us keep Telegram Media Downloader open and inclusive. Please read and follow our [Code of Conduct](https://github.com/Dineshkarthik/telegram_media_downloader/blob/master/CODE_OF_CONDUCT.md).

Help us keep Telegram Media Downloader open and inclusive. Please read and follow our [Code of Conduct](https://github.com/tangyoha/telegram_media_downloader/blob/master/CODE_OF_CONDUCT.md).
Loading

0 comments on commit 5a584cf

Please sign in to comment.