Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

执行run.sh报错 #175

Open
Jerryye17 opened this issue Dec 28, 2024 · 1 comment
Open

执行run.sh报错 #175

Jerryye17 opened this issue Dec 28, 2024 · 1 comment

Comments

@Jerryye17
Copy link

ubantu24.04
所有安装都正常
也做了python的虚拟环境
也安装了playwright
但执行run.sh或者run_task.sh都存在以下报错:
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 0; desired_concurrency = 2; cpu = 0.0; mem = 0.0; event_loop = 0.0; client_info = 0.0
[crawlee.playwright_crawler._playwright_crawler] INFO Navigating to https://www.cnaiplus.com/a/news/?btwaf=75608141 ...
[crawlee.playwright_crawler._playwright_crawler] ERROR Request failed and reached maximum retries
Traceback (most recent call last):
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/crawlee/basic_crawler/_context_pipeline.py", line 65, in call
result = await middleware_instance.anext()
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/crawlee/playwright_crawler/_playwright_crawler.py", line 171, in _navigate
response = await context.page.goto(context.request.url)
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/playwright/async_api/_generated.py", line 8973, in goto
await self._impl_obj.goto(
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/playwright/_impl/_page.py", line 551, in goto
return await self._main_frame.goto(**locals_to_params(locals()))
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/playwright/_impl/_frame.py", line 145, in goto
await self._channel.send("goto", locals_to_params(locals()))
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 61, in send
return await self._connection.wrap_api_call(
File "/home/shuopei/conda-data/envs/wiseflow/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.TimeoutError: Page.goto: Timeout 30000ms exceeded.
Call log:
- navigating to "https://www.cnaiplus.com/a/news/?btwaf=75608141", waiting until "load"
[crawlee._autoscaling.autoscaled_pool] INFO Waiting for remaining tasks to finish
[crawlee.playwright_crawler._playwright_crawler] INFO Error analysis: total_errors=1 unique_errors=1
[crawlee.playwright_crawler._playwright_crawler] INFO Final request statistics:

image

@bigbrother666sh
Copy link
Member

等待 V0.3.6 或者升级 crawlee
pip install -U 'crawlee[playwright]'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants