Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7 #128

Open
pcace opened this issue Oct 9, 2024 · 14 comments
Open

Comments

@pcace
Copy link

pcace commented Oct 9, 2024

Hi there, i am trying to get the api to run. i have these variables:

OVERPASS_META=yes
OVERPASS_MODE=init
OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/germany-latest.osm.bz2
OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/germany/minute/
OVERPASS_RULES_LOAD=10

running the container results in this output:

2024-10-09T14:29:13.636953033Z 127.0.0.1 - - [09/Oct/2024:14:29:13 +0000] "GET /api/interpreter?data=[out:json];node(8951963);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:16.102558032Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:16.106452509Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:18.745591303Z 127.0.0.1 - - [09/Oct/2024:14:29:18 +0000] "GET /api/interpreter?data=[out:json];node(4672305);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:19.187224680Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:19.190564922Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:22.272327009Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:22.275038716Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:23.872252638Z 127.0.0.1 - - [09/Oct/2024:14:29:23 +0000] "GET /api/interpreter?data=[out:json];node(2776999);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:25.357138402Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:25.360897111Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:28.442788731Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:28.446887667Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:28.997656981Z 127.0.0.1 - - [09/Oct/2024:14:29:28 +0000] "GET /api/interpreter?data=[out:json];node(9516056);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:31.527795002Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:31.531359032Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:34.077307868Z 127.0.0.1 - - [09/Oct/2024:14:29:34 +0000] "GET /api/interpreter?data=[out:json];node(9722446);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:34.612661547Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:34.615430994Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:37.698880225Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:37.702605573Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:39.144882254Z 127.0.0.1 - - [09/Oct/2024:14:29:39 +0000] "GET /api/interpreter?data=[out:json];node(7325553);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:40.782719225Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:40.786851621Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:43.867553860Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:43.870391626Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:44.223971518Z 127.0.0.1 - - [09/Oct/2024:14:29:44 +0000] "GET /api/interpreter?data=[out:json];node(3939488);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:46.954178576Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:46.958584529Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:49.326078448Z 127.0.0.1 - - [09/Oct/2024:14:29:49 +0000] "GET /api/interpreter?data=[out:json];node(4775712);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:50.038335785Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:50.042264932Z It took 1 to run the loop. Desired load is: 10%. Sleeping: 9
2024-10-09T14:29:54.442149280Z 127.0.0.1 - - [09/Oct/2024:14:29:54 +0000] "GET /api/interpreter?data=[out:json];node(4924389);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:29:59.152285506Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:29:59.155667518Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:29:59.545315805Z 127.0.0.1 - - [09/Oct/2024:14:29:59 +0000] "GET /api/interpreter?data=[out:json];node(830640);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:30:02.239166904Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:30:02.242753044Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:30:04.604736471Z 127.0.0.1 - - [09/Oct/2024:14:30:04 +0000] "GET /api/interpreter?data=[out:json];node(7866842);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:30:05.326880762Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:30:05.331050567Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:30:08.412734108Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:30:08.416323538Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:30:09.696745534Z 127.0.0.1 - - [09/Oct/2024:14:30:09 +0000] "GET /api/interpreter?data=[out:json];node(9578284);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:30:11.498004250Z runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:30:11.501876967Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:30:13.488286607Z 2024-10-09 14:30:13 INFO: Using replication server at http://download.openstreetmap.fr/replication/europe/germany/minute/
2024-10-09T14:30:13.488306416Z 2024-10-09 14:30:13 DEBUG: Using given sequence ID 6296706
2024-10-09T14:30:13.488310916Z 2024-10-09 14:30:13 DEBUG: Starting download at ID 6296707 (max 100 MB)
2024-10-09T14:30:13.489208989Z 2024-10-09 14:30:13 DEBUG: Starting new HTTP connection (1): download.openstreetmap.fr:80
2024-10-09T14:30:13.528058904Z 2024-10-09 14:30:13 DEBUG: http://download.openstreetmap.fr:80 "GET /replication/europe/germany/minute//state.txt HTTP/11" 200 104
2024-10-09T14:30:13.550707545Z 2024-10-09 14:30:13 DEBUG: http://download.openstreetmap.fr:80 "GET /replication/europe/germany/minute//006/296/707.osc.gz HTTP/11" 200 13699
2024-10-09T14:30:13.553443002Z 2024-10-09 14:30:13 DEBUG: Downloaded change 6296707. (102345 kB available in download buffer)
2024-10-09T14:30:13.590914229Z /app/bin/update_from_dir --osc-dir=/db/diffs --version=2024-10-09T14:28:45Z --flush-size=16 --meta
2024-10-09T14:30:13.941086454Z Reading XML file ... finished reading nodes. Compute current ... ready. Flushing to database ....... done.
2024-10-09T14:30:14.249195580Z Reading XML file ... finished reading ways. Compute current ... ready. Flushing to database ....... done.
2024-10-09T14:30:14.593224827Z Reading XML file ... finished reading relations. runtime error: open64: 2 No such file or directory /db/db//osm3s_areas Unix_Socket::7
2024-10-09T14:30:14.596516689Z It took 0 to run the loop. Desired load is: 10%. Sleeping: 3
2024-10-09T14:30:14.755867608Z Flushing to database ....127.0.0.1 - - [09/Oct/2024:14:30:14 +0000] "GET /api/interpreter?data=[out:json];node(7167263);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
2024-10-09T14:30:14.889795140Z ... done.
2024-10-09T14:30:15.125821878Z Update complete.
2024-10-09T14:30:15.129869915Z There are still some updates remaining
2024-10-09T14:30:15.204274243Z 2024-10-09 14:30:15 INFO: Using replication server at http://download.openstreetmap.fr/replication/europe/germany/minute/
2024-10-09T14:30:15.204292693Z 2024-10-09 14:30:15 DEBUG: Using given sequence ID 6296707
2024-10-09T14:30:15.204296913Z 2024-10-09 14:30:15 DEBUG: Starting download at ID 6296708 (max 100 MB)
2024-10-09T14:30:15.205165625Z 2024-10-09 14:30:15 DEBUG: Starting new HTTP connection (1): download.openstreetmap.fr:80
2024-10-09T14:30:15.238148861Z 2024-10-09 14:30:15 DEBUG: http://download.openstreetmap.fr:80 "GET /replication/europe/germany/minute//state.txt HTTP/11" 200 104
2024-10-09T14:30:15.239562709Z 2024-10-09 14:30:15 ERROR: Error while downloading diffs.
2024-10-09T14:30:15.263663408Z Empty version, skipping file
2024-10-09T14:30:15.264297922Z <?xml version='1.0' encoding='UTF-8'?>
2024-10-09T14:30:15.264303622Z <osmChange version="0.6" generator="libosmium/2.20.0">
2024-10-09T14:30:15.264306302Z </osmChange>
2024-10-09T14:30:15.265191475Z Update finished with status code: 3

over and over again. Is there anything i can try to make it work? or is this somehow known?

from within the container i cannot run this:

app/bin/dispatcher --osm-base --db-dir=/db/db --meta
File_Error No such file or directory 2 /db/db//osm3s_osm_base Unix_Socket::4

thanks a lot!!

@BobLamarley
Copy link

BobLamarley commented Oct 25, 2024

Hi,
I'm having the same, running this command:

docker run \
  -e OVERPASS_META=yes \
  -e OVERPASS_MODE=init \
  -e OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/monaco-latest.osm.bz2 \
  -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/monaco/minute \
  -e OVERPASS_RULES_LOAD=10 \
  -v overpass-data:/db \
  -p 12345:80 \
  -i -t \
  --name overpass_monaco wiktorn/overpass-api
image

Did you find any solutions ?

@pcace
Copy link
Author

pcace commented Nov 11, 2024

Hi, I'm having the same, running this command:

docker run \
  -e OVERPASS_META=yes \
  -e OVERPASS_MODE=init \
  -e OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/monaco-latest.osm.bz2 \
  -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/monaco/minute \
  -e OVERPASS_RULES_LOAD=10 \
  -v overpass-data:/db \
  -p 12345:80 \
  -i -t \
  --name overpass_monaco wiktorn/overpass-api
image Did you find any solutions ?

no sadly not.
any other idea what to do?

@wiktorn
Copy link
Owner

wiktorn commented Nov 11, 2024

What filesystem do you use for storing the database? Does it support unix sockets? I wonder if double forward slashes do not make a difference here.

@pcace
Copy link
Author

pcace commented Nov 11, 2024

Hi, i am running it on Ubuntu 22.04, fs is ext4

 docker --version
Docker version 27.3.1, build ce12230
docker-compose --version
docker-compose version 1.29.2, build unknown

@wiktorn
Copy link
Owner

wiktorn commented Nov 11, 2024

Just for test, can you try running:

docker exec -it <your overpass container name> /bin/bash
apt update && apt install netcat-openbsd
nc -lU /db/socket-test

If netcat shows no errors, then indeed, your /db filesystem supports unix domain sockets and I can't understand why socket is missing, although all processes started correctly. If it fails, it means that whatever the filesystem is you mounted at /db, it's not supporting unix domain sockets and you need to fix this.

@hb0
Copy link

hb0 commented Jan 15, 2025

Getting the same issue since the new wikitorn/Overpass Docker Image releases about 4 months ago.
Unfortunately, I cannot find the old releases of the Docker image, so I could just rewind back.

Machine: Github Action CI w/ Ubuntu 24.04.1.

The CI workflow worked until then.
Last time it definitely worked with :lastest was August 12, 2024 if it helps.

I try to build with a custom OSM file (which worked until then).

@wiktorn
Copy link
Owner

wiktorn commented Jan 15, 2025

I just pushed a new set of versions. Do they fail too?

@hb0
Copy link

hb0 commented Jan 15, 2025

@wiktorn Thanks for the reply. The previous error is now gone, but the container does not seem to get healthy. My CI tests the Overpass build and start process with a very small OSM test file, which previously got to a healthy status after less then one minute. Now it's stuck at "starting", even after 15 minutes:

# In my CI test I check the health status with:
HEALTH_STATUS=$(docker inspect --format='{{.State.Health.Status}}' $OVERPASS_CONTAINER_NAME 2>&1)

# And get as result: HEALTH_STATUS = "starting"

Should the health status still be working as before?

I noticed, that the Readme states that "healthcheck will verify [...] if data in instance are earlier than two days ago, healthcheck will fail." > This could be the issue in my case, the CI test runs with dummy data from timestamp 1 (osm_base="1970-01-01T00:00:01Z") and my production Overpass need to use data from a specific year, e.g. region-20220101 found on Geofabrik. But I guess this is only a special health check which can be used to check "osm update" state, not after container start?

docker log shows:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
No database directory. Initializing
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 2042k  100 2042k    0     0   767M      0 --:--:-- --:--:-- --:--:--  767M
Database created. Now updating it.
Reading XML file ... finished reading nodes. Compute current ... ready. Flushing to database ...... done.
Reading XML file ... finished reading ways. Compute current ... ready. Flushing to database ...... done.
Flushing to database ....... done.
Update complete.
Updating
No OVERPASS_DIFF_URL set. Skipping update.
Generating areas...
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>   //////// this is okay, it's a custom OSM file
</osm>
Overpass container initialization complete. Exiting.
Starting supervisord process
2025-01-15 13:20:18,229 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2025-01-15 13:20:18,230 INFO supervisord started with pid 1
2025-01-15 13:20:19,232 INFO spawned: 'overpass_dispatch' with pid 8
2025-01-15 13:20:19,234 INFO spawned: 'nginx' with pid 9
2025-01-15 13:20:19,236 INFO spawned: 'fcgiwrap' with pid 10
2025-01-15 13:20:19,238 INFO spawned: 'update_overpass' with pid 11
2025-01-15 13:20:19,239 INFO spawned: 'dispatcher_areas' with pid 12
2025-01-15 13:20:19,243 INFO spawned: 'areas_rules' with pid 13
/app/bin /
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
2025-01-15 13:20:20,518 INFO success: overpass_dispatch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-15 13:20:20,518 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-15 13:20:20,518 INFO success: fcgiwrap entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-15 13:20:20,518 INFO success: update_overpass entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-15 13:20:20,518 INFO success: dispatcher_areas entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-15 13:20:20,518 INFO success: areas_rules entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 1 to run the loop. Desired load is: 1%. Sleeping: 99
127.0.0.1 - - [15/Jan/2025:13:20:48 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:21:18 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:21:48 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 1 to run the loop. Desired load is: 1%. Sleeping: 99
127.0.0.1 - - [15/Jan/2025:13:22:18 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:22:48 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:23:18 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:23:48 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 0 to run the loop. Desired load is: 1%. Sleeping: 3
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>
</osm>
It took 1 to run the loop. Desired load is: 1%. Sleeping: 99
127.0.0.1 - - [15/Jan/2025:13:24:18 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [15/Jan/2025:13:24:48 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"

```

@wiktorn
Copy link
Owner

wiktorn commented Jan 15, 2025

I noticed, that the Readme states that "healthcheck will verify [...] if data in instance are earlier than two days ago, healthcheck will fail." > This could be the issue in my case, the CI test runs with dummy data from timestamp 1 (osm_base="1970-01-01T00:00:01Z") and my production Overpass need to use data from a specific year, e.g. region-20220101 found on Geofabrik. But I guess this is only a special health check which can be used to check "osm update" state, not after container start?

This could be an issue, if you define the healthcheck as defined here. If you are using default healthcheck, it just checks whether the instance is responding, which generally should be OK.

Do you set up custom value for OVERPASS_HEALTHCHECK or use default?

@hb0
Copy link

hb0 commented Jan 16, 2025

I'm using the default one.

I first thought it at least works on my local machine, but I didn't use a new volume name, so an old database was used.
Building locally from scratch, I can reproduce the failing health status check there, too, see docker log further below.

When I execute manually curl --noproxy "*" -s "http://localhost:6002/api/interpreter?data=\[out:json\];node(1);out;" I get a permission error:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" lang="en"/>
  <title>OSM3S Response</title>
</head>
<body>

<p>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</p>
<p><strong style="color:#FF0000">Error</strong>: runtime error: open64: 13 Permission denied /db/db//osm3s_osm_base Unix_Socket::7 </p>

</body>
</html>

My docker-compose.yml looks like this:

services:
  overpassapi:
    container_name: overpassapi-overpassapi-1-test
    image: wiktorn/overpass-api:0.7.62
    environment:
      OVERPASS_META: "no"
      OVERPASS_MODE: init
      OVERPASS_PLANET_URL: file:///osm-data/test.osm.bz2
      OVERPASS_ALLOW_DUPLICATE_QUERIES: "yes"
    volumes:
      - type: volume
        source: overpass-db-test
        target: /db
      - type: bind
        source: ./osm-data
        target: /osm-data
    restart: always
    networks:
      - mapmatching_shared
    ports:
      - "127.0.0.1:6002:80"

volumes:
  overpass-db-test:

networks:
  mapmatching_shared:
    external: true
    name: mapmatching_shared

--- docker log ---

Current status: starting. Waiting...
CONTAINER ID   IMAGE                         COMMAND                  CREATED          STATUS                             PORTS                    NAMES
0281208a7691   wiktorn/overpass-api:0.7.62   "/docker-entrypoint.…"   36 seconds ago   Up 35 seconds (health: starting)   127.0.0.1:6002->80/tcp   overpassapi-overpassapi-1-test
No database directory. Initializing
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2042k  100 2042k    0     0  2112M      0 --:--:-- --:--:-- --:--:-- 2112M
Reading XML file ... finished reading nodes. Compute current ... ready. Flushing to database ...... done.
Reading XML file ... finished reading ways. Compute current ... ready. Flushing to database ...... done.
Flushing to database ....... done.
Update complete.
Database created. Now updating it.
Updating
No OVERPASS_DIFF_URL set. Skipping update.
Generating areas...
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>


</osm>
Overpass container initialization complete. Exiting.
Starting supervisord process
2025-01-16 05:54:18,730 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2025-01-16 05:54:18,730 INFO supervisord started with pid 1
2025-01-16 05:54:19,734 INFO spawned: 'overpass_dispatch' with pid 8
2025-01-16 05:54:19,736 INFO spawned: 'nginx' with pid 9
2025-01-16 05:54:19,738 INFO spawned: 'fcgiwrap' with pid 10
2025-01-16 05:54:19,739 INFO spawned: 'update_overpass' with pid 11
2025-01-16 05:54:19,740 INFO spawned: 'dispatcher_areas' with pid 14
2025-01-16 05:54:19,741 INFO spawned: 'areas_rules' with pid 17
/app/bin /
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.62 7c32a7d5">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="1970-01-01T00:00:01Z" areas="1970-01-01T00:00:01Z"/>


</osm>
It took 1 to run the loop. Desired load is: 1%. Sleeping: 99
2025-01-16 05:54:21,003 INFO success: overpass_dispatch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-16 05:54:21,004 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-16 05:54:21,004 INFO success: fcgiwrap entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-16 05:54:21,004 INFO success: update_overpass entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-16 05:54:21,004 INFO success: dispatcher_areas entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-01-16 05:54:21,004 INFO success: areas_rules entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
127.0.0.1 - - [16/Jan/2025:05:54:23 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [16/Jan/2025:05:54:28 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [16/Jan/2025:05:54:33 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [16/Jan/2025:05:54:38 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [16/Jan/2025:05:54:43 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"
127.0.0.1 - - [16/Jan/2025:05:54:49 +0000] "GET /api/interpreter?data=[out:json];node(1);out; HTTP/1.1" 200 634 "-" "curl/7.88.1"

@wiktorn
Copy link
Owner

wiktorn commented Jan 16, 2025

Can you show output of:

docker exec -it <your_overpass_container> ls -l /db/db/osm3s_osm_base Unix_Socket

@hb0
Copy link

hb0 commented Jan 16, 2025

First it shows

ls: cannot access '/db/db/osm3s_osm_base': No such file or directory
ls: cannot access 'Unix_Socket': No such file or directory 

and on the next check after 5 seconds (and later):

ls: cannot access 'Unix_Socket': No such file or directory
srw-rw-rw- 1 overpass overpass 0 Jan 16 10:42 /db/db/osm3s_osm_base

The srw seems to point out, that osm3s_osm_base is a UnixSocket.

And if needed for /db and /db/db:

total 32K
4467691 drwx------ 4 overpass overpass 4.0K Jan 16 10:42 .
4467711 drwxr-xr-x 1 root     root     4.0K Jan 16 10:42 ..
4467726 -rw-r--r-- 1 overpass overpass  220 Aug 28 02:18 .bash_logout
4467725 -rw-r--r-- 1 overpass overpass 3.5K Aug 28 02:18 .bashrc
4467727 -rw-r--r-- 1 overpass overpass  807 Aug 28 02:18 .profile
4467723 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 changes.log
4467741 -rw-r--r-- 1 overpass overpass   29 Jan 16 10:42 cookie.jar
4467743 drwxr-xr-x 3 overpass overpass 4.0K Jan 16 10:42 db
4467732 drwxr-xr-x 2 overpass overpass 4.0K Jan 16 10:42 diffs
4467787 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 init_done
total 11M
4467743 drwxr-xr-x 3 overpass overpass 4.0K Jan 16 10:42 .
4467691 drwx------ 4 overpass overpass 4.0K Jan 16 10:42 ..
4467818 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 area_blocks.bin.idx
4467820 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 area_tags_global.bin.idx
4467819 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 area_tags_local.bin.idx
4467822 -rw-r--r-- 1 overpass overpass   21 Jan 16 10:42 area_version
4467817 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 areas.bin.idx
4467793 -rw-rw-rw- 1 overpass overpass 3.7K Jan 16 10:42 database.log
4467750 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 node_frequent_tags.bin
4467756 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 node_frequent_tags.bin.idx
4467749 -rw-r--r-- 1 overpass overpass  64K Jan 16 10:42 node_keys.bin
4467755 -rw-r--r-- 1 overpass overpass   24 Jan 16 10:42 node_keys.bin.idx
4467751 -rw-r--r-- 1 overpass overpass 752K Jan 16 10:42 node_tags_global.bin
4467754 -rw-r--r-- 1 overpass overpass  753 Jan 16 10:42 node_tags_global.bin.idx
4467748 -rw-r--r-- 1 overpass overpass 576K Jan 16 10:42 node_tags_local.bin
4467753 -rw-r--r-- 1 overpass overpass  549 Jan 16 10:42 node_tags_local.bin.idx
4467747 -rw-r--r-- 1 overpass overpass 560K Jan 16 10:42 nodes.bin
4467752 -rw-r--r-- 1 overpass overpass  184 Jan 16 10:42 nodes.bin.idx
4467746 -rw-r--r-- 1 overpass overpass 7.5M Jan 16 10:42 nodes.map
4467745 -rw-r--r-- 1 overpass overpass 108K Jan 16 10:42 nodes.map.idx
4467806 srw-rw-rw- 1 overpass overpass    0 Jan 16 10:42 osm3s_areas
4467795 srw-rw-rw- 1 overpass overpass    0 Jan 16 10:42 osm3s_osm_base
4467744 -rw-r--r-- 1 overpass overpass   21 Jan 16 10:42 osm_base_version
4467774 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_frequent_tags.bin
4467782 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_frequent_tags.bin.idx
4467773 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_keys.bin
4467781 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_keys.bin.idx
4467776 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_roles.bin
4467778 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_roles.bin.idx
4467775 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_tags_global.bin
4467780 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_tags_global.bin.idx
4467772 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_tags_local.bin
4467779 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relation_tags_local.bin.idx
4467771 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relations.bin
4467777 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relations.bin.idx
4467770 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 relations.map
4467769 -rw-r--r-- 1 overpass overpass    8 Jan 16 10:42 relations.map.idx
4467783 drwxr-xr-x 2 overpass overpass 4.0K Jan 16 10:42 rules
4467809 -rw-r--r-- 1 overpass overpass  219 Jan 16 10:42 rules_loop.log
4467794 -rw-rw-rw- 1 overpass overpass 3.0K Jan 16 10:42 transactions.log
4467762 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 way_frequent_tags.bin
4467768 -rw-r--r-- 1 overpass overpass    0 Jan 16 10:42 way_frequent_tags.bin.idx
4467761 -rw-r--r-- 1 overpass overpass  64K Jan 16 10:42 way_keys.bin
4467767 -rw-r--r-- 1 overpass overpass   24 Jan 16 10:42 way_keys.bin.idx
4467763 -rw-r--r-- 1 overpass overpass 416K Jan 16 10:42 way_tags_global.bin
4467766 -rw-r--r-- 1 overpass overpass  390 Jan 16 10:42 way_tags_global.bin.idx
4467760 -rw-r--r-- 1 overpass overpass 320K Jan 16 10:42 way_tags_local.bin
4467765 -rw-r--r-- 1 overpass overpass  284 Jan 16 10:42 way_tags_local.bin.idx
4467759 -rw-r--r-- 1 overpass overpass 352K Jan 16 10:42 ways.bin
4467764 -rw-r--r-- 1 overpass overpass  168 Jan 16 10:42 ways.bin.idx
4467758 -rw-r--r-- 1 overpass overpass  32K Jan 16 10:42 ways.map
4467757 -rw-r--r-- 1 overpass overpass   16 Jan 16 10:42 ways.map.idx

@wiktorn
Copy link
Owner

wiktorn commented Jan 16, 2025

and on the next check after 5 seconds (and later):

ls: cannot access 'Unix_Socket': No such file or directory
srw-rw-rw- 1 overpass overpass 0 Jan 16 10:42 /db/db/osm3s_osm_base

That's suspicious. Based on the other outputs that you shared, it looks like /db has 700 permissions, which prevents other processes to access files there. Can you change that to at least 711 and check if this fixes the issue?

@hb0
Copy link

hb0 commented Jan 16, 2025

This resolves the issue, the test now complete again in the usual time.

Strange, that this is not the case elsewhere, e.g. in your containers?

In the last 4 months only the base image of Dockerfile.template changed, and in my map-matching project which tests the overpass setup, I con't see any changes between last successful and first failing test.

Thanks a lot for the support and maintaining the Container setup !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants