Error in cluster_network #974

ariannaleoni · 2024-02-20T10:17:05Z

Checklist

[ x ] I am using the current main branch or the latest release.
[ x ] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

I encountered a problem when I tried to run the tutorial model of PyPSA-Earth.
I tried to debug and it seems that the problem arise on the assert in the cluster_network script: n_clusters >= len(N) and n_clusters <= N.sum() at line 315.
n_cluster is equal to 2, while len(N) is equal to 10, so the error occurs.
n_cluster is equal to 2 because the number of countries in the model tutorial is two and no bus has a subnetwork assigned, all subnetwork values are empty strings. It seems that the subnetworks are created using " n.determine_network_topology()" at line 480 in cluster_network script, after the initialization of n_cluster with value 2 at line 702 (when cluster == "min"), and no update of the n_cluster is performed.
The error that occurred is as follows:

Error Message

Error in rule cluster_network:
    jobid: 4
    input: networks/elec_s.nc, resources/shapes/country_shapes.geojson, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/shapes/gadm_shapes.geojson, resources/costs.csv
    output: networks/elec_s_6.nc, resources/bus_regions/regions_onshore_elec_s_6.geojson, resources/bus_regions/regions_offshore_elec_s_6.geojson, resources/bus_regions/busmap_elec_s_6.csv, resources/bus_regions/linemap_elec_s_6.csv
    log: logs/cluster_network/elec_s_6.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule cluster_network:
    jobid: 4
    input: networks/elec_s.nc, resources/shapes/country_shapes.geojson, resources/bus_regions/regions_onshore_elec_s.geojson, resources/bus_regions/regions_offshore_elec_s.geojson, resources/shapes/gadm_shapes.geojson, resources/costs.csv
    output: networks/elec_s_6.nc, resources/bus_regions/regions_onshore_elec_s_6.geojson, resources/bus_regions/regions_offshore_elec_s_6.geojson, resources/bus_regions/busmap_elec_s_6.csv, resources/bus_regions/linemap_elec_s_6.csv
    log: logs/cluster_network/elec_s_6.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/ariannaleoni/miniconda3/envs/pypsa-cheat/bin/python3.10 /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp3u5evlqh.cluster_network.py' returned non-zero exit status 1.
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile, line 734:
Command 'set -euo pipefail;  /Users/ariannaleoni/miniconda3/envs/pypsa-cheat/bin/python3.10 /Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp3u5evlqh.cluster_network.py' returned non-zero exit status 1.
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/Snakefile", line 734, in __rule_cluster_network
  File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/concurrent/futures/thread.py", line 58, in run

elec_s_6.log:

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units
WARNING:__main__:Keyword argument feature is only valid for algorithm `hac`. Given feature `solar+onwind-time` will be ignored.
ERROR:_helpers:An error happened in module '/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/scripts/cluster_network.py', function 'distribute_clusters': Number of clusters must be 10 <= n_clusters <= 84 for this selection of countries.
Traceback (most recent call last):
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 758, in <module>
    clustering = clustering_for_n_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 597, in clustering_for_n_clusters
    busmap = busmap_for_n_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 488, in busmap_for_n_clusters
    n_clusters = distribute_clusters(
  File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp5sc831fq.cluster_network.py", line 319, in distribute_clusters
    n_clusters >= len(N) and n_clusters <= N.sum()
AssertionError: Number of clusters must be 10 <= n_clusters <= 84 for this selection of countries.

The text was updated successfully, but these errors were encountered:

ekatef · 2024-02-20T17:48:11Z

Hello @ariannaleoni! Thanks for reporting. You are right that the number of clusters is read from the configuration file and not updated after. That is the expected behaviour, while config.yaml is used to define the parameters which are feasible for simulation.

Could you please share some details to reproduce the error you encounter? Could you please share your config.yaml?

ariannaleoni · 2024-02-21T16:23:26Z

Hi! Indeed, changing the parameter "cluster" in config.yaml solves the problem. However, there is still an error. Here is the elec_s_10.log file. The config.yaml is identical to the one reported on GitHub, with the only difference being the value of the parameter "cluster," which is now set to 10.
In any case, I used the tutorial version, so I assumed I didn't have to change any variables to run a simulation

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units
WARNING:main:Keyword argument feature is only valid for algorithm hac. Given feature solar+onwind-time will be ignored.
WARNING:main:The configured solver glpk does not support quadratic objectives. Falling back to ipopt.
ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py', function 'consense': In Bus cluster bus_id, the values of attribute bus_id do not agree:
Bus
1 1
3 3
4 4
6 6
8 8
...
185 185
198 198
200 200
207 207
209 209
Name: bus_id, Length: 75, dtype: object
Traceback (most recent call last):
File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 758, in
clustering = clustering_for_n_clusters(
File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 612, in clustering_for_n_clusters
clustering = get_clustering_from_busmap(
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 534, in get_clustering_from_busmap
buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 341, in aggregatebuses
aggregated = n.buses.groupby(busmap).agg(strategies)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1432, in aggregate
result = op.agg()
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 190, in agg
return self.agg_dict_like()
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 423, in agg_dict_like
return self.agg_or_apply_dict_like(op_name="agg")
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like
result_index, result_data = self.compute_dict_like(
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 496, in compute_dict_like
results = [
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 497, in
getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate
return self._python_agg_general(func, *args, **kwargs)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general
result = self._grouper.agg_series(obj, f)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 863, in agg_series
result = self._aggregate_series_pure_python(obj, func)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 884, in _aggregate_series_pure_python
res = func(group)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 324, in
f = lambda x: func(x, *args, **kwargs)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 136, in consense
assert (x == v).all() or x.isnull().all(), (
AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree:
Bus
1 1
3 3
4 4
6 6
8 8
...
185 185
198 198
200 200
207 207
209 209
Name: bus_id, Length: 75, dtype: object

ekatef · 2024-02-22T09:49:24Z

Hi! Indeed, changing the parameter "cluster" in config.yaml solves the problem. However, there is still an error. Here is the elec_s_10.log file. The config.yaml is identical to the one reported on GitHub, with the only difference being the value of the parameter "cluster," which is now set to 10. In any case, I used the tutorial version, so I assumed I didn't have to change any variables to run a simulation

INFO:pypsa.io:Imported network elec_s.nc has buses, carriers, generators, lines, loads, storage_units WARNING:main:Keyword argument feature is only valid for algorithm hac. Given feature solar+onwind-time will be ignored. WARNING:main:The configured solver glpk does not support quadratic objectives. Falling back to ipopt. ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py', function 'consense': In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object Traceback (most recent call last): File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 758, in clustering = clustering_for_n_clusters( File "/Users/ariannaleoni/Desktop/pypsa/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmpqj52qg59.cluster_network.py", line 612, in clustering_for_n_clusters clustering = get_clustering_from_busmap( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 534, in get_clustering_from_busmap buses = aggregatebuses(n, busmap, custom_strategies=bus_strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 341, in aggregatebuses aggregated = n.buses.groupby(busmap).agg(strategies) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 1432, in aggregate result = op.agg() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 190, in agg return self.agg_dict_like() File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 423, in agg_dict_like return self.agg_or_apply_dict_like(op_name="agg") File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 1608, in agg_or_apply_dict_like result_index, result_data = self.compute_dict_like( File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 496, in compute_dict_like results = [ File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/apply.py", line 497, in getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 294, in aggregate return self._python_agg_general(func, *args, **kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 327, in _python_agg_general result = self._grouper.agg_series(obj, f) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 863, in agg_series result = self._aggregate_series_pure_python(obj, func) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 884, in _aggregate_series_pure_python res = func(group) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pandas/core/groupby/generic.py", line 324, in f = lambda x: func(x, *args, **kwargs) File "/Users/ariannaleoni/miniconda3/envs/pypsa-cheat/lib/python3.10/site-packages/pypsa/clustering/spatial.py", line 136, in consense assert (x == v).all() or x.isnull().all(), ( AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree: Bus 1 1 3 3 4 4 6 6 8 8 ... 185 185 198 198 200 200 207 207 209 209 Name: bus_id, Length: 75, dtype: object

Hello! That is a weird issue with the mismatch of bus_id. It has been seen before and is linked with environment troubles: it seems to arise if any other dependencies have been installed into pypsa-earth environment. I'd recommend to build a fresh environment, clean-up old outputs of the worflow and run the tutorial again.

ariannaleoni · 2024-02-22T10:50:28Z

I've already tried to build a fresh environment, but it didn't work. However, I'm using version 3.14.14 of Ipopt because there's no precompiled version for osx-arm64. I've tried to compile the software from source, but I couldn't manage to make it work. However, the differences between my installed version, 3.14.14, and the required one, < 3.13.3, should be backward compatible, that is why I installed the Ipopt 3.14.14. Everything else is a fresh install newly created.

ekatef · 2024-02-22T17:23:03Z

I've already tried to build a fresh environment, but it didn't work. However, I'm using version 3.14.14 of Ipopt because there's no precompiled version for osx-arm64. I've tried to compile the software from source, but I couldn't manage to make it work. However, the differences between my installed version, 3.14.14, and the required one, < 3.13.3, should be backward compatible, that is why I installed the Ipopt 3.14.14. Everything else is a fresh install newly created.

Ok, thanks for letting us know! It seems your are facing same issue as was reported in #968. The workaround is to remove a constraint to ipop version when creating the environment.

Have you tried to re-run the workflow with a new environment?

ariannaleoni · 2024-02-23T09:28:14Z

I have already removed the ipopt constraint and re-run the workflow in a new environment, but it didn't work

ekatef · 2024-02-23T11:11:17Z

I have already removed the ipopt constraint and re-run the workflow in a new environment, but it didn't work

Thank you for trying. Let's try to debug it together. A few troubleshooting questions:

Have you also removed outputs of the previous run before re-running the workflow in a fresh environment?
Is the error the same: AssertionError: In Bus cluster bus_id, the values of attribute bus_id do not agree?
Could you please share a list of the packages installed in your environment?

ekatef · 2024-02-23T11:19:49Z

As an additional comment, the issue is unfortunately known but it is tricky to reproduce it. It would be a great achievement, if we would be able to find a way to do so.

ariannaleoni · 2024-02-23T11:36:38Z

Hello! I've removed the outputs of the previous run before re-running, using the command line snakemake -j 1 clean, and the error is always the same. Anyway, regarding the different sorting of the bus_id, if they are compared as lists (thus with a specific ordering), they are different, but if they are compared as sets (thus the order is no longer relevant), then they are equal. The installed packages in my environment are in this file (they are too many to send as a message)
packages environment .docx

ekatef · 2024-02-25T18:28:33Z

Hello @ariannaleoni!

Thank you for sharing the environment specification. Could it be that something went wrong during creation of the environment? For example, pypsa 0.25.1 was used, while currently there is a restriction <0.25:

pypsa-earth/envs/environment.yaml

Line 15 in 413e3fe

- pypsa>=0.24, <0.25

The CI doesn't show any environment problems. I have also generated a fresh environment locally removing a limitation to ipop, and seems to work properly.

If tutorial doesn't work for you, it could make sense to remove the model completely and go through the installation and tutorial following the docs.

ariannaleoni · 2024-02-29T12:01:01Z

Hi! Apologies for the delayed response, but I've managed to address the issue. Now, I'm attempting to run a new simulation, but I'm encountering some difficulties. I've set the input countries to "africa", but I'm encountering this error:
Error in rule build_renewable_profiles:
jobid: 17
input: resources/natura.tiff, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_EPSG-4326.tif, data/gebco/GEBCO_2021_TID.nc, resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, data/hydro_capacities.csv, data/eia_hydro_annual_generation.csv, resources/powerplants.csv, resources/bus_regions/regions_onshore.geojson, cutouts/africa-2013-era5.nc
output: resources/renewable_profiles/profile_solar.nc
log: logs/build_renewable_profile_solar.log (check log file(s) for error details)

After reviewing the log file, I noticed the following message: "WARNING:main:Weather data does not fully cover the requested region." Additionally, I manually downloaded the cutout files "africa-2013-era5" and "cutout-2013-era5", but encountered the same issue. Do you have any suggestions on how to proceed?

ekatef · 2024-03-04T20:50:02Z

Hi! Apologies for the delayed response, but I've managed to address the issue. Now, I'm attempting to run a new simulation, but I'm encountering some difficulties. I've set the input countries to "africa", but I'm encountering this error: Error in rule build_renewable_profiles: jobid: 17 input: resources/natura.tiff, data/copernicus/PROBAV_LC100_global_v3.0.1_2019-nrt_Discrete-Classification-map_EPSG-4326.tif, data/gebco/GEBCO_2021_TID.nc, resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, data/hydro_capacities.csv, data/eia_hydro_annual_generation.csv, resources/powerplants.csv, resources/bus_regions/regions_onshore.geojson, cutouts/africa-2013-era5.nc output: resources/renewable_profiles/profile_solar.nc log: logs/build_renewable_profile_solar.log (check log file(s) for error details)

After reviewing the log file, I noticed the following message: "WARNING:main:Weather data does not fully cover the requested region." Additionally, I manually downloaded the cutout files "africa-2013-era5" and "cutout-2013-era5", but encountered the same issue. Do you have any suggestions on how to proceed?

Hello @ariannaleoni!

Great, glad to know that the issue has been resolved.

As we discussed, the newly appeared problem has been caused by the fact that the tutorial datasets are created for a particular region only (Nigeria and Benin) and a short timeframe. Since you want to build a full-scale model, the way to go is using parameters of config.default.yaml, adjusting countries parameter.

Regarding the mismatch you observe, I do agree that it should not be the case. You can look into the details of the cutout you downloaded using jupyther notebooks provided into documentation repo. In partucular, build_cutout, build_renewable_profiles and landuse-availability.

ariannaleoni · 2024-03-05T10:13:57Z

Hello Katia,
Thank you for the advice. One possible issue could be that I modified the _simplify_polys function in build_shapes as follows:
In the place of return polys.simplify(tolerance=tolerance) (line 257) I inserted this:
counter = 0
while counter < 100:
try:
return polys.simplify(tolerance=tolerance)
except Exception as e:
counter += 1
logger.info(counter)
pass

I had to do it because frequently the old function failed to create the polygons, leading to the code breaking. By doing this, I ensure that the function retries until it succeeds.
Could it be a determining factor that causes the problem?

Thank you for the help

ekatef · 2024-03-05T21:15:38Z

Hello Katia, Thank you for the advice. One possible issue could be that I modified the _simplify_polys function in build_shapes as follows: In the place of return polys.simplify(tolerance=tolerance) (line 257) I inserted this: counter = 0 while counter < 100: try: return polys.simplify(tolerance=tolerance) except Exception as e: counter += 1 logger.info(counter) pass

I had to do it because frequently the old function failed to create the polygons, leading to the code breaking. By doing this, I ensure that the function retries until it succeeds. Could it be a determining factor that causes the problem?

Thank you for the help

Hello @ariannaleoni!

Unfortunately, I'm not quite able to trace the whole chain of possible effects. They may actually be also a result also of some combination of factors. I think, the most effective way to deal with that is to take a step back, return to the original state of the codebase and try to reproduce the error which you experienced in _simplify_polys(). If the issue will be reproducible, a bug report on that would be highly appreciated.

As a comment, if you want to contribute by improving the existing code, that is absolutely welcome! Feel free to look into a list of existing issue and join the developers meetings.

ariannaleoni · 2024-03-06T07:16:35Z

Hello @ekatef !
If I don't modify _simplify_polis() the error is:

Error in rule build_shapes:
jobid: 10
input: data/eez/eez_v11.gpkg
output: resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, resources/shapes/africa_shape.geojson, resources/shapes/gadm_shapes.geojson
log: logs/build_shapes.log (check log file(s) for error details)

ERROR:snakemake.logging:Error in rule build_shapes:
jobid: 10
input: data/eez/eez_v11.gpkg
output: resources/shapes/country_shapes.geojson, resources/shapes/offshore_shapes.geojson, resources/shapes/africa_shape.geojson, resources/shapes/gadm_shapes.geojson
log: logs/build_shapes.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile, line 259:
Command 'set -euo pipefail; /Users/ariannaleoni/miniconda3/envs/pypsa-earth/bin/python3.10 /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py' returned non-zero exit status 1.
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile", line 259, in __rule_build_shapes
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/concurrent/futures/thread.py", line 58, in run
ERROR:snakemake.logging:RuleException:
CalledProcessError in file /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile, line 259:
Command 'set -euo pipefail; /Users/ariannaleoni/miniconda3/envs/pypsa-earth/bin/python3.10 /Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py' returned non-zero exit status 1.
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/Snakefile", line 259, in __rule_build_shapes
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/concurrent/futures/thread.py", line 58, in run

And in the log file:

ERROR:_helpers:An error happened in module '/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/constructive.py', function 'simplify': IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers
Traceback (most recent call last):
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 1342, in
offshore_shapes = eez(
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 391, in eez
ret_df = ret_df.geometry.map(
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/series.py", line 4691, in map
new_values = self._map_values(arg, na_action=na_action)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/base.py", line 919, in _map_values
return arr.map(mapper, na_action=na_action)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/arrays/base.py", line 2319, in map
return map_array(self, mapper, na_action=na_action)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/pandas/core/algorithms.py", line 1743, in map_array
return lib.map_infer(values, mapper, convert=convert)
File "lib.pyx", line 2972, in pandas._libs.lib.map_infer
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 392, in
lambda x: _simplify_polys(x, minarea=minarea, tolerance=tolerance)
File "/Users/ariannaleoni/pypsa-earth-project/pypsa-earth/.snakemake/scripts/tmp_9x785vx.build_shapes.py", line 261, in _simplify_polys
return polys.simplify(tolerance=tolerance)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/geometry/base.py", line 564, in simplify
return shapely.simplify(self, tolerance, preserve_topology=preserve_topology)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/decorators.py", line 77, in wrapped
return func(*args, **kwargs)
File "/Users/ariannaleoni/miniconda3/envs/pypsa-earth/lib/python3.10/site-packages/shapely/constructive.py", line 862, in simplify
return lib.simplify_preserve_topology(geometry, tolerance, **kwargs)
shapely.errors.GEOSException: IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers

Thank you,

Arianna

ekatef · 2024-03-06T13:02:20Z

shapely.errors.GEOSException: IllegalArgumentException: CGAlgorithmsDD::orientationIndex encountered NaN/Inf numbers

Hey @ariannaleoni!

Thanks a lot for sharing the details. It looks like for some reason the workflow is not happy with geometry it obtains. Not sure what exactly is the reason, but if the issue is reproducible, it would be amazing to have a fix for it.

Could you please check that the issue can be reproduced and open a bug report for that?

ariannaleoni · 2024-03-06T13:43:58Z

Hi @ekatef
Sure, I'll do that right now

ariannaleoni added the bug Something isn't working label Feb 20, 2024

ekatef mentioned this issue Mar 6, 2024

Align network data structure with PyPSA>=0.25 #985

Closed

ekatef mentioned this issue Mar 15, 2024

pypsa>=0.25 requires upgrade of network clustering #787

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error in cluster_network #974

Error in cluster_network #974

ariannaleoni commented Feb 20, 2024

ekatef commented Feb 20, 2024

ariannaleoni commented Feb 21, 2024

ekatef commented Feb 22, 2024

ariannaleoni commented Feb 22, 2024

ekatef commented Feb 22, 2024

ariannaleoni commented Feb 23, 2024

ekatef commented Feb 23, 2024

ekatef commented Feb 23, 2024

ariannaleoni commented Feb 23, 2024

ekatef commented Feb 25, 2024

ariannaleoni commented Feb 29, 2024

ekatef commented Mar 4, 2024

ariannaleoni commented Mar 5, 2024

ekatef commented Mar 5, 2024

ariannaleoni commented Mar 6, 2024

ekatef commented Mar 6, 2024

ariannaleoni commented Mar 6, 2024

Error in cluster_network #974

Error in cluster_network #974

Comments

ariannaleoni commented Feb 20, 2024

Checklist

Describe the Bug

Error Message

ekatef commented Feb 20, 2024

ariannaleoni commented Feb 21, 2024

ekatef commented Feb 22, 2024

ariannaleoni commented Feb 22, 2024

ekatef commented Feb 22, 2024

ariannaleoni commented Feb 23, 2024

ekatef commented Feb 23, 2024

ekatef commented Feb 23, 2024

ariannaleoni commented Feb 23, 2024

ekatef commented Feb 25, 2024

ariannaleoni commented Feb 29, 2024

ekatef commented Mar 4, 2024

ariannaleoni commented Mar 5, 2024

ekatef commented Mar 5, 2024

ariannaleoni commented Mar 6, 2024

ekatef commented Mar 6, 2024

ariannaleoni commented Mar 6, 2024