Merge pull request #294 from trevorb1/issue-220

Docs updates

docs/source/config-co2-base.md

@@ -0,0 +1,11 @@
+(base_emission_values)=
+# Base Emission Values
+
+If using the `{opts}` wildcard to reduce emissions, the user must put in a `co2base` value. Provided below are historical yearly CO2 emission values for both the power sector and all sectors at an interconnect level. This data can be used as a starting point for users. **Note the units in this table are Million Metric Tons (MMT).** This data originates from the [EIA State Level CO2 database](https://www.eia.gov/opendata/browser/co2-emissions/co2-emissions-aggregates?frequency=annual&data=value;&sortColumn=period;&sortDirection=desc;), and is compiled by the script `workflow/notebooks/historical_emissions.ipynb`
+
+```{eval-rst}
+.. csv-table::
+   :header-rows: 1
+   :widths: 22,7,22,33
+   :file: configtables/emissions.csv
+```

docs/source/config-configuration.md

@@ -45,7 +45,7 @@ Planning horizons determines which year of future demand forecast to use for you
 (snapshots_cf)=
 ## `snapshots`
 
-Specifies the temporal range to build an energy system model for as arguments to `(pandas.date_range)[https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html]`
+Specifies the temporal range to build an energy system model for as arguments to [`pandas.date_range`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html)
 
 ```{eval-rst}
 .. literalinclude:: ../../workflow/repo_data/config/config.default.yaml
@@ -93,14 +93,10 @@ Specifies the types of generators that are included in the network, which are ex
    :file: configtables/electricity.csv
 ```
 
-If using the `{opts}` wildcard to reduce emissions, the user must put in a `co2base` value. Provided below are historical yearly CO2 emission values for both the power sector and all sectors at an interconnect level. This data can be used as a starting point for users. **Note the units in this table are Million Metric Tons (MMT).** This data originates from the [EIA State Level CO2 database](https://www.eia.gov/opendata/browser/co2-emissions/co2-emissions-aggregates?frequency=annual&data=value;&sortColumn=period;&sortDirection=desc;), and is compiled by the script `workflow/notebooks/historical_emissions.ipynb`
-
-```{eval-rst}
-.. csv-table::
-   :header-rows: 1
-   :widths: 22,7,22,33
-   :file: configtables/emissions.csv
+```{note}
+See [here](./config-co2-base.md) for information on interconnect level base emission values.
 ```
+
 (renewable_cf)=
 ## `renewable`
 
@@ -199,12 +195,51 @@ If using the `{opts}` wildcard to reduce emissions, the user must put in a `co2b
 
 (sector_cf)=
 ## `sector`
-```{eval-rst}
+<!-- ```{eval-rst}
 .. literalinclude:: ../../workflow/repo_data/config/config.default.yaml
    :language: yaml
    :start-at: sector:
    :end-before: # docs
 
+.. csv-table::
+   :header-rows: 1
+   :widths: 22,7,22,33
+   :file: configtables/sector.csv
+``` -->
+
+```{warning}
+Sector coupling studies are all under active development
+```
+
+```yaml
+sector:
+  co2_sequestration_potential: 0
+  natural_gas:
+    allow_imports_exports: true # false to be implemented
+    cyclic_storage: false
+  heating:
+    heat_pump_sink_T: 55.
+  demand:
+    profile:
+      residential: eulp # efs, eulp
+      commercial: eulp # efs, eulp
+      transport: efs # efs
+      industry: efs # efs
+    scale:
+      residential: aeo # efs, aeo
+      commercial: aeo # efs, aeo
+      transport: aeo # efs, aeo
+      industry: aeo # efs, aeo
+    disaggregation:
+      residential: pop # pop
+      commercial: pop # pop
+      transport: pop # pop
+      industry: pop # pop
+    scenarios:
+      aeo: reference
+```
+
+```{eval-rst}
 .. csv-table::
    :header-rows: 1
    :widths: 22,7,22,33

docs/source/config-sectors.md

@@ -1,6 +1,10 @@
 (sectors)=
 # Sectors
 
+```{note}
+More information to come!
+```
+
 (natural-gas-sector)=
 ## Natural Gas
 

docs/source/config-wildcards.md

@@ -90,26 +90,31 @@ currently:
 (sector)=
 ## The `{sector}` wildcard
 
+```{warning}
+Sector coupling studies are all under active development
+```
+
 The `{sector}` wildcard is used to specify what sectors to include. If `None`
 is provided, an electrical only study is completed.
 
-| Sector      | Code | Description                                                  |
-|-------------|------|--------------------------------------------------------------|
-| Electricity | E    | Electrical sector. Will always be run.                       |
-| Natural Gas | G    | Natural gas sector                                           |
-| Heating     | H    | Residential and commercial heating and cooling demand        |
-<!-- | Transport   | T    | Residential and light duty commercial transportation demand  |
-| Methane     | M    | Methane tracking. Requires natural gas sector.               | -->
+| Sector      | Code | Description                                    | Status      |
+|-------------|------|------------------------------------------------|-------------|
+| Electricity | E    | Electrical sector. Will always be run.         | Runs        |
+| Natural Gas | G    | Natural gas sector                             | Development |
+| Heating     | H    | Heating and Cooling                            | Development |
+| Industry    | I    | Industrial Processes                           | Development |
+| Transport   | T    | Transportation sector                          | Development |
+| Methane     | M    | Methane tracking. Requires natural gas sector. | Future      |
 
 (scope)=
 ## The `{scope}` wildcard
-Takes values `residential`, `urban`, `total`. Used in sector coupling
-studies to define population breakdown.
 
-Used in the following rules:
-- `build_heat_demands`
-- `build_temperature_profiles`
-- `build_solar_thermal_profiles`
+```{warning}
+Sector coupling studies are all under active development
+```
+
+Takes values `residential`, `urban`, `total`. Used in sector coupling studies to define
+population breakdown.
 
 
 (cutout_wc)=

docs/source/configtables/costs.csv

@@ -1,9 +1,10 @@
 ,Unit,Values,Description
-year,--,"YYYY; e.g. '2030'","Year for which to retrieve cost assumptions of `resources/costs.csv`."
-version,--,"vX.X.X; e.g. 'v0.5.0'","Version of `technology-data` repository to use."
+year,--,YYYY; e.g. ``2030``,"Year for which to retrieve cost assumptions of `resources/costs.csv`."
+version,--,vX.X.X; e.g. ``v0.5.0``,"Version of `technology-data` repository to use."
 rooftop_share,--,float,"Share of rooftop PV when calculating capital cost of solar (joint rooftop and utility-scale PV)."
 fill_values,--,float,"Default values if not specified for a technology in `resources/costs.csv`."
 capital_cost,$/MW,"Keys should be in the 'technology' column of `resources/costs.csv`. Values can be any float.","For the given technologies, assumptions about their capital investment costs are set to the corresponding value. Optional; overwrites cost assumptions from ``resources/costs.csv``."
 marginal_cost,$/MWh,"Keys should be in the 'technology' column of `resources/costs.csv`. Values can be any float.","For the given technologies, assumptions about their marginal operating costs are set to the corresponding value. Optional; overwrites cost assumptions from ``resources/costs.csv``."
 emission_prices,,,"Specify exogenous prices for emission types listed in `network.carriers` to marginal costs."
--- co2,$/t,float,"Exogenous price of carbon-dioxide added to the marginal costs of fossil-fuelled generators according to their carbon intensity. Added through the keyword `Ep` in the `{opts}` wildcard only in the rule :mod:`prepare_network``."
+-- enable,bool,``true`` or ``false``, Add cost for a carbon-dioxide price configured in costs: emission_prices: co2 to marginal_cost of generators (other emission types listed in network.carriers possible as well)
+-- co2,$/t,float,"Exogenous price of carbon-dioxide added to the marginal costs of fossil-fuelled generators according to their carbon intensity. Added through the keyword `Ep` in the `{opts}` wildcard only in the rule :mod:``prepare_network``."

docs/source/configtables/electricity.csv

@@ -1,18 +1,36 @@
 ,Unit,Values,Description
 conventional_carriers,--,"Any subset of {nuclear, oil, OCGT, CCGT, coal, geothermal, biomass}","List of conventional power plants to include in the model from ``resources/powerplants.csv``. If an included carrier is also listed in ``extendable_carriers``, the capacity is taken as a lower bound."
 renewable_carriers,--,"Any subset of {solar, onwind, offwind-ac, offwind-dc, hydro}",List of renewable generators to include in the model.
-voltage_simplified,kV,int,"Voltage level to simplify network to in rule ""simplify network"""
+voltage_simplified,kV,int,Voltage level to simplify network to in rule ``simplify_network``
 gaslimit,MWhth,float or false,Global gas usage limit (Set False for development)
 co2limit,:math:`t_{CO_2-eq}/a`,float,Cap on total annual system carbon dioxide emissions
 co2base,:math:`t_{CO_2-eq}/a`,float,Reference value of total annual system carbon dioxide emissions if relative emission reduction target is specified in ``{opts}`` wildcard.
+retirement, --,One of ``economic`` or ``technical``,"Sets the retirement method for converntional generators. If ``technical`` all generators ``p_nom_min`` are set to ``p_nom`` to prevent selling off of the asset. Retirements are then tracked in post-proccessing. If ``economic`` existing plants have their ``p_nom_min`` set as ``0``,  ``p_nom_max`` set to ``p_nom``,  and capital costs set to fixed costs. Generators with ``p_nom`` are then added to handle capacity expansion."""
+,,,
+operational_reserve:,,,Settings for reserve requirements following `GenX <https://genxproject.github.io/GenX/dev/core/#Reserves>`_
+--activate,bool,true or false,Whether to take operational reserve requirements into account during optimisation
+--epsilon_load,--,float,share of total load
+--epsilon_vres,--,float,share of total renewable supply
+--contingency,MW,float,fixed reserve capacity
 ,,,
 max_hours:,,,
 battery,h,float,Maximum state of charge capacity of the battery in terms of hours at full output capacity ``p_nom``. Cf. `PyPSA documentation <https://pypsa.readthedocs.io/en/latest/components.html#storage-unit>`_.
 ,,,
 extendable_carriers:,,,
 Generator,--,Any extendable carrier,"Defines existing or non-existing conventional and renewable power plants to be extendable during the optimization. Conventional generators can only be built/expanded where already existent today. If a listed conventional carrier is not included in the ``conventional_carriers`` list, the lower limit of the capacity expansion is set to 0."
-Storage Unit,--,"Any subset of {'battery','H2'}",Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
-Store,--,"Any subset of {'battery','H2'}",Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
-Links,--,Any subset of {'H2 pipeline'},Adds extendable links (H2 pipelines only) at every connection where there are lines or HVDC links without capacity limits and with zero initial capacity. Hydrogen pipelines require hydrogen storage to be modelled as ``Store``.
+Storage Unit,--,Any subset of {``battery``},Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
+Store,--,Any subset of {``battery``},Adds extendable storage units (battery and/or hydrogen) at every node/bus after clustering without capacity limits and with zero initial capacity.
+Links,--,Any subset of {},Adds extendable linksat every connection where there are lines or HVDC links without capacity limits and with zero initial capacity. Hydrogen pipelines require hydrogen storage to be modelled as ``Store``.
+,,,
+demand:,,,
+--profile,--,"One of {``efs``, ``eia``}",Datasource for electrical load data. ``EFS`` pulls future state level electrical demand data. ``EIA`` pulls historical balancing level electrical demand dataa.
+--scale,--,"One of {``efs``, ``aeo``}, or a float",(UNDER DEVELOPMENT) Used to scale the demand profile data. ``AEO`` will scale according to demand projections from the Annual Energy Outlook. ``EFS`` will scale according to growth rates from the Electrification Futures Study. Or can sclae according to a user defined value.
+--disaggregation,--,One of {``pop``},Method to dissagreagate load data. ``pop`` will dissagregate based on population from Breakthrough Energy.
+scenario:,,,
+--efs_case,--,"One of {``reference``, ``medium``, ``high``}",(UNDER DEVELOPMENT) Extracts EFS data according to level of adoption
+--efs_speed,--,"One of {``slow``, ``moderate``, ``fast``}",(UNDER DEVELOPMENT) Extracts EFS data according to speed of electrification
+--aeo,--,One of the AEO scenarios `here <https://www.eia.gov/outlooks/aeo/data/browser/>`_,(UNDER DEVELOPMENT) Scales future demand according to the AEO scenario
 ,,,
-retirement, --," One of ""economic"" or ""technical"""," ""Sets the retirement method for converntional generators. If ``technical`` all generators ``p_nom_min`` are set to ``p_nom`` to prevent selling off of the asset. Retirements are then tracked in post-proccessing. If ``economic`` existing plants have their ``p_nom_min`` set as ``0``,  ``p_nom_max`` set to ``p_nom``,  and capital costs set to fixed costs. Generators with ``p_nom`` are then added to handle capacity expansion."""
+autarky,,,
+--enable,bool,``true`` or ``false``,Require each node to be autarkic by removing all lines and links.
+--by_country,bool,``true`` or ``false``,Require each region to be autarkic by removing all cross-border lines and links. ``electricity: autarky`` must be enabled.

docs/source/configtables/sector.csv

@@ -1,7 +1,30 @@
 ,Unit,Values,Description
 co2_sequestration_potential,MtCO2/a,"float","The potential of sequestering CO2 in the spatial scope per year"
+,,,
 natural_gas,,,"Options when implementing natural gas network with sector wildcard 'G'"
 -- allow_imports_exports,bool,"{true, false}","Allow international imports/exports"
 -- cyclic_storage,bool,"{true, false}","Apply cyclic storage constraints on linepack and underground storage"
+,,,
 heating,,,"Options when implementing heating network with sector wildcard 'H'"
 -- heat_pump_sink_T,C,"float","The temperature heat sink used in heat pumps based on DTU / large area radiators. The value is conservatively high to cover hot water and space heating in poorly-insulated buildings"
+,,,
+demand:,,,Demand configuration options for each end use sector
+profile:,,,Demand profile source. ``EFS`` pulls future state level electrical demand data. ``eulp`` pulls End Use Load Profiles for 2018."
+--residential,--,"One of {``efs``, ``eulp``}",Datasource for residential electrical and cooling and heating data.
+--commercial,--,"One of {``efs``, ``eulp``}",Datasource for commercial electrical and cooling and heating data.
+--transport,--,"One of {``efs``}",Datasource for transportation electrical data.
+--industry,--,"One of {``efs``}",Datasource for industrial electrical data.
+scale:,,,Scales data. ``AEO`` will scale according to demand projections from the Annual Energy Outlook. ``EFS`` will scale according to growth rates from the Electrification Futures Study. Or can scale according to a user defined value.
+--residential,--,"One of {``efs``, ``aeo``}, or a float",(UNDER DEVELOPMENT) Used to scale residential demand profile data.
+--commercial,--,"One of {``efs``, ``aeo``}, or a float",(UNDER DEVELOPMENT) Used to scale commercial demand profile data.
+--transport,--,"One of {``efs``, ``aeo``}, or a float",(UNDER DEVELOPMENT) Used to scale transport demand profile data.
+--industry,--,"One of {``efs``, ``aeo``}, or a float",(UNDER DEVELOPMENT) Used to scale industrial demand profile data.
+disaggregation:,,,Dissagregation method. ``pop`` will dissagregate based on population from Breakthrough Energy.
+--residential,--,One of {``pop``},Method to dissagreagate residential load data.
+--commercial,--,One of {``pop``},Method to dissagreagate commercial load data.
+--transport,--,One of {``pop``},Method to dissagreagate transport load data.
+--industry,--,One of {``pop``},Method to dissagreagate industrial load data.
+scenario:,,,
+--efs_case,--,"One of {``reference``, ``medium``, ``high``}",(UNDER DEVELOPMENT) Extracts EFS data according to level of adoption
+--efs_speed,--,"One of {``slow``, ``moderate``, ``fast``}",(UNDER DEVELOPMENT) Extracts EFS data according to speed of electrification
+--aeo,--,One of the AEO scenarios `here <https://www.eia.gov/outlooks/aeo/data/browser/>`_,(UNDER DEVELOPMENT) Scales future demand according to the AEO scenario

docs/source/index.md

@@ -79,3 +79,9 @@ license
 changelog
 contributing
 ```
+
+```{toctree}
+:hidden:
+
+config-co2-base
+```

workflow/repo_data/config/config.default.yaml

@@ -62,9 +62,9 @@ electricity:
 
   extendable_carriers:
     Generator: [solar, onwind, offwind, offwind_floating, OCGT, CCGT, coal] #offwind, offwind_floating,
-    StorageUnit: [4hr_battery_storage] # [Xhr-battery-storage (2-10 hours), H2]
-    Store: [] #[H2]
-    Link: [] #[H2 pipeline]
+    StorageUnit: [4hr_battery_storage] # [Xhr-battery-storage (2-10 hours)]
+    Store: []
+    Link: [] 
 
   demand: 
     profile: efs # efs, eia
@@ -274,4 +274,4 @@ solving:
     glpk-default: {} # Used in CI
 
   mem: 30000 #memory in MB; 20 GB enough for 50+B+I+H2; 100 GB for 181+B+I+H2
-  walltime: "12:00:00"
+  walltime: "12:00:00"

workflow/scripts/add_electricity.py

@@ -21,7 +21,7 @@
 
 .. seealso::
     Documentation of the configuration file `config/config.yaml` at :ref:`costs_cf`,
-    :ref:`electricity_cf`, :ref:`load_cf`, :ref:`renewable_cf`, :ref:`lines_cf`
+    :ref:`electricity_cf`, :ref:`renewable_cf`, :ref:`lines_cf`
 
 **Inputs**
 

workflow/scripts/prepare_network.py

@@ -133,23 +133,24 @@ def add_emission_prices(n, emission_prices={"co2": 0.0}, exclude_co2=False):
     n.storage_units["marginal_cost"] += su_ep
 
 
-def add_dynamic_emission_prices(n):
-    co2_price = pd.read_csv(snakemake.input.co2_price, index_col=0, parse_dates=True)
-    co2_price = co2_price[~co2_price.index.duplicated()]
-    co2_price = (
-        co2_price.reindex(n.snapshots).fillna(method="ffill").fillna(method="bfill")
-    )
+"""Revisit if we add dynamic emission prices in"""
+# def add_dynamic_emission_prices(n):
+#     co2_price = pd.read_csv(snakemake.input.co2_price, index_col=0, parse_dates=True)
+#     co2_price = co2_price[~co2_price.index.duplicated()]
+#     co2_price = (
+#         co2_price.reindex(n.snapshots).fillna(method="ffill").fillna(method="bfill")
+#     )
 
-    emissions = (
-        n.generators.carrier.map(n.carriers.co2_emissions) / n.generators.efficiency
-    )
-    co2_cost = expand_series(emissions, n.snapshots).T.mul(co2_price.iloc[:, 0], axis=0)
+#     emissions = (
+#         n.generators.carrier.map(n.carriers.co2_emissions) / n.generators.efficiency
+#     )
+#     co2_cost = expand_series(emissions, n.snapshots).T.mul(co2_price.iloc[:, 0], axis=0)
 
-    static = n.generators.marginal_cost
-    dynamic = n.get_switchable_as_dense("Generator", "marginal_cost")
+#     static = n.generators.marginal_cost
+#     dynamic = n.get_switchable_as_dense("Generator", "marginal_cost")
 
-    marginal_cost = dynamic + co2_cost.reindex(columns=dynamic.columns, fill_value=0)
-    n.generators_t.marginal_cost = marginal_cost.loc[:, marginal_cost.ne(static).any()]
+#     marginal_cost = dynamic + co2_cost.reindex(columns=dynamic.columns, fill_value=0)
+#     n.generators_t.marginal_cost = marginal_cost.loc[:, marginal_cost.ne(static).any()]
 
 
 def set_line_s_max_pu(n, s_max_pu=0.7):
@@ -348,12 +349,7 @@ def set_line_nom_max(
     maybe_adjust_costs_and_potentials(n, snakemake.params["adjustments"])
 
     emission_prices = snakemake.params.costs["emission_prices"]
-    if emission_prices["co2_monthly_prices"]:
-        logger.info(
-            "Setting time dependent emission prices according spot market price",
-        )
-        add_dynamic_emission_prices(n)
-    elif emission_prices["enable"]:
+    if emission_prices["enable"]:
         add_emission_prices(
             n,
             dict(co2=snakemake.params.costs["emission_prices"]["co2"]),