Skip to content

Retrieving Data

Not all data dependencies are shipped with the git repository, since git is not suited for handling large changing files. Instead we use separate steps in the workflow (rules executed by snakemake) to download external data using the retrieve_<dataset> rules.

Data is generally retrieved in a version-controlled manner, enabling control over input data versions, reproducibility and consistency of modelling runs. The rules download data into subfolders in the data/ directory, following the structure data/{dataset}/{source}/{version}, e.g. data/jrc_idees/primary/March-2025-V1/. Which specific data version is retrieve can be controlled in the data configuration.

Below some specific retrieve_<dataset> rules are documented. For more information on the datasets retrieved, see the data sources and Data inventory section there in the documentation.

Rule retrieve_bidding_zones

Rule retrieve_cutout

See cutouts.

Rule retrieve_electricity_demand_energy_atlas

This rule downloads 1km by 1km raster of estimated annual electricity demand from the JRC Energy Atlas.

Rule retrieve_desnz_electricity_consumption

This rule downloads subnational electricity consumption data for Great Britain from the Department for Energy Security and Net Zero.

Rule retrieve_ons_lad

This rule downloads shapefiles of local authorities in the United Kingdom from the Office for National Statistics.

Rule retrieve_electricity_demand_opsd

This rule downloads hourly electric load data for each country from the OPSD platform.

Relevant Settings

None.

Outputs

  • data/electricity_demand_opsd_raw.csv

Rule retrieve_electricity_demand_entsoe

This rule downloads hourly electric load data for each country from the ENTSOE Transparency Platform.

Relevant Settings

None.

Outputs

  • data/electricity_demand_entsoe_raw.csv

Rule retrieve_electricity_demand_neso

This rule downloads hourly electric load data for the United Kingdom from the NESO Data Portal.

Relevant Settings

None.

Outputs

  • data/electricity_demand_neso_raw.csv

Rule retrieve_cost_data

This rule downloads techno-economic assumptions from the technology-data repository.

Relevant Settings

costs:
    year:

Seealso

Documentation of the configuration file config/config.yaml at costs_cf

Outputs

  • data/costs/primary/{version}/costs_{year}.csv