{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": { "mystnb": { "nb_execution_mode": "off" } }, "source": [ "# Marginal Emissions\n", "**Exploring the functionality of NEMED to extract regional marginal emitters from price setting and generator emissions data**
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data Preparation\n", "#### Import Packages" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "import nemed\n", "\n", "import pandas as pd\n", "# To generate plots shown\n", "import plotly.graph_objects as go\n", "import plotly.express as px\n", "import string\n", "\n", "# Open plot in browser (optional)\n", "import plotly.io as pio\n", "pio.renderers.default = \"browser\"" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "tags": [ "full-width" ] }, "source": [ "#### Processing Marginal Emissions Data\n", "Regional Marginal Emissions can be extracted in NEMED using `get_marginal_emissions`. The following inputs must be specified:\n", "- `start_time` define the start of the historical period to collect data for. Must be in the format: \"yyyy/mm/dd HH:MM\"\n", "- `end_time` define the end of the historical period to collect data for. Must be in the format: \"yyyy/mm/dd HH:MM\"\n", "- `cache` specify the local file directory to temporarily store downloaded files\n", "\n", "The returned dataframe will contain timeseries data with columns:\n", "| Column | Type | Description |\n", "| ------ | ---- | ----------- |\n", "| Time | datetime | Timestamp reported as end of dispatch interval. |\n", "| Region | string | The NEM region corresponding to the marginal emitter data. |\n", "| Intensity_Index | float | The intensity index [tCO2e/MWh] (as by weighted contributions) of the price-setting generators. |\n", "| DUID | string | Unit identifier of the generator with the largest contribution on the margin for that Time-Region. |\n", "| CO2E_ENERGY_SOURCE | string | Unit energy source with the largest contribution on the margin for that Time-Region. |" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "The simpliest way to collect emissions data is:\n", "```{note} \n", "Extracting 1 month of marginal emissions data was found to take approx. 2 minutes in testing.
\n", "Extracting 1 year of marginal emissions data was found to take approx. 30 minutes. Cached files amount to 3 GB.\n", "```" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "tags": [ "hide-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING: Warning: Gen_info table only has most recent NEM registration and exemption list. Does not account for retired generators\n", "INFO: Processing Price Setter Files...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "0it [00:00, ?it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Reading selected 366 JSON files to pandas, of cached files\n", "INFO: Loading Cached Price Setter Files...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 366/366 [01:42<00:00, 3.56it/s]\n" ] } ], "source": [ "result = nemed.get_marginal_emissions(start_time=\"2021/01/01 00:00\",\n", " end_time=\"2022/01/01 00:00\",\n", " cache=\"E:/TEMPCACHE2/\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```{info}\n", " Q: Why is the Intensity_Index different below although it shows same DUID?\n", "The Intensity_Index considers numerous generators that may be on the margin, and their contribution to price-setting. However, the DUID and CO2E_ENERGY_SOURCE shown only reflects a single generator with the largest influence on the margin.\n", "\n", "```" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TimeRegionIntensity_IndexDUIDCO2E_ENERGY_SOURCE
02021-01-01 00:05:00NSW10.908305VP6Black coal
12021-01-01 00:05:00QLD10.855969VP6Black coal
22021-01-01 00:05:00SA10.867177VP5Black coal
32021-01-01 00:05:00TAS10.000000GORDONHydro
42021-01-01 00:05:00VIC10.895716VP6Black coal
\n", "
" ], "text/plain": [ " Time Region Intensity_Index DUID CO2E_ENERGY_SOURCE\n", "0 2021-01-01 00:05:00 NSW1 0.908305 VP6 Black coal\n", "1 2021-01-01 00:05:00 QLD1 0.855969 VP6 Black coal\n", "2 2021-01-01 00:05:00 SA1 0.867177 VP5 Black coal\n", "3 2021-01-01 00:05:00 TAS1 0.000000 GORDON Hydro\n", "4 2021-01-01 00:05:00 VIC1 0.895716 VP6 Black coal" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Aggregate/Filter data\n", "Data retrieve from NEMED can be manually filtered. For example sorting by region we can produce..." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "result['Date'] = result['Time'].dt.date\n", "result['Hour'] = result['Time'].dt.hour\n", "result['Season'] = result['Time'].dt.month%12 // 3 +1\n", "result['Season'].replace({1:'Summer', 2:'Autumn', 3:'Winter', 4:'Spring'}, inplace=True)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Example Chart 1\n", "#### Which fuel (generator) type is most often the marginal emitter?\n", "The below chart can be reproduced using the code below (toggle the cell to view)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "result['Date'] = result['Time'].dt.date\n", "result['Hour'] = result['Time'].dt.hour\n", "result['Season'] = result['Time'].dt.month%12 // 3 +1\n", "result['Season'].replace({1:'Summer', 2:'Autumn', 3:'Winter', 4:'Spring'}, inplace=True)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "def NORD_theme():\n", " plotly_NORD_theme = pio.templates[\"plotly_white\"]\n", " plotly_NORD_theme.layout.plot_bgcolor = \"#f4f4f5\" \n", " plotly_NORD_theme.layout.paper_bgcolor = \"#f4f4f5\"\n", " plotly_NORD_theme.layout.xaxis.gridcolor = '#d8dee9'\n", " plotly_NORD_theme.layout.yaxis.gridcolor = '#d8dee9'\n", " return plotly_NORD_theme\n", "\n", "def set_font_size(layout, font_size=16):\n", " layout['titlefont']['size'] = font_size + 4\n", " layout.legend['font']['size'] = font_size\n", "\n", " for ax in [item for item in layout if item.__contains__('xaxis')]:\n", " layout[ax].titlefont.size = font_size\n", " layout[ax].tickfont.size = font_size\n", "\n", " for ax in [item for item in layout if item.__contains__('yaxis')]:\n", " layout[ax].titlefont.size = font_size\n", " layout[ax].tickfont.size = font_size\n", "\n", "def plot_marginal_fuelsrc(plt_df):\n", " df = plt_df[plt_df['Region']!=\"TAS1\"].copy(deep=True)\n", " df['Region'] = df['Region'].str.rstrip(string.digits)\n", " # Chart\n", " fig = px.histogram(\n", " df,\n", " x=\"Hour\",\n", " color=\"CO2E_ENERGY_SOURCE\",\n", " facet_col=\"Season\",\n", " facet_row=\"Region\",\n", " barnorm=\"percent\", \n", " category_orders={\"Season\":[\"Spring\", \"Summer\", \"Autumn\", \"Winter\"]},\n", " height=1000,\n", " width=1200,\n", " color_discrete_sequence=['#0C0A0C','#5E3F1C','#00527A','#5A9367','#8F6593','#FFB41F','#D2F1E4','#E9724C',\n", " '#6D250D','#F4B8A4'])\n", " # Layout\n", " fig.update_layout(title=dict(text=f\"Historical CY2021 Marginal Emitter (by Fuel Source)
\"+\\\n", " \"NEMED | Marginal Emissions Methodology | Categorisation by Fuel Source (Mainland Regions)\",\n", " y=0.95),\n", " template=NORD_theme(),\n", " legend={'title':'', 'orientation':'h', 'xanchor': 'center', 'x': 0.5, 'y':-0.1},\n", " margin=dict(l=120, r=60, t=140, b=60))\n", " \n", " # Axis Formatting\n", " fig.update_yaxes(title_text=None, mirror=True, showgrid=False)\n", " fig.update_yaxes(title=\"% of dispatch intervals\", tickvals=[0,25,50,75,100], col=1,)\n", " fig.update_xaxes(mirror=True, showgrid=False)\n", " ax_time = dict(title_text=\"Time of Day [h]\")\n", " fig.update_layout(xaxis=ax_time, xaxis2=ax_time, xaxis3=ax_time, xaxis4=ax_time)\n", " \n", " # Font Formatting\n", " FONT_SIZE = 16\n", " FONT_STYLE = \"Raleway\"\n", " fonts = dict(tickfont=dict(size=FONT_SIZE, family=FONT_STYLE),\n", " titlefont=dict(size=FONT_SIZE, family=FONT_STYLE))\n", " fig.update_layout(xaxis=fonts, xaxis2=fonts, xaxis3=fonts, xaxis4=fonts,\n", " yaxis=fonts, yaxis5=fonts, yaxis9=fonts, yaxis13=fonts,\n", " legend=dict(font=dict(size=FONT_SIZE-2, family=FONT_STYLE)),\n", " title_font_family=FONT_STYLE,\n", " title_font_size=22)\n", " fig.update_annotations(font=dict(size=FONT_SIZE, family=FONT_STYLE)) \n", " return fig" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "fig = plot_marginal_fuelsrc(result)\n", "fig.show()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```{admonition} Interactive Plot\n", "Click the image to open the plot as an interactive plotly\n", "```\n", "\n", "```{image} charts/marginal_emissions_chart_1.png\n", ":target: ../_static/html_charts/marginal_emissions_chart_1.html\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.9.12 ('nempy')", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12 (main, Apr 4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "4aab49ac747d4948ee2428bd46f4ac833ef94a37ecb38233c747c75e4d05fe4b" } } }, "nbformat": 4, "nbformat_minor": 2 }