{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "65410fbf",
   "metadata": {},
   "source": [
    "# FRED API in Python: U.S. Economic Data Analysis\n",
    "\n",
    "This notebook implements the analysis workflow described in the article brief.\n",
    "\n",
    "It covers:\n",
    "\n",
    "- Direct FRED API requests with `requests`\n",
    "- Safe API key handling with `FRED_API_KEY`\n",
    "- Cleaning FRED observations into pandas DataFrames\n",
    "- UNRATE, CPIAUCSL, FEDFUNDS, and GDPC1\n",
    "- Latest value and change calculations\n",
    "- 12-month unemployment change\n",
    "- Year-over-year CPI inflation\n",
    "- Real GDP growth\n",
    "- Multi-series alignment\n",
    "- Latest indicators summary table\n",
    "- Matplotlib charts\n",
    "- Plotly animated economic pulse chart\n",
    "\n",
    "> **Before running:** create a FRED API key and expose it as an environment variable named `FRED_API_KEY`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa315187",
   "metadata": {},
   "source": [
    "## 1. Install packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b9c4b04c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run once if needed:\n",
    "# %pip install requests pandas matplotlib plotly numpy\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f071d40d",
   "metadata": {},
   "source": [
    "## 2. Imports and API key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d7999ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import requests\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import plotly.graph_objects as go\n",
    "\n",
    "from IPython.display import display\n",
    "\n",
    "API_KEY = os.getenv(\"FRED_API_KEY\")\n",
    "\n",
    "if not API_KEY:\n",
    "    raise RuntimeError(\n",
    "        \"FRED_API_KEY is not set. Add your FRED API key to the environment before running this notebook.\"\n",
    "    )\n",
    "\n",
    "BASE_URL = \"https://api.stlouisfed.org/fred/series/observations\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd1b8f20",
   "metadata": {},
   "source": [
    "## 3. Reusable FRED downloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd70cde1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_fred_series(series_id, api_key, start_date=None, end_date=None):\n",
    "    params = {\n",
    "        \"series_id\": series_id,\n",
    "        \"api_key\": api_key,\n",
    "        \"file_type\": \"json\",\n",
    "    }\n",
    "\n",
    "    if start_date:\n",
    "        params[\"observation_start\"] = start_date\n",
    "    if end_date:\n",
    "        params[\"observation_end\"] = end_date\n",
    "\n",
    "    response = requests.get(BASE_URL, params=params, timeout=30)\n",
    "    response.raise_for_status()\n",
    "\n",
    "    payload = response.json()\n",
    "\n",
    "    if \"observations\" not in payload:\n",
    "        raise ValueError(f\"No observations returned for {series_id}\")\n",
    "\n",
    "    df = pd.DataFrame(payload[\"observations\"])\n",
    "\n",
    "    if df.empty:\n",
    "        raise ValueError(f\"Empty observation set for {series_id}\")\n",
    "\n",
    "    df[\"date\"] = pd.to_datetime(df[\"date\"])\n",
    "    df[\"value\"] = pd.to_numeric(df[\"value\"], errors=\"coerce\")\n",
    "    df = df[[\"date\", \"value\"]].sort_values(\"date\").reset_index(drop=True)\n",
    "    df[\"series_id\"] = series_id\n",
    "\n",
    "    return df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61a6861f",
   "metadata": {},
   "source": [
    "## 4. Download the core FRED series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4327031",
   "metadata": {},
   "outputs": [],
   "source": [
    "SERIES = {\n",
    "    \"Unemployment Rate\": \"UNRATE\",\n",
    "    \"Consumer Price Index\": \"CPIAUCSL\",\n",
    "    \"Federal Funds Rate\": \"FEDFUNDS\",\n",
    "    \"Real GDP\": \"GDPC1\",\n",
    "}\n",
    "\n",
    "data = {\n",
    "    name: get_fred_series(series_id, API_KEY)\n",
    "    for name, series_id in SERIES.items()\n",
    "}\n",
    "\n",
    "for name, df in data.items():\n",
    "    print(name, SERIES[name])\n",
    "    print(\"First observation:\", df.iloc[0][\"date\"].date(), df.iloc[0][\"value\"])\n",
    "    print(\"Last observation:\", df.iloc[-1][\"date\"].date(), df.iloc[-1][\"value\"])\n",
    "    print(\"Observations:\", len(df))\n",
    "    print(\"Missing:\", int(df[\"value\"].isna().sum()))\n",
    "    print()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bda5a00",
   "metadata": {},
   "source": [
    "## 5. Latest value, previous value, and change"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9e2ae0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def latest_stats(df):\n",
    "    clean = df.dropna(subset=[\"value\"]).copy()\n",
    "\n",
    "    latest = clean.iloc[-1]\n",
    "    previous = clean.iloc[-2] if len(clean) >= 2 else None\n",
    "\n",
    "    return {\n",
    "        \"latest_value\": latest[\"value\"],\n",
    "        \"latest_date\": latest[\"date\"],\n",
    "        \"previous_value\": previous[\"value\"] if previous is not None else np.nan,\n",
    "        \"change\": latest[\"value\"] - previous[\"value\"] if previous is not None else np.nan,\n",
    "    }\n",
    "\n",
    "for name, df in data.items():\n",
    "    print(name, latest_stats(df))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ded50bf",
   "metadata": {},
   "source": [
    "## 6. Unemployment analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c9849ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "unrate = data[\"Unemployment Rate\"].copy()\n",
    "unrate[\"change_1m\"] = unrate[\"value\"].diff()\n",
    "unrate[\"change_12m\"] = unrate[\"value\"].diff(12)\n",
    "\n",
    "display(unrate.tail(15))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa3417d1",
   "metadata": {},
   "source": [
    "## 7. CPI inflation analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e5eed5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "cpi = data[\"Consumer Price Index\"].copy()\n",
    "\n",
    "# CPIAUCSL is an index level. Inflation is calculated from its percent change.\n",
    "cpi[\"inflation_yoy_pct\"] = cpi[\"value\"].pct_change(12) * 100\n",
    "\n",
    "display(cpi.tail(15))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c10e1ea",
   "metadata": {},
   "source": [
    "## 8. Federal funds rate analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49125f7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "fedfunds = data[\"Federal Funds Rate\"].copy()\n",
    "fedfunds[\"change_1m\"] = fedfunds[\"value\"].diff()\n",
    "\n",
    "display(fedfunds.tail(15))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46397f5e",
   "metadata": {},
   "source": [
    "## 9. Real GDP growth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "988f1c9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "gdp = data[\"Real GDP\"].copy()\n",
    "\n",
    "# GDPC1 is quarterly real GDP.\n",
    "# Annualized quarter-over-quarter growth:\n",
    "gdp[\"qoq_annualized_pct\"] = ((gdp[\"value\"] / gdp[\"value\"].shift(1)) ** 4 - 1) * 100\n",
    "\n",
    "# Year-over-year growth:\n",
    "gdp[\"yoy_pct\"] = gdp[\"value\"].pct_change(4) * 100\n",
    "\n",
    "display(gdp.tail(12))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa76b1d0",
   "metadata": {},
   "source": [
    "## 10. Build a latest indicators summary table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d408c55e",
   "metadata": {},
   "outputs": [],
   "source": [
    "metadata = {\n",
    "    \"Unemployment Rate\": {\"frequency\": \"Monthly\", \"units\": \"Percent\"},\n",
    "    \"Consumer Price Index\": {\"frequency\": \"Monthly\", \"units\": \"Index\"},\n",
    "    \"Federal Funds Rate\": {\"frequency\": \"Monthly\", \"units\": \"Percent\"},\n",
    "    \"Real GDP\": {\"frequency\": \"Quarterly\", \"units\": \"Billions of chained 2017 dollars\"},\n",
    "}\n",
    "\n",
    "summary_rows = []\n",
    "\n",
    "for name, series_id in SERIES.items():\n",
    "    stats = latest_stats(data[name])\n",
    "\n",
    "    summary_rows.append({\n",
    "        \"Indicator\": name,\n",
    "        \"FRED series ID\": series_id,\n",
    "        \"Latest value\": stats[\"latest_value\"],\n",
    "        \"Observation date\": stats[\"latest_date\"].date(),\n",
    "        \"Previous value\": stats[\"previous_value\"],\n",
    "        \"Change\": stats[\"change\"],\n",
    "        \"Frequency\": metadata[name][\"frequency\"],\n",
    "        \"Units\": metadata[name][\"units\"],\n",
    "    })\n",
    "\n",
    "summary = pd.DataFrame(summary_rows)\n",
    "display(summary)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e697b40",
   "metadata": {},
   "source": [
    "## 11. Align monthly indicators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9378eb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "monthly = (\n",
    "    unrate[[\"date\", \"value\"]]\n",
    "    .rename(columns={\"value\": \"unemployment\"})\n",
    "    .merge(\n",
    "        cpi[[\"date\", \"inflation_yoy_pct\"]],\n",
    "        on=\"date\",\n",
    "        how=\"inner\"\n",
    "    )\n",
    "    .merge(\n",
    "        fedfunds[[\"date\", \"value\"]].rename(columns={\"value\": \"fedfunds\"}),\n",
    "        on=\"date\",\n",
    "        how=\"inner\"\n",
    "    )\n",
    "    .dropna()\n",
    "    .sort_values(\"date\")\n",
    "    .reset_index(drop=True)\n",
    ")\n",
    "\n",
    "display(monthly.tail())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd1aa95d",
   "metadata": {},
   "source": [
    "## 12. Frequency alignment note\n",
    "\n",
    "UNRATE, CPIAUCSL, and FEDFUNDS are monthly series, while GDPC1 is quarterly. Do not merge quarterly GDP directly into a monthly comparison without explicitly choosing a resampling or alignment method.\n",
    "\n",
    "For the animated comparison below, the notebook uses only the three monthly indicators.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56e2feb1",
   "metadata": {},
   "source": [
    "## 13. Static plot: U.S. unemployment rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f08e44c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(11, 5))\n",
    "ax.plot(unrate[\"date\"], unrate[\"value\"])\n",
    "ax.set_title(\"U.S. Unemployment Rate\")\n",
    "ax.set_xlabel(\"Date\")\n",
    "ax.set_ylabel(\"Percent\")\n",
    "ax.grid(True, alpha=0.25)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "709a3e35",
   "metadata": {},
   "source": [
    "## 14. Static plot: year-over-year CPI inflation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5f5ce51",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(11, 5))\n",
    "ax.plot(cpi[\"date\"], cpi[\"inflation_yoy_pct\"])\n",
    "ax.axhline(0, linewidth=1)\n",
    "ax.set_title(\"U.S. CPI Inflation, Year over Year\")\n",
    "ax.set_xlabel(\"Date\")\n",
    "ax.set_ylabel(\"Percent\")\n",
    "ax.grid(True, alpha=0.25)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "341d69f8",
   "metadata": {},
   "source": [
    "## 15. Static multi-indicator dashboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd34f8aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(11, 4))\n",
    "ax.plot(monthly[\"date\"], monthly[\"unemployment\"])\n",
    "ax.set_title(\"U.S. Unemployment Rate\")\n",
    "ax.set_ylabel(\"Percent\")\n",
    "ax.grid(True, alpha=0.25)\n",
    "plt.show()\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(11, 4))\n",
    "ax.plot(monthly[\"date\"], monthly[\"inflation_yoy_pct\"])\n",
    "ax.set_title(\"U.S. CPI Inflation, Year over Year\")\n",
    "ax.set_ylabel(\"Percent\")\n",
    "ax.grid(True, alpha=0.25)\n",
    "plt.show()\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(11, 4))\n",
    "ax.plot(monthly[\"date\"], monthly[\"fedfunds\"])\n",
    "ax.set_title(\"U.S. Federal Funds Rate\")\n",
    "ax.set_ylabel(\"Percent\")\n",
    "ax.grid(True, alpha=0.25)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc6539c0",
   "metadata": {},
   "source": [
    "## 16. Normalize monthly indicators for comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "238f65e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "animated = monthly.copy()\n",
    "\n",
    "for col in [\"unemployment\", \"inflation_yoy_pct\", \"fedfunds\"]:\n",
    "    mean = animated[col].mean()\n",
    "    std = animated[col].std()\n",
    "    animated[f\"{col}_z\"] = (animated[col] - mean) / std\n",
    "\n",
    "animated = animated.dropna().reset_index(drop=True)\n",
    "\n",
    "display(animated.tail())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c4d3a1d",
   "metadata": {},
   "source": [
    "## 17. Plotly animation: U.S. Economic Pulse Through Time\n",
    "\n",
    "The animation compares normalized z-scores, which show relative movement rather than original measurement units. Hover text retains the original indicator values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2aedada7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Downsample frames for better browser performance.\n",
    "frame_step = max(1, len(animated) // 180)\n",
    "frame_indices = list(range(1, len(animated), frame_step))\n",
    "\n",
    "if frame_indices[-1] != len(animated) - 1:\n",
    "    frame_indices.append(len(animated) - 1)\n",
    "\n",
    "series_config = [\n",
    "    (\"unemployment_z\", \"Unemployment\", \"unemployment\", \"%\"),\n",
    "    (\"inflation_yoy_pct_z\", \"Inflation\", \"inflation_yoy_pct\", \"%\"),\n",
    "    (\"fedfunds_z\", \"Federal Funds Rate\", \"fedfunds\", \"%\"),\n",
    "]\n",
    "\n",
    "def make_trace(df, z_col, label, original_col, unit, upto):\n",
    "    sub = df.iloc[:upto + 1]\n",
    "    hover = [\n",
    "        f\"{label}: {v:.2f}{unit}<br>Date: {dt:%Y-%m-%d}\"\n",
    "        for v, dt in zip(sub[original_col], sub[\"date\"])\n",
    "    ]\n",
    "\n",
    "    return go.Scatter(\n",
    "        x=sub[\"date\"],\n",
    "        y=sub[z_col],\n",
    "        mode=\"lines\",\n",
    "        name=label,\n",
    "        text=hover,\n",
    "        hovertemplate=\"%{text}<br>Normalized: %{y:.2f}<extra></extra>\",\n",
    "    )\n",
    "\n",
    "initial_idx = frame_indices[0]\n",
    "\n",
    "fig = go.Figure(\n",
    "    data=[\n",
    "        make_trace(animated, z, label, orig, unit, initial_idx)\n",
    "        for z, label, orig, unit in series_config\n",
    "    ],\n",
    "    frames=[\n",
    "        go.Frame(\n",
    "            name=str(idx),\n",
    "            data=[\n",
    "                make_trace(animated, z, label, orig, unit, idx)\n",
    "                for z, label, orig, unit in series_config\n",
    "            ],\n",
    "        )\n",
    "        for idx in frame_indices\n",
    "    ],\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    title=\"U.S. Economic Pulse Through Time: Unemployment, Inflation, and Interest Rates\",\n",
    "    xaxis_title=\"Date\",\n",
    "    yaxis_title=\"Normalized z-score\",\n",
    "    hovermode=\"x unified\",\n",
    "    updatemenus=[\n",
    "        {\n",
    "            \"type\": \"buttons\",\n",
    "            \"direction\": \"left\",\n",
    "            \"buttons\": [\n",
    "                {\n",
    "                    \"label\": \"Play\",\n",
    "                    \"method\": \"animate\",\n",
    "                    \"args\": [\n",
    "                        None,\n",
    "                        {\n",
    "                            \"frame\": {\"duration\": 70, \"redraw\": True},\n",
    "                            \"transition\": {\"duration\": 0},\n",
    "                            \"fromcurrent\": True,\n",
    "                        },\n",
    "                    ],\n",
    "                },\n",
    "                {\n",
    "                    \"label\": \"Pause\",\n",
    "                    \"method\": \"animate\",\n",
    "                    \"args\": [\n",
    "                        [None],\n",
    "                        {\n",
    "                            \"frame\": {\"duration\": 0, \"redraw\": False},\n",
    "                            \"mode\": \"immediate\",\n",
    "                            \"transition\": {\"duration\": 0},\n",
    "                        },\n",
    "                    ],\n",
    "                },\n",
    "            ],\n",
    "        }\n",
    "    ],\n",
    "    sliders=[\n",
    "        {\n",
    "            \"steps\": [\n",
    "                {\n",
    "                    \"method\": \"animate\",\n",
    "                    \"label\": animated.iloc[idx][\"date\"].strftime(\"%Y-%m\"),\n",
    "                    \"args\": [\n",
    "                        [str(idx)],\n",
    "                        {\n",
    "                            \"mode\": \"immediate\",\n",
    "                            \"frame\": {\"duration\": 0, \"redraw\": True},\n",
    "                            \"transition\": {\"duration\": 0},\n",
    "                        },\n",
    "                    ],\n",
    "                }\n",
    "                for idx in frame_indices\n",
    "            ],\n",
    "            \"currentvalue\": {\"prefix\": \"Date: \"},\n",
    "        }\n",
    "    ],\n",
    ")\n",
    "\n",
    "fig.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de3210b5",
   "metadata": {},
   "source": [
    "## 18. Optional exports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac6c4999",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Summary table\n",
    "summary.to_csv(\"fred_latest_indicators.csv\", index=False)\n",
    "\n",
    "# Clean monthly comparison dataset\n",
    "monthly.to_csv(\"fred_monthly_indicators.csv\", index=False)\n",
    "\n",
    "# Interactive Plotly animation\n",
    "fig.write_html(\"us_economic_pulse_plotly.html\")\n",
    "\n",
    "print(\"Exports complete.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "feb8a249",
   "metadata": {},
   "source": [
    "## 19. Publishing checks\n",
    "\n",
    "Before publishing results from this notebook:\n",
    "\n",
    "1. Record the data access date.\n",
    "2. State the observation date for every latest value.\n",
    "3. Remember that FRED observations can be revised.\n",
    "4. Keep series IDs visible near tables and charts.\n",
    "5. Explain any frequency conversions.\n",
    "6. Do not treat correlation as proof of causation.\n",
    "7. Do not publish a private API key.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}