{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b91f3198",
   "metadata": {},
   "source": [
    "# Process a BIOMASS L1 product on MAAP using the STAC API\n",
    "\n",
    "\n",
    "**Important note:** To use this notebook, an account on ESA BIOMASS MAAP Product Algorithm Laboratory is required.\n",
    "\n",
    "This tutorial is for users who are already familiar with STAC API and want to process data locally without explicit download or directly in the cloud. \n",
    "It is shown how to:\n",
    "- Look for a product using the STAC API\n",
    "- Use credentials to get an access token\n",
    "- Open BIOMASS raster data lazily with rioxarray (rasterio+xarray) using file urls\n",
    "- Make a valid PolSARpro dataset from these raster arrays\n",
    "- Apply a polarimetric decomposition and save the result locally\n",
    "\n",
    "### Prerequisites:\n",
    "- Access to the collaborative environment https://portal.maap.eo.esa.int/biomass/initiative-membership/\n",
    "- Credentials as described here https://portal.maap.eo.esa.int/ini/kb/books/enhanced-data-access/page/using-the-long-lasting-token-to-access-and-download-data \n",
    "- Pystac-client installed (`conda install pystac-client`)\n",
    "\n",
    "This notebook may be run locally or in the MAAP coding environment. Installation of PyPolSARpro on the MAAP environment is described here: https://gist.github.com/odhondt/f9ce88acca5f91e8f3563b32a342addc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "da5de933-73c7-467d-93e4-9e658623bcb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from pystac_client import Client\n",
    "from pathlib import Path\n",
    "\n",
    "URL_LANDING_PAGE = \"https://catalog.maap.eo.esa.int/catalogue/\"\n",
    "api = Client.open(URL_LANDING_PAGE) # Connection to the catalog"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98df8f54-4a3e-4414-9efc-710be8c537c1",
   "metadata": {},
   "source": [
    "## Search a product by ID using the STAC API\n",
    "This step may be replaced by any valid STAC API query.\n",
    "In the following, for the sake of demonstration, we only process the first product in the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d11d4168-0f3c-4f19-826b-c1196fe4fc18",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "ids = [\"BIO_S2_SCS__1S_20251216T034800_20251216T034815_T_G01_M01_C02_T017_F289_01_DJQGAN\"]\n",
    "\n",
    "results = api.search(\n",
    "    method = 'GET',   \n",
    "    collections = 'BiomassLevel1a',\n",
    "    ids = ids\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e3fdf943-f1a6-4ba0-893c-a924938cc806",
   "metadata": {},
   "outputs": [],
   "source": [
    "items = results.item_collection()\n",
    "# uncomment the following line to inspect item\n",
    "# items[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3bc134b0-2b1d-43de-9764-98072b99c649",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Extract urls of amplitude and phase rasters\n",
    "url_abs = items[0].assets[\"enclosure_i_abs_tiff\"].href\n",
    "url_phase = items[0].assets[\"enclosure_i_phase_tiff\"].href"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "970be18e-e043-4064-887b-74f94ca13526",
   "metadata": {},
   "source": [
    "## Request a token for product access\n",
    "The code in the cell below was slightly adapted from: https://portal.maap.eo.esa.int/ini/kb/books/enhanced-data-access/page/using-the-long-lasting-token-to-access-and-download-data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c1146acb-8034-45c8-9b08-6bfb25885169",
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_credentials(file_path):\n",
    "    \"\"\"Read key-value pairs from a credentials file into a dictionary.\"\"\"\n",
    "    creds = {}\n",
    "    if not file_path.exists():\n",
    "        raise FileNotFoundError(f\"Credentials file not found: {file_path}\")\n",
    "    with open(file_path, \"r\") as f:\n",
    "        for line in f:\n",
    "            line = line.strip()\n",
    "            if not line or line.startswith(\"#\"):\n",
    "                continue\n",
    "            if \"=\" not in line:\n",
    "                continue\n",
    "            key, value = line.split(\"=\", 1)\n",
    "            creds[key.strip()] = value.strip()\n",
    "    return creds\n",
    "\n",
    "\n",
    "def get_token(credential_file_path):\n",
    "    \"\"\"Use OFFLINE_TOKEN to fetch a short-lived access token.\"\"\"\n",
    "    creds = load_credentials(Path(credential_file_path))\n",
    "\n",
    "    OFFLINE_TOKEN = creds.get(\"OFFLINE_TOKEN\")\n",
    "    CLIENT_ID = creds.get(\"CLIENT_ID\")\n",
    "    CLIENT_SECRET = creds.get(\"CLIENT_SECRET\")\n",
    "\n",
    "    if not all([OFFLINE_TOKEN, CLIENT_ID, CLIENT_SECRET]):\n",
    "        raise ValueError(\"Missing OFFLINE_TOKEN, CLIENT_ID, or CLIENT_SECRET in credentials file\")\n",
    "\n",
    "    url = \"https://iam.maap.eo.esa.int/realms/esa-maap/protocol/openid-connect/token\"\n",
    "    data = {\n",
    "        \"client_id\": CLIENT_ID,\n",
    "        \"client_secret\": CLIENT_SECRET,\n",
    "        \"grant_type\": \"refresh_token\",\n",
    "        \"refresh_token\": OFFLINE_TOKEN,\n",
    "        \"scope\": \"offline_access openid\"\n",
    "    }\n",
    "\n",
    "    response = requests.post(url, data=data)\n",
    "    response.raise_for_status()\n",
    "\n",
    "    response_json = response.json()\n",
    "    access_token = response_json.get('access_token')\n",
    "\n",
    "    if not access_token:\n",
    "        raise RuntimeError(\"Failed to retrieve access token from IAM response\")\n",
    "\n",
    "    return access_token"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0ca04f1-80f0-4e37-a2b7-b1d9504c93f2",
   "metadata": {},
   "source": [
    "## Access product raster and process on-the-fly\n",
    "Requisite: create a credential file as described here https://portal.maap.eo.esa.int/ini/kb/books/enhanced-data-access/page/using-the-long-lasting-token-to-access-and-download-data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8588d626-f930-483c-b3a1-53dd336baca7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Authentication\n",
    "token = get_token(\"/data/credentials.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d2f31191-f893-4ecc-8a97-a4786add562c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import rioxarray as riox\n",
    "import rasterio as rio\n",
    "\n",
    "# Datasets lazy loading\n",
    "with rio.Env(\n",
    "    GDAL_HTTP_HEADERS=f\"Authorization: Bearer {token}\",\n",
    "    GDAL_DISABLE_READDIR_ON_OPEN=\"EMPTY_DIR\",\n",
    "    GDAL_NUM_THREADS=\"ALL_CPUS\",\n",
    "):\n",
    "    da_abs = riox.open_rasterio(url_abs, chunks={})\n",
    "    da_phase = riox.open_rasterio(url_phase, chunks={})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "da19c647-cd43-4130-86b9-b5de0315b7a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import xarray as xr\n",
    "\n",
    "# Reconstruct SLC images\n",
    "slc = da_abs * np.exp(1j * da_phase)\n",
    "dims = (\"y\", \"x\")\n",
    "# Extract polSAR channels and create a PolSARpro S matrix\n",
    "S = xr.Dataset(\n",
    "    {\n",
    "        \"hh\": (dims, slc[0].data),\n",
    "        \"hv\": (dims, slc[1].data),\n",
    "        \"vh\": (dims, slc[2].data),\n",
    "        \"vv\": (dims, slc[3].data),\n",
    "    },\n",
    "    coords={\"y\": np.arange(slc.shape[1]), \"x\": np.arange(slc.shape[2])},\n",
    "    attrs={\"poltype\": \"S\", \"description\": \"Scattering matrix\"},\n",
    ").chunk(\"auto\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "746658f2",
   "metadata": {},
   "source": [
    "## Process with (Py)PolSARpro"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "252cf18e-8460-4079-82d8-090cadb9aa6f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[########################################] | 100% Completed | 36.01 s\n"
     ]
    }
   ],
   "source": [
    "from polsarpro.decompositions import freeman\n",
    "from dask.diagnostics import ProgressBar\n",
    "import os\n",
    "\n",
    "out_file = \"/data/psp/res/freeman_biomass.nc\"\n",
    "if os.path.exists(out_file):\n",
    "    os.remove(out_file)\n",
    "\n",
    "with ProgressBar():\n",
    "    freeman(S, [7, 7]).to_netcdf(out_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1ebcc22",
   "metadata": {},
   "source": [
    "## Visualize the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b61591c1-fdba-4bc2-82b5-a78676a28ddf",
   "metadata": {},
   "outputs": [],
   "source": [
    "out_file = \"/data/psp/res/freeman_biomass.nc\"\n",
    "out = xr.open_dataset(out_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ce17a80e-e3d3-4dc2-9ccd-400a4d4fcdd7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7dbcc6b15a60>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 422988 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "out.odd.plot.imshow(vmax=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3314957c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7dbcc7582bd0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 123604 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "out.double.plot.imshow(vmax=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6bd89b57",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7dbcc409b2f0>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 279000 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "out.volume.plot.imshow(vmax=1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "psp",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
