Introduction to WaPOR and data loading
Products used: WaPOR
Keywords data used; wapor data used; crop mask
Background
The Water Productivity through Open access of Remotely sensed derived data (WaPOR) monitors and reports on agricultural water productivity through biophysical measures with a focus on Africa and the Near East. This information assists partner countries improve land and water productivity in both rainfed and irrigated agriculture (Peiser et al. 2017).
WaPOR provides numerous datasets related to vegetation productivity and water consumption, and associated meteorological and physical conditions such as soil moisture and precipitation. These datasets can be combined with Digital Earth Africa products, services, and workflows for numerous applications including: * Monitoring drought conditions * Monitoring the water use efficiency of crops * Mapping irrigated areas * Estimating crop water requirements * Irrigation scheduling and budgeting
Description
This notebook provides an introduction to WaPOR data and nomenclature, and demonstrates loading and plotting.
First, we explore the datasets available and how they are labelled.
Then, we download and plot annual evapotranspiration.
Finally, we download and plot dekadal (10 day temporal frequency) data.
Getting started
To run this analysis, run all the cells in the notebook, starting with the “Load packages” cell.
Load packages
Import Python packages that are used for the analysis.
Use standard import commands; some are shown below. Begin with any iPython magic commands, followed by standard Python packages, then any additional functionality you need from the Tools package.
[ ]:
#!pip3 install wapordl==0.9
# require lower version because wapordl 0.12.2 requires numpy<2,>=1.15, but sandbox has numpy 2.1.3 which is incompatible.
[ ]:
#!pip uninstall Tools -y
#!pip install ../Tools
[12]:
!pip3 install GDAL==3.8.4
Collecting GDAL==3.8.4
Downloading GDAL-3.8.4.tar.gz (802 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 802.5/802.5 kB 44.2 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Building wheels for collected packages: GDAL
Building wheel for GDAL (setup.py) ... done
Created wheel for GDAL: filename=GDAL-3.8.4-cp310-cp310-linux_x86_64.whl size=3650634 sha256=fdf063b25f1d6dedad1772150ebc4b62edb557cffd415cb5b1fac002906bc919
Stored in directory: /home/jovyan/.cache/pip/wheels/67/f4/aa/ad21e29df3d3124c18bc1282d3441730fb386c84497b29e250
Successfully built GDAL
Installing collected packages: GDAL
Attempting uninstall: GDAL
Found existing installation: GDAL 3.6.2
Uninstalling GDAL-3.6.2:
Successfully uninstalled GDAL-3.6.2
Successfully installed GDAL-3.8.4
[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
[13]:
import datetime
import datacube
from deafrica_tools.load_wapor import get_all_WaPORv3_mapsets, get_WaPORv3_info, load_wapor_ds
from deafrica_tools.plotting import display_map
from wapordl import wapor_map
Connect to the datacube
Connect to the datacube so we can access DE Africa data. The app parameter is a unique name for the analysis which is based on the notebook file name.
[2]:
dc = datacube.Datacube(app="WaPOR")
WaPOR Data
WaPOR data has three levels: 1. Global 300m resolution 2. National 100m resolution 3. Sub-national 20m resolution
The table below covers L1 and L2 datasets. L3 datasets can be viewed in the WaPOR maps platform which is built with the same software as Digital Earth Africa Maps. L3 datasets cover several regions of interest in northern and eastern Africa. This notebook loads level 3 20m data for Egypt. It is recommended that the WaPOR maps platform is inspected to check the availability of level, variable, and temporal frequency combinations for your area of interest. The maps platform also shows map codes in the data description.
Mapset codes are structured as level-variable-temporal frequency as shown below. The temporal frequencies available are: * A - annual * M - monthly * D - dekadal (10 days)
So, for level 3 net primary productivity at dekadal intervals the code would be L3-NPP-D.
[3]:
get_all_WaPORv3_mapsets()
[3]:
| Mapset Code | Mapset Description | |
|---|---|---|
| 0 | L1-AETI-A | Actual EvapoTranspiration and Interception (Gl... |
| 1 | L1-AETI-D | Actual EvapoTranspiration and Interception (Gl... |
| 2 | L1-AETI-M | Actual EvapoTranspiration and Interception (Gl... |
| 3 | L1-E-A | Evaporation (Global - Annual - 300m) |
| 4 | L1-E-D | Evaporation (Global - Dekadal - 300m) |
| 5 | L1-GBWP-A | Gross biomass water productivity (Annual - 300m) |
| 6 | L1-I-A | Interception (Global - Annual - 300m) |
| 7 | L1-I-D | Interception (Global - Dekadal - 300m) |
| 8 | L1-NBWP-A | Net biomass water productivity (Annual - 300m) |
| 9 | L1-NPP-D | Net Primary Production (Global - Dekadal - 300m) |
| 10 | L1-NPP-M | Net Primary Production (Global - Monthly - 300m) |
| 11 | L1-PCP-A | Precipitation (Global - Annual - Approximately... |
| 12 | L1-PCP-D | Precipitation (Global - Dekadal - Approximatel... |
| 13 | L1-PCP-E | Precipitation (Global - Daily - Approximately ... |
| 14 | L1-PCP-M | Precipitation (Global - Monthly - Approximatel... |
| 15 | L1-QUAL-LST-D | Quality land surface temperature (Global - Dek... |
| 16 | L1-QUAL-NDVI-D | Quality of Normalized Difference Vegetation In... |
| 17 | L1-RET-A | Reference Evapotranspiration (Global - Annual ... |
| 18 | L1-RET-D | Reference Evapotranspiration (Global - Dekadal... |
| 19 | L1-RET-E | Reference Evapotranspiration (Global - Daily -... |
| 20 | L1-RET-M | Reference Evapotranspiration (Global - Monthly... |
| 21 | L1-RSM-D | Relative Soil Moisture (Global - Dekadal - 300m) |
| 22 | L1-T-A | Transpiration (Global - Annual - 300m) |
| 23 | L1-T-D | Transpiration (Global - Dekadal - 300m) |
| 24 | L1-TBP-A | Total Biomass Production (Global - Annual - 300m) |
| 25 | L2-AETI-A | Actual EvapoTranspiration and Interception (Na... |
| 26 | L2-AETI-D | Actual EvapoTranspiration and Interception (Na... |
| 27 | L2-AETI-M | Actual EvapoTranspiration and Interception (Na... |
| 28 | L2-E-A | Evaporation (National - Annual - 100m) |
| 29 | L2-E-D | Evaporation (National - Dekadal - 100m) |
| 30 | L2-GBWP-A | Gross biomass water productivity (Annual - 100m) |
| 31 | L2-I-A | Interception (National - Annual - 100m) |
| 32 | L2-I-D | Interception (National - Dekadal - 100m) |
| 33 | L2-NBWP-A | Net biomass water productivity (Annual - 100m) |
| 34 | L2-NPP-D | Net Primary Production (National - Dekadal - 1... |
| 35 | L2-NPP-M | Net Primary Production (National - Monthly - 1... |
| 36 | L2-QUAL-NDVI-D | Quality of normalized difference vegetation in... |
| 37 | L2-RSM-D | Relative Soil Moisture (National - Dekadal - 1... |
| 38 | L2-T-A | Transpiration (National - Annual - 100m) |
| 39 | L2-T-D | Transpiration (National - Dekadal - 100m) |
| 40 | L2-TBP-A | Total Biomass Production (Global - Annual - 100m) |
Analysis parameters
The cell below specifies: * The area of interest. This can also be a .geojson file * The folder where the downloaded data will be stored. If you are using this script repeatedly, it is recommended you empty this folder from time to time to manage storage. It is advised that any data required for repeated or future analysis is stored outside the sandbox. * The variable of interest in the form of a mapset code, explained above. * The period of interest.
[4]:
region = [31.30, 30.70, 31.40, 30.80] #xmin, ymin, xmax, ymax
folder = "Supplementary_data/WaPOR" # folder that the data will be sent to
variable = "L3-AETI-A" # level-variable-time, see table above
period = ["2018-01-01", "2024-02-01"] # period to load
This demonstration notebook loads an area of cropland in the Nile Delta, Egypt.
[5]:
display_map(x=(region[0], region[2]), y=(region[1], region[3]))
[5]:
Download netCDF files
This cell downloads the data specified. The aeti object is the file path to the stored netCDF. Once this cell is run, netCDF files will be stored in the directory specified above.
[14]:
aeti = wapor_map(region, variable, period, folder, extension = '.nc')
INFO: Given `region` matches with `ENO` L3 region.
INFO: Found 6 files for L3-AETI-A.
INFO: Converting from `.tif` to `.nc`.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[14], line 1
----> 1 aeti = wapor_map(region, variable, period, folder, extension = '.nc')
File /opt/venv/lib/python3.10/site-packages/wapordl/main.py:585, in wapor_map(region, variable, period, folder, unit_conversion, overview, extension)
581 options = gdal.TranslateOptions(
582 **toptions.get(extension, {})
583 )
584 new_fp = fp.replace(".tif", extension)
--> 585 ds = gdal.Translate(new_fp, fp, options = options)
586 ds.FlushCache()
587 try:
File /opt/venv/lib/python3.10/site-packages/osgeo/gdal.py:523, in Translate(destName, srcDS, **kwargs)
520 if isinstance(srcDS, str):
521 srcDS = Open(srcDS)
--> 523 return TranslateInternal(destName, srcDS, opts, callback, callback_data)
File /opt/venv/lib/python3.10/site-packages/osgeo/gdal.py:4793, in TranslateInternal(*args)
4791 def TranslateInternal(*args) -> "GDALDatasetShadow *":
4792 r"""TranslateInternal(char const * dest, Dataset dataset, GDALTranslateOptions translateOptions, GDALProgressFunc callback=0, void * callback_data=None) -> Dataset"""
-> 4793 return _gdal.TranslateInternal(*args)
RuntimeError: Cannot guess driver for Supplementary_data/WaPOR/bb.ENO_L3-AETI-A_NONE_none.nc
Convert data to xarray
Below, the netCDFs are brought into the analysis environment as xarray datasets and coordinate labels and attributes are assigned. This means the data is in a common format with other Digital Earth Africa products and services, and makes the data easy to interact with. The load_wapor_ds() function brings in the data as xarray.
[ ]:
aeti_xr = load_wapor_ds(filename=aeti, variable=variable)
Inspect the xarray dataset
Now we have x, y, and time as dimensions and our variable (in this case, actual evapotranspiration and interception) as an xarray dataset. This enables us to easily deal with time and space for analysis.
[ ]:
aeti_xr
Plot annual ET
The plots show how AETI varies in space and between years. In the Egypt example, the cropland areas are easily visible as areas with higher AETI.
Note that the scalebar is labelled with information from the WaPOR metadata. This can be accessed by calling aeti_xr[variable].attrs, as below, which can be especially useful when checking units for calculation. The load_wapor_ds() function takes care of re-scaling when the data is loaded, but it is sensible to check the values are reasonable.
We can also see that the attributes include scale and offset values. These have been incorporated into the load_wapor_ds() function so the values returned are in the units shown below, in this case mm/year.
[ ]:
aeti_xr[variable].attrs
[ ]:
aeti_xr[variable].plot(col='time', col_wrap=3)
Load dekadal biomass
The cell below loads dekadal actual evapotranspiration using the same procedure as for annual. The only parameter changed is variable.
[ ]:
variable = 'L3-NPP-D'
period = ["2024-01-01", "2024-03-01"]
npp_d = wapor_map(region, variable, period, folder, extension = '.nc')
npp_d_xr = load_wapor_ds(filename=npp_d, variable=variable)
npp_d_xr
Plot dekadal net primary productivity
It’s interesting to note that in the Egypt example, some areas show very high biomass production > 30t/ha, especially in 2023. This must be considered in the context of several crop cycles occurring within a 12 month period.
[ ]:
npp_d_xr[variable].plot(col='time', col_wrap=3)
Conclusion
This notebook demonstrated the range of WaPOR variables available and how to load them in the DE Africa Sandbox environment. Subsequent notebooks will dive deeper into analysing WaPOR data alongside DE Africa data.
Additional information
License: The code in this notebook is licensed under the Apache License, Version 2.0. Digital Earth Africa data is licensed under the Creative Commons by Attribution 4.0 license.
Contact: If you need assistance, please post a question on the Open Data Cube Slack channel or on the GIS Stack Exchange using the open-data-cube tag (you can view previously asked questions here). If you would like to report an issue with this notebook, you can file one on
Github.
Compatible datacube version:
[ ]:
print(datacube.__version__)
Last Tested:
[ ]:
from datetime import datetime
datetime.today().strftime('%Y-%m-%d')