iSDA soil data¶

Products used: iSDA Soil

Keywords data used; iSDA soil, soil

Background¶

iSDAsoil is an open access soils data resource for the African continent. Soils data can be useful for a range of applications including land suitability mapping for agriculture, soil amelioration planning (e.g. liming to address acidic soils or fertiliser to address fertility constraints), and for construction/ infrastructure planning.

Description¶

This notebook demonstrates how to integrate this data with DE Africa products and workflows. For technical information on iSDAsoil, see Miller et al. 2021. Some of the code in this notebook has been adapted from the iSDA-Africa GitHub.

Load iSDA data
Visualise iSDA against DE Africa products

Getting started¶

To run this analysis, run all the cells in the notebook, starting with the “Load packages” cell.

Load packages¶

Import Python packages that are used for the analysis.

Note that we are loading a module called load_isda() for this notebook which takes a little while to load.

[1]:

%matplotlib inline

import datacube
import numpy as np
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import rasterio as rio
from pyproj import Transformer
import matplotlib.pyplot as plt
import os
import numpy as np

from urllib.parse import urlparse
import boto3
from pystac import stac_io, Catalog
from deafrica_tools.plotting import display_map, rgb
from deafrica_tools.load_isda import load_isda

/usr/local/lib/python3.8/dist-packages/geopandas/_compat.py:112: UserWarning: The Shapely GEOS version (3.8.0-CAPI-1.13.1 ) is incompatible with the GEOS version PyGEOS was compiled with (3.10.3-CAPI-1.16.1). Conversions between both will be slow.
  warnings.warn(

iSDA variables¶

iSDA offers numerous soil variables shown in the table below. Many of the variables have four layers: mean and standard deviation at 0-20cm and 20-5cm depth. We will see through the notebook that we can bring in each variable with the Access Names e.g. ‘bedrock_depth’.

Variable	Access Name	Layers
Aluminium, extractable	aluminium_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Depth to bedrock	bedrock_depth	0-200cm depth, predicted mean and standard deviation
Bulk density, <2mm fraction	bulk_density	0-20cm and 20-50cm, predicted mean and standard deviation
Calcium, extractable	calcium_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Carbon, organic	carbon_organic	0-20cm and 20-50cm, predicted mean and standard deviation
Carbon, total	carbon_total	0-20cm and 20-50cm, predicted mean and standard deviation
Effective Cation Exchange Capacity	cation_exchange_capacity	0-20cm and 20-50cm, predicted mean and standard deviation
Clay content	clay_content	0-20 cm and 20-50 cm, predicted mean and standard deviation
Fertility Capability Classification	fcc	Single classification layer
Iron, extractable	iron_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Magnesium, extractable	magnesium_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Nitrogen, total	nitrogen_total	0-20cm and 20-50cm, predicted mean and standard deviation
pH	ph	0-20cm and 20-50cm, predicted mean and standard deviation
Phosphorus, extractable	phosphorous_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Potassium, extractable	potassium_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
Sand content	sand_content	0-20cm and 20-50cm, predicted mean and standard deviation
Silt content	silt_content	0-20cm and 20-50cm, predicted mean and standard deviation
Stone content	stone_content	Coarse fragments predicted at 0-20cm depth
Sulphur, extractable	sulphur_extractable	0-20cm and 20-50cm, predicted mean and standard deviation
USDA Texture Class	texture_class	0-20cm and 20-50cm depth, predicted mean
Zinc, extractable	zinc_extractable	0-20cm and 20-50cm, predicted mean and standard deviation

Define parameters¶

Let’s define some parameters for loading the Fertility Capability Classification in a coastal area near Cape Town, South Africa. We can see in the table above that we can access this variable with `fcc’.

[2]:

var = 'fcc'

lat = -32.9
lon = 18.7
buffer = 0.5

display_map((lon - buffer, lon + buffer), (lat - buffer, lat + buffer))

[2]:

Make this Notebook Trusted to load map: File -> Trust Notebook

Link to iSDA metadata¶

There are a few complexities associated with the iSDA data, such as back-transformations / conversions that we will see later in the notebook. We need to link to metadata so we can explore these.

The cell below provides a link to the necessary metadata.

[3]:

#this function allows us to directly query the data on s3
def my_read_method(uri):
    parsed = urlparse(uri)
    if parsed.scheme == 's3':
        bucket = parsed.netloc
        key = parsed.path[1:]
        s3 = boto3.resource('s3')
        obj = s3.Object(bucket, key)
        return obj.get()['Body'].read().decode('utf-8')
    else:
        return stac_io.default_read_text_method(uri)

stac_io.read_text_method = my_read_method

catalog = Catalog.from_file("https://isdasoil.s3.amazonaws.com/catalog.json")

assets = {}

for root, catalogs, items in catalog.walk():
    for item in items:
        str(f"Type: {item.get_parent().title}")
        # save all items to a dictionary as we go along
        assets[item.id] = item
        for asset in item.assets.values():
            if asset.roles == ['data']:
                str(f"Title: {asset.title}")
                str(f"Description: {asset.description}")
                str(f"URL: {asset.href}")
                str("------------")

Transformation!¶

Importantly, much of the iSDA data needs to be transformed before it’s applied. We can use the object we stored called assets to get conversion metadata associated with each variable. This is performed for our selected variable below.

We can see that the fcc variable returns a conversion value of %3000.

[5]:

conversion = assets[var].extra_fields["back-transformation"]
conversion

[5]:

'%3000'

Below, the transformation is made and a new band added to the dataset. The values are converted to integers for easier interpretation and plotting.

[6]:

ds = ds.assign(fcc = (ds["band_data"]%3000).fillna(2535).astype('int16'))

Labels can also be drawn from the metadata. Below, classes of fertility are mapped to their respective integers for better plotting outputs.

[7]:

fcc_labels = pd.read_csv(assets['fcc'].assets["metadata"].href, index_col=0)

mappings = {val:fcc_labels.loc[int(val),"Description"] for val in np.unique(ds.fcc.astype('int16'))}

Plot Fertility Capability Classification¶

We can now plot the fcc data with labels.

[8]:

plt.figure(figsize=(12,10), dpi= 100)
im = ds.fcc.plot(cmap="magma", add_colorbar=False)
colors = [ im.cmap(im.norm(value)) for value in np.unique(ds.fcc)]
patches = [mpatches.Patch(color=colors[idx], label=str(mappings[val])) for idx, val in enumerate(np.unique(ds.fcc))]
plt.legend(handles=patches, bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0. )
plt.title("Fertility Capability Classification with labels")
plt.show()

../../../_images/sandbox_notebooks_Datasets_iSDAsoil_23_0.png

Transformation!¶

Check if and how the data needs to be transformed. In this case, there is no transformation and the data can be interpreted as %, so we can proceed.

[11]:

conversion = assets[var].extra_fields["back-transformation"]
conversion

[12]:

fig, ax = plt.subplots(2, 2, figsize=(22, 16), sharey=True)
ds[list(ds.keys())[0]].plot(ax=ax[0, 0])
ds[list(ds.keys())[1]].plot(ax=ax[0, 1])
ds[list(ds.keys())[2]].plot(ax=ax[1, 0])
ds[list(ds.keys())[3]].plot(ax=ax[1, 1])
ax[0, 0].set_title(list(ds.keys())[0])
ax[0, 1].set_title(list(ds.keys())[1])
ax[1, 0].set_title(list(ds.keys())[2])
ax[1, 1].set_title(list(ds.keys())[3])
plt.tight_layout();

../../../_images/sandbox_notebooks_Datasets_iSDAsoil_31_0.png

Conclusions¶

This notebook gave an introduction into loading iSDA soil data into the Digital Earth Africa sandbox.

Are any patterns observable between the Okavango Delta formation, clay content, and soil pH? How does the Fertility Capability Classification map align with the true colour image?

This notebook is intended to be a starting point. Users may like to try loading data for other areas or other soil variables.

Additional information¶

License: The code in this notebook is licensed under the Apache License, Version 2.0. Digital Earth Africa data is licensed under the Creative Commons by Attribution 4.0 license.

Contact: If you need assistance, please post a question on the Open Data Cube Slack channel or on the GIS Stack Exchange using the open-data-cube tag (you can view previously asked questions here). If you would like to report an issue with this notebook, you can file one on Github.

Compatible datacube version:

[18]:

print(datacube.__version__)

1.8.8

Last Tested:

[19]:

from datetime import datetime
datetime.today().strftime('%Y-%m-%d')

[19]:

'2023-01-30'

iSDA soil data¶

Background¶

Description¶

Getting started¶

Load packages¶

iSDA variables¶

Define parameters¶

Link to iSDA metadata¶

Load Fertility Capability Classification¶

Transformation!¶

Plot Fertility Capability Classification¶

Clay in Okavango Delta¶

New parameters¶

Transformation!¶

Soil pH¶

Transformation!¶

Conclusions¶

Additional information¶