Package eoreader

Source Code: https://github.com/sertit/eoreader

EOReader

This project is a multi-satellite reader allowing you to open optical and SAR data.

It also implements two additional features, non depending on the sensor:

It allows you to focus on the science instead of worrying how to access to the data, especially if you have to work with multiple sensors !

Python Quickstart

The main features of EOReader are gathered hereunder:

>>> from eoreader.reader import Reader
>>> from eoreader.bands.alias import *

>>> # Your variables
>>> path = r"path/to/your/satellite/product"  # Optical in this example
>>> # WARNING: you can leave the output_path empty, but EOReader will create a temporary output directory
>>> # and you won't be able to retrieve what's has been written on disk
>>> output = r"path/to/your/output"

>>> # Create the reader object and open satellite data
>>> eoreader = Reader()
>>> prod = eoreader.open(path, output_path=output)  # The Reader will recognize the satellite type from its name

>>> # Get the footprint of the product (usable data) and its extent (envelope of the tile)
>>> footprint = prod.footprint
>>> extent = prod.extent

>>> # Load some bands and index: they will all share the same metadata
>>> bands = prod.load([NDVI, GREEN, HILLSHADE, CLOUDS])  # Resolution not specified: use product resolution
>>> ndvi = bands[NDVI]
>>> green = bands[GREEN]
>>> hillshade = bands[HILLSHADE]
>>> clouds = bands[CLOUDS]
>>> # NOTE: every array that comes out `load` are collocated, which isn't the case if you load arrays separately
>>> # (important for DEM data as they may have different grids)

>>> # Create a stack with some other bands
>>> stack = prod.stack([NDVI, MNDWI, GREEN, SLOPE, CIRRUS])  # Resolution not specified: use product resolution

>>> # Read Metadata
>>> mtd, namespace = prod.read_mtd()

Warning

  • Sentinel-3 and SAR products need SNAP gpt to be geocoded. Ensure that you have the folder containing your gpt.exe in your PATH.

Installation

pip install eoreader

.. WARNING :: EOReader depends mainly on geopandas and rasterio. (with GDAL installation issues on Windows, so please install them from wheels that you can find here).


Main features

Read

The reader singleton is your unique entry. It will create for you the product object corresponding to your satellite data.

>>> import os
>>> from eoreader.reader import Reader

>>> # Path to your satellite data, ie. Sentinel-2
>>> path = r'S2A_MSIL1C_20200824T110631_N0209_R137_T30TTK_20200824T150432.zip'  # You can work with the archive for S2 data

>>> # Path to your output directory (if not set, it will work in a temp directory)
>>> output = os.path.abspath('.')

>>> # Create the reader singleton
>>> eoreader = Reader()
>>> prod = eoreader.open(path, output_path=output)

>>> # NOTE: you can set the output directory after the creation, that allows you to use the product condensed name
>>> prod.output = os.path.join(output, prod.condensed_name)  # It will automatically create it if needed

From there you have access to a lot of information on your product:

>>> # Product CRS (always in UTM)
>>> prod.crs
CRS.from_epsg(32630)

>>> # Full extent of the bands as a geopandas GeoDataFrame (always in UTM)
>>> prod.extent()
                                            geometry
0   POLYGON((309780.000 4390200.000, 309780.000 4...

>>> # Footprint: extent of the useful pixels (minus nodata) as a geopandas GeoDataFrame (always in UTM)
>>> prod.footprint()
   index                                           geometry
0      0  POLYGON ((199980.000 4500000.000, 199980.000 4...

>>> # Default resolution (20m for S2)
>>> prod.resolution
20.

>>> # Acquisition date and datetime
>>> prod.date
datetime.date(2020, 8, 24)
>>> prod.datetime
datetime.datetime(2020, 8, 24, 11, 6, 31)

>>> # Access the raw metadata as an lxml.etree._Element:
>>> prod.read_mtd()

See the difference between footprint and extent hereunder:

Without nodata With nodata
without_nodata with_nodata

Load

Product.load() is the function for accessing to product-related bands. It can load satellite bands, index, DEM bands and cloud bands according to this workflow: load_workflow

>>> from eoreader.reader import Reader
>>> from eoreader.bands.alias import *

>>> path = r"S2A_MSIL1C_20200824T110631_N0209_R137_T30TTK_20200824T150432.zip"
>>> prod = Reader().open(path)

>>> # Get the wanted bands and check if the product can produce them
>>> band_list = [GREEN, NDVI, TIR_1, SHADOWS, HILLSHADE]
>>> ok_bands = [band for band in band_list if prod.has_band(band)]
[GREEN, NDVI, HILLSHADE]
>>> # Sentinel-2 cannot produce satellite band TIR_1 and cloud band SHADOWS

>>> # Load bands
>>> bands = prod.load(ok_bands)
>>> bands
"""
{<function NDVI at 0x000001C47FF05E18>: <xarray.DataArray 'NDVI' (band: 1, y: 5490, x: 5490)>
array([[[0.94786006, 0.92717856, 0.92240528, ..., 1.73572724,
         1.55314477, 1.63242706],
        [1.04147187, 0.93668633, 0.91499688, ..., 1.59941784,
         1.52895995, 1.51386761],
        [2.86996677, 1.69360304, 1.2413562 , ..., 1.61172353,
         1.55742907, 1.50568275],
        ...,
        [1.45807257, 1.61071344, 1.64620751, ..., 1.25498441,
         1.42998927, 1.70447076],
        [1.57802352, 1.77086658, 1.69901482, ..., 1.19999853,
         1.27813254, 1.52287237],
        [1.63569594, 1.66751277, 1.63474646, ..., 1.27617084,
         1.22456033, 1.27022877]]])
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * band         (band) int32 1
    spatial_ref  int32 0,
<OpticalBandNames.GREEN: 'GREEN'>: <xarray.DataArray 'T30TTK_20200824T110631_B03' (band: 1, y: 5490, x: 5490)>
array([[[0.06146327, 0.06141786, 0.06100179, ..., 0.11880179,
         0.12087143, 0.11468571],
        [0.06123214, 0.06071094, 0.06029063, ..., 0.11465781,
         0.11858906, 0.11703929],
        [0.06494643, 0.06226562, 0.06169219, ..., 0.11174062,
         0.11434844, 0.11491964],
        ...,
        [0.1478125 , 0.13953906, 0.13751719, ..., 0.15949688,
         0.14200781, 0.12982321],
        [0.14091429, 0.12959531, 0.13144844, ..., 0.17246719,
         0.156175  , 0.13453036],
        [0.13521429, 0.13274286, 0.13084821, ..., 0.16064821,
         0.16847143, 0.16009592]]])
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * band         (band) int32 1
    spatial_ref  int32 0,
<DemBandNames.HILLSHADE: 'HILLSHADE'>: <xarray.DataArray '20200824T110631_S2_T30TTK_L1C_150432_HILLSHADE' (band: 1, y: 5490, x: 5490)>
array([[[220., 221., 221., ..., 210., 210., 210.],
        [222., 222., 221., ..., 210., 210., 210.],
        [221., 221., 220., ..., 210., 210., 210.],
        ...,
        [215., 214., 212., ..., 207., 207., 207.],
        [214., 212., 211., ..., 206., 205., 205.],
        [213., 211., 209., ..., 205., 204., 205.]]])
Coordinates:
  * band         (band) int32 1
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
    spatial_ref  int32 0
Attributes:
    grid_mapping:    spatial_ref
    original_dtype:  uint8}
"""

Note

Index and bands are opened as xarrays with rioxarray, in float with the nodata set to np.nan. The nodata written back on disk is -9999 by convention (for now the rasters will be written in float)

Stack

Product.stack() is the function stacking all possible bands. It is based on the load function and then just stacks the bands and write it on disk if needed.

>>> # Create a stack with the previous OK bands
>>> stack = prod.stack(ok_bands, resolution=300., stack_path=os.path.join(prod.output, "stack.tif")
"""
<xarray.DataArray 'NDVI_GREEN_HILLSHADE' (z: 3, y: 5490, x: 5490)>
array([[[9.47860062e-01, 9.27178562e-01, 9.22405303e-01, ...,
         1.73572719e+00, 1.55314481e+00, 1.63242710e+00],
        [1.04147184e+00, 9.36686337e-01, 9.14996862e-01, ...,
         1.59941781e+00, 1.52895999e+00, 1.51386762e+00],
        [2.86996675e+00, 1.69360304e+00, 1.24135625e+00, ...,
         1.61172354e+00, 1.55742908e+00, 1.50568271e+00],
        ...,
        [1.45807254e+00, 1.61071348e+00, 1.64620745e+00, ...,
         1.25498438e+00, 1.42998922e+00, 1.70447075e+00],
        [1.57802355e+00, 1.77086663e+00, 1.69901478e+00, ...,
         1.19999850e+00, 1.27813256e+00, 1.52287233e+00],
        [1.63569593e+00, 1.66751277e+00, 1.63474643e+00, ...,
         1.27617085e+00, 1.22456038e+00, 1.27022874e+00]],
       [[6.14632666e-02, 6.14178553e-02, 6.10017851e-02, ...,
         1.18801787e-01, 1.20871432e-01, 1.14685714e-01],
        [6.12321422e-02, 6.07109368e-02, 6.02906235e-02, ...,
         1.14657812e-01, 1.18589066e-01, 1.17039286e-01],
        [6.49464279e-02, 6.22656234e-02, 6.16921857e-02, ...,
         1.11740626e-01, 1.14348434e-01, 1.14919640e-01],
        [1.47812501e-01, 1.39539063e-01, 1.37517184e-01, ...,
         1.59496874e-01, 1.42007813e-01, 1.29823208e-01],
        [1.40914291e-01, 1.29595309e-01, 1.31448433e-01, ...,
         1.72467187e-01, 1.56175002e-01, 1.34530351e-01],
        [1.35214284e-01, 1.32742852e-01, 1.30848214e-01, ...,
         1.60648212e-01, 1.68471426e-01, 1.60095915e-01]],
       [[2.20000000e+02, 2.21000000e+02, 2.21000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        [2.22000000e+02, 2.22000000e+02, 2.21000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        [2.21000000e+02, 2.21000000e+02, 2.20000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        ...,
        [2.15000000e+02, 2.14000000e+02, 2.12000000e+02, ...,
         2.07000000e+02, 2.07000000e+02, 2.07000000e+02],
        [2.14000000e+02, 2.12000000e+02, 2.11000000e+02, ...,
         2.06000000e+02, 2.05000000e+02, 2.05000000e+02],
        [2.13000000e+02, 2.11000000e+02, 2.09000000e+02, ...,
         2.05000000e+02, 2.04000000e+02, 2.05000000e+02]]], dtype=float32)
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
    spatial_ref  int32 0
  * z            (z) MultiIndex
  - variable     (z) object 'NDVI' 'GREEN' 'HILLSHADE'
  - band         (z) int64 1 1 1
Attributes:
    long_name:  ['NDVI', 'GREEN', 'HILLSHADE']
"""

Optical data

Implemented optical satellites

Satellites Class Product Types Use archive Default Resolution
Sentinel-2 S2Product L1C & L2A Yes 20m
Sentinel-2 Theia S2TheiaProduct L2A Yes 20m
Sentinel-3 SLSTR S3Product RBT No 300m
Sentinel-3 OLCI S3Product EFR No 500m
Landsat-8 OLCI L8Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-7 ETM L7Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-5 TM L5Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-4 TM L4Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-5 MSS L5Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-4 MSS L4Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-3 MSS L3Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-2 MSS L2Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-1 MSS L1Product Level 1 Collection 1: No, Collection 2: Yes 60m

Satellites products that cannot be used as archived have to be extracted before use.

Optical bands

The following bands are available in EOReader, but may not be available for all sensors.

Satellite bands

These bands are mainly based on Sentinel-2 bands with some additions:

  • CA: Coastal Aerosol
  • BLUE
  • GREEN
  • RED
  • VRE_1: Vegetation Red Edge 1
  • VRE_2: Vegetation Red Edge 2
  • VRE_3: Vegetation Red Edge 3
  • NIR: Near Infrared
  • NARROW_NIR: Narrow Near Infrared (band 8A for Sentinel-2)
  • WP: Water vapour
  • SWIR_CIRRUS
  • SWIR_1
  • SWIR_2
  • PAN: Panchromatic
  • TIR_1: Thermal Infrared 1
  • TIR_2: Thermal Infrared 2

See here for more information.

Index

  • AFRI_1_6
  • AFRI_2_1
  • AWEInsh
  • AWEIsh
  • BAI
  • BSI
  • CIG
  • DSWI
  • GLI
  • GNDVI
  • MNDWI
  • NBR
  • NDGRI
  • NDMI
  • NDRE2
  • NDRE3
  • NDVI
  • NDWI
  • RDI
  • RGI
  • RI
  • SRSWIR
  • TCBRI
  • TCGRE
  • TCWET
  • WI

See here for more information.

Cloud bands

Maximum 5 cloud bands are available, according to the files provided in the data. All the bands are rasterized and orthorectified if needed (for Sentinel-2 or 3 data for example), ready to be stacked.

  • RAW_CLOUDS: Raw Cloud file as provided (the only changes are the orthorectification and rasterization). Can provide other flags, or cloud probability.
  • CLOUDS: Cloud presence (1) or absence (0).
  • CIRRUS: Cirrus presence (1) or absence (0).
  • SHADOWS: Shadows presence (1) or absence (0).
  • ALL_CLOUDS: Cloud OR Cirrus OR Shadows presence (1) or absence (0). Do not take into account missing bands ( ie. for Landsat MSS sensors, ALL_CLOUDS == CLOUDS)

See here for more information.

DEM bands

These bands need a valid worldwide DEM path positioned thanks to the environment variable EOREADER_SAR_DEFAULT_RES

  • DEM
  • SLOPE
  • HILLSHADE

See here for more information.


SAR data

Implemented SAR satellites

Satellites Class Product Types Use archive
Sentinel-1 S1Product SLC & GRD Yes
COSMO-Skymed CskProduct DGM & SCS, (others should also be OK) No
TerraSAR-X TsxProduct MGD (SSC should be OK) No
RADARSAT-2 Rs2Product SGF (SLC should be OK) Yes

Warning

Satellites products that cannot be used as archived have to be extracted before use.

SAR Bands

According to what contains the products, allowed SAR bands are:

You also can load despeckled bands:

DEM bands

These bands need a valid worldwide DEM path positioned thanks to the environment variable EOREADER_SAR_DEFAULT_RES

  • DEM
  • SLOPE

See here for more information.

Expand source code
"""
**Source Code**: https://github.com/sertit/eoreader

.. include:: ../documentation/index.md

_____

.. include:: ../documentation/main_features.md

_____

.. include:: ../documentation/optical.md

_____

.. include:: ../documentation/sar.md

"""

__pdoc__ = {
    "eoreader.data": False,
}

__version__ = "0.2.1"

Sub-modules

eoreader.bands

Band module containing: …

eoreader.env_vars

Environment variables that can change the processes

eoreader.examples

EOReader examples …

eoreader.exceptions

EOReader exceptions

eoreader.products

SAR and Optical products …

eoreader.reader

Product Factory, class creating products according to their names

eoreader.utils

Utils: mostly getting directories relative to the project