Package eoreader

Source Code: https://github.com/sertit/eoreader

eoreader_logo EOReader

EOReader is a multi-satellite reader allowing you to open optical and SAR data.

Optical SAR
Sensors + Sentinel-2 & Theia
+ Sentinel-3 OLCI & SLSTR
+ Landsats 1 - 8
+ Sentinel-1
+ COSMO-Skymed
+ TerraSAR-X
+ RADARSAT-2

It also implements additional sensor-agnostic features:

EOReader works with xarrays.DataArray and geopandas.GeoDataFrames

Python Quickstart

The main features of EOReader are gathered hereunder:

>>> from eoreader.reader import Reader
>>> from eoreader.bands.alias import *

>>> # Your variables
>>> path = r"path/to/your/satellite/product"  # Optical in this example

>>> # Create the reader object and open satellite data
>>> eoreader = Reader()
>>> prod = eoreader.open(path)  # The Reader will recognize the satellite type from its name

>>> # Get the footprint of the product (usable data) and its extent (envelope of the tile)
>>> footprint = prod.footprint
>>> extent = prod.extent

>>> # Load some bands and index: they will all share the same metadata
>>> bands = prod.load([NDVI, GREEN, HILLSHADE, CLOUDS]

>>> # Create a stack with some other bands
>>> stack = prod.stack([NDVI, MNDWI, GREEN, SLOPE, CIRRUS])

>>> # Read Metadata
>>> mtd, namespace = prod.read_mtd()

Sentinel-3 and SAR products need SNAP gpt to be geocoded. Ensure that you have the folder containing your gpt.exe in your PATH.

Documentation

The API documentation can be found here.

Examples

Available notebooks provided as examples:

Installation

pip install eoreader

EOReader depends mainly on geopandas and rasterio. (with GDAL installation issues on Windows, so please install them from wheels that you can find here).


Main features

These features can be seen in this example.

Read

The reader singleton is your unique entry. It will create for you the product object corresponding to your satellite data.

>>> import os
>>> from eoreader.reader import Reader

>>> # Path to your satellite data, ie. Sentinel-2
>>> path = r'S2A_MSIL1C_20200824T110631_N0209_R137_T30TTK_20200824T150432.zip'  # You can work with the archive for S2 data

>>> # Path to your output directory (if not set, it will work in a temp directory)
>>> output = os.path.abspath('.')

>>> # Create the reader singleton
>>> eoreader = Reader()
>>> prod = eoreader.open(path, output_path=output)

>>> # NOTE: you can set the output directory after the creation, that allows you to use the product condensed name
>>> prod.output = os.path.join(output, prod.condensed_name)  # It will automatically create it if needed

From there you have access to a lot of information on your product:

>>> # Product CRS (always in UTM)
>>> prod.crs
CRS.from_epsg(32630)

>>> # Full extent of the bands as a geopandas GeoDataFrame (always in UTM)
>>> prod.extent()
                                            geometry
0   POLYGON((309780.000 4390200.000, 309780.000 4...

>>> # Footprint: extent of the useful pixels (minus nodata) as a geopandas GeoDataFrame (always in UTM)
>>> prod.footprint()
   index                                           geometry
0      0  POLYGON ((199980.000 4500000.000, 199980.000 4...

>>> # Default resolution (20m for S2)
>>> prod.resolution
20.

>>> # Acquisition date and datetime
>>> prod.date
datetime.date(2020, 8, 24)
>>> prod.datetime
datetime.datetime(2020, 8, 24, 11, 6, 31)

>>> # Access the raw metadata as an lxml.etree._Element:
>>> prod.read_mtd()

Load

Product.load() is the function for accessing to product-related bands. It can load satellite bands, index, DEM bands and cloud bands according to this workflow: load_workflow

>>> import os
>>> from eoreader.reader import Reader
>>> from eoreader.bands.alias import *

>>> path = r"S2A_MSIL1C_20200824T110631_N0209_R137_T30TTK_20200824T150432.zip"
>>> output = os.path.abspath("./output")
>>>  # WARNING: you can leave the output_path empty, but EOReader will create a temporary output directory
>>>  # and you won't be able to retrieve what's has been written on disk
>>> prod = Reader().open(path, output_path=output)

>>> # Get the wanted bands and check if the product can produce them
>>> band_list = [GREEN, NDVI, TIR_1, SHADOWS, HILLSHADE]
>>> ok_bands = [band for band in band_list if prod.has_band(band)]
[GREEN, NDVI, HILLSHADE]
>>> # Sentinel-2 cannot produce satellite band TIR_1 and cloud band SHADOWS

>>> # Load bands
>>> bands = prod.load(ok_bands)  # resolution not specified -> load at default resolution (20.0 m for S2 data)
>> >  # NOTE: every array that comes out `load` are collocated, which isn't the case if you load arrays separately
>> >  # (important for DEM data as they may have different grids)

>>> bands
"""
{<function NDVI at 0x000001C47FF05E18>: <xarray.DataArray 'NDVI' (band: 1, y: 5490, x: 5490)>
array([[[0.94786006, 0.92717856, 0.92240528, ..., 1.73572724,
         1.55314477, 1.63242706],
        [1.04147187, 0.93668633, 0.91499688, ..., 1.59941784,
         1.52895995, 1.51386761],
        [2.86996677, 1.69360304, 1.2413562 , ..., 1.61172353,
         1.55742907, 1.50568275],
        ...,
        [1.45807257, 1.61071344, 1.64620751, ..., 1.25498441,
         1.42998927, 1.70447076],
        [1.57802352, 1.77086658, 1.69901482, ..., 1.19999853,
         1.27813254, 1.52287237],
        [1.63569594, 1.66751277, 1.63474646, ..., 1.27617084,
         1.22456033, 1.27022877]]])
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * band         (band) int32 1
    spatial_ref  int32 0,
<OpticalBandNames.GREEN: 'GREEN'>: <xarray.DataArray 'T30TTK_20200824T110631_B03' (band: 1, y: 5490, x: 5490)>
array([[[0.06146327, 0.06141786, 0.06100179, ..., 0.11880179,
         0.12087143, 0.11468571],
        [0.06123214, 0.06071094, 0.06029063, ..., 0.11465781,
         0.11858906, 0.11703929],
        [0.06494643, 0.06226562, 0.06169219, ..., 0.11174062,
         0.11434844, 0.11491964],
        ...,
        [0.1478125 , 0.13953906, 0.13751719, ..., 0.15949688,
         0.14200781, 0.12982321],
        [0.14091429, 0.12959531, 0.13144844, ..., 0.17246719,
         0.156175  , 0.13453036],
        [0.13521429, 0.13274286, 0.13084821, ..., 0.16064821,
         0.16847143, 0.16009592]]])
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * band         (band) int32 1
    spatial_ref  int32 0,
<DemBandNames.HILLSHADE: 'HILLSHADE'>: <xarray.DataArray '20200824T110631_S2_T30TTK_L1C_150432_HILLSHADE' (band: 1, y: 5490, x: 5490)>
array([[[220., 221., 221., ..., 210., 210., 210.],
        [222., 222., 221., ..., 210., 210., 210.],
        [221., 221., 220., ..., 210., 210., 210.],
        ...,
        [215., 214., 212., ..., 207., 207., 207.],
        [214., 212., 211., ..., 206., 205., 205.],
        [213., 211., 209., ..., 205., 204., 205.]]])
Coordinates:
  * band         (band) int32 1
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
    spatial_ref  int32 0
Attributes:
    grid_mapping:    spatial_ref
    original_dtype:  uint8}
"""

Note

Index and bands are opened as xarrays with rioxarray, in float with the nodata set to np.nan. The nodata written back on disk is -9999 by convention (for now the rasters will be written in float)

Stack

Product.stack() is the function stacking all possible bands. It is based on the load function and then just stacks the bands and write it on disk if needed.

>>> # Create a stack with the previous OK bands
>>> stack = prod.stack(ok_bands, resolution=300., stack_path=os.path.join(prod.output, "stack.tif")
"""
<xarray.DataArray 'NDVI_GREEN_HILLSHADE' (z: 3, y: 5490, x: 5490)>
array([[[9.47860062e-01, 9.27178562e-01, 9.22405303e-01, ...,
         1.73572719e+00, 1.55314481e+00, 1.63242710e+00],
        [1.04147184e+00, 9.36686337e-01, 9.14996862e-01, ...,
         1.59941781e+00, 1.52895999e+00, 1.51386762e+00],
        [2.86996675e+00, 1.69360304e+00, 1.24135625e+00, ...,
         1.61172354e+00, 1.55742908e+00, 1.50568271e+00],
        ...,
        [1.45807254e+00, 1.61071348e+00, 1.64620745e+00, ...,
         1.25498438e+00, 1.42998922e+00, 1.70447075e+00],
        [1.57802355e+00, 1.77086663e+00, 1.69901478e+00, ...,
         1.19999850e+00, 1.27813256e+00, 1.52287233e+00],
        [1.63569593e+00, 1.66751277e+00, 1.63474643e+00, ...,
         1.27617085e+00, 1.22456038e+00, 1.27022874e+00]],
       [[6.14632666e-02, 6.14178553e-02, 6.10017851e-02, ...,
         1.18801787e-01, 1.20871432e-01, 1.14685714e-01],
        [6.12321422e-02, 6.07109368e-02, 6.02906235e-02, ...,
         1.14657812e-01, 1.18589066e-01, 1.17039286e-01],
        [6.49464279e-02, 6.22656234e-02, 6.16921857e-02, ...,
         1.11740626e-01, 1.14348434e-01, 1.14919640e-01],
        [1.47812501e-01, 1.39539063e-01, 1.37517184e-01, ...,
         1.59496874e-01, 1.42007813e-01, 1.29823208e-01],
        [1.40914291e-01, 1.29595309e-01, 1.31448433e-01, ...,
         1.72467187e-01, 1.56175002e-01, 1.34530351e-01],
        [1.35214284e-01, 1.32742852e-01, 1.30848214e-01, ...,
         1.60648212e-01, 1.68471426e-01, 1.60095915e-01]],
       [[2.20000000e+02, 2.21000000e+02, 2.21000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        [2.22000000e+02, 2.22000000e+02, 2.21000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        [2.21000000e+02, 2.21000000e+02, 2.20000000e+02, ...,
         2.10000000e+02, 2.10000000e+02, 2.10000000e+02],
        ...,
        [2.15000000e+02, 2.14000000e+02, 2.12000000e+02, ...,
         2.07000000e+02, 2.07000000e+02, 2.07000000e+02],
        [2.14000000e+02, 2.12000000e+02, 2.11000000e+02, ...,
         2.06000000e+02, 2.05000000e+02, 2.05000000e+02],
        [2.13000000e+02, 2.11000000e+02, 2.09000000e+02, ...,
         2.05000000e+02, 2.04000000e+02, 2.05000000e+02]]], dtype=float32)
Coordinates:
  * x            (x) float64 2e+05 2e+05 2e+05 ... 3.097e+05 3.098e+05 3.098e+05
  * y            (y) float64 4.5e+06 4.5e+06 4.5e+06 ... 4.39e+06 4.39e+06
    spatial_ref  int32 0
  * z            (z) MultiIndex
  - variable     (z) object 'NDVI' 'GREEN' 'HILLSHADE'
  - band         (z) int64 1 1 1
Attributes:
    long_name:  ['NDVI', 'GREEN', 'HILLSHADE']
"""

Optical data

Implemented optical satellites

Satellites Class Product Types Use archive Default Resolution
Sentinel-2 S2Product L1C & L2A Yes 20m
Sentinel-2 Theia S2TheiaProduct L2A Yes 20m
Sentinel-3 SLSTR S3Product RBT No 300m
Sentinel-3 OLCI S3Product EFR No 500m
Landsat-8 OLCI L8Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-7 ETM L7Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-5 TM L5Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-4 TM L4Product Level 1 Collection 1: No, Collection 2: Yes 30m
Landsat-5 MSS L5Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-4 MSS L4Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-3 MSS L3Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-2 MSS L2Product Level 1 Collection 1: No, Collection 2: Yes 60m
Landsat-1 MSS L1Product Level 1 Collection 1: No, Collection 2: Yes 60m

Satellites products that cannot be used as archived have to be extracted before use.

Optical bands

The following bands are available in EOReader, but may not be available for all sensors.

Satellite bands

These bands are mainly based on Sentinel-2 bands with some additions:

  • CA: Coastal Aerosol
  • BLUE
  • GREEN
  • RED
  • VRE_1: Vegetation Red Edge 1
  • VRE_2: Vegetation Red Edge 2
  • VRE_3: Vegetation Red Edge 3
  • NIR: Near Infrared
  • NARROW_NIR: Narrow Near Infrared (band 8A for Sentinel-2)
  • WP: Water vapour
  • SWIR_CIRRUS
  • SWIR_1
  • SWIR_2
  • PAN: Panchromatic
  • TIR_1: Thermal Infrared 1
  • TIR_2: Thermal Infrared 2

See here for more information.

Index

  • AFRI_1_6
  • AFRI_2_1
  • AWEInsh
  • AWEIsh
  • BAI
  • BSI
  • CIG
  • DSWI
  • GLI
  • GNDVI
  • MNDWI
  • NBR
  • NDGRI
  • NDMI
  • NDRE2
  • NDRE3
  • NDVI
  • NDWI
  • RDI
  • RGI
  • RI
  • SRSWIR
  • TCBRI
  • TCGRE
  • TCWET
  • WI

See here for more information.

Cloud bands

Maximum 5 cloud bands are available, according to the files provided in the data. All the bands are rasterized and orthorectified if needed (for Sentinel-2 or 3 data for example), ready to be stacked.

  • RAW_CLOUDS: Raw Cloud file as provided (the only changes are the orthorectification and rasterization). Can provide other flags, or cloud probability.
  • CLOUDS: Cloud presence (1) or absence (0).
  • CIRRUS: Cirrus presence (1) or absence (0).
  • SHADOWS: Shadows presence (1) or absence (0).
  • ALL_CLOUDS: Cloud OR Cirrus OR Shadows presence (1) or absence (0). Do not take into account missing bands ( ie. for Landsat MSS sensors, ALL_CLOUDS == CLOUDS)

See here for more information.

DEM bands

These bands need a valid worldwide DEM path positioned thanks to the environment variable EOREADER_SAR_DEFAULT_RES

  • DEM
  • SLOPE
  • HILLSHADE

See here for more information.


SAR data

Implemented SAR satellites

Satellites Class Product Types Use archive
Sentinel-1 S1Product SLC & GRD Yes
COSMO-Skymed CskProduct DGM & SCS, (others should also be OK) No
TerraSAR-X TsxProduct MGD (SSC should be OK) No
RADARSAT-2 Rs2Product SGF (SLC should be OK) Yes

Warning

Satellites products that cannot be used as archived have to be extracted before use.

SAR Bands

According to what contains the products, allowed SAR bands are:

You also can load despeckled bands:

DEM bands

These bands need a valid worldwide DEM path positioned thanks to the environment variable EOREADER_SAR_DEFAULT_RES

  • DEM
  • SLOPE

See here for more information.

Expand source code
# -*- coding: utf-8 -*-
# Copyright 2021, SERTIT-ICube - France, https://sertit.unistra.fr/
# This file is part of eoreader project
#     https://github.com/sertit/eoreader
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
**Source Code**: https://github.com/sertit/eoreader

.. include:: ../documentation/index.md

_____

.. include:: ../documentation/main_features.md

_____

.. include:: ../documentation/optical.md

_____

.. include:: ../documentation/sar.md

"""

__pdoc__ = {
    "eoreader.data": False,
}

__version__ = "0.3.0"

Sub-modules

eoreader.bands

Band module containing: …

eoreader.env_vars

Environment variables that can change the processes

eoreader.exceptions

EOReader exceptions

eoreader.products

SAR and Optical products …

eoreader.reader

Product Factory, class creating products according to their names

eoreader.utils

Utils: mostly getting directories relative to the project