pysiral.auxdata.ml

module for ingesting models from machine learning

Important Note:

All mdt data handlers must be subclasses of pysiral.auxdata.AuxdataBaseClass in order to work for the Level-2 Processor. If the auxiliary class is based on a static dataset, this should be parsed in __init__.

Please review the variables and properties in the parent class, as well as the correspodning config and support classes for grid track interpolation in the pysiral.auxdata module for additional guidance.

The only other hard requirements is the presence of on specific method in order to be a valid subclass of AuxdataBaseClass:

get_l2_track_vars(l2)

This method will be called during the Level-2 processor. The argument is the Level-2 data object and the purpose of the method is to compute the auxilary variable(s) and associated uncertainty. These variable need to be registered using the register_auxvar(id, name, value, uncertainty) method of the base class. All MDT subclasses need to register at minimum the following variable:

mean dynamic topography (relative to MSS):
id: mdt name: mean_dynamic_topography

e.g., this code line is mandatory for get_l2_track_vars (uncertainty can be None):

# Register Variables self.register_auxvar(“mdt”, “mean_dynamic_topography”, value, uncertainty)

Attributes

__author__

Classes

`RetrackerThresholdModel`	This class replaces the previous same-name version but utilizizes solely pytorch
`AutoEncoderERS2`
`ERS2_TestCandidate_006_FNN`
`ERS2_TestCandidate_005_FNN`
`ERS2_TestCandidate_004_FNN`
`ERS2_TestCandidate_003_FNN_TanHLeakyRelu`	Create Feed-Forward Neural Network architecture
`ERS2_TestCandidate_001_FNN_LeakyRelu`	Creates Feed-Forward Neural Network architecture using leaky_relu activation function
`ERS2_TestCandidate_002_FNN_TanH`	Creates Feed-Forward Neural Network architecture using tanh activation function
`ERS2_TestCandidate_003_LSTM_LeakyRelu`
`TorchFunctionalWaveformModelFNN`	Create Feed-Forward Neural Network architecture
`TorchFunctionalWaveformModelSNN`	Create Self-Normalizing Neural Network architecture
`TorchFunctionalWaveformModelCNN`
`RetrackerThresholdModelTorch`	Base class for all subtype auxdata base classes (e.g. SICBaseClass).

Module Contents

pysiral.auxdata.ml.__author__ = 'Stefan Hendricks <stefan.hendricks@awi.de>'

class pysiral.auxdata.ml.RetrackerThresholdModel(*args: Iterable[Any], **kwargs: Iterable[Any])

Bases: pysiral.auxdata.AuxdataBaseClass

This class replaces the previous same-name version but utilizizes solely pytorch as pysiral’s ML/AI framework for retracker-threshold models

valid_waveforms_idx = None

n_total_waveforms = None

model_file

model_filepath

waveform_for_prediction = None

parameters_for_prediction = None

model

receive_l1p_input(l1p: L1bdataNCFile) → None

Optional method to add l1p variables to this class before get_l2_track_vars() is called. This method here store the waveform power arrays, which are input to the model, normalizes the waveforms and converts to the necessary data type.

Parameters:: l1p
Returns:

get_l2_track_vars(l2: Level2Data) → None

[Mandatory class method] Add the model prediction for the tfmra retracker threshold to the Level-2 data object. The model evaluation depends solely on waveform power.

Parameters:: l2 – Level2Data container
Returns:: None
Raises:: None

class pysiral.auxdata.ml.AutoEncoderERS2(n_in: int = 35, n_bn: int = 3)

Bases: torch.nn.Module

n_in = 35

n_bn = 3

encoder

decoder

forward(x)

class pysiral.auxdata.ml.ERS2_TestCandidate_006_FNN(n_in: int = 35, n_par=8)

Bases: torch.nn.Module

n_in = 35

n_par = 8

model

forward(x, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_005_FNN(n_in: int = 35, n_par=6)

Bases: torch.nn.Module

n_in = 35

n_par = 6

model

forward(x, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_004_FNN(n_in: int = 35, n_par: int = 6)

Bases: torch.nn.Module

n_in = 35

n_par = 6

wfm

par

cmb

forward(wfm, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_003_FNN_TanHLeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 3)

Bases: torch.nn.Module

Create Feed-Forward Neural Network architecture

n_in = 5

n_out = 1

n_par = 3

fc1

fc2

fc3

fc4

fc5

fc1_par

fc2_par

fc3_par

fc4_par

fc5_par

fc7

fc8

fc9

forward(x, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_001_FNN_LeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 2)

Bases: torch.nn.Module

Creates Feed-Forward Neural Network architecture using leaky_relu activation function using two separate branches for 1) the input of the subset waveform power of a five waveform stack and 2) the parameter input of Epsilon and Max Power for the respective waveforms of the stack.

Both branches are then input and feed through a common two-layer network and output decimal optimal retracker thresholds through a sigmoid activation finish.

Desgined/Trained for ERS2 subwaveform input using 5 bins leading and 30 bins trailing the identified first-maximum index (fmi).

REQ: Required for RetrackerThresholdModel

n_in = 5

n_out = 1

n_par = 2

fc1

fc2

fc3

fc4

fc5

fc6

fc1_par

fc2_par

fc7

fc8

fc9

forward(x, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_002_FNN_TanH(n_in: int = 5, n_out: int = 1, n_par: int = 2)

Bases: torch.nn.Module

Creates Feed-Forward Neural Network architecture using tanh activation function using two separate branches for 1) the input of the subset waveform power of a five waveform stack and 2) the parameter input of Epsilon and Max Power for the respective waveforms of the stack.

Both branches are then input and feed through a common two-layer network and output decimal optimal retracker thresholds through a sigmoid activation finish.

Desgined/Trained for ERS2 subwaveform input using 5 bins leading and 30 bins trailing the identified first-maximum index (fmi).

REQ: Required for RetrackerThresholdModel

n_in = 5

n_out = 1

n_par = 2

fc1

fc2

fc3

fc4

fc5

fc6

fc1_par

fc2_par

fc7

fc8

fc9

forward(x, par)

class pysiral.auxdata.ml.ERS2_TestCandidate_003_LSTM_LeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 2)

Bases: torch.nn.Module

n_in = 5

n_out = 1

n_par = 2

lstm

fc1

fc2

fc1_par

fc2_par

fc3

fc4

forward(x, par)

class pysiral.auxdata.ml.TorchFunctionalWaveformModelFNN(fc1_input=45)

Bases: torch.nn.Module

Create Feed-Forward Neural Network architecture REQ: Required for RetrackerThresholdModel

Note: Legacy version of the current ENVISAT implementation until rework

fc1

fc2

fc3

fc4

fc5

fc6

forward(x)

class pysiral.auxdata.ml.TorchFunctionalWaveformModelSNN(fc1_input=45)

Bases: torch.nn.Module

Create Self-Normalizing Neural Network architecture REQ: Required for RetrackerThresholdModel

Note: Deprecated!

fc1

fc2

fc3

fc4

fc5

fc6

forward(x)

class pysiral.auxdata.ml.TorchFunctionalWaveformModelCNN

Bases: torch.nn.Module

” Create Convolutional Neural Network architecture REQ: Required for RetrackerThresholdModel

Note: Deprecated!

cv1_1

cv1_2

cv1_3

mp1

fc1

fc2

forward(x)

class pysiral.auxdata.ml.RetrackerThresholdModelTorch(*args: Iterable[Any], **kwargs: Iterable[Any])

Bases: pysiral.auxdata.AuxdataBaseClass

Base class for all subtype auxdata base classes (e.g. SICBaseClass). This class defines the mandatory set of methods and properties for all auxdata classes

model_file

model_filepath

waveform_for_prediction = None

model

receive_l1p_input(l1p: L1bdataNCFile) → None

Optional method to add l1p variables to this class before get_l2_track_vars() is called. This method here store the waveform power arrays, which are input to the model, normalizes the waveforms and converts to the necessary data type.

Parameters:: l1p
Returns:

get_l2_track_vars(l2: Level2Data) → None

[Mandatory class method] Add the model prediction for the tfmra retracker threshold to the Level-2 data object. The model evaluation depends solely on waveform power.

Parameters:: l2 – Level2Data container
Returns:: None
Raises:: None