pysiral.auxdata.ml ================== .. py:module:: pysiral.auxdata.ml .. autoapi-nested-parse:: module for ingesting models from machine learning Important Note: All mdt data handlers must be subclasses of pysiral.auxdata.AuxdataBaseClass in order to work for the Level-2 Processor. If the auxiliary class is based on a static dataset, this should be parsed in `__init__`. Please review the variables and properties in the parent class, as well as the correspodning config and support classes for grid track interpolation in the pysiral.auxdata module for additional guidance. The only other hard requirements is the presence of on specific method in order to be a valid subclass of AuxdataBaseClass: get_l2_track_vars(l2) This method will be called during the Level-2 processor. The argument is the Level-2 data object and the purpose of the method is to compute the auxilary variable(s) and associated uncertainty. These variable need to be registered using the `register_auxvar(id, name, value, uncertainty)` method of the base class. All MDT subclasses need to register at minimum the following variable: mean dynamic topography (relative to MSS): id: mdt name: mean_dynamic_topography e.g., this code line is mandatory for `get_l2_track_vars` (uncertainty can be None): # Register Variables self.register_auxvar("mdt", "mean_dynamic_topography", value, uncertainty) Attributes ---------- .. autoapisummary:: pysiral.auxdata.ml.__author__ Classes ------- .. autoapisummary:: pysiral.auxdata.ml.RetrackerThresholdModel pysiral.auxdata.ml.AutoEncoderERS2 pysiral.auxdata.ml.ERS2_TestCandidate_006_FNN pysiral.auxdata.ml.ERS2_TestCandidate_005_FNN pysiral.auxdata.ml.ERS2_TestCandidate_004_FNN pysiral.auxdata.ml.ERS2_TestCandidate_003_FNN_TanHLeakyRelu pysiral.auxdata.ml.ERS2_TestCandidate_001_FNN_LeakyRelu pysiral.auxdata.ml.ERS2_TestCandidate_002_FNN_TanH pysiral.auxdata.ml.ERS2_TestCandidate_003_LSTM_LeakyRelu pysiral.auxdata.ml.TorchFunctionalWaveformModelFNN pysiral.auxdata.ml.TorchFunctionalWaveformModelSNN pysiral.auxdata.ml.TorchFunctionalWaveformModelCNN pysiral.auxdata.ml.RetrackerThresholdModelTorch Module Contents --------------- .. py:data:: __author__ :value: 'Stefan Hendricks ' .. py:class:: RetrackerThresholdModel(*args: Iterable[Any], **kwargs: Iterable[Any]) Bases: :py:obj:`pysiral.auxdata.AuxdataBaseClass` This class replaces the previous same-name version but utilizizes solely pytorch as pysiral's ML/AI framework for retracker-threshold models .. py:attribute:: valid_waveforms_idx :value: None .. py:attribute:: n_total_waveforms :value: None .. py:attribute:: model_file .. py:attribute:: model_filepath .. py:attribute:: waveform_for_prediction :value: None .. py:attribute:: parameters_for_prediction :value: None .. py:attribute:: model .. py:method:: receive_l1p_input(l1p: L1bdataNCFile) -> None Optional method to add l1p variables to this class before `get_l2_track_vars()` is called. This method here store the waveform power arrays, which are input to the model, normalizes the waveforms and converts to the necessary data type. :param l1p: :return: .. py:method:: get_l2_track_vars(l2: Level2Data) -> None [Mandatory class method] Add the model prediction for the tfmra retracker threshold to the Level-2 data object. The model evaluation depends solely on waveform power. :param l2: `Level2Data` container :return: None :raises: None .. py:class:: AutoEncoderERS2(n_in: int = 35, n_bn: int = 3) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: n_in :value: 35 .. py:attribute:: n_bn :value: 3 .. py:attribute:: encoder .. py:attribute:: decoder .. py:method:: forward(x) .. py:class:: ERS2_TestCandidate_006_FNN(n_in: int = 35, n_par=8) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: n_in :value: 35 .. py:attribute:: n_par :value: 8 .. py:attribute:: model .. py:method:: forward(x, par) .. py:class:: ERS2_TestCandidate_005_FNN(n_in: int = 35, n_par=6) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: n_in :value: 35 .. py:attribute:: n_par :value: 6 .. py:attribute:: model .. py:method:: forward(x, par) .. py:class:: ERS2_TestCandidate_004_FNN(n_in: int = 35, n_par: int = 6) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: n_in :value: 35 .. py:attribute:: n_par :value: 6 .. py:attribute:: wfm .. py:attribute:: par .. py:attribute:: cmb .. py:method:: forward(wfm, par) .. py:class:: ERS2_TestCandidate_003_FNN_TanHLeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 3) Bases: :py:obj:`torch.nn.Module` Create Feed-Forward Neural Network architecture .. py:attribute:: n_in :value: 5 .. py:attribute:: n_out :value: 1 .. py:attribute:: n_par :value: 3 .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:attribute:: fc5 .. py:attribute:: fc1_par .. py:attribute:: fc2_par .. py:attribute:: fc3_par .. py:attribute:: fc4_par .. py:attribute:: fc5_par .. py:attribute:: fc7 .. py:attribute:: fc8 .. py:attribute:: fc9 .. py:method:: forward(x, par) .. py:class:: ERS2_TestCandidate_001_FNN_LeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 2) Bases: :py:obj:`torch.nn.Module` Creates Feed-Forward Neural Network architecture using leaky_relu activation function using two separate branches for 1) the input of the subset waveform power of a five waveform stack and 2) the parameter input of Epsilon and Max Power for the respective waveforms of the stack. Both branches are then input and feed through a common two-layer network and output decimal optimal retracker thresholds through a sigmoid activation finish. Desgined/Trained for ERS2 subwaveform input using 5 bins leading and 30 bins trailing the identified first-maximum index (fmi). REQ: Required for RetrackerThresholdModel .. py:attribute:: n_in :value: 5 .. py:attribute:: n_out :value: 1 .. py:attribute:: n_par :value: 2 .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:attribute:: fc5 .. py:attribute:: fc6 .. py:attribute:: fc1_par .. py:attribute:: fc2_par .. py:attribute:: fc7 .. py:attribute:: fc8 .. py:attribute:: fc9 .. py:method:: forward(x, par) .. py:class:: ERS2_TestCandidate_002_FNN_TanH(n_in: int = 5, n_out: int = 1, n_par: int = 2) Bases: :py:obj:`torch.nn.Module` Creates Feed-Forward Neural Network architecture using tanh activation function using two separate branches for 1) the input of the subset waveform power of a five waveform stack and 2) the parameter input of Epsilon and Max Power for the respective waveforms of the stack. Both branches are then input and feed through a common two-layer network and output decimal optimal retracker thresholds through a sigmoid activation finish. Desgined/Trained for ERS2 subwaveform input using 5 bins leading and 30 bins trailing the identified first-maximum index (fmi). REQ: Required for RetrackerThresholdModel .. py:attribute:: n_in :value: 5 .. py:attribute:: n_out :value: 1 .. py:attribute:: n_par :value: 2 .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:attribute:: fc5 .. py:attribute:: fc6 .. py:attribute:: fc1_par .. py:attribute:: fc2_par .. py:attribute:: fc7 .. py:attribute:: fc8 .. py:attribute:: fc9 .. py:method:: forward(x, par) .. py:class:: ERS2_TestCandidate_003_LSTM_LeakyRelu(n_in: int = 5, n_out: int = 1, n_par: int = 2) Bases: :py:obj:`torch.nn.Module` .. py:attribute:: n_in :value: 5 .. py:attribute:: n_out :value: 1 .. py:attribute:: n_par :value: 2 .. py:attribute:: lstm .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc1_par .. py:attribute:: fc2_par .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:method:: forward(x, par) .. py:class:: TorchFunctionalWaveformModelFNN(fc1_input=45) Bases: :py:obj:`torch.nn.Module` Create Feed-Forward Neural Network architecture REQ: Required for RetrackerThresholdModel Note: Legacy version of the current ENVISAT implementation until rework .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:attribute:: fc5 .. py:attribute:: fc6 .. py:method:: forward(x) .. py:class:: TorchFunctionalWaveformModelSNN(fc1_input=45) Bases: :py:obj:`torch.nn.Module` Create Self-Normalizing Neural Network architecture REQ: Required for RetrackerThresholdModel Note: Deprecated! .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:attribute:: fc3 .. py:attribute:: fc4 .. py:attribute:: fc5 .. py:attribute:: fc6 .. py:method:: forward(x) .. py:class:: TorchFunctionalWaveformModelCNN Bases: :py:obj:`torch.nn.Module` " Create Convolutional Neural Network architecture REQ: Required for RetrackerThresholdModel Note: Deprecated! .. py:attribute:: cv1_1 .. py:attribute:: cv1_2 .. py:attribute:: cv1_3 .. py:attribute:: mp1 .. py:attribute:: fc1 .. py:attribute:: fc2 .. py:method:: forward(x) .. py:class:: RetrackerThresholdModelTorch(*args: Iterable[Any], **kwargs: Iterable[Any]) Bases: :py:obj:`pysiral.auxdata.AuxdataBaseClass` Base class for all subtype auxdata base classes (e.g. SICBaseClass). This class defines the mandatory set of methods and properties for all auxdata classes .. py:attribute:: model_file .. py:attribute:: model_filepath .. py:attribute:: waveform_for_prediction :value: None .. py:attribute:: model .. py:method:: receive_l1p_input(l1p: L1bdataNCFile) -> None Optional method to add l1p variables to this class before `get_l2_track_vars()` is called. This method here store the waveform power arrays, which are input to the model, normalizes the waveforms and converts to the necessary data type. :param l1p: :return: .. py:method:: get_l2_track_vars(l2: Level2Data) -> None [Mandatory class method] Add the model prediction for the tfmra retracker threshold to the Level-2 data object. The model evaluation depends solely on waveform power. :param l2: `Level2Data` container :return: None :raises: None