pysiral.l3proc ============== .. py:module:: pysiral.l3proc .. autoapi-nested-parse:: Created on Fri Jul 24 14:04:27 2015 @author: Stefan Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/pysiral/l3proc/alg/index Classes ------- .. autoapisummary:: pysiral.l3proc.Level3Processor pysiral.l3proc.L2iDataStack pysiral.l3proc.L3DataGrid pysiral.l3proc.L3MetaData pysiral.l3proc.Level3GridDefinition pysiral.l3proc.Level3ProductDefinition pysiral.l3proc.Level3ProcessorItem Package Contents ---------------- .. py:class:: Level3Processor(product_def) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Template for default pysiral class with logging/error handling capabilities .. py:attribute:: error .. py:attribute:: _job .. py:attribute:: _l3_progress_percent :value: 0.0 .. py:attribute:: _l2i_files :value: None .. py:attribute:: _period :value: None .. py:method:: process_l2i_files(l2i_files, period) The main call for the Level-3 processor TODO: Needs organization :param l2i_files: :param period: :return: .. py:method:: _log_progress(i) Concise logging on the progress of l2i stack creation .. py:method:: apply_orbit_filter(l2i, orbit_filter) :staticmethod: Apply a :param l2i: :param orbit_filter: :return: .. py:method:: apply_miz_filter(l2i, miz_filter) :staticmethod: Flag values based on the miz filter value :param l2i: :param miz_filter: :return: .. py:method:: _apply_processing_items(l3grid) Sequentially apply the processing items defined in the Level-3 processor definition files (listed under root.processing_items) :param l3grid: :return: .. py:class:: L2iDataStack(griddef, l2_parameter) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Template for default pysiral class with logging/error handling capabilities .. py:attribute:: griddef .. py:attribute:: l2_parameter .. py:attribute:: _n_records :value: 0 .. py:attribute:: _l2i_count :value: 0 .. py:attribute:: start_time :value: [] .. py:attribute:: stop_time :value: [] .. py:attribute:: mission :value: [] .. py:attribute:: timeliness :value: [] .. py:attribute:: _l2i_info :value: None .. py:method:: _initialize_stacks() Create all data stacks, content will be added sequentially with `add` method .. py:method:: add(l2i) Add a l2i data object to the stack Args: l2i (obj): l2i object (currently: pysiral.l2data.L2iNCFileImport) Returns: None .. py:property:: n_total_records .. py:property:: l2i_count .. py:property:: parameter_stack .. py:property:: l2i_info .. py:class:: L3DataGrid(job, stack, period, doi='') Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Container for computing gridded data sets based on a l2i data stack (averaged l2i parameter, grid cell statistics) .. py:attribute:: error .. py:attribute:: _doi :value: '' .. py:attribute:: _data_record_type :value: 'none' .. py:attribute:: _griddef .. py:attribute:: _l3def .. py:attribute:: _period .. py:attribute:: _external_masks .. py:attribute:: _time_dim_is_unlimited .. py:attribute:: _creation_time .. py:attribute:: _l2_parameter :value: None .. py:attribute:: l2 .. py:attribute:: _non_empty_grid_indices :value: None .. py:attribute:: vars .. py:attribute:: _metadata :value: None .. py:method:: set_doi(doi) .. py:method:: set_data_record_type(data_record_type) .. py:method:: get_attribute(attribute_name, *args) Return a string for a given attribute name. This method is required for the output data handler .. py:method:: add_grid_variable(parameter_name, fill_value, dtype, allow_overwrite: bool = False) Add a grid variable and fill with empty values :param parameter_name: The name of the parameter :param fill_value: the "empty" value assigned to all cell :param dtype: numpy compatible dtype :param allow_overwrite: If True, an existing variable will be overwritten :return: .. py:method:: calculate_longitude_latitude_fields() Geographic coordinates from GridDefinition .. py:method:: grid_l2_parameter() Compute averages of all l2i parameter for each grid cell. The list of l2i parameter is from the output format definition No averages are computed for grid cells that are tagged with a land flag. .. py:method:: get_parameter_by_name(name, raise_on_error=True) .. py:method:: set_parameter_by_name(name, var) .. py:method:: _init_grid_indices_mask() -> None Compute a mask of non-empty grid indices :return: .. py:method:: _init_metadata_from_l2() Gets metadata from Level-2 instance :return: .. py:method:: _init_parameter_fields(pardefs) Initialize output parameter fields .. py:method:: _get_attr_source_mission_id(*args) .. py:method:: _get_attr_source_mission_name(*_) .. py:method:: _get_attr_source_timeliness(*args) .. py:method:: _get_attr_grid_id(*args) .. py:method:: _get_attr_grid_spacing_tag(*args) .. py:method:: _get_attr_source_mission_sensor(*args) .. py:method:: _get_attr_source_mission_sensor_fn(*args) Same as source mission sensor, only a sanitized version for filenames .. py:method:: _get_attr_source_hemisphere(*args) .. py:method:: _get_attr_uuid(*args) :staticmethod: .. py:method:: _get_attr_startdt(dtfmt) .. py:method:: _get_attr_stopdt(dtfmt) .. py:method:: _get_attr_geospatial_lat_min(*_) .. py:method:: _get_attr_geospatial_lat_max(*_) .. py:method:: _get_attr_geospatial_lon_min(*_) .. py:method:: _get_attr_geospatial_lon_max(*_) .. py:method:: _get_attr_geospatial_str(value) :staticmethod: .. py:method:: _get_attr_source_auxdata_sic(*_) .. py:method:: _get_attr_source_auxdata_snow(*_) .. py:method:: _get_attr_source_auxdata_sitype(*_) .. py:method:: _get_attr_utcnow(*args) .. py:method:: _get_attr_l2_time_coverage_start(*args) .. py:method:: _get_attr_l2_time_coverage_end(*args) .. py:method:: _get_attr_time_coverage_start(*args) .. py:method:: _get_attr_time_coverage_end(*args) .. py:method:: _get_attr_time_coverage_duration(*args) .. py:method:: _get_attr_doi(*_) .. py:method:: _get_attr_data_record_type(*args) .. py:method:: _get_attr_pysiral_version(*args) :staticmethod: .. py:method:: flipud() .. py:property:: metadata .. py:property:: grid_xi_range .. py:property:: grid_yj_range .. py:property:: all_grid_indices .. py:property:: grid_indices .. py:property:: dimdict .. py:property:: grid_shape .. py:property:: griddef .. py:property:: l3def .. py:property:: hemisphere .. py:property:: time_bounds .. py:class:: L3MetaData Bases: :py:obj:`object` Container for L3S Metadata information (see property attribute_list for a list of attributes) .. py:attribute:: _attribute_list :value: ['mission_ids', 'start_time', 'stop_time', 'grid_name', 'period_label', 'time_coverage_start',... .. py:method:: get_missions_from_stack(stack) Get a list of missions that went into the stack (must be a list, since multi-mission grids are supported) .. py:method:: get_data_period_from_stack(stack) Get the first and last timestamp .. py:method:: get_time_coverage_from_period(period) Get the start and end of requested data period .. py:method:: get_auxdata_infos(l2i_info) Get information on auxiliary data sources from l2i global attributes TODO: This part is deprecated :param l2i_info: :return: .. py:method:: get_projection_parameter(griddef) .. py:method:: __repr__() .. py:property:: attribute_list .. py:property:: attdict Return attributes as dictionary (e.g. for netCDF export) .. py:property:: mission .. py:method:: set_attribute(tag, value) .. py:class:: Level3GridDefinition(l3_settings_file) Bases: :py:obj:`pysiral.grid.GridDefinition` This is a variation of GridDefinition with a mandatory link to a griddef yaml file .. py:method:: from_grid_id(l3_grid_id: str) -> Level3GridDefinition :classmethod: Create a Level3GridDefinition from a grid id .. py:class:: Level3ProductDefinition(l3_settings_file, grid, output, period) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Template for default pysiral class with logging/error handling capabilities .. py:attribute:: error .. py:attribute:: _l3_settings_file .. py:attribute:: _output .. py:attribute:: _grid .. py:attribute:: _period .. py:method:: _parse_l3_settings() .. py:method:: validate() .. py:property:: grid .. py:property:: outputs .. py:property:: n_outputs .. py:property:: l3def .. py:property:: period .. py:property:: l3_masks Return a sorted list of the masks applied to level 3 data .. py:property:: l3_external_masks .. py:property:: l3_post_processors .. py:property:: l2_parameter Extract a list of paramter names to be extracted from l2i product files .. py:property:: l3_parameter Extract a list of paramter names to be computed by the Level-3 processor .. py:class:: Level3ProcessorItem(l3grid, **cfg) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` A parent class for processing items to be selected in the Level-3 processor settings and applied in the Level3Processor .. py:attribute:: error .. py:attribute:: l3grid .. py:attribute:: cfg .. py:method:: _process_dynamic_dependencies() This method processes dynamic dependencies. Dynamic dependencies are variables that are not always required, but only if certain options are set in a certain way. All processor item classes that are inheriting this class require the property `dynamic_variable_dependencies` for this method to work. The property should return a list of tuples with (option_name, option_value, variable_name). If the option `option_name` is set to `option_value`, the variable `variable_name` is required in the Level-3 data grid. :return: .. py:method:: _check_variable_dependencies() Tests if the Level-3 data grid has all required input variables (both in the Level-2 stack as well as in the Level 3 parameters). All processor item classes that are inheriting this class require the properties `l3_variable_dependencies` & `l2_variable_dependencies` for this method to work. Both parameter should return a list of variable names. Empty lists should be returned in case of no dependency. :return: .. py:method:: _check_options() Tests if the all options are given in the Level-3 processor definition files. All processor item classes require the property `required_options` (list of option names) for this method to work. NOTE: It is in the spirit of pysiral of having all numerical values in one place only that ideally is not the code itself. :return: .. py:method:: _add_l3_variables() This method initializes the output variables for a given processing item to the l3grid. All processor item classes require the property `l3_output_variables` for this method to work. The property should return a dict with variable names as keys and the value a dict with fill_value and data type. :return: .. py:property:: allow_overwrite :type: bool If True, existing variables in the l3grid will be overwritten when adding new variables