pysiral.core.output =================== .. py:module:: pysiral.core.output Classes ------- .. autoapisummary:: pysiral.core.output.OutputHandlerBase pysiral.core.output.DefaultLevel2OutputHandler pysiral.core.output.NCDateNumDef pysiral.core.output.NCDataFile pysiral.core.output.L1bDataNC pysiral.core.output.Level1POutput pysiral.core.output.Level2Output pysiral.core.output.Level3Output pysiral.core.output.PysiralOutputFilenaming pysiral.core.output.PysiralOutputFolder Functions --------- .. autoapisummary:: pysiral.core.output.get_output_class Module Contents --------------- .. py:class:: OutputHandlerBase(output_def, applicable_data_level=None, subfolder_tags=None, default_file_location=None) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` A class that defines properties of output files (content, location, format) based on the output definition, data container and other processor settings .. py:attribute:: subfolder_format .. py:attribute:: pysiral_config .. py:attribute:: error .. py:attribute:: _basedir :value: 'n/a' .. py:attribute:: _doi :value: None .. py:attribute:: subfolders :value: None .. py:attribute:: subfolder_tags :value: None .. py:attribute:: applicable_data_level :value: None .. py:attribute:: default_file_location :value: None .. py:attribute:: output_def_filename .. py:method:: fill_template_string(template, dataset) Fill an template string with information of a dataset object (in this case Level2Data) .. py:method:: get_dt_subfolders(dt, subfolder_tags) Returns a list of subdirectories based on a datetime object (usually the start time of data collection) .. py:method:: get_template_attrs(template) :staticmethod: Extract attribute names and options (if defined) for a give template string .. py:method:: _init_from_output_def(output_def) Adds the information for the output def yaml files (either full filename or treedict structure) .. py:method:: _set_basedir(basedir, create=True) Sets and (per default) creates the main output directory .. py:method:: _create_directory(directory) Convinience method to create a directory and add an error when failed .. py:method:: _get_subdirectories(dt) .. py:method:: _get_directory_from_dt(dt) .. py:method:: _validate_outputdef() -> None Run a series of tests to check if a valid output definition has been passed. Note: these tests will only check existing items of the output definition. If the requested item is missing a separate exception will be evoked :return: .. py:property:: has_doi .. py:property:: id .. py:property:: product_level_subfolder .. py:property:: data_level .. py:property:: basedir .. py:property:: output_def .. py:property:: now_directory Returns a directory suitable string with the current time .. py:property:: variable_def .. py:class:: DefaultLevel2OutputHandler(output_def='default', subdirectory='default_output', overwrite_protection=True, period='default') Bases: :py:obj:`OutputHandlerBase` Default output handler with pysiral conventions. Uses product directory from local_machine_def.yaml as standard repository .. py:attribute:: subdirectory :value: 'default_output' .. py:attribute:: overwrite_protection :value: True .. py:attribute:: _period :value: 'default' .. py:method:: get_filename_from_data(l2) Return the filename for a defined level-2 data object based on tag filenaming in output definition file :param l2: l2data.Level2Data :return: str .. py:method:: get_directory_from_data(l2, create=True) Return the output directory based on information provided in an l2 data object :param l2: :param create: :return: .. py:method:: get_fullpath_from_data(l2) Return export path and filename based on information provided in the l2 data object :param l2: :return: .. py:method:: get_global_attribute_dict(l2) .. py:method:: remove_old(time_range) This method will erase all files in the target orbit for a given time range. Use with care .. py:method:: _init_product_directory() Get main product directory from local_machine_def, add mandatory runtag subdirectory, optional second subdirectory for overwrite protection and product level id subfolder .. py:property:: default_output_def_filename .. py:class:: NCDateNumDef(units: str = 'seconds since 1970-01-01') Bases: :py:obj:`object` Holds definition for datetime conversion to numbers and vice versa for netCDF operations .. py:attribute:: units :value: 'seconds since 1970-01-01' .. py:attribute:: calendar :value: 'standard' .. py:class:: NCDataFile(output_handler) Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Template for default pysiral class with logging/error handling capabilities .. py:attribute:: error .. py:attribute:: output_handler .. py:attribute:: data :value: None .. py:attribute:: filename :value: None .. py:attribute:: base_export_path :value: None .. py:attribute:: parameter_attributes :value: None .. py:attribute:: time_def .. py:attribute:: zlib :value: True .. py:attribute:: _rootgrp :value: None .. py:attribute:: _options :value: None .. py:attribute:: _proc_settings :value: None .. py:attribute:: verbose :value: False .. py:method:: set_options(**opt_dict) .. py:method:: set_processor_settings(proc_settings) .. py:method:: set_base_export_path(path) .. py:method:: _set_doi() .. py:method:: _set_data_record_type() .. py:method:: _write_global_attributes() .. py:method:: _populate_data_groups(level3=False, flip_yc=False) .. py:method:: _create_root_group(attdict, **global_attr_keyw) Create the root group and add l1b metadata as global attributes .. py:method:: _convert_datetime_attributes(attdict) Replace l1b info parameters of type datetime.datetime by a double representation to match requirements for netCDF attribute data type rules .. py:method:: _convert_bool_attributes(attdict) :staticmethod: Replace l1b info parameters of type bool ['b1'] by a integer representation to match requirements for netCDF attribute data type rules .. py:method:: _convert_nonetype_attributes(attdict) :staticmethod: Replace l1b info parameters of type bool ['b1'] by a integer representation to match requirements for netCDF attribute data type rules .. py:method:: _set_global_attributes(attdict, prefix='') Save l1b.info dictionary as global attributes .. py:method:: _get_variable_attr_dict(parameter) Retrieve the parameter attributes .. py:method:: _write_processor_settings() .. py:method:: _open_file() .. py:method:: _write_to_file() .. py:property:: export_path Evoking this property will also create the directory if it does not already exist .. py:property:: export_filename Returns the filename for the level2 output file .. py:property:: full_path .. py:class:: L1bDataNC Bases: :py:obj:`pysiral.core.legacy_classes.DefaultLoggingClass` Class to export a L1bdata object into a netcdf file NOTE: This class is different from the other netCDF output classes as it uses data groups. .. py:attribute:: datagroups :value: ['waveform', 'surface_type', 'time_orbit', 'classifier', 'correction'] .. py:attribute:: filename :value: None .. py:attribute:: time_def .. py:attribute:: zlib :value: True .. py:attribute:: _rootgrp :value: None .. py:attribute:: _options :value: None .. py:attribute:: _proc_settings :value: None .. py:attribute:: verbose :value: False .. py:attribute:: output_folder :value: None .. py:attribute:: l1b :value: None .. py:attribute:: parameter_attributes :value: [] .. py:method:: export() .. py:method:: _validate() .. py:method:: _create_filename() .. py:method:: _set_global_attributes(attdict, prefix='') Save l1b.info dictionary as global attributes .. py:method:: _create_root_group(attdict, **global_attr_keyw) Create the root group and add l1b metadata as global attributes :param attdict: A dictionary containing the global attributes :param global_attr_keyw: :return: .. py:method:: _populate_data_groups() .. py:method:: _convert_datetime_attributes(attdict) Change any datetime-like object to a netCDF compatible numerical representation :param attdict: dictionary-like :return: .. py:method:: _convert_bool_attributes(attdict) :staticmethod: Replace l1b info parameters of type bool ['b1'] by a integer representation to match requirements for netCDF attribute data type rules :param attdict: dictionary-like :return: .. py:method:: _convert_nonetype_attributes(attdict) :staticmethod: Change empty (None) values in an attrdict to an empty string :param attdict: dictionary like :return: None, will be changed in place .. py:method:: _get_variable_attr_dict(parameter) Retrieve the parameter attributes .. py:method:: _open_file() .. py:method:: _write_to_file() .. py:class:: Level1POutput(data, output_handler) Bases: :py:obj:`NCDataFile` Class to export a l2data object into a netcdf file .. py:attribute:: data .. py:class:: Level2Output(data, output_handler) Bases: :py:obj:`NCDataFile` Class to export a l2data object into a netcdf file .. py:attribute:: time_def .. py:attribute:: data .. py:method:: _export_content() :return: .. py:class:: Level3Output(data, output_handler) Bases: :py:obj:`NCDataFile` Class to export a Level-3 data object into a netcdf file. Differences to Level2Output are small but substantial (e.g. with the additional time dimension) .. py:attribute:: data .. py:method:: _export_content() .. py:method:: _add_time_variables() Add the required variables describing the temporal coverage and the reference time of the Level-3 data product. :return: .. py:method:: _add_grid_mapping_variables() This method adds a variable to the netCDF file that contains the grid definition and is named after the projection. E.g: byte Lambert_Azimuthal_Grid; :latitude_of_projection_origin = 90.0; // double :longitude_of_projection_origin = 0.0; // double :proj4_string = "+proj=laea +lon_0=0 +datum=WGS84 +ellps=WGS84 +lat_0=90.0"; :semi_major_axis = 6378137.0; // double :false_easting = 0.0; // double :false_northing = 0.0; // double :grid_mapping_name = "lambert_azimuthal_equal_area"; :inverse_flattening = 298.257223563; // double The information is taken 1:1 from the `netcdf_grid_description` entry in the grid definition file. :return: .. py:class:: PysiralOutputFilenaming Bases: :py:obj:`object` Class for generating and parsing of pysiral output filenames for all data levels TODO: Find out if this is still needed (probably for l1p) and remove if obsolet .. py:attribute:: error .. py:attribute:: data_level :value: None .. py:attribute:: version :value: None .. py:attribute:: hemisphere :value: None .. py:attribute:: mission_id :value: None .. py:attribute:: orbit :value: None .. py:attribute:: start :value: None .. py:attribute:: stop :value: None .. py:attribute:: resolution :value: None .. py:attribute:: grid :value: None .. py:attribute:: _registered_parsers .. py:method:: parse_filename(fn) Parse info from pysiral output filename .. py:method:: _datetime_format(dt) :staticmethod: .. py:class:: PysiralOutputFolder Bases: :py:obj:`object` Class for generating and retrieving output folders .. py:attribute:: error .. py:attribute:: data_level :value: None .. py:attribute:: path :value: None .. py:attribute:: version :value: 'default' .. py:attribute:: mission_id :value: None .. py:attribute:: year :value: None .. py:attribute:: month :value: None .. py:attribute:: hemisphere :value: None .. py:attribute:: config .. py:method:: l1bdata_from_list(mission_id, version, hemisphere, year, month) .. py:method:: l1bdata_from_l1b(l1b, version='default') .. py:method:: l2i_from_startdt(startdt, base_path, subfolders) .. py:method:: create() .. py:method:: _set_folder_as_l1bdata() .. py:function:: get_output_class(name)