demeter.demeter_io package

Submodules

demeter.demeter_io.reader module

Read and format input data.

Copyright (c) 2017, Battelle Memorial Institute

Open source under license BSD 2-Clause - see LICENSE and DISCLAIMER

@author: Chris R. Vernon (chris.vernon@pnnl.gov)

exception demeter.demeter_io.reader.ValidationException(*args)

Bases: Exception

Validation exception for error in runtime test.

demeter.demeter_io.reader.csv_to_array(f)

Read CSV file to NumPy array

Parameters:

f – Full path to input file

Returns:

array

demeter.demeter_io.reader.read_allocation_data(f, lc_col, output_level=3, delim=',')

Converts an allocation file to a numpy array. Returns final land cover class and target land cover class names as lists.

Parameters:

  • f – Input allocation file with header

  • lc_col – Target land cover field name in header (located a zero index)

:param output_level If 3 all variables will be returned (default); 2, target lcs list and array; 3, array :param delim: Delimiter type; default is comma

Returns:

[0] List of final land cover classes [1]list of target land cover classes [2] numpy array of allocation values

demeter.demeter_io.reader.read_base(config, observed_landclasses, sequence_metric_dict, metric_seq, region_seq, logger=None)

Read and process base layer land cover file.

Parameters:

  • config – Configuration object

  • observed_landclasses – A list of land classes represented in the observed data

  • sequence_metric_dict – A dictionary of projected metric ids to their original id

  • metric_seq – An ordered list of expected metric ids

  • region_seq – An ordered list of expected region ids

demeter.demeter_io.reader.read_gcam_file(gcam_data, gcam_landclasses, start_yr, end_yr, timestep, scenario, region_dict, agg_level, metric_seq, area_factor=1000, logger=None)

Read and process the GCAM land allocation output file.

Parameters:

  • gcam_data – GCAM land allocation file or data frame from gcamreader

  • name_col – Field name of the column containing the region and either AEZ or basin number

  • metric – AEZ or Basin

  • start_yr – User-defined GCAM start year to process from configuration file

  • end_yr – User-defined GCAM end year to process from configuration file

  • scenario – GCAM scenario name contained in file that the user wishes to process; set in config.ini

  • region_dict – The reference dictionary for GCAM region_name: region_id

  • metric_seq – An ordered list of expected metric ids

  • area_factor – The factor that will be a multiplier to the land use area that is in thousands km

Returns:

A list of the following (represents the target user-defined scenario): user_years: a list of target GCAM years as int gcam_ludata: Numpy array of land use area per row per year gcam_metric: Numpy array of AEZ or Basin numbers per row gcam_landname: Numpy array of the GCAM land use name per row gcam_regionnumber: Numpy array of GCAM region numbers per row allreg: Numpy array of unique region names allregnumber: Numpy array of unique region numbers allregaez: List of lists, metric ids per region

demeter.demeter_io.reader.read_gcam_land(db_path, db_file, f_queries, d_basin_name, subreg, crop_water_src)

Query GCAM database for irrigated land area per region, subregion, and crop type.

Parameters:

  • db_path – Full path to the input GCAM database

  • f_queries – Full path to the XML query file

  • d_basin_name – A dictionary of ‘basin_glu_name’ : basin_id

  • subreg – Agg level of GCAM database: either AEZ or BASIN

  • crop_water_src – Filter for crop type: one of IRR, RFD, or BOTH

Returns:

A pandas DataFrame containing region, subregion, crop type, and irrigated area per year in thousands km2

demeter.demeter_io.reader.to_array(f, target_index, delim=',')

Read file to Numpy array and slice out a single field by the target index.

Parameters:

  • f –

  • target_index –

  • delim –

Returns:

demeter.demeter_io.reader.to_dict(f, header=False, delim=',', swap=False, value_col=1)

Return a dictionary of key: value pairs. Supports only key to one value.

Parameters:

  • f – Full path to input file

  • header – If header exists True, else False (default)

  • delim – Set delimiter as string; default is comma

  • swap – Change the order of the key, value pair

  • value_col – Column index of dict values (or keys if swap is True)

Returns:

Key: value pair dictionary

demeter.demeter_io.reader.to_list(f, header=True, delim=',')

Retuns values as a list of integers where first column is row names and there is only one value column.

Parameters:

f – Full path to input file

:param header If header exists True, else False (default) :param delim: Set delimiter as string; default is comma :return: List of integers

demeter.demeter_io.writer module

Write data to multiple file outputs.

Copyright (c) 2017, Battelle Memorial Institute

Open source under license BSD 2-Clause - see LICENSE and DISCLAIMER

@author: Chris R. Vernon (chris.vernon@pnnl.gov); Yannick le Page (niquya@gmail.com); Caleb J. Braun (caleb.braun@pnnl.gov)

demeter.demeter_io.writer.arr_to_ascii(arr, r_ascii, xll=-180, yll=-90, cellsize=0.25, nodata=-9999)

Convert a numpy array to an ASCII raster.

@param arr:

2D array

@param r_ascii:

Full path to outfile with extension

@param xll:

Longitude coordinate for lower left corner

@param yll:

Latitude coordinate for lower left corner

@param cellsize:

Cell size in geographic degrees

@param nodata:

Value representing NODATA

demeter.demeter_io.writer.array_to_csv(arr, out_file)

Save numpy array as a CSV file.

Parameters:

  • arr – Input Numpy array

  • out_file – Output CSV file

demeter.demeter_io.writer.lc_timestep_csv(c, yr, final_landclasses, spat_coords, metric_id_array, gcam_regionnumber, spat_water, cellarea, spat_ludataharm, metric, units='fraction', write_outputs=False, write_ncdf=False, sce='default', resolution=0.05, write_csv=False, regrid_res=0.05)

Save land cover data for each time step as a CSV file.

demeter.demeter_io.writer.max_ascii_rast(arr, out_dir, step, alg='max', nodata=-9999, xll=-180, yll=-90, cellsize=0.25)

Return the land class index containing the maximum value in the array axis.

NOTE: Replace NaN with your nodata value. If all classes 0, then -9999 If multiple classes have the same max value, get class with largest index

@param arr:

3D array (landclass, col, row)

@param alg:

Algorithm to extract the land class index from values

@param out_rast:

Full path to outfile with extension

@param xll:

Longitude coordinate for lower left corner

@param yll:

Latitude coordinate for lower left corner

@param cellsize:

Cell size in geographic degrees

@param nodata:

Value representing NODATA

demeter.demeter_io.writer.save_array(arr, out_file)

Save numpy array to NPY file.

Parameters:

  • arr – Input Numpy array

  • out_file – Output NPY file

demeter.demeter_io.writer.to_netcdf_lc(spat_lc, lat, lon, resin, final_landclasses, years, step, model, out_dir)

Build a NetCDF file for each land class that contains the gridded fraction of land cover of that land class over all simulation years.

Parameters:

  • spat_lc – A 3D array representing fraction of land cover (lat, lon, fraction landclass)

  • lat – An array of latitude values for mapping (n)

  • lon – An array of longitude values for mapping (n)

  • resin – The input spatial resolution in geographic degrees (float)

  • final_landclasses – An array of land classes (n_classes)

  • years – A list of output years (int)

  • step – The current time step (int)

  • model – The name of the model running (str)

  • out_dir – A full path string of the output directory (str)

Returns:

A NetCDF classic file.

demeter.demeter_io.writer.to_netcdf_yr(spat_lc, map_idx, lat, lon, resin, final_landclasses, yr, model, out_file)

Build a NetCDF file for each time step that contains the gridded fraction of land cover for each land class.

Parameters:

  • spat_lc – An array of gridded data as fraction land cover (n_grids, n_landclasses)

  • map_idx – An array of cell index positions for spatially mapping the gridded data (n_grids, n_landclasses)

  • lat – An array of latitude values for mapping (n)

  • lon – An array of longitude values for mapping (n)

  • resin – The input spatial resolution in geographic degrees (float)

  • final_landclasses – An array of land classes (n_classes)

  • yr – The target time step (int)

  • model – The name of the model running (str)

  • out_file – A full path string of the output file with extension (str)

Returns:

A NetCDF classic file.

demeter.demeter_io.writer.write_transitions(s, c, step, transitions)

Save land cover transitions per time step to a CSV file.

Parameters:

  • order_rules –

  • final_landclasses –

Module contents