teca_netcdf_util::read_variable_and_attributes Class Reference

Functional that reads and returns a variable from the named file. More...

#include <teca_netcdf_util.h>

Public Types
using	data_elem_t = std::pair< p_teca_variant_array, teca_metadata >
using	data_t = std::pair< unsigned long, data_elem_t >
using	task_t = std::packaged_task< data_t()>
using	queue_t = teca_thread_pool< task_t, data_t >
using	p_queue_t = std::shared_ptr< queue_t >

Public Member Functions
	read_variable_and_attributes (const std::string &path, const std::string &file, unsigned long id, const std::string &variable)
data_t	operator() ()

Static Public Member Functions
static data_t	package (unsigned long id, p_teca_variant_array var=nullptr, const teca_metadata &md=teca_metadata())

Detailed Description

Functional that reads and returns a variable from the named file.

We're doing this so we can do thread parallel I/O to hide some of the cost of opening files on Lustre and to hide the cost of reading time coordinate which is typically very expensive as NetCDF stores unlimited dimensions non-contiguously.

Note

Thu 09 Apr 2020 05:45:29 AM PDT Threading these operations worked well in NetCDF 3, however in NetCDF 4 backed by HDF5 necessary locking eliminates any speed up.

Member Typedef Documentation

data_elem_t

using teca_netcdf_util::read_variable_and_attributes::data_elem_t = std::pair<p_teca_variant_array, teca_metadata>

Data and task types.

---

The documentation for this class was generated from the following file:

• rtd_doxygen/io/teca_netcdf_util.h