teca_netcdf_util::read_variable Class Reference

Function that reads and returns a variable from the named file. More...

#include <teca_netcdf_util.h>

Public Types
using	data_t = std::pair< unsigned long, p_teca_variant_array >
using	task_t = std::packaged_task< data_t(int)>
using	queue_t = teca_cpu_thread_pool< task_t, data_t >
using	p_queue_t = std::shared_ptr< queue_t >

Public Member Functions
	read_variable (const std::string &path, const std::string &file, unsigned long id, const std::string &variable)
data_t	operator() (int device_id=-1)

Static Public Member Functions
static data_t	package (unsigned long id, p_teca_variant_array var=nullptr)

Detailed Description

Function that reads and returns a variable from the named file.

we're doing this so we can do thread parallel I/O to hide some of the cost of opening files on Lustre and to hide the cost of reading time coordinate which is typically very expensive as NetCDF stores unlimited dimensions non-contiguously

Note

Thu 09 Apr 2020 05:45:29 AM PDT Threading these operations worked well in NetCDF 3, however in NetCDF 4 backed by HDF5 necessary locking eliminates any speed up.

Member Typedef Documentation

data_t

using teca_netcdf_util::read_variable::data_t = std::pair<unsigned long, p_teca_variant_array>

Data and task types.

---

The documentation for this class was generated from the following file:

• develop/io/teca_netcdf_util.h