TileDB C++ API Reference

Context

class tiledb::Context

A TileDB context wraps a TileDB storage manager “instance.” Most objects and functions will require a Context.

Internal error handling is also defined by the Context; the default error handler throws a TileDBError with a specific message.

Example:

tiledb::Context ctx;
// Use ctx when creating other objects:
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);

// Set a custom error handler:
ctx.set_error_handler([](const std::string &msg) {
    std::cerr << msg << std::endl;
});

Public Functions

Context()

Constructor. Creates a TileDB Context with default configuration.

Exceptions

Context(const Config &config)

Constructor. Creates a TileDB context with the given configuration.

Exceptions

void handle_error(int rc) const

Error handler for the TileDB C API calls. Throws an exception in case of error.

Parameters
  • rc: If != TILEDB_OK, calls error handler

std::shared_ptr<tiledb_ctx_t> ptr() const

Returns the C TileDB context object.

operator tiledb_ctx_t *() const

Auxiliary operator for getting the underlying C TileDB object.

Context &set_error_handler(const std::function<void(const std::string&)> &fn)

Sets the error handler callback. If none is set, the default_error_handler is used. The callback accepts an error message.

Return
Reference to this Context
Parameters
  • fn: Error handler callback function

Config config() const

Returns a copy of the configuration of the context.

bool is_supported_fs(tiledb_filesystem_t fs) const

Return true if the given filesystem backend is supported.

Example:

tiledb::Context ctx;
bool s3_supported = ctx.is_supported_fs(TILEDB_S3);

Parameters
  • fs: Filesystem to check

void cancel_tasks() const

Cancels all background or async tasks associated with this context.

Public Static Functions

static void default_error_handler(const std::string &msg)

The default error handler callback.

Exceptions

Config

class tiledb::Config

Carries configuration parameters for a context.

Example:

Config conf;
conf["vfs.s3.region"] = "us-east-1a";
conf["vfs.s3.use_virtual_addressing"] = "true";
Context ctx(conf);
// array/kv operations with ctx

Public Functions

Config(const std::string &filename)

Constructor that takes as input a filename (URI) that stores the config parameters. The file must have the following (text) format:

{parameter} {value}

Anything following a # character is considered a comment and, thus, is ignored.

See Config::set for the various TileDB config parameters and allowed values.

Parameters
  • filename: The name of the file where the parameters will be read from.

Config(tiledb_config_t **config)

Constructor from a C config object.

void save_to_file(const std::string filename)

Saves the config parameters to a (local) text file.

std::shared_ptr<tiledb_config_t> ptr() const

Returns the pointer to the TileDB C config object.

operator tiledb_config_t *() const

Auxiliary operator for getting the underlying C TileDB object.

Config &set(const std::string &param, const std::string &value)

Sets a config parameter-value pair.

Parameters

  • sm.dedup_coords If true, cells with duplicate coordinates will be removed during sparse array writes. Note that ties during deduplication are arbitrary. Default: false
  • sm.check_coord_dups This is applicable only if sm.dedup_coords is false. If true, an error will be thrown if there are cells with duplicate coordinates during sparse array writes. If false and there are duplicates, the duplicates will be written without errors, but the TileDB behavior could be unpredictable. Default: true
  • sm.check_coord_oob If true, an error will be thrown if there are cells with coordinates lying outside the domain during sparse array writes. Default: true
  • sm.tile_cache_size The tile cache size in bytes. Any uint64_t value is acceptable. Default: 10,000,000
  • sm.array_schema_cache_size The array schema cache size in bytes. Any uint64_t value is acceptable. Default: 10,000,000
  • sm.fragment_metadata_cache_size The fragment metadata cache size in bytes. Any uint64_t value is acceptable. Default: 10,000,000
  • sm.enable_signal_handlers Whether or not TileDB will install signal handlers. Default: true
  • sm.num_async_threads The number of threads allocated for async queries. Default: 1
  • sm.num_reader_threads The number of threads allocated for filesystem read operations. Default: 1
  • sm.num_writer_threads The number of threads allocated for filesystem write operations. Default: 1
  • sm.num_tbb_threads The number of threads allocated for the TBB thread pool (if TBB is enabled). Note: this is a whole-program setting. Usually this should not be modified from the default. See also the documentation for TBB’s task_scheduler_init class.Default: TBB automatic
  • vfs.num_threads The number of threads allocated for VFS operations (any backend), per VFS instance. Default: number of cores
  • vfs.min_parallel_size The minimum number of bytes in a parallel VFS operation (except parallel S3 writes, which are controlled by vfs.s3.multipart_part_size.) Default: 10MB
  • vfs.file.max_parallel_ops The maximum number of parallel operations on objects with URIs. Default: vfs.num_threads
  • vfs.s3.region The S3 region, if S3 is enabled. Default: us-east-1
  • vfs.s3.scheme The S3 scheme (http or https), if S3 is enabled. Default: https
  • vfs.s3.endpoint_override The S3 endpoint, if S3 is enabled. Default: “”
  • vfs.s3.use_virtual_addressing The S3 use of virtual addressing (true or false), if S3 is enabled. Default: true
  • vfs.s3.max_parallel_ops The maximum number of S3 backend parallel operations. Default: vfs.num_threads
  • vfs.s3.multipart_part_size The part size (in bytes) used in S3 multipart writes. Any uint64_t value is acceptable. Note: vfs.s3.multipart_part_size * vfs.s3.max_parallel_ops bytes will be buffered before issuing multipart uploads in parallel. Default: 5MB
  • vfs.s3.connect_timeout_ms The connection timeout in ms. Any long value is acceptable. Default: 3000
  • vfs.s3.connect_max_tries The maximum tries for a connection. Any long value is acceptable. Default: 5
  • vfs.s3.connect_scale_factor The scale factor for exponential backofff when connecting to S3. Any long value is acceptable. Default: 25
  • vfs.s3.request_timeout_ms The request timeout in ms. Any long value is acceptable. Default: 3000
  • vfs.s3.proxy_host The proxy host. Default: “”
  • vfs.s3.proxy_port The proxy port. Default: 0
  • vfs.s3.proxy_scheme The proxy scheme. Default: “https”
  • vfs.s3.proxy_username The proxy username. Note: this parameter is not serialized by tiledb_config_save_to_file. Default: “”
  • vfs.s3.proxy_password The proxy password. Note: this parameter is not serialized by tiledb_config_save_to_file. Default: “”
  • vfs.hdfs.name_node" Name node for HDFS. Default: “”
  • vfs.hdfs.username HDFS username. Default: “”
  • vfs.hdfs.kerb_ticket_cache_path HDFS kerb ticket cache path. Default: “”

std::string get(const std::string &param) const

Get a parameter from the configuration by key.

Return
Value of configuration parameter
Parameters
  • param: Name of configuration parameter
Exceptions

impl::ConfigProxy operator[](const std::string &param)

Operator that enables setting parameters with [].

Example:

Config conf;
conf["vfs.s3.region"] = "us-east-1a";
conf["vfs.s3.use_virtual_addressing"] = "true";
Context ctx(conf);

Return
”Proxy” object supporting assignment.
Parameters
  • param: Name of parameter to set

Config &unset(const std::string &param)

Resets a config parameter to its default value.

Return
Reference to this Config instance
Parameters
  • param: Name of parameter

iterator begin(const std::string &prefix)

Iterate over params starting with a prefix.

Example:

tiledb::Config config;
for (auto it = config.begin("vfs"), ite = config.end(); it != ite; ++it) {
  std::string name = it->first, value = it->second;
}

Return
iterator
Parameters
  • prefix: Prefix to iterate over

iterator begin()

Iterate over all params.

Example:

tiledb::Config config;
for (auto it = config.begin(), ite = config.end(); it != ite; ++it) {
  std::string name = it->first, value = it->second;
}

Return
iterator

iterator end()

End iterator.

Public Static Functions

static void free(tiledb_config_t *config)

Wrapper function for freeing a config C object.

Exceptions

struct tiledb::TileDBError

Exception indicating a TileDB error.

Inherits from runtime_error

Subclassed by tiledb::AttributeError, tiledb::SchemaMismatch, tiledb::TypeError

struct tiledb::TypeError

Exception indicating a mismatch between a static and runtime type

Inherits from tiledb::TileDBError

struct tiledb::SchemaMismatch

Exception indicating the requested operation does not match array schema

Inherits from tiledb::TileDBError

struct tiledb::AttributeError

Error related to attributes

Inherits from tiledb::TileDBError

Dimension

class tiledb::Dimension

Describes one dimension of an Array. The dimension consists of a type, lower and upper bound, and tile-extent describing the memory ordering. Dimensions are added to a Domain.

Example:

tiledb::Context ctx;
tiledb::Domain domain(ctx);
// Create a dimension with inclusive domain [0,1000] and tile extent 100.
domain.add_dimension(Dimension::create<int32_t>(ctx, "d", {{0, 1000}}, 100));

Public Functions

const std::string name() const

Returns the name of the dimension.

tiledb_datatype_t type() const

Returns the dimension datatype.

template <typename T>
std::pair<T, T> domain() const

Returns the domain of the dimension.

Return
Pair of [lower, upper] inclusive bounds.
Template Parameters

std::string domain_to_str() const

Returns a string representation of the domain.

Exceptions
  • TileDBError: if the domain cannot be stringified (TILEDB_ANY)

template <typename T>
T tile_extent() const

Returns the tile extent of the dimension.

std::string tile_extent_to_str() const

Returns a string representation of the extent.

Exceptions
  • TileDBError: if the domain cannot be stringified (TILEDB_ANY)

std::shared_ptr<tiledb_dimension_t> ptr() const

Returns a shared pointer to the C TileDB dimension object.

operator tiledb_dimension_t *() const

Auxiliary operator for getting the underlying C TileDB object.

Public Static Functions

template <typename T>
static Dimension create(const Context &ctx, const std::string &name, const std::array<T, 2> &domain, T extent)

Factory function for creating a new dimension with datatype T.

Example:

tiledb::Context ctx;
// Create a dimension with inclusive domain [0,1000] and tile extent 100.
auto dim = Dimension::create<int32_t>(ctx, "d", {{0, 1000}}, 100);

Return
A new Dimension object.
Template Parameters
  • T: int, char, etc…
Parameters
  • ctx: The TileDB context.
  • name: The dimension name.
  • domain: The dimension domain. A pair [lower,upper] of inclusive bounds.
  • extent: The tile extent on the dimension.

template <typename T>
static Dimension create(const Context &ctx, const std::string &name, const std::array<T, 2> &domain)

Factory function for creating a new dimension with datatype T with no tile extent. TileDB uses a tile extent equal to the whole domain extent in this dimension.

Example:

tiledb::Context ctx;
// Create a dimension with inclusive domain [0,1000] and implicit
// tile extent 1001.
auto dim = Dimension::create<int32_t>(ctx, "d", {{0, 1000}});

Return
A new Dimension object.
Template Parameters
  • T: int, char, etc…
Parameters
  • ctx: The TileDB context.
  • name: The dimension name.
  • domain: The dimension domain. A pair [lower,upper] of inclusive bounds.

Domain

class tiledb::Domain

Represents the domain of an array.

A Domain defines the set of Dimension objects for a given array. The properties of a Domain derive from the underlying dimensions. A Domain is a component of an ArraySchema.

Example:

Note
The dimension can only be signed or unsigned integral types, as well as floating point for sparse array domains.

tiledb::Context ctx;
tiledb::Domain domain;

// Note the dimension bounds are inclusive.
auto d1 = tiledb::Dimension::create<int>(ctx, "d1", {-10, 10});
auto d2 = tiledb::Dimension::create<uint64_t>(ctx, "d2", {1, 10});
auto d3 = tiledb::Dimension::create<int>(ctx, "d3", {-100, 100});

domain.add_dimension(d1);
domain.add_dimension(d2); // Throws error, all dims must be same type
domain.add_dimension(d3);

domain.cell_num(); // (10 - -10 + 1) * (10 - 1 + 1) = 210 max cells
domain.type(); // TILEDB_INT32, determined from the dimensions
domain.rank(); // 2, d1 and d2

tiledb::ArraySchema schema(ctx, TILEDB_DENSE);
schema.set_domain(domain); // Set the array's domain

Public Functions

uint64_t cell_num() const

Returns the total number of cells in the domain. Throws an exception if the domain type is float32 or float64.

Exceptions

void dump(FILE *out = stdout) const

Dumps the domain in an ASCII representation to an output.

Parameters
  • out: (Optional) File to dump output to. Defaults to stdout.

tiledb_datatype_t type() const

Returns the domain type.

unsigned ndim() const

Returns the number of dimensions.

std::vector<Dimension> dimensions() const

Returns the current set of dimensions in the domain.

Domain &add_dimension(const Dimension &d)

Adds a new dimension to the domain.

Example:

tiledb::Context ctx;
tiledb::Domain domain;
auto d1 = tiledb::Dimension::create<int>(ctx, "d1", {-10, 10});
domain.add_dimension(d1);

Return
Reference to this Domain
Parameters

template <typename… Args>
Domain &add_dimensions(Args... dims)

Adds multiple dimensions to the domain.

Example:

tiledb::Context ctx;
tiledb::Domain domain;
auto d1 = tiledb::Dimension::create<int>(ctx, "d1", {-10, 10});
auto d2 = tiledb::Dimension::create<int>(ctx, "d2", {1, 10});
auto d3 = tiledb::Dimension::create<int>(ctx, "d3", {-100, 100});
domain.add_dimensions(d1, d2, d3);

Return
Reference to this Domain.
Template Parameters
  • Args: Variadic dimension datatype
Parameters
  • dims: Dimensions to add

std::shared_ptr<tiledb_domain_t> ptr() const

Returns a shared pointer to the C TileDB domain object.

operator tiledb_domain_t *() const

Auxiliary operator for getting the underlying C TileDB object.

Attribute

class tiledb::Attribute

Describes an attribute of an Array cell.

An attribute specifies a name and datatype for a particular value in each array cell. There are 3 supported attribute types:

  • Fundamental types, such as char, int, double, uint64_t, etc..
  • Fixed sized arrays: T[N] or std::array<T, N>, where T is a fundamental type
  • Variable length data: std::string, std::vector<T> where T is a fundamental type

Fixed-size array types using POD types like std::array<T, N> are internally converted to byte-array attributes. E.g. an attribute of type std::array<float, 3> will be created as an attribute of type TILEDB_CHAR with cell_val_num sizeof(std::array<float, 3>).

Therefore, for fixed-length attributes it is recommended to use C-style arrays instead, e.g. float[3] instead of std::array<float, 3>.

Example:

tiledb::Context ctx;
auto a1 = tiledb::Attribute::create<int>(ctx, "a1");
auto a2 = tiledb::Attribute::create<std::string>(ctx, "a2");
auto a3 = tiledb::Attribute::create<float[3]>(ctx, "a3");

// Change compression scheme
tiledb::FilterList filters(ctx);
filters.add_filter({ctx, TILEDB_FILTER_BZIP2});
a1.set_filter_list(filters);

// Add attributes to a schema
tiledb::ArraySchema schema(ctx, TILEDB_DENSE);
schema.add_attributes(a1, a2, a3);

Public Functions

Attribute(const Context &ctx, const std::string &name, tiledb_datatype_t type)

Construct an attribute with a name and enumerated type. cell_val_num will be set to 1.

Parameters
  • ctx: TileDB context
  • name: Name of attribute
  • type: Enumerated type of attribute

TILEDB_DEPRECATED Attribute(const Context &ctx, const std::string &name, tiledb_datatype_t type, const Compressor &compressor)

Construct an attribute with an enumerated type and given compressor.

Note
This constructor is deprecated and will be removed in a future version. The filter API should be used instead.

Attribute(const Context &ctx, const std::string &name, tiledb_datatype_t type, const FilterList &filter_list)

Construct an attribute with an enumerated type and given filter list.

std::string name() const

Returns the name of the attribute.

tiledb_datatype_t type() const

Returns the attribute datatype.

uint64_t cell_size() const

Returns the size (in bytes) of one cell on this attribute. For variable-sized attributes returns TILEDB_VAR_NUM.

Example:

tiledb::Context ctx;
auto a1 = tiledb::Attribute::create<int>(ctx, "a1");
auto a2 = tiledb::Attribute::create<std::string>(ctx, "a2");
auto a3 = tiledb::Attribute::create<float[3]>(ctx, "a3");
auto a4 = tiledb::Attribute::create<std::array<float, 3>>(ctx, "a4");
a1.cell_size();    // Returns sizeof(int)
a2.cell_size();    // Variable sized attribute, returns TILEDB_VAR_NUM
a3.cell_size();    // Returns 3 * sizeof(float)
a4.cell_size();    // Stored as byte array, returns sizeof(char).

unsigned cell_val_num() const

Returns number of values of one cell on this attribute. For variable-sized attributes returns TILEDB_VAR_NUM.

Example:

tiledb::Context ctx;
auto a1 = tiledb::Attribute::create<int>(ctx, "a1");
auto a2 = tiledb::Attribute::create<std::string>(ctx, "a2");
auto a3 = tiledb::Attribute::create<float[3]>(ctx, "a3");
auto a4 = tiledb::Attribute::create<std::array<float, 3>>(ctx, "a4");
a1.cell_val_num();   // Returns 1
a2.cell_val_num();   // Variable sized attribute, returns TILEDB_VAR_NUM
a3.cell_val_num();   // Returns 3
a4.cell_val_num();   // Stored as byte array, returns
                        sizeof(std::array<float, 3>).

Attribute &set_cell_val_num(unsigned num)

Sets the number of attribute values per cell. This is inferred from the type parameter of the Attribute::create<T>() function, but can also be set manually.

Example:

// a1 and a2 are equivalent:
auto a1 = Attribute::create<std::vector<int>>(...);
auto a2 = Attribute::create<int>(...);
a2.set_cell_val_num(TILEDB_VAR_NUM);

Return
Reference to this Attribute
Parameters
  • num: Cell val number to set.

bool variable_sized() const

Check if attribute is variable sized.

TILEDB_DEPRECATED Compressor compressor() const

Returns a copy of the attribute compressor. To change the attribute compressor, use Attribute::set_compressor().

Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.

TILEDB_DEPRECATED Attribute &set_compressor(Compressor c)

Sets the attribute compressor.

Return
Reference to this Attribute
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.
Parameters

FilterList filter_list() const

Returns a copy of the FilterList of the attribute. To change the filter list, use set_filter_list().

Return
Copy of the attribute FilterList.

Attribute &set_filter_list(const FilterList &filter_list)

Sets the attribute filter list, which is an ordered list of filters that will be used to process and/or transform the attribute data (such as compression).

Return
Reference to this Attribute
Parameters
  • filter_list: Filter list to set

std::shared_ptr<tiledb_attribute_t> ptr() const

Returns the C TileDB attribute object pointer.

operator tiledb_attribute_t *() const

Auxiliary operator for getting the underlying C TileDB object.

void dump(FILE *out = stdout) const

Dumps information about the attribute in an ASCII representation to an output.

Parameters
  • out: (Optional) File to dump output to. Defaults to stdout.

Public Static Functions

template <typename T>
static Attribute create(const Context &ctx, const std::string &name)

Factory function for creating a new attribute with datatype T.

Example:

tiledb::Context ctx;
auto a1 = tiledb::Attribute::create<int>(ctx, "a1");
auto a2 = tiledb::Attribute::create<std::string>(ctx, "a2");
auto a3 = tiledb::Attribute::create<std::array<float, 3>>(ctx, "a3");
auto a4 = tiledb::Attribute::create<std::vector<double>>(ctx, "a4");
auto a5 = tiledb::Attribute::create<char[8]>(ctx, "a5");

Return
A new Attribute object.
Template Parameters
  • T: Datatype of the attribute. Can either be arithmetic type, C-style array, std::string, std::vector, or any trivially copyable classes (defined by std::is_trivially_copyable).
Parameters
  • ctx: The TileDB context.
  • name: The attribute name.

template <typename T>
TILEDB_DEPRECATED static Attribute create(const Context &ctx, const std::string &name, const Compressor &compressor)

Factory function for creating a new attribute with datatype T and a Compressor.

Example:

tiledb::Context ctx;
auto a1 = tiledb::Attribute::create<int>(ctx, "a1", {TILEDB_BZIP2, -1});

Return
A new Attribute object.
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.
Template Parameters
  • T: Datatype of the attribute. Can either be arithmetic type, C-style array, std::string, std::vector, or any trivially copyable classes (defined by std::is_trivially_copyable).
Parameters
  • ctx: The TileDB context.
  • name: The attribute name.
  • compressor: Compressor to use for attribute

template <typename T>
static Attribute create(const Context &ctx, const std::string &name, const FilterList &filter_list)

Factory function for creating a new attribute with datatype T and a FilterList.

Example:

tiledb::Context ctx;
tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});
auto a1 = tiledb::Attribute::create<int>(ctx, "a1", filter_list);

Return
A new Attribute object.
Template Parameters
  • T: Datatype of the attribute. Can either be arithmetic type, C-style array, std::string, std::vector, or any trivially copyable classes (defined by std::is_trivially_copyable).
Parameters
  • ctx: The TileDB context.
  • name: The attribute name.
  • filter_list: FilterList to use for attribute

Array Schema

class tiledb::ArraySchema

Schema describing an array.

The schema is an independent description of an array. A schema can be used to create multiple array’s, and stores information about its domain, cell types, and compression details. An array schema is composed of:

  • A Domain
  • A set of Attributes
  • Memory layout definitions: tile and cell
  • Compression details for Array level factors like offsets and coordinates

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE); // Or TILEDB_DENSE

// Create a Domain
tiledb::Domain domain(...);

// Create Attributes
auto a1 = tiledb::Attribute::create(...);

schema.set_domain(domain);
schema.add_attribute(a1);

// Specify tile memory layout
schema.set_tile_order(TILEDB_ROW_MAJOR);
// Specify cell memory layout within each tile
schema.set_cell_order(TILEDB_ROW_MAJOR);
schema.set_capacity(10); // For sparse, set capacity of each tile

// Create the array on persistent storage with the schema.
tiledb::Array::create("my_array", schema);

Inherits from tiledb::Schema

Public Functions

ArraySchema(const Context &ctx, tiledb_array_type_t type)

Creates a new array schema.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);

Parameters
  • ctx: TileDB context
  • type: Array type, sparse or dense.

ArraySchema(const Context &ctx, const std::string &uri)

Loads the schema of an existing array.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, "s3://bucket-name/array-name");

Parameters
  • ctx: TileDB context
  • uri: URI of array

ArraySchema(const Context &ctx, const std::string &uri, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Loads the schema of an existing encrypted array.

Example:

// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, "s3://bucket-name/array-name",
   TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • ctx: TileDB context
  • uri: URI of array
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

ArraySchema(const Context &ctx, tiledb_array_schema_t *schema)

Loads the schema of an existing array with the input C array schema object.

Parameters
  • ctx: TileDB context
  • schema: C API array schema object

operator tiledb_array_schema_t *() const

Auxiliary operator for getting the underlying C TileDB object.

void dump(FILE *out = stdout) const

Dumps the array schema in an ASCII representation to an output.

Parameters
  • out: (Optional) File to dump output to. Defaults to stdout.

tiledb_array_type_t array_type() const

Returns the array type.

uint64_t capacity() const

Returns the tile capacity.

ArraySchema &set_capacity(uint64_t capacity)

Sets the tile capacity.

Return
Reference to this ArraySchema instance.
Parameters
  • capacity: Capacity value to set.

tiledb_layout_t tile_order() const

Returns the tile order.

ArraySchema &set_tile_order(tiledb_layout_t layout)

Sets the tile order.

Return
Reference to this ArraySchema instance.
Parameters
  • layout: Tile order to set.

ArraySchema &set_order(const std::array<tiledb_layout_t, 2> &p)

Sets both the tile and cell orders.

Return
Reference to this ArraySchema instance.
Parameters
  • layout: Pair of {tile order, cell order}

tiledb_layout_t cell_order() const

Returns the cell order.

ArraySchema &set_cell_order(tiledb_layout_t layout)

Sets the cell order.

Return
Reference to this ArraySchema instance.
Parameters
  • layout: Cell order to set.

TILEDB_DEPRECATED Compressor coords_compressor() const

Returns a copy of the Compressor of the coordinates. To change the coordinate compressor, use set_coords_compressor().

Return
Copy of the coordinates Compressor.
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.

TILEDB_DEPRECATED ArraySchema &set_coords_compressor(const Compressor &c)

Sets the compressor for the coordinates.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
schema.set_coords_compressor({TILEDB_ZSTD, -1});

Return
Reference to this ArraySchema instance.
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.
Parameters

TILEDB_DEPRECATED Compressor offsets_compressor() const

Returns a copy of the Compressor of the offsets for variable-length attributes. To change the compressor, use set_offsets_compressor().

Return
Copy of the offsets Compressor.
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.

TILEDB_DEPRECATED ArraySchema &set_offsets_compressor(const Compressor &c)

Sets the compressor for the offsets of variable-length attributes.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
schema.set_offsets_compressor({TILEDB_ZSTD, -1});

Return
Reference to this ArraySchema instance.
Note
This function is deprecated and will be removed in a future version. The filter API should be used instead.
Parameters

FilterList coords_filter_list() const

Returns a copy of the FilterList of the coordinates. To change the coordinate compressor, use set_coords_filter_list().

Return
Copy of the coordinates FilterList.

ArraySchema &set_coords_filter_list(const FilterList &filter_list)

Sets the FilterList for the coordinates, which is an ordered list of filters that will be used to process and/or transform the coordinate data (such as compression).

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});
schema.set_coords_filter_list(filter_list);

Return
Reference to this ArraySchema instance.
Parameters

FilterList offsets_filter_list() const

Returns a copy of the FilterList of the offsets. To change the offsets compressor, use set_offsets_filter_list().

Return
Copy of the offsets FilterList.

ArraySchema &set_offsets_filter_list(const FilterList &filter_list)

Sets the FilterList for the offsets, which is an ordered list of filters that will be used to process and/or transform the offsets data (such as compression).

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_POSITIVE_DELTA})
    .add_filter({ctx, TILEDB_FILTER_LZ4});
schema.set_offsets_filter_list(filter_list);

Return
Reference to this ArraySchema instance.
Parameters

Domain domain() const

Returns a copy of the schema’s array Domain. To change the domain, use set_domain().

Return
Copy of the array Domain

ArraySchema &set_domain(const Domain &domain)

Sets the array domain.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
// Create a Domain
tiledb::Domain domain(...);
schema.set_domain(domain);

Return
Reference to this ArraySchema instance.
Parameters

ArraySchema &add_attribute(const Attribute &attr)

Adds an Attribute to the array.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
schema.add_attribute(Attribute::create<int32_t>(ctx, "attr_name"));

Return
Reference to this ArraySchema instance.
Parameters

std::shared_ptr<tiledb_array_schema_t> ptr() const

Returns a shared pointer to the C TileDB domain object.

void check() const

Validates the schema.

Example:

tiledb::Context ctx;
tiledb::ArraySchema schema(ctx, TILEDB_SPARSE);
// Add domain, attributes, etc...

try {
  schema.check();
} catch (const tiledb::TileDBError& e) {
  std::cout << e.what() << "\n";
  exit(1);
}

Exceptions

std::unordered_map<std::string, Attribute> attributes() const

Gets all attributes in the array.

Return
Map of attribute name to copy of Attribute instance.

Attribute attribute(const std::string &name) const

Get a copy of an Attribute in the schema by name.

Return
Attribute
Parameters
  • name: Name of attribute

unsigned attribute_num() const

Returns the number of attributes in the schema.

Attribute attribute(unsigned int i) const

Get a copy of an Attribute in the schema by index. Attributes are ordered the same way they were defined when constructing the array schema.

Return
Attribute
Parameters
  • i: Index of attribute

Public Static Functions

static std::string to_str(tiledb_array_type_t type)

Returns the input array type in string format.

static std::string to_str(tiledb_layout_t layout)

Returns the input layout in string format.

Array

class tiledb::Array

Class representing a TileDB array object.

An Array object represents array data in TileDB at some persisted location, e.g. on disk, in an S3 bucket, etc. Once an array has been opened for reading or writing, interact with the data through Query objects.

Example:

tiledb::Context ctx;

// Create an ArraySchema, add attributes, domain, etc.
tiledb::ArraySchema schema(...);

// Create empty array named "my_array" on persistent storage.
tiledb::Array::create("my_array", schema);

Public Functions

Array(const Context &ctx, const std::string &array_uri, tiledb_query_type_t query_type)

Constructor. This opens the array for the given query type. The destructor calls the close() method.

Example:

// Open the array for reading
tiledb::Context ctx;
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);

Parameters
  • ctx: TileDB context.
  • array_uri: The array URI.
  • query_type: Query type to open the array for.

Array(const Context &ctx, const std::string &array_uri, tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Constructor. This opens an encrypted array for the given query type. The destructor calls the close() method.

Example:

// Open the encrypted array for reading
tiledb::Context ctx;
// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ,
   TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • ctx: TileDB context.
  • array_uri: The array URI.
  • query_type: Query type to open the array for.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

Array(const Context &ctx, const std::string &array_uri, tiledb_query_type_t query_type, uint64_t timestamp)

Constructor. This opens the array for the given query type. The destructor calls the close() method.

This constructor takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

Example:

// Open the array for reading
tiledb::Context ctx;
// Get some `timestamp` here in milliseconds
tiledb::Array array(
    ctx, "s3://bucket-name/array-name", TILEDB_READ, timestamp);

Parameters
  • ctx: TileDB context.
  • array_uri: The array URI.
  • query_type: Query type to open the array for.
  • timestamp: The timestamp to open the array at.

Array(const Context &ctx, const std::string &array_uri, tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length, uint64_t timestamp)

Constructor. This opens an encrypted array for the given query type. The destructor calls the close() method.

This constructor takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

Example:

// Open the encrypted array for reading
tiledb::Context ctx;
// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
// Get some `timestamp` here in milliseconds
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ,
   TILEDB_AES_256_GCM, key, sizeof(key), timestamp);

Parameters
  • ctx: TileDB context.
  • array_uri: The array URI.
  • query_type: Query type to open the array for.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.
  • timestamp: The timestamp to open the array at.

~Array()

Destructor; calls close().

bool is_open() const

Checks if the array is open.

std::string uri() const

Returns the array URI.

ArraySchema schema() const

Get the ArraySchema for the array.

std::shared_ptr<tiledb_array_t> ptr() const

Returns a shared pointer to the C TileDB array object.

operator tiledb_array_t *() const

Auxiliary operator for getting the underlying C TileDB object.

void open(tiledb_query_type_t query_type)

Opens the array. The array is opened using a query type as input. This is to indicate that queries created for this Array object will inherit the query type. In other words, Array objects are opened to receive only one type of queries. They can always be closed and be re-opened with another query type. Also there may be many different Array objects created and opened with different query types. For instance, one may create and open an array object array_read for reads and another one array_write for writes, and interleave creation and submission of queries for both these array objects.

Example:

// Open the array for writing
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_WRITE);
// Close and open again for reading.
array.close();
array.open(TILEDB_READ);

Parameters
  • query_type: The type of queries the array object will be receiving.
Exceptions
  • TileDBError: if the array is already open or other error occurred.

void open(tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Opens the array, for encrypted arrays.

Example:

// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
// Open the encrypted array for writing
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_WRITE,
   TILEDB_AES_256_GCM, key, sizeof(key));
// Close and open again for reading.
array.close();
array.open(TILEDB_READ, TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • query_type: The type of queries the array object will be receiving.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

void open(tiledb_query_type_t query_type, uint64_t timestamp)

Opens the array. The array is opened using a query type as input. This is to indicate that queries created for this Array object will inherit the query type. In other words, Array objects are opened to receive only one type of queries. They can always be closed and be re-opened with another query type. Also there may be many different Array objects created and opened with different query types. For instance, one may create and open an array object array_read for reads and another one array_write for writes, and interleave creation and submission of queries for both these array objects.

This function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

Example:

// Open the array for writing
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_WRITE);
// Close and open again for reading.
array.close();
// Get some `timestamp` in milliseconds here
array.open(TILEDB_READ, timestamp);

Parameters
  • query_type: The type of queries the array object will be receiving.
  • timestamp: The timestamp to open the array at.
Exceptions
  • TileDBError: if the array is already open or other error occurred.

void open(tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length, uint64_t timestamp)

Opens the array, for encrypted arrays.

This function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the array at a timestamp provides a view of the array with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the array.

Example:

// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
// Open the encrypted array for writing
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_WRITE,
   TILEDB_AES_256_GCM, key, sizeof(key));
// Close and open again for reading.
array.close();
// Get some `timestamp` in milliseconds here
array.open(TILEDB_READ, TILEDB_AES_256_GCM, key, sizeof(key), timestamp);

Parameters
  • query_type: The type of queries the array object will be receiving.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.
  • timestamp: The timestamp to open the array at.

void reopen()

Reopens the array (the array must be already open). This is useful when the array got updated after it got opened and the Array object got created. To sync-up with the updates, the user must either close the array and open with open(), or just use reopen() without closing. This function will be generally faster than the former alternative.

Note: reopening encrypted arrays does not require the encryption key.

Example:

// Open the array for reading
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);
array.reopen();

Exceptions
  • TileDBError: if the array was not already open or other error occurred.

void reopen_at(uint64_t timestamp)

Reopens the array at a specific timestamp.

Example:

// Open the array for reading
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);
uint64_t timestamp = tiledb_timestamp_now_ms();
array.reopen_at(timestamp);

Exceptions
  • TileDBError: if the array was not already open or other error occurred.

uint64_t timestamp() const

Returns the timestamp at which the array was opened.

void close()

Closes the array. The destructor calls this automatically.

Example:

tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);
array.close();

template <typename T>
std::vector<std::pair<std::string, std::pair<T, T>>> non_empty_domain()

Retrieves the non-empty domain from the array. This is the union of the non-empty domains of the array fragments.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);
// Specify the domain type (example uint32_t)
auto non_empty = array.non_empty_domain<uint32_t>();
std::cout << "Dimension named " << non_empty[0].first << " has cells in ["
          << non_empty[0].second.first << ", " non_empty[0].second.second
          << "]" << std::endl;

Return
Vector of dim names with a {lower, upper} pair. Inclusive. Empty vector if the array has no data.
Template Parameters

template <typename T>
std::unordered_map<std::string, std::pair<uint64_t, uint64_t>> max_buffer_elements(const std::vector<T> &subarray)

Compute an upper bound on the buffer elements needed to read a subarray.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, "s3://bucket-name/array-name", TILEDB_READ);
std::vector<int> subarray = {0, 2, 0, 2};
auto max_elements = array.max_buffer_elements(subarray);

// For fixed-sized attributes, `.second` is the max number of elements
// that can be read for the attribute. Use it to size the vector.
std::vector<int> data_a1(max_elements["a1"].second);

// In sparse reads, coords are also fixed-sized attributes.
std::vector<int> coords(max_elements[TILEDB_COORDS].second);

// In variable attributes, e.g. std::string type, need two buffers,
// one for offsets and one for cell data.
std::vector<uint64_t> offsets_a1(max_elements["a2"].first);
std::vector<char> data_a1(max_elements["a2"].second);

Return
A map of attribute name (including TILEDB_COORDS) to the maximum number of elements that can be read in the given subarray. For each attribute, a pair of numbers are returned. The first, for variable-length attributes, is the maximum number of offsets for that attribute in the given subarray. For fixed-length attributes and coordinates, the first is always 0. The second is the maximum number of elements for that attribute in the given subarray.
Template Parameters
  • T: The domain datatype
Parameters
  • subarray: Targeted subarray.

tiledb_query_type_t query_type() const

Returns the query type the array was opened with.

Public Static Functions

static void consolidate(const Context &ctx, const std::string &uri)

Consolidates the fragments of an array into a single fragment.

You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

Example:

tiledb::Array::consolidate(ctx, "s3://bucket-name/array-name");

Parameters
  • ctx: TileDB context
  • array_uri: The URI of the TileDB array to be consolidated.

static void consolidate(const Context &ctx, const std::string &uri, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Consolidates the fragments of an encrypted array into a single fragment.

You must first finalize all queries to the array before consolidation can begin (as consolidation temporarily acquires an exclusive lock on the array).

Example:

// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
tiledb::Array::consolidate(ctx, "s3://bucket-name/array-name",
   TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • ctx: TileDB context
  • array_uri: The URI of the TileDB array to be consolidated.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

static void create(const std::string &uri, const ArraySchema &schema)

Creates a new TileDB array given an input schema.

Example:

tiledb::Array::create("s3://bucket-name/array-name", schema);

Parameters
  • uri: URI where array will be created.
  • schema: The array schema.

static void create(const std::string &uri, const ArraySchema &schema, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Creates a new encrypted TileDB array given an input schema.

Example:

// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
tiledb::Array::create("s3://bucket-name/array-name", schema,
   TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • uri: URI where array will be created.
  • schema: The array schema.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

Query

class tiledb::Query

Construct and execute read/write queries on a tiledb::Array.

See examples for more usage details.

Example:

// Open the array for writing
tiledb::Context ctx;
tiledb::Array array(ctx, "my_dense_array", TILEDB_WRITE);
Query query(ctx, array);
query.set_layout(TILEDB_GLOBAL_ORDER);
std::vector a1_data = {1, 2, 3};
query.set_buffer("a1", a1_data);
query.submit();
query.finalize();
array.close();

Public Types

enum Status

The query or query attribute status.

Values:

FAILED

Query failed.

COMPLETE

Query completed (all data has been read)

INPROGRESS

Query is in progress

INCOMPLETE

Query completed (but not all data has been read)

UNINITIALIZED

Query not initialized.

Public Functions

Query(const Context &ctx, const Array &array, tiledb_query_type_t type)

Creates a TileDB query object.

The query type (read or write) must be the same as the type used to open the array object.

The storage manager also acquires a shared lock on the array. This means multiple read and write queries to the same array can be made concurrently (in TileDB, only consolidation requires an exclusive lock for a short period of time).

Example:

// Open the array for writing
tiledb::Context ctx;
tiledb::Array array(ctx, "my_array", TILEDB_WRITE);
Query query(ctx, array, TILEDB_WRITE);

Parameters
  • ctx: TileDB context
  • array: Open Array object
  • type: The TileDB query type

Query(const Context &ctx, const Array &array)

Creates a TileDB query object.

The query type (read or write) is inferred from the array object, which was opened with a specific query type.

The storage manager also acquires a shared lock on the array. This means multiple read and write queries to the same array can be made concurrently (in TileDB, only consolidation requires an exclusive lock for a short period of time).

Example:

// Open the array for writing
tiledb::Context ctx;
tiledb::Array array(ctx, "my_array", TILEDB_WRITE);
Query query(ctx, array);
// Equivalent to:
// Query query(ctx, array, TILEDB_WRITE);

Parameters
  • ctx: TileDB context
  • array: Open Array object

tiledb_query_type_t query_type() const

Returns the query type (read or write).

Query &set_layout(tiledb_layout_t layout)

Sets the layout of the cells to be written or read.

Return
Reference to this Query
Parameters
  • layout: For a write query, this specifies the order of the cells provided by the user in the buffers. For a read query, this specifies the order of the cells that will be retrieved as results and stored in the user buffers. The layout can be one of the following:
    • TILEDB_COL_MAJOR: This means column-major order with respect to the subarray.
    • TILEDB_ROW_MAJOR: This means row-major order with respect to the subarray.
    • TILEDB_GLOBAL_ORDER: This means that cells are stored or retrieved in the array global cell order.
    • TILEDB_UNORDERED: This is applicable only to writes for sparse arrays, or for sparse writes to dense arrays. It specifies that the cells are unordered and, hence, TileDB must sort the cells in the global cell order prior to writing.

Status query_status() const

Returns the query status.

bool has_results() const

Returns true if the query has results. Applicable only to read queries (it returns false for write queries).

Status submit()

Submits the query. Call will block until query is complete.

Note
finalize() must be invoked after finish writing in global layout (via repeated invocations of submit()), in order to flush any internal state. For the case of reads, if the returned status is TILEDB_INCOMPLETE, TileDB could not fit the entire result in the user’s buffers. In this case, the user should consume the read results (if any), optionally reset the buffers with set_buffer(), and then resubmit the query until the status becomes TILEDB_COMPLETED. If all buffer sizes after the termination of this function become 0, then this means that no useful data was read into the buffers, implying that the larger buffers are needed for the query to proceed. In this case, the users must reallocate their buffers (increasing their size), reset the buffers with set_buffer(), and resubmit the query.
Return
Query status

template <typename Fn>
void submit_async(const Fn &callback)

Submit an async query, with callback. Call returns immediately.

Example:

// Create query
tiledb::Query query(...);
// Submit with callback
query.submit_async([]() { std::cout << "Callback: query completed.\n"; });
Note
Same notes apply as Query::submit().

Parameters
  • callback: Callback function.

void submit_async()

Submit an async query, with no callback. Call returns immediately.

Example:

// Create query
tiledb::Query query(...);
// Submit with no callback
query.submit_async();
Note
Same notes apply as Query::submit().

void finalize()

Flushes all internal state of a query object and finalizes the query. This is applicable only to global layout writes. It has no effect for any other query type.

std::unordered_map<std::string, std::pair<uint64_t, uint64_t>> result_buffer_elements() const

Returns the number of elements in the result buffers from a read query. This is a map from the attribute name to a pair of values.

The first is number of elements for var size attributes, and the second is number of elements in the data buffer. For fixed sized attributes (and coordinates), the first is always 0.

For variable sized attributes: the first value is the number of cells read, i.e. the number of offsets read for the attribute. The second value is the total number of elements in the data buffer. For example, a read query on a variable-length float attribute that reads three cells would return 3 for the first number in the pair. If the total amount of floats read across the three cells was 10, then the second number in the pair would be 10 (totaling 10*sizeof(float) bytes of data).

For fixed-length attributes, the first value is always 0. The second value is the total number of elements in the data buffer. For example, a read query on a single float attribute that reads three cells would return 3 for the second value. A read query on a float attribute with cell_val_num 2 that reads three cells would return 3 * 2 = 6 for the second value.

If the query has not been submitted, an empty map is returned.

Example:

// Submit a read query.
query.submit();
auto result_el = query.result_buffer_elements();

// For fixed-sized attributes, `.second` is the number of elements
// that were read for the attribute across all cells. Note: number of
// elements
auto num_a1_elements = result_el["a1"].second;

// Coords are also fixed-sized.
auto num_coords = result_el[TILEDB_COORDS].second;

// In variable attributes, e.g. std::string type, need two buffers,
// one for offsets and one for cell data ("elements").
auto num_a2_offsets = result_el["a2"].first;
auto num_a2_elements = result_el["a2"].second;

template <typename T = uint64_t>
Query &set_subarray(const T *pairs, uint64_t size)

Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive. For the case of writes, this is meaningful only for dense arrays, and specifically dense writes.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_READ);
int subarray[] = {0, 3, 0, 3};
Query query(ctx, array);
query.set_subarray(subarray, 4);
Note
set_subarray(std::vector<T>) is preferred as it is safer.

Template Parameters
  • T: Type of array domain.
Parameters
  • pairs: Subarray pointer defined as an array of [start, stop] values per dimension.
  • size: The number of subarray elements.

template <typename Vec>
Query &set_subarray(const Vec &pairs)

Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive. For the case of writes, this is meaningful only for dense arrays, and specifically dense writes.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_READ);
std::vector<int> subarray = {0, 3, 0, 3};
Query query(ctx, array);
query.set_subarray(subarray);

Template Parameters
  • Vec: Vector datatype. Should always be a vector of the domain type.
Parameters
  • pairs: The subarray defined as a vector of [start, stop] coordinates per dimension.

template <typename T = uint64_t>
Query &set_subarray(const std::initializer_list<T> &l)

Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive. For the case of writes, this is meaningful only for dense arrays, and specifically dense writes.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_READ);
Query query(ctx, array);
query.set_subarray({0, 3, 0, 3});

Template Parameters
  • T: Type of array domain.
Parameters
  • pairs: List of [start, stop] coordinates per dimension.

template <typename T = uint64_t>
Query &set_subarray(const std::vector<std::array<T, 2>> &pairs)

Sets a subarray, defined in the order dimensions were added. Coordinates are inclusive.

Note
set_subarray(std::vector) is preferred and avoids an extra copy.
Template Parameters
  • T: Type of array domain.
Parameters
  • pairs: The subarray defined as pairs of [start, stop] per dimension.

template <typename T>
Query &set_coordinates(T *buf, uint64_t size)

Set the coordinate buffer.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
// Write to points (0,1) and (2,3) in a 2D array with int domain.
int coords[] = {0, 1, 2, 3};
Query query(ctx, array);
query.set_layout(TILEDB_UNORDERED).set_coordinates(coords, 4);

Note
set_coordinates(std::vector<T>) is preferred as it is safer.
Template Parameters
  • T: Type of array domain.
Parameters
  • buf: Coordinate array buffer pointer
  • size: The number of elements in the coordinate array buffer

template <typename Vec>
Query &set_coordinates(Vec &buf)

Set the coordinate buffer for unordered queries

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
// Write to points (0,1) and (2,3) in a 2D array with int domain.
std::vector<int> coords = {0, 1, 2, 3};
Query query(ctx, array);
query.set_layout(TILEDB_UNORDERED).set_coordinates(coords);

Template Parameters
  • Vec: Vector datatype. Should always be a vector of the domain type.
Parameters
  • buf: Coordinate vector

template <typename T>
Query &set_buffer(const std::string &attr, T *buff, uint64_t nelements)

Sets a buffer for a fixed-sized attribute.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
int data_a1[] = {0, 1, 2, 3};
Query query(ctx, array);
query.set_buffer("a1", data_a1, 4);

Note
set_buffer(std::string, std::vector) is preferred as it is safer.
Template Parameters
Parameters
  • attr: Attribute name
  • buff: Buffer array pointer with elements of the attribute type.
  • nelements: Number of array elements

template <typename Vec>
Query &set_buffer(const std::string &attr, Vec &buf)

Sets a buffer for a fixed-sized attribute.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
std::vector<int> data_a1 = {0, 1, 2, 3};
Query query(ctx, array);
query.set_buffer("a1", data_a1);

Template Parameters
  • Vec: buffer. Should always be a vector type of the attribute type.
Parameters
  • attr: Attribute name
  • buf: Buffer vector with elements of the attribute type.

template <typename T>
Query &set_buffer(const std::string &attr, uint64_t *offsets, uint64_t offset_nelements, T *data, uint64_t data_nelements)

Sets a buffer for a variable-sized attribute.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
int data_a1[] = {0, 1, 2, 3};
uint64_t offsets_a1[] = {0, 8};
Query query(ctx, array);
query.set_buffer("a1", offsets_a1, 2, data_a1, 4);

Note
set_buffer(std::string, std::vector, std::vector) is preferred as it is safer.
Template Parameters
Parameters
  • attr: Attribute name
  • offsets: Offsets array pointer where a new element begins in the data buffer.
  • offsets_nelements: Number of elements in offsets buffer.
  • data: Buffer array pointer with elements of the attribute type. For variable sized attributes, the buffer should be flattened.
  • data_nelements: Number of array elements in data buffer.

template <typename Vec>
Query &set_buffer(const std::string &attr, std::vector<uint64_t> &offsets, Vec &data)

Sets a buffer for a variable-sized attribute.

Example:

tiledb::Context ctx;
tiledb::Array array(ctx, array_name, TILEDB_WRITE);
std::vector<int> data_a1 = {0, 1, 2, 3};
std::vector<uint64_t> offsets_a1 = {0, 8};
Query query(ctx, array);
query.set_buffer("a1", offsets_a1, data_a1);

Template Parameters
  • Vec: buffer type. Should always be a vector of the attribute type.
Parameters
  • attr: Attribute name
  • offsets: Offsets where a new element begins in the data buffer.
  • data: Buffer vector with elements of the attribute type. For variable sized attributes, the buffer should be flattened. E.x. an attribute of type std::string should have a buffer Vec type of std::string, where the values of each cell are concatenated.

template <typename Vec>
Query &set_buffer(const std::string &attr, std::pair<std::vector<uint64_t>, Vec> &buf)

Sets a buffer for a variable-sized attribute.

Template Parameters
  • Vec: buffer type. Should always be a vector of the attribute type.
Parameters
  • attr: Attribute name
  • buf: pair of offset, data buffers

Public Static Functions

static Status to_status(const tiledb_query_status_t &status)

Converts the TileDB C query status to a C++ query status.

static std::string to_str(tiledb_query_type_t type)

Converts the TileDB C query type to a string representation.

Filter

class tiledb::Filter

Represents a filter. A filter is used to transform attribute data e.g. with compression, delta encoding, etc.

Example:

tiledb::Context ctx;
tiledb::Filter f(ctx, TILEDB_FILTER_ZSTD);
int level = 5;
f.set_option(TILEDB_COMPRESSION_LEVEL, &level);

Public Functions

Filter(const Context &ctx, tiledb_filter_type_t filter_type)

Creates a Filter of the given type.

Example:

tiledb::Context ctx;
tiledb::Filter f(ctx, TILEDB_FILTER_ZSTD);

Parameters
  • ctx: TileDB context
  • filter_type: Enumerated type of filter

Filter(const Context &ctx, tiledb_filter_t *filter)

Creates a Filter with the input C object.

Parameters
  • ctx: TileDB context
  • filter: C API filter object

operator tiledb_filter_t *() const

Auxiliary operator for getting the underlying C TileDB object.

std::shared_ptr<tiledb_filter_t> ptr() const

Returns a shared pointer to the C TileDB domain object.

Filter &set_option(tiledb_filter_option_t option, const void *value)

Sets an option on the filter. Options are filter dependent; this function throws an error if the given option is not valid for the given filter.

Example:

tiledb::Filter f(ctx, TILEDB_FILTER_ZSTD);
int level = 5;
f.set_option(TILEDB_COMPRESSION_LEVEL, &level);

Return
Reference to this Filter
Parameters
  • option: Enumerated option to set.
  • value: Value of option to set.
Exceptions
  • TileDBError: if the option cannot be set on the filter.

void get_option(tiledb_filter_option_t option, void *value)

Gets an option value from the filter.

Example:

tiledb::Filter f(ctx, TILEDB_FILTER_ZSTD);
int32_t level;
f.get_option(TILEDB_COMPRESSION_LEVEL, &level);
// level == -1 (the default compression level)

Note
The buffer pointed to by value must be large enough to hold the option value.
Parameters
  • option: Enumerated option to get.
  • value: Buffer that option value will be written to.
Exceptions
  • TileDBError: if the option cannot be retrieved from the filter.

tiledb_filter_type_t filter_type() const

Gets the filter type of this filter.

Public Static Functions

static std::string to_str(tiledb_filter_type_t type)

Returns the input type in string format.

Filter List

class tiledb::FilterList

Represents an ordered list of Filters used to transform attribute data.

Example:

tiledb::Context ctx;
tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});

Public Functions

FilterList(const Context &ctx)

Construct a FilterList.

Example:

tiledb::Context ctx;
tiledb::FilterList filter_list(ctx);

Parameters
  • ctx: TileDB context

FilterList(const Context &ctx, tiledb_filter_list_t *filter_list)

Creates a FilterList with the input C object.

Parameters
  • ctx: TileDB context
  • filter: C API filter list object

operator tiledb_filter_list_t *() const

Auxiliary operator for getting the underlying C TileDB object.

std::shared_ptr<tiledb_filter_list_t> ptr() const

Returns a shared pointer to the C TileDB domain object.

FilterList &add_filter(const Filter &filter)

Appends a filter to a filter list. Data is processed through each filter in the order the filters were added.

Example:

tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});

Return
Reference to this FilterList
Parameters
  • filter: The filter to add

Filter filter(uint32_t filter_index) const

Returns a copy of the Filter in this list at the given index.

Example:

tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});
auto f = filter_list.filter(1);
// f.filter_type() == TILEDB_FILTER_BZIP2

Return
Filter
Parameters
  • filter_index: Index of filter to get
Exceptions

uint32_t max_chunk_size() const

Gets the maximum tile chunk size for the filter list.

Return
Maximum tile chunk size

uint32_t nfilters() const

Returns the number of filters in this filter list.

Example:

tiledb::FilterList filter_list(ctx);
filter_list.add_filter({ctx, TILEDB_FILTER_BYTESHUFFLE})
    .add_filter({ctx, TILEDB_FILTER_BZIP2});
uint32_t n = filter_list.nfilters();  // n == 2

Return

FilterList &set_max_chunk_size(uint32_t max_chunk_size)

Sets the maximum tile chunk size for the filter list.

Return
Reference to this FilterList
Parameters
  • max_chunk_size: Maximum tile chunk size to set

Compressor

class tiledb::Compressor

Represents a compression scheme. Composed of a compression algorithm + a compression level. A compression level of -1 indicates the default level.

Example:

auto a1 = tiledb::Attribute::create<int>(ctx, "a1");
a1.set_compressor({TILEDB_BZIP2, -1});

Note
This class is deprecated and will be removed in a future version. The filter API should be used instead.

Public Functions

Compressor(tiledb_compressor_t compressor, int level)

Create a compressor with a given algorithm and level.

Parameters
  • compressor: Enumerated compression algorithm.
  • level: Compression level (-1 for the compression algorithm’s default).

tiledb_compressor_t compressor() const

Returns the compressor algorithm.

int level() const

Returns the compression level.

Public Static Functions

static std::string to_str(tiledb_compressor_t c)

Converts the input compressor type to a string.

static tiledb_filter_type_t to_filter(tiledb_compressor_t c)

Converts the input compressor type to a filter type.

Group

void tiledb::create_group(const Context &ctx, const std::string &group)

Creates a new group. A Group is a logical grouping of Objects on the storage system (a directory).

Return
void
Parameters
  • ctx: The TileDB context.
  • group: The group URI.

Map

class tiledb::Map

A Map is a key-value store backed by a TileDB sparse array. A Map supports multiple key types and the value is defined by the set of attributes in a MapSchema.

Example:

// Make the map
MapSchema schema(ctx);
schema.add_attribute(Attribute::create<int>(ctx, "a1"));
schema.add_attribute(Attribute::create<std::string>(ctx, "a2"));
schema.add_attribute(Attribute::create<std::array<float, 2>>(ctx, "a3"));
Map::create("my_map", schema);

// Open the map and write to it.
tiledb::Map map(ctx, "my_map");
std::vector<double> key = {2345.1, 345.2};

// Attribute values
int t1 = 3;
std::string t2{"ccc"};
std::array<float, 2> t3({{3.1f, 3.2f}});

map[key][{"a1", "a2", "a3"}] = std::make_tuple(t1, t2, t3);
map.flush(); // Flush to disk

// Read value from map
std::tuple<int, std::string, std::array<float, 2>> vals = map[key];

// Close map
map.close();

Public Functions

Map(const Context &ctx, const std::string &uri, tiledb_query_type_t query_type)

Load an existing map for reading/writing.

Parameters
  • ctx: TileDB context
  • uri: URI of map to open.
  • query_type: The mode in which the map is opened (for reads or writes).

Map(const Context &ctx, const std::string &uri, tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Load an existing encrypted map for reading/writing.

Parameters
  • ctx: TileDB context
  • uri: URI of map to open.
  • query_type: The mode in which the map is opened (for reads or writes).
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

Map(const Context &ctx, const std::string &uri, tiledb_query_type_t query_type, uint64_t timestamp)

Load an existing map for reading/writing.

This constructor takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the map at a timestamp provides a view of the map with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the map.

Parameters
  • ctx: TileDB context
  • uri: URI of map to open.
  • query_type: The mode in which the map is opened (for reads or writes).
  • timestamp: The timestamp to open the map at.

Map(const Context &ctx, const std::string &uri, tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length, uint64_t timestamp)

Load an existing encrypted map for reading/writing.

This constructor takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the map at a timestamp provides a view of the map with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the map.

Parameters
  • ctx: TileDB context
  • uri: URI of map to open.
  • query_type: The mode in which the map is opened (for reads or writes).
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.
  • timestamp: The timestamp to open the map at.

~Map()

Destructor. Calls close() on this map.

template <typename T>
bool has_key(const T &key)

Check if a key is in the map.

Example:

// Load a map
tiledb::Map map(...);
std::vector<double> key = {2345.1, 345.2};
bool has_key = map.has_key(key);

Return
True if the key is in the map.
Template Parameters
  • T: Key type
Parameters
  • key: Key to check

template <typename T>
MapItem get_item(const T &key)

Returns a MapItem from the map corresponding to the given key.

Example:

// Load a map
tiledb::Map map(...);
std::vector<double> key = {2345.1, 345.2};
auto item = map.get_item(key);

Return
The item
Template Parameters
  • T: Key type
Parameters
  • key: Key of item to retrieve
Exceptions

template <typename T>
MapItem operator[](const T &key)

Get an item with a given key. If the item doesn’t exist, it is created.

Example:

// Load a map
tiledb::Map map(...);
std::vector<double> key = {2345.1, 345.2};
auto item = map[key];

Return
The item
Template Parameters
  • T: Key type
Parameters
  • key: Item key

Map &add_item(const MapItem &item)

Add the given item to the map. The item is buffered internally and periodically flushed to persistent storage. Map::flush() forces flushing the buffered items to storage.

Return
Reference to this Map
Parameters
  • item:

void flush()

Flush any buffered items to storage.

const MapSchema &schema() const

Get the schema of the map.

const Context &context() const

Get the underlying context.

const std::string &uri() const

Get the map URI

void open(tiledb_query_type_t query_type)

Opens the Map, preparing it for reading/writing. This is called automatically by the constructor.

Parameters
  • query_type: The type of queries the Map will be receiving.

void open(tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Opens the Map, for encrypted Maps.

Parameters
  • query_type: The type of queries the Map will be receiving.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

void open(tiledb_query_type_t query_type, uint64_t timestamp)

Opens the Map, preparing it for reading/writing. This is called automatically by the constructor.

This function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the map at a timestamp provides a view of the map with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the map.

Parameters
  • query_type: The type of queries the Map will be receiving.
  • timestamp: The timestamp to open the map at.

void open(tiledb_query_type_t query_type, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length, uint64_t timestamp)

Opens the Map, for encrypted Maps.

This function takes as input a timestamp, representing time in milliseconds ellapsed since 1970-01-01 00:00:00 +0000 (UTC). Opening the map at a timestamp provides a view of the map with all writes/updates that happened at or before timestamp (i.e., excluding those that occurred after timestamp). This is useful to ensure consistency at a potential distributed setting, where machines need to operate on the same view of the map.

Parameters
  • query_type: The type of queries the Map will be receiving.
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.
  • timestamp: The timestamp to open the map at.

bool is_open() const

Checks if the Map is open.

void reopen()

Reopens the Map. This is useful when there were updates to the Map after it got opened. This function reopens the Map so that it can “see” the new fragments.

Note: reopening encrypted Maps does not require the encryption key.

Example:

// Load a map
tiledb::Map map(...);
// Some updates to 'map' here, then reopen.
map.reopen();

void reopen_at(uint64_t timestamp)

Reopens the Map at a specific timestamp.

Example:

// Load a map
tiledb::Map map(...);
uint64_t timestamp = tiledb_timestamp_now_ms();
map.reopen_at(timestamp);

uint64_t timestamp() const

Returns the timestamp at which the Map was opened.

void close()

Close the map. All buffered written items will be flushed to persistent storage. This is called automatically by the Map destructor.

std::shared_ptr<tiledb_kv_t> ptr() const

Returns a shared pointer to the C TileDB kv object.

bool is_dirty()

Checks if the Map is dirty, i.e., if the user added items to the Map that are buffered in main-memory and have not been flushed to persistent storage.

Public Static Functions

template <typename T>
static MapItem create_item(const Context &ctx, const T &key)

Create a map item with the given key. Once populated with attributes, it can be added to a Map with map.add_item().

Example:

std::vector<double> key = {2345.1, 345.2};
auto item = Map::create_item(ctx, key);
item.set("a1", 123);

static void create(const std::string &uri, const MapSchema &schema)

Create a new empty map at the given URI with the given schema. Example:

// Make the map
tiledb::MapSchema schema(ctx);
schema.add_attribute(Attribute::create<T>(...));
Map::create("my_map", schema);

Parameters
  • uri: URI where the map will be created
  • schema: Schema for the map

static void create(const std::string &uri, const MapSchema &schema, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Create a new encrypted empty map at the given URI with the given schema. Example:

// Make the map
tiledb::MapSchema schema(ctx);
schema.add_attribute(Attribute::create<T>(...));
// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
Map::create("my_map", schema, TILEDB_AES_256_GCM, key, sizeof(key));

Parameters
  • uri: URI where the map will be created
  • schema: Schema for the map
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

template <typename MapT, typename Key = typename MapT::key_type, typename Value = typename MapT::mapped_type>
static void create(const Context &ctx, const std::string &uri, const MapT &map, const std::string &attr_name)

Create a TileDB map from a std::map. The resulting TileDB map will be accessible as map[key][attr_name].

Example:

// Create map
std::map<int, std::string> map;
map[0] = "0";
map[1] = "12";
Map::create(ctx, "map_name", map, "attr");
// Load map and read items
Map map2(ctx, "map_name", TILEDB_READ);
auto a = map2[0]["attr"].get<std::string>(); // "0"
auto b = map2[1]["attr"].get<std::string>(); // "12"

Template Parameters
  • MapT: std::map type
  • Key: Key type for std::map
  • Value: Value type for std::map
Parameters
  • ctx: TileDB context
  • uri: URI of map to create
  • map: std::map instance to read from
  • attr_name: Name of attribute to create

template <typename MapT, typename Key = typename MapT::key_type, typename Value = typename MapT::mapped_type>
static void create(const Context &ctx, const std::string &uri, const MapT &map, const std::string &attr_name, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Create an encrypted TileDB map from a std::map.

Example:

// Create map
std::map<int, std::string> map;
map[0] = "0";
map[1] = "12";
// Load AES-256 key from disk, environment variable, etc.
uint8_t key[32] = ...;
Map::create(ctx, "map_name", map, "attr",
   TILEDB_AES_256_GCM, key, sizeof(key));

Template Parameters
  • MapT: std::map type
  • Key: Key type for std::map
  • Value: Value type for std::map
Parameters
  • ctx: TileDB context
  • uri: URI of map to create
  • map: std::map instance to read from
  • attr_name: Name of attribute to create
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

static void consolidate(const Context &ctx, const std::string &uri)

Consolidates the fragments of a Map into a single fragment.

Parameters
  • ctx: TileDB context
  • uri: URI of map

static void consolidate(const Context &ctx, const std::string &uri, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Consolidates the fragments of an encrypted Map into a single fragment.

Parameters
  • ctx: TileDB context
  • uri: URI of map
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

Map Schema

class tiledb::MapSchema

Represents the schema of a Map (key-value store).

Example:

tiledb::Context ctx;
tiledb::MapSchema schema(ctx);
schema.add_attribute(Attribute::create<int>(ctx, "a1"));
schema.add_attribute(Attribute::create<std::string>(ctx, "a2"));
schema.add_attribute(Attribute::create<std::array<float, 2>>(ctx, "a3"));
// Create an empty map with the above schema.
Map::create("my_map", schema);

Inherits from tiledb::Schema

Public Functions

MapSchema(const Context &ctx)

Creates a new empty map schema.

Parameters
  • ctx: TileDB context

MapSchema(const Context &ctx, const std::string &uri)

Loads the schema of an existing Map with the given URI.

Parameters
  • ctx: TileDB context
  • uri: URI of map to load

MapSchema(const Context &ctx, const std::string &uri, tiledb_encryption_type_t encryption_type, const void *encryption_key, uint32_t key_length)

Loads the schema of an existing encrypted Map with the given URI.

Parameters
  • ctx: TileDB context
  • uri: URI of map to load
  • encryption_type: The encryption type to use.
  • encryption_key: The encryption key to use.
  • key_length: Length in bytes of the encryption key.

MapSchema(const Context &ctx, tiledb_kv_schema_t *schema)

Loads the schema of an existing kv with the input C array schema object.

operator tiledb_kv_schema_t *() const

Auxiliary operator for getting the underlying C TileDB object.

void dump(FILE *out = stdout) const

Dumps the map schema in an ASCII representation to an output.

Parameters
  • out: (Optional) File to dump output to. Defaults to stdout.

MapSchema &add_attribute(const Attribute &attr)

Adds an attribute to the schema.

Example:

tiledb::Context ctx;
tiledb::MapSchema schema(ctx);
schema.add_attribute(Attribute::create<int>(ctx, "a1"));

Return
Reference to this MapSchema.
Parameters

std::shared_ptr<tiledb_kv_schema_t> ptr() const

Returns a shared pointer to the C TileDB domain object.

void check() const

Validates the schema.

Example:

tiledb::Context ctx;
tiledb::MapSchema schema(ctx);
// Add attributes etc...

try {
  schema.check();
} catch (const tiledb::TileDBError& e) {
  std::cout << e.what() << "\n";
  exit(1);
}

Exceptions

std::unordered_map<std::string, Attribute> attributes() const

Gets all attributes in the array.

Return
Map of attribute name to copy of Attribute instance.

unsigned attribute_num() const

Returns the number of attributes in the schema.

Attribute attribute(const std::string &name) const

Get a copy of an Attribute in the schema by name.

Return
Attribute
Parameters
  • name: Name of attribute

Attribute attribute(unsigned int i) const

Get a copy of an Attribute in the schema by index. Attributes are ordered the same way they were defined when constructing the schema.

Return
Attribute
Parameters
  • i: Index of attribute

MapSchema &set_capacity(uint64_t capacity)

Sets the tile capacity.

Return
Reference to this MapSchema.
Parameters
  • capacity: Capacity value to set.

uint64_t capacity() const

Returns the tile capacity.

Map Item

class tiledb::MapItem

Object representing a Map key and its attribute values.

Public Functions

MapItem(const Context &ctx, tiledb_kv_item_t **item, Map *map = nullptr)

Load a MapItem given a pointer.

bool good() const

Checks the goodness of the key-value item. Useful when checking if a retrieved key-value item exists in a map.

Example:

tiledb::Context ctx;
tiledb::Map map(ctx, "map_name");
int key = 1;
tiledb::MapItem item = map.get_item(key);
bool item_exists = item.good();

template <typename T>
void set(const std::string &attr, const T &val)

Sets an attribute to the given value for this item.

Example:

tiledb::Context ctx;
// Load or create a map.
tiledb::Map map(...);
int key = 1;
auto item = Map::create_item(ctx, key);
item.set("attr", 123);

Template Parameters
Parameters

template <typename T>
T key() const

Returns the key for this item.

Template Parameters
  • T: Key type

std::pair<tiledb_datatype_t, uint64_t> key_info() const

Returns a pair of the key datatype and size (in bytes) for this item.

template <typename T>
std::pair<const T *, uint64_t> get_ptr(const std::string &attr) const

Returns the value for a given attribute for this item.

Note
This does not check for the number of elements, but rather returns the size (in elements) retrieved.

template <typename T>
T get(const std::string &attr) const

Returns the value for an attribute of this item.

Return
The attribute value of this item
Template Parameters
  • T: The value type
Parameters
  • attr: The attribute

impl::MapItemProxy operator[](const std::string &attr)

Operator that enables setting/getting an attribute value of this item with [].

Example:

tiledb::Context ctx;
// Load or create a map.
tiledb::Map map(...);
int key = 1;
auto item = Map::create_item(ctx, key);
item["attr"] = 123;
// Equivalent to:
// item.set("attr", 123);

Return
”Proxy” object supporting operator[].
Parameters

impl::MultiMapItemProxy operator[](const std::vector<std::string> &attrs)

Operator that enables setting/getting multiple attribute values of this item with [].

Example:

tiledb::Context ctx;
// Load or create a map.
tiledb::Map map(...);
int key = 1;
auto item = Map::create_item(ctx, key);
// Assumes attribute types of a1 int, and a2 std::string.
item[{"a1", "a2"}] = std::make_tuple(123, "abc");
// Equivalent to:
// item.set("a1", 123);
// item.set("a2", "abc");

Return
”Proxy” object supporting operator[].
Parameters
  • attrs: Attributes to get/set

std::shared_ptr<tiledb_kv_item_t> ptr() const

Ptr to underlying object.

Map Iter

class tiledb::MapIter

Iterate over items in a map.

Inherits from std::iterator< std::forward_iterator_tag, MapItem >

Public Functions

MapIter(Map &map, bool end = false)

Construct a iterator for a given map.

~MapIter()

Flush on iterator destruction.

template <typename T>
void limit_key_type()

Only iterate over keys with some type. Only keys with the underlying tiledb datatype and num will be returned.

void all_keys()

Disable any key filters.

bool operator==(const MapIter &o) const

Iterators are only equal when both are end.

Object Management

class tiledb::Object

Represents a TileDB object: array, group, key-value (map), or none (invalid).

Public Types

enum Type

The object type.

Values:

Array

TileDB array object.

Group

TileDB group object.

Invalid

Invalid or unknown object type.

KeyValue

TileDB key-value (map) array object.

Public Functions

std::string to_str() const

Returns a string representation of the object, including its type and URI.

Type type() const

Returns the object type.

std::string uri() const

Returns the object URI.

Public Static Functions

static Object object(const Context &ctx, const std::string &uri)

Gets an Object object that encapsulates the object type of the given path.

Return
An object that contains the type along with the URI.
Parameters
  • ctx: The TileDB context
  • uri: The path to the object.

static void remove(const Context &ctx, const std::string &uri)

Deletes a TileDB object at the given URI from disk/persistent storage.

Parameters
  • ctx: The TileDB context
  • uri: The path to the object to be removed.

static void move(const Context &ctx, const std::string &old_uri, const std::string &new_uri)

Moves/renames a TileDB object.

Parameters
  • old_uri: The path to the old object.
  • new_uri: The path to the new object.

class tiledb::ObjectIter

Enables listing TileDB objects in a directory or walking recursively an entire directory tree.

Example:

// List the TileDB objects in an S3 bucket.
tiledb::Context ctx;
tiledb::ObjectIter obj_it(ctx, "s3://bucket-name");
for (auto it = obj_it.begin(), ite = obj_it.end(); it != ite; ++it) {
  const tiledb::Object &obj = *it;
  std::cout << obj << std::endl;
}

Public Functions

ObjectIter(Context &ctx, const std::string &root = ".")

Creates an object iterator. Unless set_recursive is invoked, this iterator will iterate only over the children of root. It will also retrieve only TileDB-related objects.

Example:

// List the TileDB objects in an S3 bucket.
tiledb::Context ctx;
tiledb::ObjectIter obj_it(ctx, "s3://bucket-name");
for (auto it = obj_it.begin(), ite = obj_it.end(); it != ite; ++it) {
  const tiledb::Object &obj = *it;
  std::cout << obj << std::endl;
}

Parameters
  • ctx: The TileDB context.
  • root: The root directory where the iteration will begin.

void set_iter_policy(bool group, bool array, bool kv)

Determines whether group, array and key-value objects will be iterated on during the walk. The default (if the function is not invoked) is true for all objects.

Parameters
  • group: If true, groups will be considered.
  • array: If true, arrays will be considered.
  • kv: If true, key-values will be considered.

void set_recursive(tiledb_walk_order_t walk_order = TILEDB_PREORDER)

Specifies that the iteration will be over all the directories in the tree rooted at root_.

Parameters
  • walk_order: The walk order.

void set_non_recursive()

Disables recursive traversal.

iterator begin()

Returns an object iterator at the beginning of its iteration.

iterator end() const

Returns an object iterator at the end of its iteration.

Public Static Functions

static int obj_getter(const char *path, tiledb_object_t type, void *data)

Callback function to be used when invoking the C TileDB functions for walking through the TileDB objects in the root_ diretory. The function retrieves the visited object and stored it in the object vector obj_vec.

Return
If 1 then the walk should continue to the next object.
Parameters
  • path: The path of a visited TileDB object
  • type: The type of the visited TileDB object.
  • obj_vec: The vector where the visited object will be stored.

class iterator

The actual iterator implementation in this class.

Inherits from std::iterator< std::forward_iterator_tag, const Object >

struct ObjGetterData

Carries data to be passed to obj_getter.

VFS

class tiledb::VFS

Implements a virtual filesystem that enables performing directory/file operations with a unified API on different filesystems, such as local posix/windows, HDFS, AWS S3, etc.

Public Types

using filebuf = impl::VFSFilebuf

Stream buffer for Tiledb VFS.

This is unbuffered; each read/write is directly dispatched to TileDB. As such it is recommended to issue fewer, larger, operations.

Example (write to file):

// Create the file buffer.
tiledb::Context ctx;
tiledb::VFS vfs(ctx);
tiledb::VFS::filebuf buff(vfs);

// Create new file, truncating it if it exists.
buff.open("file.txt", std::ios::out);
std::ostream os(&buff);
if (!os.good()) throw std::runtime_error("Error opening file);

std::string str = "This will be written to the file.";

os.write(str.data(), str.size());
// Alternatively:
// os << str;
os.flush();
buff.close();

Example (read from file):

// Create the file buffer.
tiledb::Context ctx;
tiledb::VFS vfs(ctx);
tiledb::VFS::filebuf buff(vfs);
std::string file_uri = "s3://bucket-name/file.txt";

buff.open(file_uri, std::ios::in);
std::istream is(&buff);
if (!is.good()) throw std::runtime_error("Error opening file);

// Read all contents from the file
std::string contents;
auto nbytes = vfs.file_size(file_uri);
contents.resize(nbytes);
vfs.read((char*)contents.data(), nbytes);

buff.close();

Public Functions

VFS(const Context &ctx)

Constructor.

Parameters
  • ctx: A TileDB context.

VFS(const Context &ctx, const Config &config)

Constructor.

Parameters
  • ctx: TileDB context.
  • config: TileDB config.

void create_bucket(const std::string &uri) const

Creates an object-store bucket with the input URI.

void remove_bucket(const std::string &uri) const

Deletes an object-store bucket with the input URI.

bool is_bucket(const std::string &uri) const

Checks if an object-store bucket with the input URI exists.

void empty_bucket(const std::string &bucket) const

Empty a bucket

bool is_empty_bucket(const std::string &bucket) const

Check if a bucket is empty

void create_dir(const std::string &uri) const

Creates a directory with the input URI.

bool is_dir(const std::string &uri) const

Checks if a directory with the input URI exists.

void remove_dir(const std::string &uri) const

Removes a directory (recursively) with the input URI.

bool is_file(const std::string &uri) const

Checks if a file with the input URI exists.

void remove_file(const std::string &uri) const

Deletes a file with the input URI.

uint64_t file_size(const std::string &uri) const

Retrieves the size of a file with the input URI.

void move_file(const std::string &old_uri, const std::string &new_uri) const

Renames a TileDB file from an old URI to a new URI.

void move_dir(const std::string &old_uri, const std::string &new_uri) const

Renames a TileDB directory from an old URI to a new URI.

void touch(const std::string &uri) const

Touches a file with the input URI, i.e., creates a new empty file.

const Context &context() const

Get the underlying context

std::shared_ptr<tiledb_vfs_t> ptr() const

Get the underlying tiledb object

Config config() const

Get the config

Utils

namespace tiledb

Functions

template <typename T, typename E = typename std::vector<T>>
std::vector<E> group_by_cell(const std::vector<uint64_t> &offsets, const std::vector<T> &data, uint64_t num_offsets, uint64_t num_data)

Convert an (offset, data) vector pair into a single vector of vectors. Useful for “unpacking” variable-length attribute data from a read query result in offsets + data form to a vector of per-cell data.

Example:

std::vector<uint64_t> offsets;
std::vector<char> data;
...
query.set_buffer("attr_name", offsets, data);
query.submit();
...
auto attr_results = query.result_buffer_elements()["attr_name"];

// cell_vals length will be equal to the number of cells read by the query.
// Each element is a std::vector<char> with each cell's data for "attr_name"
auto cell_vals =
  group_by_cell(offsets, data, attr_results.first, attr_results.second);

// Reconstruct a std::string value for the first cell:
std::string cell_val(cell_vals[0].data(), cell_vals[0].size());

Note
This function, and the other utility functions, copy all of the input data when constructing their return values. Thus, these may be expensive for large amounts of data.
Return
std::vector<E>
Template Parameters
  • T: Underlying attribute datatype
  • E: Cell type. usually std::vector<T> or std::string. Must be constructable by {std::vector<T>::iterator, std::vector<T>::iterator}
Parameters
  • offsets: Offsets vector. This specifies the start offset of each cell in the data vector.
  • data: Data vector. Flat data buffer with cell contents.
  • num_offsets: Number of offset elements populated by query. If the entire buffer is to be grouped, pass offsets.size().
  • num_data: Number of data elements populated by query. If the entire buffer is to be grouped, pass data.size().

template <typename T, typename E = typename std::vector<T>>
std::vector<E> group_by_cell(const std::pair<std::vector<uint64_t>, std::vector<T>> &buff, uint64_t num_offsets, uint64_t num_data)

Convert an (offset, data) vector pair into a single vector of vectors. Useful for “unpacking” variable-length attribute data from a read query result in offsets + data form to a vector of per-cell data.

Example:

std::vector<uint64_t> offsets;
std::vector<char> data;
...
query.set_buffer("attr_name", offsets, data);
query.submit();
...
auto attr_results = query.result_buffer_elements()["attr_name"];

// cell_vals length will be equal to the number of cells read by the query.
// Each element is a std::vector<char> with each cell's data for "attr_name"
auto cell_vals =
  group_by_cell(std::make_pair(offsets, data),
                attr_results.first, attr_results.second);

// Reconstruct a std::string value for the first cell:
std::string cell_val(cell_vals[0].data(), cell_vals[0].size());

Return
std::vector<E>
Template Parameters
  • T: Underlying attribute datatype
  • E: Cell type. usually std::vector<T> or std::string. Must be constructable by {std::vector<T>::iterator, std::vector<T>::iterator}
Parameters
  • buff: Pair of (offset_vec, data_vec) to be grouped.
  • num_offsets: Number of offset elements populated by query.
  • num_data: Number of data elements populated by query.

template <typename T, typename E = typename std::vector<T>>
std::vector<E> group_by_cell(const std::vector<uint64_t> &offsets, const std::vector<T> &data)

Convert a generic (offset, data) vector pair into a single vector of vectors.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
std::vector<uint64_t> offsets = {0, 5};
auto grouped = group_by_cell<char, std::string>(offsets, buf);
// grouped.size() == 2
// grouped[0] == "abcde"
// grouped[1] == "fghi"

Return
std::vector<E>
Template Parameters
  • T: Underlying attribute datatype
  • E: Cell type. usually std::vector<T> or std::string. Must be constructable by {std::vector<T>::iterator, std::vector<T>::iterator}
Parameters
  • offsets: Offsets vector
  • data: Data vector

template <typename T, typename E = typename std::vector<T>>
std::vector<E> group_by_cell(const std::vector<T> &buff, uint64_t el_per_cell, uint64_t num_buff)

Convert a vector of elements into a vector of fixed-length vectors.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
auto grouped = group_by_cell(buf, 3, buf.size());
std::string grp1(grouped[0].begin(), grouped[0].end());  // "abc"
std::string grp2(grouped[1].begin(), grouped[1].end());  // "def"
std::string grp3(grouped[2].begin(), grouped[2].end());  // "ghi"

// Throws an exception because buf.size() is not divisible by 2:
// group_by_cell(buf, 2, buf.size());

Return
std::vector<E>
Template Parameters
  • T: Underlying attribute datatype
  • E: Cell type. usually std::vector<T> or std::string. Must be constructable by {std::vector<T>::iterator, std::vector<T>::iterator}
Parameters
  • buff: Data buffer to group
  • el_per_cell: Number of elements per cell to group together
  • num_buff: Number of elements populated by query. To group whole buffer, pass buff.size().

template <typename T, typename E = typename std::vector<T>>
std::vector<E> group_by_cell(const std::vector<T> &buff, uint64_t el_per_cell)

Convert a vector of elements into a vector of fixed-length vectors.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
auto grouped = group_by_cell(buf, 3);
std::string grp1(grouped[0].begin(), grouped[0].end());  // "abc"
std::string grp2(grouped[1].begin(), grouped[1].end());  // "def"
std::string grp3(grouped[2].begin(), grouped[2].end());  // "ghi"

// Throws an exception because buf.size() is not divisible by 2:
// group_by_cell(buf, 2);

Return
std::vector<E>
Template Parameters
  • T: Element type
  • E: Cell type. usually std::vector<T> or std::string. Must be constructable by {std::vector<T>::iterator, std::vector<T>::iterator}
Parameters
  • buff: Data buffer to group
  • el_per_cell: Number of elements per cell to group together

template <uint64_t N, typename T>
std::vector<std::array<T, N>> group_by_cell(const std::vector<T> &buff, uint64_t num_buff)

Convert a vector of elements into a vector of fixed-length arrays.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
auto grouped = group_by_cell<3>(buf, buf.size());
std::string grp1(grouped[0].begin(), grouped[0].end());  // "abc"
std::string grp2(grouped[1].begin(), grouped[1].end());  // "def"
std::string grp3(grouped[2].begin(), grouped[2].end());  // "ghi"

// Throws an exception because buf.size() is not divisible by 2:
// group_by_cell<2>(buf, buf.size());

Return
std::vector<std::array<T,N>>
Template Parameters
  • N: Elements per cell
  • T: Array element type
Parameters
  • buff: Data buffer to group
  • num_buff: Number of elements in buff that were populated by the query.

template <uint64_t N, typename T>
std::vector<std::array<T, N>> group_by_cell(const std::vector<T> &buff)

Convert a vector of elements into a vector of fixed-length arrays.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
auto grouped = group_by_cell<3>(buf);
std::string grp1(grouped[0].begin(), grouped[0].end());  // "abc"
std::string grp2(grouped[1].begin(), grouped[1].end());  // "def"
std::string grp3(grouped[2].begin(), grouped[2].end());  // "ghi"

// Throws an exception because buf.size() is not divisible by 2:
// group_by_cell<2>(buf);

Return
std::vector<std::array<T,N>>
Template Parameters
  • N: Elements per cell
  • T: Array element type
Parameters
  • buff: data buff to group

template <typename T, typename R = typename T::value_type>
std::pair<std::vector<uint64_t>, std::vector<R>> ungroup_var_buffer(const std::vector<T> &data)

Unpack a vector of variable sized attributes into a data and offset buffer.

Example:

std::vector<char> buf = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'};
// For the sake of example, group buf into groups of 3 elements:
auto grouped = group_by_cell(buf, 3);
// Ungroup into offsets, data pair.
auto p = ungroup_var_buffer(grouped);
auto offsets = p.first;  // {0, 3, 6}
auto data = p.second;   // {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'}

Return
pair where .first is the offset buffer, and .second is data buffer
Template Parameters
  • T: Vector type. T::value_type is considered the underlying data element type. Should be vector or string.
  • R: T::value_type, deduced
Parameters
  • data: Data buffer to unpack

template <typename V, typename T = typename V::value_type::value_type>
std::vector<T> flatten(const V &vec)

Convert a vector-of-vectors and flatten it into a single vector.

Example:

std::vector<std::string> v = {"a", "bb", "ccc"};
auto flat_v = flatten(v);
std::string s(flat_v.begin(), flat_v.end()); // "abbccc"

std::vector<std::vector<double>> d = {{1.2, 2.1}, {2.3, 3.2}, {3.4, 4.3}};
auto flat_d = flatten(d);  // {1.2, 2.1, 2.3, 3.2, 3.4, 4.3};

Return
std::vector<T>
Template Parameters
  • V: Container type
  • T: Return element type
Parameters
  • vec: Vector to flatten

namespace impl

Functions

void check_config_error(tiledb_error_t *err)

Check an error, free, and throw if there is one.

Version

std::tuple<int, int, int> tiledb::version()

Get the Major, Minor, and Patch version.

Stats

class tiledb::Stats

Encapsulates functionality related to internal TileDB statistics.

Example:

// Enable stats, submit a query, then dump to stdout.
tiledb::Stats::enable();
query.submit();
tiledb::Stats::dump();

Public Static Functions

static void enable()

Enables internal TileDB statistics gathering.

static void disable()

Disables internal TileDB statistics gathering.

static void reset()

Reset all internal statistics counters to 0.

static void dump(FILE *out = stdout)

Dump all statistics counters to to some output (e.g., file or stdout).

Parameters
  • out: The output.