TileDB is an efficient multi-dimensional array management system which introduces a novel on-disk format that can effectively store dense and sparse array data with support for fast updates. It offers numerous features, including excellent compression, high IO performance on multiple data persistence backends (e.g., HDFS and S3), and easy integration with ecosystems used by today’s data scientists (e.g., Python Numpy).
TileDB is open-sourced under the permissive MIT License.
TileDB was originally created at the Intel Science and Technology Center for Big Data, a collaboration between Intel Labs and MIT. The research project was published in a VLDB 2016 paper. TileDB, Inc. was founded in February 2017 to continue the further development and maintenance of the TileDB software.