TileDB is an efficient multi-dimensional array management system which introduces a novel on-disk format that can effectively store dense and sparse array data with support for fast updates. It features excellent compression, high IO performance on multiple data persistence backends (e.g., HDFS, S3, etc.), and easy integration with ecosystems used by today’s data scientists (R, Python, Numpy, Scipy, Pandas, etc.).
TileDB is open-sourced under the permissive MIT License.
TileDB was originally created at the Intel Science and Technology Center for Big Data, a collaboration between Intel Labs and MIT. The research project was published in a VLDB 2017 paper. TileDB, Inc. founded in February 2017 to continue the further development and maintenance of the TileDB software.