What are modalities? How TileDB Carrara unifies diverse data with omnimodal data intelligence

Data is only useful when we can gain insights from it. This is a foundational truth of data science that applies whether we’re a biopharma corporation searching for new genomic targets, an intelligence agency analyzing geospatial data or a public health organization reviewing population health reports. But no matter what vertical they’re in, many organizations struggle to extract the true value of all their data—not because they have too much of it, but because their data exists in diverse modalities that their legacy data infrastructure cannot harmonize and integrate.

If we are going to master the challenge of managing all kinds of data modalities, we must begin by defining what data modalities are. By establishing this understanding, we can articulate a data management vision that’s truly omnimodal, treating all data as discoverable, securely accessible and able to be analyzed effectively and efficiently.

The definition of data modality and TileDB’s omnimodal vision

A data modality refers to a reachable and shareable data type that has some form of structure and meaning. The word “structure” here is key, but not in the way many think. Traditionally, data has been divided into structured and unstructured based on whether or not it’s been formatted in a table such as a financial database or Excel file. 

However, just because data doesn’t fit neatly into a table does not make this data useless, or even unstructured. Images with pixels are structured to appear a certain way. PDFs with text are structured with grammar and spelling. Complex datasets of single cell data follow the structure of DNA and RNA. Organizations can no longer afford to dismiss such data as unstructured and store it as blobs stripped of context and metadata. Why? Because these data modalities are all backed by some kind of structure and have all kinds of insight to share.

As organizations think beyond their tabular databases and data lake houses to explore how AI platforms can compile all their data into new insights, they must expand how they think about data modalities. This is foundational to the omnimodal vision of TileDB Carrara, which manages all types of data as different modalities in one platform:

1. 1

   External databases: All external databases from platforms like Microsoft Fabric, Snowflake or Databricks become modalities, with a single catalog and point of governance in TileDB.

2. 2

   Files and chats: TileDB interprets every “unstructured” file, email, chart or other communication from platforms like Office, Outlook and Teams with attached metadata, APIs, tools and preview widgets. This data provides invaluable context for AI and ML to learn from in order to create new insights.

3. 3

   Complex data: This valuable, high-resolution data like multiomics and imaging is no longer labeled “unstructured” but instead integrated into TileDB where it can be easily managed and queried.

4. 4

   Code: Github repos, Jupyter notebooks and workflow scripts are also all modalities that can be securely launched in TileDB, equipping engineers and data scientists to work seamlessly with their preferred tools and frameworks.

5. 5

   Apps: Powerful dashboards, chat bots, visualization tools and other standalone applications are also modalities that TileDB can securely launch.

6. 6

   Compute: User-defined functions, task graphs that represent workflows, even standalone server hardware are all modalities TileDB can integrate and manage.

7. 7

   Agents: Agentic AI is a growing part of the unified data foundation in TileDB, where they are developed and securely launched to easily learn and act based on all the other modalities in the platform.

TileDB Carrara: The omnimodal intelligence platform

By centering its architecture around these modalities, TileDB Carrara integrates all multimodal data types into a unified platform that maintains semantic context and metadata. This makes all this multimodal data FAIR and easily searchable by human users and AI agents, establishing TileDB as a true omnimodal intelligence platform that can derive insight from any dataset regardless of its type, size or complexity.

To learn more about how TileDB Carrara helps organizations master different data modalities, watch our full Tech Talk here.