> For the complete documentation index, see [llms.txt](https://doc.thordata.com/doc/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.thordata.com/doc/faq/product-problem/multimodal-dataset/what-is-a-multimodal-dataset.md).

# What is a multimodal dataset?

A multimodal dataset is a structured collection of data that combines two or more data types — such as text, images, audio, and video — with aligned relationships between them, used to train AI models that can understand multiple input formats at the same time.

Unlike single-modality datasets, multimodal datasets capture the way information appears in the real world. A video clip paired with its transcript, an audio recording aligned word-by-word with its text, or an image linked to a caption are all multimodal pairs. They are the foundation for training today's most advanced AI systems: speech recognition engines, video-understanding LLMs, vision-language models, and embodied AI agents that must see, hear, and act simultaneously. The quality of alignment between modalities — not just the raw volume — is what determines how well a model learns cross-modal reasoning.