Multimedia data, which are direct reflections of the real world, have unlimited semantic interpretations within different application domains. In order to reuse information resources, in this paper, we provide a layered architecture of multimedia schema - raw multimedia layer, domain knowledge layer, and user profile layer. Raw multimedia layer provides non-semantic management of data, including storage allocations, buffer algorithms and strategies, and multimedia preprocessing facilities. Domain knowledge layer provides semantic-oriented management and access of multimedia data. Different domains can utilize different knowledge bases to support semantic interpretations. We use Frame and Semantic Network as the representation models for domain knowledge. Content-based retrievals can be performed on different levels of visual object constructions or concept abstractions. Furthermore, since different individual users may have different interpretations for fuzzy specifications in their queries, user profiles are created and used to support fuzzy retrievals of multimedia data.