Annotation and Image Markup (AIM) Project

Images, in particular medical and scientific images, contain vast amounts of information. While this information may include metadata about the image, such as how or when the image was acquired, the majority of image information is encoded in the images pixels; however, information about how images are perceived by human or machine observers is not currently captured in a form that is directly tied to the images. A wealth of information pertaining to image content is thus disconnected from the images, limiting the value of radiology imaging to be related to other non-imaging data. We need tools that will allow both human and machine image annotations to be created and stored in a standard format that is syntactically and semantically interoperable with the infrastructure with other biomedical resources while supporting standards such as DICOM, HL7 and those being created by the W3C semantic Web community.

We are developing methods to describe the semantic content in images using ontologies, explicit representations of the entities and relations in biomedicine. We are also creating tools to compose ontology-based descriptions of image content and associate them with images. This work will change the paradigm of medical imaging—instead of clinical systems storing just pixels, they will store image data plus the image meaning. This will enable a broad range of computational analytic functionality, including semantic search (see IQ Project), integration of image- and non-image data, statistical modeling of disease, and intelligent decision support applications for image-based personalized care.