Carnegie Mellon University
School of Computer Science
5000 Forbes Avenue
Pittsburgh, PA 15213
This site was the original web site for the NSF Digital Libraries Initiative Phase-1 and is now an archive. For more updated information on the continuation of Informedia under the NSF Digital Libraries Initiative Phase 2 , please go to the Informedia Home Page, or go straight to Informedia-II.Project Description
Vast digital libraries of information will soon be available on the nation's Information Superhighway as a result of emerging technologies for multimedia data processing. These libraries will profoundly impact the conduct of business, professional,and personal activity. However, it is not enough to simply store and play back video (as in currently envisioned commercial video-on-demand services); to be most effective, new technology is needed for searching through these vast data collections and retrieving the most relevant selections.
The Informedia project is developing these new technologies for data storage, search, and retrieval, and embedding them in a video library system for use in education, training, sports and entertainment. Additional applications include the recording and subsequent search of corporate meetings and professional conferences. The new digital video library technology will allow more independent, self-motivated access to information for self-teaching and exploration, which can bring about a revolutionary improvement in the way education and training are delivered.
The Informedia Digital Video Library uses intelligent, automatic mechanisms that provide full-content search and retrieval from an extremely large (scaling to several thousand hours) online digital video library. We will build the library initially using video from the archives of WQED (a PBS producer), the British Open University's BBC-produced video courses, and public information videos from Federal agencies. We are developing tools that can automatically populate the library and support access via desktop computers on local, metropolitan and wide area networks.
Our approach uses combined speech, language and image understanding technology to automatically transcribe, segment and index the linear video. These same tools will be applied to accomplish intelligent search and selective retrieval. Initially, we apply a highly accurate, speaker-independent speech recognizer to automatically transcribe video soundtracks which are then stored in a (time-track corresponding) full-text information retrieval system. This text database in turn allows for rapid retrieval of individual corresponding "video paragraphs" which satisfy an arbitrary subject area query based on the words in the soundtrack. Another innovative concept is our implementation of "video skimming". This enables an accelerated viewing of the key video and audio sequences without the perceptual disturbance of simply speeding up the frame rate and audio. We thus create a video abstract that conveys the essence of the content in 5 to 20% of the time.
Figure 1: The Informedia TM Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.
Initial versions of our library will use commercial compression and deliver VHS quality playback. We anticipate that our primary media-server file system will require one terabyte (1,000 gigabytes) of storage to archive over 1000 hours of video. We will also incorporate pay-per-view accounting as subsequent commercial versions of the system will require such data for protection and commerce of intellectual property. Understanding real-world economics of digital libraries will also be critical for their successful implementation.
Our modular plan will allow us to install early testbeds in Carnegie Mellon and local area public schools. We will also facilitate the establishment of commercial information services to provide metropolitan area network access over local, high-bandwidth networks and wide area access via the Internet.
Figure 2: Informedia (TM) Digital Video Library system
Two main factors distinguish our research from similar but purely technology-driven efforts. We will provide a highly usable library of commercial- broadcast-quality resources and a mechanism for disseminating and commercializing our research products. Our work also addresses human factors issues: interaction, motivation, and effective usage modes for K-12, post-secondary, and life-long learning.
Established by Carnegie Mellon University, in cooperation with QED Communications (WQED Pittsburgh) and with initial support from the Heinz Endowments, the Informedia project integrates regional and national resources and focuses on creating and applying interactive multimedia to learning and communication in grades K-12 and beyond. Funding for this project is provided by a grant from the National Science Foundation, DARPA, and NASA. Founding corporate sponsorship is provided by Digital Equipment Corporation, Microsoft, Intel, CNN, Telecom Italia, Boeing, Bell Atlantic, and Visa, amongst others.