Informedia Digital Video Library:  Digital video library research at Carnegie Mellon School of Computer Science
nav graphic
















  Carnegie Mellon University
  School of Computer Science
  5000 Forbes Avenue
  Pittsburgh, PA 15213

  About Informedia-I  |  Sponsors  |  Media coverage  |  Other projects  |  Reports  |  MPEG Collection
Informedia-I: Integrated Speech, Image and Language Understanding for Creation and Exploration of Digital Video Libraries
Howard Wactlar
Takeo Kanade, Michael Mauldin, Raj Reddy, Marvin Sirbu, Scott Stevens, Doug Tygar
National Science Foundation , Digital Libraries Initiative Phase-1

This site was the original web site for the NSF Digital Libraries Initiative Phase-1 and is now an archive. For more updated information on the continuation of Informedia under the NSF Digital Libraries Initiative Phase 2 , please go to the Informedia Home Page, or go straight to Informedia-II.

Project Description

Vast digital libraries of information will soon be available on the nation's Information Superhighway as a result of emerging technologies for multimedia data processing. These libraries will profoundly impact the conduct of business, professional,and personal activity. However, it is not enough to simply store and play back video (as in currently envisioned commercial video-on-demand services); to be most effective, new technology is needed for searching through these vast data collections and retrieving the most relevant selections.

The Informedia project is developing these new technologies for data storage, search, and retrieval, and embedding them in a video library system for use in education, training, sports and entertainment. Additional applications include the recording and subsequent search of corporate meetings and professional conferences. The new digital video library technology will allow more independent, self-motivated access to information for self-teaching and exploration, which can bring about a revolutionary improvement in the way education and training are delivered.

The Informedia Digital Video Library uses intelligent, automatic mechanisms that provide full-content search and retrieval from an extremely large (scaling to several thousand hours) online digital video library. We will build the library initially using video from the archives of WQED (a PBS producer), the British Open University's BBC-produced video courses, and public information videos from Federal agencies. We are developing tools that can automatically populate the library and support access via desktop computers on local, metropolitan and wide area networks.

Our approach uses combined speech, language and image understanding technology to automatically transcribe, segment and index the linear video. These same tools will be applied to accomplish intelligent search and selective retrieval. Initially, we apply a highly accurate, speaker-independent speech recognizer to automatically transcribe video soundtracks which are then stored in a (time-track corresponding) full-text information retrieval system. This text database in turn allows for rapid retrieval of individual corresponding "video paragraphs" which satisfy an arbitrary subject area query based on the words in the soundtrack. Another innovative concept is our implementation of "video skimming". This enables an accelerated viewing of the key video and audio sequences without the perceptual disturbance of simply speeding up the frame rate and audio. We thus create a video abstract that conveys the essence of the content in 5 to 20% of the time.

Figure 1: The Informedia TM Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.

Figure 1: The Informedia TM Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.

Initial versions of our library will use commercial compression and deliver VHS quality playback. We anticipate that our primary media-server file system will require one terabyte (1,000 gigabytes) of storage to archive over 1000 hours of video. We will also incorporate pay-per-view accounting as subsequent commercial versions of the system will require such data for protection and commerce of intellectual property. Understanding real-world economics of digital libraries will also be critical for their successful implementation.

Our modular plan will allow us to install early testbeds in Carnegie Mellon and local area public schools. We will also facilitate the establishment of commercial information services to provide metropolitan area network access over local, high-bandwidth networks and wide area access via the Internet.

Figure 2: Informedia Digital Video Library system screen shot

Figure 2: Informedia (TM) Digital Video Library system

Two main factors distinguish our research from similar but purely technology-driven efforts. We will provide a highly usable library of commercial- broadcast-quality resources and a mechanism for disseminating and commercializing our research products. Our work also addresses human factors issues: interaction, motivation, and effective usage modes for K-12, post-secondary, and life-long learning.

Established by Carnegie Mellon University, in cooperation with QED Communications (WQED Pittsburgh) and with initial support from the Heinz Endowments, the Informedia project integrates regional and national resources and focuses on creating and applying interactive multimedia to learning and communication in grades K-12 and beyond. Funding for this project is provided by a grant from the National Science Foundation, DARPA, and NASA. Founding corporate sponsorship is provided by Digital Equipment Corporation, Microsoft, Intel, CNN, Telecom Italia, Boeing, Bell Atlantic, and Visa, amongst others.


nav graphic
About Informedia-I   |  Sponsors  |  Media coverage  |  Other projects   | Reports  |  MPEG Collection  |  INFORMEDIA HOME
topCopyright 1994-2002 Carnegie Mellon and its licensors.  All rights reserved.