More and more videos are finding a niche on the Internet. But how do Web users find what they're looking for? Current video search technologies depend on semantic annotation; videos are manually tagged with keywords and located via a text-based search, a time-consuming and expensive effort.
An EU-funded team of researchers is about to change all that. The DIVAS ('Direct video & audio content search engine') project, which received EUR 1.94 million under the 'Information society technologies' Thematic area of the Sixth Framework Programme (FP6), has developed a multimedia search engine based on advanced direct video and audio search algorithms — so-called digital media fingerprints.
The DIVAS consortium, led by the Greece-based information technologies group Archetypon, targeted the development of technology capable of indexing and searching compressed video files without many headaches.
'We wanted to develop a way of indexing and searching compressed video files quickly and easily regardless of their compression format or how or where they are stored,' explained Nick Achilleopoulos, DIVAS' technology development manager.
The DIVAS team successfully developed two advanced software engines: the first creates fingerprints from compressed audio and/or video, and the second uses unique identifiers to carry out content-based searches for audiovisual material.
What is unique about DIVAS' indexing software is that it does not need the video to be uncompressed to work. So computer processing power and storage space are decreased, but the indexing process speed increases.
'The fingerprint extraction software defines audio and video features much as a human viewer perceives audiovisual elements,' Mr Achilleopoulos said. 'It builds the fingerprint based on visual features, such as scene changes, the way the camera cuts and moves, the brightness level, and the movement of people and objects.'
The DIVAS team said audio features including music and speech are part of the fingerprint makeup. In the case of audiovisual fingerprints, they are stored in the XML file format in combination with the MPEG-7 multimedia content description standard, thus creating quick and easy access to a searchable video index.
'Say you saw a short clip of a TV series and wanted to see more of it but did not know the name,' said Mr Achilleopoulos, 'you could easily upload the clip to a DIVAS search engine and then use this to find not only the series, but also the season, episode and the exact minute of a scene the clip is from.'
The only challenge is that the searcher would have to have an indexed database of video content to compare the fingerprinted clip to, according to the researchers.
In terms of commercialisation, the team said their product would be a cheaper alternative to what is currently used on the market. 'A lot of companies are interested in monitoring broadcasts to make sure TV stations are airing their adverts in the time slots and with the frequency they pay for,' the DIVAS project manager said.
'Currently, they do this by recording broadcast on expensive equipment and even have people watch the TV, but a much cheaper alternative would be to record compressed files and have software automatically creating fingerprints of the content,' he added. 'These could then be matched with the advertiser's content, letting them know precisely when and how often their adverts are shown.'
The DIVAS partners have entered talks with an advertising company to launch the technology on the market for monitoring TV broadcasts. Other companies have also expressed an interest in using DIVAS' technology to improve Internet searches of their video databases.
The DIVAS partners are from Austria, Belgium, France, Germany, Greece, Israel and Russia.