Automatic Acquisition of Visual Landmarks / Automatisk indlæring af visuelle landemærker
A mobile robot can autonomously navigate through for example a production facility by using a video camera to monitor known visual landmarks in the environment. Landmarks can be light switches, posters, door signs, etc. An especially flexible system is achieved if the robot can automatically acquire (learn) such landmarks for future navigation use. This project has developed techniques for automatic learning of visual landmarks by acquiring images of the environment and extracting positions and appearances of landmark candidates. These candidates are then stored in the robot's memory for subsequent use in controlling robot movements. CVMT has special focus on analysis of how accurately this navigation method can perform, in order to optimise the use of the available landmarks.
Project under VIRGO, (EU, TMR).
(Claus B. Madsen; Salvatore Livatino, Italy)
Segmentation of skin colour under different illuminations and estimation of the colour of illuminations / Segmentering af hudfarve under forskellige lyskilder og estimering af lyskilders farve
Automatically detecting and tracking faces and hands of humans in motion is important in applications such as interfaces for human computer interaction (HCI), in- and outdoor surveillance, and automatic camera men. An often-used feature in face and human motion tracking is skin colour because it is fast to compute and invariant to size and orientation. However, the colour of skin changes as the illumination changes. This project aims to develop adaptive methods for segmenting human skin under changing illumination conditions. A physics based reflection model of human skin has been developed, and successfully applied, e.g., to estimate the illumination colour and adapt statistical models for skin colour, respectively.
Partly funded by ARTHUR
(Moritz Störring, Erik Granum, Hans J. Andersen)
Assessment
of crops by outdoor computer vision/Afgrødevurdering og udendørs computer
vision
Visual information of crops has in centuries been used by mankind for assessment
of crops growth condition. By visual information the spatial distribution
of reflection from the canopy is used for detection of specific patterns'
related to lack of nutrients or infestation by diseases, i.e. lack of manganese
will turn out as light brown spots on the leaves whereas lack of calcium will
turn as miss colouring of leaves from the leaf edge and inwards.
So far development of sensor for detection of plant nutrient deficiency
has been oriented towards nitrogen and with sensor types that does not take
the spatial pattern of the reflection into account.
The objective of this project is
to developed computer vision methods that may enable a more detailed analysis
of the reflection from canopies by: 1) identification of areas without specular
reflection and 2) correct images for uneven illumination conditions.
In this way a reliable quantitative analysis of the canopy may be obtained
for optimal growth control.
Funded by: The Danish Agricultural and Veterinary Research Council.
(Hans J. Andersen)
ACROSS: Autonomous spatial-temporal
crop and soil surveying/Autonom spatio temporal afgrøde- og jordmonitorering
The general vision is effective precision farming,
which in harmony with the environment utilises resources optimally. This
requires continuous selective and adaptive control of growth, weeds, diseases
and pest. In turn such control is conditioned on corresponding continuous
monitoring in the field using appropriate methods of measuring the current
conditions of and for the plant growth.
The objective of this project is to develop methods of measuring and managing
such information to support the above vision in a way that invites also
new innovative approaches to precision farming by providing the necessary
information on demand and in time for planning and decision making.
More concretely the project will develop methods and technology for: 1)
Computer vision and laser range methods for on-site and real-time monitoring
of information of the crop growth (nutrients, diseases, etc.). The methods
will allow diagnostics of crop condition based on reflection patterns (e.g.
miss-coloured areas) down to single leaf scale. 2) Implementation and integration
of the above methods on an autonomous platform with a suite of existing crop
and soil measuring facilities for on-site operation. 3) Repeated test and
evaluation for development and Proof of concept: "Autonomous Crop and Soil
Surveillance"
Hence the project through new research and practical development will contribute
to a new scope precision agriculture, which until now has not been seen in
its full perspectives, due to the lack of precise and timely information.
Funded by: The Danish Technical Research and Agricultural and Veterinary
Research Councils and the Danish Ministry of Food, Agriculture and Fisheries.
(Erik Granum, Hans J. Andersen, Michael Nielsen)
Computer vision-based human motion capture using kinematics constraints / Computer vision registrering af humane bevægelser vha. kinematiske begrænsninger
In man-machine-interfaces there is a great need for interaction methods more natural to humans, e.g. via speech and body language. The latter is based on a computer capturing the movements of the individual body parts and recognises their meaning. In this project computer vision is utilised to investigate the capturing problem. The key approach is to have a geometric model of the articulated body parts. The model is used to predict possible configurations given the past configurations. The predicted configurations are compared with the image measurements and the true configuration of the human body is captured. To optimise this process detailed kinematics constraints related to the articulated body parts are introduced to limit the search space of possible configurations.
(Thomas B. Moeslund, Erik Granum)
Computer vision based action detection and classification / Computer Vision baseret detektion og klassifikation af handlinger
Real time detection and recognition of the motions and actions of dynamic objects is of interest in various applications like surveillance and monitoring of behaviours of humans and for real time visualisation of humans in (collaborative) virtual environments. The approach is based on scale-space analysis of spatial-temporal representations of the camera input. Different behaviours and motion patterns have typical ranges in spatial and temporal scale spaces, respectively, and such a priory knowledge is taken into account to achieve robust performance. Experimental tests are made with real time computer vision systems using monochrome as well as colour cameras.
(Peter Jürgensen, Erik Granum)
Reconstruction of 3D surface models using video cameras / Rekonstruktion af 3D overflademodeller vha. almindelige videokameraer
The aim of this project is to develop an image processing based system for construction of metrically correct 3D models from monochrome video camera images. The method is based on calibrated 2D images captured under controlled illumination conditions, as changes in the illumination play an active role in the processing of the images. From the 2D images, a set of corresponding 2.5D surface patches can be calculated and a surface based alignment method is being developed for joining these surface patches into a common 3D model of the surface. The generated 3D models are expected to have an accuracy compatible with existing methods, e.g., MRI-scanning, but with a significantly lower use of resources.
(Jørgen Bjørnstrup, Erik Granum)
VR MediaLab, Virtual Reality MediaLab
VR MediaLab is the Aalborg University Centre for Virtual Reality and Interactive Media. It was inaugurated in August 1999 with unique computing and visualisation facilities in a dedicated building complex of NOVI, the Science Park of Aalborg University. The VR-facilities comprises an sgi super computer (16 cpu’s, 6 graphics pipes) and three visualisation arenas: (6-sided CAVE; Panorama screen of 160 dg., 7.1 m diameter; and 3D Power Wall, 8 m wide). The Centre hosts research groups from various departments of the university, which wants to operate in the well-supported interdisciplinary environment with both research and teaching activities. CVMT played a major role in the establishment of the VR Centre, and is now located within it, contributing to, and benefiting from the interdisciplinary environment and the facilities.
Supported by Det Obelske Familiefond, Spar Nord Fonden, and EU Funds for Regional Development.
(Erik Kjems)
i3net, European Network of Excellence in Intelligent Information Interfaces
CVMT is a founding member (1996) of a European Network of Excellence, i3net (i-cube), in Intelligent Information Interfaces, which is lead by professor N. O. Bernsen, Odense Universitet. CVMT has in various periods been member of the Co-ordinating Group, CG, of the network, and was chairman for its first annual conference. i3 net formed a co-ordinating function for 13 i3-projects, which started in 1997, and for 10 ESE-projects (Experimental School Environments) starting in 1998. These projects were supported through separate IT-initiatives under EU ESPRIT Long Term Research (now FET, Future and Emerging Technologies). One of the projects under ESE, is the PUPPET project mentioned below, which CVMT co-ordinated. The network had support until 2002, and efforts are ongoing to establish a follow-up network.
(Erik Granum)
PUPPET, The educational puppet theatre of virtual worlds / Indlæring ved leg med dukketeater i den virtuelle verden
EU-project (ESPRIT EP-29335) under ESPRIT Long Term Research (now FET), i3/ESE, Experimental School Environments, October 1998 to January 2002. CVMT is co-ordinating contractor and the three other partners are early learning psychologists from Sussex University, England, dramaturgs from Århus Universitet, Denmark, and AI-experts from DFKI, Saarbrücken, Germany. The goal of the project was to develop and evaluate new virtual reality (VR) tools to support early learning for children (4-8 years old). Specifically was developed a 3D virtual world as a puppet theatre to facilitate externalisation through play. This should reduce the cognitive load of children and hence allow them to concentrate more on learning new concepts. The consortium has developed gradually improved versions of the virtual environment as a “theatre stage” with autonomous agents performing as virtual actors. A farm scenario with a farmer and farm animals was chosen for empirical investigations, and it has successfully been demonstrated and tested several times with 6-8 years old children.
One of the major technological contributions was the use of autonomous agents, which appeared to be harder to develop with real time performance of interesting behaviours than anticipated. The autonomy also contradicted with the intentions of supporting free story building and story telling by the children. A dramaturgical framework for improvisational theatre with a constraint and yet flexible space for spontaneous interaction and participation of the children was developed. The final version of the PUPPET theatre was tested by sixteen children playing with the virtual theatre in four different roles: (1) audience, observing the ever changing animated actions of the virtual actors in a designed conflict; (2) participating actor, by choosing and controlling an animal as avatar (representative in the virtual world) to interact and interfere in the ongoing conflict; (3) author, by supplying recordings for all the elements of speech and sound in the communication between the virtual actors; - and eventually (4) editor, by recalling and editing the audio recordings previously supplied. The results supported the project thesis, but indicated also that agent technology and interfaces needed significant improvements in order to show all the potential of the technology.
(Erik Granum, Claus B. Madsen, Panteleimon Kampolis)
STAGING, Staging of virtual inhabited 3D spaces / Iscenesættelse af virtuelle beboede 3D rum
STAGING is a national, interdisciplinary project (1998-2002) focusing on the use of a theatre metaphor for exploration, analysis, and construction of interactive, inhabited Virtual Reality environments. The project has partners ranging from dramaturgy and media science over architecture to engineering science, with representatives from several Faculties of Aalborg Universitet, Århus Universitet, Roskilde Universitet, and Center for Sprogteknologi (CST). The project is divided into three teams: on analysis, methods, and construction, respectively, and CVMT is leading the construction-team and responsible for implementation of a series of concrete interactive Virtual Reality systems, based on design input from project partners. CVMT has had a very fruitful collaboration with Institute for Dramaturgy at Århus Universitet (IDAU) and with CST. See also Virtual Reality Platform for interactive, inhabited virtual worlds, and Autonomous Agents.
Funded by the Danish Research Councils.
(Erik Granum, Claus B. Madsen, Thomas Moeslund, Rasmus Agerholm)
Virtual Reality Platform for Interactive, Inhabited Virtual Worlds / VR platform til interaktive, beboede verdener
To construct interactive virtual worlds a software platform is needed which can 1) simulate and maintain a dynamic 3D model of a scenario, 2) present this simulated world for users using computer graphics and audio, and 3) provide one or more users with interaction facilities. In conjunction with the STAGING and PUPPET projects CVMT has designed and developed such a Virtual Reality platform. The platform supports arbitrary scenarios but specifically for the PUPPET project an interactive farm scenario has been developed, where computer controlled animals and humans interact. Such computer-controlled characters are called Autonomous Agents, or simply agents. The user can freely navigate within this virtual world, which the software platform visualises in real-time on a computer screen (or in either of the VR MediaLab's VR arenas). The platform also enables the autonomous agents to move around (supported by path planning), play animations, utter recorded sounds, and change facial expressions. In this way agents can communicate themselves to, and interact with, other agents or the user(s). The platform also enables agents to continuously sense the virtual world, i.e., the agents have simulated vision, audio and tactile senses. A special aspect of CVMT's VR platform is that it supports the user in being represented by an avatar (an agent controlled by the user) in the virtual world. This allows the user to interact with the virtual world and its inhabitants on equal terms with the agents. The work on this platform in the reporting period has mainly focused on providing unique sound related interaction possibilities for the user. For example the user can communicate with the autonomous agents using sound (the agents can 'hear' the user through a microphone), and the user can record the sounds he/she wants the agents to use in particular situations.
Project under STAGING (Danish Research Councils) and PUPPET (ESPRIT).
(Claus B. Madsen, Panteleimon Kampolis, Erik Granum)
Interaction with virtual worlds and their inhabitants / Interaktion med vituelle verdener og deres beboere
This project focused on interaction in virtual reality by exploring and further develops the facilities at the VR-Centre. The project formed a part of the STAGING-project and in collaboration with the VR Centre. In spite of the experience from a substantial history for interaction with computers, the interaction with VR-applications is often rather primitive. A range of basic hardware and software problems regarding interaction in the VR installation was uncovered and solved. Interaction processes were analysed in the light of application contexts and the relevant combinations of input devices and display types. A series of interaction techniques for VR was developed, implemented and tested.
Subproject of STAGING.
(Erik Granum, Jens Peter Vester, Henrik Rojas Nagel)
Computer vision based interface to virtual reality / Computer vision baseret interface til virtual reality
The current interfaces in the VR-Media Lab's CAVE is based on physical devices with long wires. In this effort it has been investigated how natural gestures and computer vision can be applied to create more intuitive and convenient interacting methods. The approach is based on four infrared cameras tracking markers attached, e.g. to the user's index finger in order to estimate the user's pointing direction. This approach suggests that computer vision allows a more natural interaction with virtual reality by using gestures instead of traditional interfaces. Furthermore, this way of interaction is wireless and by that less disturbing for immersiveness.
Partly funded by VR-Media Lab.
(Moritz Störring, Thomas B. Moeslund, Niels Tjørnly Rasmussen, Jesper Kjeldskov, Erik Granum)
3D Visual Data Mining
Both private companies and public institutions regularly collect large databases, but much of the information content in those databases is difficult to extract. With VR-technology it is possible to create virtual visual worlds based on the characteristics of the data, so that visual data explorers can be immersed in these worlds and observe from within. The project develops and investigates the applicability of temporal visualisation methods for the purpose of detection of previously unknown structures and relationships in data. A new and flexible VR visualisation system has been developed and implemented, which allows visualisation of arbitrary
temporal developments of data, and furthermore makes it possible to study new forms of interaction. The VR visualisation system is being used by the members of the 3D Visual Data Mining project, as well as by other research
groups at Aalborg University. It has allowed the participants of the 3D Visual Data Mining project to develop new methods for exploring data in Virtual Reality based on arbitrary temporal data visualisation. The project is interdisciplinary with participation of computer scientists, statisticians, and psychologists, - all from Aalborg University.
Supported by the Danish Research Councils, 1999-2002. (Henrik R. Nagel, Erik Granum; M. Böhlen, Department of Computer Science; Steffen Lauritsen, Department of Mathematical Science; Peer Mylov, Department of Communication)
ARTHUR: Augmented Round Table for Architecture and Urban Planning / Augmenteret “rundbords-designværktøj” for arkitekter og byplanlæggere
ARTHUR is a 6 partner EU-IST-RTD project (IST-2000-28559), Key Action 4, Mixed Realities, that started in 2001 and will continue until 2004. ARTHUR will bridge the gap between real and virtual worlds by enhancing the users' current working environment with virtual 3D objects. The project focuses on providing an intuitive environment, which supports natural interaction with virtual objects while sustaining existing communication and interaction mechanisms. Real world objects will be used as tangible interfaces to make 3D environments attractive even to non-experts. ARTHUR will develop new types of user-friendly head mounted see-through displays (HMD), non-intrusive object tracking mechanisms and intuitive user interface mechanisms within a location independent multi-user real-time augmented reality environment. CVMT develops object and head tracking mechanisms based on computer vision using cameras mounted on the HMD. Furthermore, a computer vision based gesture interfaces will be developed allowing to recognize gestures without disturbing the user in his or her natural behaviour by cumbersome and wired hand tracking devices.
(Thomas B. Moeslund, Moritz Störring, Claus B. Madsen, Yong Liu, Abilio Langa Herrero, Erik Granum)
Real and Virtual Shadows in Augmented Reality / Virkelige og kunstige skygger i Augmented Reality
In Augmented Reality virtual objects are visually combined with real objects to create the illusion that the virtual objects are in fact just a part of the real scenario. While this presents many interesting challenges one area has so far received very little attention, namely the issue of shadows, or more precisely the issue of ensuring that the virtual objects are lit and cast shadows in the same way as the real scene. CVMT is developing techniques for estimating the positions of real scene light sources, and for estimating the spectral properties of real shadows, in order to apply this information in the rendering of the virtual objects. The result is that the virtual objects mix with the real scenario in a much more realistic manner. In the reporting period activities have primarily been focused at detecting shadows in real scenes, and using this information to develop a model for how virtual shadows should appear in the scene.
Project under BENOGO (EU FET), and ARTHUR (EU IST)
(Claus B. Madsen, Mads Sørensen)
FG-NET - Face and Gesture Recognition Working Group / Ansigts- og gestusgenkendelsesarbejdsgruppe
FG-NET is a 6 partners EU-IST Concerted Action/Thematic Network (IST-2000-26434), that started in 2001 and will run until 2004. The aim of this project is to encourage technology development in the area of face and gesture recognition. The precise goals are: (1) to act as a focus for the workers developing face and gesture recognition technology; (2) to create a set of foresight reports defining development roadmaps and future use scenarios for the technology in the medium (5-7 years) and long (10-20 years) term; (3) to specify, develop and supply resources (e.g. image data sets) supporting these scenarios, and (4) to use these resources to encouraging technology development. The use of shared resources and data sets to encourage the development of complex processes and recognition systems has been very successful in the speech analysis and recognition field, and in the image analysis field in the specific cases where it has been applied. The basis of this project is that when properly defined and collected, such resources would also be of benefit in the development of solutions to wider problems in face and gesture recognition.
(Moritz Störring, Thomas B. Moeslund, Michael Nielsen, Erik Granum)
BENOGO: Being There – Without Going / At få oplevelsen uden at tage turen
BENOGO is a 6 partner EU-FET project (IST-2001-39184), which started in 2002, and will continue until 2005. The project is coordinated by CVMT. BENOGO develops and investigates novel computer graphics rendering techniques (the so called Image Based Rendering approach) with the purpose of optimizing users’ sense of being present at a location, without actually being there. The BENOGO system visualizes existing, physical locations in stereo on a Head Mounted Display, or in any of VRMediaLab’s 3D arenas. The project’s rendering technique is based forming new images in real-time, and in response to user movements, using only data from previously acquired real images of an existing place. With this technology the project can circumvent the 3D modelling problems traditionally associated with standard Virtual Reality, and at the same time achieve a very high level of visual realism (photo-realism). The project consortium includes experts in the field of psycho-physics, human perception, and presence research, and these research domains continuously evaluate the project’s rendering technology to optimize the performance, and to develop a theoretical understanding of how the sense of presence is best provided. In addition to visualizing a world to the user the project also investigates the use of 3D sound, and the visualization is also augmented with virtual objects.
(Erik Granum, Claus B. Madsen, Mads Sørensen, Michael Vittrup, Moritz Störring, Henrik Nagel)
Software Platform for Real-Time Image Based Rendering / Software system til real-tids billedbaseret visualisering
Image Based Rendering (IBR) is a visualization technique offering some clear advantages over traditional 3D model-based computer graphics. Primarily IBR provides much higher level of visual realism. The disadvantage of IBR is that it cannot (yet) be supported by fast purpose-designed graphics hardware. CVMT is developing a software platform specifically for the IBR requirements of the BENOGO project. This platform integrates IBR software, traditional model-based computer graphics rendering, and 3D sound rendering with user tracker technology. The platform will enable a user to visually explore (move around in) a world being presented through visualization and sound. The primary challenge is to develop sufficient support for the massive data exchange and transformation required to do Image Based Rendering in real-time.
Project under BENOGO (EU-FET)
(Claus B. Madsen, Michael Vittrup, Henrik Nagel, Erik Granum)