signalspot.com

With the use of WebCamera we have tried to create an alternative way of controlling a movie and sound-clip. We have succeeded in developing a system where you can play, pause and fast-forward the clip, zoom in/out and adjust the volume by using your hand and a red glove. We have used color tracking to track the red glove and afterwards performing blob tracking to output coordinates of the movements of the hand. These coordinates are used to perform decisions on which actions to do with the movie- and sound-clip. We encountered different problems during the experimental work. Some of them were predicted already in the starting phase such as camera noise, resolution, change in illumination, incoming objects andsome Groundnut Problem. The only problem still standing is the problem concerning incoming objects with the same color as the glove. The well-known illumination problem was partly solved by using the HSI color model instead of RGB. The Groundnut Problem, where the system doesn’t know if it is active or not, were solved by defining an open/close command that included clenching the fist. All in all we have reached our goal, and we are satisfied with the outcome of the project and the final prototype of the Automatic Remote Control.

(Note: This Paper was presented in ICSIP Please Download the paper for proper formatting, images and symbols)

1. Introduction

In this project we will try to create a Automatic remote control that will suit the purpose of a standard remote control, as we know them nowadays. However as we define the Automatic Remote Control (ARC) it is not going to fulfill all the needs of a common remote control, only some of the basic features. The future goal of the Automatic Remote Control (ARC) will be to teach the computer common human gesticulation. A computer will at best be able to recognize some patterns and respond to them accordingly. Therefore the aim of the project is not to produce a fully developed consumer product that will replace the standard remote control, but rather to make a prototype and to learn from the process of experimenting during the project. The report will serve as documentation of our experiments, both successful and not, and as a summary of the theory used.

This project tries to answer the following question:

What is required to develop a system where users are capable of controlling movie- and sound-clips using their hands as controller?

2.The Web Camera Experiments

Thisreport is divided into three steps:

- The red hand experiments, where the goal is to isolate the red hand using blob tracking and

extract the coordinates from the blob.

- Offline experiments (recorded footage), the goal of this step is to implement the interaction

with the movie clip (on/off, play, stop etc.).

- Online experiments (live footage), the aim of this step is to use the offline experiment patch

and make it work with live input.

Problems to be considered :-

Before conducting the work with the experiments we tried to pin down likely problems that might influence our work, and the functions of the final patch. These problems are camera noise, resolution, change in illumination, incoming objects and the notorious “Groundnut problem”[1].

Camera noise

When working with digital video and digital image processing in general, noise is always a parameter to be taken into consideration. Noise is not part of the ideal signal and may be caused by a wide range of sources, e.g. variations in the detector sensitivity (camera quality) and different environmental variations. Noise may cause problems when applying color tracking, because some of the pixels vary slightly in color value or intensity over time. There are different ways to get rid of noise, such as the median filter.But all pre-processing steps we apply to our digital video will slow down the process and we may run into problems if the Web Camera patch gets too

extensive .

Resolution

In addition to the noise problem, there may be a problem with the camera resolution. We will use a digital video camcorder with a resolution of 720×576 pixels and a frame rate of 25. This resolution is most likely too high, because of all the computations we will need to apply on the captured video.

Change in illumination

The illumination of an image is affected by the lightning and the reflections from different objects. If the lightning in the scene changes, objects will reflect more or less light and therefore change appearance. A change in illumination will cause all the pixels in the image to change intensity and color. This is a major issue when working with digital image processing. If the color of the red glove changes during the experiment, caused by illumination changes, the color tracking process might be difficult to perform. Therefore, we want to do the preliminary experiments in a closed environment, where we can manage the lightning ourselves. The problem of illumination is almost impossible to solve totally, therefore we don’t expect to solve this problem.

Incoming objects

If our experiments are disturbed by unwanted motion, such as a person walking in the background, it may give us problems when we want to extract the movements of the person wearing the glove. We want to know exactly where the glove is in the image, and how the position of the glove changes over time, so we can control a multimedia clip accordingly. But if a person or some other object disturbs this process, it will probably fail. Especially if the object has pixel values close to the color we want to track.

“The Groundnut Problem”

When will the system know it is being used or that the user is just eating Groundnut ? This is a problem we will have to solve in some way. We have to find a way to tell the system, whether it is active or not. Otherwise, the user can’t do anything else with his hands while using the system, e.g. reaching for the Groundnut bowl. A possible solution could be that the system only works when the user moves his hand in a certain area. Then, if the hand is outside this area, the system won’t react to the movement of the hand.

3.The Red Hand Experiments

Here various experiments were made to test out the color theory, including color spaces and blob tracking in connection with Web Camera.

These experiments took us only a little closer to the final goal of getting value out of a tracked blob that could be used in controlling a movie clip. Although we might have taken the wrong approach, they proved as experiments, to give valuable information about how the different tracking features ofWeb Camera worked. The x,y coordinates from the blob tracking in the Coordinates experiment coincide with where the red hand is at, at a given moment. The task is now, to take these coordinates further and transform them from being just that, coordinates, and into values that can control properties of a movie clip.

Before we go further with the interaction we have to stabilize the tracking, because without stability everything else is useless. The values from an unstable patch could cause the volume to rapidly go up and down and/or pause and fast-forward the movie clip continually[1].

4. Offline Experiments :-

These experiments are built on the foundation from the red hand experiments. We will try out different experiments on a number of different backgrounds with various amounts of movements and different lighting situations[3].

Offline Experiment Set up :-

In order to have good uniformed footage we had to make the exact same set up for every environment we recorded in. We put the camera on a tripod to get good full view of the “actor”, and placed tape on the floor for the actor to know where to stand on each recording. We measured the distance from the actor’s feet to the wall, from the actor to the camera, and from the lens of the camera to the floor. In addition the zoom of the camera were the same for every recording as well. Therefore the setup was the same on every location. The set-up shown in Figure 2.

The actor moved a red glove upwards/downwards, sideways and by turns closed and opened the hand. These were all moves, which we were going to work with in Web Camera. We actually did recordings with both a blue but a red glove and ended up using only the red.

Attached Files:

• dip02228.doc (140 KB)

Attached Files:

• dip022281.doc (140 KB)