Performances evaluation

Position Coverage

Manufacturer specifications:

Following specification were tested when distance between emitter and receiver is 29.53" ( 75 cm ). Operation at distance from 30" to 60" with reduced accuracy.

Our results:

What follows is the actual plot of the 56 sample points we recorded with the Wintracker unit. The measurement procedure consisted of placing the tracker sensor (labelled R on the following graph) at positions that were previously measured to define a perfect 7 x 8 points grid. The grid steps were as follow: On the x axis (the axis going out of the emitter front toward the user), we put grid marks starting at 10 cm with subsequent values spaced by 40 cm up to 290 cm. On the y axis (the axis side way with the positive direction toward the left side of the user facing the emitter), we put grid marks at -150 cm, -100 cm, -50 cm, 0 cm, 50 cm, 100 cm and 150 cm. The receiver was moved from one sample point to the next. For each sample position on the grid, our software took 1000 samples. For each sample point, we logged information about the minimum, maximum and average values for position and orientation information (6DOF x,y,z,h,p and r). Hardware data filtering was set off during our measurements and we only activated the first tracker receiver.

The emitter (labelled E on the following graph) was placed at the origin of both axes as shown below and remained static during experiment. Note that our measurements only took place on the x-y plane, where z remained near zero.

The x and y averaged results are shown at each sample points on the above grid. The red rectangle shows the area for which the position data was closely matching the theoretical values we had expected. Since we placed the receiver by hand at each sampling point, human error must also be considered in these results. Error margin for this experiment is of +/- 1.0 cm on each axis. Also keep in mind that these values are computed by averaging 1000 position samples. They must be considered as filtered positional values and not raw position values. Thus, it does not represent in any way the dynamic behaviour (i.e., stability over time) of the tracked position in space.

Angular Coverage

Manufacturer specifications:

all-attitude

Our results:

The returned attitude information was covering the full range of -180 to +180 degrees with the resolution stated in the manufacturer specifications.

Static Accuracy and Resolution

Manufacturer specifications:

0.12" (0.3 cm) RMS for the X,Y,or Z receiver position, and 0.6 ° RMS for azimuth, elevation, or roll receiver orientation.

Position resolution: 0.01 inch (0.03 cm)

Orientation resolution: 0.1 °

Our results:

By resolution, we assume that it represents the smallest value that the tracker can return for any given axis parameter. This was not measured as such by our test. What we measured was the static accuracy, which is linked to the dynamic nature of the data, and has a more critical impact on the usability of the system while in use in a real-world application.

To evaluate the static accuracy, we measured the minimum and maximum values returned for any given static position of the receiver. Next, we compute the deltas (difference between the smallest and biggest values) for every of the 6 DOF parameters. For each sample position on the grid, our software took 1000 actual samples.

The following graph shows the total summed error on the x-y plane for the position data components.

Regarding positional information, we can see that the operational range is quite acceptable from about 10 cm to 160 cm (5.3 feet) for all 3 axes. If we compare with the manufacturer specifications, we can see that the unit performed much better overall than what was given in the specifications. The operational range for positional information is quite satisfactory.

The following graph shows the total summed error on the x-y plane for the rotation data components.

The angular error on the heading, pitch and roll parameters was also satisfactory and well beyond the manufacturer initial specifications. The operational range of the unit regarding rotation information is located between 10 cm and over 180 cm (6 feet) away from the x axis of the emitter. While we took many precautions to avoid electro-magnetic field disturbance, the yellow zones on each sides on the graph shown above were apparently caused by nearby metallic objects in our lab. The red area on the above graph takes this fact into account.

The summed error for heading, pitch and roll values is now typically less than 0.3 radians on a surface that covers 400 cm wide by 290 cm deep from the emitter. This is really good news for HMD users that want to use the Wintracker over a medium-sized tracking volume. We actually tested the Wintracker using the new firmware on a typical VR application using an HMD.

In conclusion, considering all 6 parameters of the 6 DOF data, we can say that the operational range following the y axis is within acceptable limits from –120 cm to +120 cm. This is to say that the volume of operation of this tracking unit seems to be:

• 30 cm to 160 cm (5.3 feet) on the x axis
• –120 to +120cm on the y axis

The z axis of the operational volume can not be defined since we did not measure any samples along the x-z plane. Assuming the behaviour is similar on the x-z and x-y planes, we could expect to see a range from about -120 cm to 120 cm in the z axis direction of the volume.

The Wintracker is thus able to reliably track 6DOF data in a volume covering about 8 feet wide by 5 feet long and 8 feet vertically where the emitter is placed at middle height and in front or to the back of the user.

Here are some spatial views of the error repartitions we just discussed above.

This graph above shows the total summed error of the x, y and z components in centimeter units. It gives a good idea of the volume of operation which is essentially a function of the radius from the emitter to the receiver, forming a more or less spherical volume. Considering the units used here, the region covered by the darkest green (from 0 to 10 cm) seems an acceptable region of operation. Anything above 5 cm of total error will probably be too noisy to allow a stable and fluid tracking of a position in space.

Here we can see the total summed error of the heading, pitch and roll components in radians units. Since the error is given in radians, even a minimal error value may have a huge impact on the angles received by the host application. Taking that into consideration, we clearly see than any value above even 0.26 radians (0.09 radians * 3), which represents 5 degrees on each axis, is probably the worst we can allow. Thus, in the preceding graph, only the region covered by the darker green levels (1 to 3) would be acceptable.

The tracking error will be more or less critical depending on what the positional data is used for. In a virtual reality system, tracking the head will be much more critical than tracking a hand for example. Why? Because a tracking error of 1 cm at the hand level will barely be noticeable to the user eye. On the other hand, the same 1 cm of error at the head level will induce so much vision instability to the tracked user viewpoint that it may induce nausea. The angular error will be noticeably more critical regarding the user's viewpoint stability. It is thus important to take into account the above graphs considering what the tracker data will be the used for.