Latency

Manufacturer specifications:

15 milliseconds unfiltered and 30 milliseconds filtered, from beginning of measurement period to beginning of transfer from output port.

Our results:

We have not measured latencies in our tests but it is worth mentioning a few things about it. Latency, also called "transport delay", is the time it takes to see the effect of an external stimulus entering the system up to the point where a change of state is perceived at the other end of that system, the output. For example, let say the user moves his hand which is tracked by a sensor. At the very moment the sensor begins to move, the tracker unit will perceive a change of position. Some data processing may also take place on that positional information. By the time the corresponding 6DOF data packet is received by the host computer, some time has passed. The time it took for the use hand movement to concretize into a new 6DOF data inside the host VR application is the latency or transport delay of the tracking system. We could compare latency to a time shift of the appearance of a data at the system output, when it is travelling inside a system, from its input to its output. So, it is the time it takes for that data to move from the input to the output. If the system had zero latency, the data would instantly travel through the system.

Latency must not be confused with throughput. Even thought both refer to the notion of speed at which data propagates inside a given system, they define quite different phenomena. The Latency is, like we said earlier, a time shift of data traveling inside a system. On the other hand, throughput refers to the amount of data that crosses a given point of the data path, on a specific period of time. Going back to our tracked hand example, lets say the user starts to move the hand at time = 0 second. After 32 milliseconds, we still do not see any change at the output of the main tracker unit. Exactly at time = 32 milliseconds, the first bits of the data packets start to arrive at the tracker output. From that moment, the data flows at a rate of 1 megabit per second, uninterrupted. During all that time, the user has not stopped to move so that its hand is constantly at a different position in space. It is also worth noting that, at any given time, the user's hand position and the position dictated by the current tracker data will be referring to a different spatial position. At time = 2 seconds, the user stops moving. At the tracker output, we still receive bits of data until time = 2 seconds + 32 milliseconds. It is only 32 milliseconds after the user has stopped moving, that the outputted data reflect that change. What does it tell us? That the latency or transport delay of the tracking system is of 32 milliseconds, regardless of the amount of data that it can send to the host computer. As you noticed, the latency is not simply the time it takes for a bit to travel from point A to point B, it is a measure of the induced processing time that a system adds to the time of travel of that data.

Now that we have a better understanding of what is latency, let us look at its impact on a case more specific to virtual reality systems. Lets assume the VR system is composed of a tracked HMD that the use wears to see a virtual environment. The manufacturer specification states that tracker latency can be either 15 or 30 milliseconds, depending on filtering options. This represents either 1 or 2 frames of delay for a virtual reality system displaying 60 frames per second of 3D imagery. Practically speaking, if you are tracking the user head movement, the effect of turning the head will not be visible before two images have been rendered by the VR system and thus, displayed to the user wearing the VR HMD. This may seem negligible, but it is not. The human visual system is very sensible to even small delays like these ones. It is often accepted that the human brain can tolerate latencies of less than 100 milliseconds. Anything higher than 150 milliseconds will more than likely cause nausea and other severe physical indispositions.

So, it becomes clear that this parameter will be of premier importance to consider while integrating a tracker system to other elements of the global VR system. The VR system designer has to take into account that all these latencies add up to give the global system latency. It is that global latency that must be within 100 milliseconds or less, not each individual latencies. In the present case, this means that, assuming the tracker has a 30 ms latency, the rest of the VR system has a latency margin of no more than 70 ms. This still give room to work, but keep in mind that it is common to have latencies of 2 to 3 frames (30 to 48 ms) just on the rendering stage of the VR system. In such a situation, our worst case would be 58 + 30 = 78 milliseconds of global system latency. This represents close to 5 frames of latency on a frame rate of 60 images a second. It does not seem too bad in numbers, but consider that most humans will notice flicker at update rates equal or below 30 images per seconds looking at fast moving scenes. This is to say that a latency of as little as 32 ms may already be noticeable to the eye in some situations. This being said, there is a distinction to be made between latency and frame rates. A latency is a time shift of some punctual or repeated event that occurs in time. A frame rate is the time spacing between two events occurring inside a system, it is the throughput. For example, a simulator rendering 60 frames per second effectively dispose of a maximum of 16 milliseconds to render each of these 60 frames. Thus, a frame represents 16 ms. Even if this simulator was only rendering 30 frames per seconds, it could be said to have no latency at all. That is, the system is rendering each frame at a certain rate. As long as that rate is kept constant and predictable , the system is stable and does not "lag", a jargon term often used to refer to latency or its resulting visible consequences. No matter the frame rate of the system, it can also have latency. So in that case, the system is taking 16 ms to render a frame, but it may take even more time to react to external events such as user movements. Back to our tracked HMD example, even thought the VR system is sending a sustained image throughput at a rate of 60 images per seconds (60 Hz), the scene will not move instantly with the user head movements. It will take 30 milliseconds before the head movement translates into a scene movement as shown inside the HMD.

The bottom line is, always keep an eye on latency sources in your design and try to keep it as low as possible. The global system latency should never go beyond 100 milliseconds in order to avoid extreme physiological discomforts, often referred to as "motion sickness" symptoms.