Drones and wireless video

Lately, drones have become very popular both as professional tools and for recreation and air sports competitions. In a series of articles we will present the different systems in a drone, including the ground support equipment. This article will focus on how wireless video systems are implemented to give a First Person View (FPV) to the pilot remote controlling the drone.

 

drone in flight

Drone or UAV (Unmanned Aerial Vehicle) are generic terms that includes many types of unmanned, remote controlled aerial vehicles. This includes fixed wing planes, helicopters and multi-rotors. Professional drones have a wide range of applications. Aerial photography during sports events does not have to rely on expensive full size helicopters, and real-estate agents frequently use drones for documentation. Drones discover missing people, and can monitor habitats exposed to risk of pollution. Electricity companies are now inspecting some of their high-voltage lines without expensive power outages and risky climbs. Even a conservative industry like the railway companies are considering drones for inspection of disrupted tracks in areas with limited access. Several companies plan to deliver small packages by drones, but it is not a commercial reality yet. This is partly due to regulatory limitations.

man with small helicopter drone

Fig. 1 Prox Dynamics Nano UAV (Photo from Prox Dynamics)

Larger military drones have been common for decades, but recently small stealthy nano UAVs have been developed for shorter reconnaissance missions. Figure 1 shows the Prox Dynamics Black Hornet which is also intended for use by rescue workers. It can give situational awareness, or discover victims trapped in collapsed building structures.

man using drone goggles

Fig. 2 FPV with video goggles

Wireless video for first person view

Drones can be piloted in two different ways, either line of sight by visually observing the drone, or by First Person View (FPV). In an FPV system the video image from an onboard camera is transmitted by radio to a personal video display on the ground in the form of a screen or video goggles. Figure 2 shows a typical set of video goggles with circular polarised antenna and embedded receiver. Systems span from simple low-cost systems, to advanced systems with high-power video transmitters and ground receivers with directional tracking antennas that offer ranges of tens of km.

Challenges and antenna solutions for wireless video

The range of the wireless video link is limited by a number of factors. The path loss itself will diminish the signal when distance increases, and obstacles in the line of sight can give additional attenuation. However in a natural environment there are some less obvious challenges to the radio-link that require clever solutions. We will take an in-depth look at the two main issues.

Interference

Other sources of radio transmission in the environment can interfere with the main signal. If the interfering signals occur in the same frequency band as the wireless video link it will act as inband noise. This will reduce the signal to noise ratio, resulting in a noisy video image and limited range of the link. A typical interferer can be the video transmitter of another drone in the area, a nearby WiFi hotspot or mobile phone. The problem can be minimised by selecting a channel as far away in frequency from the interferer as possible, or by moving the video receiver and antenna. If the source of interference is powerful, but outside the band of the wireless link, it is called a blocker. The blocking signal can penetrate insufficient front-end channel filtering, and decrease the dynamics of the Low Noise Amplifier (LNA). A simplified diagram of a receiver signal chain is shown in Figure 3. Typical high power blockers can be radars, broadcast towers or military radios. Technical measures for handling interference can be to ensure good front-end channel filtering of the video receiver, and use a directive ground antenna to minimise interference from other directions. Directional antennas with a narrow beam and high directional gain will also increase the receive strength of the signal from the drone. The antenna can even be equipped with a tracker that automatically directs the antenna at the moving drone. A tracker takes GPS coordinates from the drone as input for the control system and the tracking algorithm.

illustration of simplified receiver signal chain

Fig. 3 Simplified Receiver Signal Chain

Multipath fading due to reflection

Even with a strong, noise free signal, a radio link can get sudden dropouts, especially in cluttered or urban environments. This can be due to the reflected propagation path cancelling the direct propagation path. The cancelling occurs because of the phase shift associated with different propagation delay. This occurs at a specific point of the receiving space, and can disappear just by moving the antenna less than one wavelength. In addition to signal cancellation, multipath propagation also results in symbol delay spread. The symbols from the various paths arrive at different time, causing bit errors if the delay is significant. Figure 4 shows the principle of multipath propagation and delay spread. The two main strategies to deal with multipath fading are avoidance or constructive combination of reflected signals.

illustration of multipath propagation

Fig. 4 Multipath propagation

diversity receiver

Fig. 5 Diversity Receiver with circular polarised antennas, one directional and one omnidirectional

Avoidance of reflected signals

The most intuitive way to avoid reflected signals is to have a directional focus at the line of sight. As explained previously this can be accomplished with directional antenna systems. The reflected signals will approach at an angle outside the main lobe of the antenna, and will be attenuated. Another less intuitive, but yet simple way is by adding antenna polarisation. The most effective type for this purpose is circular polarisation, where the radio wave due to antenna shape will have a twisting propagation. The receiving antenna with matching polarisation will pick up this signal but will suppress un-polarised signals. Reflected signals are avoided because the reflection disrupts the polarisation. Figure 5 shows circular polarised antennas connected to a diversity switching FPV receiver. An important factor to consider is antenna positioning on the drone, because losses are introduced when the angle between the transmit and receive antennas is increased. At exactly 90° the loss is theoretically infinite, so a combination of antennas with different angles could be beneficial for a drone. This strategy could as well diminish the impact of flying into the antenna radiation pattern null. Omni directional antennas have a doughnut-like radiation pattern, with a null at the perpendicular axis. When flying directly over the ground station antenna, the drone can hit this spot. The strategy of reflection avoidance has one major downside, and that is the dependence of a line-of-sight path. When reflected signals are suppressed, major obstacles in the radio path cannot be handled efficiently resulting in a rapidly declining signal.

Constructive combination of rejected signals

Constructive combination of reflected signals can solve not just the multipath fading problem, but also maintain sufficient signal strength when path obstacles occur. A commonly used mechanism for constructive combination of multipath signals is diversity. Diversity can be employed both on the transmitting and receiving side, by having two or more antennas connected. When the antennas are spaced by a distance of around a wavelength or more, the probability for severe signal cancelling is greatly reduced. The signals from the antennas can be passively summed before entering the receiver, or the receiver could actively switch to the antenna with the strongest signal. The radio can also have duplicated receive chains and sum the signals at base-band.

illustration: mimo with base band weighting

Fig. 6 MIMO with base-band weighting

A more sophisticated mechanism evolving from diversity is Multiple Input Multiple Output (MIMO). Here a higher number of antennas are used to be able to receive and analyse several multipath signals. The phase and amplitude of the signals at each antenna is measured and compared, and an advanced and fast algorithm calculates how each signal must be processed to make up the wanted signal. Each signal must be precisely phase shifted and gained to enable constructive combining. This process is called weighting. Combination and individual treatment of a higher number reflected paths, gives a much higher signal to noise ratio than diversity. Consequently more advanced modulations can be employed, resulting in higher link bandwidth. The weighting of each signal can be done at different stages in the receiver signal chain. With the advances in compact and low-power digital processing, it is most common to do this on the base-band samples after analog to digital conversion. Figure 6 shows a simplified block diagram of this principle. The MIMO concept is widely used in LTE (Long Term Evolution) or 4th generation mobile data transmission, and has recently been adapted for use in high-end digital video links for FPV.

FPV camera

The camera used in FPV setups is typically light and compact, but robust and very light sensitive. Apart from the optics, the image sensor is the most critical part for high quality video. There are currently two main types of image sensors available: Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD). CMOS sensors are most commonly used. They are inexpensive, require low operational voltage, and have lower power consumption than CCD sensors. Most drone pilots demand high performance video, even when moving fast and during vibrations In this area CMOS image sensors have some shortcomings. Images are actually scanned pixel-by-pixel, line-by-line, and there is a limit to this scanning speed. This way of capturing the image is called rolling shutter. On a fast moving drone with high frequency vibration from the motors and propellers, artifacts, distortion and even wobbling will be noticeable on the image. CCD cameras on the other hand has a different way of collecting the image. The light-induced charge of each pixel is collected from all lines simultaneously. This is referred to as global shuttering and is the reason why CCD cameras can handle fast movement and vibration without image distortions. The cells or pixels require very small amounts of light, and hence the sampling of the full frame can be fast. Since the CCD readout and signal conversion requires less active circuitry on the sensor itself, the raw pictures will contain less noise than the output from a CMOS sensor. CCD sensors will also have a higher dynamic range, i.e. a bigger difference between low-light threshold and saturation of a pixel. The best CCD cameras for consumer FPV setups are sufficiently light sensitive even for night flights, with only a few light sources present. CCD cameras however are more expensive, and require a higher operational voltage to run the charge-coupled cells in the image detector. The signal conversion circuitry also consumes more power than the distributed CMOS pixel amplifiers.

 

Analog FPV transmission

camera, analog transmitter and circular polarised antenna

Fig. 7 Camera, analog transmitter and circular polarised antenna

Most consumer, and even professional FPV systems still use analog wireless video transmission, with very compact and low-cost systems. Figure 7 shows a complete analog FPV transmitter system giving approximately 1 km range (an AA-cell is shown for size comparison). Analog video is real-time and requires no image compression and advanced processing. This results in near zero latency between the image captured by the camera and the one viewed by the pilot. Latency of just tens of milliseconds is very noticeable when flying fast with FPV, and the hundreds of milliseconds often seen in standard digital High Definition (HD) systems for consumer use will pose a safety threat. Another surprisingly favorable property of analog video is the gradually increasing picture noise when the video link starts to break down. This occurs when reaching the boundaries of the radio range, or encountering interference or obstacles. A pilot experiencing this instinctively knows that he has to turn around the drone or avoid an obstacle. With a consumer grade digital video link over radio, a rapidly decreasing signal to noise ratio will often result in stuttering and frozen video frames.

Digital FPV transmission

The last couple of years intense development has gone into creating compact and low latency HD video links. The digital radio transmission system needs robustness to preserve an acceptable framerate even under rapidly deteriorating radio conditions. Fast adapting dynamic modulation schemes are applied, to narrow the required bandwidth during a period with reduced signal quality. A reasonable framerate is maintained while restricting video resolution. A wireless high framerate HD videolink has to carry very high bitrates, and hence an advanced modulation is needed. This puts a strict requirement on the signal to noise ratio, so a variant of the antenna MIMO concept previously explained is applied, together with Orthogonal Frequency Division Multiplexing (OFDM). OFDM is very effective for communication over channels with frequency selective fading (different frequency components of the signal experience different fading). With a traditional single wideband carrier, frequency selective fading is complex to handle. OFDM mitigates the problem by converting the high speed serial data into parallel low bandwidth subcarriers. Some subcarriers are reserved as pilot carriers (used for channel estimation/ equalisation and to combat magnitude and phase errors in the receiver) and some are left unused to act as guard bands. The reservation of subcarriers to guard bands helps to reduce the out of band radiation, and thus eases the requirements on transmitter front-end filters. In the receiver fading correction is applied to each subcarrier. This can be seen as a form of diversity, called Frequency Diversity. OFDM is a well-known principle from WiFi and Wimax systems, and DAB broadcasting. Figure 8 shows simplified block diagram of the OFDM concept.

Illustration: OFDM transmitter principle

Fig. 8 OFDM transmitter principle

The underlying modulation is dynamically selected depending on the available signal to noise ratio of the channel. Quadrature Modulation (QAM) is a modulation form where each constellation represents certain amplitude and phase of the radio signal. At 64QAM one symbol equals 6 bits and yields a high bitrate per subcarrier, but this constellation can be used only under good signal to noise ratio. Under deteriorating channel conditions the modulation steps down to 16QAM, and continues to simpler constellations if the channel worsens. For very bad conditions Binary Phase Shift Keying (BPSK) with only two constellations is employed, where each constellation represents a phase shift.

Interference outside the channel

Fig. 9 Interference outside the channel

Interference inside the channel

Fig. 10 Interference inside the channel

With one bit per symbol, it gives a very modest bitrate, but is correspondingly robust for noisy conditions. In this state the video resolution and framerate is just sufficient to fly safely for a short duration. Figure 9 and 10 shows a simulation of how interference outside and inside the channel affects a 16QAM constellation.

 

 

videosnap2

Fig. 11 Preserved video during reduced signal strength

videosnap1

 

 

Unlike WiFi radiolinks, a low latency FPV link is cannot be dependent on acknowledge for every data frame, and does not support re-sending of failed data frames. Instead Forward Error Correction coding (FEC) is employed, to handle most of the occurring bit errors. The remaining few failed frames will be part of the displayed video stream. This can be observed as occational macroblocking in the image, but will not affect the image quality much if the adaptive modulation reacts quickly during worsening radio conditions. The pictures in figure 11 shows screenshots taken during a drive-test of a digital wireless HD video link. It is a good illustration of an adaptive modulation scheme operating.

Future development

Wireless video for FPV drone piloting is still immature technology, and we will see compact and low-cost HD FPV systems emerge in the near future. The key to lowered cost is increased integration of systems on chip and a resulting high volume. Paradigm shifts will occur when totally new radio, camera, or display concepts emerges. The next generation of cellular and WiFi technology, termed 5G will exploit dynamic beamforming to increase system gain and keep interference low. Together with more sophisticated MIMO this will increase performance and transmission bandwidth further. It is likely that these concepts will be applied for future FPV systems when the technology matures. This will result in higher performance with extended range, higher image quality and better reliability. It will enable drones to handle more of our present challenges, and those we have not come to think of yet.