The following blog is an edited transcript from a presentation by Lior Weiss, VP Marketing and Business Dev., Celeno, filmed at Wi-Fi NOW 2019.
Welcome to the Celeno session on Wi-Fi Doppler Imaging. My name is Lior, I'm responsible for marketing and business development at Celeno. For anyone unfamiliar with our company, Celeno is a Wi-Fi chip vendor. We provide connectivity Wi-Fi chips, Wi-Fi four, five and six, to OEMs that design home networking devices for service providers such as broadband gateways, set-top boxes and related devices. But in this presentation, I'm not going to talk about Wi-Fi connectivity, rather about Wi-Fi radars or Doppler imaging technology as we call it.
I'm going to discuss the driving forces in the industry that bring sensing technology into Wi-Fi, and specifically about the technology itself and how it comes together. Then I'll wrap up with some use cases and applications.
Radars have been around for a long long time, in the last few years, we've seen a flurry of activity and innovation around them, driven primarily by two industries. The automotive industry and IoT.
Whether in the home or in industrial and commercial applications, the IoT Industry has been driving a lot of advancements in RF sensors. Lately, we've seen innovation from chip vendors for more advanced radar technology. These include, UWB radars for IOT applications, 77 gigahertz advancements for radar in the autonomous driving application side, and a few months ago, Google announced that they are going to embed a 60 gigahertz radar in their pixel for gesture sensing and analysis.
So, we at Celeno took up the challenge and developed the first radar over Wi-Fi. This may sound crazy but there is definitely good business sense around it. Wi-Fi is so ubiquitous, it exists everywhere, from most homes to most businesses factories and manufacturing floors. It’s only logical to leverage and capitalize on this invested hardware for sensing and insightful information as well. When we talk about radar technology, we're referring primarily to two main capabilities: Object classification and motion classification.
Let's talk about how this is done by first looking at an active radar. We're not referring to a passive radar, leveraging off signals distributed by another device or another system. We're not relying on Wi-Fi links or Wi-Fi endpoints. Basically, this is a Wi-Fi device that serves as an active radar ,taking care of its own mission and distributing the packets it needs, in order to analyze and make deductions. It's an OFDM pulse radar which makes perfect sense since Wi-Fi as we all know is based on OFDM modulation.
So, we're leveraging that. Also, when one considers the indoor environment, it's a very challenging one. Radars have been around for a long time in the aerospace industry and the outdoors. Indoors presents a new set of challenges primarily around clutter coming back from walls and static objects like furniture. One needs to know how to deal with this clutter in order to identify objects of interest. Most importantly, I think, is the Wi-Fi Doppler Imaging technology that we have developed in compliance with the Wi-Fi standard, with Wi-Fi RF characteristics and all.
Lastly, it coexists with Wi-Fi networks, and has simultaneous networking and radar functionality. I believe we share the same vision and destiny with regards to Wi-Fi as many industry analysts. It only makes sense to leverage on Wi-Fi beyond just connectivity and utilize this technology for additional applications and use cases, such as Wi-Fi radar sensors. This in turn, will help drive the IOT industry. We've heard in the last day and a half, a lot about how Wi-Fi 6 drives and fuels the IOT industry and how it’s tailored to this set of applications, and we believe that Wi-Fi Doppler Imaging is going to assist in accelerating it even further.
Let's talk a little bit about the technology itself and how it actually works. Wi-Fi Doppler Imaging, as the name implies, relies heavily on the Doppler effect. The Doppler effect is basically the shift in frequency of a returning wave echoing back from a moving object. Obviously, the frequency shift is proportional to the relative velocity of the moving object. Micro dopplers are essentially the aggregation of the Doppler shift as it occurs from the movement of complex bodies that have multiple parts moving in different trajectories and in different velocities. The human body is a perfect example of a complex body. The human body has joints and limbs, arms, legs, head, and chest, and each one of them moves slightly differently.
So, the aggregation of a Doppler shift of an echoing signal from a complex body such as the human body generates what we call a Doppler signature. The evolution of a Doppler signature over time and the visual representation for it is known as a spectrogram. Here are some real captures of spectrogram tech, taken in the Wi-Fi domain of different body movements. We can see, a walk, a run, someone slipping backwards, someone falling forward and so forth. This generates very distinct and unique spectrograms. In fact, we believe that Doppler imaging is the perfect tool to classify these movements - emotions and body motions in particular.
In fact, it has such a fine resolution in granularity, it can even detect breathing. If you think about breathing, the chest and rib-cage movement is a very fine movement of millimeters per second in speed and velocity. But the Doppler Imaging radar is sensitive enough to identify that type of movement and even differentiate between different respiration rates as we see in this example of two different people breathing at different rates. So, that's a bit about the theory. Let's talk about how it's being implemented. The implementation consists primarily of three phases or three stages.
The first stage is the air interface. So, while very similar to a conventional Wi-Fi chip (which has a baseband modem signal processing) we've additionally implemented radar signal processing. The air interface is identical to that of a Wi-Fi system. The chip is generating packets back and forth between itself and other Wi-Fi and endpoints for data communication. But at the same time, it generates Wi-Fi packets specific for the radar operation and its receiver picks up the echo, the aggregated echo set that comes back from those packets after propagating in the room and reflecting back from various objects. The next layer is the signal processing layer and there are two main outcomes or outputs of this single processing layer.
The first one is the Doppler range intensity heat map, as we call it. This is a visual representation of all the echoes picked up at a certain moment in time from the overall coverage area of the device. This is the fundamental element which is used by the higher layers to deduct context, location, context classification and what not on the upper layer of the applications. The second output of the signal processing layer is the spectrogram. that we've discussed already. So, it's picking up on the Doppler reading, isolating the object of interest, collecting the Doppler over time and generating those spectrograms. Those two outputs are moving forward to the third layer which is basically the inference layer.
An inference layer includes a set of algorithms including some machine learning algorithm to generate two types of analytics. The first output is the object identification, location and tracking. That's on the left-hand side and the output from that is basically XYZ coordinates of the objects of interest. The second output is the classification of an object. It could be labeling the object, for example pet versus human, and it could be motion classification, for example a fall versus a sit-down and so forth.
At the end of the day, those are the stream of events that correlate to actual movements and motions happening in the coverage range of the device. Those two outputs are basically fed into an upper layer of business applications and we believe this technology is going to ignite the application development community to come up with legacy applications as well as new innovative applications that we have not even started to think about or anticipate. A few examples of applications and I'll talk about more common applications. I will try to show how Doppler imaging technology can actually enforce better applications with enhanced capabilities compared to other technologies.
Then we'll move on to home use cases and commercial use cases. So, on the home for example, we start with home monitoring applications.
Some home monitoring applications are intrusion detection, peace of mind applications and so forth. Some capabilities of the Wi-Fi Doppler imaging lend themselves very well. Obviously, motion detection is one, but the accurate tracking capability enables geofencing calculations as well. That means benchmarking or comparing the accurate location of the object of interest against the perimeter of the house or apartment in order to reduce false alarms due to a neighbor’s activity. Also, the fact that we can classify objects and differentiate between paths and humans also helps in reducing false detection. Another set of applications are around the entertainment system, TV analytics as we call it. This brings high interest to set-top box manufacturers, TV manufacturers, as well as service providers that provide TV services. The idea here is to understand who is sitting in front of the TV and if anyone is sitting in front of the TV at all. If no one is watching TV at this very moment, why not shut off the setup box or perhaps drop down the broadband bandwidth consumed by the set-top box?
Knowing how many eyeballs are in front of the TV, can drive a lot of personalized advertising applications and this is definitely the next frontier in some of the service providers’ advertisement strategy. Proximity and accurate location can help the operation of voice assistance and directive speakers on the entertainment system in terms of understanding precisely where people sit. Elderly care is a very sought-after application, a very lucrative application. Family members, consumer service providers, governments, all have a high interest in improving the life and safety of elderly people. I've mentioned before that Doppler imaging is the right tool to identify falls.
Fall is a very high dynamic motion that can be classified accurately by a Doppler radar. Breathing detection during the night is especially important. Contextual and analysis means tracking the daily dynamics of an older person. This helps to predict health deterioration over time as a result of abnormal behavior and such in advance. Accurate location can help with wander alert. Of course, this technology does not require line-of-sight. It doesn't inflict any privacy concerns, doesn't rely on any wearables, which are all pretty preventive with the elderly community.
Lastly in industrial and commercial use cases. There are plenty, so we're not going to get to each one of them, but factory floors, office buildings, smart high risers and so forth. Safety applications, energy, better energy distribution applications in large buildings and so forth, and of course all of those facilities use Wi-Fi today for networking. So, to wrap things up, there are a lot of capabilities in Wi-Fi Doppler Imaging technology that are beneficial as we've discussed.
The Wi-Fi spectrum basically enjoys a pretty long range. It doesn't require line-of-sight. It can go through walls. It doesn't bring any privacy concerns as it doesn't recognize face or body contours. Of course, it can present great Capex advantages by layering together networking hardware that are already there as well as sensing capabilities. People say that a picture is worth a thousand words, so perhaps a video is worth a million. So, here's a short clip that kind of brings it all together. This colleague is basically walking around in a room and what we can see on the left-hand side is basically the tracking output of the Doppler Imaging technology and on the right hand side, the stream of events out of the classifier of different motion and movements as the person here is simulating common actions.
So, with that, thank you very much. We'd be happy to answer any questions.
Audience: What additional costs would incur, for a systems’ company buying these chips, in order to enable this and the impact on throughput. Also, would this kind of a system benefit from being cloud connected?
Lior: As far as the impact on throughput, we estimated about three to five percent. It really depends on the motions that you're trying to pick up and so on. The air interface is kind of comparable to about a 40 kilobits per second stream. So, you can think of it like another Wi-Fi client with a slow stream requirement. Then, with regards to the cost, we basically positioned the chip as a Wi-Fi chip that can do more. So, it enjoys the same price points as any Wi-Fi chip out there because we have to compete under it anyway. In terms of the software licensing, it really depends on what the customer is interested in.
We have some license options for inference engines that we've developed but the fundamental signal processing is embedded within the chip.
Regarding the cloud services, I think the last layer is the business layer, maybe a dashboard for a service provider that delivers home intrusion services. Or it could be a mobile app in for elderly care to alert a family member in the event of an accident. All of those are of course driven from a cloud which is provided by a service provider or nursing home caregiver and so forth. So, there definitely is a place for cloud-based applications.
Audience: How long do you think it will take before it becomes mainstream for end user devices to start using Wi-Fi Doppler Imaging and Wi-Fi radar sensors?
Lior: We announced the technology in early summer. We've already announced a few partnerships, including with British Telecom, Orange, and we're also partnering with Comcast. I believe it's going to be about a year before we see some commercial grade solutions out there.
Audience: When do you see enterprise and industrial uses becoming mainstream?
Lior: The value chain is a little bit simpler in those types of segments. So, we are working with industrial applications, OEMs like some of the big names that are very common. Primarily for production floor safety, as well as a smart building real estate optimization and energy management solutions. These are typically delivered by OEMs and SI’s and again it's probably going to take about a year to design those systems.
Audience: Would you license the technology to folks that may not be using your connectivity chip?
Lior: So, I think there is a little bit of a technical challenge there. Wi-Fi radar sensor operations actually requires specific silicon related implementation. If you think about it, communication is a half-duplex operation, while radar is a full duplex operation, therefore, the architecture of the transceiver system needs to be a little different. So, right now it's offered only on our end.
Audience: This technology could also potentially be used for things like gesture recognition, are you looking at that as well?
Lior: Yeah, it's funny you’re bringing it up. In fact, we're in a dialogue with a TV manufacturer and indeed proximity and gestures is one big area of interest for them.
Audience: For gaming and things like that –
Lior: I think this is a pretty technical crowd so it's pretty straightforward. The wavelength basically dictates the type of gesture. So, the pixel for 60 gigahertz it's more about fingers - we're talking about hand gestures and that's what I guess most of the TV manufacturers are interested in.