This article comes from a recent Forbes Futures In Focus podcast interview with Ron Martino, executive vice president of global sales for NXP Semiconductors. He discusses the edge economy and what edge technology may need to look like moving forward.
getty
We’re entering a new age of compute. We may not see the edge, but we will surely experience a dramatically different world of compute ushering in new experiences and economic models. How would you define the edge compute world and what it looks like?
Ron Martino: Edge computing goes beyond the traditional type of computing, such as existing mobile phones or personal computers. It involves small, distributed computers embedded in all the devices around you. Those devices are sensing local environments; they’re establishing and understanding and the context of that environment. And then we’re enabling those devices to perform actions that are independent of human interaction or command, but consistent with what you desire. These devices also interact to build a collective intelligence among themselves and then take action based on how they understand the world in which they exist — whether that’s in your home, work environment or even in your car.
Can you work through some examples and where you think we will be in the next decade or so?
Martino: Let’s consider agriculture. Farms around the world use about 70% of our fresh water, in terms of annual consumption. It’s estimated that half of the water is used for irrigation — this was determined via devices placed out in the field. You can use that data to get insights in terms of the conditions of the crops and the needs around irrigation, specifically for a given field or crop. You could apply the data to determine whether what’s needed is water or fertilizer or pesticide, and it can be applied in a much more efficient way. These are just a few examples of how we can dramatically eliminate waste and also improve the productivity of farming.
The ability is there today to collect the data. It can come in via more very efficient small devices that can run on different energy sources, whether that’s the sun or extracting energy from the soil through a photosynthesis process. Or it could be part of equipment that is used to actually do the farming, integrated into the sensors in the larger compute inside those vehicles. And all of that can then be leveraged through a different type of local compute, as well as combined with cloud computing to determine and optimize the productivity and efficiency of that farming operation.
You speak about cloud computing and farms. How far is this from the chalkboard to actual practical examples? Have you seen applications in large-scale agriculture where this is now happening, or is this still mostly on the whiteboard?
Martino: This is happening today. The majority of farming is done in an autonomous way. And the data is being fed in from tractors and other farming practices into a cloud. There is a local capability that efficiently operates the vehicles, and then there’s a collection of data that can be preprocessed locally and then sent for higher compute analysis or more complex analysis of the data. This is in play today. It’s in play in your home as well. Not just from an agricultural point of view, but in terms of placing more compute capability in your thermostats in your home or in controls for your lighting, and looking at how to operate those at more efficient times in terms of power consumption and cost. The data can help determine when to shut off lights and use energy when the environment is not active.
So these are the sorts of inside mechanisms or applications that we may not always see with our own eyes. What are other applications where a very small-footprint, high-powered processing skill that uses AI can really start making substantial differences to our daily lives?
Martino: So let’s talk about it at a very high level first — this type of computing and how its evolution will enhance productivity. In automotive, injuries and fatalities can be reduced when you’re mapping the environment and taking action when dangerous situations are identified. Let’s also consider the improvement of your personal health when optimizing sleep quality of the air, as well as the overall conditions of the environment that you’re living in. The biometrics or the data you collect monitoring your body conditions — that data can help lead to making better choices. With this type of computing, where more and more data is being created, if it’s deployed properly, it can better protect your personal data.
It can also provide another meaningful impact on society around the achievement of sustainability goals such as creating a greener world with more efficient use of power. All of this can be done through the proliferation of these billions and billions of connected devices and managing new sources of information. We can take all this data and collect and process it to be able to leverage new insight and take action.
These are independent devices functioning on the edge in the cloud. They are super-intelligent and will be better at managing data in real time than human beings are. These billions of devices — that’s not a small population — will engage with all of us in various ways and places. What sort of societal changes do you think may come out of this, or start to occur 10 years from now when this becomes a common process?
Martino: Current forecasts estimate that there will be 75 billion connected devices by the next decade. Just as an interesting fact: Today we generate about 64 zettabytes of data — that’s 64 followed by 21 zeros. It’s a massive amount of data and it’s only going to increase by orders of magnitude in the next decade. When you think about the amount of information that becomes available by deploying this, and the desire to not waste it by extracting valuable information, you’re going to need compute devices that must operate locally where the data’s generated, not use significant energy, and then transfer the information most efficiently. In terms of its impact to society and how it will be deployed, it’s going to come into play in meaningful ways.
For example, it can help in reducing your energy bill. Or it can improve the safety of your family. In terms of entertainment, as you may interact with media and sound in environments in a way that is more enjoyable. We’ll also have to consider the issue of data privacy and security of personal information, especially with the addition of devices that can sense, hear and record your daily life. What’s taking place in an environment will require appropriate control by the owner and doing things that they desire while protecting them against bad actors that might be interested in gaining access. From an industry point of view, addressing fundamental issues regarding security and data privacy becomes a key part of the discussion. We’ll need to determine how to best deploy technology in a way that properly manages the needs of individuals as well as the benefits that technology will bring.
Let’s dive further into the idea of security. For example, would blockchain technology play an important role in the solution? It would require cloud-to-device interaction and integration.
Martino: There are different forms of security, and we’ve been working with financial and banking industries and governments in order to protect data and ensure secure transactions. The types of security discussions that are critical include enabling system protection under different conditions. The ability to securely bring up a system so it doesn’t have vulnerabilities and can identify that a device is secure and has an identification that’s unique becomes very important. We must also think about how you provision it, then how you commission it into an environment that is targeted for the use of that device — be it your home or work environment — and then how you manage its security on an ongoing basis.
For example, how do you protect from physical attacks or remotely connected attacks and enable updating these devices over time, or effectively remove them from an environment so that they can’t be used to gather information? Then there are concepts around confidential computing that come into play, on collecting data from multiple sources and not losing privacy but getting the benefit of processing the data to extract valuable information to protect your personal information. A great example of that could be personal health data — looking for information to help society or a community but not revealing the personal information of a specific individual.
Yes, for example things like orphan diseases, where there is a vast array of some very harmful but disparate, low-incidence diseases. If you could share some of that data blind, between one patient to the next, you could create a much larger and very beneficial database of knowledge around a particular orphan disease. It’s tough to measure unless you’re in a particular institution database.
Martino: Absolutely. Medical applications around the tracking of disease, as well as other aspects of sharing information that could be useful for a community, clearly come into play when you’re looking at that type of processing of information. It’s a capability that exists today, and it’s is evolving to higher levels of energy efficiency and ease of use. Management of these devices plays a key part of growing broad adoption. People have to trust these devices, trust the security and find it easy to deploy without the challenges of managing a high-tech environment.
As you think about the lifecycle process, safety, security and all the things that really matter, what are the triggers that are going to turn this into an endemic norm?
Martino: I’ve touched on a few of them. First is examining the meaningful end result of, say, putting a device into your home or your work or your car. As the benefits of having these different devices around you become clear, they’re going to be more broadly adopted. If I tell you I can cut your energy bill in half, you’re going to be very interested. And if it’s easy to deploy, you will have little reason to not deploy it.
What will make it even more attractive is the ease of putting the device into your home and the interoperability of that device with other devices in a simple interface for easy control. And then when an issue occurs, having the ability to resolve it in a very simple way or have the system identify what needs to be fixed.
When it comes to industrial applications, it’s going to be about enhancing the output of the factory through automation. Improvement of the safety of your workers, which will make your workplace a more healthy and attractive place to join are other examples of addressing economic and social motivations.
This is interesting because it actually puts that data accumulation into practical or contextual action in an environment. In a factory environment, currently there must be some unique outputs, where organizations are using insights not just for safety, which is important, but also for productivity. It must be opening up windows of opportunity for companies to look at the data and the interactions as being as valuable as the actual product they produce.
Martino: Yes. Specific to industrial applications, industrial control and automation are important areas for us. Half of global waste is caused by manufacturing inefficiencies. If you combine these devices with sensors and machine learning at the edge, you have more energy-efficient devices and you can conduct a real-time reaction. You can significantly reduce waste and improve the output of the factory. We’re working with many different industrial customers to put scalable platforms of computing in place that can automate and control their manufacturing environments in a much more efficient way. That’s done through a range of different computing capabilities combined with other capabilities to make the computer more intelligent.
And what do I mean by that? It’s applying machine learning. And that machine learning is basically a model that will allow the machine to understand the scenario, what activities are taking place, who is in the environment and whether they are in a distracted condition or in danger — and then take action. Those are just small examples of where industries are looking to either improve efficiencies or improve safety and leverage this to drive enhancements for their company.
Right. You can already see this type of ability with the distraction mode in cars. If you take your hands off the steering wheel or there’s a sudden motion, there’s a warning that you might be distracted. And I think that there’s a natural extension of that experience into a more complex industrial landscape.
Martino: Yes. Another area of significant growth is the way you interact with machines. The human–machine interaction and interface is an area of significant growth. It is relevant, relative to health and being able to interact with different systems and machines without touching them. Having a cleaner environment and ease of interaction, whether it’s using a voice- or a camera-based system in order to achieve new efficiencies. If you’re hitting buttons or using interfaces through more of a mechanical interaction versus a natural human interaction that the machine can understand, you can work with them in a much more effective way. That’s another significant aspect of investment in innovation: enhancing that human–machine interaction so that it feels natural and creates a safer as well as healthier environment.
Right. People have a lot more sensing capability than just trying to use fingers on a keyboard, which is a really limited way of interacting with the machine.
Martino: Absolutely right. There’s a lot that you can do to make it a seamless experience. You can easily detect where an individual is staring, for instance, and you can adjust the environment to make it more effective and comfortable for that person to use, or make it safer to use. You can combine a vision-based classification of who you are and that you’re speaking, so that the machine can not only understand the words that are being stated but who’s stating them. And conveying your state in terms of emotion — are you happy? Are you laughing? This can provide context for what you’re saying, in addition to the words themselves. It’s really understanding the context of the environment by using more data so that the interaction becomes more effective.
It’s a fascinating new world. In the next decade or so, what may be some interesting surprises that are not readily obvious to us now?
Martino: To build on the idea of human–machine interaction and the experiences you have today, here’s a fun one: You have a multimedia experience where you’re immersed in sound. You hear it in three dimensions, so you can place sound in specific locations with certain technologies today. That’s usually optimized for a point in a room. Well, future technology will allow that to adjust for your location and give you that environment at many points as you move. Same thing with lighting. So the ability for an experience or an environment to adjust based on your physical location, even to where you’re looking, becomes an interesting capability.
In the case of modifying your environment with sound, if you have a child sleeping, perhaps you shift the sound away from that area. You only target the sound in areas that are relevant to the listeners, and you keep a quiet environment for the sleeping child. You can even use it for safety reasons, say in a car when you want to direct the driver’s attention to a source of possible danger, which could be a car in the blind spot. Or imagine being alerted to a security issue occurring in your house as windows are being broken: The sound is directed toward the point of the security issue, as opposed to your going to a panel and interpreting a code to understand where you should pay attention.
Another example is how we efficiently obtain energy to run these devices. One of the demonstrations that we provided at an industry convention showed the use of plants and the photosynthesis process to harvest energy and convert it to a current in a voltage that can operate the device. We are literally extracting energy from the soil to operate devices, using a natural process that occurs all around us. Creating sustainable devices to perform tasks in a way that is consistent with the environment — build from that, and we can create many different and exciting applications where this local computing and access to information can be meaningful to the individual or the business or other aspects of our lives.
There’s an interesting paradox here. We talked about about 70+ billion devices in the near future. What percent of those devices could effectively be run using alternative energy sources?
Martino: That’s an interesting question. While I haven’t really come up with a percentage of those devices, many if not all will need to have energy efficiency factored in. The effective use of energy to achieve sustainability goals and help drive a greener world will be an additive effect of efficiency through the devices we enable. Alternatively, energy can involve solar, it can involve wind. It can also involve techniques like I mentioned previously, such as extracting energy from the soil through a photosynthesis process. So a broad set of applications can be driven through alternative energy. At the end of the day, it comes down to efficient operation and efficient systems. You don’t want to send all the data to the cloud, because you’re spending energy going from a local environment to a remote environment to then sending information back. And that action is very inefficient. The more you can do at the source of the data, the more you save energy as a whole. That really ties in to the sustainability focus that we have as a company and what we are doing in terms of enabling these devices to operate in the most efficient way.
When we look at the new edge economy with 70 billion devices that will be capable of self-orchestration and different sensing interactions with humans, how much does this edge environment replace the existing data center/PC business in terms of compute? It may be difficult to calculate, but there must be some sort of transition point at which the edge becomes the prime place where compute occurs and not the core. What will be the impact? When might that tipping point happen?
Martino: Edge internet economies are projected to reach 4–5 trillion by 2030, and this is a somewhat conservative estimate. When you talk about the edge and the deployment of all these devices, it’s transforming how computing is done. There will be all sorts of new data sources, and we’ll need to efficiently process that data locally and then send information appropriately. This computing is complementary and additive, and it is not replacing what you would typically do with your PC. More use cases and an increased need for higher compute levels in a data center will be needed, but it will need to do it efficiently as we scale massive amounts of data. We need to think through how to do that in a way that’s efficient from an energy usage and management perspective, how to process it locally and then use the other forms of computing in a more targeted way that is effective from the perspective of cost of ownership. These use cases will also require us to handle real-time local tasks for specific applications that need quick response.
In this future we’re discussing, there will be a fundamental shift from what used to be compute or gather and sense centrally, to what will be gather, adjust and compute locally and in the moment.
Martino: The way that I always talk about this is that the central compute model and the edge compute model are complementary. They build on each other. The optimization or the end goal is how to use the combination of those two in the most effective way. At NXP, we work in partnership with all of the cloud businesses as partners in order to enable these many devices to securely connect to the cloud and then let our end customers, as well as ourselves, look at different ways to optimize that balance between the two for a particular application. Not all applications will have their own optimization. Not everything will be the same. Some will benefit from a heavier cloud usage, some will benefit from a local usage.
The other aspect of this that becomes important is where you have a need to use a cloud environment. When you get disconnected from that environment, you want to also have local capability that allows that experience to continue, or allows the operation of your environment to continue without disruption. The local environment capability also allows for the backup or the continuation of the local capability. Consider your home and security monitoring or the interaction with your lighting system. You want that to occur, whether you’re connected to a cloud service or not. It’s the same thing with factory automation and operation — you can’t have disruption locally even if you have a cloud connection. So that becomes a key complementary aspect of how edge and cloud together make the strongest solutions.
NXP is on a fascinating journey, and we’ll be eager to see what develops now and through the next decade.
Martino: We work with many customers and industries and applications, both in consumer and business applications, as well as connecting information with the cloud. We are leaders in the automotive industry in terms of supplying of electronics for all functions in the car. And so we are present in a variety of environments and have a worldwide footprint that engages with many types of customers. There are multiple ways to connect with NXP and multiple forums where we’re talking about new developments. We look forward to continuing to share our work around edge computing and the proliferation of this capability.
The conversation has been edited and condensed for clarity.