AI Fire Detection: Computer Vision Guards the Forest

In the age of global warming, forest fires are becoming more frequent and faster-growing. In California, for example, 8,619 wildfires in 2021 scorched  2.7 million acres, destroying more than 3,500 structures and causing billions of dollars’ worth of damages.

Clearly, the world needs sustainable solutions to preserve our natural resources, protect human lives, and avoid economic devastation. As an environmental advocate and sustainability enthusiast, I got to thinking about whether a technological solution can help with this daunting task. Fortunately, I am also a computer scientist, one who is all too aware of how tedious and time-consuming research can be.

Computer Vision Keeps an Eye on the Forest

In such times, I often choose to play my ace in the hole by going straight to Intel’s rich ecosystem—the Intel® Partner Alliance. Not surprisingly, it led me to an ingenious solution: the AAEON Intelligent Forest Fire Monitoring System (Figure 1).

The AAEON Intelligent Forest Fire Monitoring System uses AI to detect smoke.
Figure 1. The AAEON Intelligent Forest Fire Monitoring System uses AI to detect smoke. (Source: Digital Business Innovation)

Deceptively simple, the solution consists of cameras capable of monitoring a large area, detecting smoke, and activating an alarm. Yet on closer analysis, its innovative architecture makes it a technologically advanced solution.

AI Fire Detection Cuts Through the Fog to Detect Smoke

Indeed, image processing is done locally (near the camera) through an edge computing device. This industrial-grade computer analyzes images from dual video cameras–one for visible light and the other for infrared–and identifies any signs of smoke. If the system detects smoke, it immediately activates the alarm for central operations to alert local fire departments.

To avoid false alarms, the device can distinguish between smoke and fog. Let that sink in for a minute. The system has been trained to recognize fog, so that it cannot be fooled.

The AI can predict the direction and speed at which fire will spread—and alert relevant fire departments in advance. @AAEON via @insightdottech

All of this is made possible by the Intel® Movidius™ Myriad™ X Vision Processing Unit. This extraordinary processor performs image analysis through a specialized architecture that can perform deep-learning inference on a remarkably low power budget.

Central Monitoring and Prediction for Firefighters

Among other benefits, the ability to process video at the edge reduces the amount of data that must be sent to the central operations center—an important consideration given that many of the cameras will be installed at remote locations with limited network connectivity.

Once at the data center, images from all cameras are jointly analyzed for a forest-wide perspective of all fire activity. What happens next is truly amazing. The system can predict the direction and speed at which fire will spread—fully accounting for environmental factors such as wind and humidity. Next, it alerts relevant fire departments in advance to give them time to evacuate residents and try to contain the spread.

Already exceptional, the system takes it a step further by storing data and using it for iterative improvements. That’s not a surprise considering that AAEON is a leader in AI for the real world.

At this point you may be thinking: But how much does it cost?

Anything other than cost-prohibitive or exclusive, to be honest. I was surprised when AAEON’s executive team showed me the total cost of ownership. Instead of digits followed by a seemingly infinite series of zeros, the cost of the system and subsequent management is quite modest considering its ability to safeguard human life, the natural environment, and foster sustainability.

Using AI to Meet Global Goals

The AAEON solution illustrates the many ways we can use the powerful predictive influence of artificial intelligence at our fingertips or in technological proximity where local intelligence is required. This seems to be the future that awaits us, humans and artificial intelligence together to build a better future for us and our planet.

It is a shared goal of world leaders to such an extent that many governments feel the need to create a shared platform to explore and mitigate the disruptive impact of AI on society and the economy.

In Europe, for example, the AI4EU project was launched to harmonize, equalize, and promote innovation and technology transfer. I have been honored to be involved with this project as an external expert on the evaluation committee, giving me a front-row view of the ways AI can be used for good.

Returning to my central narrative: There is a strong sense in which the fire detection system is a manifestation of sustainability, insofar as it prevents CO2 emissions from forest fires; avoids overconsumption of electricity by transmitting all data to mega computers in the cloud; and is universally accessible by developing and developed countries alike.

In a word, it has the power to engage each of us in a sustainable movement to protect our one and only planet.

 

This article was edited by Christina Cardoza, Associate Editorial Director for insight.tech.

This article was originally published on February 25, 2021.

Q&A: Everything AI at embedded world 2021

embedded world is going virtual in 2021. But the event, held March 1-5 online, is now easier than ever for people to attend. And it’s still packed with experiences from the entire world of embedded technology—from AI to IoT.

Kenton Williston, Editor-in-Chief of insight.tech, and Dr. Sally Eaves, CEO of Aspirational Futures, discuss what to expect from this year’s event, where it fits in the broader context of the industry, and how attendees can get the most of their time strolling—that is, scrolling—the show.

Want more? Find links to demos and Q&A sessions from embedded world 2021 on the blog page for this podcast, and learn more about the technologies discussed in their conversation. Or join Kenton and Sally as they live tweet several of the sessions and explore what’s happening there in real time.

Megatrends of embedded world 2021

Kenton Williston: Sally, can you tell our listeners a little bit about yourself?

Sally Eaves: I’m CEO of Aspirational Futures, which basically looks at enhancing inclusion in technology and also education. I’m a CTO by background, and I’m now a senior policy advisor for the Global Foundation of Cyber Studies and Research. I do a lot around emergent technology advisory, and I’m also a professor in that area. I’m really active around cloud computing, cybersecurity, IoT, IIoT, AI, blockchain, 5G, etc., but also the cultural aspects of those topics. And the people factors around sustainability and social impact too.

Kenton Williston: All things very relevant to what is coming this year to embedded world. I think the big trends I am looking at for this year’s show really center around embedded transforming into IoT—and building on that IoT migration trend is a greater and greater emphasis on AI. It seems to be part of just about everything that is happening this year.

From your point of view, Sally, what do you see as the megatrends for 2021 as they relate to the Internet of Things and AI? And just generally, what’s happening in this commercial-technology space?

Sally Eaves: I think it’s really exciting times. There’s a great deal of convergence. And you mentioned two of the key factors there—with IoT and industrial IoT alongside AI and machine learning. But I’d also add 5G into the mix, as that becomes more and more mainstream. We need to look at not just the technology, but the skill sets alongside that. Time-sensitive networking is one to look out for as well—things to make it easier for developers so they can really maximize their time.

Kenton Williston: One of the demos that I’m really looking forward to seeing on this point is what IEI (one of Intel’s partners) has called their AIoT kit, which—as its name suggests—combines AI plus IoT. And of course it brings together the AI and IoT sides of things, but it also brings together with that the 5G technology that you were mentioning.

Another demo from Vecow brings together AI pre-trained models and ROS, the Robot Operating System. It’s like a one-stop shop for everything you need to create an intelligent robot—like an autonomous robot that might be running around a warehouse floor.

Sally Eaves: I love the sound of that demo. I’ll definitely be looking at that.

There are some  challenges I see, as well. I think one would be security and, in particular, safeguarding critical data within industrial and embedded IoT. And also thinking more on the network side of things around timeliness. I think that’s so, so vital for industrial automation, AR, VR, and also robotics use cases.

Making Tech “Simpler, But Not Simple”

Kenton Williston: From your point of view, what should developers and engineers be on the lookout for that will help them actually put together these increasingly complex, multifaceted sorts of applications that folks are trying to build in 2021?

Sally Eaves: We’ve got this increased sophistication that’s offered by convergence, but at the same time it’s this juxtaposition around complexity. So it’s about making it simpler but not simple, for another way of putting it. I think one thing definitely to look out for are the 5G elements. 5G and Edge computing together—connecting more devices, more efficient processing of data.

And agility—I think it’s the key word probably for 2021, as your workloads are fluctuating. Distributed Edge computing is going to give that flexibility to scale on demand, to deploy your applications to any Edge location, conserving memory and power. And because apps are being processed at the Edge, you’re reducing bandwidth as well. I’m seeing some very interesting collaborations in that space—definitely would shout out for that for developers to have a look at.

Kenton Williston: Intel’s got a whole new web presence that it’s launched within the last year called the Intel® Edge Software Hub, which I think is a pretty interesting effort to bring together all of these commonplace technologies. And it’s not just for the things like the AI or the robot operating systems, but even things like the connectivity—pre-packaged modules for 5G connectivity—that allow you to easily configure the Open Network Edge Services Software—or OpenNESS platform—on that Edge device.

So, I think all these kinds of approaches, where you’re almost building things out of Legos, as it were—this is the sort of thing that I think everybody has been talking about for a long, long time. To your point about the agility and how quickly folks are wanting to, not only deploy IoT designs but be able to update them—I think it’s just more important than ever.

Sally Eaves: The example you gave just now about the Edge Software Hub is such a strong one, because you’re right, you’ve got that pillar, that pre-optimized pillar of deployment-ready software packages, which is fantastic. But, equally, you’ve got the ability to customize, so it’s that best-of-both-worlds approach. I think that’s absolutely the way we need to be going.

Making AI Trustworthy

Kenton Williston: A lot of these trends are kind of longer term, but I think something that has changed here is just how pervasive the AI element is in just about every space. I saw a couple of examples from ADLINK, who will be demonstrating how they use AI for everything from inspecting contact lenses to automating palletization and tracking of items for shipping.

Sally Eaves: Absolutely. And supply chain—I think one of the things that’s come to the fore so much over the pandemic is the fragility around that, and embedding a transparent audit trail. I’ve seen some really interesting things with AI and blockchain coming together. So that marriage, for want of a better word, between AI and blockchain I think is one to watch as well. Pharmaceuticals, for example, would be a classic example of that.

The Ethos of Tech-for-Good

Kenton Williston: Another thing that I think is worth adding to the mix here is the safety element of things. As exciting as it is to see this amazing intelligence being applied in all these amazing, creative ways, there’s also a lot of caution that we should exercise about how we’re deploying these technologies—particularly as we’re increasingly automating systems and making them hands off.

One thing that comes to mind for me there as an example is Intel and its latest hardware platforms: the Intel Atom® x6000E series processors incorporates functional safety technology to help protect the physical world. And there’s a really great demo from NEXCOM showing exactly how that works, and how you can deploy that in all kinds of different applications to keep things from causing harm.

Sally Eaves: I think that’s one of the absolute key issues of the entire year. In certain sectors around manufacturing, operational technology, health, education, there’s been such an increase—I think it’s around a 300% increase around identity attacks over the past year, as one example. I think we’ve also seen where there’s been continual investment in infrastructure, but maybe less so around patching and around refreshes. So that’s created an area of security vulnerability.

Kenton Williston: Another thing that comes to mind for me is ethical AI, which is something I know you’re passionate about. I’m thinking about a demo from a company called EverFocus that is offering an in-transit network video recorder box that incorporates analytics—both forward-looking to see what’s happening in traffic, as well as inward-facing to understand what’s happening inside the vehicle.

There’s potential for misuse there, but there’s also potential for really amazing benefit in terms of keeping people safe and healthy, and cities running efficiently and minimizing their carbon footprint. So it’s all about how you deploy it. And I like this EverFocus demo as a kind of example of how to do it the right way.

Sally Eaves: I think leadership in this area is so, so important. And one of the things that’s also impressed me over recent months is a bit of a change in the narrative around this. But what we’ve been able to see over the pandemic experience is some fantastic examples of collaboration. One of those that springs to mind for me would be the HPC Consortium—the High Performance Computing Consortium—of which Intel is a member.

And it’s a great example of leading tech companies coming together—partnering up with research and academia and governments across the world, as well—and really coming together. That ethos of tech-for-good collaboration—basically bringing computing capacity, bringing computing power together to look at how we can better fight COVID-19. I think that’s a great example of turning the narrative on AI as something for good. Supporting that further, building that momentum of greater trust around AI, I think is really, really important

I also think this comes down to education. People have to be empowered to be able to ask the right types of questions, and we need to get better diversity of teams into who’s building AI, as well. And that goes beyond aspects like gender, to all sorts of different characteristics. But diversity of experience—it matters so, so much. And every piece of research going—and our practical experience as well—says that the teams that are diverse are happier, they’re more creative, they’re more satisfied, and you get so much more innovation, and you reduce the risk of implicit bias as well. So that has to be the way to go forward.

embedded world 2021 vs. embedded world 2020

Kenton Williston: I’m very excited to see for myself where things are going as the industry gets more complicated, more sophisticated. But I think there’s an overriding theme here of bringing together so many different technologies that have been in development—whether it’s AI, whether it’s 5G, whether it’s safety and security—bringing so many of these things together in ways that I think really are noteworthy, and notably different than what I saw last year at this time. How about you? What are you looking forward to?

Sally Eaves: I think there’s a real acceleration in innovation, and around the actualization. The speed of change has been unlike things we’ve seen before, absolutely. So I think there is a real change this year vis-a-vis the one before. That’s really, really exciting.

There are so many sessions. Fourteen sessions on Internet of Things, platforms and applications. But I think what I like about this year’s event is it’s five days long—you can really tailor it to your particular organization and also what you want to learn about. There are so many opportunities to really dive in deep and ask questions.

I also like the matching application they’ve put together. It feels like a proper personalized experience. Because if you can’t be there in person, then making an event feel like a true interactive experience matters so much.

I really miss the socialization aspect of events and things, but I must admit I’m really impressed by how the agenda for this has been curated. There’s a really strong attention to detail there, and opportunities to build that network connection and match people together. So I really like what’s been done in terms of curating the event.

Kenton Williston: Should our listeners be coming to this podcast after the fact, where can they find you online?

Sally Eaves: I think the one to go for, number one, would be @sallyeaves on Twitter. But I’m on all major channels— LinkedIn, my own website, etc.

Kenton Williston: That just leaves me to thank you for joining us today. Really appreciate all your insights.

Sally Eaves: Absolute pleasure. And really looking forward to the event.

Everything AI at embedded world 2021

A conversation with Dr. Sally Eaves @sallyeaves

[podcast player]

With embedded world 2021 going all-digital, attending the world’s premier IoT event is easier than ever. And it’s one of the year’s best opportunities to get up to speed on the latest in AI and advanced programming techniques. You won’t want to miss this chance to discover the trends that will define the year.

In this podcast, multitalented IoT and AI expert Dr. Sally Eaves joins insight.tech Editor-in-Chief Kenton Williston to preview the show, pick out must-see conference sessions, and highlight key trends attendees should look for at the event. We explain:

  • Why AI will play a critical role at embedded world 2021
  • How new tools are changing the way IoT applications are created
  • How developers and engineers can get the most out of the event
Apple Podcasts  Spotify  Google Podcasts  

Transcript

Kenton Williston: Welcome to the IoT Chat podcast, where we talk about the trends that matter to developers, engineers, and system architects. Today we’ll be talking about the upcoming embedded world 2021 show, which of course has now gone all virtual. Kind of sad and disappointed myself not to be going this year, because I love taking a flight out to Germany every year around this time. But the great thing about it is now it’s going to be easier than ever for folks to attend. And so, on that point, I’ve invited our guest, Dr. Sally Eaves, to join me to talk a little bit about what’s going to be coming up in the show, where it fits in the broader context for the industry, and how attendees can get the most out of their time. So, Sally, welcome so much to the show.

Sally Eaves: Oh, thank you so much. Pleasure to be here. Really looking forward to our conversation today.

Kenton Williston: Excellent. And can you tell our listeners a little bit about yourself?

Sally Eaves: Yeah, absolutely. So, I’m CEO of Aspirational Futures, which basically looks at enhancing inclusion in technology, and also around education as well. I’m a CTO by background, and I’m now a senior policy advisor for the Global Foundation of Cyber Studies and Research. I do a lot around emergent technology advisory, and I’m also a professor around that area as well. So, really active around cloud computing, cybersecurity, IoT, IIoT, AI, blockchain, 5G, etc., but also the cultural aspects of that, and the people factors around sustainability and social impact too.

Kenton Williston: All things that people are very much focused on these days, I think. Very relevant to what is coming this year to embedded world. So, on that point, I think two of the big trends that I am looking at for this year’s show really center around, of course, the longer-standing trend of Embedded transforming into Internet of Things, or IoT, technology. And I think that what really distinguishes those two is a combination of the connectivity. Of course, you can’t have an Internet of Things without having an internet.

And then I think also the intelligence, and I think what’s becoming particularly prominent, and building on that IoT migration trend, is a greater and greater emphasis on AI. It seems to be part of just about everything that is happening this year. So, I’m wondering, from your point of view, Sally, what you see as the megatrends for 2021 as they relate to the Internet of Things and AI, and just generally what’s happening in this commercial-technology space.

Sally Eaves: Yeah, absolutely. I think it’s really exciting times. There’s a great deal of convergence. And you mentioned two of the key factors there, with IoT and industrial IoT alongside AI and machine learning. Absolutely. But I’d also add 5G into the mix there as well, as that becomes more and more mainstream. A lot of the advantages that brings—particularly, for example, building low-latency applications—I think that’s going to really have an influence as we move to the latter side of this year as well. So I’m excited to see what happens there.

I think, just generally, the latest predictions are something like 41 billion connected devices by 2025, and the whole market around embedded software is set to increase to something like 130 billion by two years later—so, 2027. So we’ve got a huge demand for this area. So we need to look at not just the technology, but the skill sets alongside that. And maybe some of the challenge trends I see as well. So I think one would be security and, in particular, safeguarding critical data within industrial and embedded IoT, and also thinking more, maybe, on the network side of things around timeliness. I think that’s so, so vital for industrial automation, AR, VR, and also robotics use cases as well.

So, things like time-sensitive networking, which I know is one of the sessions at the upcoming event as well. I think it’s one to look out for as well. So, really exciting times, and AI specifically—things to make it easier for developers so they can really maximize their time. So I think there’s some interesting things we can explore there in depth as well.

Kenton Williston: Yeah, absolutely. I feel like you’ve almost read my mind there on several points. So, one thing for sure I should mention up front here is that listeners can visit the blog page for this podcast, and they will find there links to a whole bunch of different demos and Q&A sessions that folks can attend to learn more about all of the technologies we’ll be discussing today. And that’s very much inclusive of one of the first things you mentioned—that a lot of things are converging together. I had already mentioned there’s AI plus IoT; you talked about 5G, security, and I absolutely agree.

This, of course, is nothing new, to say that applications keep becoming more and more complicated. It’s almost trite to say that. But I think there’s a truth to that that is different from how things used to be complicated in the past—that the kinds of complexity people are dealing with is quite a bit different than in the past. And bringing all of these different technologies together, I think, really has a meaningful impact on what kinds of applications you can even consider.

And one of the demos that I’m really looking forward to seeing on this point is, there’s a really cool, what IEI—one of Intel’s partners—has called their AIoT kit, which as its name suggests combines AI plus IoT to get AIoT. And of course it brings together the AI and IoT sides of things, but it also brings together with that the 5G technology that they were mentioning, which I do agree is going to be a very important part of where the application space is headed this year.

To another point that you made about the challenges and the complexity—I’m wondering, from your point of view, what developers and engineers should be on the lookout for that will help them actually put together these increasingly complex, multifaceted sorts of applications that folks are trying to build in 2021.

Sally Eaves: Yeah, absolutely. I love the sound of that demo, by the way. I’ll definitely be looking at that. That looks fantastic. I think, for me, you really touched on it around convergence. So, for me, it’s—we’ve got this increased sophistication that’s offered by convergence, but at the same time it’s this juxtaposition around complexity. So it’s about making it simpler but not simple, for another way of putting it. So, I think one thing definitely to look out for—which I think is going to really help, and I’ve just been part of an incubation program along this very basis—are the 5G elements. So I think that’s really exciting in terms of speed and the new possibilities that’s going to create.

So, 5G and Edge computing together—connecting more devices, more efficient processing of data. And agility—I think the key word probably for 2021 as your workloads are fluctuating. Distributed Edge computing is going to give that flexibility to scale on demand, to deploy your applications to any Edge location. So that’s really exciting, and conserving memory and power, and, because apps are being processed at the Edge, you’re reducing bandwidth as well. So, looking out around opportunities for that, I’m seeing some very interesting collaborations in that space.

So, definitely would shout out for that for developers to have a look at. But on top of that there’s a range of tools supporting alternative software-development paradigms, and they’re becoming more available. Using graphical programming methods or domain-specific languages. So that’s an exciting area. It’s very much model-based software development, but also in particular—and we’ve touched on it a little bit already—but opportunities for almost, like, all-in-one Edge development offerings. So you’re integrating software and hardware with pre-trained AI models and comprehensive tools. So it’s all around that help, that seamless help, for developers to take care of that baseline to a certain extent, to give you more time to do the actual innovation and agile development.

So that’s really exciting. Anything that’s going to help you with—or active prototyping, experimentation, get up and go and running quickly—those are the things to look out for. And I’ve seen a lot of things in the agenda coming up that really touch on that in depth. So I’m really excited to see where that goes.

Kenton Williston: Yeah, I’m seeing much the same. And I mentioned already that there’s this IEI kit that they’ll be demoing that brings together a number of things. And another one that really struck me from this angle is there is one from a company called Vecow that brings together some AI pre-trained models, like you were describing, as well as the ROS—that is the Robot Operating System platform—and I think this is a really good example of the sort of thing that you were laying out as the path forward.

You need to have pretty robust platforms that have got a lot of the basics done for you. And more than that, I think, shared platforms, whether they be open source or at least open APIs. I mean, I’m thinking, for example, like the way the big cloud players like Microsoft and AWS have these APIs you can leverage. I think using the standard methodologies and approaches to design your IoT devices is going to be increasingly important, just because you’re trying to combine so many complicated technologies in one single device, in one single software package.

So, again, the one that was really interesting in this regard was this demo that Vecow will be showing that’s kind of like a one-stop shop for everything you need to create an intelligent robot. So—think an autonomous robot that might be running around a warehouse floor, or something like that, would be a good example of where this would be really useful.

Sally Eaves: Absolutely. No, I spotted that as well. I think that looks excellent and really reducing time consumption, and particularly around challenges of integration of various software stacks as well. I think it’s so strong on that, but also things, maybe, around stable and reliable version management as well. I think that’s an excellent example.

Kenton Williston: Yeah, absolutely. Absolutely. So, and in fact, some of the links we’ll be providing on our site will be to exactly those sorts of things. I briefly touched on the way that folks are increasingly doing development in a fashion that resembles what you would see in the IT world. So, doing things in containers, developing in the cloud, and pushing to the Edge, make it a lot easier to—to your point—maintain versioning, make it a lot easier to maintain visibility across distributed systems, make it a lot easier to—as I was saying earlier—take a bunch of standardized things and package them together in a platform that everybody else is already using, so you don’t have to figure it all out for yourself.

Sally Eaves: Absolutely. Absolutely. I couldn’t agree more.

Kenton Williston: And there’s even a really interesting session that’s talking about some of what Intel’s doing. It’s got a whole new web presence that it’s launched within the last year called the Edge Software Hub, which I think is a pretty interesting effort to bring together all of these commonplace technologies. And it’s not just for the things like the AI or the robot operating systems, but even things like the connectivity—pre-packaged modules for 5G connectivity—that allow you to easily configure the open network Edge services software—or OpenNESS platform—on that Edge device.

So, I think all these kinds of approaches, where you’re almost more like building things out of Legos, as it were—again, this is the sort of thing that I think everybody has been talking about for a long, long time. And I think there’s nothing new conceptually about this, but I think, just given all the many different things, and—to your point about the agility and how quickly folks are wanting to, not only deploy IoT designs but be able to update them—I think it’s just more important than ever.

Sally Eaves: Yeah, I think the example you gave just now about the Edge Software Hub is such a strong one, because you’re right, you’ve got that pillar, that pre-optimized pillar of deployment-ready software packages, which is fantastic. But, equally, you’ve got the ability to customize as well, so it’s that best-of-both-worlds approach. So, it’s so strong on the actualization side of things, and what you mentioned there about continuous integration, continuous deployment—I come from a telco background; I’ve just been doing some work specifically on this area, and I think it’s so, so strong for that granular changeability, which I think is fantastic to move into the IoT space. So, absolutely. I think that’s absolutely the way we need to be going.

Kenton Williston: Yeah. And I think one point that I can’t overemphasize is, I think when I’m thinking back to the past years and—I have to say, I have so, so loved going to this show. Nuremberg is just such a beautiful city. I really love the old town. The last time I went, in fact, we stayed in the old part of the city, right on the river. We were on the bottom floor of the building that was who knows how many hundreds of years old, and there are all these ducks swimming about, and I could just about reach out and touch them. And, oh, so lovely. I really miss that.

Sally Eaves: Oh, me too. Me too. I’m used to remote working, but I’m used to mobile remote working, if you see what it means? So that’s—

Kenton Williston: Exactly. Yes.

Sally Eaves: But, yeah, absolutely. I really miss the socialization aspect of events and things, but I must admit I’m really impressed by how the agenda for this has been curated. There’s a real strong attention to detail there, and opportunities to build that network connection and match people together. So I really like what’s been done in terms of curating the event.

Kenton Williston: Yep, absolutely. Absolutely. And the thought that got me on our little rabbit trail here was thinking back to previous years. A lot of these trends, like I’ve been saying, are kind of longer term, but I think something that has changed here is just how pervasive the AI element is in just about every single space that we’re looking at. And there’s some really fun and interesting examples. I’m finding AI cropping up in places that I wouldn’t even expect, despite the fact that I’m constantly in this space thinking about IoT applications. It still surprises and delights me to find out where and how it’s being deployed.

So, I saw a couple of examples from ADLINK, where they’re going to be demonstrating how they use AI to inspect contact lenses. And as a guy who’s just going out to get their latest prescription, I really appreciate that. Someone is making sure those are very well made. And even things like palletization of items for shipping, and automating that process, and automating the tracking of things within a pallet. It’s just—no matter what kind of application, big or small, it’s like AI has a way to help out it seems like just about everywhere.

Sally Eaves: Absolutely. I couldn’t agree more. And supply chain. I think one of the things that’s come to the fore so much over the pandemic—and in some cases the fragility around that—and embedding a transparent audit trail. I’ve seen some really interesting things with AI and blockchain coming together. And, again, we were talking about actualization earlier. I think in blockchain, in particular, it’s something that has been associated with particular use case studies more than others, but it’s shown the real art of the possible now, and really tangible, actionable case studies. So that marriage, for want of a better word, between AI and blockchain I think is one to watch as well. It’s been really heightened. I think trust has been built over this process. So pharmaceuticals, for example, would be a classic example of that.

Kenton Williston: Yes, absolutely. Absolutely. Another thing, too, that I think is worth adding to the mix here—speaking of the trust—there’s also the safety element of things. So, as exciting as it is to see this amazing intelligence being applied in all these amazing creative ways, there’s also a lot of, I think, caution that we should exercise as technical professionals about how we’re deploying these technologies.

And, I think, particularly as we’re increasingly automating systems and making them hands off. So here in the States, for example, there was just a story where someone had accessed a water treatment plant and increased the amount of lye that was going into the water, which normally would just handle the acidity and get it to a reasonable pH balance, and they had increased it to—it was either 100- or 1,000-fold the desired amount. And it was just a coincidence that someone happened to be in the plant looking at a screen while someone was in there maliciously mucking about, that they caught that. And that sort of thing is scary, right? And I think reasonably so.

So, I think for all of us to be thinking about putting the safeguards around this amazing technology is also a very important consideration going forward. And one thing that comes to mind for me there as an example is Intel and its latest hardware platforms: the—what’s been known as Elkhart Lake, now as the—Atom 6000 Series incorporates some functional safety technology to help in the physical world keep things safe. And I think that’s very, very important. And there’s a really great demo from NEXCOM showing exactly how that works, and how you can deploy that in all kinds of different applications to keep things from causing harm.

Sally Eaves: Absolutely. No, I think that’s one of the absolute key issues of the entire year. And I think in certain sectors around manufacturing, operational technology, health, education—there’s been such an increase, I think it’s around 300% increase around identity attacks over the past year, as one example. But I think we’ve also seen where, for example, there’s been continual investment in infrastructure, but maybe less so around patching and around refreshes. So that’s created an area of security vulnerability as well. So, so much lookout there, so that sounds a really, really excellent investment and advance. So it’s great to hear that.

Kenton Williston: Another thing that comes to mind for me is, I know that ethical AI is something you’re passionate about. And I think, just on a personal level, it’s something I care about as well. So, I’m based in Oakland, California myself, which is known as a hotspot for, let’s say, socially progressive sorts of movements. And there has been just recently here, in the past week or so, some movement among the activists in this community to stop the Oakland Police Department from using automated license plate recognition—which I’ll leave whether that’s a good idea or not for the listener to decide—but I appreciate very much the ethical challenges there.

I mean certainly these kinds of technologies have been abused already, and there’s potential for abuse for sure. Facial recognition has also been, I think, a really, really hot and hotly debated topic. And I know we’ve been talking about software packages that can help with AI. And Intel, I think, has done a really fantastic job with its OpenVINO platform, pre-packaging a lot of commonplace AI workloads together. And they took the step of actually removing any of the facial recognition elements from those packages, just to really say, “Hey, let’s take a pause here and think about how these technologies can be best deployed. And see what we can do to address issues of things like racial profiling, and the differences in how well these things perform depending on your gender and ethnicity, and so forth.” So I’d love to hear some of your thoughts on these issues.

Sally Eaves: Absolutely. And one thing I’d also applaud there is Intel have done some really good work about building an ethical AI toolkit, and they’ve got some really great examples there, and it’s backed up by research with Stanford and other places as well. So, really, really impressed by that, because I think leadership in this area is so, so important. And I think one of the things that’s also impressed me over recent months is a bit of a change in the narrative around this.

So, in the past when we talked about AI there had been quite a lot of headlines that were quite scary. It would focus on—if a research report came out—it would focus more around words like “destruction” and “elimination” around certain types of jobs. But what we’ve been able to see over the pandemic experience is some fantastic examples of collaboration. So, one of those that springs to mind for me would be the HPC Consortium—so, the High Performance Computing Consortium—of which Intel is a member.

And it’s a great example of leading tech companies coming together—partnering up with research and academia and governments across the world as well—and really coming together, and that ethos of tech-for-good collaboration—basically to bring computing capacity, to bring computing power together to look at how we can better fight COVID-19. So, I think that’s a great example of turning the narrative on AI as something for good. And to support that further, to build that momentum of greater trust around AI—I think for me it’s ethical development and aspects like the explainability of AI which I think is really, really important. I’ve seen a lot of work—and I contribute to some of this myself with some of my research—about value frameworks to build common understanding, common language, common commitments about the development of AI, but I also think this comes down to education.

People have to be empowered to be able to ask the right types of questions, and we need to get better diversity of teams into who’s building AI as well. And that goes beyond aspects like gender, to all sorts of different characteristics—but diversity of experience, it matters so, so much. And every piece of research going and, you know, our practical experience as well—the teams that are diverse are happier, they’re more creative, they’re more satisfied, and you get so much more innovation and you reduce the risk of implicit bias as well. So that has to be the way to go forward.

There’s been a lot of research by groups such as Endelman—they benchmark trust, for example, over at least 17 years now—and even before the pandemic it was a low ebb across all sectors, even around charity, for example, as well. So there’s always been a lot of work to do here, but I believe that the positive things we’ve seen coming out over the last year—let’s harness that. Let’s build a contagion of change around these types of subjects. I’m really, really super passionate about social impact and around inclusion. I think we can really build on what we’ve seen here, and some of the collaborations and the movement forward, and really make this a change for good.

On the technology side, I’ve also seen some developments, for example, helping end users have a better understanding about why a specific result’s been generated, helping developers be able to more easily debug and tune and optimize their models. So we’ve always had a trade-off between accuracy and explainability in driving model selection—particularly obviously in highly regulated environments as well. So seeing enhancements around interoperability, and with bias and explainability tools across all stages and model development, I think is hugely important. And things like shapely values—the ethos of that around visibility and transparency into model decision-making is so, so important. I think it can help shorten the path to success for us all.

Kenton Williston: Yeah, absolutely. And obviously—even from my own personal experience I have found the diversity of folks that I work with on this insight.tech program to be really wonderful in terms of opening my eyes to all kinds of possibilities, and just having people come from such different perspectives. Currently I’ve been saying, as much as I am really deeply immersed in this world, there’s just so many things that constantly surprise me nonetheless. And I think having this diversity of perspective is just incredibly valuable for that.

Sally Eaves: Absolutely, absolutely. It’s enriching in every aspect, isn’t it? It really is, and the foundation of Aspirational Futures I mentioned at the top, that’s what we specialize in a lot—really democratizing access into tech careers, which I think is so, so important for the future and building those skills and skills confidence as well.

Kenton Williston: Absolutely. And so I want to just mention in passing—all of these different factors—I think there’s some really good examples of how they can be applied. So, there’s a demo from a company called EverFocus that is offering an in-transit network video recorder box that incorporates analytics—both forward-looking onto the road to see what’s happening in traffic and help the driver perform at their best, and to help the folks who are routing all the transit vehicles understand what the situation is on the ground. As well as inward-facing to understand what’s happening inside the transit vehicles and help make sure everything is secure and everyone is doing just fine. And of course these days lots of new concerns—like making sure everyone’s masked up.

So, I think this is a really good example of, you could take some of these things that are potentially problematic, like the forward-looking cameras potentially doing some things that you may or may not like—to recognize people and cars and such on the road—as well as the inward-looking, recognizing people in the vehicle. I mean, there’s potential for misuse there, but there’s also potential for really amazing benefit in terms of keeping people safe and healthy, and cities running efficiently and minimizing their carbon footprint. So it’s all about how you deploy it. And I like this EverFocus demo as a kind of example of how to do it the right way.

Sally Eaves: Absolutely. That sounds fantastic. And I think enabling all voices to be heard in that as well. There’s a great example coming out of Helsinki at the moment, which is very much like that demo you were describing there, but ensuring, for example, different voices—so, citizens inputting to the development of their city. So, the example you were saying there about the city development environment and mobility—I think we’re seeing some great things. So what you were just describing there, I think, would be a great fit for that use case.

Kenton Williston: Yeah, absolutely. Absolutely. So, to just kind of wrap a bow around all of this, again, for me as I look forward to this show—and I should mention, too, for our audience that you and I will be live Tweeting some of the sessions. So, absolutely—let me try saying that again. So I should mention, of course, that you and I will be live Tweeting some of these sessions, and I absolutely invite our listeners to come join us on Twitter—follow along as we explore what’s happening as it goes down.

Sally Eaves: Absolutely.

Kenton Williston: Yeah, absolutely. So, on the whole, I’m very excited to see for myself where things are going as the industry gets more complicated, more sophisticated. But I think, for me, there’s an overriding theme here that I opened with, of bringing together so many different technologies that have been in development, on the horizon—whether it’s AI, whether it’s 5G, whether it’s safety and security—bringing so many of these things together in ways that I think really are noteworthy, and notably different than what I saw last year at this time. How about you? What are you looking forward to?

Sally Eaves: Absolutely, absolutely. As I said earlier, I think there’s a real acceleration in innovation, and around the actualization. The speed of change has been unlike things we’ve seen before, absolutely. So, I think there is a real step change this year vis-a-vis the one before. So I think that’s really, really exciting.

One of the sessions I’m going to definitely be looking at is Peter Fang—who’s talking about bridging Orchestrator and hard, real-time workload consolidation. I think that’s a really interesting one—with Edge computing, smart factories. It’s really pushing that demand for consolidation orchestration across mixed and critical workflows. So that’s definitely a session I’ll be looking at in detail.

But there’s so many. It really is a kind of smorgasbord really, isn’t it? Fourteen sessions on Internet of Things, platforms and applications—just fits into a lot we’ve been talking about in our conversation today. But I think what I like about this is it’s five days long—you can really pick and mix this, tailor it to your particular organization and also what you want to learn about. There’s so many opportunities to really dive in deep and ask questions.

I also like the matching application they’ve put together as well. So you can really tailor it. It feels like a proper personalized experience. So that’s really exciting because, if you can’t be there in person, then making an event feel like a true interactive experience matters so much. And I think there’s been a real effort around that curation. So I love to see that.

Kenton Williston: Fabulous. Well—should our listeners be coming to this podcast after the fact, where can they find you online?

Sally Eaves: That’s a great question. Well, I think the one to go for, number one, would be @sallyeaves on Twitter, but I’m on all major channels, basically. But Twitter is the easiest starting point. But LinkedIn, my own website, etc., as well. And I’ll share details after the podcast.

Kenton Williston: Lovely. All right. Well, that just leaves me to thank you for joining us today. Really appreciate all your insights.

Sally Eaves: Absolute pleasure. Thank you so much. And really looking forward to the event.

All right, we’ll see you there.

Quantum-Resistant IoT Security

A conversation with Louis Parks @Veridify

[podcast player]

Many IoT systems remain in the field for years or even decades, creating major challenges for security. Building automation and industrial systems are prime examples. Conventional IoT security techniques may be sufficient for now, but advances in technology like quantum computing will soon break popular methods like ECC and RSA.

What’s the best way to protect valuable infrastructure in the long term? Join us as we dig deep into this question with Louis Parks, the Chairman, CEO, and co-founder of Veridify, the creator of quantum-resistant, public-key security tools for low-resource IoT environments. We discuss:

  • Why technologies like firewalls are difficult to deploy in multi-vendor IoT systems
  • Why device authentication is a critical element for building and industrial IoT security
  • How to use bump-in-the-wire security to retrofit legacy infrastructure
  • Why quantum-resistant encryption is needed for long-term IoT security
Apple Podcasts  Spotify  Google Podcasts  

Transcript

Louis Parks: We need to be thinking about: we can keep a building safe today, and certainly for the next five, six, eight years, but how do we keep it safe for the long term? And that’s where you will need to turn to quantum-resistant methods.

Kenton Williston: That was Louis Parks, the Chairman, CEO, and co-founder of Veridify. And I’m Kenton Williston, the Editor-in-Chief of insight.tech.

Every episode on the IoT Chat, I talk to industry experts about the technology and business trends that matter for developers, system integrators, and end users.

Today I’m talking to Louis about the security challenges for smart buildings and industrial automation, and some cool new ways you can lock down your assets—including the quantum-resistant cryptography Louis just mentioned.

Louis, welcome to the show. Can you tell me a little bit about yourself and what Veridify does?

Louis Parks: Sure. Good to be here. Veridify is focused on security for very, very low-resource devices, and really has been since its inception. And what I mean by that, since security’s a very big landscape, we’re focused on identification and authentication—which, not always working—but we take it for granted when we do our banking and do things on large, powerful platforms like PCs or smartphones. But when you have very, very low-resource processors—perhaps in an embedded device or in the Internet of Things—authenticating them and knowing they’re your device can be difficult. So we develop methods for doing that type of authentication identification.

I have three partners who are mathematician-cryptographers and specialists in the area, who helped develop these very efficient protocols. And I’m a co-founder of the company, who helps in figuring out how we take these things to market and turn them into the products that we market today.

Kenton Williston: So, this is a really interesting topic and a timely topic, I think, because the just general security landscape has been—it’s an ever-shifting landscape, to be sure. But I think last year, with the pandemic and people moving to remote work, there’s been just a real, I think, significant shift broadly on the sort of threats people are encountering. So I’m wondering, when it comes to the areas you specifically are looking at—things like industrial IoT and building automation and control systems—how you see the landscape shifting in those specific areas.

Louis Parks: Sure. Well, so first of all, the change in how we work has really brought attention to the whole idea of security. Privacy, of course, is something that comes out of that. But the idea now that—whether it’s your video call on whatever platform you’re using, the ordering, banking, what have you—suddenly we’re all aware that it’s a very digital world, from retail to socialization. So that’s a heightened awareness that we’re operating against.

What we see now, and what has continued as a level of sophistication—because as we’ve tried to connect more things together to make it efficient, so we could work remote, work from home, make the supply chain more efficient, whatever—has given a broader horizon for the hackers and attackers out there to infiltrate and/or go after things. Our focus, again, is on the small devices that run these things.

Specifically, we’ve been very busy the last year in the area of building security. That’s not getting into a building or the cameras on a building, but rather the fact that for years, and even more than decades, buildings have relied on processors to manage the heating system, the HVAC systems—more recently, the lighting systems and the elevators. So even before the term “smart building” came into vogue, buildings relied on processors.

And in fact, unfortunately, as those have been now connected to the IoT, it’s given another access route for an attacker to get to the IT systems in the buildings, and places where valuable data may be. So, really, as we’ve gotten so much more connected and better at operating digitally, so have the attackers.

Kenton Williston: Yeah, absolutely. And, like I said, this is not exactly at an all-new trend—it’s something that’s been happening. This example’s getting pretty dated by now, but people have been talking about, like the Stuxnet attack as an example of how the security landscape is not just about the servers, but it’s about all the equipment that’s out there.

I think it’s fair to say there’s a broad sense that, hey—you’ve got to protect your IoT systems. But I think it’s also the case that people don’t really fully appreciate all the time just what exactly the threat landscape is. So I’m wondering, from your point of view, are you seeing some significant risks that people generally are not aware of?

Louis Parks: Oh, absolutely. When you talk about attacks, and what have you, the threat landscape goes back before we got the label “IoT.” I remember well over 10-plus years ago being in Washington at a meeting where they were discussing technologies to help with border security. Many people listening to this and maybe on the podcast have a car where you can look at the air pressure in your tires on a dashboard or in a display to know if the air pressure’s good or not.

That technology comes to you courtesy of RF (radio frequency), little broadcasters in the wheels talking to your cars that have been paired. And if anybody’s had a damaged wheel, like I have, you know that the dealership will charge you dearly to pair a new wheel to your car. But because it’s RF (radio frequency), it’s not only talking to your car, but it’s broadcasting outbound too.

So the discussion was, “Gee, people are driving across the border. Perhaps we could use that broadcast, that radio-frequency broadcast, as metadata to identify a vehicle.” So, that’s arguably a friendly use, if we believe in border security. But the point being is: that is what would now be considered an IoT device. A car itself is probably, to many people, an IoT thing now. So, these threats have been around for a long period of time. And probably a lot of people have not thought about their wheels on their car betraying them to somebody for the purposes of location, or tracking, or other nefarious activities.

Kenton Williston: Yeah, that’s a great example. I’ve got a pretty old beater myself, so no RF in my tires, but I have to admit that that was a security risk I was completely unaware of until you mentioned it just now. Now, having said all that, our audience is probably smarter than me, very well aware of the many different security risks that are out there, and doing a really good job of trying to secure their IoT systems. So, I’m just wondering, from your perspective—to the extent that there ever is any such a thing as a standard approach to anything in the world of IoT, where every system is a little bit different—what the standard/typical approach to security looks like today. And where you see it being strong, and where you see there being some gaps in the current approaches.

Louis Parks: There’s a couple of things that you want to do, or people are doing. And, in general, there’s a lot of attention being paid now, unfortunately again, because the news is not always great. And we always get reminders—although not IoT—things like SolarWinds remind us that if we’re going to be digital, we are all potentially susceptible to various types of attacks.

The challenge in IoT is significant, in the sense that we have a really wide range of devices—whether you look at industrial, whether you look at a commercial building, or a home—because the devices, number one, may come from many vendors. We’ve all seen the value of a single-vendor solution and the ability to control your world if you come from Apple. And then the value from a marketing perspective if you allow many players to play, like in the Android world. But at the same time, ensuring that all those players are good people.

So in the IoT, when you have a mix of technologies, it becomes a challenge. People are understanding that more and more. So there really isn’t one security thing that you should do. There’s probably many. Certainly the first thing is to know if you have an issue, and there’s a lot of really good anomaly detection, network-monitoring technologies, that are being developed. So that people who want to know, or should know, if they have an issue can know. That doesn’t prevent an attack, doesn’t prevent somebody from stealing data. But, arguably, a very critical issue to know is—is it happening? So that you need to increase or improve whatever it is you’re doing.

Of course, all the other technologies have been around for years and decades. Whether it’s malware protection, firewalls—on and on the list goes—you need to employ when you’re talking about networks. But the IoT and a lot of devices, number one, operate outside of these very controlled networks—the three floors of your office building. A lot of these devices are out in the open.

The other thing is that a lot of these devices are engineered or designed very eloquently to use absolutely the smallest processor that will deliver all the features. So one view of some of the audience might be, “I have all the tools I need. I’m using them today.” They might be on a tablet, or a gaming PC, or a smartphone.

But when you go down to a very, very small 32-bit, or 16-, or even 8-bit processor that’s been optimized to provide a single function in a building or embedded platform network, you don’t have the luxury of the computational capability to put that authentication technology on it—to put that digital certificate and all of the signing and verification capabilities on it that you take for granted—the TLS or SSL solutions you use when you’re on a network.

So there’s a lot of attention being paid to that. There’s a lot of innovative technologies: from how do you take public-key or asymmetric technologies, as we do, both legacy things like ECC or ECDSA—which some of your viewers will know are 30- 35-year-old technologies that still lead the way for legacy—to upcoming quantum-resistant methods. How do you shrink them and make them work? As well as other technologies like PUFs—physically unclonable functions—which are fingerprint technologies, and enable you to provide unique identification on a per device basis, or a seed of identification, a root of trust.

So there’s really a lot of areas that are being brought together, again, because you have a really, really broad mix of devices. And a lot of them need to be out there by themselves, which again is why we focus on device-to-device as an area. But you would not look to us as a single solution. It would be us in combination, arguably, with some of these other technologies to make yourself secure.

Kenton Williston: Yeah, so let’s talk about your solution a little bit, because, like you said, I think when everything’s said and done one of the biggest challenges you really pointed to is—whether you’re talking about an industrial setting or a building-automation setting—you’ve got a landscape with a lot of existing legacy devices that aren’t going to go anywhere anytime soon. You’ve got a landscape with a lot of things that were designed for minimal cost, minimal power. So, what are you bringing to the table to help protect this very diverse, fragmented landscape that’s not really set up, like you said, for the kind of things you would think about in, like a data center or your own home PC kind of setting?

Louis Parks: So, pretty interesting, and we’ve been immersed for about a year now with our platform, DOME, that we developed a few years ago as a platform for device management, not unlike many IoT-product or device-management platforms that are out there. The difference, again, with ours was we were using, or we are using, the ability to shrink protocols, asymmetric or public-key protocols, that allow authentication capability down to fit on the actual device. So a device in the IoT, or a device out there, can actually manage its credential, manage its authentication, without the need to connect to a cloud or a server to do that.

Of course, connecting to a cloud and server is a very valid way with larger devices that come embedded with URLs to authenticate them. But, again, if you have a very small device it’s only going to operate in a limited network, but could provide an exposed platform. That was something that we were focused on.

So we developed DOME, a device ownership and management solution, where we manage a credential in the cloud in a blockchain for the device. But the device actually challenges and ensures it’s talking to something authentic. We took that and translated it to the building-automation world, where a building, again, as I mentioned earlier in the podcast, for years has run on processors managing elements of a building’s operation today.

And, of course, it’s getting even more sophisticated. There’s some really brilliant use of technology to make building smart, more comfortable, more adapted to our use. All of that involves introducing more processors on the operational-technology side. To manage them you connect them to the IT side. Of course, in the IT side is where we find the networks, and then the databases, and the back offices of the people in the building. That’s where the danger emerges. So that has been an interesting challenge and a great use.

There’s one additional element you alluded to, or may not realize you alluded to, and that is that 99% of the market that we’re talking about protecting exists. It’s already there. The buildings have been built, they’re running. So if you’re designing a brand-new smart building today, and if you were just fresh on the plane back—well, you wouldn’t be fresh—fresh off your Zoom or digital call from CES with ideas for all the new technology going to put in it, likely there’ll be some good security tools.

But if your building’s only two, three, five years old, you probably still want to use that very expensive air handler cooling system, what have you, you have installed, but it probably has not got the protections you need. So retrofitting security to a preexisting infrastructure is also a challenge, and something that we’ve addressed with something we call bump-in-the-wire technology, that we’d looked at for a period of time. And, in fact, developed some solutions with our partner Intel to deliver to industrial IoT, and have now adapted it for the building-automation protocols like BACnet, and later Modbus and KNX, to retrofit security to a preexisting infrastructure—in this case a building—which is another challenge in making things secure in current days.

Kenton Williston: Yes, I want to dig into that a little bit more, and here is just a little shameless plug. We’ve got an article that corresponds to this same conversation over on insight.tech, so I encourage our listeners to go check it out. You can get more details on this bump-in-the-wire solution, how it works, and why you might be interested in it. But, just to look a little bit closer at that here and now, can you tell me a little bit more about what this architecture looks like? And you mentioned that it’s got some Intel technology—what kind of technology is incorporated there?

Louis Parks: Sure. So bump-in-the wire’s not a unique solution to us; many industries and areas have it or contemplated it. What we’ve done here—a couple of things that are unique. Number one, we’ve based our initial solution on an Intel FPGA—a small, very powerful, low-cost FPGA. So not only does it ensure that we can address the security issues today, but an investment in this relatively low-cost device will give us the adaptability going forward—because the horizon for the attacks, the nature of the attacks, is continuously evolving.

And, typically, as you’ll see in many, many articles when they talk about buying something that’s connected, or the IoT, they always say, “Make sure you have a way to update to the latest patches and fixes, and what have you.” So not only do we have a very powerful platform to provide the technology, but we have one that’ll allow adaptability.

For the building space what was critical is that we had a relatively simple plug-and-play solution. So it’s a simple plug in plug out between the controllers and the Edge devices that are already installed—typically running on some sort of IP platform or network in the building-automation space. Our initial solutions are designed for the BACnet world, which, again, is a building-automation standard for how devices and buildings communicate.

So, our device is running; it runs the initial ones, NIST-approved, legacy—what I refer to as legacy protocols and methods for certification purposes. But other versions of it will run a quantum-resistant crypto—and we should talk about that for a minute—which is critical for long-term protection. And of course, finally, this is BACnet, which is a building standard. It runs over BACnet IP. We’ve developed other technologies that coordinate with it to ensure that you can also monitor the discussions that are going on.

The summary is: we were creating a secure tunnel from the controller to this bump-in-the-wire device with encrypted data flowing over a BACnet-compliant communication. So we don’t replace anything that a building currently has, or anything in the standard. Then it protects the device it’s plugged into behind it. So, that’s a very simple description of this device. It’s designed to be flexible in the protocols it manages, and what have you. A lot of that power and flexibility, again, comes from the ability of having this FPGA-background platform that will allow us to adapt it. And so, unique functionality capabilities, as we move through the building space.

Kenton Williston: You’re talking about this bump-in-the-wire solution protecting the device that’s behind it. So, are we talking about something where you would need to deploy, like a one-to-one everywhere you’ve got a device you’d want one of these bumps-in-the-wire? Is it per floor, per building? What’s the architecture look like?

Louis Parks: The architecture needs to address a couple of different scenarios. We would suggest the ultimate protection, of course, would be one-to-one, and ensure that every device has this secure, encrypted element—authenticating all the inbound traffic, and encrypting and delivering back all the outbound data back to the controller in the building. That’s not always possible, or feasible, and sometimes it’s just probably not the right architecture.

So, although we do have these relatively low-cost, powerful, FPGA bump-in-the-wires, we also have a similar technology in a router form. Again, the secure connectivity—which we call S Link—so we can run it to a router, which then could have several devices. So it could be a one-to-one, or one-to-many configuration—as is exactly what you find in the building spaces today.

Kenton Williston: That makes sense. I do want for sure to ask you about the quantum cryptography. So, this is certainly, if you’re up to speed on the latest and greatest security, a hot topic. But in some ways it kind of feels like, “Gee, if we’re just talking about a building-automation system, isn’t this really kind of overkill?” So, what’s the rationale behind this, and why have you taken this really hardcore approach?

Louis Parks: Sure. It’s not overkill. As a matter of fact, in addition to providing DOME with NIST-approved methodologies, we—ourselves and my partners, their background is in the mathematics of asymmetric and public-key methods—we’ve developed and published methods which are quantum resistant, as well as we are working with several methods that NIST now has under review for the purposes of standardization.

But focusing just on the question about quantum resistance—again, many of your listeners will be aware—but quantum computers since the late ’70s, early ’80s, were a white paper/physics idea that was out there. And about three, four years ago, actually maybe five now as I think about it, IBM and MIT simultaneously managed to create working prototypes. Now, these are not full-functioning, or were not at the time full-functioning, but proving the science, the technology, behind a quantum computer.

And again, these computers are not in the future going to replace our current computing. It’s a different type of computing. You’re probably not going to have a smartphone running on quantum. But they do manage and process data differently than our current conventional computers. And, again, there’s a lot of articles—it’s years later—many of your readers would be familiar with it. But the reason we’re talking about it—and they have evolved, and they’re getting better, and they’re getting more stable, and they’re getting larger, which is a key element. So they become more practical to use—likely in a data center-type fashion. So they will be great for solving DNA-sequencing issues, discovery of new drugs, etc.

And, unfortunately, there are at least two algorithms that have been developed to run on quantum computers that have been mathematically proven that will attack a weakened—and in one case, break—the legacy methods which I’ve referred to a few times—elliptic curve, RSA, Diffie-Hellman—when you have a large enough quantum computer. So, the part I can’t answer—and it’s hard for anybody—when will that be? It’s not next year or the year after. Could it be five years out, or seven years out? Don’t know. People commercially are working on it, as are nation states. So, it will happen, but we don’t know the timeframe.

Which brings us back to the discussion today on a building, where you put up a building—not unusual to stand for decades, if not 100 years-plus. Arguably the systems get replaced, but they get replaced every 15, 20 years. So a system going in today will likely be around when there’s a large enough quantum computer. And that quantum computer will break the ECC or the public-key methods. You cannot increase the security of ECC or RSA to avoid it. They will actually be broken by Shor’s algorithm in particular, and weakened by Grover’s algorithm.

So we need to be thinking about: we can keep a building safe today, and certainly for the next five, six, eight years, but how do we keep it safe for the long term? And that’s where you will need to turn to quantum-resistant methods.

Kenton Williston: Makes sense. Then the follow-up question there, is: why use FPGAs for this role? Is there something particularly advantageous that they offer?

Louis Parks: I guess the fair answer is, yes and no. So, there are equal processing-capable technologies and microcontrollers and ASICs, and one could even argue in some cases even more optimizable technologies than an FPGA. But the critical element for what we’re doing today—and I think for a lot of the building space, which we have found to be years behind where the general processing community is, and certainly years behind a lot of the new IoT—is we’re providing the tools that we believe and think are critical today, and that landscape is shifting.

I think the key characteristic of the platform that we’re operating with is that it’s field programmable. So, we’re delivering solutions that are going through third-party testing and all the verifications you want to make sure that they’re secure, but will also give us the capabilities to adapt these devices, not only to different building and industrial IoT operations, but also to adapt to the market, and the threats, and the nature of what we’re looking to address, as we’ve been discussing.

So, although in some cases—certainly people are probably familiar with FPGAs—they can cost 1,000s of dollars, the ones we’re working with—and in particular with our partner Intel— are still powerful but are a fraction of that cost. So there is not a penalty from that side, but there is a significant dividend from the flexibility and our ability to address the market.

In some cases, even specific projects that we’re working on, where we’ve had discussions with building owners—sophisticated building owners, who have very extensive networks operating already within their buildings, understand all the operational technologies—and they have several requests that frankly we hadn’t contemplated in the basic platform, but because we’re working in the FPGA world we can answer, we can deliver. So we think it’s an ideal solution that the cost benefit—there is significant benefit from this FPGA approach.

Kenton Williston: So, I’m glad you mentioned the cost aspect, because I think historically there’ve been two big factors that have caused people to shy away from FPGA solutions. So, one is certainly been historically the cost—although, like you said, today there are a lot of very moderate-cost solutions that are available. The other, though, has just been the programming model. The way you configure an FPGA is very, very different from how you would program a microcontroller, for example.

So, if I were considering how I wanted to secure my building or my industrial systems, the thought of adding an FPGA in there I could see making me a little nervous, like, “Is this going to be something that I’m going to actually be able to manage? Or is it going to require me to learn a whole new skill set?” So, what do you say to that?

Louis Parks: So, first of all, to a lot of the industry this process will be obfuscated because we’re working with other partners, and this is their area of specialty—developing products and solutions based on FPGA technology. So, again, where the functionality of the device does need to be provisioned—whether it’s a microcontroller or an ASIC, and the other partners and other areas where we are doing similar solutions at a microcontroller setting—the FPGA, when it’s being provisioned both with the functionality of the platform will also be provisioned with the security technology.

So, again, it may not have the overall efficiency en masse for deployment, but the vendors who are working with it have the basic tools for doing the volumes that we’re talking about here. So, again, I think it’ll be proportional. If this was a high-volume consumer, low, low, low, low cost—yes, this would create probably a larger component of the cost of the device. So, we’re not in pennies; we’re 10s of dollars to low-100s of dollars in some of these cases.

So the provision and costs, I think, are proportional to the device. And certainly, again, the overall payback for this type of platform—I think certainly in the early stages of this industry—is easily there. This has not been an issue so far in the projects that we’ve been looking at or working with.

Kenton Williston: Got it. Great. So, I think we’ve covered a ton of ground here. So I’m going to ask you a very challenging question, which is: if you could wrap this all up and leave our audience with like one key takeaway, what would that be? What would be the one message you would want to convey?

Louis Parks: I think the message I’d want to convey is that we all need to have a realization that behind the things we’re using today there are processors. And just because it doesn’t have a screen and a keyboard, or it’s something that you’re not entering your credit card information into, or your banking, you still need to be thinking about security and protection because of the interconnectivity.

And, again, there are many, many examples—way beyond the couple of simple ones I gave, and much more eloquent ones. But I would suggest that everybody needs to stay aware that, whether it’s you’re working from home, or the fact that you can find a car spot easier in a car parking lot because of some new technology, it’s because things are connected and they’re communicating. And when they’re doing that it’s a convenience, but it’s also a threat platform. And they should recognize that just because, again, it doesn’t have your credit card in it, doesn’t mean that it can’t possess an equal threat. We should all be aware, and hopefully be seeking these solutions to try and stay even—maybe even get ahead of what’s happening in the world of attacks and hacks.

Kenton Williston: Very good. Well, with that, listen, I want to thank you for joining us on the program today. Really appreciate your insights.

Louis Parks: Great. Well, thank you for having us.

Kenton Williston: Absolutely.

And thanks to our listeners for joining us. If you enjoyed listening, please support us by subscribing and rating us on your favorite podcast app.

This has been the IoT Chat podcast. We’ll be back next time with more ideas from industry leaders at the forefront of IoT design.

The Currency of Retail? Engagement with Digital Displays

The consumer shopping experience has been on the cusp of change. Retailers have had to compete with the rise of one-click online purchasing, same-day delivery, and in many cases, free shipping. The COVID pandemic accelerated that change seemingly overnight.

Still, the full sensory experience of in-store shopping has many advantages. While 2020 was a challenging year for retailers, many have realized there is no better time than now to embrace technology.

Quividi, a pioneer in digital audience and campaign intelligence platforms dedicated to digital displays, is perfectly poised to help with that transition. Its solution, implemented across 80 countries, serves two major markets—digital out-of-home (DooH) and retail digital signage.

While digital displays have long been a staple for advertising and in-store marketing, innovations in AI, deep learning, computer vision (CV), and powerful edge compute are opening new opportunities.

“Our platform and data science provide marketers with the unique ability to test, measure, optimize, and deliver data-driven contextualized content,” says Laetitia Lim, CEO of Quividi. “In other words, they can use Quividi to turn DOOH and retail digital signage into a powerful medium boosting engagement, traffic, and sales—all while respecting privacy.” Video 1 shows how a Westfield shopping center in Australia is achieving these results.

Video 1. Smart displays and experiential content react and adapt to their audience to increase engagement. (Source: Quividi)

Measuring Engagement with Computer Vision and AI

From the beginning, the company’s philosophy has been “privacy by design.” Its solution uses Anonymous Video Analytics (AVA) versus biometrics. It can tell a viewer’s gender, age, mood, position, and attention with good accuracy.

As it turns out, you can optimize your communication on AI-enabled displays and gather consumer insights by merely keying in on basic demographics.

“Understanding which content works best for every audience based on one-to-many or one-to-one retailers can do two things,” says Lim. “One, you can analyze the data afterward to adjust your communication and marketing strategy moving forward. Second, you can even use that to adjust in real time. So on the fly, you can optimize for each viewer the content based on what you know has worked best, on which screens, where, and for who.”

But with the rise of online ordering and curbside pickup, today’s shoppers have a strong intent once they get to the store, and it’s more of a challenge to engage them. While online retailers can feature additional products based on what customers have viewed, it is more difficult to upsell the unknown consumer. Smart screens allow retailers to suggest products based on gender and age.

Marketers can use @quividi to turn DOOH and retail digitalsignage into a powerful medium—boosting engagement, traffic, and sales—all while respecting privacy.

Reducing the Friction of Digital Display Adoption

Despite the benefits, these robust systems can be a difficult sell for retail systems integrators (SIs). They may be experts in hardware, software, and sensors, but not in AI and data science. The first run of digital advertising screens hasn’t required that knowledge. But budgets are tight, and getting retailers to invest requires that SIs convey a strong vision and value proposition. They need to show retailers and advertisers that the latest AI-enabled smart screens are not simply another expense, but that they are a business driver.

It’s a problem Quividi understands and validates.

To help SIs, the company is launching its Data Academy for SIs and Kickstarter Kits for retailers. Through a dedicated website, the Data Academy uses text, video, illustrations, and webinars to provide SIs with valuable knowledge, expertise, and convincing selling points.

And to reduce friction for retailers, the Kickstarter Kit allows SIs to offer zero-cost, no-obligation, Quividi-powered proof-of-concept (POC) systems. “In parallel with the Data Academy, the Kickstarter kit is enabling us to embark the end users onto the data journey,” says Lim.

Right off the bat, retailers can see the operational benefits of the screens. For example, they no longer need staff to stand outside and count the number of people entering a store or be responsible for enforcing social-distancing guidelines. The screens can do that for them. In this time of COVID, the technology helps protect the health and safety of shoppers and staff.

And over time, they will have data to show the long-term benefits of the Quividi solution, and that allows them to make informed decisions and increase engagement, which drives more sales. The insights that retailers and brands gain can also show where they need to adjust their marketing and communication overall.

Retail Analytics Enabled Through a Larger Ecosystem

To deliver these types of benefits, you need an extensive ecosystem. Quividi is deeply integrated with many different hardware manufacturers, software vendors, and advertising agencies. Its technology currently supports some 30 different CMS systems. That compatibility goes a long way to increase the rate of implementation because its retail and advertising customers can use the tools they already have in place.

Intel® has played a major role in educating the ecosystem and is an essential partner for Quividi, on both technology and marketing fronts. For example, the Intel® OpenVINO Toolkit is core to its computer vision and AI development, enabling the company to scale the solution in a cost-effective way. And Intel processors provide the powerful edge computing essential to generating real-time data.

“Intel is deeply embedded in our solution,” says Lim. “The technology allows us to deliver the right content, to the right audience, at the right time. And in the end, AI-enabled intelligent screens offer a stronger value proposition for SIs and clear business benefits for retailers.”

Building Automation Offers Safer Workplaces

In the face of a global pandemic, building owners and operations managers have been challenged with providing a healthy and safe environment as employees return to work. Many have been racing to come up with a plan while others were ready to hit the ground running. What’s their secret? The answer lies in smart building technologies, powered by the IoT and AI.

Building automation and asset control are not new. They offer the monitoring and analytics required for use cases such as reducing energy use and lowering the cost of facilities management. Now, the same technologies are being put to work in new ways, as organizations welcome back their staff. And as time is of the essence, modifying an existing solution to address response and recovery is far quicker than building something new.

Intelligent control and IoT solution provider IAconnects Technology Ltd. expanded its MobiusFlow commissioning solution to help companies adhere to health and safety requirements. The IoT edge platform allows connectivity of an ever-increasing number of sensors, control systems, HVAC, lighting, and office workflows. And now its IoT Smart Monitoring Solution Kit social distancing management system monitors and reports on the occupancy status of communal areas in real time.

Before the pandemic hit, office managers might have pulled occupancy reports once a week to give building owners and managers a sense of changes in use over time and to help manage energy costs. But today, real-time data is critical in managing occupancy at any given moment.

“A lot of our solutions that are more COVID-focused were already solutions in their own right,” says Pete Smith, Head of Sales and Marketing at IAconnects. “Today there’s a need to have occupancy data immediately so building users can see either how many people are in a particular area, whether they’re allowed to enter, or if it’s at maximum capacity.”

The IAconnects newest plug-and-play solutions—a room management kit and a desk management kit—come with the MobiusFlow Edge Gateway and software (edge or cloud), protocol converter connectors, and a range of sensors.

The room management system feeds real-time data directly to its management system that indicates if entry is allowed and when cleaning is required. The desk management system shows if a workstation is currently, or has previously been, occupied and therefore requires cleaning.

.@iaconnects expanded its IoT smart building platform to help companies adhere to health and safety requirements in real time.

Building Upon Today’s IoT Tech

Adapting an existing system may be a faster way to a solution during a pandemic, but addressing a few crucial deployment concerns can further help speed up implementation.

“A common challenge in creating IoT solutions is reducing the end customer’s cost while also providing a single, scalable control system,” says Smith. “You want to provide the best sensors or devices for the job. Having everything on the same protocol is often less expensive, but may not meet all solution requirements.”

The company strategy is to develop hardware-agnostic systems that integrate with the sensors, and controllers that a building might already have in place. So rather than having one solution for lighting, a different option for heating, and something else for occupancy, that decision allows its clients to choose the best devices for the situation while also having a single point of control (Figure 1).

Smart Building AI technology computer vision building automation
Figure 1. A typical IAconnects sensor schematic demonstrates a scalable IoT ecosystem. (Source: IAconnects)

IAconnects has designed its solution to be an accessible, no-coding-required tool that almost anyone could use with minimal training. To further assist customers, the company provides an extensive library of resources, and a user’s group provides an online community where peer advice can be given and received.

As a result, whether companies are new to smart building technologies or have been using them for years, implementation is a smooth and easy process and has a quick return on investment.

“Doing that development upfront allows us to work with companies that might have an office with just a few employees who need only a small amount of data, right up to big contractors that have cloud services or third-party applications already in place,” says Smith. “Between the support site and our engineers, they can access whatever they need.”

Smart Partnerships Lead to Scalable Solutions

Intel Atom® processors power the MobiusFlow gateway providing the scalability and capacity to support real-time data analytics required for smart building applications. “The reason we went with an Intel®-based gateway is because it provides that relevant power for scalable solutions,” says Smith. “The gateway can work for a very small organization, or it can also handle thousands of sensors in a single building for a large corporation.”

IAconnects’ relationship with Intel benefits the company beyond just technology. Arkessa is another important partner for the company, integrating Arkessa 3G/4G technology into the MobiusFlow gateway.

“It’s a win-win for us to work with Arkessa,” says Smith. “We’ve known them for a number of years and are continuing to build our partnership on an almost weekly basis with projects we’re working on together. Intel and its wider ecosystem has been a great decision for us, and we hope to continue that well into the future.”

AI and Thermal Imaging Streamline Temperature Checks

Current health monitoring systems, such as human-operated temperature checks at entry points, were hastily thrown together to fight COVID-19, but their limitations could actually compound problems. The time-consuming, arduous process can be inconvenient for people trying to make their way to events, meetings, and appointments, and long lines impede social distancing, potentially spreading germs.

Organizations looking for a more efficient, scalable solution to ensure health and safety in the future can use technology combining AI-powered video analytics and thermal screening, such as the Heat Detection Camera, created by a partnership between Digital Barriers and Vodafone.

“What makes the solution so powerful is the Heat Detection Cameras’ remote monitoring capability,” says Kenny Long, Business Development Manager for UK-based edge intelligent video technology company Digital Barriers. “With holistic connectivity, the solution can encompass cameras installed in multiple locations, and alarms can be viewed locally or remotely via a central monitoring center or on a smartphone. This means that staff don’t need to provide direct supervision to the device on-site.”

The solution combines unique low-bandwidth video streaming capability and intelligent analytics with Vodafone’s innovative IoT connectivity and support service. This gives end users or systems integrators a fully managed secure end-to-solution straightaway, rather than having to source the hardware and each capability from multiple vendors.

“The Heat Detection Camera system combines unique low bandwidth video streaming, intelligent analytics, and Vodafone’s connectivity and support service.”

How AI and Video Analytics Improve Safety

The Heat Detection Camera system uses a thermal camera, a laptop, and a blackbody tool that gathers the ambient temperature of the room (Figure 1). “On a really hot day, you might get lots of false alerts if you did not have that blackbody,” says Long. “It’s a key part of getting accurate results.”

A person entering building and walking past thermal camera for temperature check.
Figure 1. Heat Detection Camera uses thermal imaging to detect visitors with high temperatures. (Source: Digital Barriers)

Heat detection cameras, which are powered with Intel® processors, scan people as they enter a location and display encrypted body temperature thermal imaging on a laptop, smartphone, or another device in a control room. The system can register up to eight people at a time, and as many as 100 people in 60 seconds, making it a good choice for mass screening at large businesses or public places, such as airports or stadiums.

“If you try to use heat detection guns at a stadium with 60,000 spectators, it would take about seven hours to get everyone inside,” says Long. “This system will take less than half an hour.”

The solution also operates on cellular connectivity using pioneering video transmission, which compresses and encrypts the footage, providing huge data savings—an important consideration for organizations that want to minimize expenses. The live-streaming cameras measure available bandwidth and adjust the amount of detail before sending video to a device so that it doesn’t exceed the bandwidth and cause a delay. “An ultra-low bandwidth can save the customer up to 60% on data costs compared to anyone else,” says Long.

The system can be set up by the end user, a systems integrator, or through Vodafone. It is typically connected to via a Vodafone SIM card. “It’s a very simple out-of-the-box solution,” says Long. “You’re literally ready to go.”

Gaining Real-Time AI Insights

Digital Barriers’ partnership with Vodafone allows customers to have a managed solution, and current Vodafone customers can add the service to their existing contracts. Customers also get access to a 24/7 helpline for support.

When the solution is delivered as a managed service, companies gain an added layer of efficiency and protection, including live-stream monitoring to multiple devices, providing real-time information. Otherwise, camera latency can cause a lag of up to six seconds or more.

“If you don’t have a security guard standing with that camera, the person may have walked away by the time the information reaches the control room,” says Long. “If a high temperature is detected with real-time viewing, an alert is sent to the devices, pinpointing the location of the individual and giving the organization time to follow its safety protocol.”

Making Buildings Smarter and Safer

The Heat Detection Camera is currently deployed in several types of venues, including sporting arenas and airports. For example, the Wasps rugby team in Coventry, England uses the solution to screen players and staff members as they enter the stadium and training facility to help keep everyone safe (Video 1).

Video 1. The Wasps rugby team deployed the Heat Detection Camera to get players back on the field. (Source: Vodafone)

The system has also been deployed at other sporting venues, as well as England’s Bristol Airport.

“You can’t stop the virus from spreading without identifying people who have symptoms like high temperatures. It’s absolutely vital to have this sort of technology at airports,” says Long.

While COVID containment is a central focus today, the solution’s camera technology and AI capability can also be used to aid security operations and building surveillance. “The system can become part of your security efforts for the whole organization, with an upgrade path to other enhancements,” says Long. As well as commercial entities, Digital Barriers provides tried and trusted IoT and edge-AI technology to military and government agencies, and follows all privacy procedures and encryption regulations.

“Digital screening solutions have three essential components. It should be seamless, secure, and safe,” says Long. “I think that’s pretty much how you can describe our solution. We offer seamless entry and secure encryption in a tool that helps keep organizations safe.”

AI and Computer Vision Create Safe Spaces

Imagine the safety control room at the heart of a bustling public space like a busy train station. A pair of employees is scanning a handful of monitors that display rotating frames from cameras positioned around the property. But that means precious minutes can tick by before a specific view pops back up; if an incident happens in the meantime, this lag allows it to go unnoticed until the image reappears and alerts the control room. By then the damage could be done.

Now, picture this control room in a “smart city,” with AI technology that uses object detection to surface a potential issue just as it arises. By notifying the team immediately, they can take charge and address the issue before it escalates. Crisis averted.

AI Technology Helps Identify and Measure Risk

That’s the promise of Sensing Feeling, a specialist in computer vision, machine learning, and AI. The company’s SensorMAX platform proactively detects and prevents incidents, using existing infrastructure, such as the station’s closed-circuit television cameras.

“We add value to the cameras or visual technology that a customer has already invested in by dramatically improving its performance,” says Jag Minhas, CEO and founder of Sensing Feeling. “Instead of having to make resource-intensive decisions, like adding more people or screens in order to provide a safer environment, our solution uses software and AI technology to identify the risks for them.”

The AI software will process the feed from every camera connected to the system and give the control room operators a dynamic, real-time assessment of each zone’s risk, based on a repertoire of pre-trained models combined with behavioral analytics.

Each client determines the events that it believes reflect risk in their particular scenario, and Sensing Feeling uses that information to attach a “risk index” to every camera. For example, in a train station in a smart city, that could mean the software has sensed an unaccompanied child; a crowd that is developing at a time of day when it normally wouldn’t or is running in panic; or a person riding a bike against regulations.

Given the limited number of displays in a traditional control room, SensorMAX then uses its AI-powered software to prioritize what the client is most interested in or worried about, and ensures that the riskiest camera zones are being displayed. “We surface on their screens where we believe the highest risk is playing out at any moment in time,” Minhas says. “In this way, it can improve an existing system by offering a live indication of risk as it’s developing so you can potentially de-escalate situations, where before you would have had to rely on a postmortem approach.”

Of course, public safety is at its heart, but using AI software proactively has a financial benefit for the customer, as well. Preventing incidents rather than reacting to them allows clients to avoid costs that could otherwise arise.

.@sensingfeeling adds value to the visual technology that a customer has already invested in by dramatically improving its performance.

Partnering With Industry Leaders

To bring its solutions to the marketplace, Sensing Feeling works with channel partners that already have sector experience. “As a group of AI and ML experts, we find our most successful business development activities come through partnering with someone who understands what SensorMAX can offer, and then is able to package it to add value to a solution they’re already selling to an end user,” Minhas says.

For example, the company might join with a design firm that can use information collected by SensorMAX to identify motion paths and clustering behavior in commercial buildings. These measurements allow them to understand how people gather to better plan space to promote productivity and collaboration—or change traffic flows to meet new COVID-related regulations.

With an architecture based on the Intel® OpenVINO Toolkit, the versatile SensorMAX technology can be applied to a wide range of situations in a variety of different industries—beyond smart city, office building, or public space applications.

Other notable use cases Minhas cites include reducing accidents in the oil and gas industry or alerting managers to early signs of stress and fatigue among manufacturing workers. “Its practicality extends to any customer who wants to improve user experience and enhance an environment by managing safety and risk, while preserving privacy,” Minhas says.

Ethical Mandate at Its Core

Personal privacy is top of mind today—as it should be—and Minhas emphasizes that the system’s architecture is designed with that as a priority. As he explains, the system uses sensors, rather than surveillance cameras, which allows it to process data, but not record, store, stream, or transmit images. That means that Sensing Feeling doesn’t track specific activities or any biometric profiling, such as facial recognition, that would identify an individual.

“Our tagline is ‘ethical by design,’ and we aim to uphold that prominent position,” he says. In fact, they encourage their customers to be transparent with end users about how and why the technology is being deployed. Minhas adds, “The reality is that if you can’t communicate a benefit, then we don’t think our solution should be adopted at all.”

15 IoT Twitter Influencers to Follow in 2021

What better way to kick off 2021 than with 15 hot follows in the IoT Twitterverse. Whether you’re looking to keep your finger on the IoT pulse or interested in keeping up with industry trends, these are the influencers to follow.

1. Kevin Jackson

@Kevin_Jackson

A Top 5G Influencer and a Top 20 Tech Blogger, this DC-area author is a globally recognized cybersecurity and cloud computing expert. Take his crash course in digital transformation on this recent episode of IoT Dev Chat.

2. Sarah-Jayne Gratton

@grattongirl

One half of the tech power couple The Grattons, Sarah-Jayne Gratton is a digital strategist and technology influencer who covers AI, 5G, AR, and Big Data. Catch her on our UK-focused podcast mini-series: Retail Tech Chat.

3. Dean Gratton

@grattonboy

The second half of this London-based power couple, Dean Gratton is a tech influencer, analyst, and futurist, covering AI, IoT, IIoT, SmartHomes, SmartMeters, Energy, and Digital Transformation.

4. Neil Cattermull

@NeilCattermull

This London-based analyst and tech influencer is fluent in tech, IoT, cloud, blockchain, and AI. His Twitter feed is curated from across the IoT and embedded space. Bonus: He’s into balloon animals and pasta.

5. Diana Adams

@adamsconsulting

Looking for digital transformation, IoT, 5G, AI, ML, big data, and automation? This Atlanta tech journalist is the follow for you.

6. Antonio Grasso

@antgrasso

A big-picture thinker, Antonio’s feed is a great place to track the trends that matter, like AI, blockchain, FinTech, and IoT.

7. Ajit Jaokar

@AjitJaokar

Ajit is your go-to for all things AI, IoT, and Bioinformatics, plus he’s Course Director of Artificial Intelligence: Cloud and Edge Implementations at the University of Oxford.

8. Rob van Kranenburg

@robvank

Rob is the founding member of IoT News and voted one of IoT Day’s Top 20 thought leaders in industrial IoT. Follow his feed for all things IoT.

9. Peggy Smedley

@ConnectedWMag

Peggy is a podcaster, influencer, and futurist educating the world about the IoT and emerging tech in an effort to inspire next-gen women and men as innovators.

Unleash your inner geek with @insightdottech’s top 15 IoT influencers to follow in 2021.

10. Chris Isak

@chrisisak

Into tech, gaming, and geeky things? Chris is your man. He also dabbles in all things AV and IoT.

11. Beverly Eve

@BevEve

This London-based co-founder’s feed is the place to be for tech, innovation, IoT, AI, Cloud, 5G, Big Data, and Digital Transformation.

12. Shawn Hymel

@ShawnHymel

Believe that education is the best form of marketing? You’re in good company with this freelance content creator. Added bonus: his quirky videos.

13. Evan Kirstel

@EvanKirstel

This Boston-based B2B Tech Influencer supports Enterprise Clients with virtual events in telecom, 5G, IoT, and Cloud.

14. Ronald van Loon

@Ronald_vanLoon

A Top 10 Influencer, follow Ron for the latest on AI, machine learning, and Big Data + live coverage of IoT shows around the world.

15. Dr. Sally Eaves

@sallyeaves

Professor Sally Eaves aka the torchbearer for ethical tech, has recently been ranked 8th in the world in blockchain impact and is ranked in the top 10 for digital disruption and across frontier technology subjects.

 

Have more influencers to add to the list? Share them with us on Twitter: @insightdottech.

Catch our top influencers from 2019 and 2020.

Hot AI Trends for 2021

A conversation with Ray Lo @OpenVINO

[podcast player]

2020 was an eventful year—and AI played a major role. Whether guarding against overcrowding or helping factories ramp up mask production, AI truly showed its value.

So what’s next for AI and its cousins, deep learning (DL) and machine learning (ML)? We put that question to Ray Lo, an OpenVINO evangelist at Intel. Join us for a lively discussion of the state of the industry and the big trends ahead in 2021. We explore:

  • Why AI applications like natural language processing (NLP) will be hot in 2021
  • How developers can strengthen their skill in AI, ML, and DL
  • How to create ethical AI applications
Apple Podcasts  Spotify  Google Podcasts  

Transcript

Ray Lo: I always find people are too ambitious about AI. That’s how I find that was a pitfall. I’m an engineering background. We have to be realistic about exactly what this can do and what it’s good at.

Kenton Williston: That was Ray Lo from Intel. And I’m your host, Kenton Williston, the editor-in-chief of insight dot tech. Every episode I talk to a different expert about the latest ideas and trends that are pushing IoT innovation forward. Today’s show is a look back at the ways AI changed in 2020, and a look forward to what’s ahead in 2021. There’s a lot to talk about, so let’s get to it!

So, Ray, I just want to welcome you to the show. Could you tell me a little bit about who you are and what you do at Intel?

Ray Lo: Great. Yes. Hi Kenton. So, my name is Raymond. I’m an Intel software evangelist for OpenVINO. So, OpenVINO stands for Open. VINO is visual inference. And then, NO means neural network optimizations. So, it’s a big name, but what it means is when you have a CPU, you want to run the fastest possible neural network on Intel, you run this tool called OpenVINO. And that’s what I do. I’ve been giving this news to many people at Intel.

Kenton Williston: Very cool. And how long have you been in this role?

Ray Lo: Pretty recent. I joined about… Let’s see. Hold on. I’m doing my finger math. Oh, four months ago. And now been giving talks at Intel and all that.

Kenton Williston:  Well, one of the first things I wanted to ask you given your background there is what exactly AI is. Put some context around that. We’ve been on the insight.tech program doing a lot of work around the OpenVINO platform. And its applications in everything from machine vision to predictive analytics. So, there’s a pretty broad scope of stuff that people think about when they say AI. And of course, there are related terms, deep learning and machine learning. And I think oftentimes all these things get conflated and it’s a little confusing as to which thing is which. So, you want to give us your primer on what in the world AI is and how it differs from these other ideas.

Ray Lo: Sure. Maybe I’ll put a one line about my background. So, from a perspective from my side. So, I did my computer science from Toronto, and then, I did my PhD there as well for computer engineering degree. So, my thinking about AI is that what AI stands for, artificial intelligence, right? So, we always think about there’s a way to emulating, simulating, just want to make a brain that behave like human, right? So, things like predicting things like object recognition and all that.

But what I always see people confuse is there’s a part called machine learning, there’s a part called deep learning. So, those three categories people always think about them in a mixed way. I always think AI is a big umbrella that cover many of that. And within that, you have machine learning. And within machine learning, you have something called the deep learning. It’s one category, like the neural network, where people… I will say more recently because of the computation power allow us to do that. So, it became a lot more popular recently because back then, when I was starting school about 15 years ago, when you’re doing this kind of math, it may take a year before the training was finished. But today, we talk about weeks, maybe days. And if you’re very smart about it, maybe in couple of hours, you can get some results done.

Kenton Williston: Yeah. It’s amazing how much progress has been made, which leads me to ask. This whole podcast I want to talk to you about what are the trends that you’ve been seeing in 2020. So, just kind of open-ended question for you. Beyond the amazing continuing increase in processing power, what do you think some of the biggest trends of the year were? Not only AI, but deep learning, machine learning, all the related areas.

Ray Lo: Right. Because in the last year, I’ll say you hear a lot of podcasting about computer vision side, which is my background too. But I start to see the trend of kind of beyond vision. So, we will be seeing application, like NLP, natural language processing. It’s matured a lot recently. For example, one trend I saw were something called a BERT, was a new I would say a framework [inaudible] from people created for doing natural language processing. And the result is astonishing. So, it can actually… What they can do is they optimize and fine tune it for applications or tasks called SQuAD. It’s Stanford question and answering database. So, they can literally answer questions better than humans. So, if today I take the SAT test, I don’t think I can win. It’s things like that.

So, it’s kind of like, okay, there is certain tasks that now today machine can do so fast and so much better than human. So, that’s one trend I saw is in call center, especially this year is such a crazy year, we’ve seen a lot of disasters. Like, bad things happen. So, one trend were call center are now automated a lot better than before. So, they have machine learning behind it to answer the call, and then, translating what you said, direct you into the right system, or sometimes even answer questions for you.

Kenton Williston: Yeah. For sure. Happily I haven’t been in an emergency situation or anything like that where I needed to get a quick response from a call center, but even on my own daily experience, I’ve got an iPhone and Apple Watch and all the rest. When Siri first came out, it was just really a joke. You could ask it to set a timer maybe. And maybe it would get that right, but it was pretty bad. And now, it is gotten to be very perceptive.

Like, the other day, I happened to be reading my daughter a book that talked about the design called the fleur de lis. And I tried to make a drawing of it to show her what it was. I was like, “Well, this looks terrible. Let me just see if Siri could help me out.” So, I just raised my wrist to my mouth, and asked Siri to show me a fleur de lis, and sure enough there is an image of a fleur de lis on my watch. It’s gotten to be very good at answering broad questions. Same for all the rest. Alexa and all the rest of those too. They’re much, much better than they used to be, even just, say, a year ago.

Ray Lo: Exactly. I even forget how to set an alarm sometimes. I have to really like Google Alexa. I just tell a story read and getting into the menu and all that. So, that’s a lot of task, like what we talked about, become a lot more natural to human. And behind the scenes, you can actually see all the data center crunching all this data for us. And then, doing all this heavy lifting. And that’s why I really find is really cool in this year.

Kenton Williston: Yeah. For sure. Of course, you mentioned the difficulty we’ve had this year and everything that’s happened around the pandemic, of course, has been really dominated not just the tech industry, but what’s happening in the world at large. But I think as difficult as the situation has been, there’s also a lot to be excited about in terms of how all these smarter technologies helped the world respond to COVID.

Ray Lo: That’s correct. I actually did a study… I was at Google before Intel, so I was looking at some of the case studies they did. How they scale the call center. So, that was really lifesaving because with all those emergencies, they take about millions of call in a day. When I think about where do you get millions of people, right? Especially people at that level of stress. They want to get simple answer. And those are really I think… It’s really the future. We always think about oh we worry about the jobs, but those are the jobs that’s not even we can be able to scale to. And then, sometimes essential for us. So, I find that is something very new to us.

Kenton Williston: Yeah. For sure. I think you’re making a really good point there about the longstanding concerns that the robots are going to come and take all of our jobs, which there is some merit to that. Certainly automation has changed the job landscape broadly speaking, but I think AI is really poised to do jobs that just weren’t possible before. And also free people up from the really ugly bad jobs to do things that are more pleasant. So, one example that, again touches on the pandemic situation we’ve been in, are the many different kinds of machine vision applications to do things like scan crowds for fevers or…

One nice one that I was just reading about is a simple digital display that’s paired with a vision system to tell you, “Hey, there’s X number of people in the store.” So, it’s a really non-invasive way, non-confrontational way of saying should you feel safe entering the store or not, is this meeting with regulations to enter the store or not. And that’s a job that would be pretty unpleasant to do as a human being.

Ray Lo: Right. I think I see… I work with partners a lot in Intel. For example, I talk with ADLINK. So, they released tools to gather in logistic house. So, it’s Christmas time guys, right? We’re getting a lot of gifts. Just millions of millions, maybe billions of packages sending around the world. And then, they scan that, double check that for you, before you receive them. And I think just reducing maybe 0.1% of the error rate, just 0.1 maybe, it’s such a huge deal. You can imagine all the gas you waste, all the energy you waste, to deliver something raw. Having those checks in place is such a great thing that’s happening in the industry.

And so, is inspections, right? Safety, I saw, like… for example, it will check the tool for you. If see anything defected, like for car for example, or inspect the wheels. Those are lifesaving for me, and I find those a type of job that even if you give to human, you don’t want to afford that 0.1% error.

Kenton Williston: Yeah. Absolutely. And just goes back to your point you’re making about how in many applications, machines are doing much better than a human could ever hope to do. It’s not even a question of are you replacing a human? It’s just something a human just could not do absolutely.

I’m curious though. We’ve talked about a couple of key areas. I think one of the key areas in 2020 for sure was machine vision. There was a lot going on there, whether it was in an industrial setting like you were describing doing inspection of packages and parts and whatnot, or in the more public sphere to do things like tell if people are wearing masks or not. And of course, you talked a little bit about language processing, I think, has also been really, really important. What other areas have you seen some important movement in?

Ray Lo: For AI, right? I want to give some suspenses because I see a lot of things in the industry. I want to talk about AR as well coming up. Something that I personally work on. Before Intel, I was a CTO for a company building augmented reality headset. So, more recently, I think you may have seen from various companies like Facebook. They’re releasing augmented reality headset. But behind the scenes we start to realize a lot of machine learning will get into a place like recognizing places, landmarks. A lot of decision that will make for you, it’s not going to be done by human behind the scenes to say, “Okay. Trigger this. Show you this.” So, they start to look into a lot of those efforts that I have seen. Quite amazing.

For example, six, seven years ago when I did a SLAM tracking… And once you have the landmark, I always have to question, “Now what?”

Kenton Williston: Right. Now what? Right.

Ray Lo: “I have a landmark. Now what?” Right? So often the after delayer is how do you take this data, and then, generalize it or create methodologies so that people can utilize them? Like, one way I’ve seen is, okay, now you have a scene. Now you recognize the chair. You recognize the table. And you make a scene of information that you can use for content. So, I had a application at one point, they generated a workspace. When you see a desk, a chair, it automatically generate a virtual screen. And people recognize everything, like a setup, as if it’s in the real world. I find that super cool because it’s like the sci-fi movie, but I work on research for many years, and that’s fascinating.

Kenton Williston: Absolutely. And it strikes me that that’s also a really great example of the different kinds of machine intelligence because there is, I’m sure, elements of deep learning and machine learning in terms of recognizing the scene. And then, some AI to decide, “Well, what should I do now that I recognize the machine?” And I think really illustrates how all these different concepts play together.

Ray Lo: Mm-hmm. Yeah, so if ask me, I never say it’s one application. But I see a set of tools that work together, turning to a new experience for human. And it’s like today. You’re going to shopping, right? You often may pull up your phone to look for the barcode and look for the discount. And et cetera. Et cetera. But think about we can automate the entire process. You just walk in the store, pick up the best thing, the coupon automatically applied. You just focus on the shopping instead of trying to go through that painful experience. And then, that’s what we’ve been seeing the retails. A lot of automations are happening and behind the scenes it’s real machine learning driven. Some of them of course what we’ve talked about the tracking and helping the people.

Kenton Williston: And absolutely. I did a podcast series recently talking about retail. And there’s so many interesting examples there. One that really made me laugh was there is an application where they used the RFID to do some analysis of theft that was happening in the store. And they discovered that one of their biggest sources of loss was actually people taking products from one floor, and then, going up to another floor and saying, “Oh, I need to return this. And I don’t happen to have the receipt.” Et cetera. So, there was theft that was happening without anything actually leaving the facilities. So, lots of interesting applications, for sure.

And that makes me think, with all these concepts becoming so prevalent across I think pretty much every industry, would you say that for developers getting skilled in machine learning and AI are becoming really a requirement?

Ray Lo: I will say… Okay. I felt like today when you think about using machine learning, AI, it’s like back then when I was doing math on top of calculus and linear algebra. It’s like fundamental that if you don’t use those tools, you may be missing out for a lot of potential applications. Of course, you don’t have to use it for everything. For example, you just want to print a “Hello World” on the screen, you don’t have to take up your machine learning textbook tonight. Okay, it’s not design for. It’s just doing something simple, right? But I see that as a momentum that I see a lot.

I did some research on the trend on the machine learning. I think it’s published by Stanford too. So, in the last 10 years, the growth was close to exponential. So, the number of conferences with attendees, like they double every year. So is the publications in Europe, China, America and the patents that we file related to machine learning and deep learning. So, this is something I find… just like back then when we talk about internet and this is pretty much happening again. It’s something that if your phone doesn’t have a camera or internet, that’s like it’s not working. That’s how I feel now if you try to get into this field today without having some fundamental, it may block you from your creativity.

Kenton Williston: Yeah. And that makes sense to me. But I think on the other side, when I think someone who’s new to this field, starts looking at the diagrams of convolutional neural networks and things like that, it can be a little overwhelming.

Ray Lo: Hmm. Exactly why we have OpenVINO. That’s exactly what we… I’m not trying to sell this, but… Well, that’s why we have OpenVINO to encapsulate a lot of the automization steps, which I don’t think you want to get a whole PhD on that problem. And it’s really hard. Just getting the quantization problem right is very difficult. So, that’s why in Intel, OpenVINO, we have a lot of engineers just focus on those big problems. Like, how to get the… exactly what I talked about, performance-wise. Or just getting the tool together so that you don’t have to learn everything, but you have to know it of course fundamentally what mimetic is, what does it do.

But for the deployment perspective, for the development perspective, not engineering perspective. When I always think about development is like copy and paste code, make something quick and easy first, and prove your concept. Like, a prototype. Now today we had a couple of hackathons. In a week they built something I spent six years on in my PhD. I was like, “Oh, that’s not fair.” But that’s the reality. That’s the reality, right? That’s what’s happening.

Kenton Williston: Well, that just seems to me broadly speaking how AI platforms and deep learning platforms are evolving in general. Like you said, even just a couple of years ago, to develop some of these applications would be a huge amount of work. And now, there’s so many platforms that offer pre-packaged models or even things like… You were talking about using SLAM. You get a little developer kit that has a mobile robot and ROS operating system and the SLAM already built into it. So, it’s giving you a tremendously advanced foundation to start from. And you still need to do the work of course to implement whatever it is you’re doing for your specific application, but you don’t have to get bogged down in all of the fundamentals as it were.

Ray Lo: Exactly. And that’s why I believe too SLAM took me millions of dollars to build. It was no joke. My journey where I have to find a professor. Then do collaboration. Then from the collaboration I have to sign a contract. From the contract I get a source code, I have to maintain a source code. Then from the source code, I have to debug. And then, we went back and forth for half a year. That was the reality I was facing. But today, you download a package, it’s been tested, calibrated. Hardware, software, all working together. And then, that’s the new reality we’re facing.

Kenton Williston: Yeah. Exactly. Much better.

Ray Lo: Much better. I’m so happy.

Kenton Williston: So, if you are a developer looking to get into this field, what would you suggest is a way to get started?

Ray Lo: I would definitely recommend people start looking at existing tools because we spend a lot of time and effort… I’m not going to say only OpenVINO, but TensorFlow, all the open tool in the market. And get familiar with the framework and then the understanding of the mathematics. I still think mathematic wise you have to go for it. Even if you don’t have the math background, there’s a lot of good lessons from Coursera. Even OpenVINO have open courses. They can take and get that understanding. Once you have that understanding, now you see the possibility.

Then you get into the nitty gritty details. So, we have a lot of demo code you can try. Trial the demos. I love demos because they open up the imaginations, right? When I work with a lot of developers, surprisingly a lot of them from India, they are students. They come up with new ideas that I never even thought about because… and I ask like, “How do you think about those?” “Oh, I remember I tried this demo. And I tried this demo. I tried this demo. If I combine all this demo together, I get a new demo.” I was like, “Wow. That reminds me of Legos.” Right?

Kenton Williston: Yeah.

Ray Lo: Yeah. So, having that understanding, having that flexibility, having things working in a modularized way, and putting them together is the new trend. So, I think that’s where I think a lot of people should focus on the beginning. Just don’t get bogged down on just one technical detail now, but instead think bigger. See if you can solve the world problem. And once people understand it, love it, get a team together, now the resource will come to you because now you are proving your point. So, I think it’s much better than before. You got to be in a research for four years on particular small problem, and then, that’s it. That’s how I see it differently.

Kenton Williston: Yeah. I think one question that leads me to is you’re kind of painting a picture here of almost a blue sky environment where you can just really be creative and put all kinds of new ideas together in ways that people haven’t thought of. But obviously anything you’re doing has to fit within the budgetary constraints, which not just the dollars, but you got power constraints, or you got some kind of rugged environment where you might have a different thermal constraint or whatever. So, where do you see the state of the art in hardware now? I’m wondering in particular if there are advances that the broad developer audience might not know about that would raise the ceiling on what’s possible inside of these constraints?

Ray Lo: That’s a very fundamental problem when I work for my partners, right? So, once every use case. I think now I will say sky or the space is the limit because we had one success story where someone put the Movidius VPU on a satellite. So, that has a much, much harsher requirement than anything else because beyond just thermal, they have to think about radiations. They’re going up to the space. So, things like that I think when we are building product today, today we have a lot more flexibilities to. Back then, you’re constrained to a extremely power hungry GPU or maybe at that time may not be a powerful enough CPU that’s not optimized for the code. Now, it’s a lot better and a lot better. Or you will be really stuck on this extremely low power, low performance, like a Raspberry Pi at one point.

But today I think we have a lot of our hardware accelerator platforms available. Like, just recently OpenCV released a project called O-A-K, OAK. And now you have a camera with a billion Intel hardware accelerator process in it. And then, it just changed the landscape how we think about processing. We always think about processing as a device, a processor, maybe an extra processor like a GPU or maybe something on top. Then you have a cable that connect everything. But with those kinds of newer approach right now is everything in one chip. Like, you have the Intel chip next to the image processor. And then, you may even have a slightly underpowered CPU there just to do some easy crunching. And you can connect that to a host to do even heavy lifting. And that’s how I see the architecturally hardware is converging a little bit. Back then was a duct tape, I call it duct tape. Just you have something on a USB cable.

Kenton Williston: Yes.

Ray Lo: USB 3.0 cable. It was a horrible thing to me. Latency was crazy hard. You have so many issues. Powering. So, today you see a lot more condensed into one single element. And then, I see that as one of the next things.

Kenton Williston: Yeah. And I think it’s fair to say that basically any hardware you look at these days, it’s starting to acquire some AI capabilities. Like, the most recently released Intel Atom processors, which you wouldn’t really think of as being super high-performance processors, you have some AI acceleration built into them. So, even at that level, there’s a lot you can do.

Ray Lo: Exactly. That’s the one that went on the satellite.

Kenton Williston: Ah okay.

Ray Lo: There we go. You picked on the right one. Given all these choices and platforms, now even on a space program, people are able to think, “Okay. Now I have one more other power available. What can I do?” Because all they have is a solar panel. But now they can do so much more because with that project… Now, the problem is not just power, right? They have bandwidth. It takes so much time to transmit one image, so every image is so important. But they can crunch couple of images because they have enough power from the sun. So, then they actually process the image, make sure it’s not garbage image, it’s nice, it’s like satellite image, right? When you take picture of a cloud, what do you see? Cloud, right? You want to see houses. You want to see landscape.

Kenton Williston: Yeah.

Ray Lo: Now because of that processing, they’ve saved… I don’t remember the exact number, but that changed the whole dynamic about the whole efficiency there. And that’s I think that’s the innovation that people are thinking now. And just like re-adjusting the problem statement.

Kenton Williston: Yeah. Exactly. I’m glad you said that because that was exactly what I was thinking that it’s not just, “Oh, you can do all these new things” but it’s a matter of you can come at the problem from a totally different angle than you would have before. So, it’s good to rethink your architecture. And very simplistic example of this would just be the way that all this machine learning, deep learning has very often been split up into train it in a big power-hungry data center or cloud or whatever. And then, deploy the inferencing at the edge on something really, really lightweight. Put the right processing, the right smarts, in the right place.

And to your point, all the other things you can do too. What can you do to rethink where the data flows? So, maybe you do processing in a location that previously would have just been a transmitter of data, et cetera.

Ray Lo: Exactly. And people still get confused between the training and deployment. They always think AI must be extremely power hungry. Yes, the training phase because you’re trying to teach the neural network, but once you have the network ready, the deployment, that’s I think we have to really think twice. The deployment is a different problem than the training. And of course, there’s different type of machine learning problems that may require real-time training. But for most of the stuff with detection, like what we’ve talked about, detecting the cloud, once you train it, the neural network will actually be able to detect those very quickly. And then, we will be able to deploy them very differently.

Kenton Williston:  I do think it’s useful to explore where the biggest challenges lie in AI and machine learning. What some of the common pitfalls are and what developers can do to avoid those?

Ray Lo: Yeah. I always find people are too ambitious about AI. That’s how I find that was a pitfall. I’m an engineering background. We have to be realistic about exactly what this can do and what it’s good at. So, I did a challenge about doing image classifications. And I gave it to many candidates. I said, “Okay. Run this code. Put on your own image. And see what it can do.” Even as amazing at 90 something percent, 80% of accuracy, but that 20% of error is hilarious. So, if you think about deploying a tool for use cases, you have to really understand the use case, and align with your expectation on accuracy. Is 80% acceptable? A lot of times it’s a no, right? And it’s amazing, but it’s a no. It’s a big no. And people have to really learn that in an early time before they deploy.

And why is it funny is we did that challenge. People put a Tesla on it. It’s really funny. So, the Tesla is not… So, the Cybertruck is not part of the database. It came up with the answer it’s a jeep plus a beach wagon. I was like, “That’s correct, but I don’t think the marketing team will appreciate that.” So, think about things like that. You have to really learn what you’re doing and make sure they align with your use case.

Kenton Williston: Yeah. Absolutely. There’s some instances I’ve seen some pretty funny examples of AI trying to classify whether what it was looking at was a muffin or a dog. It looks so similar.

Ray Lo: Exactly.

Kenton Williston: And that’s pretty funny. But of course, if you’re doing something like…

Ray Lo: Medical.

Kenton Williston:… when you predict when a very expensive machine is going to fail or anything that has the kind of ethical implications, all of a sudden, it’s much less funny. You need to be very, very thoughtful. And I think that there is some big lessons learned this year about being ethical with AI. I think conversations that really needed to happen.

Ray Lo: In Intel, we actually formed a group just on that topic. I think it’s extremely important to understand what you do, does it hurt people? Does it have any damages? It’s ethical, right? That term is such an important thing because it’s like you have great power… What’s it called? When you do Sudo on a Linux, right? Great power come we think great responsibility. Yes, sounds a little bit old, but it’s happening. So, that’s something I feel we have to all look very carefully into.

Especially with medical. Think about this. You’re doing diagnosis, right? Is that 1% good enough? Is it ethical to say, “I can accept a 1% error”? Is it going to do something harmful with the people? Those kind of have to go through a lot of rigid testing approval and making sure things are right.

Kenton Williston: For sure. For sure. Going back to an earlier point you made, there’s some things that machines can do now far better than humans, but there are definitely times when you really need a human in the loop.

Ray Lo: Mm-hmm.

Kenton Williston: And it’s really even in the training part. So, I just mentioned, for example, monitoring expensive machinery. It would be unwise for a developer who’s not familiar with whatever equipment this is to think that they could just go out and collect some data and interpret it. You really need the human being who’s been operating that machine to help you understand what the data really means.

Ray Lo: Mm-hmm. This is very important because back then we have data biases, and then, create problem down all the way to recognizing… Become racist, becomes manipulative. Bad things can happen to the system when it’s not really carefully reviewed and monitored.

That’s one thing I think we got to be really careful. And then, I think as long as everyone have the good heart, it’ll be okay.

Kenton Williston: Right. Absolutely. So, kind of zooming back out to the big picture. Wanted to again recap what we’ve seen happen in 2020. So far we’ve talked about things that have happened in terms of the advancement of platforms, on the hardware side, and on the development of software, development side. The ways people are coming at problems in different ways. How important this has all been to the pandemic response. Any other big picture trends that you’re keeping your eye on?

Ray Lo: It’s open source. I think that’s one thing we always undertook. It’s the whole OpenVINO effort, all of the TensorFlow effort, all of AI effort, that are open source. So, it’s something that is not very common back then with a lot of the corporate I worked with in the past. So, oftentimes you may have a solution. One off, you have to pay for license fee. Or you don’t even see anything. And there’s no way you can adopt and change with the rest of the community.

So, the open source and community, and then, that’s why I talk about OpenVINO as open. I find is very empowering because I’ve seen a lot of use cases that are done by the community that I’d never seen. Like, for example, OpenCV is our partner. They have their own open community. And then, within the community, they take on both tools. And then, they create new tools. And that’s one thing that’s happening in the next two to three years. We will see those new tools that open source are mature and are getting to the point that will be the new standard. Open standard, open source for AI is the new big thing for me.

Kenton Williston: So, what do you think that will enable? Is it just a matter of increasing the ability to come up with these creative ideas and put things together in a new way? Or is there something more beyond that do you foresee?

Ray Lo: I will see it’s like two or three phases, right? It’s a Linux in the beginning will be like, “Oh, it’s a small community.” But eventually become a standard for all server that we’re running today. And then, become a thing, right? Becomes the gold standard. And I see those will happen in many of those. It will just change the way we approach things. And because of that openness, now advancement is in exponential speed because all those blockers are going. And that’s why I very care and interested in… It’s literally viral. It’s one to two, two to three, two to five. So much faster than before.

Kenton Williston: Yeah. I agree with that because again, thinking about how you want to be focusing on innovative ways to tackle a problem and not the basics of the technology. As more and more gets contributed to this community, and again, as a very simplistic example. Just all of the pre-trained models that are now out there. Boy, that gives you so much of a faster start and makes it so much easier to focus on whatever is unique about what you’re doing.

Ray Lo: That’s correct. Especially with the pre-trained model I think is a big deal because not everyone have the powerful GPU, can train everything from scratch. A lot of people interested in the outcome. Like, the use cases. For example, the BERT, I don’t have the database, I don’t have all those, but I can turn that into a cooking recipe, which I built for demo. And then, now instead of reading the recipe, you can ask questions about what the recipe can do. Like, how many eggs do you need, things like that. And you can run that in real time. And that’s very different because before when I think about that problem, I think, “Oh dear. I got stuck collecting all the recipe in the world. I got to think about a language model. I got to think about who I got to hire. And I didn’t even have a dollar in my bank yet.” So, that’s a huge difference.

Kenton Williston: So, just to make sure I hadn’t missed anything important yet, could you tell me what BERT is? That B-E-R-T.

Ray Lo: It’s a language model that’s published by Google. It stands for bidirectional encoder representation. And what this will do is… So, back then when we did machine learning, there’s many ways to do this. This one is published by Google that has one feature that we all love that it’s called fine tuning. What it means by fine tuning is when you do natural language processing to understand what the language means and all that, the effort oftentimes it’s very much like one task it can do. Like, find out where noun. Where’s the verb. Things like that. But this one, you can fine tune to do things specifically like what I talked about question and answering. So, it will be able to do that, but without retraining the entire model.

So, you can think of it as like a new processing model that Google came up with a lot of researchers together. And it’s something, really, I would say popular now because it’s a new gold standard. Because of that now you do Google search and all that, you get much better accuracy. So, you wonder why. “Why is this so good now?” It’s actually behind the scenes one of the models they use.

Kenton Williston: Got it. Makes sense. So, before we go, I want to get your thoughts on the coming year. So, this is actually going to publish in January. And this will go live in January. So, we’ll take a little risk here and see by the time folks are listening to this, if any of our predictions are maybe even coming to pass. But what are some of the main trends you foresee happening in this domain in 2021?

Ray Lo: Mm-hmm. So, I got to summarize, I think NLP will be a big thing in the next couple of years. Those will change the way we interact with devices. We saw it in the early time, but now time where you get call center, things are happening.

A deployment of IoT will kick in very soon. You’ll see a lot of… all the warehouse, all those automations, you’ll see machine learning in every bit of our industry.

And last and not least, I think the growing trend of augmented reality and virtual reality. We know we talk about a lot. It seems like it’s the hype back then, but today, when I look at the technology maturity, the integration of AI and AR and VR will happen because I crave a good content all the time from virtual reality and augmented reality headset. And I think once we put those elements we talk about, recognizing things, how it can create relevant things about your life, your surroundings, it’ll be a killer app for many things we’re doing today.

Kenton Williston: Nice. Well, with that, let me just say thank you so much for joining us today. Really enjoyed this conversation.

Ray Lo: Thank you. And I have as well.

Kenton Williston: And thanks to our listeners for joining us. If you enjoyed this podcast, check out insight.tech for more innovative IoT ideas. This has been the IoT Chat podcast. We’ll be back next time with more ideas from industry leaders at the forefront of IoT design.