Medical Panel PCs Fulfill Hospitals’ Rigorous Requirements

From MRIs and CT scanners to electrocardiograms, oximeters, and blood pressure monitors, today’s medical machines collect an enormous amount of data about patient health. But getting that information into the hands of hospital doctors and nurses who need it is often a struggle.

One problem is that medical machines and devices are made by a wide range of manufacturers. As a result, the information they produce is scattered in incompatible formats across multiple data bases and IT systems, making it hard for doctors and nurses to assemble a coherent picture of patient health.

Another issue is hospitals’ restrictive hardware requirements. Nursing stations, operating rooms, and intensive care units—as well as the labs and pharmacies they connect with—must use computers that meet rigorous hygienic standards. And to successfully transmit vital patient data, the computers must also be very fast, reliable, easy to use, and secure.

Modern medical Panel PCs are designed to meet these challenges. They support data integration from a diverse range of medical equipment via an all-in-one, compact computer that meets hospital sanitary, usability, security, and reliability requirements. And they have the computing power to run the sophisticated medical software that provides caregivers the data they need not only to respond to emergencies but to consistently measure patient progress.

Unifying Patient Monitoring Solutions

Because healthcare technology continuously advances, most hospitals contain a wide range of medical equipment makes and models.

“Hospitals modernize their technology in phases, so they have a very heterogeneous hardware and software environment. Communication protocols are a big pain point,” says Guenter Deisenhofer, Product Manager at Kontron AG, a manufacturer of IoT equipment.

Machines that don’t intercommunicate erode procedural efficiency and make treatment decisions difficult—a problem Deisenhofer recently experienced firsthand after taking his son to a local hospital. First, an intake worker measured the boy’s heart rate, blood pressure, and oxygen level, recording the information on a slip of paper. His son was later seen by a doctor, who took his vital measurements again, writing them on a different paper before sending him off for X-rays—where the process was repeated yet again.

“At the end of the day, there were probably five pieces of paper. They had not continuously monitored his condition and nobody had an overview of it,” Deisenhofer says.

Fortunately, the situation turned out not to be serious. But with vital information arriving from different devices at different times—whether it’s recorded on paper or encoded in incompatible software programs—doctors can never be sure of what they might miss.

Medical edge computers, such as Kontron’s MediClient Panel PC, close the information gap, using a standard protocol to integrate data from machines, wearables, and patient health records. The Panel PC satisfies hospitals’ strict sanitary regulations and is readily accessible to caregivers, even if they’re wearing gloves. It conveys information from patient monitoring machines through wired or wireless connections to hospital communication hubs, such as nursing stations. High-performance Intel® processors enable the monitoring machines’ software to run near-real time analytics on incoming results, helping doctors and nurses see patterns and spot anomalies that may suggest a diagnosis, or point to the need for specific tests.

“With continuous monitoring data available, doctors aren’t just reacting to an emergency. They can see, for example, if the heart rate is dropping and recovering over time. It helps them make better diagnostic and treatment decisions,” Deisenhofer says.

Monitoring can continue after patients are released from the hospital, with wearable devices seamlessly transmitting their information to the MediClient, where it can be integrated with patients’ previous records.

With hospitals increasingly relying on advanced #medical equipment, machine #manufacturers must be vigilant in keeping them updated, both to enable new #IoT functions and to guard against the latest cyberthreats. @Kontron via @insightdottech

Improving Machine Life Cycle Management with Medical Panel PCs

With hospitals increasingly relying on advanced medical equipment, machine manufacturers must be vigilant in keeping them updated, both to enable new IoT functions and to guard against the latest cyberthreats. Making these changes often involves not only software but firmware and sometimes hardware. That means devices like the MediClient PC must also be updated to continue providing hospitals the machines’ vital data.

As technology innovation accelerates, machines that were built to last 10 or 15 years often require several major updates. “Machine life cycle is getting more and more difficult to manage,” Deisenhofer says.

Kontron works closely with medical equipment manufacturers to keep up with planned changes and is often included in early product planning cycles. Because every hardware change requires extensive testing and recertification, close communication saves time. Manufacturers can get their products recertified in one go, instead of having to do so again after making modifications. Kontron also does third-party software installation and electrical testing of manufacturers’ equipment to help them resolve potential problems before release.

Collaboration with Kontron allows manufacturers to deliver upgrades to hospitals sooner—and hospitals can integrate the new capabilities into their medical systems as soon as they are available.

Bringing New Capabilities to the Edge

Working together, OEMs and Panel PC makers can extend the value of monitoring machines as technology advances. The more data the machines collect, the more system builders can improve their AI software, reducing false alarms and pinpointing problems. “For example, you could use AI to create warning scores for recognizing conditions well in advance of a critical situation,” Deisenhofer says.

And the processing speed of edge Panel PCs will improve, helping caregivers respond to identified health threats sooner.

Benefits like these suggest a bright future for patient monitoring machines and the medical Panel PCs that connect their data to doctors and nurses. As Deisenhofer says, “Our device cannot make decisions. But by bringing all the data together at the edge, it can help doctors make the right decisions.”

Edited by Georganne Benesch, Editorial Director for insight.tech.

Staffing AI in the OR: With Caresyntax

Did you know that edge AI can provide better outcomes for surgical operations? A number of factors can influence what happens on the operating table, but with edge AI physicians can get real-time data and information about a procedure at their fingertips, minimizing patient risks.

In this podcast, we explore the ways edge AI can assist physicians—both inside and outside the operating room—examining the benefits as well as the challenges, and what to expect from AI in the OR in the future.

Listen Here

[Podcast Player]

Apple Podcasts      Spotify      Google Podcasts      Amazon Music

Our Guest: Caresyntax

Our guest this episode is Dennis Kogan, Founder and CEO of Caresyntax, a digital surgery platform provider. Dennis founded Caresyntax in 2013, where he works to bring AI-powered solutions to the healthcare space with the goal of improving patient care and outcomes.

Podcast Topics

Dennis answers our questions about:

  • (2:04) Physicians’ and patients’ expectations
  • (4:36) Becoming comfortable with new OR technologies
  • (7:50) Different ways AI assists in surgical procedures
  • (11:31) The importance of AI combined with human expertise and data
  • (16:51) Implementing the latest and greatest technologies and innovations
  • (19:14) AI-assisted surgical operations in action
  • (22:54) The different types of partnerships that make AI in the OR possible

Related Content

To learn more about AI-assisted surgical technology, follow Caresyntax at @caresyntax and on LinkedIn.

Transcript

Christina Cardoza: Hello, and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech. And today, we’re going to be talking about the use of AI in the OR with Dennis Kogan from Caresyntax. But, as always before we jump into the conversation, let’s get to know our guest. Hey, Dennis, thanks for joining us.

Dennis Kogan: Hi, Christina. Thanks for having me.

Christina Cardoza: What can you tell us about yourself and Caresyntax before we jump into the topic?

Dennis Kogan: Well, I am a tech entrepreneur first and foremost. You know, I’ve started this company, together with my partner, Bjorn von Siemens, about 10 years ago. We had a different healthcare company right before that. I am not a physician; I’m actually more of a technology guy. So, I graduated from Carnegie Mellon University with an information systems background and then ended up working in data science consulting, doing quite a bunch of healthcare.

But my link to surgery is actually more personal. My father is a surgeon, my grandfather is a surgeon, and my great-grandfather is a surgeon. So it’s kind of a dynasty of urologists that stopped with me, but I like to think that I’m contributing to surgery in a different way, perhaps more scalable than if I were a surgeon.

Christina Cardoza: Yeah, absolutely. So you found sort of a back-end way to get into the family business, so to speak.

Dennis Kogan: Isn’t that interesting how it happens?

Christina Cardoza: Yeah, absolutely. That’s really interesting that you’ve been in the healthcare space for so long and you have so many—a deep history, family history, in the healthcare space, because I’m sure you’ve seen this space has just evolved rapidly, especially over the last couple of years. And that’s sort of where I wanted to start off the conversation.

We’ve had so many technological advancements and innovations in this space. What have you seen over the last couple of years, and how has this advancement, how has it changed expectations that surgeons and physicians and patients have coming into quality healthcare?

Dennis Kogan: Yeah. Well, just kind of piggybacking on the prior question, when I was at CMU, at Carnegie Mellon, I dealt a bit with other innovations in other industries, like sports analytics and security and military. And I was at the time already talking to my dad, who, you know, was a surgeon. And I was telling him, “Hey, you know that athletes get this and this for performance management and situational awareness and get analytics? And every decision could be supported.” And he told me, “We have nothing like this. Yes, we have very interesting and very important medical devices, and we’re continuously getting clinical innovation in our hands, but there isn’t really a lot of data usage and decision-making support.”

And I think that hasn’t changed that much up until a few years ago, to your point. I think we had a ton of innovation around medical devices. You’re probably familiar with the da Vinci robot; Intuitive Surgical brought robotics into the space. But, at the end of the day, it’s still helping the surgeon operate with his or her hands. The advancements that we are seeing, and obviously we’re part of, is actually enabling the surgical teams to not only have better tools in their hands but also have better decision-support mechanisms, right?

So, with the volume of patients I think there is more and more expectation that the surgeon cannot be just thinking herself or himself about the risks of the procedure, right? They do want support. They do want more information to stratify risks more. And doing it in their heads, as it happened before, is probably no longer acceptable, given how much background technology there is in healthcare and outside healthcare.

Christina Cardoza: Absolutely. And to your point, I could see a little bit why the surgical field may be a little bit slower to adopt some of these technologies or to advance in this space, given that you’re operating on people and it’s such a mission-critical surgery or application. And some people, consumers, may not feel comfortable having this technology being used in the operating room.

But now we’ve seen just all these advancements, how it can help, and it’s really providing more, like you said, it’s mitigating some of the risks that we did have before. So how do you see now the surgical space adding some of this technology to the OR in a safe way to ensure that it’s accurate, the physicians are comfortable with it, but then also the patients are comfortable with it?

Dennis Kogan: Yeah. I mean, I think patients, I would say patients are probably—relative to other types of therapies or chronic diseases—are less aware of what’s happening in the OR, naturally. You’re under anesthesia as a patient. But what patients really do want, they want to understand how likely are they to have a good outcome? They want to understand data about their surgical teams so that they can really weigh their choices, right? So I think they expect and they probably are surprised, would be surprised, to know that not as much integrated decision-making support is available to surgical teams that will be operating on them.

And so I think the challenge has been, to your point, is that surgery is a real-time intervention, right? And so in order to integrate new technology and AI, any automation to be able to jog decisions. You really have to get a lot of new infrastructure in that would be able to pool data and create the necessary backbone to actually make the software and AI run in that real-time setting. And there is a pretty high threshold of quality and operational effectiveness, right? So, for example, anything that is used in the operating room should have almost no lag, right? So these should be real-time, near real-time decision-support mechanisms. And that, by itself is a higher hurdle than a lot of other information technology that has been used in healthcare.

So I think over the last decade, I would say, probably, since we started, I think the biggest change that’s happened in the operating room is actually bringing live all these different disparate devices into one system that is able to, A: drive the safer and more automated workflow in the operating room, but also be able to capture data and receive data so that you can start building even more advanced innovation around AI to really bring even more decision-making support to the surgical teams.

Christina Cardoza: Yeah, and that’s always seemed to be a big hurdle in the healthcare space, is having these silos of data—if patients have different data, depending on the devices they’re using or the doctors they’re seeing or the space they’re in. So I can see why it’s been a little bit difficult on the physician side to be able to get access to all this data and really make those real-time decisions. And of course with automation and AI, like you mentioned, it’s all bringing it in one place so that they can react quicker, they can make better decisions, they can have that top-of-mind rather than information getting lost.

But when we’re talking about AI in the operating room and technology advancements, I think a lot of people automatically go to what we were talking about earlier, about the robotics and robot arms and robots being used to do the surgery. In this context we’re not necessarily talking about AI-guided surgery, right? It’s more AI-assisted surgery, where AI is providing the up-to-date and accurate information for the physicians to actually help surgery and improve patient outcome. Is that correct?

Dennis Kogan: Yeah, no, that’s 100% correct. I mean, I think there isn’t a very high probability of surgeons and surgical teams being replaced by technologies for a long, long, long time. The environment is extremely dynamic. And it’s not only quantifiable activities and techniques; it’s also communication and teamwork, right? I mean, it’s actually a team sport. So, part of the outcome depends on risk stratification and how well a surgeon does a certain maneuver. But part of it is how well does a surgeon communicate with the nursing staff and anesthesiologist, and how do they adapt to changing clinical picture during the procedure? It’s so complex that it’s almost impossible to foresee how this could be replaced by artificial intelligence in the foreseeable future.

But because of that same dynamism, AI has a lot to give in terms of bringing the right information and options to the fingertips of physicians in this dynamic setting, right? I mean, a procedure that’s lasting hours or, say, an hour, and a physician team that may be operating from early morning into late evening with very different types of patients. You could be having a healthy 25-year old female or a very sick 85-year-old male, right? And you have to be able to adjust a lot of inputs and a lot of decisions throughout the procedure dynamically.

And that cognitive overload often does cause mistakes or suboptimal decisions. So at the end of the day there is—we call this variability. The change, the risk, in surgery is, unfortunately, there. There’s probably one out of seven cases has some sort of significant complication, so over 15%. And so proactive risk management through situational awareness, through certain automation, is what we’re talking about. It’s about reducing and removing variability that was unwarranted, that’s driven by the cognitive overload and changing clinical picture.

And so I think the best use cases that we see right now for AI are really proactively managing risk by showcasing specific information about that given patient, about that procedure—before the procedure, and in real time, and after the procedure—to be able to guide the entire pathway, and the outcome to be better than it would be without that support.

Christina Cardoza: I love all the patient examples that you provided, because it really just showcases not only do physicians have a number of different things going in their heads with different patients and just different operations that they have to do throughout the day, but not every patient is a clear-cut case. Not every surgery is a one-surgery-fits-all. And there’s unforeseen complications and decisions you have to make while you are performing the surgery.

So I can see how AI being brought in really provides that real-time information that allows them to react fast and to give that best possible outcome, surgical outcome, for those patients. We were talking a little bit about the infrastructure that is necessary for implementing AI into the OR. So I’m wondering if you can tell us a little bit more about how we can actually get this information—we can get AI into the OR and have that combined with the patient data and the human expertise to really transform the surgery and the outcome.

Dennis Kogan: Yeah. And the operating room is obviously the place where the actual therapy happens, and it’s very important, but because a surgery is actually a treatment to a disease, everything that happens before and after is also extremely important. So actually the best-integrated kind of platforms allow for the connectivity between the operating room and the pre- and post-operative space and time and activities, right?

Because decisions you make right before the patient enters the operating room are extremely important—about preparing the right tools, the right medications, the right people at the table. And then, of course, knowing with what level of risk that patient is exiting the OR may change the protocol of how that patient is going to be taken care of. Maybe that patient can go home; maybe that patient needs to be in ICU; maybe they need an extra dose of antibiotics because of extra bleeding, right?

So, first and foremost, truly integrated surgical-decision support touches on all points of the perioperative cycle, right? You have to connect clinical and operational information that’s coming directly from the workflow to really get a sense for how you can reduce this variability. And so that means that you actually have to connect a lot of different systems, right? We’re talking about—and we did talk about—the inside-the-OR situation, where you can connect medical devices and video cameras. The real, unstructured data, just like pilots or athletes use it to really understand, in more granular detail, how exactly that intervention is done.

But it also includes the classic, usual suspects. The electronic medical record, because it has a trove of data about the patient and his or her predispositions. The ERP, the operational data to understand the length of certain things, because that also can lead to important insights. So, the truth of the matter is that in order to get the best, smartest insights, you have to have a full perioperative clinical and operational record, with the crown jewel being the intraoperative space, because that is the most mission-critical piece where things can really go wrong.

And so because it’s real time and because it’s mission critical, it has a specific, added level of, let’s say, sophistication that’s needed. And, of course, it’s not, in technical terms, a cloud-friendly territory, right? Like, we’re used to a lot of innovation being rolled out very fast using cloud technologies, and you can open your phone and you can use it. And even in healthcare that’s been the case, right? With desktop usage—maybe not cellphones, up until recently.

In the OR, it’s all on the edge, right? You cannot rely on two-second upload and download from a cloud. So this edge computing, the Internet of Things–technology toolkit, is extremely important. And, again, it’s very similar to mission-critical segments outside healthcare: you have to have very high level of service.

And at the same time it has to be very robust and attractive from the perspective of deployment and cost solution, right? Because at the end of the day everything that is overly expensive or unwieldy—another huge machine being rolled in into already a very packed operating room—is just not, doesn’t work, right? You’re expected to have ergonomics in the OR, you’re expected to have plug-and-play capabilities, but you’re expected to have a high threshold of real-time, high-integrity flow of information.

So it took us at Caresyntax, for example, with the help of a few technology partners, years to develop this platform in a way that achieves these parameters that I just mentioned. And early adopters, of course, learned with us, seven, eight years ago. But at this stage we’ve achieved that level of quality and efficiency and are able to integrate operating rooms and add the context around it to create this proactive management of risk.

And so I know it’s possible. I think it’s still sort of in the beginning in a way, right? I think the next decade will probably have every OR being equipped with these kinds of systems. And in 10 years physicians will be wondering how they were doing work without it.

Christina Cardoza: So, given that most operating spaces or most healthcare organizations, they have these devices that don’t really talk to each other or play nicely with each other, what type of investment does a hospital or physician need to make to make everything plug and play, to be integrated, more interoperable? And how can they make those investments with Caresyntax, ensuring that they are future-proofing any investments that they do make? That they can add more capabilities, take away capabilities if they need to. They can take advantage of the latest and greatest technology and innovations coming out without making these technological investments that are going to put them in a vendor lock-in or to stagnate their innovation.

Dennis Kogan: Yeah, that’s a great question. I mean, every industry has gone through a cycle of having first a few vendors create kind of a walled garden, and then gradually the users expecting more and more flexibility and open nature to be able to add value and add new applications. And I think surgery and healthcare need to undergo the same change.

I mean, I think the medical device world, for good reasons as well, has a lot of proprietary intellectual property. And so a lot of device owners and vendors are naturally quite protective of the ecosystem they create around their therapy and their device. And so historically that’s been a dominant mindset also for physicians, is to a certain degree align with specific vendors and think of the operating room through a prism of a device.

So the first investment that needs to be made is to reinvent and recalibrate the mindset towards the operating room being not an extension of a leading device platform, but actually a care setting that belongs to that horizontal process of achieving outcome. And that having infrastructure that is vendor neutral, having infrastructure that is open to capturing any kind of data and feeding any type of algorithms into it, regardless of what vendor you use, is of course a significant change in historical patterns of acquiring such platforms.

But from a perspective of capabilities, obviously flexibility, but also total cost of ownership, having a neutral platform that is really there to add value, regardless of what inputs it’s connected to, vendor neutrality doesn’t mean any compromise in quality and safety of these platforms.

Christina Cardoza: So I know this is still a little bit early days, and more hospitals and physicians are looking to add this in the OR, so I’m curious if you have any customer examples already or any use cases of how Caresyntax really came in and helped an OR setting that you’re able to share with us or talk a little bit about.

Dennis Kogan: Yeah, no, I mean, I think there are multiple examples. I think the ones that I’m really excited about are the ones that we’re able to achieve things at scale. So we have an example of a medical-insurance company, actually, that insures surgical risk in terms of medical malpractice and safety aspects, partner with us to create proactive risk-management solutions that are using some of the understanding of safety that they have, and processes and governance, with our technology, creating a proactive risk-management offering that, both in the OR and in terms of governance, is fostering this culture of improvement and safety using the data feeds from our platform.

And this partnership has been very successful in Europe—actually in multiple countries, in hundreds of operating rooms. Basically convincing the hospitals that the combination of technology and good governance of how to achieve a high-reliability organization—if you combine this into one solution set—it’s in their best interest. And we’ve had a lot of positive traction and a lot of indications of actual improvement in technical skills of physicians, improvement in outcomes, improvement in managing surgical-site infections.

So I think that’s kind of a more complex partnership example, but of course there are numerous end users that have seen the same, right? We’ve published several studies with surgeons—I think from 20 different countries—about how usage of intraoperative data, including video, can improve technical skills of physicians, which is one of the biggest determinants of success, right? At the end of the day, you may have a very sophisticated robotic assistant or a Gamma Knife, but at the end of the day it’s still the skill set many times that decides whether it’s an excellent or an average outcome.

And so we’ve been able to show that using these advanced platforms in the OR can lift that performance level. And that’s not only surgeons; it’s also other physicians and clinical collaborators. So, for example, nursing, right? We’re increasingly starting to deploy almost like interactive, step-by-step navigation guides in the OR.

After the pandemic a lot of folks entered the workforce without maybe as much training as they would have beforehand. So there are a lot of new, for example, nurses who very quickly need to catch up in the environment, where there’s a lot of new volume because so many surgeries came back after the pandemic. So being able to quickly ramp up and actually get the step-by-step and move-by-move support in the right moment of the procedure is extremely helpful for somebody who is still lacking confidence and the experience to know what to do next, right?

Christina Cardoza: You mentioned a lot of different technologies that go into this, so I imagine there’s a technological partnership that goes into this. You know, I should mention that the IoT Chat and insight.tech as a whole, we are sponsored by Intel. But I can see you’re using the edge, you’re using the AI. So I can see there’s probably some Intel hardware and Intel software that goes into this, and there’s a lot of different moving pieces to make this happen. So I’m curious, what’s the value of working with partners like Intel to make this a reality, to bring this to ORs in a safe, secure, and accurate way?

Dennis Kogan: Yeah, no, and even though it’s sponsored, I can kind of rave about some of the support we’ve been getting here, because at the end of the day, being a surgery specialist, we have a very good view for what the end application and use case should be, but we don’t have as much experience building that infrastructure, and we don’t have the benchmarks and comparables from other use cases that may be similar in terms of the rigor and in terms of the actual architecture.

And so Intel is indeed an important partner that helped us and is helping us to meet those criteria that I described, right? I mean, having an integrated smart-surgery platform that is sort of plug and play, that is very smart and not very heavy in terms of hardware content, something that is able to generate information but also have the capability and the bandwidth to receive algorithm and actually produce AI and showcase it in real time—it’s a pretty sophisticated set of requirements.

And Intel has been one of the partners who have really plugged in with us, almost inside our team, to make this happen, right? So being able to design the architecture, find the right components, utilize some of their components that they developed, like OpenVINO, which allows for this AI penetration and usage—all of these were very important. I think without a partner like Intel we would’ve been, at the least, much slower, looking for every piece ourselves, probably making more mistakes.

In the end, I think, if you really think about the speed at which we’ve created some new, very key, new-generation components together, I think it’s probably half or even less than if we would have tried to embark on this ourselves. But we’re excited to be working with them. And I think there’s going to be continued innovation.

And alongside Intel, of course, we also work with cloud-solution providers—AWS and Google Cloud. Because, at the end of the day, you kind of have to have an edge-to-cloud transition. As I mentioned, it’s a preoperative, intraoperative, and postoperative space. So if you really wanted to have an integrated performance or decision-support system, you continuously have to go to the edge and back to the cloud and make the information interchangeable.

And so it’s been very rewarding as well to kind of build with our partners, because actually they all collaborate in between themselves—Intel and Google, Intel and AWS. So there’s a lot of big-technology-company support here to enable this, and then some of the secret sauce and the knowledge of what really makes it different for the users that we bring to the table.

Christina Cardoza: Yeah, it’s great to see all the different partnerships happening, because it’s not that you have to solve this alone, especially in a sensitive environment, like the surgical environment that you’re working in. It’s good to see that you’re using the expertise and the knowledge that you have as a company, but then also leveraging support and expertise and technology from other companies as well, to really make this as high quality as possible, as it can be.

Dennis Kogan: Of course the pandemic has been an impediment to any innovation, given the distraction on real, existential/day-to-day issues, but that’s subsided. And I think everybody’s really looking at surgery and saying, “Okay, well, it’s very important. You cannot prevent surgery very often. And yet it’s still not as safe as it could be.” It’s not as safe as flying; it’s not as safe, even as some other medical procedures. It’s time to improve it. It’s time to let everybody change certain ways of doing things.

And to your point about partnership, it takes an ecosystem of players to achieve that. And I think we’re shaping that, and Intel is one of the partners we are extremely grateful to have alongside us.

Christina Cardoza: Absolutely. I can’t wait to see how else this space continues to grow, and where else that partnership with Intel and other partners you’re working with will bring these technologies, make this more mainstream, and see that adoption over the next couple of years. And like you said, physicians in the future are going to say, “I can’t believe that we were working without this.” And I think patients are even going to say, “I can’t believe I had gotten surgery back in the day without all of this information at the physician’s fingertips.” So it’s great to see all these transformations and innovations happening.

Unfortunately we are running out of time, but before we go, Dennis, this has been a wealth of knowledge in this conversation. Is there anything you want to leave our listeners with? Any final thoughts or key takeaways?

Dennis Kogan: Well, I mean, I think I just want the ecosystem to kind of acknowledge a couple of things, right? I mean, I very often see that folks—and I think there are reasons for that—think of surgery as something that’s been figured out, something that’s reached maturity and doesn’t require innovation. It doesn’t give me any pleasure to say that this is not the case. I think 15%-complication rates, mortality rates, going into a procedure not knowing how it’s going to turn out, is still a daily occurrence.

And yet, it’s a stark size that surgery has in terms of a share of volume in treatments, right? Next to pharmaceutical therapies, surgical therapies are the second-most-used way of correcting a disease, right? If you think about spend, I think it’s over 20%, 30% of all of healthcare spend in the US is connected to surgery. And so if you think about the variability and the risk that’s still in the system, and you also even convert this to spend, this is a huge problem that has clinical implication; it has cost implications for our society, for the government who’s insuring a lot of people. So it hasn’t been fully solved, right? It hasn’t been fully solved, and it has opportunity and room to get to the same place as we have gone with aviation, you know? I don’t think you and I would accept getting on the plane with a 15% chance of something going wrong in that flight.

Christina Cardoza: No. Absolutely not.

Dennis Kogan: I think we should have that feeling going into surgery: that everything is going to be okay, backed by real statistics. And in order to change the statistics, you have to let all boats rise. And in order to do that, you have to deploy these kinds of solutions into your workflow. And—together with the ecosystem of providers, insurers, technology vendors—you really can make surgery safer and smarter, and it will have broad impact on patient health, millions of cases. And it’ll have broad impact on cost as well.

So I think my message is, please think about surgery as part of innovation, where faster, better cures to certain diseases can be achieved. And, ample room for improvement in every provider organization, as long as the mindset is there. And we’re always happy to be there together with our partners to help move forward towards that objective.

Christina Cardoza: When you think about all the risks and the complications that can happen in surgery, and then all the benefits that patients and physicians get by having this data, having AI assistance in the operating room and at their fingertips, it seems like a no-brainer to do some of this stuff. So I’m excited to see how else this space moves forward.

I invite all our listeners to visit the Caresyntax website to see how else you guys are going to continue to innovate, or how they can partner with you to add some of these technologies into their operating room. And I just want to thank you again for the insightful conversation, Dennis. Thank you for joining us on the IoT Chat. And thanks to our listeners for listening and tuning in today. Until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel® trademark information.

This transcript was edited by Erin Noble, copy editor.

Smarter, Faster, Connected with 5G and Intelligent Networks

Recently on the IoT Chat, we hosted Martin Garner and Bola Rotibi from CCS Insight to discuss the company’s annual IoT predictions report, specifically looking at the edge and AI spaces. But there was so much about 5G and networks in the report and in that conversation. So this time we decided to focus specifically on the rise of 5G and those intelligent networks with Ian Fogg, Director of Networks at CCS Insight.

Pretty much all industries around the world need to connect to the internet these days. They want fast, reliable, real-time information, and that requires the network to be fast, too. More and more intelligent devices connect to the network as well, and the grid needs to handle all that connectivity. At the same time, industries and companies try to meet sustainability goals. Could 5G—even 6G—and AI actually help achieve those goals? Let’s find out (Video 1).

Video 1. CCS Insight’s Ian Fogg discusses recent evolutions in the 5G and IoT network space and what’s still to come in 2024 and beyond. (Source: insight.tech)

How has the landscape for 5G and network changed over the past few years?

5G first launched back in 2019, in a really very early version of the standard. What we’ve seen more recently is tremendous enhancement in what it is capable of. One of the things we’ve seen it used for recently, for example, is private networks. 5G was deployed in 45% of the private networks we saw announced in 2023.

We’re also seeing non-terrestrial networks, which are part of the upcoming Release 17 of the 5G standard. By 2027 we expect that 15% of smartphone users will have satellite-enabled devices. Now, what does that mean for IoT? Well, what has often happened in the past is that the consumer space has driven innovation that then gets reused for other things because there’s a commonality. Once that satellite capability is built, the satellite players can choose to support more than one type of customer.

Another thing we’ve seen happen in the past year is the announcement of the OpenAPI Initiative, which is something the GSMA—the operators organization—is very big on. And that includes network APIs, too, that manage network quality and other network-type settings.

What is the next step for 5G in these IoT networks?

One of the predictions we have is that by 2025 a digital marketplace for app-based network functionality will offer more than a hundred versions of network capabilities and APIs. We’ve seen some small initiatives already, but we expect it to expand tremendously over the next 18 months or so.

We’re also expecting that hybrid private-public 5G will emerge as the dominant option for private networks by 2030. A standalone private network uses a dedicated network just for that offering. But a hybrid solution uses some of the new capabilities of standalone 5G and network slicing to provide a quality of service that is different from what other people using the wider cellular 5G network are getting. That bridges the gap between locations where an enterprise may have a dedicated private network but wants people or devices to move between dedicated sites—a hybrid solution. It’ll happen gradually over time, but that’s something we see becoming very dominant by 2030.

How can 5G and the network edge keep up with the demand of IoT?

I think this hybrid capability is particularly important here, because it does bridge that gap between a dedicated private network, which is maybe in a port or in a manufacturing facility or some other limited location, and the need for a high quality of service across a wide range of areas.

One of the other things we see as important, and increasingly so, will be using AI in a whole range of areas and types of products in different parts of the network. We think that AI will enable 5G networks to significantly improve their availability, perhaps even to move beyond five-nines availability, by managing the traffic patterns better and by making sure that the network is offering a good enough quality of service as well as managing outages or downtime issues.

Talk more about how AI can be used to enhance 5G and IoT networks.

The network guys are using AI on all kinds of areas. For example, they’re using it to improve RAN management. As we’ve gone from original 4G to 4G Advanced or LT Advanced, and then on to 5G, and then on to Release 17 of 5G, and onward, the complexity of the RAN has gotten much greater, and there are more settings that need to be managed. The interaction between the base stations, between different frequency bands, is much more complex, and AI is a key way of enabling that ongoing RAN management to improve the coverage and the performance.

We see AI being very important in the Open RAN rollout, too. Historically, service providers have bought a base station from a network vendor, and everything is basically integrated and included from that same vendor. The concept of Open RAN is that there are interfaces within the base station so that a service provider can mix and match different suppliers. And we think the complexity there is something that AI can help improve upon.

Green issues are another thing we think is interesting and important: hitting carbon targets, managing energy costs on the network. If you’re a service provider, you want to drop energy usage while maintaining the network experience that users need. So, how far can you cut back network resources and still offer that? RAN optimization is a place that a machine learning tool can help. And one of our predictions is that by 2025 a combination of intelligent radio access network technology, automation, and AI-driven power-down techniques will enable at least three leading operators to bring forward their carbon-neutral targets by several years.

As we’ve gone from original #4G to 4G Advanced or LT Advanced, and then on to #5G, and then on to Release 17 of 5G, and onward, the complexity of the #RAN has gotten much greater. @CCSInsight via @insightdottech

How can 5G and IoT network capabilities help industries reach sustainability goals?

One of the sustainability angles here is around smart grid, because a lot of people are looking to solar power and wind power as cost-effective and versatile ways of generating green electricity. The challenge is that solar and wind power generation are not always predictable: They depend on cloud cover, on time of year, and on what the weather systems are doing at the time. And there can be times when there’s too much power being generated and you want to encourage end users to consume some of that electricity. There may be other times when you want users to drop their energy consumption because it’s too expensive or there isn’t enough green power being generated.

So we think an increased use of solar and wind power necessitates smart grid technology to manage the supply and demand, and we expect that smart grid technology will become widely adopted in most advanced economies from 2028, if not before. We can see signs of it happening even now.

How can 5G help the grid become both smarter and more sustainable at the same time?

If you take EV charging as an example, you can see a range of ways that network technology comes in to it. Many EVs have cellular capability, so the user can remotely set the power-saving modes and tell it where and when to schedule the charge. Many EV chargers in the home have a similar remote control, on Wi-Fi or something else. But then you also have smart meters in the home that typically have cellular connectivity. The power company can then monitor how much power is being used and charge people accordingly, in some cases in very granular ways.

So you can see it in in the EV connectivity, in the EV charger, in the smart meter—three different places just involved in the end user-EV charging process where you can see network technology becoming very important.

How much longer will 5G continue to be dominant?

Actually, even 4G is still important in the 5G era. In that EV scenario I mentioned, most EVs have a 4G cellular radio. You have a smart meter—it might even have a 2G radio in it. These network eras do tend to overlap. So even when 6G arrives, 5G will continue to be important. But there is absolutely work happening on 6G; we can already see 6G spectrum discussions being advanced.

There’s also work happening on the use cases for 6G. One of these is using the cellular network to sense what’s happening in a particular location. It could be sensing how much traffic there is on the roads, or sensing people walking down the pavement. You can see examples of that happening now: Some of the Wi-Fi access points can sense whether people are at home, and that’s being used as a crude alarm system.

But part of the thinking around the 6G network is that it would have a wider capability—the ability to sense things happening across large areas, across cities. And then of course that means you could draw analytics from that information and take actions on the back of it.

In terms of timescale, if you are a company looking to deploy something today, 6G isn’t probably relevant. If your product roadmap is much longer—towards 2029, 2030, or onward—then 6G should be something that’s in that roadmap. If you are a network vendor, then your R&D labs are fast and furiously working on 6G things at this moment, and you are much more advanced in your 6G thinking.

Related Content

To learn more about 5G and IoT network predictions, read the report IoT-Related Predictions for 2024 and Beyond and listen to The Rise of Intelligent Networks with 5G and IoT and Top IoT and Edge AI Predictions for 2024: With CCS Insight. For the latest innovations from CCS Insight, follow them on Twitter at @ccsinsight and on LinkedIn.

 

This article was edited by Erin Noble, copy editor.

Generative AI Solutions: From Hype to Reality

Last year was a breakout year for generative AI (GenAI), with businesses in all industries interested in the technology and its advanced capabilities. As we head deeper into 2024, its evolution is expected only to grow.

Organizations have learned over the past year that while general-purpose models are a good place to start, to really capitalize on GenAI solutions, they need more task-specific, custom AI models. And as GenAI solutions mature into production, these custom models will be key to hitting performance, accuracy, and cost targets for a particular deployment.

Healthcare offers a great example of the benefits of these approaches: A general-purpose model might struggle with medical terminology, and that could put patient data at risk. Instead, businesses can create a custom GenAI model that can pinpoint a specific healthcare discipline, anonymize it sufficiently, optimize it for deployment, and so on.

There are several ways to create these custom AI models to solve all kinds of problems, such as fine-tuning a large language model (LLM) or creating a small specific model (SSM). Whatever the approach, there is a growing ecosystem of tools, platforms, and services to streamline the effort.

“When these models become more critical to the business, we’re also seeing companies want to take ownership and to control their own destiny,” says Teresa Tung, Cloud First Chief Technologist at Accenture, a global management consulting and technology company.

Tools for Taking GenAI to Market

To help businesses get a jump-start on AI model development, there are solutions like the AI Playground—which came out of a collaboration between meldCX, the University of Southern Australia, and Intel—that gamifies the AI developer learning experience and simplifies getting started with model creation.

On the other end of the spectrum is the Anyscale Platform, which adds enterprise-ready management to the open-source Ray project. The Ray project is a framework for scaling and productionizing AI workloads that provides a robust environment for training, fine-tuning, and inferencing in which some impressive efficiencies can be achieved. In one recent example, Anyscale experimented with translating natural language to SQL queries and achieved similar performance to GPT-4 with a much smaller model of only 7 billion parameters—which equates to about 1/100th of the cost.

The Ray project is just one example of a trend toward open-source GenAI tooling that will accelerate in 2024. Others include LlaMA 2 from Meta (delivering promising results) and LAVA Realtime AI for video and vision-based processing.

One reason these models get so much attention is that many companies want the flexibility to deploy models in private data centers or cloud environments like Amazon Web Services (AWS). Companies also think more about who owns their models and how to build platforms that can be adapted to multiple industry-specific applications.

“By creating smaller, custom models that are cost- and energy-efficient, these models can help aid domain experts and other stakeholders in completing complex tasks that require retrieving/summarizing information from multiple sources, generating new customer-facing content, brainstorming new ideas, and more,” says Ria Cheruvu, AI SW Architect and Evangelist at Intel.

As #GenAI advances at speeds impossible for any organization to match, collaborative #partnerships will be important to addressing infrastructure and compute resource requirements. @anyscalecompute and @Accenture via @insightdottech

This desire for flexibility highlights the importance of collaboration. As GenAI advances at speeds impossible for any organization to match, collaborative partnerships will be important to addressing infrastructure and compute resource requirements and managing the development, monitoring, and deployment of models through machine learning operations (MLOps). For example, Accenture uses cnvrg.io, an MLOps tool, to facilitate collaboration among data scientists and engineers.

Accenture is also an interesting example because it leverages industry collaborations to help deliver on the promise of cutting-edge technology. For example, Intel and Accenture have come together to create a set of AI reference kits designed to accelerate digital transformation journeys.

Libraries and Tools Optimize GenAI

The platforms and reference kits we’ve looked at so far are just a small sample of a larger trend that will undoubtedly accelerate throughout this year and beyond. The spread of libraries and optimizers is also part of this trend. For example, the Optimum Habana Library and the Optimum Intel Library help make Intel’s deep learning accelerators easily accessible to the Hugging Face open-source community.

In terms of optimization, two noteworthy examples come to mind. On the model creation side, the computer vision AI platform Intel® Geti is designed to create highly accurate vision models with limited input data and computational resources. On the deployment side, the Intel® Distribution of OpenVINO Toolkit compresses AI models to a size that’s suitable for edge computing.

As AI development increasingly focuses on cost reduction, there will be increased use of tools like these within collaborative supplier ecosystems that enable comprehensive GenAI applications.

The Big GenAI Challenges for 2024

Although there are many reasons to be excited about the future of GenAI, there are also major challenges ahead. First and foremost, consumers and customers alike can be skeptical of AI, so it is critical to avoid delivering disappointing solutions. One of the biggest problems is the difficulty of eliminating “hallucinations” where models generate false or irrelevant responses.

The issue is particularly important for applications with regulatory or ethical implications. Developers continue to face difficulties aligning their models with regulatory standards and ethical norms. And it’s worth pointing out that this is not just a moral issue: GenAI systems have the potential to violate laws and cause real harm.

There isn’t a complete solution for these challenges today, but developers should look to responsible AI practices that are beginning to emerge. Among other things, they should strive to develop and use AI in a manner that is ethical, transparent, and accountable. As a practical example, an AI can be built to explain how it makes decisions. 

Of course, responsible AI is not just a technical matter; it also implies cultural and societal change. At a company introducing AI into its workflows, for example, this could involve directly addressing concerns about AI’s impact on jobs, training employees to work with AI, or adjusting business processes, among other steps.

Exciting Year Ahead

In 2024, the GenAI landscape will be reshaped by increased collaboration and innovation, with a focus on custom model development for specific industry needs. The trend toward open-source projects, the continued growth of tools and platforms, and the growing awareness of ethical concerns will all come together in ways that will continue to surprise us.

“It’s exciting to see the breadth of technologies in this space making it convenient and fast to create optimized AI models that can best fit a business’s needs and values,” says Cheruvu.

This article was edited by Christina Cardoza, Editorial Director for insight.tech.

The Rise of Intelligent Networks with 5G and IoT

As demand for the Internet of Things and intelligent solutions continues to grow, the 5G landscape must evolve with it. According to CCS Insight, over the next couple of years, artificial intelligence is expected to gain more traction in this space—enhancing availability, managing traffic, and providing self-healing capabilities to the network.

Moreover, the rise of private networks and hybrid solutions, facilitated through network slicing, is anticipated to become the predominant approach for ensuring quality service.

In this podcast, we look at how 5G and IoT networks have to adapt to the rapid pace of innovation, what’s still to come in the 5G space, and when it will be time to start thinking about 6G.

Listen Here

[Podcast Player]

Apple Podcasts      Spotify      Google Podcasts      Amazon Music

Our Guest: CCS Insight

Our guest this episode is Ian Fogg, Director of Networks at CCS Insight. Before joining CCS Insight, Ian was VP of Analysis at Opensignal, Senior Director of Mobile and Telecom at IHS Technology, and Principal Analyst at Forrester Research.

Podcast Topics

Ian answers our questions about:

  • (1:42) The changing 5G and IoT network landscape
  • (3:42) 2024 5G and IoT network predictions
  • (8:12) The growing demand for IoT and intelligent solutions
  • (10:21) AI’s role in the future of 5G and IoT networks
  • (12:57) Sustainability efforts with intelligent networks
  • (17:47) A look toward the future of 6G

Related Content

To learn more about 5G and IoT network predictions, read the report Edge Computing and IoT Predictions for 2024 and Beyond and listen to Top IoT and Edge AI Predictions for 2024: With CCS Insight. For the latest innovations from CCS Insight, follow them on Twitter at @ccsinsight and on LinkedIn.

Transcript

Christina Cardoza: Hello and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech. And today we’re going to be talking about the rise of intelligent networks thanks to 5G and AI with Ian Fogg from CCS Insights. Hey, Ian, how are you doing?

Ian Fogg: Hey, doing well.

Christina Cardoza: Thanks for joining us today. Before we dive into the conversation, can you tell our audience a little bit more about yourself and what you do at CCS Insight?

Ian Fogg: Sure. So, we’re a research and analysis company. I lead the networks or the network-technology research at CCS Insight. Before CCS Insight I’ve worked with operators, analytics companies, other analyst firms. So, really many, many years of experience in network technology.

Christina Cardoza: Awesome. Looking forward to dive into it. We just had your colleagues Martin Garner and Bola Rotibi on the podcast. We were talking about IoT protections for 2024, specifically in the edge and AI space. But as part of that report—and I encourage all our listeners to go take a look at that report to see what’s coming over the next couple of years—but as part of that report there was a lot about 5G and networks in there. So I wanted to chat with you today a little bit more about what can we expect in that space, and these things sort of run parallel sometimes.

So before we jump into what we have coming, let’s take a look about where we are today, or what got us here today. So can we just start off explaining how has the 5G- and IoT-network landscape changed over the last few years? What have you been seeing?

Ian Fogg: Sure. Well, I mean 5G first launched back in 2019, but that was really very early versions of the 5G standards. And what we’ve seen more recently is tremendous enhancements in what 5G is capable of. We’ve seen 5G being used for more and more different things.

So one of those is, for example, private networks, and we’re tracking the number of private networks that are launching or are being announced. For example, in 2023 up to Q3, we saw 1,279 private networks announced with revenues of over €100,000. That’s up from 1,081 in 2022 and 761 in 2021. 5G was deployed in 45% of those networks we saw announced in 2023.

We’re seeing other things happen too. We’re seeing non-terrestrial networks arrive. We saw a lot of activity in that last year around this time, with announcements from, at the time, Qualcomm, and that’s part of the upcoming release 17 5G standard as well. We’ve seen more focus around REDCap—reduced capability—on 5G starting to be talked about by the vendors, but not really out there yet in the market.

One of the other things we’ve seen happen last year is the operators announce OpenAPI initiative, something the GSMA—the operators organization—has been very big on. And that includes network APIs too, to manage network quality and other network-type settings. So we’re seeing all kinds of things happening over that last year.

Christina Cardoza: Absolutely. A lot happening and, like you mentioned, 5G started to come around, maybe around 2019. The conversation started well before that, and the conversations are still happening. It always amazes me how much 5G is being adopted or deployed, and how much there is still to come and how much we still have work in this space.

So, what would you say, looking to 2024 or even beyond that, what would be the next step for 5G in these IoT networks coming up?

Ian Fogg: Well, this takes us back to the predictions that we put together at the end of last year. So, some of these are related to what I just said, some of them are looking a bit beyond that.

So, one of the predictions we had was that by 2025 a digital marketplace for app-based network functionality offers more than a hundred versions of network capabilities and APIs. We’ve seen some small initiatives, but we expect that to expand tremendously over the next 18 months or so.

We’re also expecting that hybrid private and public 5G, through network slicing, emerge as the dominant option for private networks by 2030. So what that is, is that a standalone private network is using a dedicated network just for that offering. But a hybrid solution is using some of the new capabilities of standalone 5G, network slicing to give you a quality of service that is different to other people using the wider cellular 5G network. And that then bridges the gap between a location where an enterprise may have the dedicated private network, but they maybe want people or devices to move between those dedicated sites. And the macro network, the regular 5G network with a network slice, bridges the gap—a hybrid solution. So that’s something we see becoming very dominant by 2030. And obviously it’ll happen not just suddenly in 2029; it’ll happen gradually over time as we see something happening.

Something else we see—although there’s been some negativity around NTN—we see that continue to grow. It’s valuable for the IoT space. If you think about container tracking, it doesn’t require very high bandwidth services. But there are other things that we see happening very near term, like the smartphone space, where we see that’s still growing despite some of the negativity; there’s a lot of activity in that space. It’s part of the upcoming 5G standards, part of release 17 to have a non-terrestrial network capability. And by 2027 we expect 15% of smartphone users have satellite-enabled devices.

Now, what does that mean for IoT? Well, often what has happened in the past is the consumer space has driven innovation that then gets reused for other things because there’s a commonality. Once you’ve built that satellite capability, the satellite players can choose to support more than one type of customer.

Christina Cardoza: Great. Now, you mentioned more of a rise in hybrid private and public networks. When I think of these terms, I equate them to the cloud: hybrid cloud, private cloud, public cloud. Is this the same concept of using networks? So, for instance, when you have more mission-critical solutions or applications, those would be on a private network, much like they’d be on a private cloud. What’s the distinction there, or the similarities between the different types of cloud uses and then the network uses? Will these things be parallel? Also, if you’re using a hybrid cloud, would you likely be using a hybrid network?

Ian Fogg: I think it depends on the use case. You think about a hybrid network, the key thing that’s really driving that’s, that’s enabling that, is this switch from the initial versions of 5G that started back in 2019, which were using what’s called non-standalone access. They relied on a 4G core network, which means you didn’t get all the capabilities of 5G that were being hyped in the 2017, 2018, 2019 period. You need to have the 5G core network up and running at the mobile operator, at the service provider. And that enables a whole load of new network functions, and one of those is this thing called network slicing, where you can have essentially a dedicated quality of service. So that’s something that’s happening.

I think the cloud space is parallel, but I think it’s an interesting analogy to say, well, there’s more than one type of cloud; there’s more than one type of network that can offer a high quality of service.

Christina Cardoza: Great. And at the same time that all of this is happening, 5G is getting more advanced and we’re adding more capabilities to it. The demands and the needs of the Internet of Things is growing also; companies want to connect to the internet more. They want fast, reliable, real-time information, and that requires the network to work fast also. So how can the network and 5G keep up with this ever-growing demand of IoT and some of these more intelligent solutions and devices coming online?

Ian Fogg: Well, I think this hybrid capability is particularly important, because it does bridge the gap between maybe a dedicated network that’s maybe a dedicated private network that’s maybe in a port or in a manufacturing facility or in maybe a warehouse or a barn or something.

I mean, one of the areas we looked at here actually for hybrid private networks or hybrid 5G was around precision agriculture. And if you imagine agriculture, you have the farm, you have some farm buildings where you may have a dedicated capability, but you can’t put that over the whole farm area. It’s far too big, far too problematic. And that’s a classic example where a hybrid solution enables an agricultural offering to have a high quality of service right really across a wide range of areas.

One of the other things I think we see as an important thing at the moment, and increasing so, will be using AI in a whole range of different areas. We think AI is going to be important in a range of types of products in different parts of the network. But one of them is we see that AI enables 5G networks to improve their availability significantly, perhaps even to move beyond five-nines availability by managing the traffic patterns better, by making sure that the network isn’t just on, it’s offering a good enough quality of service and managing around outages or downtime issues to give a self-healing element to the experience. So that’s something else we see increasingly important on the network side.

Christina Cardoza: I love that example of the agriculture and in the farm, because that’s one of the examples that IoT is just growing and demand for it—farming, you wouldn’t really think becoming online or using these advanced capabilities, but that’s just the reality today. Every industry around the world is leveraging Internet of Things and and network.

I want to dig deeper a little bit into the role of AI. You mentioned that it’s going to be improving availability and it can do things like self-healing. So exactly what’s going on in this space? How can AI be utilized and be applied to do these things and enhance the 5G and IoT networks?

Ian Fogg: Well, the network guys are using AI on all kinds of areas. They’re using it to improve the RAN management, because as you’ve gone from original 4G to 4G Advanced or LT Advanced, and then onto 5G and then onto release 17 and onwards, the complexity of the RAN has got much greater, and there are more settings that need to be managed. The interaction between the base stations, between different frequency bands, is much more complex, and AI is a key way of enabling ongoing management of that RAN to improve the coverage and improve the performance.

We see it being very important in the Open RAN rollout too. So, Open RAN is something where, historically, service providers have bought a base station from a network vendor, and everything is basically integrated and included from the same vendor. The concept of Open RAN is that there are interfaces within that base station so that a service provider can mix and match different suppliers. And we think the complexity there is something that AI can help improve the Open RAN experience and help drive Open RAN adoption. So we think that that’s very important too.

Something else we think is interesting and important is around green issues: hitting carbon targets, managing energy costs on the network. And, again, what you need to do there, if you’re a service provider, is you want to keep the performance for your user base good, but minimize the energy usage. And so you want to drop the energy usage but still maintain the network experience. So how far can you cut back network resources and yet still offer the experience that users need.

And that’s something where a machine learning tool can help with that RAN optimization. And we feel by 2025 a combination of intelligent radio access network–technology, automation, and AI-driven power-down techniques enables at least three leading operators to bring forward their carbon-neutral targets by several years. It’s one of the predictions in our report.

Christina Cardoza: Yeah, I was just going to ask that. It’s interesting because I think over the last couple of years sustainability and green issues have been a top concern for industries all over the place and being able to use AI and even these 5G networks to hit some of these goals. So I was just going to ask what you predict for the next couple of years: is it going to be more important, and do you see organizations and industries actually being able to reach some of their goals, or those goals really becoming a reality over the next couple of years with some of the capabilities happening in 5G and IoT networks?

Ian Fogg: One of the sustainability angles here is around smart grid. There’s obviously a massive drive to remove coal, oil, and gas carbon technologies from power generation. One of the options to replace there is obviously nuclear. Nuclear has enormous capital cost, long lead times, a lot of complexity. So a lot of people are looking to solar and wind power as a cost-effective and versatile way of generating green electricity.

The challenge there is that solar and wind power generation is not always predictable—depends on cloud cover, on the solar side, and time of year. And on the wind power, it just depends on what the weather systems are doing at the time things are happening. And you could have night times where there’s too much power being generated and you want to encourage end users to consume some of that electricity. There may be other times you want users to drop their energy consumption because it’s too expensive or there isn’t enough green power being generated.

So we think greater use of solar and wind power necessitates that smart grid technology to manage the supply and demand. And we expect smart grid technology will become widely adopted in most advanced economies from 2028, if not before. But we can see some signs of that happening even now.

Something else we see as important is power-as-a-service, an integral part of tower infrastructure services by 2025, because one of the challenges or one of the difficulties of getting power to tower sites: it’s difficult, it can be time consuming, it can depend on permits. But what we see happening there potentially is the tower company managing the power to the site. So, taking a greater role in the site management than they have done before. And we think that’ll become a cornerstone of tower companies’ offerings by 2025.

Christina Cardoza: One thing that interests me about the smart grid is obviously we have all of these new and intelligent devices connecting to the network and going online, and we need to prepare the grid to be able to handle all of this at the same time we’re trying to be more sustainable. And then as part of being sustainable people are using more—electric vehicle technology or electric technology and plugging that into the grid.

So is there anything happening in the 5G or network space that is going to be able to help that demand or help the smart grid really become smart and become more sustainable at the same time?

Ian Fogg: So, if you take electric car EV charging, typically you want to do that overnight if you can, if it’s on a residential solution. So you can see a range of ways that network technology come in there. Many EVs have cellular capability, so the user can remotely control their car and set the power-saving modes, tell it where and when to schedule the charge. Many EV chargers in the home have also a similar remote control—often that’s on Wi-Fi or something else.

But then you have smart meters in the home, which typically have a cellular connectivity. So the power company can monitor what’s being used and charge people—in some cases charge people in very granular ways. There’s a power company in the UK, for example, which has a tariff which has half-hourly pricing. So the pricing varies dynamically during the day based on what the power generation and the overall consumption is. So you can see it in different places—whether it’s in the EV, you have connectivity; whether it’s in the EV charger; whether it’s in the smart meter—three different places just involved in the end-user EV charging process where you could see network technology becoming very important.

Christina Cardoza: Great. Yeah, it’s definitely very interesting to see. We always talk about these elements different from one another, as silos: this is what AI’s doing in this space, this is what network’s doing in this space. But it’s really interesting to see it as an end-to-end solution, how we’re using all of these technologies to really meet these goals and to make some of these ideas as a reality.

One thing I wanted to talk about is, obviously from this conversation it sounds like there’s still a lot of work to go in 5G. 5G is still going to be around for a while and keep improving itself and improving industries and businesses and other technologies. But we’re already hearing some rumblings about 6G come up. So I’m not sure if it’s too early to start talking about 6G—what’s the reality there? People are getting hyped about this, but where are we with 6G? When should we prepare for 6G, and how long will 5G still be dominant?

Ian Fogg: So there’s absolutely work happening on 6G, and I’ll talk about that in a second. But I think you are absolutely right: that 4G is still important in the 5G era. You think about that EV scenario I mentioned a second ago: most EVs, they have cellular; it’s a 4G cellular radio. You have a smart meter: it might even have a 2G radio in it, let alone 3G or 4G. Although they’re obviously progressing and upgrading them to 4G to enable a 2G switch-off to happen. So these network eras tend to overlap.

So even when 6G arrives, 5G will continue to be important. 4G will still be probably around as well, because these things are interoperable and they will continue to exist. The reason 6G is interesting and the things we can already see happening are we can see spectrum discussions being advanced.

We saw WRC—the international conference that happens every few years to coordinate spectrum alignments globally—so there are large economies of scale in any of these offerings. Discussion happened there around 6G bands—typically in the cellular space—people looking at the 7 to 16 gigahertz range. But there’s also some terahertz capacity, which is very, very high frequency. If you’re familiar with some of the 5G offerings using millimeter wave, this is even higher frequency than that, which is also being discussed—very line of sight, very, very high capacity, probably very short range. So there’s work happening.

There’s also work happening on the use cases for 6G. One of the use cases, which is different to what we’ve seen before, is using the cellular network to sense what’s happening. It could be sensing how much traffic there is in the roads, sensing people walking down a pavement. And that is something which is—you can see examples of that are happening now. Some of the—in the home, for example, some of the Wi-Fi access points can sense whether people are at home and use that as an alarm system, like a crude alarm system. But that’s quite a crude offering at the moment.

But part of the thinking around 6G is the 6G network would have a wider sense of capability to sense things happening across cities, across large areas. And then of course that means you can then draw analytics from it and make actions on the back of that. And that’s one of the key new use cases that’s being discussed around 6G.

In terms of timescale, which you asked about, is stuff’s happening now. I guess it depends who you are. If you are a company looking to deploy something today, 6G isn’t probably relevant. If your product roadmap is much longer, if you’re looking at kind of around 2029, 2030 or onwards time, and if you are starting to work on those sorts of things in your roadmap now, then 6G should be something that’s in that roadmap. They’re obviously not imminent, but back off. If you are a network vendor, then your R&D labs are fast furiously working on 6G things at the moment. And you are much more advanced in your 6G thinking.

Christina Cardoza: I think that point you made about there will still be some overlap and these technologies are interoperable, so I think that’s a really important takeaway for listeners. Also, obviously you just mentioned all of these benefits and capabilities and new use cases that are coming with 6G that can make the industry very excited to hop on it, but it doesn’t have to be a “6G is here now; move everything to this technology.” There is overlap, there is interoperability, so you can move things over slowly, what makes sense and with time, and be more purposeful about these changes coming, rather than going in all or nothing. So I think that that’s a great point. This has been a great conversation.

We’re running a little bit out of time, but I’m just curious, in this space, are there any final key thoughts or any predictions you want to leave us with, what we can expect to come?

Ian Fogg: I think just one last one on the 6G side. So, one of the predictions we had around that was that by 2030 the first 6G-powered, massive twin city is announced. Digital twinning is this idea of having a replica of the physical world in the digital world. And we see 6G being important in that, partly because what we just talked about around sensing and around smart city–type uses, 6G will have a lot more capacity; it’ll probably have much lower latency, more consistency, and it’ll have this sensing capability too.

I mean, one of the classic areas I think this might get deployed is perhaps in the Middle East, in the Gulf region, where there’s enormous efforts to build a number of massive new cities basically from scratch. Greenfield—well, not greenfield, because it’s the Middle East—but in new locations, a planned city. And if you are building something from scratch, you are able to do things differently, and it may well be that the first massive digital twin city 6G enabled happens in that region for those reasons.

Christina Cardoza: Wow, 2030 sounds like a long ways away, but we’re already in 2024, so, yeah. It’s going to be here before you know it. So I’m excited to see how this space progresses to get us there over the next six years, where else 5G and IoT networks are going.

But it’s been a pleasure talking to you today, Ian, about this. Again, it’s been an insightful conversation, and for our listeners who want to learn more about the CCS Insight predictions, we will have that ready for you. You could dig into the edge AI and this network space also. So thank you again for joining us, and, until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel® trademark information.

This transcript was edited by Erin Noble, copy editor.

10 Tech Influencers Shaping the Digital Landscape in 2024

In the rapidly evolving landscape of technology and innovation, influencers play a pivotal role in shaping conversations, trends, and the direction of various industry trends. As we step into 2024, insight.tech rings in the new year by identifying 10 influential voices in the tech industry, all using their digital presence to make a significant impact in their respective domains.

These influencers bring a wealth of knowledge and experience to the table, making them instrumental voices in shaping the future of technology. Whether you’re interested in the strategic implementation of digital transformation, the role artificial intelligence plays in shaping the future, or the security considerations within the AI landscape, following these thought leaders will provide valuable perspectives and foresight as they continue to pioneer influencing the 2024 narrative.

Stay up to date with all the industry trends, news, and event coverage from edge AI to sustainability by following insight.tech on X (formerly Twitter) and on LinkedIn. And make sure to follow these top tech influencers and thought leaders in the AI and tech space.

Kirk Borne, Ph.D.

Expertise: Big Data, IoT, AI

Why to Follow: LinkedIn “Top Voice” Kirk Borne is a luminary in big data, IoT, and AI. His expertise bridges these domains, offering a holistic perspective on the intersection of data and technology. Fun fact: He’s also an astrophysicist.

LinkedIn   X

Marsha Collier

Expertise: Retail, AI

Why to Follow: Collier’s expertise in retail and AI makes her a must-follow for those interested in the evolving landscape of consumer experiences and the role of AI in shaping the retail sector.

LinkedIn   X

Glen Gilmore

Expertise: Tech for Good, AI

Why to Follow: Gilmore’s advocacy for using technology for positive change and his expertise in AI make him a beacon for those interested in the ethical implications of technological advancements.

LinkedIn   X

Antonio Grasso

Expertise: Digital Transformation, AI, Cybersecurity

Why to Follow: Another “Top Voice” on LinkedIn is Antonio Grasso, a renowned figure in technology innovations. His expertise extends from the broader vision of digital transformation down to the granular intricacies of implementing robust cybersecurity measures. Interested in discovering where digital evolution meets people’s revolution? Antonio’s latest publication, Toward a Post-Digital Society: Where Digital Evolution Meets People’s Revolution, is the place to start. He also has a bite-sized monthly newsletter.

LinkedIn   X

As we step into 2024, @insightdottech rings in the new year by identifying 10 influential voices in the #tech industry, all using their #digital presence to make a significant impact in their respective domains.

Kevin Jackson, CISSP, CCSP

Expertise: Digital Transformation, AI, Data

Why to Follow: USA Today and Wall Street Journal bestselling author Kevin Jackson stands as a leader in digital transformation, providing expert commentary on the integration of AI, machine learning, and data analytics. His insights offer a roadmap for organizations navigating the complexities of the digital age.

LinkedIn   X

Jean-Baptiste Lefevre

Expertise: AI, Robotics, Cloud

Why to Follow: Lefevre’s insights delve deep into the realms of AI, robotics, and cloud computing. His thought leadership provides a roadmap for navigating the future of these technologies and beyond.

LinkedIn   X

Tamara McCleary

Expertise: AI, IoT, Digital Transformation

Why to Follow: McCleary is a leading voice in digital transformation, exploring the synergies of AI, IoT, and the evolving digital landscape.

LinkedIn   X

Franco Ronconi

Expertise: AI, Robotics, Future of Work

Why to Follow: Ronconi’s focus on AI, robotics, and the future of work provides valuable insights into the changing dynamics of the workplace and the role of automation in shaping our professional lives.

LinkedIn   X

Dr. Monika Sonu

Expertise: Healthcare, AI

Why to Follow: Sonu’s insights into the intersection of healthcare and AI offer a glimpse into the transformative potential of technology in revolutionizing the healthcare industry.

LinkedIn   X

Helen Yu

Expertise: Digital Transformation, IoT, Cybersecurity

Why to Follow: As a Wall Street Journal bestselling author and LinkedIn “Top Voice,” Helen Yu is a dynamic influencer making waves in the realms of digital transformation, IoT, and cybersecurity. Her expertise lies at the intersection of these critical domains, offering a comprehensive understanding of how organizations can secure their digital landscapes while embracing transformative technologies. Catch Yu on CXO Spice, a series on her YouTube channel where thought leaders share their point of view on innovations and explain how to implement them in the real world.

LinkedIn   X

Edited by Georganne Benesch, Editorial Director for insight.tech.

AI and Beyond: Forecasting the Future of IoT Edge AI

It’s that time again: prediction season. Here at insight.tech we make an annual tradition of highlighting the top IoT predictions from CCS Insight, this year including plenty of edge and AI. We’ll break it all down with the CCS Insight Head of IoT Research, Martin Garner, and Bola Rotibi, its Chief of Enterprise Research (Video 1). And once again we’re hosting the complete research paper for our readers: Edge Computing and IoT Predictions for 2024 and Beyond. Be sure to check it out.

Video 1. CCS Insight’s Martin Garner and Bola Rotibi discuss upcoming trends and predictions for IoT and edge AI. (Source: insight.tech)

It’s clear that there are lots of benefits and opportunities associated with AI; it has the ability to transform industries and our lives. But it also comes with a certain level of discomfort on the part of the public—how is their data going to be used, how secure is it, and who is doing what to safeguard it in this Wild West of innovation and expansion? And nervousness, too, on the part of businesses trying to future-proof their investments. But one thing is sure: No one camp is responsible for getting the AI situation right. It’s going to be a collaborative effort from everyone building and using these solutions. 

How is the push toward edge and AI impacting IoT predictions for 2024?

Martin Garner: Obviously, 2023 was a huge year for both edge and AI. ChatGPT clearly had a massive impact and has really opened the world’s eyes to what AI can do—and to quite a few things that it can’t do yet. In our predictions for last year, we had lots about edge and some about AI. We really dialed up the AI predictions this year. And it’s not just generative AI, like ChatGPT; it’s not just what individuals can do with AI.

One prediction that’s an example of this is that by 2028, a major healthcare provider will offer its customers a digital twin service that monitors their health proactively. Now, today there is a lot of data about you scattered around—online health records, fitness bands, smart watches, data that’s on your phone, and so on. But it’s all a bit messy and not joined up at all. So we think that healthcare providers will start to combine these sources. They’ll use AI and basically do what the industrial guys are already doing—predictive maintenance—but on people. And of course the aim is early intervention, which often means smaller intervention, and usually cheaper intervention, too. So that’s an example of how we think AI will start to develop from here.

Will there be a continued move to the cloud because of these advancements?

Martin Garner: I think that the cloud-versus-edge debate is going to remain live for a good few years. I think in many countries there are worries about the economy, and we’re not quite out of all the pandemic effects yet, either. So one prediction is that recession fears push workloads from the cloud to on-premises through 2024. The best candidates for doing that are companies that are using hybrid cloud already. And it may be that a hardware refresh is a good opportunity for some companies to repatriate some of their workloads to on-premises—and take some cost savings while they’re doing it. That’s the short term.

Longer term, we think there are several areas where there are pendulum swings—like insourcing versus outsourcing—and edge to cloud could be one of those. But one area where we have another prediction is that by the end of 2028, there will be a repricing of cloud providers’ edge-computing services.

Now, what does that mean really? The major cloud providers all have edge-computing services, but actually the public cloud is the big part of the business. And the edge services, they’re sort of the on-ramp for the public cloud, and they’re priced that way. But the forecasts for edge computing show it growing bigger than the public cloud over five years—possibly a lot bigger. And if that happens, then we’d be in a position where the big part is subsidized by the smaller part, which makes no sense at all.

So we really expect the prices of edge-cloud services to move upward over three to five years, and that could be a significant price rise. Many industrial companies might want to consider their own edge computing to put themselves in a better position, one where they have more options. It’s edge as a hedge, if you like.

What can we expect in terms of AI solutions development over the next year?

Bola Rotibi: 2023 was a year of launches, especially from IT solution providers, with a wealth of new tools—obviously ChatGPT spawned massive interest. But I would say that the development of AI has actually been happening for quite some time, with machine learning and other sorts of models and algorithms that people have been using behind the scenes. Searching through pictures on your mobile phone—those are AI solutions, AI models.

What we are seeing is the power of generative AI, especially as a productivity solution. That ability of generative AI to simplify complex queries and bring back concise information that is also relevant information. So everyone’s jumping on that. Over the past year we’ve seen pretty much every provider—Intel among them—bring out generative AI capabilities, as well as beefing up their AI solutions. We’ve seen Microsoft launch with its AI-powered assistance Copilot and AWS with Amazon Q.

So we have a prediction that AI investment and development will accelerate in 2024—despite some calls for caution. Because quite a few of the main protagonists over recent months have said, “Hold on just a minute. We need to kind of slow this down.” People are also a bit worried about security; they’re worried about whether the regulations are out there and whether they are effective enough. But at the same time, I think there’s a real thirst to get AI and to develop it, because people have been just blown away by the new experiences and the engagement levels.

What is the reality of generative AI over the next year?

Bola Rotibi: On one hand it’s going to be really, really great and really fast; on the other hand, we’re going to see some slowdown. And another prediction—despite all of the froth and the fact that we’ll see lots of new tools—is that we do think there will be some level of slowdown in 2024. That’s partly because people will get to grips with the reality of the costs and some of the risks and complexities that have started to be exposed this year.

The excitement of 2023 will start tempering down into more of a level-headed, nuanced approach, and people will start to play with generative AI properly, delving into some of the capabilities like generated code. We’ll start seeing it across different types of workplace solutions, helping knowledge workers but also helping expert professionals.

“2023 was a huge year for both edge and #AI. #ChatGPT clearly had a massive impact and has really opened the world’s eyes to what AI can do—and to quite a few things that it can’t do yet”. – Martin Garner, @CCSInsight via @insightdottech

As investment accelerates, do you anticipate an increase in ethical-AI initiatives?

Martin Garner: The short answer is: Yes, there’s going to be a lot more of that. AI has the potential for many good uses in society, but used wrongly, it has the potential to do a huge amount of damage. It’s a bit like medicine, with regulated and unregulated drugs. But the big difference is that there’s no professional body, there’s no Hippocratic oath. You can’t be struck off as an AI practitioner, at least not yet.

At the moment we have the opposite situation, where as soon as something new is developed, the AI-leading companies open source it and push it out into the world as fast as possible. That obviously puts a huge imperative on suppliers and developers to take an ethical stance in how they use it, as well as on companies that are using AI as customers. There’s lots to get right there.

We do have a prediction that AI-oversight committees will become commonplace in large organizations in 2024—committees of ethics experts, AI experts, legal advisors, data scientists, HR teams, representatives from the different business units—to review use of AI across the company. Their job will be to bridge the gap between the tech teams—who are all engineers and not typically ethicists—and the organization and its goals.

That’s going to require quite a significant amount of overhead for a lot of companies, and lots of training to come up to speed and to stay on top of it. All that because the AI industry is largely not doing a good job of self-regulation.

What does the EU’s AI Act mean for development of AI solutions?

Bola Rotibi: The EU has been first out the door with this AI Act, which will be like GDPR. And we’ve already seen the ratification in the EU of the Digital Markets Act. But the EU is not the only one; there’s the US, the UK, China, and other regions as well. So I do think that the regulators are coming together, and we’re going to start seeing some level of improvement toward the end of 2024. But I think there will be a sort of bedding-in process, as people try to get used to it and understand what it all means—teething problems. But I think it will become something for people to rally around.

The other thing that’s actually happening is regulation at the industry level. Recently, 50 organizations—including IBM, Meta, and Intel—have launched the AI Alliance. It’s aimed at bringing the industry together to work collectively on standardizing; to bring working groups together; to come up with ideas for strategies and approaches to handling certain AI challenges and opportunities; and to be a hub for interactions between end users.

What are some considerations for developers building AI solutions?

Bola Rotibi: It isn’t just on the developers. In the same way that security is the responsibility of everyone in the workflow, so is an ethical approach to AI. Of course, developers could ask themselves, “Well, just because I can do it, should I do it?” But, at the same time, if you want a level of consistency, you have to provide guidelines and principles that are distributed and circulated right across the organization. It needs to come from the ground up, and it needs to go from the top down.

So I see that there will be a layered approach going forward. That may mean the oversight committee Martin mentioned thinks about where the organization is from an ethical standpoint and starts building policies. And then those policies will be put into the tools to act as guardrails. But there’s also going to be guidance and training of developers in terms of them taking a responsible-AI approach in the development process.

Lots of organizations have been thinking about impact, about sustainability, and all those kinds of things, so there is a wealth of ideas and initiatives already for making people think at multiple levels, not just about responsible AI but about doing the right thing in general.

Where does 5G fit into this? And when is it time to start looking at 6G?

Martin Garner: One of the things 5G will do is enable a lot more use of AI, thanks to the very high capacity, time-sensitive networking location services it will bring in. We’ll see a lot more AI in use around domains like autonomous vehicles, and the 5G that we have now—as well as the newer bits of 5G that are nearly here—is one of the key enablers of that.

But I think the other interesting bit is the impact of AI on 5G. The 5G networks—they’re complicated things; the whole optimization and management is a big deal. And we have a prediction around that, which is that AI will enable 5G networks to move beyond five-nines availability. That would come through analyzing traffic patterns and ensuring that the network is set up best to handle that particular type of traffic, to identify problems, to do predictive maintenance, and to configure the network so it has graceful degradation or can even become self-healing if things go wrong.

It is a tiny bit early for 6G, but work is going on, of course. Over the next five years or so, we’re going to be building 6G networks, and we think 2030 is going to be a bit of a headline year for it. So we do have a few 6G predictions. One is that by 2030 the first 6G-powered massive twin city will be announced. We think that cities will be a great showcase for 6G, and massive twinning will be one of the best use cases because of all the layers of a city that could potentially be included in the model. 6G would be needed just for the sheer volume and speed of the real-time data that runs through a city. We think 2030 will be a big headline year for that.

Related Content

To learn more about edge AI trend predictions, listen to Top IoT and Edge AI Predictions for 2024: With CCS Insight and read the report Edge Computing and IoT Predictions for 2024 and Beyond. For the latest innovations from CCS Insight, follow them on Twitter at @ccsinsight and on LinkedIn.

This article was edited by Erin Noble, copy editor.

Beyond Chatbots: The Potential of Generative AI Solutions

AI has been making headlines for quite some time already. But as we head into 2024, generative AI emerges as the latest breakthrough. Much of the attention has revolved around ChatGPT, and with it a lot of misunderstanding and misconceptions about exactly what generative AI is.

Breaking it down, we spoke to Waleed Kadous, Chief Scientist at Anyscale, an AI application platform provider; Teresa Tung, Cloud First Chief Technologist at global management consulting and technology company Accenture; and Ramtin Davanlou, AI and Analytics Principal Director as well as CTO in the Accenture and Intel partnership. They discuss the business opportunities around generative AI solutions, the challenges involved, and what comes next (Video 1). Because generative AI is here to stay, and there’s a lot to look forward to.

Video 1. Industry experts from Anyscale and Accenture discuss the implications and opportunities of generative AI solutions. (Source: insight.tech)

Please explain generative AI, the business opportunities, and challenges.

Ramtin Davanlou: In summary, companies like OpenAI, Google, and AWS use their massive compute resources and massive data sets to train AI models—or LLMs, large language models—to generate new content, to build net new knowledge. This content comes in different forms: text, images, video, voice, or even computer code. But text is especially important since it is the main means of communication for most businesses.

Many of these AI models are able to generate responses that are really good on any given topic—better than an average person or even an average expert on that topic. Companies can then take these models and fine-tune them a little bit so that the model behaves in certain ways and gains more knowledge about a specific context. That creates a lot of opportunities.

Companies can use generative AI to do things like send emails or create slides—all of this content we’re creating to communicate with each other—or to enhance those things. This has huge implications for service industries, and also for manufacturing when combined with robotics.

But what LLMs cannot do now, but may soon be able to do, is create net new knowledge from scratch.

What considerations should businesses think about when developing GenAI solutions?

Waleed Kadous: One consideration is the quality of the output from these models. There’s a problem with LLMs called hallucination, where they confidently assert things that are completely untrue. So how do you evaluate to make sure that the system is actually producing high-quality results? What base data are you using? So over the last six months we’ve seen developments in an area called retrieval augmented generation that helps to minimize the hallucination problem.

A second consideration is data cleanliness, which is in regard to the information these LLMs have access to. What are they disclosing? What do they have the power to inform people of? Is there leakage between different users? Can someone reverse-engineer the data that was used to train the models? It’s still a new frontier, so we’re still seeing issues that crop up in that front.

And then the final one is that LLMs are expensive. I mean, really expensive. You can easily blow a hundred thousand dollars in a month on GPT-4.

How can businesses get started and take GenAI solutions to the next level?

Teresa Tung: Most companies have their proofs of concept, and many are starting with managed models like OpenAI. And these amazing general-purpose models address many use cases and can offer a really great way to begin. But, as Waleed mentioned, cost in the long term is a factor; it could be an order of magnitude bigger than many companies are willing to pay. So companies now need to look at rightsizing that cost and rightsizing it for the performance.

When AI models become more critical to a business, we’re also seeing companies want to take ownership of them. Rather than using a managed model, they might want to create their own task-specific, enterprise-specific model. There are sub-10 billion parameter models that can be customized for different needs. There will still be the general-purpose models available but fit-for-purpose models as well.

Waleed Kadous: One of the experiments we did at Anyscale was in translating natural language to SQL queries. The general-purpose model, GPT-4, was able to produce an accuracy of around 80%. But by training an SSM—a small specific model—that was only 7 billion parameters, which was about one one-hundredth of the cost, we were able to achieve 86% accuracy in conversion. So small specific models versus large language models is an evolving discussion that’s happening in the industry right now.

Where are the biggest generative AI opportunities for your customers right now?

Waleed Kadous: The first kind of use case opportunity is summarization. Are there areas where you have a lot of information that you can condense and where condensing it is useful?

The second is the retrieval-augmented-generation family, which I mentioned before. That’s where you don’t just ask the LLM questions naively, you actually provide it with context—with an existing knowledge base of answers—that helps answer those questions.

Another interesting application is what you might call a “talk to the system” application. Imagine it as a dashboard you can talk to, a dashboard on steroids. This is especially interesting in IoT. I’ve seen a company that does this expertly; it does Wi-Fi installations for retailers. You can ask this dashboard questions like, “Where are we seeing the routers working too hard?” And it will query that information in real time and give you an update.

The final one is an in-context application development. Perhaps the best-known one is Copilot, where when you’re writing code, it will give you suggestions about how to write even better, higher-quality code. In-context applications are the most difficult, but they also have the highest potential.

Teresa Tung: Waleed gave a great overview, so I’m going to bring a different perspective—in terms of things you can buy, things you can boost, and things you can build. “Buying” is being able to buy generative AI-powered applications for software development, for marketing, for enterprise applications. They use a model trained on third-party data and enable anyone to capture these efficiencies. This is quickly becoming the new normal.

“Boosting” is applying a company’s first-party data—data about your products, your customers, your processes. To do that you’re going to need to get your data foundation in order, and retrieval-augmented generation is a great way to start with that.

“Building” is companies maintaining their own custom models. This would likely be starting with a pre-trained, open model and adding your own data to it. It gives you a lot more control and a lot more customization within the model.

Where do partnerships like the one Accenture has with Intel come in?

Ramtin Davanlou: Partnerships are very important in this area, because companies that are trying to build an end-to-end GenAI application typically have to solve for things including infrastructure and compute resources. For example, you need a very efficient ML Ops tool to help you handle everything you do—from development to managing, monitoring, and deploying the models in production.

“Partnerships are very important in this area, because companies that are trying to build an end-to-end #GenAI application typically have to solve for things” – Ramtin Davanlou, @Accenture via @insightdottech

We’ve used some of the Intel software, like cnvrg.io, an ML Ops tool that allows data scientists and engineers to collaborate in the same environment. It also allows you to use different compute resources across different cloud platforms—like in your on-prem environment, on Intel® Developer Cloud, and on AWS.

Partnerships are also an effort to reduce the total cost of ownership, especially the cost when you scale. Instead of building new platforms for every new use case, why not build a platform that you can reuse? For example, with Intel we have built a generative AI playground using Intel Developer Cloud along with GaudiTools, an AI accelerator specifically built to fine-tune the models for deep-learning applications. And then for deploying those models in scale, you can use AWS.

Another example is needing a tool to help distribute the workloads. There is a library called TGI from Hugging Face that is very helpful. So you can see that there are a lot of different components and pieces that need to come together so that you can have an end-to-end GenAI application.

Waleed Kadous: Another thing that has come up is the idea of open source—both open-source models and, of course, open-source software. One example that Meta has released is a model called Llama 2 that we’ve seen very, very good results with. It’s maybe not quite on par with GPT-4, but it’s definitely close to GPT-3.5, the model one notch down. There is vLLM out of Berkeley, a really high-performance deployment system; and also Ray LLM. vLLM manages a single machine; Ray LLM gives you that kind of scalability across multiple machines, to deal with spikes and auto-scaling and so on.

We’re seeing a flourishing of open source because not everybody likes entrusting all their data to one or two large companies, and vendor lock-in is a real concern. Also for flexibility: I can deploy something in my data center or I can deploy it in my own AWS Cloud, and nobody has access to it except me.

And for cost reasons—open-source solutions are cheaper. We did a profile of what it would take to build an email-summarization engine, where if you used something like GPT-4, it would cost $36,000, and if you used open-source technologies, it would be closer to $1,000.

We’ve seen a lot of interest in open-source models—from startups that tend to be more cost-focused to enterprises that tend to be more privacy- and data-control focused. It’s not that open-source models and technologies are perfect, it’s just that they’re flexible and less expensive. And there is availability of models at every size—from 180 billion down to 7 billion and below. It’s just a really dynamic space right now.

What needs to happen to make generative AI more mainstream?

Waleed Kadous: One of the increasing trends is an effort to make LLMs easier to use. But another is that we haven’t completely worked out yet how to make them better over time. If an LLM makes a mistake, how do you correct it? That sounds like such a simple question, but the answer is actually nuanced. So we’re seeing a massive improvement in the evaluation and monitoring stages.

And then, so far the focus has been on large language models—text in, text out—because every business in the world uses language. But we’re starting to see the evolution of models that can process or output images as well. Just as there is Llama for text, there’s now LLaVA for video and vision-based processing, even though not every business in the world needs to process images.

What should business leaders be conscious of on the topic of generative AI?

Teresa Tung: Hopefully the takeaway is realizing how easy it is to begin owning your own AI model. But it does start with that investment of getting your data foundation ready—remember that AI is about the data. The good news is that you can also use generative AI to help get that data supply chain going. So it is a win-win.

Ramtin Davanlou: I think regulatory and ethical compliance, as well as dealing with hallucinations and other topics under what we call responsible AI, are the biggest challenges for companies to overcome. And navigating the cultural change that’s needed to use GenAI at scale is really key to its success.

Waleed Kadous: It’s important to get started now, and it doesn’t have to be complicated. Think about it as a staged process. Build a prototype and make sure that users like it. Then come at cost and, to some extent, quality as secondary issues.

And you can give people tools to optimize their own workflows and to improve the LLMs themselves. I think that’s really one of the most exciting trends—rather than seeing GenAI as a substitute technology, seeing it as an augmentative technology that helps people do their jobs better. Help people to use LLMs in a way that makes them feel empowered rather than eliminated.

Related Content

To learn more about generative AI, listen to Navigating the Generative AI Solutions Landscape. For the latest innovations from Accenture and Anyscale, follow them on:

 

This article was edited by Erin Noble, copy editor.

Top IoT and Edge AI Predictions for 2024: With CCS Insight

AI and edge computing are taking IoT to the next level, with new and exciting use cases expected over the next year and beyond. But as the rapid pace of advancements and investments unfolds, it raises the question: Are we accelerating too fast?

In this podcast, we dive into the forefront of technological evolution, unveiling CCS Insight’s top IoT and Edge AI predictions for 2024. We also navigate through responsible development of solutions and explore the impact AI is set to make in the coming years.

Listen Here

[Podcast Player]

Apple Podcasts      Spotify      Google Podcasts      Amazon Music

Our Guests: CCS Insight

Our guests this episode are Martin Garner, Head of IoT Research, and Bola Rotibi, Chief of Enterprise Research, at CCS Insight. Martin has been with CCS Insight for more than 14 years, specializing in the Internet of Things. Bola has been with the analyst firm for more than four years, researching the software development space.

Podcast Topics

Martin and Bola answer our questions about:

  • (1:33) How the rise in edge AI impacts IoT
  • (4:05) If the cloud versus edge debate will continue
  • (7:36) What the future of AI development looks like
  • (11:20) The reality of generative AI
  • (15:08) Business considerations for AI investments
  • (17:55) Upcoming AI regulations and implications
  • (21:25) Developing AI with privacy in mind
  • (25:43) Where 5G fits into AI predictions

Related Content

To learn more about edge AI trend predictions, read the report Edge Computing and IoT Predictions for 2024 and Beyond. For the latest innovations from CCS Insight, follow them on Twitter at @ccsinsight and on LinkedIn.

Transcript

Christina Cardoza: Hello and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech. And it’s that time of year again where we’re going to look at the top IoT and AI predictions for 2024. And joining us again on this topic we have Martin Garner and Bola Rotibi from CCS Insight.

So for those of you who aren’t familiar or didn’t get to hear our last podcast with these two experts, let’s get to know them a little bit before we jump into the conversation. Bola, I’ll start with you. What can you tell us about what you do at CCS Insight?

Bola Rotibi: I’m the Chief of Enterprise Research at CCS Insight. I have a team that covers software development, workplace transformation, cloud, AI of course, security—quite a whole load, but everything to do with the enterprise.

Christina Cardoza: Great. And Martin, welcome back to the show. What can you tell us about CCS Insight and yourself?

Martin Garner: Thank you, Christina. Well, yes, I’ve been at CCS Insight for 14 years. I lead the work we do in IoT, mostly focusing on the industrial side, and so work very closely with Bola on those areas.

Christina Cardoza: Great. And, Martin, since you’ve been at the company for, like you said, 14 years now, we’ve worked a bit over the last couple of years. You guys always have an IoT prediction you put out at the end of the year, and we’ve always worked to talk about that and to see what’s coming next.

So I’m wondering, in the last year or even the last couple of years there’s been a big push to edge and AI. So I’m wondering how that impacted this year’s predictions and what we see coming for 2024.

Martin Garner: Sure. Well, obviously 2023 was a huge year for both of those areas. Edge was already growing strongly coming into this year, and I think 13 months ago ChatGPT had just launched and has clearly had a massive impact during this year and has really opened the world’s eyes to what AI can do—and quite a few things that it can’t do yet. So I think it’s fair to say that the world is now thinking rather differently about the two areas, and we are too.

So in our predictions last year we had lots about edge and some AI. I think for anyone who downloads the booklets at the end of the podcast you’ll see that we really dialed up the AI predictions this year. And it’s important: it’s not just generative AI like ChatGPT, and it’s not just what individuals can do with AI.

And there’s one prediction I’d like just to give as a little example of that, which is that by 2028 a major healthcare provider offers its customers a digital twin service that proactively monitors their health. Now today there is a lot of data about you scattered around—so we have online health records in many countries; we have fitness bands; we have smart watches and they show activities, sleep, and so on. There’s lots of data on your phone, but it’s all a bit messy and it’s not joined up at all.

So we think that healthcare providers over three to four years will start to combine these sources. We’ll use AI and we’ll basically do what the industrial guys are already doing, which is predictive maintenance on people. And of course the aim is early intervention gives smaller intervention, and it’s usually cheaper intervention too. The outcomes are just better if you get there earlier. So that’s an example of how we think AI will start to develop from here.

Christina Cardoza: Yeah. And I love what you said—we’ve learned a lot about what we can actually do with AI, or what’s possible. There’s always that hype or that trend that everybody wants to jump on right away. So I want to dig a little bit into that.

But before we get there, just this movement to—with AI and this movement to edge and having, wanting to get information at real time and faster—there’s been a lot of movement also to the cloud with all of these solutions and applications. And I know there’s always that cloud-edge versus on-premise debate. Will that continue? Or will people continue to move to the cloud over the next couple of years, or is on-premise still going to remain steady?

Martin Garner: Well, it’s a good question, and I think that debate is going to remain live for a good few years. And I want to draw out just a couple of ways of that. So one is quite short term. I think in many countries there are fears about where the economy’s going. We’re not quite out of all the pandemic effects yet. So one prediction is that recession fears push workloads from the cloud to on-premises through 2024.

Now, the best candidates for that are companies which are using hybrid cloud. It’s not so much those who’ve gone all-in on the cloud. And there are a few new initiatives, like HPE GreenLake and Dell APEX, which bring the flexible consumption model that we have in the cloud—they bring it to hardware and allow you to do that at the edge as well. So it may be that a hardware refresh in some of those companies is a good opportunity to rationalize some aspects, sort of repatriate some of their workloads to on-premises and take some cost savings while they’re doing it. So that’s the short term.

Longer term, we think there are several areas where there are sort of pendulum swings, like insourcing versus outsourcing, and edge to cloud could be one of those. And it’s partly also that there are new tools and different economics coming up in both areas. So it’s all changing quite fast.

But one area where we have another prediction is that a repricing of cloud providers’ edge computing services takes place by the end of 2028. Now, what does that mean really? Well, the big cloud providers, they all have edge computing services, but actually the public cloud is the big part of the business. And the edge services, they’re sort of on-ramp for the public cloud, and they’re priced to do that. But the forecasts for edge computing, they show it growing bigger than the public cloud over five years, possibly a lot bigger than the public cloud.

Now, if that happens, then we’d be in a position where the big part is subsidized by the smaller part, and that makes no sense at all. So we really expect the edge-cloud services, their prices to be moved upwards over three to five years, and that could be a significant price rise. So we think many industrial companies might want to consider their own edge computing, just in case—put themselves in a better position so they have more options. It’s edge as a hedge, if you like, and kind of gives them more options about how they go forwards.

Christina Cardoza: Yeah, absolutely. And that’s a great point. Companies, they don’t want to be locked in with a vendor. Like you said, more opportunities. They want to make sure any investments they make—because they do make a lot of investments to move to these solutions to build these—they want to make sure any investments they do make they can future-proof them. I think a lot of it driving it too is the type of solution that they’re building. If it’s more critical workload or they need more security, they tend to move on premise. If they need real-time analytics or they need faster results, that’s sometimes the cloud and the—where the edge computing comes in.

So I want to talk a little bit about AI development with you, Bola. What can we expect in terms of the AI solutions or the developments happening over the next year within enterprises?

Bola Rotibi: Well, I think—I mean it’s a good question actually, because I think we’re going to expect quite a lot, and I think, whilst 2023 has been kind of a year of—we’ve seen a year of launches, especially from many of the IT solution providers, there’s been a wealth of new tools. I mean, obviously when ChatGPT launched last in the end of November 2022, it really spawned massive interest, and we’ve seen it exponentially grow, especially with generative AI.

But I would say that the development of AI has actually been happening for quite some time, because it’s been machine learning and various other sort of models and algorithms that people have used behind the scenes. So anything that you’ve been using on your mobile phone to search through pictures and all those, those are kind of like AI solutions, AI models. So this AI itself is not new, right?

But what we are seeing is the power of generative AI, especially as we think about the productivity. You know, everyone has sort of latched on to the conversational aspect of generative AI and its ability to simplify even sort of complex queries and bring back concise information that is relevant information. So everyone’s jumping on that because we see that as a productivity solution.

So what we’re seeing now is a launch of lots and lots of solutions. Over the last year we’ve seen pretty much every provider—Intel amongst those as well—bringing out generative AI capabilities, as well as beefing up their own AI solutions. But one of the things that we’ve seen here, we’ve seen Microsoft launch with the Copilot AI-powered assistance. We’ve seen others from other providers like AWS with Amazon Q.

But I think what we’re—what one of the things that we do think is that despite all this frothiness around the interest, we did have a prediction which actually said AI investment and development will accelerate in 2024, it does accelerate in 2024—despite calls for caution. And we’re already seeing the throughput of this already. And the reason we see this is that actually quite a few of the main protagonists, I mean over the recent months have said, “Well, hold on just a minute. We need to kind of slow this down. Things are moving rather fast.” And people are a bit worried about security; they’re worried about whether the regulations are out there, whether they are effective enough. And all of this caution is still kind of wrapped around.

But at the same time I think there’s a real thirst to get AI, to develop it, to really get their hands on it. Because people have been blown away by the new experiences and the engagement levels that they can have, especially with generative AI. So I think there’s going to be even more acceleration next year.

Christina Cardoza: Yeah, absolutely. And a lot to dig in there. I want to start off with the generative AI piece. And, like you mentioned, the possibilities—it brings so many benefits that a lot of companies are trying to jump on this, and I always hear in the back of my head, one of our guests on the podcast was saying, “You want to create solutions that solve a problem, not create solutions just to create a solution, just to jump on the latest trend.”

So I’m curious with generative AI, because obviously, we’re seeing all of these companies talk about it—all of these releases and with AI toolkits adding generative AI capabilities, there’s been a lot of hype. What’s the reality? What do you think will actually come out of this space in the next year, and what can we be looking for? What are the applications or use cases that are possible?

Bola Rotibi: Oh gosh, that’s lots and lots of questions. Well, first of all I think there is a lot of froth around with generative AI, but that’s because everyone’s been, as I said earlier, been blown away by the productivity that they’ve seen from it and actually the intuitiveness of it all, and just really being able to talk or put together natural language queries and then get back concise information.

That said, I mean, one of our predictions that we outlined is that despite all of the excitement around generative AI and the fact that we’ll see lots and lots of new tools, we do think that, 2024, we’ll see some level of slowdown—partly because as people kind of get to the grips with the reality of the costs, some of the risks and the complexities that really, that become, start to be exposed and have been starting to be exposed this year, I think that we’ll see a slow, a slight slowdown.

But, despite that, I mean I’ve—it’s a bit of a thing where you’re sort of saying on one hand it’s going to be really, really great and really fast. On the other hand we’re going to see some slowdown. What I think, as with any technology, there will be people just trying all sorts of capabilities.

Nowadays you can go in and you can start typing in queries, and then immediately get very, very concise answers back. And what we’re also seeing is that we’re seeing generative AI in multiple perspectives. So we’re seeing it as an assistant to providing information to relevant information across lots and lots of different data types. We’ve also seen it being able to create, generate code that is very good quality—that at least is a starting point, especially the experts to finalize it and sort of do a little bit more quality control.

So I think we’re going to see—so on one side I think 2024 is really where people get to start to play with it properly. And the tools, they’ll get to start to understand some of the limitations of the tools, as well as what they’re going to be able to achieve. So I think it’s going to be, on one hand the hype of 2023 will start tempering down into much more of a more level-headed sort of approach to it and more nuanced, as well as the excitement of actually delving into some of the capabilities, like generated code. We’ll start seeing it across different types of workplace solutions, helping knowledge workers, but also helping expert professionals—whether those be expert developers or other professions as well.

Christina Cardoza: Yeah, it’s interesting because we may see a slowdown in some of the hype or everybody jumping onto this trend. But, like you mentioned, we’re going to see an increase in acceleration in AI investments maybe in other areas. And, like you mentioned, that we’re going to see an acceleration despite the need to maybe take a slowdown to take a step back and look how we’re developing things.

And, Martin, I know we’ve spoken in conversations outside of the podcast that AI oversight committees, this might be something that is coming. So can you talk a little bit more about what that means? How can we develop, or how do you anticipate ethical AI initiatives to spring up as this investment and this acceleration continues?

Martin Garner: Well, yes, of course. And you’re right, we touched on it before. It’s a big subject and we could talk all day about that. We’re not going to. So the short answer is, yes, there’s going to be a lot more of that. And I think you already said it, Christina, that AI does have the potential for many, many good uses in society. But also used wrongly it has the potential to do a huge amount of damage, and it’s a bit like medicine, where regulated drugs are generally good for society and unregulated drugs like opioids and fake Wegovy and things, generally not so good.

And the big difference with medicine is that there’s no professional body, there’s no Hippocratic oath. You can’t be struck off as an AI practitioner, at least not yet. And so at the moment instead we have the opposite, where the AI leading companies, they seem to be on a big push. As soon as something new is developed they open source it and push it out into the world as fast as possible. And that obviously puts a huge imperative on suppliers and developers to take their own ethical stance in how they use it—which customers are they going to deal with or not deal with, how to train their staff, internal governments. There’s lots to get right there. But it also puts an onus on companies who are using AI as customers, and they need to step up too.

And so we do have a prediction, which is that AI oversight committees become commonplace in large organizations in this coming year, 2024. And those are likely to be committees of ethics experts, AI experts, legal advisors, data scientists, HR, representative from the different business units, and so on. And they’re going to have to review the use of AI across the company and in the company’s products. And their job really is to bridge the gap between the tech teams who are all engineers—they’re not typically ethicists—and the organization and its goals and what it wants to do with it.

And we think that’s going to be quite a significant overhead for a lot of companies. Quite difficult to get right. Lots of training to come up to speed and to stay on top of it, because it’s all moving so fast that the committee will have to move fast too. And all that because the AI industry is largely not doing a good job of self-regulation.

Christina Cardoza: One thing I’m curious about is the EU is putting together the AI Act, and I’m wondering what will this mean for developing solutions with the GDPR? You know that that was in the EU, but that was a global impact. So do you anticipate the same thing happening when the AI Act has passed?

Martin Garner: I think probably Bola is the better one to talk on that, if I may. Can I hand that one to Bola? I know you’ve been looking—

Bola Rotibi: Yeah, no. Yes, I think definitely actually, because at the end of the day I do think that the regulators are coming together. The EU has been first out the door for starters with the EU AI Act. But when the act does come in, it will be like the GDPR Act. And I think what will happen, and we’ve already seen the ratification of the Digital Markets Act, and we’ve already seen the effects of that with some of the companies who are being highlighted are still trying to get bedded in into whether they are the main gateways.

But I think when it finally does come through, I think there will be sort of a bedding-in process as people try to get used to it, try to understand what it means, what all the constructs, see all of the rules and regulations. So there will be kind of like teething problems, but I think it will actually, it will become a thing, a regulation for people to rally around.

But the EU is not the only one. We’ve got the acts in the US, and we’ve also got the UK pushing to having a really strong play with AI regulations. And then we’ve got China and in other regions as well. So I think we’re going to start seeing some level of improvement in terms of—towards the end of 2024, certainly, as people rush out to see the frameworks regulation. And I think that will be really important.

The other thing that’s actually happening, and it’s not just about regulation at the international level and the national level, but it’s also what the industry is doing. And I think there’s some really exciting things that have been happening. Recently IBM and Meta and Intel amongst—as part of 50 organizations that have launched the AI Alliance, which is aimed to bring the industry together to work on collectively—like any other body—to standardize, like working groups, to bring working groups together, to come up with ideas for strategies and approaches to handling certain AI challenges and opportunities, to be the home or the hub for interactions between end users as well.

Christina Cardoza: I love seeing all of these AI efforts, especially from big tech companies, because AI has a lot of benefits, it has a lot of opportunities and ability to transform industries and transform our lives for the better. But there is that level of uncomfortability from end users—how their privacy is going to be used, how secure the data is, what’s it going to do, how it really is going to transform their lives. So I think this helps put a safeguard and minimize some of the risks that people think that are out there.

And I’m curious—because obviously we can’t wait for some of these acts or some of these efforts to be passed or to take off before we start developing in a more ethical way—so I’m just curious, Bola, you work with developers. How can they continue to develop these AI solutions with privacy and ethics in mind and ensure the safety and security of their solutions and of the end users?

Bola Rotibi: Well, I mean it’s a good one actually, because I think invariably what’s going to happen is that I think lots and lots of—how to say—there’s likely to be oversight committees within organizations. That’s something that we’ve actually put down in the prediction of ours. But I think one of the things is that there will be communities that will work within organizations to kind of understand what it is that they can do. Many of the tools that they are actually using now and many of the providers are building in that responsible AI, ethical AI, from the ground up.

What developers have to then start thinking about—and this isn’t just on the developers, because actually at the end of the day a developer might think, “Well, actually, I’m building code, I’m generating—I’m building a code, I’m writing an application.” But at the same time, in the same way, that security is actually down to everyone. Everyone in that workflow. So is responsibility—responsible AI and an ethical approach. It’s down to everyone.

So I don’t see it’s just a developer requirement, but at the same time you have to say there needs to be frameworks in place, and that needs to be driven around the whole organization. It needs to come from the ground up. It needs to go from the top down. So there needs to be some principles that are used and distributed and circulated across the organization. And there needs to be some sort of guidelines.

Now, the tools are coming with guidelines and guardrails, right? In terms of, “Okay, how do I prevent certain things happening?” Because you can’t expect anyone who’s always developing to have everything in their heads about, “Oh, okay, is this ethical?” Of course, you could ask yourself, “Well, just because I can do it, should I do it?” You know? And that’s always the case.

But, at the same time, if you want a level of consistency, we need to have, provide, those guidelines and to the organization, to the development organization, right across the board. But there needs to be guardrails. And I think many of the tools are recognizing this so that they can actually allow organizations of any size to put their own policies in.

So I see going forward that there will be a layered approach. So that may be an oversight committee within the organization that thinks about what it is that the organization is from an ethical standpoint, from a responsibility standpoint, and start building policies. And those policies will be driven and put into the tools so that they act as guardrails. But there’s also going to be guidance and training of developers in terms of taking an ethical approach, taking a responsible-AI approach.

So I think, going forward, it’s a bit like how do you do right, knowing right from wrong? But this is something that the development community already is aware of. There’s a lot of things out there, a lot of programs, a lot of initiatives out there, things like aiming for doing code for good. Lots of organizations have been thinking about impact, about sustainability, and all those kind of things. So there is a body, sort of a body, a wealth of already ideas and initiatives to make the people think at multiple levels, not just about responsible AI, but doing the right thing, thinking about sustainability. So I think the approach is actually is already there for many developers, but they do need help.

Christina Cardoza: Yeah, and I love what you said: it’s not—one camp is not responsible for this. It’s really a collaborative effort from everybody building it, everybody using these types of solutions, everybody touching AI. So I love that.

We are running a little bit out of time, but before we go I just want to change the conversation to a different topic real quickly while I have Martin on the podcast, because Martin, we’ve talked over the last couple of years: We’ve been talking about 5G. Are we looking to 6G? Where are we in the 5G efforts? Is it still beginning? So I’m just curious, as we’re talking about AI and Edge, where 5G fits into all of this, what the impact of AI is going to be on f 5G networks, and when is it time to start looking at 6G?

Martin Garner: Yeah, 6G, I love that. We’re not, we don’t quite have all the good bits of 5G yet, do we? They’re coming, but they’re not quite here yet. So—but there is work going on 6G.

So in terms of AI and 5G I think the first thing is that organizations who are starting to use 5G in their factory, in their warehouse, in private networks up and down the country and so on—one of the things 5G will do is enable a lot more use of AI thanks to the very high capacity, the time-sensitive networking location services it will bring in. And we’ll see a lot more AI in use around those domains. A lot more autonomous vehicles and things. We can already see good examples of autonomous trucks used in mines in Latin America and in ports in many different continents and so on. Lots and lots more of that to come with 5G and the newer bits of 5G, which are nearly here, as one of the key sort of enablers of that.

But I think the other interesting bit is the impact of AI on the network itself. Now, there are several tricky aspects if you try to buy and use 5G. So things like coverage planning on public and private networks. Very good candidate for using AI to make that simpler so that more people can do it. Also, 5G networks, they’re complicated things. They really are a very high number of settings and things. And so the whole optimization and management is a big deal in a 5G network.

And we have a prediction around that, which is that AI enables 5G networks to move beyond five-nines availability, which anyone who’s used a cellular network will appreciate the importance of that. And that would come through by analyzing traffic patterns and ensuring that the network is set up best to handle that type of traffic, identify problems, predictive maintenance, and configure the network—if things are going to go wrong—so it has graceful degradation or even become self-healing.

And we think there’s a lot more that networks themselves can do with AI to become much better quality and really support not only people up and down the country on the public networks, but also really the OT world—which absolutely, if they use it a lot, they will depend on it, and it’ll be very expensive when things go wrong. So supporting that requires that degree of quality.

Anyway, then 6G. It is a tiny bit early for 6G, but work is going on of course. And over the next five years or so we’re going to be building 6G networks. We think 2030 is going to be a bit of a headline year for 6G. And we have a few 6G predictions. Here’s one: which is that by 2030 the first 6G-powered massive twin city is announced. And we think that cities will be a great showcase and massive twinning is one of the best use cases, because all the layers of a city could be potentially included in the model there. And the 6G network could enable a sort of full representation of the whole city environment.

We don’t think that’s likely to start with older cities—much more with a new project such as they’re doing in Saudi Arabia at the moment. And if we do that, they’re going to need 6G just for the sheer volume and speed of the data in real time that runs through a city. So that was a really good example. We think 2030, big headline year for that.

Christina Cardoza: Yeah, absolutely. And I agree, we still have to fix or finish the 5G aspects that we have there, but of course you have businesses or companies out there already trying to talk about 6G. And I agree that for all of these AI solutions and benefits and opportunities we want to take advantage of we need to make sure the infrastructure is there to make it possible first.

Martin Garner: Yeah, and working as well as it can.

Christina Cardoza: Yeah, absolutely. So this has been a very big conversation. Obviously there’s lots to look forward to. And we have only touched a little bit of the research paper, the CCS Insight IOT predictions that come out every year. We’ve only touched a small subset and this has already been such a big conversation.

So before we go I just want to throw it back to you guys. Are there any final thoughts or key takeaways that you want to leave our listeners with today? Bola, I’ll start with you.

Bola Rotibi: Well, yes, I think the one thing that I do feel that is going to become even more prominent, if people haven’t already identified it yet, which is—and we have a wonderful prediction about this—which is that proficiency in generative AI is a common feature of job adverts for knowledge workers by 2025. And I think it’s actually going to be probably that you could stretch that out to more than knowledge workers—probably for everyone.

And the reason we say that is because what we’ve seen, this swathe of recent announcements for generative AI, we’ve seen them being embedded into workplace tools such as Microsoft Office, Google Workspace—all of the different types of solutions that are out there. I mean, Martin actually mentioned earlier that pretty much you now get generative AI capability right across the board—almost any tool you open up and there’s a generative AI chat box there.

And I think the thing is we have data already showing that the actual—the more proficient you are, i.e., the better that you are in creating those prompts, which means if you’re trained as well, the better the productivity and the better the quality. And I think people are going to recognize that that is going to be a real factor to having an efficiency factor and an effectiveness factor for the workplace, workflows, and especially if you’re trying to streamline your workflows, make them more effective.

So I think that proficiency side is going to be a real big thing. So my advice is to start looking at how best to build out those prompts, and taking some sort of training support in that. It’ll follow the way of many technologies—you know, the better you are with them, the more effective that you can get them to be.

Martin Garner: And I think there is already a job title called Prompt Engineer, isn’t there? I’ve heard about this.

Bola Rotibi: Exactly, yes.

Martin Garner: And what I think you’re saying, Bola, is that that’s going to evaporate and everybody’s going to be good at it.

Bola Rotibi: Yeah, exactly. I mean, prompt engineering is going to be like, well, that’s it. Just another engineering. But the thing is is that we can joke about it, but I think really in the same way that—and it’s going to be a little bit more than just being good at search terms. I think everybody’s going to have to really sort of learn how best to construct your prompts to get the best insights and the best information out of it.

Christina Cardoza: Yeah, absolutely. It’s almost like the AI developers, you don’t really call them AI developers anymore because everyone’s developing for AI. Everyone needs to know how to develop for AI, so they’re just developers. But, Martin, anything you want to leave us with today?

Martin Garner: No, no, I think that sort of wrapped it up really quite nicely. Thank you, Christina.

Christina Cardoza: Yeah, absolutely. Well, thank you guys again. It’s always a pleasure to have both of you on the podcast, and it has been a great, insightful conversation. I invite all of our listeners to take a look at the CCS Insight 2024 prediction for IoT. We will host it and make sure that we make a link available to our listeners to check that out. And I want to thank you guys for listening to the podcast. Until next time, this has been the IOT chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel® trademark information.

This transcript was edited by Erin Noble, copy editor.

API Security Fills Critical Gap in Cyber Protection

Anytime you digitally access your bank account, make an online purchase, or log into a cloud application, you’re using an application programming interface (API). Acting as gateways among applications, APIs are the glue of digital connections. Yet they remain poorly understood, and too often unsecure.

Organizations rely on hundreds, even thousands, of APIs, but CISOs, CIOs, and CTOs often lack an accurate inventory. This explains why APIs have become a common cyberattack vector. Problems involving APIs include poor authentication practices, misconfigurations, and lack of monitoring.

API security is a huge challenge and, to a large extent, a consequence of how cybersecurity solutions are applied. Most solutions take a “horse blinders” approach, securing specific parts of the computing environment, such as endpoints, servers, and cloud applications, says Ryan B., Technical Director, Strategic Alliances at Noname, an API security company. “They’re only going to see what’s in their track and in front of their eyeballs.”

The result: a yawning security gap in IT environments that affects a company’s internal assets as well as its connections to ever-growing stacks of cloud-based and Software-as-a-Service (SaaS) applications. Noname fills that gap, using AI and ML algorithms to analyze traffic, and identify and block malicious behavior wherever assets reside—in the cloud, on-premises, or a combination of both. “We’re purpose-built for the API problem,” says Ryan.

Noname approaches the API challenge from the perspective of the enterprise, rather than that of the security vendor or the cloud provider, taking a panoramic view of the entire environment to secure all APIs.

“We find that many solutions out there do not identify malicious activity pertaining to APIs,” says Peter Cutler, Vice President, Global Strategic Alliances at Noname. “By the time they find them it’s too late. That’s why Noname integrates out-of-the-box with cybersecurity solutions such as SIEM (security information and event management) and endpoint protection.”

#API #security is a huge challenge and, to a large extent, a consequence of how #cybersecurity solutions are applied. @NonameSecurity via @insightdottech

Uncovering API Vulnerabilities with AI and ML

The most common API vulnerabilities, according to OWASP, an open-source foundation for application security, include:

  • “Broken object level authentication,” which lets a user access data based on the user’s role without verifying if the person is authorized to access the data
  • “Broken authentication,” which occurs when attackers compromise credentials such as passwords, session tokens, and user account information
  • “Broken object property level authentication,” which involves one user accessing another’s data

Malicious bots are often behind the attacks. Humans trying to counter the actions of bots can’t compete with their speed and capacity, hence the need for AI and ML.

“This is a superhuman problem to solve. All the people you’ve hired, all the technologies you’ve purchased—you thought they were the right solution to the job. You purchased firewalls, hired security consultants, ran some penetration testing. Guess what? It wasn’t good enough,” says Ryan B.

Noname fights bots with bots. In the first week of implementation at a customer’s environment, the Noname API Security Platform is in learn mode, observing the patterns of traffic moving among applications that use APIs. The platform memorizes API specs, requests, and response schemas, and looks at parameters of communications involving confidential information such as payment card data.

Starting in the second week, the security platform uses this baseline knowledge acquired in the first week to identify activities that stray from normal patterns. AI then determines if the anomalies are malicious. Suspicious activity is flagged and blocked. Noname applies a confidence score to the process. It’s based on at least 80% machine learning derived certainty that a specific action is malicious and can be traced to attackers and their known locations, Ryan B. says.

To keep IT defenses up to date, Noname uses Active Testing, a technology that simulates cyberattacks. Whenever customers make changes to their environment, this runtime testing checks if a new software version, endpoint, or other component is properly protected. This prevents introducing new vulnerabilities into the environment.

Without active testing, Cutler says, organizations launch new production APIs “with their fingers crossed that the API gateway or web application firewall (WAF), and other security layers, will identify and protect them. That is very risky and certainly not a good strategy.”

API Security Awareness Requires Performant Compute

ML and AI, of course, will continue to play a central role in API security. ML and AI require lots of processing power as traffic volumes grow. “We start out with eight CPUs for about 3,000 messages per second. Our machine learning engine then is hungry for more CPUs as API traffic scales to 7,000; 10,000; 20,000 messages per second,” says Ryan B.

Noname works closely with Intel to take advantage of the performance required to run AI and ML. The company benchmarks the 5th Gen Intel® Xeon® processor to benefit from significant performance gains. Work also is underway to leverage Intel embedded encryption to prevent malicious actors from compromising the Noname technology. 

As Noname looks into the future, the company wants APIs to be better understood. This requires education, something the company delivers as part of its mission.

“What I see, moving forward into 2024, is people are going to take security even more seriously. They’re going to buy the right tool for the job and secure their systems in a way that they haven’t even tried to before by leveraging these innovations,” Ryan B. says.

Edited by Georganne Benesch, Editorial Director for insight.tech.