Multisensory AI Revolutionizes Real-Time Analytics

Posted on by Micah Kaplan

Humans use their eyes, ears, and nose to understand their environment. iOmniscient’s AI-based analytics emulates these capabilities with video, sound, and smell analytics, all working together.

iOmniscient CEO Rustom Kanga cofounded the company 23 years ago—long before AI became fashionable. “It was different then in the sense that you didn’t have the same power in computers, so we had to design our algorithms to be much more efficient, minimizing the amount of training required,” Kanga says. This experience has been very useful in that today the company’s AI algorithms require minimal training and can operate without a GPU, significantly reducing both operational costs and time required to implement systems.

And because GPUs are very power-hungry, iOmniscient systems make the entire system more sustainable with a significantly lower carbon footprint. The company leverages the Intel® OpenVINO™ toolkit to optimize models while minimizing resource requirements and enabling use of more cost-effective hardware.

The company’s #AI algorithms require minimal training and can operate without a #GPU, significantly reducing both operational costs and time required to implement systems. @iOmniscient1 via @insightdottech

Delivering Outcomes with Industry-Specific Packs

With more than 70 international patents, the company offers many unique capabilities—such as being able to understand behaviors in crowded and complex scenes—but iOmniscient’s focus is not on selling products. Rather, it understands its customers’ problems and ensures outcomes. It has developed algorithms to address more than 300 use cases, which can be used together in different permutations to solve many industry-specific needs. Based on these, it has put together comprehensive industry packs for industries as varied as retail, railways, and intelligent traffic management. Today the company’s products have been implemented in 70 countries across 30 different industries.

For many industries, iOmniscient’s comprehensive industry pack solves a variety of problems.

Consider the retail industry, where management is interested in understanding the demographics of their customers and how long they spend in different parts of the store. Retailers may offer loyalty programs to provide their customers with special offers specific to their known interests. When shoppers move through the store, digital advertising displays can reflect the interests of people approaching the display based on their demographics.

In buildings, the iOmniscient system can provide gateless access management without the need for a database. And in a manufacturing plant, it can help with quality control and predictive machine maintenance.

Autonomous Response

“We realized very early that the real challenge was not just to generate information about what was happening in an environment,” said Ivy Li, cofounder of iOmniscient. “Our customers require systems to act autonomously to solve their specific problem.”

Take, for example, a typical incident in a public place like an airport or railway station. The iOmniscient system can detect an abandoned parcel even in a crowded location. The person who left it can be tracked anonymously. When they are located, the system can find the nearest appropriate responder, provide them with a video of the incident on their phone and tell them where to go and what to do.

So rather than being focused on a single detection algorithm, the system dynamically combines multiple algorithms to achieve the action required, which in the above case was to guide a first responder to handle the incident.

This Autonomous Response, another internationally patented iOmniscient capability, can reduce the response time for an incident by around 80%. In a traffic accident situation, such a fast response can be life-saving.

Solving Complex Problems at Lower Cost with Multisensory AI

“Our customers tend to be price-sensitive,” Kanga explains. “We’ve designed our solution such that it requires 90% less storage, 90% less network bandwidth, and less computing than today’s GPU-based AI systems, and it can work with existing low-resolution cameras.”

So while the company continues to build “best in class” solutions for its customers, it is also focused on doing it at minimal cost, which is achieved by designing the system to minimize its infrastructure requirements. The result is an intelligent multisensory analytics solution that can usually be installed for less than the cost of a video recording system.

 

This article was edited by Teresa Meek, Contributor for insight.tech.

Few-Shot Learning Speeds AI Model Training and Inferencing

Posted on by Micah Kaplan

To increase profitability, optimize production, and succeed in a highly competitive business landscape, manufacturers increasingly turn to computer vision technology. But developing solutions that work on the factory floor is an extremely complex and time-consuming process.

That’s because the AI algorithms that computer vision models use to assemble products or spot defects in parts and machinery require heavy training. While pre-trained models are available, they are almost never accurate enough for deployment. And training custom models usually requires huge data sets, skilled workers to guide the training, and months of effort.

It’s a serious development bottleneck for manufacturers looking to implement vision-enabled solutions, and it hampers digital transformation in the sector. But a new approach to AI model training called “few-shot learning” may hold the key to deploying AI solutions much faster.

How Few-Shot Learning Speeds AI Model Training

To see why few-shot learning is such a game changer, it’s useful to understand how computer vision models are usually developed.

Typically, a custom AI model begins with a pre-trained model. Take, for example, a picking use case in an assembly line setting. A development team might start with a generalized computer vision model for object recognition. But that model wouldn’t be able to identify specific components used by the company. To make it accurate enough for deployment, developers often take an approach called “supervised learning,” providing the AI model with annotated training data to help it learn what a particular part or defect can look like and differentiate it from another.

But this task often requires thousands of images. In supervised learning scenarios, the images must also be labeled by domain experts (“this is a widget, this is not”) to teach the model what it needs to know. It’s an expensive, labor-intensive task, because skilled employees must first annotate the images and then tune the model’s hyperparameters over multiple rounds of training.

“Even under the best of circumstances, supervised learning can involve many hours of skilled labor and take months to complete,” explains Lu Han, an Algorithm Researcher at Lenovo Group, a global computing intelligence company.

In some scenarios, supervised learning may not be feasible. For instance, if a manufacturer needs to train a model to spot a new type of part defect, there may not be enough images of faulty parts to use in customizing the model.

Few-shot learning overcomes such challenges by taking a different approach. The “shot” in “few-shot learning” refers to the number of examples of a type of object that a model is given during the training process.

In this process, the AI may be taught to identify the degree of similarity or difference between objects in general. This capability can then be applied to matching a never-before-seen object to a small number of reference examples. For a simplified instance, a model might be given labeled images of nuts, bolts, and screws, with two examples from each category, plus a test image of a bolt, and then be asked to predict which of the three object categories the test image of the bolt most nearly resembles.

Few-shot learning requires far fewer labeled images to customize an AI model—just a few dozen instead of several thousand—and generally takes days or weeks instead of months. The result is a greatly simplified AI development workflow that already helps companies deploy AI solutions more quickly than ever before.

A Few-Shot Learning System for Defect Recognition

Case in point: Lenovo’s defect-recognition implementation at a textile manufacturer.

For quality assurance, the manufacturer had to be able to identify more than 80 different types of surface defects in the textile products they made. Manual inspection did not achieve the desired level of quality control.

The company hoped to develop an AI-powered defect recognition solution but faced several prevalent implementation barriers. Customized model training would be difficult because they had very few defect samples to work with. In addition, they made a wide variety of products using the same production line, so any AI solution would have to be able to quickly update its models at the edge as products and materials changed. Using a traditional training approach, it would take an estimated six to 12 months to construct a working AI solution—an unpalatable time frame, and one that augured poorly for future iteration.

Lenovo, a leader in smart manufacturing, has incubated the Lenovo Edge AI direction within its research division, Lenovo Research. It has strong technical capabilities to solve the above-mentioned difficulty. Working with the manufacturer, Lenovo EdgeAI developed an end-to-end computer vision defect recognition solution using few-shot learning techniques. The initial training and local updates of the model were completed in just one week. The accuracy was impressive: zero missed detections in key items.

The result is a greatly simplified #AI development workflow that’s already helping companies deploy AI solutions more quickly than ever before. @Lenovo via @insightdottech

To reduce latency and enable local management, Lenovo ran the AI inferencing workloads on an edge industrial personal computer (IPC). This enabled near real-time switching between the different AI models used for various product types. It also allowed factory quality assurance workers to retrain models on-site to accommodate for future product modifications or newly appearing defects.

Lenovo credits its technology partnership with Intel in helping to deploy the solution effectively. “Our system uses Intel chips, which provide powerful computing resources in the kind of edge scenarios needed by our customers,” says Han.

Lenovo also used the Intel® OpenVINO™ toolkit. Han says, “Our inference engine is widely compatible with various toolkits, especially OpenVINO. For inferencing, running on Intel silicon can support around 20 edge AI models. Many of our customers prefer devices built on Intel chips, helping us bring this solution to market more quickly.”

Accelerating Industry 4.0 with Few-Shot Learning

Because few-shot learning is an effective strategy for speeding AI solutions development, it will likely become an attractive choice for solutions developers, systems integrators (SIs), and manufacturers. It could also help lower costs, since few-shot systems can more easily be retrained to adapt to operational changes like new-product models or never-before-seen defects.

As AI uptake increases, the industrial AI ecosystem is likely to mature as well, with SIs and solutions developers offering comprehensive service and consulting packages. Lenovo already markets its defect detection system as an end-to-end solution, with full-lifecycle service packages and subscription-based support for the software.

“Computer vision is becoming indispensable in manufacturing,” says Han. “Few-shot learning helps companies implement innovative computer vision-based solutions faster, speeding their digital transformation and realizing ROI sooner. Both computer vision and few-shot learning have a bright future in this sector.”

 

This article was edited by Teresa Meek, Contributor for insight.tech.

Digital Twins Make Building Management Greener and Smarter

Posted on by Micah Kaplan

Companies with portfolios of assets in the built environment face a frustrating problem. They want their buildings—retail stores, factories, hospitals, or other structures—to be smart. After all, a smart facility consumes less energy, creates less waste, reduces maintenance costs, prolongs the life of assets, and delivers a host of additional operational efficiencies. But the data that businesses need to deliver such desirable outcomes is difficult to access, often locked behind proprietary protocols in separate legacy systems.

Fortunately, digital twins—functional models that represent physical assets and capture their interdependencies—can solve this problem. By unlocking and uniting data across systems, they give enterprises valuable operational insights, which building owners can use to make their facilities smarter, more sustainable, and easier to manage.

A Digital Twin Platform for Legacy Asset Management

“Organizations have a tremendous opportunity to leverage technology to make their businesses and their facilities smarter,” says Dale Kehler, Vice President of Business Development at e-Magic, a provider of digital twin solution TwinWorX®. Liberating asset-driven data from silos enables enterprises to make data-driven decisions instead of relying on unreliable parameters or gut intuition.

TwinWorX® does that by creating a digital model of physical assets and tracking their usage and interdependencies. As a result, enterprises can monitor and control their equipment through digital equivalents. TwinWorX® is also vendor-agnostic, aggregating live telemetry data from disparate assets onto a single platform. That means enterprises no longer have to work with 10 different software products for 10 different assets; they can simply view all relevant information through a single pane of glass.

Critically, digital-twin solutions like TwinWorX® capture the interdependencies among assets. By connecting thousands upon thousands of data points across systems, TwinWorX® enables asset managers to easily see how changing one variable would affect others. For example, dialing up the temperature setting in a building can reveal the effects it would have on the longevity of the furnace, carbon emissions, and energy costs. “Because there’s contextual data to support what you’re seeing, digital twins represent a new way of modeling data,” Kehler says.

Live telemetry data from assets doesn’t just flow one way. “By having bidirectional communication with the assets, you can actually control them from the digital twin,” Kehler says. The digital twin can change settings such as temperature, gas flow, or any other accessible parameter that enterprises want to manipulate.

With real-time data feeds, companies can receive notifications about operational anomalies that need immediate attention. And gathering all relevant data in one digital twin platform allows them to apply technologies such as machine learning and AI—not just to understand what’s happening now, but to foresee problems coming down the pike, and in some cases, apply predictive maintenance to prevent breakdowns.

Digital Twin Solutions Applications Range Across Industries

Digital twins have a wide array of applications depending on the setting, which can range from a single office tower to a sprawling manufacturing plant or medical complex.

TwinWorX® is most often used to centrally manage assets across multi-building environments, Kehler says. For example, Temple University wanted to create one centralized platform where it could view data from all building management systems across the campus. Because of the vendor-agnostic legacy asset management system TwinWorX® provides, the university now saves time by managing data in one place. It also saves money by avoiding redundant licensing and analytics tools.

#DigitalTwins have a wide array of applications depending on the setting, which can range from a single office tower to a sprawling #manufacturing plant or #medical complex. @eMagic02 via @insightdottech

Centralizing data and shaking off expensive vendor contracts, as Temple did, are early steps in the journey toward a smart environment, Kehler says. A digital-twin solution answers the question many building owners have: How can we architect our systems strategically and set ourselves up to manage a smart environment? With TwinWorX®, companies can start small, connecting a single asset or just a few, adding more as their goals become more ambitious.

Companies often start by seeking ways to lower energy bills and make their operations greener. “A lot of companies are dealing with rising energy costs and are addressing decarbonization efforts in order to meet sustainability commitments,” Kehler says. “Gaining visibility into how energy is used provides opportunities to find cost savings through reducing energy use and mixing energy composition.” Digital-twin solutions can show companies how using one kind of renewable energy—or a mix of renewable and traditional energy sources—will affect their overall sustainability profile.

The TwinWorX® platform runs on Intel® servers and processors, and e-Magic often recommends Intel® processors and IoT systems for customers’ edge computing implementations, which require low latency. e-Magic provides the digital-twin platform and connectivity tools for legacy asset management. Its systems integration partners provide additional advisory services for setting up specialized functions, such as carbon accounting.

The Future of Legacy Asset Management

While it’s still early days for digital twins, Kehler expects adoption to ramp up in the near future, especially for companies addressing sustainability goals, as legislation forces building owners to realize energy efficiencies quickly.

As implementations increase, digital twins may someday help to create entire ecosystems of interconnected systems, with built environments and smart cities working synchronously toward a common set of efficiency goals.

 

This article was edited by Teresa Meek, Contributor for insight.tech.

Leverage In-Store CCTV Systems for Smart Retail Solution

Posted on by Micah Kaplan

When you shop online, you leave behind a trail of digital breadcrumbs for retailers to analyze. By tracking your mouse clicks and even abandoned shopping baskets, retailers can tell which products you love, which you ignore, and which ones to bring back to your attention through targeted ads.

While e-commerce retailers can harness such shopper insights, their brick-and-mortar counterparts have historically not been able to do so as easily. Charting the customer journey from when a buyer is drawn to a product display to the final sale has been difficult within the walls of a physical store.

But thanks to computer vision analytics (CVA), retailers can now use the CCTV systems they might already have to access data about the in-store shopper. By bringing more precise journey tracking to brick-and-mortar, these technologies empower retailers to make better decisions about merchandising displays, product assortment, and staffing to improve sales.

#Technologies empower #retailers to make better decisions about merchandising displays, product assortment, and staffing to improve sales. @johnsoncontrols via @insightdottech

Harnessing CCTV Systems for Computer Vision Analytics

The most frictionless approach to shopper data access in a brick-and-mortar location is to leverage the store’s existing technology infrastructure whenever possible, says Dustin Ares, General Manager of Video Analytics, AI and Incubation at Sensormatic Solutions, a provider of digital solutions for the retail industry.

In-store CCTV systems can track shopper behavior so retailers can tell which products are noticed or ignored, and how long it takes to make a sale. Sensormatic Solutions CVA piggybacks on CCTV units, using them as camera sensors in an edge inferencing system. The company feeds this camera data to proprietary AI algorithms that conduct real-time inferencing to provide actionable insights about on-the-floor events. “The edge device allows us to process data, make very quick low-latency decisions and then notify the store associate or manager to take a specific action,” Ares says.

That “specific action” may include reassigning staff to new areas of the store or restocking certain items, depending on shopper traffic and interest. In a few cases, through a headset, a retail associate can receive immediate instructions on how and where to work with customers to nudge a sale.

The very same camera-driven analytics can help retailers address another major pain point: theft. “Red” shoppers are potential shoplifters who often behave differently than their well-intentioned, “green” counterparts. They may pick up items at random or constantly scan the store for associates—and such visual cues from the camera can help nudge associates.

Smart Retail Solutions and Analytics

Retailers who are worried about using cameras to conduct analysis of shopper behavior can access analytics while abiding by privacy and consumer data protection laws like Europe’s GDPR. Sensormatic Solutions models are built with a privacy-by-design mindset; they don’t retain or process privately identifiable information, processing only metadata at the edge.

The company is also conscious of the potential for bias in AI models. “We vet our models rigorously to ensure that we’re not propagating bias, especially if we’re doing demographics evaluation,” Ares says. Sensormatic Solutions regularly checks in with its parent company, smart building solutions provider Johnson Controls, to keep abreast of global regulations and compliance in the field. “We want to make sure that we’re not only compliant but also thinking ahead about our product and where we think the [trendlines] are headed in the future,” Ares says.

A Variety of Smart Retail Use Cases

While understanding shopper behavior demographics and loss prevention are significant retailer concerns, they are by no means the only issues on the industry’s mind.

A European chocolatier used Sensormatic Solutions CVA to test-drive the efficacy of a central store display. The merchant discovered lower customer engagement than expected, which led it to route associates closer to the display to improve associate-customer interactions.

In South Korea, Sensormatic Solutions helped a convenience store chain develop an inventory management solution. The retailer was wasting staffing resources by having employees constantly checking inventory levels on fresh-food shelves. Installing camera sensors not only gave accurate information about inventory, but also enabled the retailer to proactively predict when levels might run low so they could take necessary preemptive action.

Sensormatic Solutions is itself a systems integrator. The company has relationships with many channel partners in the industry, especially providers of security solutions for retailers. The company can use in-store systems as is and suggest additional camera units depending on the metrics that need to be measured.

Sensormatic Solutions’ smart retail technology portfolio runs on Intel hardware, specifically the Intel® Core™ 13th Gen processors and the most recent Intel® Core™ 14th Gen processors. Intel’s global availability has been an asset for the company, which also has a global footprint.

The Future of Data-Driven Retail

Expect Sensormatic Solutions to deliver more holistic solutions that address a wider swath of retail-related challenges. Ares predicts the company will continue to evolve as a retail data expert, delivering insights not just for individual clients but also providing parameters and analytics for industry-wide benchmarking.

With omnichannel shopping and multiple outlets competing for consumer eyeballs, retail is as complex an endeavor as it has ever been. Fortunately, even brick-and-mortar stores can use CCTV systems to leverage insights and make business less complex. Data-driven analytics that underlie smart retail solutions can help retailers of all stripes achieve an impressive set of efficiency goals.

 

Edited by Georganne Benesch, Editorial Director for insight.tech.

API Security Fills Critical Gap in Cyber Protection

Posted on by Micah Kaplan

Anytime you digitally access your bank account, make an online purchase, or log into a cloud application, you’re using an application programming interface (API). Acting as gateways among applications, APIs are the glue of digital connections. Yet they remain poorly understood, and too often unsecure.

Organizations rely on hundreds, even thousands, of APIs, but CISOs, CIOs, and CTOs often lack an accurate inventory. This explains why APIs have become a common cyberattack vector. Problems involving APIs include poor authentication practices, misconfigurations, and lack of monitoring.

API security is a huge challenge and, to a large extent, a consequence of how cybersecurity solutions are applied. Most solutions take a “horse blinders” approach, securing specific parts of the computing environment, such as endpoints, servers, and cloud applications, says Ryan B., Technical Director, Strategic Alliances at Noname, an API security company. “They’re only going to see what’s in their track and in front of their eyeballs.”

The result: a yawning security gap in IT environments that affects a company’s internal assets as well as its connections to ever-growing stacks of cloud-based and Software-as-a-Service (SaaS) applications. Noname fills that gap, using AI and ML algorithms to analyze traffic, and identify and block malicious behavior wherever assets reside—in the cloud, on-premises, or a combination of both. “We’re purpose-built for the API problem,” says Ryan.

Noname approaches the API challenge from the perspective of the enterprise, rather than that of the security vendor or the cloud provider, taking a panoramic view of the entire environment to secure all APIs.

“We find that many solutions out there do not identify malicious activity pertaining to APIs,” says Peter Cutler, Vice President, Global Strategic Alliances at Noname. “By the time they find them it’s too late. That’s why Noname integrates out-of-the-box with cybersecurity solutions such as SIEM (security information and event management) and endpoint protection.”

#API #security is a huge challenge and, to a large extent, a consequence of how #cybersecurity solutions are applied. @NonameSecurity via @insightdottech

Uncovering API Vulnerabilities with AI and ML

The most common API vulnerabilities, according to OWASP, an open-source foundation for application security, include:

  • “Broken object level authentication,” which lets a user access data based on the user’s role without verifying if the person is authorized to access the data
  • “Broken authentication,” which occurs when attackers compromise credentials such as passwords, session tokens, and user account information
  • “Broken object property level authentication,” which involves one user accessing another’s data

Malicious bots are often behind the attacks. Humans trying to counter the actions of bots can’t compete with their speed and capacity, hence the need for AI and ML.

“This is a superhuman problem to solve. All the people you’ve hired, all the technologies you’ve purchased—you thought they were the right solution to the job. You purchased firewalls, hired security consultants, ran some penetration testing. Guess what? It wasn’t good enough,” says Ryan B.

Noname fights bots with bots. In the first week of implementation at a customer’s environment, the Noname API Security Platform is in learn mode, observing the patterns of traffic moving among applications that use APIs. The platform memorizes API specs, requests, and response schemas, and looks at parameters of communications involving confidential information such as payment card data.

Starting in the second week, the security platform uses this baseline knowledge acquired in the first week to identify activities that stray from normal patterns. AI then determines if the anomalies are malicious. Suspicious activity is flagged and blocked. Noname applies a confidence score to the process. It’s based on at least 80% machine learning derived certainty that a specific action is malicious and can be traced to attackers and their known locations, Ryan B. says.

To keep IT defenses up to date, Noname uses Active Testing, a technology that simulates cyberattacks. Whenever customers make changes to their environment, this runtime testing checks if a new software version, endpoint, or other component is properly protected. This prevents introducing new vulnerabilities into the environment.

Without active testing, Cutler says, organizations launch new production APIs “with their fingers crossed that the API gateway or web application firewall (WAF), and other security layers, will identify and protect them. That is very risky and certainly not a good strategy.”

API Security Awareness Requires Performant Compute

ML and AI, of course, will continue to play a central role in API security. ML and AI require lots of processing power as traffic volumes grow. “We start out with eight CPUs for about 3,000 messages per second. Our machine learning engine then is hungry for more CPUs as API traffic scales to 7,000; 10,000; 20,000 messages per second,” says Ryan B.

Noname works closely with Intel to take advantage of the performance required to run AI and ML. The company benchmarks the 5th Gen Intel® Xeon® processor to benefit from significant performance gains. Work also is underway to leverage Intel embedded encryption to prevent malicious actors from compromising the Noname technology. 

As Noname looks into the future, the company wants APIs to be better understood. This requires education, something the company delivers as part of its mission.

“What I see, moving forward into 2024, is people are going to take security even more seriously. They’re going to buy the right tool for the job and secure their systems in a way that they haven’t even tried to before by leveraging these innovations,” Ryan B. says.

Edited by Georganne Benesch, Editorial Director for insight.tech.

Navigating the Generative AI Solutions Landscape

Posted on by Micah Kaplan

By now, almost everyone has heard of generative AI and its potential to revolutionize many industries. But does the hype align with the reality? As businesses search for ways to harness its power and gain a significant competitive edge, there are many questions: Which current generative AI opportunities can businesses start to take advantage of today? And what limitations do we still need to overcome?

In this podcast, we dive into key benefits as well as challenges of building generative AI business solutions. We also talk about how to successfully build and deploy solutions that can help businesses take their efforts to the next level.

Listen Here

[Podcast Player]

Apple Podcasts      Spotify      Amazon Music

Our Guests: Accenture and Anyscale

Our guests this episode are:

Podcast Topics

Teresa, Ramtin, and Waleed answer our questions about:

  • (3:03) The rise of and interest in generative AI
  • (7:09) Considerations for deploying generative AI solutions
  • (12:10) Taking generative AI out of the experimentation phase
  • (14:34) Biggest opportunities and use cases for generative AI
  • (16:47) The importance of partnerships and collaboration
  • (25:20) What’s next for generative AI
  • (27:32) Final thoughts and key takeaways

Related Content

For the latest innovations from Accenture and Anyscale, follow them on:

Transcript

Christina Cardoza: Hello and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Editorial Director of insight.tech, and today we’re going to be talking about generative AI trends and opportunities with Waleed Kadous from Anyscale, Teresa Tung from Accenture, and Ramtin Davanlou, also from Accenture. But before we get started, let’s get to know our guests. Teresa, I’ll start with you. What can you tell us about yourself and Accenture?

Teresa Tung: I’m our Cloud First Chief Technologist, where I have the best job because I get to predict the next generation of impactful technologies. In this role I’ve gotten to work on big data, edge computing, AI, and lately you can bet it’s generative AI almost all the time.

Christina Cardoza: Great. That’s not an easy job you have there—having to predict what’s coming next or stay on top of the trends. But excited to get a little bit in to this conversation with you.

Before we go there, Ramtin, you’re also from Accenture. But what do you do there, and what can you tell us about your role at the company?

Ramtin Davanlou: Sure. Thanks a lot for having me, Christina. I’m a Director from Accenture Data and AI Group, based out of San Francisco, and I also serve as the CTO in the Accenture and Intel partnership, where we have a big focus on scaling GenAI applications and solutions with Intel hardware and software.

Christina Cardoza: Yeah, absolutely. I know Intel, the hardware and the software—they’re powering a lot of the use cases and the opportunities that we’re going to get into in just a moment.

But before we get there, Waleed, last but not least, what can you tell us about yourself and Anyscale?

Waleed Kadous: Yeah, so my name’s Waleed Kadous. I’m the Chief Scientist at Anyscale. Anyscale is a company that makes it easy for companies to scale up their machine learning infrastructure, with a special focus on scaling up generative AI—both the training, fine tuning, and inference stages. And we help our customers to deploy applications that run at huge scale.

We’re the company behind an open source project called Ray that’s very popular for doing distributed computing. It’s used by OpenAI and Cohere to train their models. So, yeah, it’s a really exciting job I have helping customers to work out how to deploy really cutting edge technologies, including IoT.

Christina Cardoza: Yeah, absolutely. And I can imagine open source—that behind a lot of the use cases it’s helping developers and businesses get started, and just the community out there to work together and build upon some of the successes that we’re seeing in that community.

So, like Teresa mentioned, generative AI—it seems to be the next big thing. We keep hearing about it; at all the latest conferences we’re talking about generative AI. A lot of it is around ChatGPT, so we’re not exactly sure—it’s a new thing, not exactly sure what this is. So, Ramtin, I’d love to start there, if you can give us a little explanation about, when we’re talking about generative AI and especially the opportunities out there for businesses, what do we—what is this idea of generative AI, and what are really the opportunities businesses are looking for?

Ramtin Davanlou: Yeah, yeah, sure. I think everybody has used generative AI or knows about it by now—unless they’ve been living off grid in the past year and this is the first content that they watch after they come back. But I think I can just put it in terms of its potentials and a summary of what is this basic entity that we have created. And it’s like—this is especially true when, over Thanksgiving dinner, all the conversation is centered around generative AI. So that’s really, really popular, and everybody is trying to make sense of it.

And so what a lot of companies like OpenAI, Google, AWS, and many more were able to do was they use their massive compute resources and massive data sets—a majority of the texts that you can find on the internet or on specific subjects—and they train these AI models—aka large language models—capable of generating new content. And this content is in many different forms. It is text, images, video, voice, or even code, computer code. And text is especially important because that’s what all humans basically do in communication, right?

So it boils down to text, and many of these AI models are able to generate responses that are really good on any given topic—better than an average person or even an average expert on that topic, right? And that creates a lot of new opportunities. Companies can now take these models, fine tune them a little bit so that the model behaves in certain ways and gains knowledge more about a specific context. And if you think about most of what white collar labor is doing every day at work—the conversation that we are having right now, it is basically text that we are generating, right? And that is the means of communication and building net new knowledge.

So what these LLMs cannot do now, but may soon be able to do, is creating that net new knowledge. But they can do everything that has been created, and that poses an opportunity for us to just focus on that net new value that we can generate. So the current workforce will focus on creating net new things.

And companies are thinking about how to use this entity to kind of enhance or substitute a lot of things that we are doing these days—sending emails, creating slides, right? All of the content that you’re creating to be able to communicate with each other. So this has huge implications for service industries and for manufacturing when you combine it with robotics. And at Accenture we are helping a lot of our clients to reinvent themselves using this transformative technology.

Christina Cardoza: Yeah. You mentioned the manufacturing spaces and other industries: it’s exciting when you have a capability like this that can really help transform all industries, and everybody’s using it in different ways. And I love how at the beginning of your response you talked about how it was dominating the Thanksgiving conversations much better than politics. But I’m sure—you know what? You can find, having these everyday conversations about this, is there comes a lot of misconceptions. A lot of interest comes with a lot of misconceptions.

So, Waleed, since you’re working in the open source world, you’re working with a lot of developers and people who are looking to develop, deploy, and manage these types of applications. Curious what you think. What are some of the considerations out there that businesses should be thinking about when developing these solutions? What are some of the misconceptions you’re seeing out there?

Waleed Kadous: Yeah. So in terms of the things that businesses should consider, I think one of them is to consider the quality of the output from these models. And in particular there’s a problem with LLMs called hallucination, and that’s where they may—they confidently assert things that are completely untrue. Now, over the last six months we’ve seen developments in an area called retrieval augmented generation which helps to minimize that. And we can talk a little bit about that, but making sure that the quality of the responses is accurate is one of the key considerations.

A second consideration is data cleanliness and making sure what is the information that these LLMs have access to? What are they disclosing, and what do they have the power to inform people of? Is there leakage between different users? Can someone reverse engineer the data that was used to train these models from the data themselves? It’s still a new frontier, and we’re still seeing issues that crop up in that front.

And then the final one is LLMs are expensive. I mean, really expensive. If you naively go and use GPT-4, which is probably the leading LLM out there today, you can easily blow a hundred thousand dollars in a month. So you really have to think about the cost and how you keep the cost under control.

And finally, there’s the final thing I would say is evaluation. How do you make sure that the system is actually producing high-quality results? What base data are you using to ensure that it’s doing the right thing? And that’s an area—I mean, the other misconception that we sometimes see people have is there’s this mechanism called fine tuning: let’s just fine tune and solve the problem. There’s a lot more nuance to solving problems with LLMs than we see, but there’s also a lot of potential applications.

And if I try to think about the applications, often when we talk to our users, they’re like, “Well, I can’t conceptualize what I need in terms of using an LLM.” And so we’ve worked out: these are the top things to look for in terms of use cases. The first one is summarization. Are there areas where you have a lot of information that you can condense, and where condensing it is useful? So we have a company called Merlin that provides a Chrome plugin that summarizes data.

The second is the retrieval-augmented-generation family. And that’s situations where you don’t just naively ask the LLM for questions; you actually provide it with context that helps answer the questions. So, for example, if it’s a customer support application, you wouldn’t just ask it about the customer support. You have an existing knowledge base of answers to questions that you pull the data from, and then say, “By the way, Mr. LLM or Mrs. LLM, here are the data points that you can use to help answer this question.”

I think one of the most interesting applications was what you might call “talk to the system” applications—especially interesting in IoT. So, imagine this as kind of a dashboard on steroids, a dashboard you can talk to. And I’ve seen a company that does expertly, that does Wi-Fi installations across companies for retailers, and what you can do is you can ask it questions, like, “Hey, which areas are we seeing the routers working too hard in?” And it will go, and it will query that information in real time and give you an update. And I really think that model is kind of the most interesting one for IoT.

And then the final one is really this in-context development and in-context application development. Perhaps the best known one of those is Copilot, right? When you’re writing your code, as Ramtin was talking about, it would give you suggestions about how to write even better, higher-quality code. And we’ve seen some of our companies deploying that. And, roughly, that order is the order of difficulty. In-context applications are the most difficult, but they’re also the highest potential.

And one thing I’d especially like to highlight is the idea that I know a lot of people are not quite sure: is this hype, or, is this not real? The really interesting thing to me is already we’re seeing massive proof points of the effectiveness of generative AI. We’ve seen GitHub do studies that show that it boosts the developer productivity by 55%. We’ve seen research about a Fortune 500 company that has shown that it can boost customer support quality and responsiveness by 14%. It’s just like, this is not kind of a hype thing. There’s already, even at this early stage, some incredible proof points that this works. But we’re still at the genesis of this technology, and we’re still collectively all learning how to use it.

Christina Cardoza: Yeah, absolutely agree. There’s always that concern: is the hype reality? Am I going to invest X amount of dollars in this and then tomorrow there’s going to be a new thing? So businesses are always making sure—are looking to future proof the investments that they are making. But, like you said, we have seen some proofs of concept, some success in the early days of experimentation. But, like you mentioned, cost is a big factor, and how can we go from those proofs of concept, experimentation solutions, and actually bring these solutions to life at production and at scale?

So, Teresa, I know Accenture does a lot of work with your end users trying to help them get to the next level of some of their efforts. So what can you tell us about where your customers are and what the next level is?

Teresa Tung: I would say, as an industry, most companies have started their proofs of concept, and many are starting with managed models like OpenAI. And these amazing general-purpose models address many use cases and offer a really great way to get started. But, as Waleed mentioned, cost in the long term is a factor, and this could be an order of magnitude bigger than many companies might be willing to pay. So, as generative AI pilots mature into production, companies now need to look at rightsizing that cost, rightsizing it for the performance, and even factoring in their sustainability goals.

When these models become more critical to the business, we’re also seeing companies want to take ownership and to control their own destiny, right? Rather than using a managed model, they might want to be able to take and create their own task-specific, enterprise-specific model. So these sub–10 billion parameter models are just more customized for them. And so what we’re seeing is companies beginning to adopt a number of models for different needs. So, yes, there will still be the general-purpose model available, but we’ll also have these fit-for-purpose models as well.

Waleed Kadous: So, to give a concrete example of what Teresa’s talking about: one of the experiments we did at Anyscale is we looked at translating natural language to SQL queries. And the general-purpose model, GPT-4, was able to produce an accuracy of about 86%, 80%. But by training a small, specific model that was only 7 billion parameters, that cost about one 100th the cost, we were able to achieve 86% accuracy in conversion. So this challenging mode of what are now being called SMSs versus LLMs—small specific models versus large language models—is kind of the evolving discussion that’s happening in the industry right now.

Christina Cardoza: Great. And we’ve been talking about these industries and these use cases at a high level, but I would love to dig in a little bit more and learn exactly how customers are using this—in what industries, where really are they finding that the biggest opportunities are, or the biggest areas that they want to start applying these generative AI solutions. So, Teresa, I’m wondering if you have any examples or use cases that you can share with us?

Teresa Tung: Yeah, I have a few. And I think Waleed had already done a great overview of some. I’m going to give a different perspective; I’m going to think about it in terms of things you can buy, things you’re going to boost, and things you’re going to build, right? So buying, being able to buy these generative-AI-powered applications for things like software development, marketing, some of these enterprise applications—that’s quickly becoming the new normal. So these applications use a model trained on these third-party data, and it gives everyone a jumpstart. And that’s the point—everyone is going to be able to capture these efficiencies, so don’t get left behind.

Boost is a second category, and this is where things like knowledge management or being able to apply a company’s first-party data—so, data about your products, your customers, your processes. And to do that you’re going to need to get your data foundation in order. And so using something like retrieval augmented generation is a great way to start, right? You don’t have to get a whole lot of data, and as you go along you can be able to create that data foundation.

And then, finally, in terms of build, we’re talking about companies being able to even maintain their own custom models. So, likely starting with the pre-trained open model and adding their own data to it. And this, again, gives them a lot more control and a lot more customization within the model.

Christina Cardoza: Great. And as we’re talking about building and boosting, deploying, managing these applications, and being able to work with large-scale models, it just comes to me that it’s—there’s a lot that goes into building these applications, and at insight.tech we’re always talking about “better together.” It’s usually not one company that’s doing this alone; it’s really an ecosystem or partnership that’s making this happen.

And, Ramtin, you talked about Intel hardware and software at the beginning of the conversation. I should mention, the “IoT Chat” and insight.tech, we are owned and sponsored by Intel, but of course it creates some of these great partnerships with companies like Anyscale and Accenture. So, I’m just curious, in terms of the hardware and the software, and the build and boost, how important it is to work with companies like Intel or any other partners that you have out there, Ramtin?

Ramtin Davanlou: Yeah, yeah, of course. I think partnerships are essentially very important in this area. Because if you are trying to build an end-to-end GenAI application and a developer ecosystem, you’re going to need a few of these suppliers, technology suppliers, to come together, right? And companies typically have to solve for a few things. This includes infrastructure and compute resources. You need a very efficient ML Ops, basically, tool to help you kind of manage this—everything you do, from development to managing and monitoring and deploying the models in production, right? And you also need a third-party software in a lot of cases, or open source software. A lot of clients are thinking about building larger platforms that could support several different use cases.

And this is an effort, like what Teresa mentioned and Waleed mentioned as well, to reduce the cost of this when you do this at scale. So instead of just using or building new platforms for every new use case, companies realize that they’re going to need this for many, many different use cases. So why not build a platform that you can reuse for all of these different cases? And this helps basically with total cost of ownership at the end of the day. And that means bringing several different technology pieces together, right?

For example, what we have built with Intel is a generative-AI playground, where we have used Intel Cloud, Intel Developer Cloud, and Gaudi tools—which is an AI accelerator specifically built for deep-learning applications, both training and—. So you can basically use GaudiTools in Intel Developer Cloud to fine-tune your models. But once you want to deploy that in scale you can go and use AWS, right? And that’s what we have done. In this playground you can basically bring in—do the development and fine-tuning of your models in IDC—Intel Developer Cloud—and then deploy at scale on AWS.

And we’ve used some of the Intel software like Converge, which is the ML Ops tool that you can use to make your data scientists and engineers collaborate in the same environment. And one of the big advantages of Converge is that it also allows you to use different compute resources across different cloud. So you can use compute resources in your on-prem environment, on Intel Developer Cloud, and on AWS all in the same workflow, right? Which is a huge advantage.

So at the time of deployment we realized that we need a tool that—or library that—helps us distribute the workloads. So if you’re getting 1,000 inferences per second, and then this has fluctuating demand—it goes up and down based on what’s going on, the time of day, and stuff like that—so you need to have a very efficient way to distribute the workloads. And we learned that this library called TGI from Hugging Face is very helpful. And that’s when you see there’s a lot of these different components and pieces that need to come together so that you can have an end-to-end GenAI application.

Christina Cardoza: Absolutely. It all goes back to the future proofing and protecting your investments, making sure that the hype is reality. I love, like you mentioned, being able to reuse some of the models or the applications and solutions you have out there—that’s always important.

So, Waleed, given that you are in the open source community and you are working with a lot of customers to make sort of these partnerships happen, I’m curious what you’re seeing. The use cases that you’re seeing, and how partners like Intel or any other partners you’re working with make those use cases a reality.

Waleed Kadous: Yeah, we’re definitely seeing a lot of interest in the different stages—the training, the functioning, and inference stages. So I think the points that Ramtin is making are valid, but one particular thing that has come up is this idea of open source. So there’s both open source models—so, for example, Meta has released a model called Llama 2 that we’ve seen very, very good results with. It’s maybe not quite on par with GPT-4, but it’s definitely close to GPT-3.5, the model one notch down.

And so there’s both open models and of course open source software. So TGI, for example, is not quite open source. It’s got a bit of a weird license, but there are systems like VLLM out of Berkeley, which is a really high-performance deployment system, as well as Ray LLMs. VLLM manages a single machine; Ray LLM gives you that kind of scalability across multiple machines, to deal with spikes and auto-scaling and so on.

We’re seeing a flourishing of the open source world in particular, because there are certain things that people like. Not everybody likes entrusting their entire data to one or two large companies, and vendor lock-in is a real concern. So we’re seeing people flock to open source solutions for cost reasons—they are cheaper—but mainly for flexibility reasons in terms of: I can deploy this in my data center, or I can deploy this in my own AWS Cloud and nobody has access to it except me.

And that flexibility of deployment and that availability of models at every size—from 180 billion down to 7 billion and below—these are the reasons that people are tending towards open source. And we’ve seen many of our customers—we did a profile of what it would take to build an email summarization engine, where if you used something like GPT-4 it would cost $36,000, and if you used open source technologies it would be closer to $1,000, for example.

And what that shows is not just that—it’s the other question, is are you really comfortable sending all of the emails that are in your company to any third party, whether it’s OpenAI or someone else. And so we’ve seen a lot of interest from all levels, from startups that tend to be more cost focused, to enterprises that tend to be more privacy- and data-control focused in open source models.

It’s not that open source models are perfect and open source technologies are perfect, it’s just that they’re flexible and you become part of a community. I think when Teresa was talking earlier about the—or you were talking, Christina, earlier—about this idea of building things together, that’s really the ethos behind the open source movement that we’ve seen, and we’ve really seen a lot of dynamism in that area, and every week there’s new models. It’s just a really, really dynamic space right now. And maybe open source models lag a little bit, but they’re continually improving at a very, very fast rate.

And if you look at the history of things like Linux and so on, you see that same pattern, that sometimes they lag a little bit, but just the breadth of applications that they end up being part of becomes the reason that people flock to these open source models. Just the fact that they’re part of a community that exists, that’s also one of the reasons that there are places like Hugging Face that are incredibly popular in the community as locuses of this open source movement.

Christina Cardoza: Absolutely. And I think, being in this open source space, it’s also when we talk about hype versus reality. There’s always the hype of what businesses want or what they think these solutions can do, and the reality of what they actually can do—not only if generative AI is hype. So I think being in the open source space you’re in an interesting area, where you can see the limitations. We’re still need to figure out exactly how to work with LLMs in all different types of use cases and scenarios and industries.

So I’m curious, Waleed, how do you think this space will mature? What do you think really needs to happen within the community—open source or outside of open source—to really make generative AI more mainstream?

Waleed Kadous: What we also see is open source models becoming easier to use. So, for example, Anyscale now offers a hosted version of many of these models that’s both price effective but also is identical to the OpenAI API. So you can literally just change a few variables and use it.

And that effort to make LLMs easier to use is one of the increasing trends. I think what will also happen is the continual looking of the life cycle. So really what we haven’t worked out so far is how to make LLMs better over time. If an LLM makes a mistake, how do you correct it? That sounds like such a simple question, but the answer is actually nuanced. And so what we’re seeing is a massive improvement in the evaluation and monitoring stages that companies like LangChain and LangFusion are really taking the lead on.

And then, finally, so far the focus has been on Large Language Models—text in, text out—but as Ramtin pointed out, we’re starting to see the evolution of both open source and closed, multimodal models—things that can process images or output images as well as text. And just as there is Llama for text, there’s now LLaVA for video and vision-based processing. And so we’re going to see some multimodal applications start to come up in the coming years.

I would say though, that I still think much of the focus would be on Large Language Models. Every business in the world uses language, uses words, as Ramtin pointed out. Not everybody, not every business in the world really needs to process images. Lawyers probably don’t need to process images all that often, right? So I think much of the growth will be in the single mode, text-based mode as things move on.

Christina Cardoza: Well, I’m excited to see where else this space is going to go. Like we’ve talked about in this conversation, there are a lot of opportunities, a lot of places to get started, but there’s obviously a lot of work still to be done and a lot we can still look forward to.

So, we are running a bit out of time, but before we go I would love to throw it back to each one of you—any final thoughts or takeaways? So, Teresa, I’ll start with you. Is there anything else you wanted to add, or anything looking towards the future you’re excited that we’re going to get to?

Teresa Tung: I think we still just need to remember that AI is about the data, and so being able to use this new class of generative AI for your business and to differentiate. One, hopefully the takeaway is that you could realize how easy it is to begin owning your own model. But it starts with that investment—getting your data foundation ready. And the good news is you could also use generative AI to help get that data supply chain going. So it is a win-win.

Christina Cardoza: Yeah. I love that final thought, that AI is about the data. There’s always such a—when something like this happens and there’s a big trend in a space, there’s always that instinct to jump on it.  But you have to actually be solving a problem or resolving something that the businesses need. Don’t just jump on it just to jump on it. What is the data telling you, and how can you use these solutions to help tell that story through your data? So that’s great to hear.

And, Ramtin, anything you would like to add?

Ramtin Davanlou: Yeah. I think regulatory and ethical compliance and dealing with hallucinations, like Waleed mentioned, and other topics under what we call responsible AI are the biggest challenges for companies to overcome. And also navigating the cultural change that’s needed to use GenAI at scale is really key to success.

Christina Cardoza: Yeah. And I think it’s ethical AI and responsible AI—it’s not only an issue for businesses, but also consumers or customers. They can be skeptical of these solutions sometimes, so building these with responsible AI and ethical AI in mind, that’s definitely important.

Waleed, last but not least—again, is there anything else you wanted to add to round out this conversation?

Waleed Kadous: I think it’s important to get started now, and it doesn’t have to be complicated. The tools for prototyping are becoming easier and easier. OpenAI, for example, recently released GPTs, that make it very easy to build a retrieval augmented generation system with a clean UI.

There will be these stages, but think about it as a staged process. Let’s build a prototype, make sure that users like it, even if it costs a lot of money. We’ll build that on top of GPT-4 turbo just to prove that there’s value there. And then come at the cost—and to some extent the quality issues—as a secondary issue, as the usage of those particular tools come up. So it’s now becoming much easier to prototype.

And one other thing is to not just think about this in terms of central enterprise organizations, but how can you empower employees to build their own LLMs? And for that initiative around LLMs to come not from some central machine learning group or something like that, but to give people tools to optimize their own workflows and to improve themselves.

And I think that’s really one of the most exciting trends. Rather than seeing this as a substitute technology, to see it as an augmentative technology that helps people do their jobs better. And I really like that mode of us focusing in AI on that, and empowering people to use LLMs in a way that makes them feel empowered rather than eliminated.

Christina Cardoza: Yeah, absolutely. I think my biggest takeaway from this conversation is generative AI is here to stay, and it’s only going to get bigger and help us do better things. So, like you said, Waleed, get started now, and if you’re skeptical or you don’t know where to start, or you don’t know how to take your small wins to the next level, we have great partners like the ones on this podcast: Anyscale or Accenture.

I encourage all of our listeners to visit their websites to stay up to date in this space, and to see how they can help you take your solutions and efforts to the next level, as well as visit insight.tech as we continue to cover these partners in this space. So I want to thank you guys all again for joining the conversation and for the insightful, informative conversation. And thanks to our listeners for tuning in. Until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel® trademark information.

This transcript was edited by Erin Noble, copy editor.

Mastering the Tools for the Next Generation of AI

Posted on by Micah Kaplan

AI is changing the world—evolving in, and into, different industries every day. That is in part because developers are building new solutions and working hard to stay on top of each new generation of AI possibilities. And Intel helps to make it all possible—giving those developers the technologies and tools they need, such as OpenVINO™ and Intel® Geti™. What’s more, OpenVINO and Geti are designed to connect developers and business domain users together, further facilitating the next generation of AI solutions and use cases.

To learn more about the next generation of AI development, we talked with Paula Ramos, AI Evangelist at Intel. She discusses real-world problems that AI solves, Intel capabilities involved, and the democratization and spread of AI (Video 1). Because it’s not only about making tools and technologies available to developers; it’s also crucial to provide education and resources to bring more people into the field in the first place.

Video 1. Paula Ramos, AI Evangelist at Intel, looks at the trends and technologies making the next generation AI possible. (Source: insight.tech)

How do you see AI advancing and solving real-world problems right now?

AI is advancing fast; new things are coming every single day. Right now there is even more awareness for it than ever, as those real-world problems turn into amazing solutions. Startups are using silicon power together with the capability of AI to solve these problems.

For example, AI helps people to communicate using translations—translating text between a hundred languages. Another great example is a self-driving-car system that some vehicle brands are using to control vehicle steering, acceleration, and braking, with the potential to reduce traffic fatalities and accidents. AI can help doctors to diagnose cancer, to develop personalized treatment plans, or to accelerate the deployment of new drugs and treatments. It is helping farmers reduce the use of pesticides and herbicides. AI is helping humanity to solve their problems faster.

What is the importance of making AI more accessible for developers?

I have three thoughts on this. First, creating more accessible information for developers can accelerate the speed of AI innovation. The more developers who have access to AI tools and the easier it is, the faster the technology will advance to accelerate the speed of innovation. Access to the latest hardware improvements is also important. It is really another tool, because it enables developers to build and deploy applications more efficiently and effectively.

The second point is democratizing AI. We need to be sure that AI is for everybody, and that every single developer has the opportunity to benefit from the technology. By making it more accessible, we can help to bridge the AI adoption gap.

And the third one is solving the AI talent shortage. Right now there is high demand for AI developers and not enough developers in the world. By making AI easier to learn and use, we can help train more developers and close the AI talent gap.

What tools and hardware have come out to address the democratization of AI?

I’m so excited about the Intel announcement of the new AI PC. It is built with the Intel® Core™ Ultra processor that incorporates GPU and also a new element called an NPU—Neural Processing Unit. I showed it at the generative-AI booth at the Intel® Innovation conference in September, and it had a great performance. Pat Gelsinger, the CEO of Intel, demonstrated running a Llama 2 chatbot—an LLM model—on an AI PC, locally on a Windows machine. It is a lot more secure to utilize AI without an internet connection, without sending that data out to the cloud, for sure.

“The more #developers who have access to #AI tools and the easier it is, the faster the #technology will advance to accelerate the speed of innovation.” – Paula Ramos, @intel via @insightdottech

You know, a lot of people think that Intel is just a hardware company, but we are doing a great job of showing developers how they can easily improve their solutions using frameworks or systems such as OpenVINO, the inference and deployment framework that Intel has. The AI PC also has the capability to run OpenVINO. And OpenVINO is showing us the potential we have at Intel to leverage AI. OpenVINO is everywhere now: in smart cities, manufacturing, retail, healthcare, and also agriculture. Downloads of it have increased 90% in the past year.

Can you tell us more about that relationship between OpenVINO™ and AI?

OpenVINO is a toolkit that Intel provides to developers on client and edge platforms; it is powering AI at the edge and making AI—generative AI—more accessible, optimizing neural network inference across multiple hardware platforms. However, the main goal of OpenVINO is to run the optimization and quantization of the models so we can reduce their size. We can reduce the memory footprint and also run the models faster in a wide range of hardware—Intel and non-Intel hardware. What we are doing with OpenVINO is solving the real pain points for developers.

Another important thing is that once you have a model in the intermediate representation format—that is, the OpenVINO format—you can deploy it everywhere. This is one of the differences between Intel and some of its competitors.

How are developers leveraging Intel® Geti™?

Intel Geti is a platform that helps organizations to rapidly develop computer vision models. In short, it brings all the necessary things together—annotation, training, optimization, testing. You can create projects, upload your annotation, upload new production data or previous data. You can modify trainable features, test the deployment running in the server, and also download your deployment for running it locally.

So there are a lot of benefits to using it. Data scientists, machine learning professionals, systems integrators, and domain experts can work together using the same platform. This is because it is easy to use and has the potential to control multiple aspects of the training and optimization process. It can also provide modeling in different formats, and one of these formats is that intermediate representation format—OpenVINO, which is behind the scenes of the Geti platform for optimization and quantization.

The Intel Geti platform also offers an SDK that helps users to take advantage of easy-to-use functionalities. It utilizes OpenVINO to build deployment pipelines and to accelerate inference on various Intel hardware platforms that include CPUs and GPUs, without the need for computer vision expertise. That is the beauty of this platform.

Deployment with the Intel Geti SDK makes it super easy for developers since the SDK is agnostic to the computer vision task and also agnostic to the model architecture. Developers don’t need to prepare the data for model input and don’t need to prepare the model output to show the results.

What are some of the challenges women face entering the AI space?

I’m so passionate about this topic. Personally, I represent two minorities in the global tech workforce. The first is women in tech, and the second is Latin women in tech—and you can interpret “tech” as artificial intelligence or engineering in general. Fifty percent of the global tech workforce are women but just two percent are Latin women. So this is a huge underrepresentation, and I want to contribute to reducing the lack of access to education and training that we have in Latin countries, particularly.

But I also want to inspire more women in general to work in AI. I want to reduce discrimination and bias. Everyone deserves the opportunity to have success in tech, regardless of gender. Women can also sit at the table and have serious discussions about technology. Women bring a unique perspective about how to solve problems; we have unique skills to create products and services that meet the needs of all users.

And we have made some significant contributions to the field of AI. One example is Dr. Fei-Fei Li, an AI researcher and Co-Founder of the Stanford Institute for Human-Centered AI. She has contributed to the birth of deep learning—she developed the ImageNet initiative, and that initiative has played a major role in the deployment of deep learning. This is a remarkable impact that one woman has made to AI, so I can imagine how many ideas that other women are also capable of contributing to the field.

Is there anything you would like to add about the future of AI development?

There is one important question that I want to share with developers: Where are your dreams? You can achieve those goals, because AI is a powerful tool that has the potential to make the world a better place for everyone.

So try new technologies, new models and algorithms. Try to participate and be an active contributor in an open-source project. Stay tuned to the latest trends that will make AI more practical. We need to build an AI that is inclusive, fair, and beneficial to all. The engine is in your imagination.

Related Content

To learn more about AI trends, read Evangelizing AI: The Key to Accelerating Developers’ Success and listen to Exploring the Next Generation of Artificial Intelligence: With Intel. For the latest innovations from Intel, follow it on Twitter @IntelIoT and on LinkedIn at Intel Internet of Things.

 

This article was edited by Erin Noble, copy editor.

Interactive Signage Boosts Digital Engagement with Video

Posted on by Micah Kaplan

When was the last time a video really grabbed your attention as you walked by? The increasing ubiquity of the medium in our common spaces means there’s a growing risk of it becoming mere background wallpaper.

Simon Carp recommends a solution to move past the ho-hum. “Instead of using video as a passive medium, treat it as a platform for digital engagement,” says Carp, the Head of Product Management for Uniguest, a provider of audio-video solutions. This means interactive digital signage and dynamic and informative content that can change how customers consume video.

Improving Engagement Through Interactive Digital Signage

It might seem like a no-brainer, but improving digital engagement requires video solution providers to deliver content that customers actually want, Carp says. That content needs to be fun, eye-catching, informative—or all the above.

For a high street retailer in Australia, rolling out dynamic content on screens is a perfect example of using eye-catching and engrossing video as a way of engaging customers. What started as fixtures at the ends of store aisles has expanded to window displays and even custom video at checkout counters. A bonus: The retailer sells advertising space on the platform, adding to revenue.

Through a combination of screens, no matter the location, Uniguest’s Tripleplay platform can help clients customize and deliver personalized content. The tools give customers the ability to essentially create their own channels of content, which is delivered to screens. Tripleplay technology enables more targeted customization of content. Personalization can extend down to an individual screen—a bank branch can add information about its customer service team, for example—or to various other displays.

Such an ability to scale content up or down is one of the significant advantages that Tripleplay delivers. “Customizing content and making it relevant and actually configuring it for hundreds if not thousands of screens is difficult,” Carp points out. “You can create a huge overhead in planning and configuration. But what we try to do is build tools into our technology that allows you to localize content for all these screens in a scalable way.”

Clients can deliver the dynamic content needed for engagement—at scale and through a single platform.

There’s a lot of #DigitalSignage out there, but the Tripleplay complementary suite of #technologies adds breadth and enables the end user to craft a legitimate and fully #digital engagement platform. @uniguest via @insightdottech

Interactive Media in the Workplace

It’s the behind-the-screen tools that help Tripleplay deliver implementations in a variety of markets.

The Tripleplay Reserva Edge solution, for example, is a digital meeting-room signage tool that helps employees schedule meetings and access other functionalities related to the management of space and equipment. The physical units are small, ready-to-use interactive display screens installed next to meeting areas in corporate and teaching spaces within university campuses.

Employees can use the Reserva Edge unit’s touchscreen to find out information about the facility’s usage a week or two in advance, and can book their own meetings. To make things more accessible, employees can perform the same functions anywhere through a mobile app. The tool requires attendance confirmation when the room is reserved so the space can be released to others when vacant. Using the Reserva Edge solution, employees can also alert maintenance about malfunctioning equipment and optionally alert other users of the space about potentially disruptive issues.

AI Driving Digital Engagement

Intel powers the hardware that underpins most of Tripleplay’s AV solutions, Carp says. The digital-signage media players and content management systems all run on Intel.

Uniguest uses AI selectively based on end use cases. Voice-to-text AI systems, for example, can transcribe meeting and lecture audio streams. Students can even search these lecture notes by keywords. AI can also be used to measure screen engagement: whether or not someone is looking at the screen (and for how long); and basic demographic information like age. The AI solutions follow privacy protocols by not personally identifying people but simply registering their presence, Carp says.

Collecting even basic demographic information will help Uniguest customers target their video campaigns even better. Customers could use audience measurement as a basis for their campaign planning and figure out what content they would want to advertise accordingly.

Future directions for video will include tools to accurately measure metrics, something Uniguest is working toward. “The big area of focus for us going forward is what we can do to measure engagement and provide insights back to the business to demonstrate the benefits,” Carp says.

Uniguest brings not just the video platforms and AV solutions to the table but also delivers ways of facilitating and managing the content. There’s a lot of digital signage out there, but the Tripleplay complementary suite of technologies adds breadth and enables the end user to craft a legitimate and fully digital engagement platform.

Carp sees a bright future for video. An increasing number of organizations want to communicate with people. Technology underpins that engagement; it helps companies beyond passive media to really engage their customers using dynamic and interactive media.

 

Edited by Georganne Benesch, Editorial Director for insight.tech.

AI Integration Platform Delivers a Wealth of Data Insights

Posted on by Micah Kaplan

When it comes to technology implementations, companies want readily consumable pizzas they can sink their teeth into, not Lego building blocks they need to assemble, says Ken Mills, CEO of EPIC iO, a provider of IoT and AI solutions.

Yes, IoT and AI are revolutionary and can deliver dramatic efficiencies. But the number of building blocks—connectivity, data integration, visualization—to make these technologies work can make any IT team’s heads spin. “Customers aren’t at the technology adoption curve to just figure out AI solutions on their own; they need help,” Mills says.

Developing an AI Integration Platform

EPIC iO helps by making these technologies work for their clients—essentially crafting pizzas for AI and IoT solutions.

The company does so through EPIC iO DeepInsights, a software platform that makes the different technology components work together. DeepInsights is like the crust on the pizza, tying together IoT sensors, AI and computer vision, and connectivity. “We’re bringing the whole stack to the customer so they don’t have to go buy five different other software packages to make these technologies work,” Mill says. EPIC iO is especially useful in delivering context to data, which leads to sharper insights.

To extend the pizza metaphor further, companies can pick and choose the “toppings” (sensors) they want to include, whether those are cameras for video data analytics or connectivity sensors to provide and monitor edge devices.

The advantage of the open #AI + #IoT #software platform is that you don’t have to deliver bespoke solutions every time. @EpicIO_Tech via @insightdottech

A Template for Democratizing AI

The advantage of the open AI + IoT software platform is that you don’t have to deliver bespoke solutions every time. “We’re putting together validated templates so our customers can automate workflows and make more money,” Mills says.

The creation of reproducible AI solutions also democratizes access to the technology by lowering the barriers to adoption. It’s part of EPIC iO’s mission to make the world safer, smarter, and more connected. The cost of AI is decreasing because the cost and performance of the necessary hardware and compute are all improving. These factors also help improve access.

The templated format is especially useful when companies want to focus solely on their strengths. For example, data analytics firm SAS partnered with EPIC iO so they could layer their skills on top of edge data gathered by EPIC iO-installed solutions. “EPIC iO and SAS are a great partnership because we can collect all that data at the edge and generate events and rules through our computer vision and machine learning models and we can then pass that data onto SAS for deeper analysis,” Mills says.

Efficiencies from IoT Analytics

On the other hand, in many cases, companies know they need to improve their processes, but they are not sure if AI solutions will do the trick. A major utility company in California, for example, would dispatch workers to check the fill levels on tanks and follow up if levels were low. After the first pass to evaluate process efficiencies, the company explored installing cameras that would be monitored by humans. But “the person-in-the-loop solution is not very cost-effective or practical at scale, especially when you have technology like AI,” Mills says.

Instead, EPIC iO suggested installing IoT camera sensors and using AI and computer vision to measure fill rates and proactively detect problems. Workers need to visit only when the levels dip. Even better, because of the digital transformation that AI has delivered, the company receives expanded and more useful data that includes how and when the site is being used instead of focusing on the narrow problem of fill rates.

The utilities case is an illustration of how to use AI to solve operational efficiencies. Companies need to ask what problems they haven’t figured out a solution for, and which ones are the most meaningful for their business. “Once you have identified those two vectors, then you’ll probably have a solution where AI can help,” Mills says.

Working With Partners

DeepInsights is built on the Intel® OpenVINO™ toolkit and powered by Intel CPUs and GPUs—providing high-performance computing while keeping power consumption low. The software is also cloud-native and not tied to a single cloud environment. DeepInsights’ portability enables customers to control where their AI-related data is stored and inferenced, which matters for abiding by privacy and data handling regulations.

While EPIC iO delivers easy solutions to customers, the company still works with systems integrators to set up physical infrastructure and ongoing onsite support if needed. Integration with external software programs and establishing operational workflows also fall under the purview of systems integrators.

The Future of Democratized AI

Endless possibilities for AI implementations use computer vision and sensors: A robust posture analysis model, for example, can determine if a hospital patient has fallen in their room so staff can intervene when needed.

Using crowd analytics to monitor retail floor activity is another possibility as is using AI to help improve air quality and safety in cities.

Mills is particularly enthusiastic about using generative AI to make getting insights easier for the average user. Another development he’s looking forward to? The democratization of AI. “I’m really excited that more and more people will be able to take advantage of AI across all industry segments and that it’s not just restricted to the biggest companies and cities,” he says. The democratization of AI is a sure path toward a safer, smarter, and more connected world.

 

Edited by Georganne Benesch, Editorial Director for insight.tech.

Smart Factory Tech Proves Data Is Power

Posted on by Micah Kaplan

Comprehensive smart factory solutions have arrived at last, helping manufacturers take control of their operational technology (OT) data for the first time. The result is greater efficiency, increased profitability, and greener operations.

The power of smart factory solutions lies in how they allow manufacturers to collect, coordinate, and visualize a disparate mass of data from the factory floor in a unified digital format—in essence, merging the spheres of OT and IT. This might sound like a fundamental step in the digital transformation of manufacturing, but until recently it has been surprisingly difficult to implement.

“OT data is quite challenging to manage because of the diversity of proprietary data protocols used by industrial machines,” says Eric Lo, Associate Vice President of Strategic Marketing for NEXCOM, the parent company of NexAIoT, a specialist in industrial computing and smart factory solutions. Legacy machines can also be hard to integrate with modern IT networks.

The encouraging news is that specialists like NexAIoT have now developed the hardware and software capabilities needed to implement end-to-end smart factory solutions. Some of these solutions are already in deployment—and the early results are extremely promising.

A Smart Factory Comes to Life

Case in point: NexAIoT’s implementation at the production facility of a prominent notebook computer manufacturer. The company wanted to modernize its factory operations. But their vision was highly ambitious, encompassing comprehensive digitization in which every step of the production process could be traced and managed from a centralized platform.

“They needed visibility into all kinds of data, from production status, manufacturing parameters, and factory environmental conditions to the bill of materials for current and finished products,” recalls CL Chiang, Director of IoT Automation at NexAIoT. The company also had many machines with proprietary data protocols—especially in their component assembly and motherboard fabrication areas. Data from this equipment needed to be made available to the factory IT network for centralized monitoring, management, scheduling, and capacity planning.

Working with the computer manufacturer, NexAIoT developed a complete smart factory solution that provided the insight and control the company needed. Industrial personal computers (IPCs) were used to collect and collate data from industrial machinery. These IPCs acted as gateways and edge server devices, translating the machines’ different data languages into the widely used OPC Unified Architecture (OPC UA) industrial communication protocol.

With a common data format in place, information from the production line could be integrated with the facility’s core IT systems: customer relationship management (CRM), incoming quality control (iQC), enterprise resource planning (ERP), and the manufacturing execution system (MES). NexAIoT also helped the manufacturer incorporate a computer vision-based defect recognition system, allowing quality assurance workers to spot defects and potential issues in real time. In addition, NexAIoT provided features beyond the company’s specifications, including predictive maintenance capability and a centralized dashboard where managers can visualize production data, helping them make better business decisions.

#SmartFactories are already delivering tangible benefits to #manufacturing businesses. These solutions, and the wider ecosystem in which they are developing, will likely bring even greater upside to the sector. @NEXCOMUSA via @insightdottech

The result is an end-to-end smart factory solution that has helped the company bring its operations into the industry 4.0 era.

IPC Technology Boosts Control and Efficiency

The computational heart of the smart factory solution is the TT300-A30 Fanless System IPC—a rugged, powerful machine built for performance at the industrial edge.

NexAIoT used industrial computers running on Intel® Core™ processors, allowing them to deliver features including Intel® Time Coordinated Computing (Intel® TCC) to reduce latency and provide real-time control. “The Intel processors also support multi-display outputs, which allows them to work on different workloads at the same time for better efficiency,” says Mark Tuo, Product Manager of NexAIoT’s IPC system.

When IPCs are combined with sensors, AI, and data visualization tools, factories gain several key benefits:

  • Profitability: Greater insight into operational data enables better business decisions and predictive maintenance, while computer vision can be used for quality assurance. Manufacturers boost their bottom line by increasing efficiency, optimizing processes, and reducing downtime and defects.
  • Sustainability: Detailed power consumption data from the production line helps facilities use energy more efficiently through load balancing and the timely replacement of aging equipment that is no longer energy-efficient. The result is a lowered carbon footprint and less overall waste.
  • Safety: Because smart factories automate the collection of information from industrial machines, it is no longer necessary to manually examine equipment to gather performance data. This keeps workers out of harm’s way since they no longer have to enter noisy and risky environments.

Extending the OT – IT Pipeline

Smart factories are already delivering tangible benefits to manufacturing businesses. But in coming years, these solutions, and the wider ecosystem in which they are developing, will likely bring even greater upside to the sector.

The continued integration of artificial intelligence into manufacturing is one part of this story. NexAIoT, for example, is looking at ways to use AI to optimize factory operations to achieve ESG goals, and is also investigating the use of AI to identify and mitigate production process bottlenecks.

In addition, the current wave of industrial digitization may one day expand to encompass a factory’s customers. By integrating the purchasing process into the smart factory system, buyers will be able to order from manufacturers more directly and efficiently, and manufacturers will gain greater control over inventory management and demand forecasting.

There will undoubtedly be many challenges along the way, as well as new possibilities. But Lo says that this is to be expected—and embraced: “Industry 4.0 is not just another solution. It’s a long-term journey to continue improving the manufacturing process for greater efficiency, sustainability, and profitability.”

 

This article was edited by Teresa Meek, Contributor for insight.tech.