Christina Cardoza: Hello, and welcome to the IoT chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Associate Editorial Director of insight.tech. And today we’re talking about omnichannel customer experiences with Stephen Borg from meldCX, and Chris O’Malley from Intel. Hey guys, thanks for joining the podcast today.
Chris O’Malley: Thank you. How are you doing today?
Christina Cardoza: Great. Great.
Stephen Borg: Thank you. Thanks for having me.
Christina Cardoza: Yeah, of course. Before we get started, I think our listeners would love to hear a little bit more about you, and who we’re about to speak to. So Stephen, I’ll start with you. What can you tell us about yourself and why you started meldCX?
Stephen Borg: I’m the Co-Founder and CEO of meldCX. I—at the time I was working for a few large groups consulting in this area, and we actually designed meldCX on a napkin seven years ago, but the tech was just not there to build it. So we decided around five years ago, when we saw some of the tech emerging that was relevant to us to really start the process. We really built it to—and that’s what it stands for—is to take legacy and meld it with current technology and create great experiences. And that’s really what we do. We realize our customers have that—some legacy debt that they have to deal with, and we take that, take that information and bring it out and really create an experience for customers.
Christina Cardoza And Chris, welcome back to the podcast. For our listeners, why don’t you refresh their memories, what you’re up to at Intel these days.
Chris O’Malley: Sure. So my name is Chris O’Malley. As Christina mentioned, I’m at the Intel Corporation. I am a Marketing Director for the Internet of Things group. I focus on essentially the technology used in retail, hospitality, banking, and entertainment segments. Primarily, you know, we’re thinking of, how does technology drive experiences in stores? And our whole goal is to support technologies like Stephen’s company, meldCX, works on.
Christina Cardoza: Great. And I should mention that insight.tech, the program and the IoT Chat podcast, are owned by Intel. So it’s great to have somebody from the company representing this conversation today.
Stephen, I love how you mentioned in your intro that when you started the company, that technology wasn’t there, but now we’re seeing the technology rapidly advance today. And in addition to that, over the last couple of years these customer experiences in retail, hospitality—all of these different industries have just completely changed, and have had to change. But, you know, it’s great that we have this technology to be able to do that. Not everybody knows how to utilize the technology or how to change—what’s the right change for them. So why don’t you start by telling us a little bit about what you’re seeing in this customer experience, how you were seeing this evolve across different industries.
Stephen Borg: It’s interesting. I think the whole COVID situation, multiple lockdowns, has really accelerated the curve, right? We’ve had customers and we talk to them, and we still got the same issues where they need to increase the level of service without as many resources, either being budgetary, or they can’t get access to the correct staff. So you’ve got this expectation of needs increasing, while you have less ability to facilitate those customers. So what we’re finding is that when customers do venture out—and this is feedback from our customers or our key enterprise customers—when their customers do venture out, they expect a higher degree of service. They expect the demonstration of cleanliness, right? That they’re respecting the situation and following process. And they also expect a higher degree of engagement.
So how do you do that while either reducing costs or with less resources? And we have customers that can’t even hire to the needs they have to facilitate. So what we’re seeing is that they’re starting to turn to technology that takes the transactional elements or these elements that use up resources that are not customer facing, and redirecting those resources to creating great experiences. And we’re seeing that in hotels—from seamless check-in and check-out; in grocery, transactional items that slow customers down in either self-checkout or in actual checkout—you know, things like processing fresh produce. And we’re seeing that really across the board: what are the opportunities to reduce friction, create automation, but increase engagement?
Christina Cardoza: You mentioned the pandemic, how that sort of accelerated things for businesses. And I imagine a lot of these businesses were forced into a digital transformation they weren’t quite prepared for. So they’ve had to make a lot of changes on the fly to stay competitive. And now that we’ve had some time to look back at those changes that were made, Chris, I’m wondering from your perspective how well have these industries, like Stephen just mentioned—retail, grocery stores, hospitality—how well have they been dealing with the changes that need to be happening and taking on this digital transformation?
Chris O’Malley: You know what? It’s a, it’s a mixed bag, to be honest. You know, all the trends or the challenges that retail was facing prior to COVID, they still exist. You know, three years ago we were talking about frictionless—the millennials, digital natives, have been growing. They don’t like to talk to people. So it’s been self-checkout, wayfinding key, that type of thing—engaging with technology as opposed to humans. There’s been inventory and supply challenges. There’s been this—you know, there’s been some increasing theft, there’s been the need for inventory. But the reality is they were, we were, kind of at the slow level of growth. You know, it was nice to have this technology, but it wasn’t absolutely necessary.
What we found is the people that, or the companies that, started to invest in this type of technology prior to COVID, now that COVID’s hit us the ones who invested previously are doing really well. The ones who don’t are trying to build this entire technology structure without any previous investment. And they’re struggling greatly with it. You know, the biggest thing Stephen mentioned—it’s amplified or accelerated the trends. That’s what we’re seeing. It’s absolutely accelerated. And I think it’s because of this worldwide labor shortage. There’s a lot of jobs that they literally cannot hire for, or if they can hire for it, they have to hire at a wage rate that quite frankly they can’t sustain profitably. So they’re looking at, how can I automate, how can I use insights of computer vision to give people the experience they want?
After, you know, during COVID, many companies went in with almost no one-to-one digital contact with their customers. You know, for two years we did online ordering, we did mobile ordering, we did curbside pickup. So companies now have this massive relationship with customers digitally. If they don’t know how to deal with that data, if they don’t know how to personalize with that data, they’re really struggling. So we’re really seeing the companies that invested prior to COVID taking off, and the other ones are playing come-from-behind, really struggling to put the IT infrastructure in place, how to use computer vision and things like that, to make it really valuable.
Christina Cardoza: Stephen, I imagine you’re working very closely with a lot of these businesses that are struggling with these things, like infrastructure or to adapt to these changes. What have been the top pain points or challenges that you’ve been seeing, and how can they address these now going forward? Or how can they implement these new technologies and work with meldCX to go down a better path?
Stephen Borg: Yeah, I think there’s a few areas. One, when we started out with meldCX, I mentioned earlier that we took a little bit of a pause and made sure the technology was available. The reason for that is when we went out with meldCX, we wanted to create a solution out of the box where you could simply plug and play. You don’t need data scientists, you don’t need a massive team to stand up what we see as the most common aspects of computer vision. So, analytics tracking, inventory, those types of things you can just plug and play out of the box and get going. So that was one of the first things we wanted to do.
And then, secondly, we wanted to create a method where you can take—and we found a lot of customers in this state where they had to furlough some of their team members during the pandemic and couldn’t get them back—so we found a lot of customers that were, say three-quarters through a project where they had existing investment in some models. You can take meld and pump those models down meld, because we use OpenVINO, and mix them with other models we have to complement. So we’ve got this thing called a mixer, and it blends models together and gives you an outcome.
And then, thirdly, we actually created a service that if you have a specific use case or a problem you’re trying to solve, we can go ahead and create that model for you. We have a synthetic data lab because we face the same issues of getting content or getting the right video to create these scenarios. So we have a synthetic data lab.
So what we’re finding is that now that the level of engagement is cross business, customers are very invested, and we’re finding that we have—unlike the past, where we might have an IT stakeholder or a marketing stakeholder—we have everyone at the table, because they see the benefit and they really drill down into understanding what they need. So we sort of advise customers to start with computer vision, start with out-of-the-box modules and then go from there, because they really don’t, most of them don’t comprehend the power of it. And what we try and explain to customers is that this is an amplification of your existing capability, right? So you put your machine vision in, or your models, and it either feeds you data, or automates a function, or creates a cause and effect to enable your staff to do more with less. So really we say, start at what’s out of the box, experiment with it. And then we have our team work with them to try and really drill down into that problem-solving phase for future growth.
Christina Cardoza: Chris, is there anything you wanted to add about challenges that you’re seeing, and how tools and technologies today can address those?
Chris O’Malley: You know, yeah. The big thing, and I think Stephen addressed it already, is especially with the name meldCX, melding the old with the new technology. So computer vision is great. It’s a great technology and it really can, you know, figure out how to deploy the people where you need them most. But what’s exciting about some of the technology that meld offers is, say you don’t want to do the full investment into a new camera setup right now. You may have security cameras already there. You know, meldCX could take those feeds right away, load some models on that, and get basic data just from the get-go. So you don’t have to do this massive investment to start to get data.
Now, what we find is once customers actually start to realize that, and they see what the computer vision can provide, then they’re interested in investing further. And they say, “Wow, you know, instead of just looking at operational-type stuff—is there a liquid on the floor that I need to clean up? As you know, 30 people enter a bathroom and now I need to go clean the bathroom. And that slot machine’s been used 150 times, I want to clean it now.” Or something like that. They start to add it to more and more. They see the power of it, so they start to add more and more and more. And what I find about that is it’s great, because this is, this type of technology, is not coming to an end.
We’re at the beginning right now. You know, I wouldn’t be Intel if I didn’t say Moore’s law, you know, we’re doubling the performance of our technology every two years. The corollary to that is that technology becomes cheaper every single year, too. I mean, technology is reduced by more than half every two years. So you start adding compute to everything. And when you start adding compute to everything, there’s an immense amount of data. So you need technology like this to start making insights, those actionable insights that are valuable for your company.
Christina Cardoza: Now, we’ve been discussing sort of how these physical spaces can transform themselves. Grocery stores, restaurants we’ve mentioned, but it’s really a bigger piece to this, is the online aspect of it. I think sometimes we tend to think of e-commerce and retail, physical retail, as two separate things, but today they’re sort of merging and melding, like you explained. So Stephen, when we talk about a retail omnichannel experience, what does this really mean? And what is it touching?
Stephen Borg: Yeah. And I think it’s really touching on every aspect of your customer’s journey, right? I think there’s been a lot of focus on mobile, a lot of focus on web, but connecting mobile to web in a single, seamless experience has not been something that we’ve seen when it comes to connecting those two to an in-store or a physical contact point. And often we find they’re completely different experiences, right?
So what we’re finding is by using data or connecting those dots—one thing about meldCX, we not only have our base that gives you computer vision, but we have these modules that you can load on existing devices, legacy devices, and new, that allows you to connect those dots.
So for example, connecting that computer vision to an event that occurs locally—it could be providing access to a digital locker based on your token, or having that seamless experience of you’re doing it anonymously, but having your last order come up on the screen. It doesn’t know who you are, but it just knows who your last order—what your last order is. Or when you go to a self-service device and it knows you’ve used it multiple times, it doesn’t go through all the instructions again. It just goes to your last left-off point. So all of these little, subtle things that are done anonymously but create convenience and context is what we’re starting to see. And it seems to be best practice, and we’re seeing some good results from it.
Christina Cardoza: So how would you suggest businesses get the best of those—both worlds, and really connect the dots? How do you start on this omnichannel journey and make sure that you’re providing the right value to each platform, and those platforms are all connected together so that you’re getting even more value into your business?
Stephen Borg: I think we start with, and as Chris was saying, we start with simple measurement—understanding your environment the best you can and trying to connect those contact points.
So we had a recent customer have a scenario that no one anticipated from our data. They’re a large electronics retailer, and they sell CDs and DVDs. And you think that’s a dwindling area, right? Netflix and online streaming. You think that would be something that a retailer wouldn’t give much credence to. But what we’re finding is, especially during holiday season, when people travel, in those travel locations, that they might not have the data that is required, or they might not have the setup in an Airbnb that is required for them to use their own Netflix or their own Hulu. And we’re finding that people will go into these destinations and look for CDs and DVDs. And typically won’t buy them, they’ll go buy something like an Xbox or an Apple mini or gaming. So we found that in this client, although all the online data indicated they weren’t interested in these things unless they had that destination or that base still there, they wouldn’t cross sell to these other items, right? And when they reduced it, you thought there wouldn’t be an impact because it’s not a high-selling area, but when they reduced it, their peripheral sales reduced. And without that data, they would never know that.
So they were relying on online data to dictate what behavior is in-store. And that simple measurement task indicated that if they don’t keep this area, they don’t get peripheral sales, because it’s actually a destination for browsing, especially when going to regional areas. Or those customers are destination shopping as they’re about to go away. And we actually increased the sales in one and it increased peripheral sales. So that type of data, you wouldn’t pick it up in sales data. You wouldn’t pick it up in online data, but you’re picking it up by using that anonymous tracking data, hotspot data, and associated sales data.
Chris O’Malley: If I could interject here, what he references is pretty interesting. So, in the last 10, 15 years, advertisement online has been really eating up a lot of the market share. And a lot of it’s been because you’ve been able to track behaviors. If you showed an advertisement to you, Christina, and you clicked on it, they’d know that it had some sort of an influence and they could pay for that.
When you go in-store, there was none of that information. There was no attribution. There was no success. Did my digital advertisements in-store do anything? But with the technology that meld is offering, or computer vision is offering, you now have that ability to figure out, is my campaign working? Was it actually influencing the people? Were they happy with it? Were they unhappy with it? Were they engaged with it? And you can change that. That’s never really existed before until you have the advent of computer vision.
And that’s pretty powerful, especially for a retailer, because you can now start to monetize some of those things as well, but you can also figure out, how do you change your display? How do you change the technology you’re using? How do you change associate activities? All those different things, because you’re going to pick up all this powerful data, which you never had access to before. You know, I’m in marketing, we always say 50% of the money that we spend is useless, 50% is valuable. We just don’t know which one is which. With the technology that Stephen has, we’re starting to be able to figure that in-store. What’s valuable? What’s not. And then you can really start to target these things that make them a lot better.
Stephen Borg: For example, we have another retailer that’s taken their front-end bay, or the bay that you see when you walk in, and they monitor it and monetize it based on you, the customer, touching the product that’s on the shelf. So instead of paying to be in that bay, now they pay for every click or every touch of individual product that’s in that bay. And then monetizing like a website.
Christina Cardoza: We’re talking about massive amounts of data that we can collect now—custom behavior, how they’re moving, and what they’re doing online—connecting those dots together. Now that we know how to collect all of this data and what we want to be collecting and looking at, how can we make sure that we’re making sense of the data? How do we analyze it and process this data and make sure that it’s accurate to make more informed decisions? Chris, I’ll start with you.
Chris O’Malley: Sure. Yeah. You know, that’s the—it’s the mixed blessing of data. You know, there’s a huge amount of data out there that goes unused, and it’s very valuable. So I think one of the first things that has to do with any retailer, and a lot of legacy retailers in particular have very, like, siloed data. So, the POS data is here and it never shares it; there’s kiosk data in-store here; there’s mobile data over here; there’s online data over here. The data is never shared between them all. That doesn’t do a lot of valuable—we desire personalization. You only know a little snippet about them in each of the separate activities.
When you move to a modern kind of an edge, or a modern microservices-based architecture, where you have kind of a shared data or a data lake, and every single one of these experiences can access that same data, that’s when you can start to make sense of all of that data. The other thing that’s incumbent is you’ve got to make sure that you standardize the data. You know, how do you store the data so that every app that you’re running on top can kind of access the same set of data, understands exactly the importance of that data, and then figure that out?
And then the other thing that frankly starts to happen—and Stephen’s already referenced this a little bit—with big data analytics, and we’re at the advent of that as well, you could start to look at pieces of data that outwardly to us make no sense. There’s no correlation, there’s no relation. But if you see it over and over on big data, you can actually make the correlation and figure out that actually, yes, this product A does influence product B. And you can start to set that up that way. Those are things that you don’t even know about, but it all, it all comes down to how you set your architecture up—make sure that data is shared by all the different apps.
Christina Cardoza: And I imagine this data we talked about coming from cameras, we’re doing online data, we’re watching customers in-store, tracking their movements, their patterns, seeing what attracts them, depending on where products are placed. So I can imagine that you’re not just using one solution, or you can’t just do it alone with one company. Stephen, are you working with partners like Intel? How do you use the ecosystem that’s available out there to make this possible for businesses?
Stephen Borg: Yeah, and I guess we see there’s multiple types of data. So there’s some data that are—that is immediately actionable. So for example, we work with a large hotel chain, and when their room keys are out on their vending or on their kiosks, or there’s something that needs to be filled, we actually push that data through intermediate alert. So they use like a Salesforce communication app with their staff. We will notice things. We don’t necessarily store that data. We’ll just have an event that we executed that command. So sometimes we don’t store the data; it’s immediately actionable.
Or, as Chris mentioned earlier, we feel that front desk has hit a threshold and it needs to be cleaned, right? So there’s that type of data. And there’s also that historical data or multisource data that you were trying to get insights out of. In that case, yeah, we do. We work with Intel from an OpenVINO perspective and to make sure that our models are optimized, they can coexist with other applications. And also that we’re not—one thing that we found with OpenVINO in particular, it means we need less heavy infrastructure at the edge, which significantly reduces cost. So that’s a great aspect.
And we work with partners such as Microsoft, Google, Snowflake, to provide customers the data set in the way they wish to consume them. In addition, we have a very comprehensive—and this is one of the things that we initially struggled with—we were providing the data, and customers did not have either the resources or the understanding of how to mine that data effectively.
So we have a comprehensive suite of dashboards that you can use depending on your role in retail. So if you are operational, you can use the operational dashboards; or if you’re marketing or product, you’ll use those bot dashboards. In addition, you can feed your existing data lake or data warehouse. So what we’re finding is customers have a hybrid. They use our reports, which are customizable, and they’ll feed their main data source and start to do integrating into their reporting system.
So one of the aspects that we found, or one of the blockers that we found, is that we didn’t want customers to need to make a big change to their data warehouse or data lake just to experiment with the technology, which we found that being a blocker because they’re really resource poor. So you sign on, pulling your persona, and I’ll give you the data that’s relevant to your role.
Chris O’Malley: So, one thing I’d like to—you know, Stephen referenced the importance of real-time actionable insights. So, one of the things we’re seeing is that if you go to the cloud, you’re going to encounter some element of latency. And for some things that’s perfectly fine, latency doesn’t matter. And those things are perfectly fine to store in the cloud or put into the cloud. But a lot of activities that happen at a retail store, you may want to have absolute real time and you need to do it at the edge.
And the other thing that’s happening—we already referenced that compute is getting so cheap that they’re adding more and more compute. So there’s smart building technology, there’s IoT sensor data all around these tools, there’s computer vision technology. There’s lots of things like that. So your data is actually growing significantly faster than the cost of your connectivity to the cloud is reducing. So you really can’t—in theory you can run all of this video data, computer vision, you can run it in the cloud. The problem is your cost of connecting to the cloud to make those insights is going to explode. And it’s going exceed the value. You have to do this type of stuff at the edge. And that’s where Intel with OpenVINO is very much optimized for efficiently optimized or efficiently using the edge capability to do the inference and get those real time analytics that you need. That’s where we’ve been focused. And that’s where we see really important—
I think some of the data we’re seeing is that data is going so large that about 95%-plus of the data is actually going to be dispensed with and disposed of at the edge. And only a certain amount, let’s say 5% or less, is ultimately going to go to the cloud for permanent storage, for analytics and things like that. But it’s key metadata. The rest is going be processed at the edge. So the edge is absolutely growing rapidly.
Stephen Borg: And that’s what we’re finding. We don’t send any video to the cloud, so we strip out everything we need at the edge. And we do that for two reasons. One, to reduce the cost. And, most importantly, we do it because we abstract all content at the edge for privacy reasons. So that way there’s no instance of any private data going into the cloud or going through our system. It’s all stripped out by that edge device and OpenVINO.
Chris O’Malley: Yeah. And with GDPR, that is really important. So any type of anonymous analytics, anything like that, has to be deleted at the edge. So you can just—you just gather the data that’s important, but no images, no nothing, is ever sent to the cloud. It’s not allowed to be sent to the cloud.
Christina Cardoza: Since you’ve both brought up cost being an aspect of this, Chris, I want to come back to something you mentioned earlier, which is you have to look at your infrastructure and change things and consider the legacy technology that you do have. But I know some businesses can be worried about introducing new technologies to their infrastructure, whether or not it’s going to be a smart investment in the long run. So how can they ensure that the technologies that they’re using, the infrastructure that they’re changing, is going to meet their needs today, but also be able to scale to meet their needs tomorrow?
Chris O’Malley: Got it. So, yeah, I mean, from an architecture standpoint, I mean the first thing that you—you have to be future proof. So whatever you do, it has to be future proof. And that’s why we think that you need kind of an open—what we call a microservices-based architecture. The siloed architecture, which worked well in the past, it fails as you continue to add new technologies and new technologies and integrate new data; it becomes so difficult. The cost of integration is just going to overwhelm you.
If you build your modern—and you could start it, by the way, from your online, and then you can add your mobile, and then you build it down. Most people go down until, like, either the restaurant or the retail level. The last thing to be integrated into that modern architecture is probably the point of sale. You know, that’s kind of one of these sacrosanct things, but when you build everything else, eventually the POS can be sort of an app right on that as well, too. And then, because you’ve got the data infrastructure set up, everything like that, as soon as I add a new technology, it’s much easier just to drop it in. It almost becomes like a new app placed on top of an existing infrastructure. And it’s very easy to launch those new apps, and you can really get to market a lot quicker than if you had to integrate in the old ways with the silo technology.
Christina Cardoza: And we’ve been talking a lot about the business benefits that these organizations are getting by introducing these technologies and creating these omnichannel customer experiences. But Stephen, I’m wondering, how are the customers dealing with all of these changes? What are the benefits that they’re getting from digital signage or video analytics?
Stephen Borg: What we’re finding is that if it’s done with privacy in mind, that customers respond to it quite well, in that either they’ve had a less—a frictionless experience; they’re getting through the checkout quicker, or the staff member has information that’s relevant to them at the time, or maybe tailored. So they’re getting content that’s tailored to either their persona type or based on their frequency of visits. All this can still be done anonymously, but it can create context or awareness. So we’re finding that if you’re providing a frictionless experience, that staff member is not just focused on the transaction. And we found this—we do some financial institutions as well, where we found those staff members could have more of a conversation rather than focusing on some of the transactional aspects—that increasing customer engagement is welcomed, but they still want this degree of knowing that it’s a clean and safe environment where it’s contact—physically contactless where possible, but there is still a rich need for some engagement. And that’s what we’ve found. One of the aspects we’ve found from the pandemic is that now some shopping has become even more social because some countries have just still have lockdown restrictions, and when they do get out they want to be engaged.
Christina Cardoza: So, unfortunately we’re nearing the end of our time today, but Chris, I wanted to give you a chance to add any final thoughts or key takeaways to our listeners today as they go on this omnichannel customer-experience journey, and continue to refine it in the years to come.
Chris O’Malley: Got it. You know, I think the critical thing that we’ve mentioned already is that customers want this frictionless experience. They want the personalized experience, that they can get someone online—they want that in-store, but they also still like the socialization in-store. You know, that type of stuff is still very important, especially in today’s environment. And it can be done with this technology. It can absolutely be done.
You can have the great parts of, like, shopping that everybody still loves, but you can bring in that goodness of online through all of these tools. But from a retailer or a casino venue or hospitality venue, you also have this ability to replace human resources in some instances. There is the worldwide labor shortage that we’ve referenced is real. These venues are struggling to hire people. They’re desperate to hire people. I’ve—many restaurants, they can’t fill up all of their seats because they don’t have enough staff. The same thing is happening in hospitality and entertainment venues. If you can take some of those things that are, perhaps were, done by humans or still could be done by humans, if you can automate that, if you can replace that with compute, then you can serve, you can hold back your valuable human resources to do the stuff that people really like, which is the interaction. It’s the talking, it’s the setting up your experience.
That’s what you really need to do. Focus your human resources or your human talent on that interaction that people really like to really drive experiences, and all of that stuff in the background, all the operational, all the inventory, all the insights and stuff, set that up with computer vision and automation that excels at it. That’s what it’s really good at.
Christina Cardoza: Yeah. That’s a great point. You know, as an end user to some of these things—grocery stores, checking into hotels, ordering online food to go—I’m already seeing such a huge benefit with the inflammation of these technologies. And I can’t wait to only see it advance in the years to come. So with that, I just want to thank you both for joining the podcast today.
Chris O’Malley: Alrighty. Thank you very much. Have a good day. We’ll see you, Stephen.
Christina Cardoza: And thanks to our listeners for joining us today. If you liked this episode, please like, subscribe, rate, review, all of the above on your favorite streaming platform. And, until next time, this has been the IoT Chat.
The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel® trademark information.
This transcript was edited by Erin Noble, copy editor.