Insight Tech

Christina Cardoza: Hello, and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Associate Editorial Director of insight.tech. And today we’re talking about machine learning as part of your data strategy with Erik Bovee from MindsDB. But before we jump into the conversation, let’s get to know our guest. Erik, welcome to the show.

Erik Bovee: Thank you, yeah, it’s great to be here.

Christina Cardoza: What can you tell us about MindsDB and your role there?

Erik Bovee: So MindsDB is a machine learning service, and I’ll get into the details. But the goal of MindsDB is to democratize machine learning, make it easier and simpler and more efficient for anybody to deploy sophisticated machine learning models and apply them to their business. I’m the Vice President of Business Development, which is a generic title with a really broad role. I do—I’m responsible for our sales, but that’s kind of a combo of project management, some product management, kind of do everything with our customers. And then a really important aspect that I handle are our partnerships. So one of the unique things about MindsDB is that we enable machine learning directly on data in the database. So we connect to a database and allow people to run machine learning, especially on their business data, to do things like forecasting, anomaly detection. So I work with a lot of database providers, MySQL, Cassandra, MariaDB, Mongo, everybody. And that’s one of the key ways that we take our product to market: working with data stores and streaming brokers, data lakes, databases, to offer machine learning functionality to their customers. So I’m in charge of that. And also work with Intel^®. Intel’s provided a lot of support. They’re very close with MariaDB, who’s one of our big partners, and Intel also provides OpenVINO™, which is a framework which helps accelerate the performance of our machine learning model. So I’m in charge of that as well.

Christina Cardoza: Great. I love how you mentioned you’re working to democratize machine learning for all. I don’t think it’s any surprise to businesses out there that machine learning has become a crucial component of a data management strategy, especially, you know, when all the influx of data is coming from all of these IoT devices, it’s difficult to sift through all of that by yourself. But a challenge is that there’s not a lot of machine learning skills to go around for everybody. So I’m wondering, let’s start off the conversation, if you can talk about what the state of machine learning adoption looks like today.

Erik Bovee: Yeah, I mean, you summed up a couple of the problems really well. The amount and the complexity of data is growing really quickly. And it’s outpacing human analytics, and even algorithmic-type analytics, traditional methods. And also, machine learning is hard. You know, finding the right people for the job is kind of difficult. These resources are scarce. But in terms of the state of the market, there are a couple of interesting angles. First, the state of the technology itself, and core–machine learning model, is amazing. You know, just the progress made over the last five to ten years is really astonishing. And cutting-edge machine learning models can just solve crazy hard real-world problems. If you look at things like what OpenAI has done, with their large GPT-3 large language models, which can produce human-like text or even consumer applications, there’s a—you’ve probably heard of Midjourney, which you can access via Discord, which, based on a few key words, can produce really sophisticated, remarkable art. There was a competition—I think it was in Canada recently—that a Midjourney-produced piece won, much to the annoyance of actual artists. So the technology itself can do astonishing things.

From an implementation standpoint though, I think the market has yet to benefit broadly from this. You know, even autonomous driving is still more or less in the pilot phase. And the capabilities of machine learning are amazing in dealing with big problem spaces—dynamic, real-world problems, but adapting these to consumer tech is a process. And they’re just—there are all kinds of issues that we’re tackling along the way. One is trust. You know, not just, can this thing drive me safely? But then also, how do I trust that this model’s accurate? Can I put my—the fate of my business on this forecasting model? How does it make decisions? So those are, I think those are important aspects to getting people to implement it more broadly.

And then I think one of the things that’s really apparent in the market, and as I’m dealing with customers, are some of the hurdles to implementation. So, cutting-edge machine learning resources are rare, which we said, but then also a lot of the simpler stuff, like machine learning operations, turns out to be more of a challenge, I think, than people anticipated. So the data collection, the data transformation, building all the infrastructure to do this ETL link data, extracting, transforming it, loading it from your database into a machine learning application, and then maintaining all this piping and all these contingencies. Model serving is another one. Your web server is not going to cut it when you’re talking about large machine learning models for all kinds of technical reasons. And these are all being solved piecemeal as we speak. But the market for that is in an early stage. Those are dependencies that are really important for broad adoption of machine learning.

But there are a few, I would say there are a few sectors where commercial rollout is moving pretty fast. And I think they’re good bellwethers for where the market is headed. Financial services is a good example and has been for a few years. Big banks, investment houses, hedge funds, they’ve got the budgets and the traditional approach to hiring around a good quant strategy. They’re moving ahead pretty quickly, and often with well-funded internal programs. Those give them a really big edge, but they’ve got the money to deploy this, and it’s this narrow business advantage for things like forecasting, algorithmic trading are tremendously important to their margins. So I’ve seen a lot of progress there. But a lot of it is also throwing money at the problem and kind of solving internally these MLOps questions, not necessarily applicable to the broader market.

The next are, I would say, industrial use cases. You had mentioned IoT. That’s where I see a lot of progress as well, especially in things like manufacturing. For example, taking tons of high-velocity sensor data and doing things like predictive maintenance. You know, what’s going to happen down the line? When will this server overheat, or something? That’s where we’ve seen a lot of implementation as well. I think those sectors, those market actors are clearly maturing quickly.

Christina Cardoza: So, great. Yeah. I want to go back to something you said about trust. Because I think trust goes a little bit both ways. Here you mentioned how businesses have to trust the solution or the AI to do this correctly and accurately. But I think there’s also a trust factor that the person deploying the machine learning models or training the machine learning models, knows what they’re doing. And so, when you democratize AI, how can business stakeholders be confident and comfortable that a business or an enterprise user is training and creating these models and getting the results that they’re looking for?

Erik Bovee: Yeah. I think a lot of that starts with the data. Really understanding your data, make sure there aren’t biases. Explainable AI has become an interesting subject over the last few years as well. Looking at visualizations, different techniques like Shapley values or counterfactuals to see where, how is this model making decisions? We did a big study on this a few years back. Actually, one of the most powerful ways of getting business decision makers on board and understanding exactly how the model operates—which is usually pretty complex even for machine learning engineers, once the model is trained what the magic is that’s going on internally is not always really clear—but one of the most powerful tools is providing counterfactual explanations. So, changing the data in subtle ways that you get a different decision. Maybe the machine learning forecast will change dramatically when one feature in the database or a few data points, or just a very slight change, and understanding where that threshold is. It’s like, here’s what’s really triggering the decision making or the forecasting on the model in which columns or which features are really important. If you can visualize those, it gives people a much better sense of what’s going on and how the decisions are weighted. That’s very important.

Christina Cardoza: Absolutely. So, I’m also curious, you know we mentioned some of the challenges, a big one being not enough skill set or data scientists available within an organization, but I think even if you do have the skills available, it’s still complex to train machine learning models or to deploy these two applications. So can you talk about some of the challenges businesses face beyond skill set?

Erik Bovee: Interestingly—so, skill set is one, but that’s, I think that will diminish over time. There are more and more frameworks that allow people to get access, just data analysts or data scientists to get access to more sophisticated machine learning features that AutoML has become a thing over the past few years. And you can do a lot, you can go a long way with automobile frameworks, like DataRobot or H2O. What is often challenging are some of the simple things, some of the simple operational things in the short term, on the implementation side. You know, a lot of the rocket science is already done by these fairly sophisticated core machine learning models, but a huge amount of a data scientist’s or ML engineer’s time is spent on data acquisition, data transformation, cleaning the data and coding it, building all the pipeline for preparing this data to train and retrain a model. Then maintaining that over time.

You know, the data scientist tool set is often based on Python, which is where a lot of these pipelines are written. Python’s not necessarily, arguably not very well adapted to data transformations. And then what happens, you’ve often got this bespoke Python code written by a data scientist, and maybe things that are being done in a Jupyter Notebook somewhere, then it becomes a pain to update and maintain. What happens when your database tables change? Then what do you do? You’ve got to go back into this Python and it’s all reliant on this one engineer to kind of update everything over time. And so they—that’s, I think, the MLOps side is one of the biggest challenges. How do you do something that is efficient and repeatable and also predictable in terms of cost and overhead over time? And that’s something that we’re trying to solve.

And one of the theories behind that, behind our approach, is just to bring machine learning closer to the data and to use existing tools like SQL to do a lot of this stuff. They were very—SQL’s pretty well adapted to data transformation and manipulating data—that’s what it’s designed for. And so why not find a way where you can apply machine learning directly, via connection to your database, and use your existing tools, and not have to build any new infrastructure. So I think that’s a big pain point—one of the bigger bottlenecks that we’re trying to solve actively.

Christina Cardoza: So, you touched on this a little bit, but I’m wondering if you can expand on the benefits that the data scientists will actually see if we democratize machine learning. How can they start working with some of those business users together on initiatives for machine learning?

Erik Bovee: Yeah. So one of our goals is to give data scientists a broader tool set, and to save them a lot of time on the operational, the cleanup and the operational tasks that they have to perform on the data, and allow them really to focus on core machine learning. So the philosophy of our approach—we take a data-centric approach to machine learning. You’ve got data sitting in the database, so why not bring the machine learning models to the database, allow you to do your data prep, to train a model. Let’s say, for example, in an SQL-based database, using simple SQL with some modifications as SQL syntax from the MindsDB standpoint. We don’t—we’re not consuming database resources; you just connect MindsDB to your database. We read from the database, and then we can pipe machine learning predictions, let’s say business forecasts, for example, back to the database as tables that can then just be read like your other tables.

The benefit there for data analysts and any developer who’s maybe building some application on the front end that wants to make decisions, algorithmic trading, or, you know, anomaly detection. You want to send up an alert when something’s going wrong, or you just want to visualize it in a BI tool like Tableau is that you can use the existing code that you’ve got. You simply query the database just like you have from another application. There’s no need to build a special Python application or connect to another service. It’s simply there. And you access it just like you would access your data normally. So that’s one of the business benefits, is that it cuts down considerably on the bespoke development, is very easy to maintain in the long term, and you can use the tools you already have.

Christina Cardoza: So you mentioned you’re working to bring machine learning closer to the data, or bringing machine learning into the database. I’m wondering, is this how, traditionally, machine models have—machine learning models have been deployed, or is there another way of doing it? So, can you talk about how that compares to traditional methods—bringing it into the database versus the other ways that organizations have been doing this?

Erik Bovee: So, traditionally machine learning has been approached like people would approach a PhD project, or something. It’s, you would write a model using an existing framework like TensorFlow or PyTorch, usually writing a model in Python. You would host it somewhere, probably not with a web server there are Ray and other frameworks that are well adapted to model serving. And then you have data you want to apply. It might be sitting all over the place, and maybe it’s in a data lake, some in Snowflake, some is in MongoDB, wherever. You write pipelines to extract that data, transform it. You often have to do some cleaning, and then data transformations and encoding. Sometimes you need to turn this data into a numerical representation, to a tensor, and then feed it into a model, train the model. The model will spit out some predictions, and then you have to pipe those back into another database, perhaps, or feed them to an application that’s making some decisions. So that would be the traditional way. So you can see there’s a bespoke model that’s been built. There’s a lot of bespoke infrastructure, pipelines, ETLA that’s been done. That’s the way it’s been done in the past.

With MindsDB what we did is we have two kind of—MindsDB has two components. One is a core suite of machine learning models. There’s an AutoML framework that does a lot of the data prep and encoding yourself. And we’ve built some models of our own, also built by the community. But I forgot to mention MindsDB is a large, one of the largest machine learning open source projects. We have close to 10,000 GitHub stars. And there’s a suite of machine learning models that are adapted—regression models, gradient boosters, neural networks, all kinds of things that are adapted to different problem sets. MindsDB can make a decision looking at your data what model best applies and choose that.

The other piece of this core, this ML core of MindsDB, is that you can bring your own model to it. So if there’s something you like particularly—Hugging Face, which is like an NLP model, language processing model—you can actually add that to the MindsDB ML core using a declarative framework. So, back in the day you would have to, if you wanted to make updates to a model or add a new model, you’d have to root around in someone else’s Python code. But we allow you to do this to select models—select the model you want, bring your own model. You can tune some of the hyper-parameters, some things like learning rate, or change weights and biases using JSON, using human-readable format. So it makes it much easier for everybody to use.

And then the other piece of MindsDB is the database connector—a wrapper that sits around these ML models and provides a connection to whatever data source you have. It can be a streaming broker, Redis, Kafka. It can be a data lake like Snowflake. It can be an SQL-based database where MindsDB will connect to that database, and then using the natural—using the query language, native query language, you can tell MindsDB, “Read this data and train a predictor on this view or these tables or this selection of data.” MindsDB will do that and then it will make the predictions available. Within your database you can query those predictions just like you would a table. So it’s a very, very different concept than your traditional kind of homegrown, Python-based machine learning applications.

Christina Cardoza: And it sounds like a lot of the features that MindsDB is offering with its solution, data scientist can go in themselves and expand on their machine learning models and utilize this even more. So if you do have a data science team available within your organization, what would be the benefit of bringing MindsDB in?

Erik Bovee: This is the thing that I think it’s important to make really clear. We are not replacing anybody, and it’s not really an AutoML framework. It allows for far more sophisticated application machine learning than just a tool that gives you in a good approximation of what a hand-tuned model would do. So it basically, for a machine learning engineer or a data scientist internally, MindsDB, we would just save a tremendous amount of, you know, that 80% of their work that goes into data wrangling. Cleaning, transforming, and coding. They don’t have to worry about that. They can really focus on the core models, selecting the data they want to train from, and then building the best models, if that suits them, or choosing from a suite of models that work pretty well within MindsDB, and then also tuning those models. A lot of the work goes into kind of adapting, changing the tuning, the hyper-parameters of a model to make sure you get the best results, make that much simpler; you can do that in a declarative way rather than rooting around in someone’s Python code. So the whole thing is about time savings, I think, for data scientists.

And then, in the longer term, if you connect this directly to your database, what it means is you don’t have to maintain a lot of the ML infrastructure that up until now has been largely homegrown. If your database tables change, you just change a little bit of SQL—what you’re selecting and what you’re using to train a predictor. You can set up your own retraining schema. There are just lots and lots of operational time- and cost-saving measures that come with it. So it allows data scientists and machine learning engineers really to focus on their core job and produce results in a much faster way, I think.

Christina Cardoza: Great. Yeah. I love that point you made that it’s not meant to replace anybody or data science per se, but it’s really meant to boost your machine learning efforts and make things go a little bit smoother.

Erik Bovee: Yeah. In a nutshell, it just saves a data scientist tons of time and gives them a richer tool set. That’s—that was our goal.

Christina Cardoza: So, do you have any customer examples or use cases that you can talk about?

Erik Bovee: Yeah, tons. I mean, we concentrate. They fall into two buckets. We really focus on business forecasting, often on time-series data. And time-series data can be a bit tricky even for seasoned machine learning engineers, because you’ve got a high degree of cardinality. You’ll have tons of data, let’s say, where there are many, many unique values in a column, for instance—by definition that’s what a time series is—and if you could imagine you’ve got something like a retail chain that has maybe thousands of SKUs, thousands of product IDs across hundreds of retail shops, right? That’s just the complex data structure, and trying to predict what’s going to sell well—maybe a certain SKU sells well in Wichita, but it doesn’t sell well in Detroit. And how do you predict that? That’s a sticky problem to solve because of the high degree of cardinality in these large, multi-variate time series. But it also tends to be a very common type of data set for business forecasting. So we’ve really focused our cutting-edge large models on time-series forecasting. So that’s what we do. We will tell you what your business is going to look like in the future, in weeks or months.

The other thing that we see in the use cases—so it’s forecasting on time series, and then also anomaly detections. So it’s fraudulent transactions, or, is this machine about to overheat. Getting down into the details, I can tell you, across the board, all kinds of different use cases. One very typical one is for a big cloud service provider. We do customer-conversion prediction. They have a generous free-trial tier, and we can tell them with a very high degree of accuracy based on lots of columns in their customer data store and lots of different types of customer activity and the structure of their customer accounts who’s likely to convert to paying tier and when. And precisely when, which is important for their business planning. We’re working with a large infrastructure company, Telco, on network planning, capacity planning. So we can, we can predict fairly well where network traffic is going to go, and where it’s going to be heavy and not, and where they need to add infrastructure.

We’ve also worked on—this is a typical IoT case—manufacturing process optimization and semiconductor. So we can look at in real time sensor data coming in from the semiconductor process. And we can say, when do you stop and go on to the next phase of the process, and where default’s also likely to arise based on some anomaly detection on the process. That’s one we’ve seen working on one project in particular, but we’ve seen a couple like that in pilot phases. Been doing credit scoring real estate, like, payment-default prediction, as well as part of the business forecasting. So, those are all typical, and we, across the board, we see forecasting problems on time series.

One of the actually most enjoyable projects, it’s unique and interesting, but it’s really close to my heart, is we’re working with a big esports franchise building forecasting tools for coaching for video games. For professional video game teams. Like, how would you—what can you predict what the other team’s going to do for internal scrimmages and internal training for their teams? And what would be the best strategy given a certain situation on some complex, like MOBA games, like League of Legends or Dota 2? So that’s something we’re working on right now. They’ve already built the tools in the front end of these forecasting tools. And they’re—we’re working with very large data sets, proprietary data sets of internal training data to help them optimize their coaching practices. It’s an exotic case, but I guarantee you that’s going to grow in the future. So that’s one of the most interesting ones.

Christina Cardoza: So, lots of different use cases in ways that you can bring these capabilities into your organization efforts. But I’m wondering, in your experience, what is the best place to start on democratizing machine learning for a business? Where can businesses start using this? And where do you recommend they start?

Erik Bovee: Super easy. “Cloud.mindsdb.com.” It’s—we have a free-trial tier. It’s super easy to set up and get you signed up for an account. And then we have—God knows how many, 50-plus data connectors. Wherever your data’s living, you can simply plug in MindsDB and start to run some forecasting and do some testing and see how it works. I mean, you can take it for a test drive immediately. I would—that’s one of the first things that I would recommend that you do. The other thing is you can join our community. If you go to MindsDB.com, we’ve got a link to our community Slack and to GitHub, which is extremely active. And there you can find support and tips. And if you’re trying to solve a problem, almost guaranteed someone solved it before and are available on the Slack community.

Christina Cardoza: Great. I love when projects and initiatives have a community behind it, because it’s really important to learn what other people have been doing and to get that outside support or outside thinking that you may not have been thinking about. And I know you mentioned, Erik, in the beginning, you guys are also working with Intel on this. I should mention the IoT chat and insight.tech as a whole are sponsored by Intel. But I’m curious how you are working with Intel and what the value of that partnership has been.

Erik Bovee: Yeah, so that’s actually been—Intel has been extremely supportive on a number of fronts. So, obviously, Intel has a great hardware platform, and we have implemented their OpenVINO framework, whichoptimizes machine learning for performance on Intel hardware. So make great performance gains that way. And on top of that, they just, Intel provides tons of technology and kind of go-to-market opportunities. We work with them on things like this. I’ll be presenting at the end of the month, if anybody wants to come check us out at Intel Innovation in San Jose, I think it’s on the 27th, 28th, 28th, 29th of this month at the San Jose Convention Center. And we’ll have a little booth in the AI/ML part of their innovation pavilion. And I’ll be demoing how we work, running some machine learning on data in MariaDB, which is an Intel partner. Actually MariaDB introduced us to Intel, and that’s been really fruitful. Their cloud services are hosted on Intel. So if anybody wants to come and check it out, that’s—Intel has provided us this forum. So they’re—we’re extremely grateful.

Christina Cardoza: Perfect, and insight.tech will also be on the floor at Intel Innovation. So, looking forward to that demo that you guys have going on there at the end of the month. Unfortunately, we’re running towards the end of our time. I know this is a big, important topic and we could probably go on for a long time, Erik, but before we go, are there any final key thoughts or takeaways you want to leave our listeners with today?

Erik Bovee: I would just urge you to go—I mean, we love the feedback, even if you’re, you know—go test it out. It’s actually fun. MindsDB is pretty fun to play with. That’s how I got involved. I discovered MindsDB by chance and installed it and started using it and found it was just useful in all kinds of data sets and just doing science experiments. We love it. If you take it for a test drive and provide feedback on the community Slack, we’re always looking for product improvements and people to join the community. And so we’d really welcome that—“ cloud.mindsdb.com.” And thanks very much for the opportunity, Christina.

Christina Cardoza: Yep, of course. Thank you so much for joining the podcast today. It’s been a pleasure talking to you. And thanks to our listeners for tuning in. If you like this episode, please like, subscribe, rate, review, all of the above on your favorite streaming platform. And until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel^® trademark information.

This transcript was edited by Erin Noble, copy editor.

Christina Cardoza: Hello and welcome to the IoT Chat, where we explore the latest developments in the Internet of Things. I’m your host, Christina Cardoza, Associate Editorial Director of insight.tech, and today we’re talking about creating innovative solutions without compromise with Scott Chmiel from Arrow, and Steen Graham from Scalers AI. But before we jump into the conversation, let’s get to know our guests. Scott, I’ll start with you. Welcome to the show. Please tell us more about yourself and your role at Arrow.

Scott Chmiel: Thank you very much, Christina. With Arrow for more than 17 years, multiple roles starting at field sales and engineering, moving through my current roles: business development for cloud and edge development, helping customers realize their solutions on edge with Arrow orchestrating and helping where we can. Multiple roles, very focused on Microsoft and other software platforms.

Christina Cardoza: Great. Looking forward to digging in more on where Arrow fits into this IoT, innovative landscape. But let’s introduce Steen as well. Welcome to the show. What can you tell us about yourself and Scalers AI?

Steen Graham: Christina, thanks so much for having me, and I must admit you guys produce some great content and have had some great guests. So, just really thrilled to be here with Scott showcasing some of the great work we’ve done together. And I’m the CEO and co-founder of Scalers AI, and we’re an enterprise AI company that’s focused on deploying AI in the physical world to drive industry transformation and ultimately enrich our lives.

Christina Cardoza: Great. And now something you both sort of mentioned in your introduction is really helping customers navigate through this new digital transformation, intelligent IoT world, and it certainly is a lot easier said than done. There’s a lot of benefits that companies want to get to, but a lot of challenges that are standing in the way. So, Scott, I want to start with you to jump into this conversation. What can you tell us about the current industry transformation efforts? How well have they been applied, and what are the challenges that businesses are coming across?

Scott Chmiel: Well, the first thing that comes to mind is the challenges have changed quite a bit over time. The complexity of solutions has just increased. From days in the past where we were talking about simple appliances or devices that we’re doing, everything contained in the single piece of hardware or software, we’re adding cloud, we’re adding complexity, we’re adding new technologies which not only require more from the technology standpoint, but different skill sets from the development. As you create these solutions, you have to integrate them and deploy them in existing environments or customer environments that differ from one to another. There’s just a complexity of those environments. And, for instance, connected devices now require that additional operational technology security be applied and looked at to make sure that not only is the solution working but it’s secure, it’s not creating vulnerabilities, and, obviously, we can do new things that weren’t possible for the advancement to machine learning and AI, it’s possible to solve new business problems that we couldn’t even address in the past.

Christina Cardoza: Definitely seeing a lot of complexity and challenges on our side too, as we talk to partners on insight.tech, and I think what I see from my standpoint is that this IoT ecosystem, it’s really a partner ecosystem and there’s a lot of players involved. So it’s not only if a business is failing or a business is struggling that can have ripple effects to other issues in the industry or in society. I’m thinking about the supply chain, for instance. So, Steen, I’m wondering if you can expand on the impact that you’ve seen those challenges being, both on a business level, but a society level too.

Steen Graham: Absolutely, Christina, and I think Scott said it well as he kind of outlined the challenges that we have in this physical world in deploying artificial intelligence in the IoT in the physical world to drive industry transformation. It’s really challenges across development and developing these new unique technologies. Deployment and data is incredibly important, and when you look at something like a port, obviously these ports and the infrastructure for ports has been around for decades, and so what you get is you get a mix of existing applications that are working just fine, just fantastic in a port environment, but then you want to implement some technologies that Scott’s really familiar with, like those cloud and native technologies, and how do we actually deploy these cloud-native methodologies, including artificial intelligence, on existing infrastructure, and so can cohabitate with the existing infrastructure and provide these added enhancements so we can do things like analyze the efficiency of the ports, monitor the CO₂emissions so we reduce the fossil fuel burn associated with the ports as well. And so this combination of existing infrastructure and new infrastructure, both from a hardware perspective and from a software perspective, and being able to acquire the right data so you can make these near real-time AI decisions, are all very critical in driving these industry transformations and solving some of the challenges we face, for example, in our supply chain crisis.

Christina Cardoza: Great, and I want to get into more about how businesses and companies can actually apply new and innovative capabilities and technologies to some of their existing and legacy technology. But one thing that stands out to me when we’re talking about the ports—this is the part in the supply chain where you’re delivering product and containers, and trucks have to come and offload those from ships, and it’s a huge bottleneck to do that right now, which is causing a lot of delays in getting product out for businesses and end users. And there’s not one single company or business that owns a port like that. So, I’m interested with all of these different players involved in something like a port, which is just one aspect of a supply chain, how can you make innovative and impactful changes when you have multiple people and businesses involved?

Steen Graham: So, I think contextually maybe we’ll start at the macro level. The US federal government administration has been fantastic, and supporting port modernization. They recently passed the infrastructure bill, which is about $17 billion allocated to modernizing our ports. In addition to that, in the newly approved inflation reduction act, there was another—a few billion dollars around monitoring CO₂emissions at ports. But one of the really unique things about ports is they’re actually managed by the local municipalities, which is quite interesting. Insofar as you can look at the ports of Long Beach or Los Angeles and see that 30% of traffic goes through Los Angeles and Long Beach—and that’s managed or containerized traffic in the United States goes through these two ports managed by local municipalities. So, what those leaders do locally impacts all of us at a US scale, so that’s where not only those local municipalities and their leadership is critically important, unions are also critically important as well. And so, one of the job roles that is sustained in the United States is crane operations, and what we’ve automated is the front end, actually removing the containers from the ship. But where we really have heavy, invested, human, union-based roles is in loading and unloading those trucks at ports, and so that’s one of the key bottlenecks that you find as well, but those three critical parties: the federal government on igniting the opportunity financially and creating the frameworks, the local municipalities are really the leaders in this front, and then finally the unions are incredibly important. They do occasionally go on strike too, so you can imagine the impact it would have if the unions went on strike in our current challenge as well. And so, that’s really the key players, and obviously I didn’t talk about technology at all, because I think those three angles are tough enough to navigate. Deploying the technology performantly. We’ll talk a little bit more about that, but that’s a broad challenge in itself as well.

Christina Cardoza: So, Scott, I’m interested from your perspective when you take all the challenges that Steen just talked about, as well as the industry business challenges that you spoke about in the beginning, how do you see businesses making impactful transformations and changes?

Scott Chmiel: Well, it’s obviously the challenge to understand what business outcome they’re seeking. That’s often the first step, is what are they trying to accomplish and who are the stakeholders. As Steen pointed out the port of Los Angeles—there’s not just one company, there’s the municipality, there’s the people handling the containers, there are the truck drivers—there are dozens, if not hundreds of subcontractors who all run that and have to dance around to move the ports. When one of those things breaks down, the whole system breaks down. So our solution focuses a little bit on one of the challenges they have there, around safety and just tracking in and out.

Christina Cardoza: Great. And seeing you mentioned tools and technologies as being a part of this, we haven’t gotten too deep into it yet, but we were talking about the ports. We have crane operations, we have different loading and offloading technology already in place—has been in place for years—but at the same time, these governing bodies and businesses want to start adding new and innovative solutions to start tackling and addressing these problems, and that can be challenging when you’ve had these other things in place for years, if not decades. So how do you go about deciding what to replace, what to upgrade, and get over challenges where you just can’t add a new technology or capability because the tools in place aren’t allowing you to do so?

Steen Graham: I think what Scott and I have looked at is how to deploy in a no-compromise way, and from a simplistic operating system perspective, which is foundational to technology, we know that there’s two pervasive operating systems in the world: notably Windows and Linux. And those cloud-native workloads in the modern AI workloads are written in Linux, whereas a lot of existing workloads and applications have been written in Windows. For example, one of the common operating systems for cranes is Siemens SIMOCRANE, which is a Windows-based operating system. For example, there’s Windows 32 applications that are alarm managers to notify the crane operators when it’s safe to proceed based on sensor-based data proximity around them. And so, with that kind of foundational element of no compromise between Windows and Linux, existing applications and new applications, we’re able to retrofit these modern AI applications on existing infrastructure and make sure that they work better together. That avoids the long conversation, Christina, that you alluded to, of when to upgrade our infrastructure, what to keep, what to remove. All of the retraining associated with employees, as well, on that digital transformation effort ultimately becomes a multiyear effort if you start replacing existing applications that are working just fine today. So layering on the modern, cloud-native attributes and AI capabilities was really the approach we used in this solution.

Christina Cardoza: So, when we’re looking at adding cross-platform capabilities to some of these technologies and looking at the port-modernization efforts, as an example, getting Linux on cranes instead of the traditional Windows, or being able to use both together—how do you actually go about that? What is driving that cross-platform interoperability? Scott, if you want to answer that one.

Scott Chmiel: Well, a lot of times it’s the existing hardware, and I’m going to let Steen answer it a little bit after. The example we use for EFLOW was a specific issue we identified, or challenge that can be addressed, but the technology, the infrastructure—whether it’s the hardware or it’s the codebase—can be applied to many different solutions depending on where you are in the port, where you are—whether it’s a retail application or within a smart port—there’s many different places—or in a warehouse. It’s all the same types of challenges or same technology can be used and can be customized or repackaged. And retraining AI, they have different Windows applications, and the integration that’s being done—that’s the opportunity for SIs or the people building the solution, is bringing this off-the-shelf technology and building solutions they couldn’t do anymore for the specific vertical, for the specific problem they have or identify. It’s bringing additional value to the existing hardware they have, adding value with things they couldn’t do. And in the particular example we did with the smart port was adding safety. One of the most significant things is the amount of people and how things fast things have to move. A mistake is critical. It’s devastating if a mistake is made. If somebody’s standing where they’re not supposed to be. And that that’s an example we try to illustrate. Obviously, in retail it might not be as critical as a fault. But then again, some people aren’t supposed to stand something—or stand someplace, or before a crane moves through a warehouse making sure this place is void. Anything you can attribute to safety is value, and ultimately a cost benefit for the customers and their customers. Steen, would you like to add a little bit to that?

Steen Graham: I think just to expand upon that is the gift that we were given, notably by Microsoft and Intel, is that underlying technology, which we use the acronym EFLOW, which is Edge for Linux on Windows. It’s now accurately described as, I think, Azure IoT Edge for Linux on Windows. Quite a mouthful, but the reason the mouthful is important is it gives us that no-compromise capability across Windows and Linux. I mean, that’s something that’s really unique to running applications that are just running great—existing applications and these modern applications. And so that’s been fantastic, and maybe the hidden gem from Intel is that Intel invested in hardware-acceleration capabilities via its integrated-graphics capability that allows us to do these workloads on deployed Intel-based CPUs today without having upgrade to expensive GPS. Why that’s so unique is typically you’re abstracted from access to the integrated graphics if you’re running an AI model and Linux via Windows, but that was a really hidden gem that Intel produced, and now we can run multiple AI models, multiple camera feeds on affordable, off-the-shelf technologies like Intel’s net platform, and Windows and Linux as well. So, an incredible array of technology that allows us to deploy these modern workloads and make sure they’re interoperable with existing infrastructure.

Scott Chmiel: And I’ll add one more thing too, is the investment companies have done in their Windows architecture, their infrastructure, they have people who understand Windows. They have people who manage those Windows devices. When you bring something into an existing application OT, there usually is an IT element there managing those devices. The value is you’re still utilizing that skill set, that expertise you bring to the table, and now you’re adding that modern, cloud-based, machine learning AI, so they’re leveraging additional value on top of their skill set. So, once again, all the investments that would’ve been made can be reused and can be built on top of.

Christina Cardoza: Now, I should mention the IoT Chat, and insight.tech as a whole, is owned by Intel, so always love hearing about what Intel is doing and how they’re helping others in the industry. But I want to expand a little bit more on EFLOW—what the process was getting this into the port system, how hard, or where the initiative came from, and then the benefits that we’ve seen in the supply chain because of EFLOW. Steen, I’ll throw that one to you.

Steen Graham: So, in regards to the port system, it’s quite an extensive, multiyear process of RFIs and RFPs. So, this was technology—the EFLOW technology—was released just late last year, so we really built this solution within the past year and are still in the engagement phase on a number of opportunities with ports to demonstrate in RFP and RFI mode. So, even though the technology’s highly ready to solve some of these challenges, as you alluded to earlier, the multilayered decision making process is what you kind of run into on these sophisticated situations, and so that’s certainly something we’re working through. From a business-outcome perspective, the problem that we were trying to address is the bottleneck associated with the turn times: the operational-technology metric of how fast the containers can be loaded and unloaded. You may have seen at the height of the crisis you would find many times the truck drivers blaming the crane operators for being unfamiliar with their workstation because the union assigns them a workstation based on their seniority on a daily basis, and as COVID was in place as well, it’s even more dynamic union positioning. Meanwhile, the crane operators were not happy with the truck drivers because they were no-showing about 50% of their appointments, and because of the goods shift in US consumption from services to physical goods, we had peak traffic of containers as well. So you’ve got more people than ever on the port. And so the business problem that we’re trying to solve is how do we optimize those turn times of those cranes? How fast can they be unloaded and loaded? How do we make sure the truck’s in the right place at the right time, and efficiently does that while providing enhanced safety experiences for the workers on site as well. And we also are tracking CO₂ emissions based on the fossil fuel consumption of those cranes. Although, I should say there is a mix between hybrid cranes as well as diesel cranes, so many ports have a nice mix—like, for example, the Port of Houston has about one-third hybrid cranes and two-thirds diesel cranes. And so that’s another metric that we’re tracking, is how efficient are those hybrid cranes? Are the operators trained and familiar with that as well?

Christina Cardoza: So, a lot of the things that we’ve been talking about—this problem between Linux versus Windows, and legacy technology versus new technology—we’ve been talking about these in regards to ports and smart ports. But these sound like challenges that every industry deals with in some shape or form. So, Scott, I’m wondering where else do you see EFLOW being used outside of ports, and what other challenges or benefits is EFLOW looking to solve and giving to businesses?

Scott Chmiel: Well, I think there’s, depending on who the audience is, it’s going to be different verticals they’re focused on. I know there’s a strong focus on the retail from both Microsoft and Intel: the opportunities there to do workload consolidation. I think the example we’ve shared or talked about it a little bit is a consolidation of surveillance and point of sale, where one machine could do both or, once again, new services you couldn’t do before. Now what you have, a visual element with a transaction, what kind of value can you generate out of that? What kind of benefit to the business, benefit to the customer? Obviously the transportation, when we talk about smart ports, we can expand on that and say transportation in general—whether it’s warehouse, workflow management, the smart port, we’re talking crane—but there’s a lot going on in ports of entry and warehouses, and if you think about just a logistical hub where trucks are coming and going, and how much volume of material is exchanged between trucks or put on shelves. There’s the accelerator—we did both talk about efficiencies—but also safety using AI—Visual AI to, for instance, detect where people shouldn’t be—giving warnings: things are moving fast; things are big that are moving; people and those machines don’t always interact very well if they’re in the wrong place.

So there’s lots of opportunities, and I think transportation, industrial and retail are a lot of them. I’m sure somebody innovative in a different vertical, like medical, has applications as well. But the great thing is the people who understand those industries, the people who have the IP and the investment in those industries, they understand the solutions they’re trying to solve, and a lot of that—the code, the underlining technology—can be repurposed for those verticals. And, once again, a lot of the work’s already done for them with the accelerators and the tools that Microsoft and Intel with OpenVINO have provided to be leveraged for their use. It’s pretty exciting, what they can do, and I think if you get Steen and I going, we can talk about and get pretty excited about, and start making up solutions, but I think the end customers, the people who live and breathe in those industries, know the kind of problems, and, like I said, being aware of the new technologies and what EFLOW can do, they’re going to start coming up with some pretty exciting ways to apply it from very simple, solving simple problems, to more complex problems they might have on their sites or locations or their vertical.

Steen Graham: Just to maybe extend Scott’s thoughts on a couple of these industries, and maybe starting with healthcare where he left it, is if you look at medical-imaging equipment, such as ultrasound, a lot of ultrasound vendors are Windows-based applications, but they’re looking to add new, AI-based features. And so, for example, to make sure that mother and child are safe through pregnancy, you can look at the fetal position of the baby as it exits the womb. And that’s something that you can take an existing Windows-based ultrasound equipment with, and then overlay modern deep-learning capability as well. Another example is anesthesiologists occasionally have challenges finding the veins on folks, and it could be a material difference if you obviously don’t hit the vein correctly. And so you can use ultrasound equipment—again, Windows-based with modern deep-learning skills—to determine the accuracy of the vein, and then pinpoint the associated anesthesiology inputs with that one as well. So that’s a couple of use cases in medical where you have this demand to run—my existing ultrasound works great on Windows, and now I’d like to overlay some modern deep-learning capability as well.

And so that’s a good use case in factories—I think in manufacturing Windows is pervasive, but we’ve also seen just an incredible demand around things like computer vision to do defect detection in line in the manufacturing-process flow, and I think it’s an incredible use case. And I think perhaps kind of the hidden cool factor in that computer vision for quality detection use case, if you do in-line AI defect detection, you can actually find products that are essentially having quality issues earlier in the manufacturing flow, which is nice because of course you can address product-quality issues. But simultaneously, if you address that earlier in the flow, you use less fossil fuels to actually run through the rest of the process; you use less raw commodities as well, and so there’s a sustainability effort that’s quite impressive in doing some of those use cases as well. And again, huge install base of Windows and manufacturing facilities as well, so across transportation, healthcare, manufacturing, there’s really some incredible opportunities to pair modern Linux-based, cloud-native apps and AI on top of Windows via this technology. So, kudos again to the teams developing it and continuously updating it at Microsoft.

Christina Cardoza: Absolutely. I love those examples because it really shows you these benefits go beyond businesses and really have a ripple effect to society. Now, you guys have mentioned Intel and Microsoft ,and you’re from Scalers AI and Arrow, so, I’m curious about the partnership that goes into this. Typically businesses, you don’t think that they synergize or work well together, but really this IoT landscape—and when you look at supply chain as another example—it really takes a team effort. So how are you guys working together, and working with some of those other partners you mentioned: Microsoft and Intel?

Steen Graham: I think Arrow is kind of a natural fit for working across partners and solving these multipartner solutions, because they’re one of the leading solution providers in the industry. And Arrow is always, I think, looking to figure out how they can make one plus one equal three across their partnerships. And that’s really where Scott actually came to us with an incredible idea about showcasing the value of this underlying technology, which is quite primitive relative to deployment with a real business outcome, and we were able to take those technologies from Intel and Microsoft, a number of open source projects as well. So there’s many other we could modern—we would call them modern microservices in many cases: full open source projects as well, so I have a hard time calling them a microservice because they’re a macro set of code. But we were really building upon numerous open source projects, numerous technologies from Intel and Microsoft and others, and building that solution code and where Scalers fits in is really understanding how to fit all those things together into a solution and providing that high-fidelity enterprise AI solution, as well as building the custom AI models for deployment. And so Scott, would you like to add anything on?

Scott Chmiel: Arrow’s one of those companies that, unless you’re in electronics, a lot of people don’t know who we are. We’re over $30 billion in 2021, and we cover quite a wide breadth of different things Arrow does, from the enterprise to components and everything in between. The thing that Arrow focuses on, and the intelligent solutions of Pacific group that I’m in, is we call ourself an orchestrator and aggregator, bringing the different technologies—the trusted advisor role orchestrating different partners. Because, once again, we talked about the complexity—it’s hard for one company who has a vision or a challenge to be able to necessarily have the resources, the skill sets in house to do everything for an end-to-end solution. Obviously they might have a component—that might be their IP, their technology, a device, a sensor, something that does something really well—but, as I said, the solutions, especially in operational technology, are wide and deep within the end use.

So what Arrow looks to do is work with the end user, the prime whoever’s coming up with the solution, and bring in appropriate partners. Those could be technology partners, they can be existing technology-solutions systems, the compute from Intel, different form factors, the IO—and if the customer has something which doesn’t exist in the market, help them build it, help them pick the right solutions, not only for their end use, but looking at the longevity, the overall life cycle of that solution. Smart ports—that’s not something that’s going to be deployed and done in a couple of years. And, once again, you don’t want something on each crane being different. That’s the important point, something that’s repeatable. And, obviously if you can do it in smart port—Port of Los Angeles—I’m sure you want to move it down to the Port of LA—move it to, excuse me, Port of Houston, other locations, if you can reuse that. There’s even inland port. We think of ports as next to water—no, there’s a lot of hubs for transporting and redoing containers throughout the US, and where they distribute from the rail systems, from the trucking systems—if you can reuse it, the company who’s developing that solution or who is bringing these pieces together can reuse it, and, once again, create more scale, create more value across the ecosystem.

So Arrow works on orchestrating, bringing the appropriate partners when a company deploys a solution. I’m going to go back to operational technology—security’s a huge thing. There are a lot of options in what you need to do to deploy into a system—making sure the sensors, the gateways, all the devices are secure in that operational technology. Because, keep in mind, it might be not just their network, it might be on a facilities network. How are you securing those devices? How are you making sure that data’s safe? That the devices aren’t vulnerable to people with devious intentions. And, of course, just making sure it works. That’s very important. So, once again, Arrow is orchestrating—whether it’s services, whether it’s components, whether it’s helping them with design—helping them make the right technology solutions. That’s really important. Talking about Intel—the long relationship we have with Intel, there’s a lot of products to look at. There’s a lot of ways to do things. We want to make sure that the customers are educated and selecting the right product for their solution, for their usage model.

Christina Cardoza: Great. Well, I can’t wait to see what else comes out of your partnerships with each other and Intel and Microsoft, as well as where else EFLOW is going to make some of these innovative transformations for the industry. But, unfortunately, we are running out of time on the podcast. Before we go, I know we’ve covered a lot, I just want to throw it back to each of you quickly if there’s anything that we didn’t go over that you think it’s important for our listeners to know, or any final thoughts or key takeaways you want to leave them with today. So, Steen, I’ll throw it to you first.

Steen Graham: Thank you, Christina. Well, it’s been a pleasure talking with you and Scott about this transformative technology. I think, to the listeners, I think as we talk about the cost of development and software engineering and really solutioning these, it’s incredibly important that we write the code to integrate these partnerships. And there’s so many incredible companies with great technologies, but what many times is missing is the single line of code that connects the APIs to really drive transformation as well. And so I think that’s really important that we take that step to fortify our partnerships so we can drive these industry transformations. And, as an industry, we really got to come together on the deployment challenge, because building capabilities in the cloud is fantastic, and it’s really affordable and easy to do these days. And so, Applied AI is affordable as it’s ever been, it’s democratized as it’s ever been. But where challenges occur is deploying it in the physical world, and the continuous learning, the transfer learning, the continuous annotation requirements to do that. And so, I think as an industry we’ve got to continuously focus on deployment and DevOps capabilities as well. And finally, I think, although, we’re getting really good at synthetic data and creating AI models with small data sets, if we want really want to move society forward, we have to be able to build models with high fidelity on good data sets and do it in a way with explainable AI so we know why the AI is making its determinations, which is very frequently required for mini RFPs these days because we want to make sure we know why the artificial intelligence is coming to its conclusions to make sure it’s as inclusive as possible and accurate as well. So those are certainly the three areas that I would emphasize here. You’re fantastic listeners here, Christina, and it’s been a pleasure. Thanks for having me.

Christina Cardoza: Great points, yeah, absolutely. And, Scott, any final thoughts or takeaways you want to leave today?

Scott Chmiel: I’m always amazed when I talk to companies in specific verticals—whether it’s somebody running a warehouse, somebody in a port, somebody in surveillance or whatever, the medical industry—the amount of knowledge they have with what they do, their particular industry—their particular solutions are amazing. And as these solutions get more complex, I want to make sure people understand there’s no need to go alone on some of the more complex solutions. It’s no longer the days of building a device that does one thing. It’s not an MRI which just does visioning; it’s how it integrates with that hospital. It’s how additional leveraging, additional technology—whether helping the visual inspectors for all that vision scanning, finding—being zeroed in certain things. And let’s talk honestly, with machine learning AI you’re not only bringing the expertise of one person; it’s how you train that and the thousands of models and thousands of things you can leverage. Companies don’t need to do it alone. They really can’t do it alone when they do more complex solutions, and Arrow has a lot to do that they can leverage to help them kit that end-to-end solution. And, once again, learning about the technology, learning about them, them learning about what can be done, what they hadn’t thought about doing, or the thought was outside the scope of what they can actually accomplish in their solution. The bar is moving down for what can be done, and it’s amazing—business solutions that couldn’t be solved in the past can be. It’s just getting the right partners, technology, hardware, ready-made solutions to be available for you to look at and to leverage, get your time-to-market faster, and it can save you money on—once again, why reinvent the wheel if there’s a piece of your solution that you can reuse from somebody else? It’s the final solution that’s brought together, that’s the important thing, and most likely the customer that knows that vertical is trying to solve the problem. They’re the experts on that, but let’s help them get that, and solve those problems with what’s out there.

Christina Cardoza: Yeah. I love that, that last thought: the bar is getting low for what we can do, but expectations are high for what others think we should do. So having the right partners is definitely going to help you be successful creating these new and innovative apps. So with that, I just want to thank you both for joining the podcast today. It’s been a pleasure talking to you. And thanks to our listeners for tuning in. If you liked this episode, please like, subscribe, rate, review, all of the above, on your favorite streaming platform. Until next time, this has been the IoT Chat.

The preceding transcript is provided to ensure accessibility and is intended to accurately capture an informal conversation. The transcript may contain improper uses of trademarked terms and as such should not be used for any other purposes. For more information, please see the Intel^® trademark information.

This transcript was edited by Erin Noble, copy editor.