Exclusive: Schneider Electric Chief AI Officer on Using Custom ChatGPT

Schneider Electric is leveraging Microsoft’s Azure OpenAI platform to develop chatbot solutions to improve worker productivity and enhance interactions with customers.

Among the AI tools being used is Resource Advisor Client, a copilot for data analysis and decision support, Jo-Chat GPT, an internal conversational assistant, and Knowledge Bot, a chatbot assisting customer care representatives.

There is also Finance Advisor, a conversational assistant to help financial analysts in accounting, and Conversational Search to help customers search for the products in natural conversation style.

Schneider Electric is also set to integrate GitHub Copilot, the code generation tool, in a bid to improve creation processes and operations.

The Azure OpenAI offering is designed to allow companies to build their own secure versions of ChatGPT using proprietary data.

In an exclusive interview, Philippe Rambach, chief artificial intelligence officer at Schneider Electric explained how the French energy solutions giant built its new AI tools and offered advice on how to approach generative AI projects at scale.

What follows is an edited transcript of that conversation.

AI Business: What is Schneider Electric’s thinking around generative AI and what made you comfortable in handling its risks, like hallucinations, biases, etc. when other companies are more hesitant?

Related:Schneider Electric CDO: Merits of an AI Hub-and-Spoke Structure

Philippe Rambach: This is why we've developed our own ChatGPT. We didn't want to forbid people to use ChatGPT because it brings lots of value. We're a tech company. We want to use tech. But on the other side, we don't want our data to leak. We have created our own version called Jo-Chat GPT using a private, OpenAI on Azure version. We still have the risk of hallucination, but we don't have any risk of data leakage.

We trained our staff on what are the risks of using it including that you cannot use the output without checking it first. One example is Knowledge Port, where we provide our customer care centers with tools to extract information from our user manuals and FAQs to answer questions.

To avoid data leakage, … we only gave it information coming from our corpus of knowledge. We use OpenAI for the capacity of embedding vectorization - we don't ask ChatGPT. We've also developed a tool that links to where the information is coming from. If the customer care center employee has a doubt, they can click and look at the FAQ and check for themselves. The rate of hallucination is never zero, but it's much lower because we only take information from our corpus of data and we give the link to where it came from.

Related:Microsoft Offers Private, Customizable ChatGPT to Enterprises

What made you choose Azure?

We don't work only with Azure. We work with other LLMs and we investigated. But I have to admit that being an industrial company in need of security that requires industrial versions, we decided to not accumulate too many unknowns and difficulties to start and we went for the safest, shortest, easiest path. Microsoft is a long-term partner of Schneider beyond AI so that was another good reason to work with them on this. But that doesn't mean that we will forever only use Microsoft and OpenAI. To start our gen AI journey, it was safer, faster, easier and more industrialized than going to a totally open source LLM from the market. The reason was really efficiency and speed.

Any other systems you’re experimenting with?

We can't mention everything, but we have worked with Google Bard. We will look at all the names that you know, and we investigate and evaluate all of them. For example, it's not directly linked with the LLM itself, but for all our knowledge application management we use quite a lot of LangChain. We usually try to use the best of breed.

What are the primary use cases of these new systems you’re announcing?

If I talk about generative AI there are two different things that we look at. One is what's available off the shelf, which is generic enough that we can just buy it. An obvious example of that is generative AI for code generation. We will not develop our own solution; we're going to buy one from the market. You know the names, there are not that many. but we are still investigating a bit.

Something pretty new with generative AI means that with other techniques of AI, sometimes off-the-shelf solutions can work but not very often. Quite often, you need to train it on your data and make it more specific. There are applications of generative AI like software code generation that we believe you can buy off the shelf and use out of the box.

The first thing we did was to ask, ‘What do we need to develop in-house and what should we just buy?’ Code generation we buy. Probably content generation, marketing generation, image generation: I'm not sure we're going to need to develop. What we believe we're going to need to develop internally at least today, is everything connected with our knowledge management, because they are proprietary: our data, our knowledge, and of course, everything that is enriching our offers to our customers, making offers easier to use, like the Copilot for Resource Advisor software, or what we have just shown today.

To summarize again: First, what can we buy off-the-shelf - there is more that can be bought off-the-shelf with generative AI than traditional AI. And second: What we think we need to develop in-house. And that can move fast, by the way, some stuff we need to do now. So maybe tomorrow we can buy but today we believe that the knowledge management team has to do quite a lot of internal stuff.

In terms of approaching these projects, were solutions you could access the first consideration, or was it more use case-focused?

That has not changed in generative AI. I see many customers, partners who made the mistake of being impressed that they fall into the trap of starting from the technical: 'Oh, I've got this generative AI. What can I do with generative AI?'

We stick with 'What are our business problems? What do we need to solve? And yes, with generative AI, there are things that we couldn't solve in the past that we can solve now - having people coding faster, generating marketing content faster and having better access to our knowledge for everybody in Schneider. But the starting point is always the business values of a business case when it is deployed at scale. Otherwise, my belief is with these techniques, you can do hundreds of thousands of super exciting stuff, but we're not paid for that, we are here to bring value.

When did Schneider begin its AI journey?

We started two years ago. And we started to deliver a strong value with use cases a year after that. I would say more or less 12 months between when we started and when we started to deliver real value and then it continues to accelerate. Now we are reaching cruising speed where we deliver at scale use cases.

Two years seems like a long time – is taking the time on this is a good approach to adoption or has it been a challenge?

I would challenge the idea of two years being long. We have now in all our customer care centers in North America a fully deployed solution where our customer care people use a generative AI tool to access up to thousands of pages of FAQs that we have and access the knowledge it's now in production for a month. Between July and when people started to be excited, we did that in three months and we did that at scale. I would not say it took us two years to get it and do that and so on. If I take a specific use case like this one, it's four, five months between identification and being at scale.

Also, we started developing value after one year, starting from AI scientists working centrally in research to a fully at-scale team with processes, with a tech platform able to deliver use cases at scale that are used by our employees or customers for real, bringing a lot of value.

For a large company, (it’s more than just doing) a nice pilot that we showcase at an event but rather deploy a solution that I use at scale, for real, bringing value. I don't think competition goes much faster, at least in our industry. Of course, some startups can do stuff fast. But here when we speak about scale, it means that our business in North America with (€9 billion annual) revenue, the customer care centers now use a gen AI solution at scale to get the knowledge from our FAQs and user manuals.

The big risk in AI is (just) to do pilots and demonstrations. … If you want people to do it at scale, take this customer care center example, you need to have it embedded in the tool that people use in production. You don't just need to create a webpage that people access to get the model. You need to have it in the tool. You need to have it validated. You need to have your users trained on it.

In terms of using your proprietary data, how did you break down data silos and aggregate it into a training dataset? What challenges did you encounter?

At the core of the model we have put in place, we started from the business case, the business value, something that we see deployed at scale and not an idea for a pilot that come up with later. We think, ‘what is the at-scale value in that case? How do we have all our customer care centers having good access to the knowledge, and having good support in answering customers?’ Then when identified with a business value, we'll create a team mixing people from the AI hub and people responsible for owning the business case. In this case, they were people from sales in charge of customer support. And that team delivers the value, delivers a project at scale and completely within that. That squad doesn’t stop when we’ve made a pilot, a demonstration and then IT will implement it.

The solution is deployed for use or training. Or if it’s designed for customers, the sales force is trained and we sell it and so that’s how we … break the silos as you say. What I've seen too often is a team doing a pilot and then someone is tasked with industrializing and someone is supposed to scale. We put that into one team, which starts from the idea.

What considerations went into safety when implementing these new systems?

We treat AI products as a product, which means that we will apply them with all the Schneider standard procedures for cybersecurity, for data privacy plus our responsible AI framework to also address the specific cities of AI.

What’s the cost considerations for these new AI systems?

That's where for me in the long term, we could have a discussion on what is the right LLM. For example, for our customer care center, we decided that GPT 3.5 was good enough. There was no reason to support the cost of GPT-4. The reason is not technical; we stayed with GPT 3.5 for cost reasons, and one day we may decide to move to another LLM for cost reasons.

The market is very early, competition still has to come. When we speak about running costs, we look at two things, the running costs in terms of money, but also in terms of impact of carbon emissions and CO₂. The two reasons not to use GPT-4 was, one, it's more expensive. Second, it's also much more expensive in terms of carbon impact, carbon emission and energy consumption. We try to be as frugal as possible in the sense that we use the minimum technique to solve the problem - and we don't do technique for technique’s sake.

You wrote a thought leadership article for AI Business last year on AI at scale and sustainability. What energy considerations did your team discuss in terms of these new deployments?

For everything we do in terms of energy optimization for our customers, in most cases you don't need generative AI. In most cases, optimizing, forecasting, anomaly detection techniques are good enough. They use much less energy, they cost much less, and in many cases, they probably do a much better job than generative AI. We're also very careful[BW1]  not to be blinded by the beauty of what is new.

Yes, generative AI is a total game changer for everything around NLP, knowledge management, content generation. But for energy optimization for our customers, in terms of forecasting the need, optimizing the loads, detecting anomalies and so on, we stay with conditional neural nets. They use much less energy, costs less money, and does probably a better job.

What's next? What can we expect from Schneider Electric in terms of AI?

More Copilot offers, more knowledge management, more support of employees using generative AI. More support to our customers are probably the things that you will see coming in the coming weeks and months.

Finally, what lessons have you learned from these deployments that you can apply to future projects?

Still very early, but I would say that the biggest learning for me is to forget the past. Whatever millions you have put into technology, if a new one is coming which is much better, stop (using it) and stop it fast. For example, we decided quickly in June to stop investing in our (existing) chatbots … and decided to move all our chatbots to generative AI.

It was a bit hard because you put quite a lot of money in building databases, Q&As, all of that. But the question is never how much you have spent, but how much more you need to spend to reach what you want to reach. … This will happen again, technology is moving fast. After deciding to build something by ourselves in three months (for example), we'll be just able to get it from the market. But we have to be courageous enough to say, “OK, we stopped what we were developing and we buy or we use what is open source.’ For me, the big lesson is ‘be ready to stop what you were developing if the new technology brings enough (capabilities). At the same time, don't go for new technologies if traditional neural networks are better and cheaper (than, for example), gen AI. But in some cases be ready to stop what you were doing, even if you already invested quite a lot of effort and money.