Which skills do you need in your team?

What are the costs for running the service?

How can you optimize them?

These are three key questions when setting-up and running an AI and analytics service.

As discussed in my last post, AI and analytics service organizations can provide two service types:

In the following article, we look at the needed capabilities to deliver such services, and at costs.

The three core capabilities for strategic advisory

When looking at the timeline of AI and analytics strategic advisory projects, they start with understanding the business question, translating the business question to an AI and analytics problem, getting the necessary data, creating (and potentially applying) the model on the data, and presenting the results.

So, while the AI and analytics knowhow of data scientists gives the team and service the name, this is just one capability (Figure 1). The often-underestimated capabilities are the analytics translator and data preparation and management. Typically, projects spend more time on these topics than on the actual data science tasks.

The capability data management and preparation covers the following tasks and topics:

The reason to distinguish between data preparation and management and data science is the different skillsets required. Any SQL developer knowing the company’s databases outperforms any machine learning genius for data management and preparation. However, creating machine learning models requires completely different knowledge on algorithms and methodologies.

Then, there is the capability of an analytics translator. His or her task is to understand the business question and formulate an AI and analytics task – and to present and discuss the results with the business. Again, this requires different skills than creating machine learning models: understanding commercial challenges, asking the right questions when talking with users and customers, and presenting results to the management. No software project sends their Java guru to the C-level to discuss the strategic goals for the software development project – and neither should an AI and analytics service owner send their AI experts.

Figure 1: Capabilities for Analytics and AI Service Teams

Strategic advisory: Understanding and influencing the service costs

Important costs categories are inevitable costs versus costs-by-design and fixed versus variable costs. In general, project costs are variable. If there is work, there are costs. If there is no work, there are no costs. The costs for a strategic advisory project can be influenced partially. All project phases are necessary: understanding the business questions, translating them into an AI and analytics problem etc. However, data preparation and model creation and optimization can be influenced. Do you consider all data sources, including the more complex ones requiring a higher effort? Do you stop after the first model that seems to work well – or do you go for a second, third, fourth? A second cost-driver is the interaction time with the customer. How much do you explain to the customer, how many meetings do you organize? How much time does the project team invest into slides or training?

A service organization has fixed costs, even if it runs mostly strategic advisory projects. The service organization has engineers and data scientists etc. They have to be paid whether or not they work on a project. If there is overcapacity, e.g., to ensure that there are always resources for urgent projects, these costs have to be covered as well. Furthermore, there are costs for a tool chain, for licenses, and for software maintenance. These are often fixed costs. Finally, there might be (variable) costs for external capacity for projects if there are not enough internal resources or if there is a know-how mismatch. Figure 2 summarizes the discussion.

Figure 2: Cost structure for analytics and ai strategic advisory

The six capabilities for ongoing operational insights services

An ongoing operational insights service requires an initial set-up phase before the actual service delivery and operations starts (Figure 3). During the set-up phase, one task is to create the machine learning model. This requires the already mentioned capabilities analytics translator, data science, and data preparation and management. Most of the time, there is also some integration with other systems and/or the model is deployed to a production environment. This is the task of the second subproject named integration engineering.

Integration engineering is usually a task for the future application management and operations engineers. They make installations and configure interfaces they take care of later. For example, they ensure that the analytics component such as RServer with all models and all its interfaces work. They solve issues, install patches, and coordinate releases with other applications and with the underlying infrastructure. Often, they rely on the IT department’s central monitoring solution to detect issues before they impact the users.

The support capability is needed for ongoing operational insights services. Support specialists interact with the users and the customers once the service is up and running. A list with results is not delivered on time, an interface breaks down, something should be changed – the support is the face to the customer. While there is often an overlap with the staff for application management and operations, these are separate capabilities. The service design is different, and so are the cost models.

During the operations phase, the model has to be updated from time to time as well. This is a data science job. However, the support or application management can take over running mature scripts, if they have clear metrics to check the model quality. Then, data scientists can focus on larger and methodological challenging tasks and new projects. Interaction with the business via analytics translators and adding new data sources are also tasks that are necessary during operations, just on a low scale compared o the initial project.

There is one capability that might or might not be needed. This is Report Provisioning, Presentation, and Visualization. Result are often lists, but more complex visualizations are needed as well. Also, when reports are updated on a daily or weekly base and the users have to get a copy, emailing Excel sheets is not a good solution. It can be work-intensive or not, but it is for sure an information security risk. Emails can be sent by mistake to the wrong persons. Thus, a platform for providing reports is important – and the more users, the more important is a proper design.

Figure 3: Ongoing Operational Insights - Understanding the Phases of a Service

Ongoing operational insights: Understanding and influencing costs

The main challenge when designing an ongoing operational insights service is to balance the need of having staff with the right expertise available with preventing that the costs explode. This requires in-depth understanding of how to design and manage a service, not necessarily AI and analytics know-how.

Basically, all capabilities are pivotal and have to be there. The service design decisions circle around how much to invest into which. The service organization can be one intern reading emails and solving technical issues on Monday morning, and Wednesday and Thursday afternoon – or there can be a 7/24 phone support hotline, three senior engineers on call, and a professional monitoring and alarming infrastructure. The costs are completely different. However, if the service quality is too low, other costs emerge suddenly. Thus, it is important to understand the following three important cost types (Figure 4): costs for running the service, negligence costs, and potential losses.

Costs for running the service is the money the organization (willingly) spends e.g. for the support and application management and operations staff. These are directly visible costs. This is a risk when there are cost-cutting initiatives. This looks perfect for quick savings, but saving too much causes unexpected, hidden costs: negligence costs.

Negligence costs are the result of handling customer requests and technical incidents inadequately, causing work disruption for the AI and analytics service team. Improper, unprofessional handling of adequate and even inadequate user and customer requests can result in escalations. Escalations are time consuming for the management, but for engineers as well. The latter have to collect data, search for old email communication, etc. This causes costs and impacts ongoing projects. If there is no adequate support organization, simple requests are routed to senior specialists and disturb their project work, e.g. on the next machine learning model.

Finally, the business impact or the potential losses reflect the monetary impact if the AI and analytics service is not working. If the AI and analytics component make upselling and cross-selling recommendations which account for 20% of the revenues of the web shop, these revenues are gone when the component is down or starts making bad suggestions. This is the key point why the organization invests in the support and application management and operations team. They want to reduce the risk of potential losses.

The two main cost blocks for running the services are, first, handling incoming technical issues and, second, user requests. Engineers and support specialists have to paid for being available and waiting for a problem to occur or handle incoming request (certainly, they can do engineering task in low periods). The needed staff depends on the following:

Business continuity management and monitoring can be additional cost blocks. Details depend on whether there is a solution and processes in place by the IT organization and whether they are charged internally to an AI and analytics team. Also, the service management might have to invest some effort there as well.

Then, there are costs and efforts that are a result of the overall set-up and processes. What is the quality of the implemented solution? The ready-to-use machine learning models are code-wise not complex, but ETL processes for preparing data are – and if applications with interfaces to the AI and analytics components fail and create issues on the interfaces this generates work for the AI and analytics team as well.

Figure 4: Understanding Costs for Continuous Data Science Services

Klaus Haller is a Senior IT Project Manager with in-depth business analysis, solution architecture, and consulting know-how. His experience covers Data Management, Analytics & AI, Information Security and Compliance, and Test Management. He enjoys applying his analytical skills and technical creativity to deliver solutions for complex projects with high levels of uncertainty. Typically, he manages projects consisting of 5-10 engineers.

Since 2005, Klaus works in IT consulting and for IT service providers, often (but not exclusively) in the financial industries in Switzerland.