Researchers Optimize AI Agent Workloads Innovatively

Researchers from MIT and the University of Washington have developed a new method that allows businesses to enhance the decision-making capabilities of AI agents, even when operating on limited hardware.

Agent-based AI systems are commonly deployed as customer service bots or for predicting inventory needs in supply chain management. However, such systems require real-time processing to achieve results, which can be expensive to run.

The researchers contend that instead of deploying the most powerful model or building a complex underlying architecture, improving the success of the AI agent involves understanding available computing resources.

Traditionally, agent-based systems would compensate for limited compute by randomly introducing errors into otherwise optimal responses.

In a newly published paper, the researchers propose that setting a “budget” for an AI system allows it to better produce outcomes.

Rather than forcing a model to produce a result within a set time, the researchers propose setting what is effectively a cap on the computing power necessary for each task the AI system performs.

The concept of “latent inference budgets” sets a level of compute that the AI agent should use when processing responses which makes the system more efficient without sacrificing accuracy.

Related:Future AI Could Share Knowledge Like the Borg in ‘Star Trek’

The approach enables an AI model to simulate decision-making processes, even when it cannot process all available information instantaneously.

Such a budget is designed to help the AI decide how deeply it should think about a problem before generating a response. If the budget is low, the AI may use faster methods to generate a response, which may not be as accurate but would be sufficient for less complex problems. Setting a higher budget would use up more computing power but provide a more accurate answer.

A business user could set a budget for his or her AI agent based on the complexity of tasks it needs to handle. For example, a customer support system could be allocated a lower budget to ensure faster responses.

While testing, the researchers applied the budgeting idea to tasks including maze navigation and playing chess.

They found that systems with a budget in place could make more accurate predictions and offer improved decision-making.

“In three domains, maze navigation, pragmatic language understanding and playing chess, we demonstrated that it can outperform classical models of bounded rationality while imputing meaningful measures of human skill and task difficulty,” the paper read.

Related:OpenAI is Developing AI Agents