California Issues Generative AI Tool Use Guidelines

California has introduced guidelines on the use of generative AI technologies requiring officials to conduct audits and risk assessments before adopting new systems.

The guidelines directly follow California Governor Gavin Newsom’s September 2023 executive order, which imposed rules on the use of generative AI in state government.

Among the new guidelines, state agencies must show when a company is offering to provide generative AI products or services. Any potential solution provider must show the generative AI aspects of what they are offering the state.

They also offer specific guidance on “incidental” and “intentional" purchases of generative AI technologies.

“A solicitation may receive bid responses that include generative AI, regardless of whether it is a non-IT goods and services, IT goods and services, or telecommunications purchase,” the guidelines read.

The rules make it tougher for providers to offer their services, as state agencies can only purchase AI systems from providers who have undergone a written solicitation process.

State agencies must also bring in a generative AI expert to help draft and check contracts for new systems.

Before they can go ahead with procurement, agency CIOs must also conduct risk assessments of generative AI technologies to ensure no system can harm citizens. Staff tasked with conducting risk assessments must use the California Generative Artificial Intelligence Risk Management Principles, which provide a framework for evaluating generative AI risks.

Related:California to Regulate State Use of Generative AI

Before adopting generative AI technologies California state agencies will be forced to conduct workforce training.

Executive-level staff will receive initial training followed by legal and privacy specialists. General staff will receive training later before generative AI tools are deployed. All employees will need to take information privacy and security training.

Additionally, state entities are mandated to submit an inventory of all generative AI uses to the California Department of Technology for review. 

Agencies have also been supplied with a toolkit agencies can use to help procure generative AI technologies. It is written in plain language, so non-technical staff can make sense of the rules and will be continuously updated as best practices evolve.