Alan Dix explores AI and Social Justice
Author of Introduction to Artificial Intelligence
AI Business is part of the Informa Tech Division of Informa PLC
This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 3099067.
The British government has published its contracts with businesses tasked with building a COVID-19 datastore for the National Health Service.
Microsoft, Google, Palantir, and Faculty were drafted to develop various cloud and artificial intelligence tools as part of the NHS' pandemic response strategy. The contracts were released hours before the start of the court proceedings launched by openDemocracy over the government’s controversial and secretive data arrangements.
More than 13,000 people signed a petition demanding a public release.
“It shouldn’t have taken a lawsuit threat to get us these documents, but we’re pleased to share them with the public now," Mary Fitzgerald, OpenDemocracy’s editor-in-chief, said.
The group learned that the original contracts allowed tech corporations to retain intellectual property rights, train AI models on citizen health data, and profit from the deal. The government now claims this has been amended.
“Government lawyers have now claimed that a subsequent, undisclosed amendment to the contract with Faculty has cured this problem, however they have not released the further contract. openDemocracy and [‘tech justice’ organization] Foxglove are demanding its immediate release. It is not clear if other contracts were altered as well.”
Microsoft's contract appears to primarily focus on cloud and online services, as well as compliance with the UK's Data Protection Act.
Google's contract notes that the deal with the NHS is free of charge, "purely as a service to the public." The company promises to provide technical, advisory, and other support to NHSx, the digital division of the NHS.
The document states Google will not have access to personal data, and does not aim to develop any intellectual property with this project.
But, notably, the contract states: "Google owns all rights, title, and interest in (a) Google's background IP, (b) all IP and know-how applicable to Google products and services, and (c) all IP arising in connection with the Support that has general application to Google's other customers, including derivative of and improvements to Google background IP."
Put simply, it appears that if Google learns anything from the project that improves its products (be it AI or Cloud), it will be able to keep those improvements, and commercialize them.
Google has a long history with the NHS - its AI subsidiary DeepMind previously teamed up with the national healthcare provider to research eye disease. This grew to include contracts with five NHS trusts, and data on more than a million patients. The contracts were hugely controversial when they were made public, and in 2017 the UK's Information Commission ruled that a study into detecting kidney injuries broke privacy laws. In 2019, DeepMind Health became a part of Google Health.
Then there’s Palantir. Founded by Peter Thiel and Alex Karp, the company is best known for taking the types of contracts that many in Silicon Valley would pass on: like vast AI-powered surveillance deals with the US military, Homeland Security, Immigration and Customs Enforcement, and the CIA.
"Our product is used on occasion to kill people," Karp told Axios last month, admitting that many thought of the company's ICE contract as unethical. "I had people protesting me, some of whom I think ask really legitimate questions. I have asked myself if I were younger at college: 'Would I be protesting me?'"
Palantir has been awarded multi-million dollar COVID-19 tracing contracts in the US. But in the UK, the arrangement is worth £1. This prompts “further speculation about how the firm was due to benefit from the deal,” OpenDemocracy noted.
The project involves a pilot trial of Palantir Foundry Services, with the company covering the costs of hosting on Amazon Web Services.
Other, more detailed, documents are referenced, but not shared. Still, the Palantir contact provides much more information than others: "The aim of the project is to create a data store which will be used to: 1. Track and predict the spread of COVID-19; 2. Model interventions including guidance for public & patients; 3. Optimise health & community resources."
Also mentioned in the contract are the types of personal data Palantir will have access to, which may include "name, personal email address, home address... gender, nationality, place of birth... employment details... working hours," as well as "any other personal data that may be useful."
In the contract, the company notes such data would be pseudonymised, a technique that replaces or removes information in a dataset that can be traced back to an individual.
Pseudonymising personal data can reduce the risks to the data subjects and help meet data protection obligations, according to the UK’s ICO. But the regulator notes that the technique “is effectively only a security measure. It does not change the status of the data as personal data."
The contract states the NHS will also provide Palantir with aggregated data, which it admits carries the "risk of re-identification in the absence of proper controls."
For a deeper understanding of what can be done with Palantir's technology without proper controls, we recommend Bloomberg's 2018 report on how a security expert hired by JPMorgan used the company’s software to conduct an extensive spying program on the bank's staffers and executives without their knowledge.
Elsewhere in the contract, Palantir stated: "For the avoidance of doubt, Palantir is acting as a technology provider and not a clinical decision maker. The Customer is responsible for making decisions informed by the use of Palantir’s software and services."
That brings us to Faculty - perhaps the most controversial of the companies on the list. This UK-based business has ties to Boris Johnson’s chief adviser, Dominic Cummings (who recently made headlines for breaking lockdown rules), and the wider Brexit movement.
Faculty, which received at least eight government contracts worth almost £1.6m, all in the space of 18 months, worked on the winning Vote Leave campaign. Former employee Ben Warner then followed Cummings into government.
His brother Marc Warner, Faculty CEO, was revealed to have attended the government’s Scientific Advisory Group for Emergencies (SAGE) meetings. The company is known to have at least two other pandemic contracts, and one of its shareholders is Cabinet Office minister Lord Agnew.
The contracts provide some insight into what Faculty is seeking to do with the NHS - at a price ceiling of £930,000 plus VAT.
The mission is to "design with the AI team what the NHS AI Lab will look like, with options on physical vs virtual settings and how it will sit alongside other government AI Labs," the document states.
"Development of change management frameworks and service redesign for adoption of AI - covering national, regional, and local levels and in line with the four priority areas of the AI in Health and Care Award."
Another task is listed as "modelling and simulation: using data from across the healthcare system to model scenarios to better understand the impact of the spread of COVID-19 on healthcare resources."
In a separate section, the document explains that the "the Faculty team will work with Palantir" on strategy and oversight of the overall program, as well as on the health system dashboard.
Data for another collaboration, the chest X-ray database, will also be pseudonymized, and stored in an Amazon S3 bucket. Faculty said it will provide its data science platform "at zero licensing cost to provide sandboxing infrastructure for the validation of AI models." Salaries of individual employees are redacted, but include senior AI roles, data scientists, and a senior economist.
Further details, including the outcome of Data Protection Impact Assessments, are yet to be published.
Author of Introduction to Artificial Intelligence