The government has been asked to understand AI use on an individual public servant basis to accurately determine how the technology is being used.
The joint committee of public accounts and audit yesterday handed down its findings on public sector AI use. [pdf]
The committee ultimately calls for sweeping changes – including legislation governing how the public sector can use AI, mandatory rules and requirements, and standardised definitions and training.
A working group would be needed to be assembled to coordinate this effort, with some urgency.
“The committee’s very grave concern that if the Australian government does not implement effective and coordinated governance frameworks for AI systems now, this technology will outpace its ability to do so in the very near future,” committee chair Linda Burney wrote.
However, the first step is to better understand how AI is used across the Australian public service.
The committee said the annual APS employee census could be used to collect this data; last year, it had over 140,000 responses.
The committee wants to know “the specific types and sources of the technology being used, from automated decision-making through to generative AI; activities and tasks for which these technologies are being utilised and the impact of this on specific decisions and actions; and the levels and types of training that had been provided for these systems and the level of confidence of the respondent to effectively use them.”
It also wants to understand how the outputs of these technologies are managed, and “the level of understanding of the risks associated with AI-generated decision-making, including potential biases.”
That detailed dataset could flush out ‘shadow’ or undocumented ways AI is being used, as well as inform the future direction of AI use in government.
At present, AI use is being reported at an agency level, most recently via published transparency statements. However, these have been fairly vague so far, and do not go to specific uses of the technology.
Even without that granular data, the committee said there are some areas where prohibition of AI should be considered.
It warned against AI for decision-making that “directly impacts individuals’ lives”, such as “welfare benefits, criminal justice, or immigration”, noting the potential for algorithmic bias, false positives and “decisions that do not align with administrative law requirements”.
“AI systems are currently not accountable to the Australian people and may have algorithmic bias which may rely on faulty data, contribute to human rights violations, or erroneously flag individuals,” it noted.
The committee was also against AI being “used for surveillance purposes in the public service or elsewhere”.
This was not specifically defined, but could cast a wide net, from the use of the technology in productivity software, to its use in conjunction with CCTV.
