In total, the department removed nine items from its website — several of which appeared to be identical use cases listed under two different agencies — and changed the bureau for a handful of the remaining items. The State Department didn’t provide a response to FedScoop’s requests for comment on why those uses were removed or changed.

The deletions came roughly a week after the Office of Management and Budget released draft guidance for 2024 inventories that says, among many other requirements, that agencies “must not remove retired or decommissioned use cases that were included in prior inventories, but instead mark them as no longer in use.” OMB has previously stated that agencies “are responsible for maintaining the accuracy of their inventories.”

AI use case inventories — which are public, annual disclosures first required by a Trump-era executive order — have so far lacked consistency. Other agencies have also made changes to their inventories outside the annual schedule, including the Department of Transportation and the Department of Homeland Security. OMB’s recent draft guidance and memo on AI governance seek to enhance and expand what is reported in those disclosures.

OMB declined to comment on the removals or whether it’s given agencies guidance on deleting items in their current inventories.

Notably, the department removed a use case titled “forecasting,” which was a pilot using statistical models to forecast outcomes that the agency told FedScoop last year it had shuttered. The description for the use case stated that it had been “applied to COVID cases as well as violent events in relation to tweets.” 

Several of the other deleted State Department uses were related to media and digital content. 

For example, the agency removed the disclosure of a “Facebook Ad Test Optimization System” that it said was used to collect media clips from around the world, a “Global Audience Segmentation Framework” it reported using to analyze “media clips reports” from embassy public affairs sections, and a “Machine-Learning Assisted Measurement and Evaluation of Public Outreach” that it said was used for “collecting, analyzing, and summarizing the global digital content footprint of the Department.” 

State also removed its disclosure of “Behavioral Analytics for Online Surveys Test (Makor Analytics),” which the agency said was a pilot that “aims to provide additional information beyond self-reported data that reflects sentiment analysis in the country of interest.” That use case had been listed under the Bureau of Information Resource Management and the Under Secretary for Public Diplomacy and Public Affairs. Both references were removed.

Two of the removed items had been listed under two agencies but had only one disclosure removed: an AI tool for “identifying similar terms and phrases based off a root word” and a use for “optical character recognition and natural language processing on Department cables.”

Another removed use was for a “Verified Imagery Pilot Project” by the Bureau of Conflict and Stabilization Operations. That pilot tested “how the use of a technology service, Sealr, could verify the delivery of foreign assistance to conflict-affected areas where neither” the department nor its “implementing partner could go.”

While the use case inventory was trimmed down, the department also appears to be adding uses of AI to its operations. State Chief Information Officer Kelly Fletcher recently announced that the department was launching an internal AI chatbot to help with things like translation after staff requested such a tool. 

Rebecca Heilweil and Caroline Nihill contributed to this report.

The post State Department trims several uses from public AI inventory appeared first on FedScoop.

In an email to FedScoop, the NRC spokesperson said Nelson will be leaving the agency effective Jan. 26. Taking his place as acting chief AI officer and CIO is Scott Flanders, the commission’s current deputy CIO. 

Nelson was appointed as the regulatory agency’s CIO in 2016, leaving his previous position as CIO and director of the Office of Enterprise Information for the Centers for Medicare and Medicaid Services. 

Nelson was recently appointed as the NRC’s CAIO, in light of a long-awaited executive order on AI from President Joe Biden. While the order did not include the NRC as an agency that will be required to eventually name a CAIO, the commission told FedScoop previously that it was “assessing whether and how it applies.”

Additionally, the NRC spokesperson confirmed that Victor Hall, the deputy director of the Division of Systems Analysis in the Office of Nuclear Regulatory Research, serves as the responsible AI official under Executive Order 13960, issued by the Trump administration. The NRC was also exempted from that requirement as an independent regulatory agency.

The post Nuclear Regulatory Commission CIO David Nelson set to retire appeared first on FedScoop.

The Government Accountability Office report released Tuesday focuses on how 23 agencies — the civilian agencies under the Chief Financial Officers Act — have progressed with a requirement that they must annually inventory their current and planned deployments of AI and disclose non-sensitive, non-classified uses publicly. That obligation stems from a Trump-era executive order (EO 13960) focused on AI. 

The report found that those agencies’ inventories did not always follow requirements and sometimes contained “data gaps and inaccuracies.” Of the 20 agencies that submitted inventories to the Office of Management and Budget, just five “provided comprehensive information” for each use case, GAO found.

“Without accurate inventories, the government’s management of its use of AI will be hindered by incomplete and inaccurate data,” the report stated.

The report also outlined the extent to which CFO Act agencies are using artificial intelligence. While more than 700 use cases were publicly disclosed in a consolidated spreadsheet uploaded to the AI.gov website in October, the GAO reports that agencies publicly disclosed about 888 use cases and there are about 1,200 reported by those agencies in total. 

NASA had the most uses of any agency, followed by the Department of Commerce and the Department of Energy, GAO found, which differs from the totals in the existing consolidated inventory. NASA, for example, which has 375 total public uses of AI according to GAO, has just 33 in the consolidated list online.

About half of government AI use cases are focused on science, while the rest span areas including internal management, agriculture, transportation, public services and law enforcement, the GAO said. 

A major Stanford report, along with a series of subsequent investigative pieces produced by FedScoop, had previously highlighted challenges, including inaccuracies and discrepancies, within these inventories. The inventories were also discussed in a House Oversight subcommittee hearing earlier this fall. 

The disclosures could become more important. In a November interview with FedScoop following President Joe Biden’s recent AI executive order, Conrad Stosz, director of AI in the White House Office of the Federal Chief Information Officer, said the disclosures are expected to be more central to government and public insight into AI uses going forward. 

In addition to not following requirements, the GAO also found that several inventories included duplicative use cases, and two included references to technology that were not, in fact, AI. Nine agencies also included references to AI used within research and development, which were supposed to be excluded.

Reasons for the issues, agencies told the watchdog, included staff errors and different interpretations of the CIO Council’s guidance for the inventories. 

Department of Veterans Affairs officials, for example, told GAO that they used different responses for life cycle stage than the ones in the template provided by the CIO Council “because the selections did not accurately reflect the life cycle stage for their AI use cases and there was no option for ‘other,’” according to the report.

FedScoop similarly reported that DOE said it was able to produce a more accurate and larger use case inventory in the second year of reporting, following “enhanced” guidance.

The GAO report also found that agencies have taken only initial steps to implement existing AI requirements in executive orders and statutes. OMB and the Office of Personnel Management, for example, didn’t “fully implement” requirements for AI acquisition and workforce assessments, respectively.

Ultimately, the GAO made 35 recommendations to 19 agencies. That included recommendations that 15 agencies update their inventory to include information required, and that OMB, OPM and the Office of Science and Technology Policy implement the outstanding AI requirements.

Of the 19 agencies, GAO said 10 agreed with their recommendations, three partially agreed, four didn’t agree or disagree and one ultimately didn’t agree with a recommendation.

The post Most agency AI inventories are ‘not fully comprehensive and accurate,’ GAO reports appeared first on FedScoop.

The report will focus on the Chief Financial Officer Act agencies — not including the Defense Department — as well as the Office of Management and Budget and the Office of Science and Technology Policy, and is expected to be published next week, Kevin Walsh, a director with the GAO Information Technology and Cybersecurity team conducting the report, said in an email to FedScoop. 

Speaking on a panel at Informatica’s Data in Action Summit on Wednesday, Stephen Sanford, the GAO’s managing director of strategic planning and external liaison, described the forthcoming report as a look at how the CFO Act agencies are doing with the AI inventory process and where they are with requirements in various AI executive orders and legislation.

Federal agencies’ annual AI use case inventories, which were initially required under a Trump-era executive order, have so far lacked consistency and received criticism from academics and advocates as a result. A major 2022 report by several Stanford researchers analyzed progress on the existing AI requirements under statute and existing executive orders, detailing compliance issues with the inventories in the first year they were required. FedScoop has continued to report on issues with those public postings in recent months.

Following President Joe Biden’s recent AI executive order, the White House said the inventories are intended to be a more expansive resource for the government and the public. A list published on AI.gov ahead of that executive order consolidated the public inventories into one file, totaling more than 700 uses across the government. 

The coming report was initiated by GAO under the Comptroller General’s authority, Walsh said, as opposed to a congressional request. The Comptroller General’s authority is typically used for work on emerging issues, broad interest areas for Congress, and to respond to events of national or international significance, Walsh explained.

While DOD won’t be included in the new report, GAO has previously looked at the department’s AI-related work. Earlier this year, the GAO published a report recommending that DOD establish department-wide guidance for AI acquisition, and previously published a report recommending improvements to its own AI inventory process.

In addition to the use case report, Sanford said the GAO is also preparing a report on how the Department of Homeland Security and some of its components are doing with the implementation of the watchdog’s AI accountability framework. He said they expect to put it out “early next year.”

“We have a lot in the pipeline,” Sanford said. “The goal here is, I think, as a federal community, to try to learn from all this work. And these are, I think, some of the first wide-scoped evaluative jobs that we’re … doing that are going to be coming out.”

The post GAO preparing report on agency artificial intelligence use case inventories appeared first on FedScoop.

Lynne Parker, a former AI official in the Trump and Biden White Houses, told FedScoop in a recent interview that OMB could enhance its parameters for agency AI inventories by focusing on what those disclosures were intended to achieve: Improving public transparency, helping agencies see what others in the government are using, and informing policy guidance on responsible use of AI.

For example, Parker suggested structuring a call to agencies “to put use cases into various different kinds of buckets or to ask key questions that are important for informing the development of that policy guidance.”

“By thinking through those ultimate purposes, then you can structure the information that you’re requesting from the agencies such that it can actually serve those purposes,” said Parker, who is currently associate vice chancellor and director of the AI Tennessee Initiative at the University of Tennessee Knoxville.

Her comments come as OMB prepares to release guidance on federal government use of AI as part of its efforts to regulate and harness the capabilities of the budding technology. That guidance was expected to be released for public comment over the summer, according to a May 2023 announcement, but has yet to be published.

AI inventories are required to be published annually and publicly under a Trump-era order (EO 13960), but those disclosures so far have varied widely. Stanford’s RegLab found widespread lagging compliance in the first year of agencies’ use case inventories, and FedScoop reporting has found inconsistencies in reporting have persisted. 

Those inconsistencies include variations in format, timeline, and level of detail, along with notable use case omissions. These issues have caught the attention of some members of Congress and were recently discussed in a House Oversight subcommittee hearing on federal agency use of AI.

Parker, who led the interagency committee that drafted the order, said the “spotty” and “inconsistent” nature of the disclosures is due in part to the timing of the executive order — which came in the late days of the Trump administration — as well as new priorities. 

“With any administration, you come in with a lot of different priorities,” Parker said, adding that the Biden administration’s priorities required a lot of work by OMB and agencies in other areas. “They simply, I don’t think, had the cycles, nor were they being messaged by the administration that things like this executive order were a priority.”

The order, among other stipulations, created a framework for how the U.S. government should approach the nascent technology and established requirements for additional guidance and transparent disclosure of AI being used in the government through use case inventories. The inventories are required of all agencies except for the Defense Department, intelligence community agencies, and independent regulatory agencies. 

Although the Federal CIO Council, which is led by OMB officials, did come up with guidance for the first year, Parker said, the effort “was, frankly, trying to check a box” and focused more on trying to have something reported, “as opposed to thinking through how the information that’s reported could be useful.” 

The CIO Council’s most recent version of the guidance was more detailed than it was in the first year, which has helped some agencies. The Department of Energy, for example, said clarification from OMB on what constitutes a use case and what is considered research and development — which doesn’t have to be reported — allowed the agency to more comprehensively compile its inventory.

OMB didn’t respond to a FedScoop request for comment for this story. A White House official, however, recently told lawmakers on a House Oversight subcommittee that the inventories are important.

“The initiative to start cataloging those use cases was an important one and it’s very much work in progress,” said Arati Prabhakar, director of the White House Office of Science and Technology Policy. With respect to coming OMB guidance, she also said: “The Office of Management and Budget is working in a very focused manner on what they clearly understand is a priority.”

Parker pointed to other actions the order mandated that don’t appear to have been completed by OMB and Office of Personnel Management, too. The former was supposed to have issued a roadmap for their planned policy guidance on how to implement the principles the order outlined, and the latter was required to inventory rotational programs and make recommendations on how to increase the number of government employees with AI expertise.

OMB also didn’t respond to a request for comment on whether its forthcoming AI guidance will address that requirement from the order. OPM told FedScoop in July it could start compiling its report after a data call was complete and pointed to a memo on AI competencies for federal workers. It didn’t respond to a request for comment on the status of that work.

The coming OMB guidance, Parker said, is paramount – even more than inventories of use cases. “I think having consistent policies for how government shall ensure that those principles from the executive order have held – I think that is kind of more fundamental,” Parker said.

The post Ex-White House official says improved artificial intelligence inventories could help OMB guidance appeared first on FedScoop.

According to a posting on the agency’s public AI inventory, PHMSA is weighing an “artificial intelligence support for rulemaking use case.” The project, according to the posting, involves using ChatGPT to support the rulemaking “processes to provide significant efficiencies, reduction of effort, or the ability to scale efforts for unusual levels of public scrutiny or interest.” The agency told FedScoop that, right now, it has no official plans to implement such technology.

Interest from PHMSA, which creates regulations for the movement of potentially dangerous materials, comes as other agencies, including NASA and the Defense Department, begin considering the role of generative AI tools in their work.

Still, PHMSA’s concept for a technology pilot that would use ChatGPT to analyze comments submitted to the agency about regulations it’s considering raises concerns about what role, if any, the technology should play in the regulatory process, according to an expert on AI and civil liberties.

“The idea that agencies will use a tool notorious for factual inaccuracies for development of rules that forbid arbitrary and capricious rule-making processes is concerning,” Ben Winters, an attorney and the leader of the AI and Human Rights project at the Electronic Privacy Information Center, said in an email to FedScoop. “Especially, the PHMSA, whose rules often concern potentially life-altering exposure to hazardous materials.”

The Transportation Department’s AI inventory states that the OpenAI chatbot would be used to conduct sentiment analysis on comments sent to the agency about proposed rules. The tool could be used for analyzing the “relevance” of the comments, providing a “synopsis” for comments, “cataloging of comments,” and identifying duplicates.

When asked about the use case, PHMSA emphasized that the project is still in a very early stage.

“PHMSA, like many other federal agencies, is exploring the responsible and ethical use of AI through limited pilots and demonstration projects,” the agency told FedScoop in a statement. “These pilots and projects are designed to ensure alignment with recent guidance from the Administration on the appropriate use of AI in the federal government.”

The agency continued: “At this time, PHMSA is not using, and does not plan on using any generative AI tools or commercial software for generative AI like OpenAI to influence the rulemaking process. PHMSA is working with our stakeholders to assess both short term and long term risks from generative AI.”

In the agency’s AI inventory, which was last updated in July, the project is described as “a pilot initiative” that’s “planned” and “not in production.”

Winters, from EPIC, questioned whether ChatGPT is an appropriate technology for the rulemaking process. He argued that relevance analysis could ultimately result in an agency missing a novel point they hadn’t considered before, and added that sentiment analysis isn’t a “relevant consideration” of the Administration Procedure Act’s rulemaking process.

“[S]ummaries by ChatGPT are prone to factual inaccuracies and a limited and outdated corpus of information,” he said. “Most of these functions could not be reliably achieved by ChatGPT.”

OpenAI did not respond to a request for comment by publication time.

There are other instances where the DOT’s AI activities, at least as described on the agency’s official AI inventory, have raised questions. Earlier this year, the Department removed a reference to the Federal Aviation Administration’s Air Traffic Office using ChatGPT for code-writing assistance in response to FedScoop questions.

Stanford researchers highlighted major issues in the AI inventory compliance process at the end of last year. FedScoop has reported on ongoing issues related to these inventories and the AI use cases they’ve revealed.

Madison Alder contributed reporting. 

The post Pipeline safety agency’s proposed pilot for ChatGPT in rulemaking raises questions appeared first on FedScoop.

The DOE’s 2023 AI use case inventory, which is required under a 2020 executive order, includes roughly 180 applications of the technology in various stages of deployment throughout the agency. The majority of those uses began development or acquisition between 2017 and 2023. Roughly 130 are being operated by a single subsection of the agency, the National Energy Technology Laboratory, which is focused on environmentally sustainable energy sources. 

That new inventory — which appears to include very few, if any, of the uses the agency reported previously — is the result of updated guidance provided by the Office of Management and Budget, according to a DOE spokesperson.

“One of the primary challenges faced in the earlier inventory was a lack of clear guidance on what precisely constitutes a use case and what falls under the domain of research and development (R&D),” the spokesperson said in an emailed statement. “In response, the Office of Management and Budget (OMB) provided enhanced guidance that clarified these ambiguities, enabling a comprehensive capture of relevant AI projects within the agency.”

As a result, many of DOE’s uses of the emerging technology are being reported for the first time this year.

AI use case inventories are required of many federal agencies — except the Department of Defense, those in the intelligence community, and independent regulatory agencies — under a Trump-era executive order (EO 13960). Those inventories must be posted publicly and annually, though how agencies have responded to the order and guidance from the Federal Chief Information Officers Council — which is led by OMB officials — has varied in the first two years. 

Researchers at Stanford’s RegLab reported widespread lagging compliance in the first year of agencies’ use case inventories in a December 2022 report about the country’s AI strategy. Meanwhile, recent FedScoop reviews of agency use case inventories found inconsistencies in reporting have persisted. In response to FedScoop reporting, the National Archives and Records Administration has publicly shared its AI use cases and the Department of Transportation removed a reference to the Federal Aviation Administration using ChatGPT. 

“Agencies are responsible for maintaining the accuracy of their inventories. Additionally, per the instructions for the 2023 use case inventory, agencies are instructed to exclude AI research and development activities, including any AI use case that is unlikely to be incorporated into agency operations,” an OMB spokesperson recently told FedScoop. “We will continue to update the required guidance, as deemed necessary, to mature this effort. We have no further comment at this time.”

Fluctuating applications

It appears the majority, if not all, of the 45 use cases from the DOE’s previous inventory aren’t included in the new one. The bulk of uses reported in DOE’s previous inventory were operated by the Idaho National Laboratory, but the lab has only two use cases reported in its 2023 list. While both of those uses share similarities with applications reported last year, it’s not clear they’re the same. Similarly, the Office of Electricity had 10 uses last year, but none were reported for 2023. 

When asked about those changes, a DOE spokesperson said projects might not have been included in the 2023 inventory if they weren’t active during the reporting period.  Projects can have periods of inactivity caused by things like reassessing objectives, budgetary constraints, and changing priorities, the spokesperson added. 

They also said researchers may have shifted focus. “It is not uncommon for researchers to transition from one project to another based on emerging needs, technological breakthroughs, or evolving strategic directions,” the spokesperson said.

The inventory also contains at least two use cases that appear to be listed more than once. The DOE didn’t respond to an inquiry about those instances before publication.

Most uses reported in DOE’s inventory include a date only for when they were developed or acquired, which potentially suggests that most uses are still in the development phase. Nine use cases had implementation dates reported, however, including one for groundwater modeling that’s been in use since 2003. 

Among newly disclosed AI use cases: The creation of a resiliency framework for power grids that uses machine learning to protect against cyberattacks; AI-based hazard detection for coal ash and tailings storage facilities; and a machine-learning-driven process to predict locations of undocumented orphaned wells. All three began development or acquisition in 2022 and are being operated by the National Energy Technology Laboratory.

The post Increase in Energy’s reported AI uses reflects ‘enhanced’ guidance from White House appeared first on FedScoop.

In its 2023 AI use case inventory, the agency charged with managing U.S. government documents disclosed it wants to use an AI-based system to autofill metadata for its archival documents. Similar to some other agencies, the National Archives also disclosed its interest in using the technology to help respond to FOIA requests.

While NARA shared these planned applications, it did not include any current, operational use cases of AI.

The list of AI use cases is required of most federal agencies under a 2020 executive order (EO 13960). Those inventories must be posted publicly and annually.

The agency’s public release of its AI inventory comes after FedScoop reported that the National Archives had published its list only on MAX.gov, a platform for sharing information within the government. The Office of Management and Budget later re-emphasized that agencies are required to release a list on their agency website, in addition to the MAX portal.

“The National Archives is excited about the use of AI/ML/RPA and how we can utilize these technologies to help with natural language processing, search, and process automation,” said NARA Chief Information Officer Sheena Burrell in a previous statement to FedScoop. 

Burrell also said the agency was in the process of developing a governance life cycle for AI “along with the evaluation criterias to assess our AI solutions for compliances in accordance with the Executive Order.”

While the agency provided most details required under the Federal CIO Council’s 2023 guidance for the inventories — and additional optional information — it appears to follow a format consistent with the guidance for the previous year’s inventories. As a result, it doesn’t include whether the use is contracted or consistent with the executive order. It also doesn’t include columns for dates that note when stages in a use case’s life cycle take place.

Researchers at Stanford’s RegLab reported widespread lagging compliance in the first year of agencies’ use case inventories in a December 2022 report about the country’s AI strategy. Recent FedScoop reviews of agency use case inventories found inconsistencies in reporting have persisted.

The post National Archives discloses planned AI uses for record management appeared first on FedScoop.