Artificial intelligence is often discussed with a simple safeguard in mind: Keep a human in the loop. But what happens when AI is already shaping the information, options and recommendations that inform a decision?
That question is part of new research by Lorien Jordan, an assistant professor in the USF College of Education. In a , Jordan examines recent disputes involving major AI companies and the U.S. Department of Defense, challenging the idea that AI ethics can be reduced to choosing one company or platform over another.
Lorien S. Jordan
The debate between Anthropic and the Defense Department over how its AI could be used illustrates the question: Even when a human remains responsible for the final decision, what happens when AI has already shaped the information behind it? Jordan's research looks at the broader systems that connect AI technologies to universities, publishers and researchers.
鈥淥ne of the things that surprised me and that I will continue to research is how we always say we need a human in the loop when we use AI, but as we鈥檙e scaling decisions so quickly and so massively, what does it mean to be responsible for the decisions that follow if the information reaching a researcher has already been shaped by the systems behind an AI tool?鈥
Her research considers the larger systems behind AI tools and their 鈥渉eritage,鈥 since many of the major frontier large language models -- including ChatGPT, Claude and Copilot -- are built on shared research and techniques.
鈥淚 think awareness is an important aspect of AI,鈥 she said. 鈥淢any debate whether one company appears more 鈥榬esponsible鈥 than another, but they are often overlooking the broader infrastructures that connect these technologies.鈥
The question extends beyond research, as AI is increasingly being used to assist decision-making in fields ranging from health care and education to business and national security.
For qualitative researchers like Jordan, the findings call for greater awareness of the systems in which AI research and use are already embedded. To further explore these concepts, she is in the process of developing a new study to analyze the choices researchers make with AI that will allow participants to see how AI interprets their words in real time.
Jordan鈥檚 work contributes to a growing body of research examining the ethical implications of AI and the larger systems and relationships that make these technologies possible.