As technology evolves, the interplay between humans and artificial intelligence (AI) becomes increasingly critical, especially in fields where safety and security are paramount. A project spearheaded by Dr. Mancenido at the Center for Advanced Operational Efficiency (CAOE) is working to refine this dynamic, focusing on improving human-AI collaboration within digital screening systems, such as those used by Transportation Security Administration (TSA) officers at airports.
The Human-AI Synergy
At the heart of Dr. Mancenido’s project is a key question: How can humans and AI work together more effectively? Drawing from a rich background in industrial engineering and statistics, Dr. Mancenido’s expertise in systems thinking has allowed him to approach the challenge in a unique way. “I think of everything in terms of systems. Systems have components, inputs, and outputs. When you add humans into the equation, those variables become even more complex,” Mancenido explained. The project aims to answer this question in the context of security screening systems, where AI technologies, such as facial recognition and face verification, are increasingly integrated.
Dr. Mancenido’s work begins with exploring the potential for humans and AI to complement each other. One example highlighted is the collaboration between TSA officers and automated facial recognition systems at airport checkpoints. While AI handles the heavy lifting of face matching, the TSA officers, with their specialized training and knowledge, ensure the accuracy and relevance of the AI’s decisions. “We want to explore how these systems can work better together, rather than as separate, individual entities,” Mancenido said.
A core component of the project is to develop a measure of this cooperation. “We need to know how to evaluate which human-AI team is performing better,” Mancenido said, noting that the interdependence between humans and AI in these systems is complex and nuanced.
From Concept to Application: Collaborating with DHS Stakeholders
The project has made significant strides over the past two years, with ongoing collaboration with key stakeholders in the Department of Homeland Security (DHS). Dr. Mancenido highlighted the importance of working closely with subject matter experts (SMEs) in the TSA. “When designing human subject studies, it’s critical to understand that what might be easy for an expert in the field could be difficult for an average person,” Mancenido said. The challenges posed by this dynamic have led the team to shift their focus towards involving TSA officers in the design process, ensuring that the project remains relevant and useful to the people who will ultimately benefit from it.
During the project’s biennial review, Dr. Mancenido reflected on how the methodology had evolved. “We’ve had to adjust our metrics and data collection methods, incorporating more feedback from professionals working in the field. Our focus is on producing results that are not just scientifically novel, but practically useful for the TSA,” Mancenido explained.
This practical orientation is a defining feature of CAOE projects. Unlike traditional academic research, which often prioritizes scientific merit, CAOE’s work is driven by real-world applications. “We want to give our DHS stakeholders something that’s usable, not just hypothetical,” Dr. Mancenido said.
Addressing Challenges and Ensuring Impact
Throughout the course of the project, Dr. Mancenido and the research team have faced several challenges, particularly in gathering accurate data from TSA officers. “Because the work involves high-stakes security measures, we had to ensure that our data collection was directly informed by the experiences of TSA officers, not just generalized from crowd-sourced participants,” Mancenido said. The solution was to design focus group discussions and engage with the experts in a way that allowed for actionable insights.
One of the project’s most significant contributions is its focus on human-AI teaming in safety-critical contexts. Dr. Mancenido emphasized the importance of considering both the likelihood of an event occurring and the severity of its potential impact. “We’re assessing risk in a very contextual way,” Mancenido explained. This approach ensures that the project’s findings are not only scientifically valid but also operationally relevant.
The project’s impact is measurable in several ways. One key metric is the involvement of TSA officers in the study’s data collection phases. By engaging over 120,000 participants across three airports, the team is ensuring that their findings resonate with the people who will use them. Dr. Mancenido also noted the project’s importance for students involved in the research. “It’s a great opportunity for students to see how real-world processes work,” Mancenido said, adding that many students contribute to the research as part of the project’s broader community outreach.
Recommendations for the Future
The project’s success is evident not only in its collaboration with DHS stakeholders but also in its potential to generate actionable recommendations. Dr. Mancenido emphasized the importance of producing insights that directly improve the TSA’s operations. “One of the measurable impacts will be the number of recommendations we can provide that will help the TSA improve their work environment and efficiency,” Mancenido said.
Although the challenges of optimizing human-AI systems are complex, Dr. Mancenido remains optimistic about the project’s future. “We might not have a perfect measure of optimal performance, but by refining these human-AI systems, we can make a meaningful difference,” Mancenido concluded.
As the project continues to develop, the team remains focused on enhancing the safety and effectiveness of security systems. By fostering collaboration between humans and AI, they are not only advancing technology but also creating a more secure and efficient future for everyone.
