Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations

Mangesh Pujari ; Anil Kumar Pakina ; Anshul Goel
International Journal Science and Technology (IJST... 📅 2023 Vol 2, No 3 🔗 DOI: 10.56127/ijst.v2i3.1956
Abstract
The rapid deployment of large language models (LLMs) on mobile devices has introduced significant privacy concerns, particularly regarding data collection, user profiling, and compliance with evolving AI regulations such as the GDPR and the AI Act. While these on-device LLMs promise improved latency and user experience, their potential to inadvertently leak sensitive information remains understudied. This paper proposes a red teaming framework to systematically assess the privacy risks of phone-based LLMs, simulating adversarial attacks to identify vulnerabilities in model behavior, data storage, and inference processes.
We evaluate popular mobile LLMs under scenarios such as prompt injection, side-channel exploitation, and unintended memorization, measuring their compliance with strict privacy-by-design principles. Our findings reveal critical gaps in current safeguards, including susceptibility to context-aware deanonymization and insufficient data minimization. We further discuss regulatory implications, advocating for adaptive red teaming as a mandatory evaluation step in AI governance. By integrating adversarial testing into the development lifecycle, stakeholders can preemptively align phone-based AI systems with legal and ethical privacy standards while maintaining functional utility.
Keywords
Large Language Models (LLMs) Red Teaming AI Privacy Mobile AI Privacy Regulations Adversarial Testing Model Evaluation Data Security Innovation and Ethics Privacy-Preserving AI
How to Cite

Mangesh Pujari, et al. (2023). Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations. International Journal Science and Technology (IJST), 2(3). https://doi.org/10.56127/ijst.v2i3.1956

Mangesh Pujari; Anil Kumar Pakina; Anshul Goel, "Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations," International Journal Science and Technology (IJST), vol. 2, no. 3, 2023.

Mangesh Pujari; Anil Kumar Pakina; Anshul Goel. "Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations." International Journal Science and Technology (IJST), vol. 2, no. 3, 2023.

Mangesh Pujari; Anil Kumar Pakina; Anshul Goel. "Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations." International Journal Science and Technology (IJST) 2, no. 3 (2023).

Mangesh Pujari, et al. (2023) 'Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations', International Journal Science and Technology (IJST), 2(3). doi: 10.56127/ijst.v2i3.1956.

Mangesh Pujari; Anil Kumar Pakina; Anshul Goel. Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations. International Journal Science and Technology (IJST). 2023;2(3).

Artikel Terkait

Heart Disease Prediction Using Logistic Regression and K-Nearest Neighbor: A Comparative Study of Classification Algorithm Performance

Sururi, Yan Risa Aspi; Ghozi, M. Asadullah Al

Perception and Behavior of Farmers Towards the Use of Organic Fertilizers in Rice Plants in Kewayuhan Village, Pejagoan District, Kebumen Regency

Furqon, Ngafiful; Aprilia, Rennanti Lunnadiyah

Integrated Technical, Thermal, and Techno-Economic Assessment of Hot-Oil and Electric Heating Systems for 15–22 MT Asphalt Storage Tanks

Arifuddin, Arifuddin; Wiseno, Elbi

Generative AI Risk Governance in Organizational Information Systems: A Conceptual Model Integrating Security, Ethics, and Digital Trust

Nurdiyanto Yusuf

Design of a SCADA-Based Control System for Hybrid Power Generation Integrating a Screw Turbine and Solar Panels

Mekel, Jonah Alfred; Bawano, Franklin; Mekel, Alfred Noufie; Saroinsong, Tineke

Performance Evaluation of an N₂ Membrane Generator under Different Operating Temperatures

Pradito, Okttu Kristi; Korawan, Agus Dwi; Mawarni, Drajat Indah
Tren Sitasi Jurnal