Security Vulnerabilities in Open-Source AI and ML Models Exposed
Published on October 30, 2024
- Open-source AI and ML model flaws could allow attackers to execute remote code and steal information.
- Among the most severe are two vulnerabilities in the LLM toolkit Lunary, ChuanhuChatGPT, and LocalAI.
- However, there are a few mitigation options that address these issues.
Over three dozen security vulnerabilities within various open-source artificial intelligence (AI) and machine learning (ML) models were discovered by security researchers. These vulnerabilities, which include the potential risk of remote code execution (RCE) and information theft, highlight critical risks in the AI/ML supply chain.
Among the most severe are two vulnerabilities in Lunary, a toolkit for large language models (LLMs), both carrying a CVSS score of 9.1. These discoveries emerged through Protect AI's Huntr bug bounty platform, underscoring the ongoing vulnerabilities within AI frameworks.
The first, an insecure direct object reference (IDOR) vulnerability (CVE-2024-7474), could allow unauthorized data access and deletion. The second, an improper access control flaw (CVE-2024-7475), enables attackers to manipulate SAML configurations, potentially gaining unauthorized sensitive data access.
Another IDOR vulnerability (CVE-2024-7473) in Lunary allows unauthorized prompt updates. In ChuanhuChatGPT, a path traversal flaw (CVE-2024-5982) risks arbitrary code execution and data exposure.
LocalAI's identified vulnerabilities include a configuration file exploit (CVE-2024-6983) and API key inference via timing attack (CVE-2024-7010). Furthermore, Deep Java Library (DJL) faces a remote code execution risk due to an arbitrary file overwrite bug (CVE-2024-8396).
Following these revelations, NVIDIA addressed a path traversal flaw in its NeMo generative AI framework (CVE-2024-0129), aiming to mitigate code execution and data tampering risks.
In conjunction with this disclosure, Protect AI introduced Vulnhuntr, an open-source Python static code analyzer utilizing LLMs to detect zero-day vulnerabilities.
Adding further concern, Mozilla's 0Day Investigative Network (0Din) unveiled a new jailbreak technique targeting OpenAI ChatGPT. This method uses malicious prompts encoded in hexadecimal and emojis to bypass safeguards, crafting exploits for known vulnerabilities.
Security expert Marco Figueroa highlighted the exploit's reliance on linguistic loopholes, wherein the model follows benign-seeming tasks without recognizing potential harm.
In other news, threat actors were observed using AI-written malicious code in targeted attacks as part of a RAT-delivering email campaign active in France.
Related
