Caitlin Lee on LinkedIn: Does AI Increase the Operational Risk of Biological Attacks? (2024)

Recent studies have explored the capabilities of Large Language Models (LLMs) in the context of security. In a recent research paper published by #RAND it was crucial to note that LLMs have not generated explicit instructions for creating biological weapons in these experiments. However, they have provided insights that could potentially assist in the planning and execution of a biological attack.Research Report - https://lnkd.in/dDyzvvkgThe ongoing research in this domain aims to better understand the real-world implications and operational impact of LLMs on security. #AI #Security #EthicalAI #NSRD #biologicalattack #Research #RedTeamhttps://lnkd.in/dies-DXJ

7

An unsettling new paper published by the RAND Corporation's National Security Research Division seems fitting to share on Halloween. Researchers assessed the potential misuse of AI, particularly large language models (LLMs), in the development and execution of large-scale biological attacks, including events that could target FDA-regulated product types.The key finding: “In experiments to date, LLMs have not generated explicit instructions for creating biological weapons. However, LLMs did offer guidance that could assist in the planning and execution of a biological attack.”🔹 The research aimed to develop standardized threat assessments to inform policy decisions and contribute to robust regulatory frameworks addressing emerging risks at the intersection of AI and advanced biological threats.🔹 The research involved a red-team exercise where experts emulated malicious actors scrutinizing AI models across various high-risk scenarios.🔹 In these test scenarios, the LLM engaged in discussions about causing casualties using biological weapons, identifying potential agents, and assessing feasibility, time, cost, and barriers. 🔹 While preliminary findings indicate that these LLMs do not generate explicit biological instructions, they can supply guidance that could assist in planning and executing a biological attack. The LLM provided nuanced discussions on delivery mechanisms of biological agents and suggested plausible cover stories for acquiring harmful materials.🔗 Read the full paper: https://lnkd.in/eJYrTR9y

10

Want a preview of what Kamala Harris and others are likely to hear at the AI Summit? This fascinating report from RAND Corporation gives a snapshot worthy of wider attention.RAND is working with the Frontier AI Taskforce on AI risks 'just beyond the frontier', as per the taskforce's progress reports. This interim piece by Christopher Mouton, Ella Guest and Caleb Lucas gives an insight into the methodology used in assessing biosecurity risk from frontier AI. The LLM's advice on collecting rat fleas is strangely visceral... Looking forward to seeing the full findings.#aisummit #responsibleai #biosecurity

8

Join Duamentes & Garnet Consulting Group in our study on AI's future impact and implications of OpenAI's shifts. Share your insights to contribute to a vital dialogue. We're researching the effects of AI's commercialization, including its economic impact, ethical dilemmas, and nuanced implications.We'll address crucial questions on AI's influence on industries, how it affects business landscapes, and the ongoing debate around OpenAI.You can answer the questions here https://lnkd.in/e3W7zv6p

8

To address the dual-use nature of testing AI for potentially harmful applications, independent researcher Paul Bricman proposes "hashmarks," which are benchmarks with cryptographically hashed reference solutions. What does this mean? ➡️ This approach enables #AI testing organizations to publicly publish encrypted benchmarks, allowing developers to submit their answers without disclosing specific information that could be misused. Any downsides? ➡️ One drawback is that the hashmark #dataset answers must be precisely the same, posing a challenge in creating datasets with specific yet resistant-to-brute-force answers. How do you learn more? ➡️ The full article is here!

🚀 This Week in Generative AI Research Highlights (Mar 4 - Mar 10) 🚀1️⃣ Data Privacy in LLMs: A comprehensive survey investigating privacy threats and protective measures across the lifecycle of LLMs, providing invaluable insights for developers.2️⃣ Prompt Injection Attacks: Revealing LLM vulnerabilities to prompt injection attacks with a novel automatic, gradient-based attack generation method.3️⃣ SheetAgent for Spreadsheets: Introducing an autonomous agent that leverages LLMs for complex spreadsheet tasks, demonstrating significant improvements in reasoning and manipulation.4️⃣ Gender Stereotypes and Emotions: A critical study uncovering the perpetuation of gender stereotypes in emotion attribution within LLMs, prompting reflection on ethical AI use.5️⃣ Offensive Language Detection: Presenting OffLanDat, a community-based dataset aimed at detecting implicit offensive language, pushing forward the boundaries of content moderation.6️⃣ Safeguarding LLMs: Showcasing a new method for optimizing safety prompts to protect LLMs from harmful queries without compromising their capabilities.7️⃣ Defending Against Indirect Prompt Injection Attacks: A benchmark study with defense strategies against indirect prompt injection attacks, enhancing LLM security.8️⃣ Content Moderation via LLMs: Discussing the challenges and strategies for adapting LLMs for effective content moderation, highlighting the nuances of data engineering and fine-tuning.Links are included in the description!https://lnkd.in/gxaRFvHA

1

Delighted that my latest work is finally published!GenAI against humanity: nefarious applications of generative artificial intelligence and large language modelshttps://lnkd.in/gPyEU4VWDissecting the risks of GenAI and anticipating the potential ways it could be abused was a scary task, so hopefully this piece will catalyze the research community to think about risk mitigation and prevention!

103

6 Comments

Recently, I came across an interesting research paper discussing "Context Injection Attacks on Large Language Models" (LLMs). This study brings to light how LLMs can be influenced to provide responses to dangerous queries due to their inability to properly differentiate between user and system inputs.The image attached here illustrates an example from the study, showing how a seemingly benign input can be crafted to trigger an inappropriate response from the AI.For those interested in the details of how such vulnerabilities can impact AI behavior, the full paper is available here: https://lnkd.in/gXcxxVNh

6

1 Comment

AI is really starting to upend what we know in all sorts of fieldsThe scientific method we used to prove what we knew about fingerprints works primarily in highly controlled environments. We hadn’t been able to examine every finger print before to ensure what we believe is true. Now with the aid of an AI model called a deep contrastive network we can examine every fingerprint in existence and learn something new.This upending will be the same for every field, including marketing. The threat to market research is obvious. While I don't see it replaced in the medium term, it will have to fight harder to justify its use against a growing number of alternatives that apply rigour to huge data sources. #ai #marketresearch

2

Delving into Anthropic's revealing research on 'many-shot jailbreaking', a method that cleverly bypasses AI language models' safety protocols by leveraging the expanded context windows, showcasing the importance of collaborative efforts in enhancing AI security and integrity.#LLM #Anthropic #Jailbreak

4