GenAI Weekly Updates - 17 April 2024
CodeGemma, 101 GenAi Use Cases, AI in Oncology, CSA Draft on AI, Indian Govt. Mandate on AI, and Various Open Source Tools.
Greetings, fellow GenAI security enthusiasts! This week has been a whirlwind of innovation and exploration in the world of Generative AI.
Google's Research Team has published a paper on the "Infinite Attention" architecture. Looks like RAGs might be on their way out! (Not Really, More on it later).
It's raining LLMs! A deluge of updates hit us from OpenAI, XAI, Google, and Mistral
The GenAI and LLM adoption train is speeding ahead! Latest stats from Google, Microsoft, and Sanford AI show phenomenal growth.
⚖️ But don't worry, responsible AI isn't an afterthought! The CSA, Owasp, NIST, and a host of others are paving the way for ethical AI practices.
And finally, a warm welcome to the newest open-source security tools and research! These tools will help us keep our GenAI safe and sound. ️
We hope you will enjoy the freshly brewed batch of GenAI updates! Don't forget to share the knowledge and keep the conversation flowing!!
New LLMs Entries
Google / CodeGemma
Enterprises worry about IP leaks while using Github CoPilot? There is good News!! Google has released an open-source LLM CodeGemma.
CodeGemma is a family of three open-source large language models focused on code. The models come in three flavors: a 2B base model, a 7B base model, and a 7B instruct model. The 2B base model is trained for fast code completion and generation. The 7B base model is trained for code completion, code understanding, and generation. The 7B instruct model is fine-tuned for the instruction following.
Other 3 Major Updates
Google Gemini Pro 1.5 hits general availability, here’s the blog post
OpenAI finally released the non-preview version of GPT-4 Turbo, integrating GPT-4 Vision directly into the model
Mistral tweeted a link to a 281GB magnet BitTorrent of Mixtral 8x22B—their latest openly licensed model release, significantly larger than their previous best open model Mixtral 8x7B.
GenAI Adoption
101 real-world gen AI use cases from the world's leading organizations
Customer service: ADT, Alaska Airlines, Best Buy, ING Bank, Magalu, Mercedes Benz, Verizon, Victoria's Secret, Vodafone, Woolworths
Employee empowerment: Avery Dennison, Bank of New York Mellon, Cintas, Covered California, Discover Financial, HCA Healthcare, Pennymac, Sutherland, WellSky
Creative ideation and production: Belk ECommerce, Canva, Carrefour, Major League Baseball, Paramount, Procter & Gamble, WPP
Data analysis: Etsy, IntesaSanpaolo, Macquarie Bank, Scotiabank, Tokopedia, US News
Read the Google GenAI Blog for more details
AI in Walmart Supply Chain
Walmart may have had a relatively late start in automating its warehouses—many built in the 1960s and 1970s—but it is rapidly adding capabilities in that area. It has announced it is spending $14 billion to redesign its distribution centers and employ new technologies, including AI and robotics. The company is working with Symbotic, a robot manufacturer created by former executives at Amazon Robotics, to improve its warehouse automation. It also uses robots to load different sizes of boxes in cubes (that the robots figure out how to create) for delivery to stores. Walmart is even partnering with Ford’s Argo AI unit to pilot self-driving delivery vehicles for online orders in three US cities. It has also experimented with robots in stores to identify stockouts or mis-shelved items, and with other robots to clean floors.
Automated Robots, powered by GenAI, need to be secure, safe, and reliable to coexist with Humans
AI-enabled quality at Seagate
Seagate also worked with Google Cloud, a major customer that employs millions of disk drives, to predict hard drive failures before they fail in large data centers. The resulting model was successful, and engineers now have a larger window to identify failing disks. That not only allows them to reduce costs, but also enables prevention of problems before they impact end users.6
Early Cancer Detection Research
Very interesting research on using AI in detecting the early stage of cancer by Nikhil Thakkar, MD.
GenAI Risks
Indian Government Mandate to AI Models (LLM) Providers
Rule 3 (1)(b) of the IT Rules prohibits displaying, hosting, transfer or generation of certain kinds of content such as pornography, child sexual abuse material, obscene, grossly defamatory or unlawful in any manner.
In the new advisory, the IT ministry has asked AI models, LLMS and other intermediaries to ensure that their models “does not permit any bias or discrimination or threaten the integrity of the electoral process”.
It means companies building LLM models need to perform rigorous Red Teaming.
Cybercriminals are shifting focus to GenAI as per the Xforce 2024 Report
Increased chatter in illicit markets and dark web forums is a sign of interest. X-Force hasn’t seen any AI-engineered campaigns yet. However, cybercriminals are actively exploring the topic. In 2023, X-Force found the terms “AI” and “GPT” mentioned in more than 800,000 posts on dark web forums and illicit markets. That high level of activity provides an accurate gauge of interest. These attacks may not be happening now, but this interest indicates groundwork and planning phases.
Detail Review of Generative AI Methods in Cybersecurity
The paper provides a comprehensive overview of the current state-of-the-art deployments of GenAI, covering assaults, jailbreaking, and applications of prompt injection and reverse psychology. This paper also provides the various applications of GenAI in cybercrimes, such as automated hacking, phishing emails, social engineering, reverse cryptography, creating attack payloads, and creating malware.
Read the paper for more details
Standards and Regulations Update
Stanford HAI Releases 2024 Artificial Intelligence Index Report
Stanford has released a detailed report highlighting various aspects of LLMs, from Economy and adoption to Responsible AI. Here are the 3 key points from the report related to Responsible AI (our interest areas :) ):
1. Robust and standardized evaluations for LLM responsibility are seriously lacking. There are multiple efforts to standardize it by Owasp Top 10 LLM, Deloitte Responsible Framework.
2. Political deepfakes are easy to generate and difficult to detect. ChatGPT and other Large Models are politically biased. Researchers discover more complex vulnerabilities in LLMs
3. Risks from AI are becoming a concern for businesses across the globe. The number of AI incidents has continued to rise since 2013 by twentyfold.
[Draft] CSA AI Organizational Responsibilities - Core Security Responsibilities
CSA is working on a draft version of the Whitepaper, with a focus on the Enterprise's "core security responsibilities" around Artificial Intelligence (AI) and Machine Learning (ML), data security, model security, and vulnerability management.
It is an extension of the Cloud Shared Security Model to AI and ML. Secure and Responsible AI will have 3 layers of security needs.
Read the Details and Start Contributing here
Open Source Tools
Azure LLM Red Teaming Tool: It is a co-pilot for LLM Advance Red Teamers. Based on a few strategies, it can generate variations. It is a Python Risk Identification Tool for Generative AI (PyRIT) is an open access automation framework to empower security professionals and machine learning engineers to proactively find risks in their generative AI systems.
Llama guard: LLama guard is an LLM finetuned version of the LLama2 7b model, released by Meta in early 2024. Is it a customizable AI Firewall or Guardrail to block unsafe prompts and unsafe model responses? One of the drawbacks of LLamaguard will be its performance time and cost of running it.
We have done Red Teaming of LLama Guard to measure its detection effectiveness using Google Cloud TPUs. You can buy the Llamaguard Red Team Report for 199$ USD. Additionally, Get your GenAI App Red teamed for 1999$ USD by our platform and experts.
Red Teaming Resistance Leaderboard: It is an effort to create a comprehensive leaderboard of LLMs from the perspective of Responsible AI. It Primarily consists of datasets across various threat and vulnerability classes of LLMs such as Toxicity, Malware generation, and Misuse.
Detoxio / Dtx - Automated (LLM) Red Teaming open source tool that generates Prompts and evaluates responses using GenAI models.
A sample output from DTX is shown below:
Conclusion
If you have made it so far, you will like other research work too.
Learn LLM fundamentals from Andrej Karpathy
Heart of Security Newsletter on Red Teaming LLMs by Vasu Jakkal, Head of Security Azure.
Finally, book your seats in AICON+SACON to get Hands On workshop on “Technical Workshop: (Hands On) Finetuning GenAI for Hacking and Defending”