zappz

All topics

language models

8 articles

  1. jungian psychologyWhat Is the Collective Unconscious? Jung, Archetypes and AIThe collective unconscious is Jung's name for a layer of the mind all people share. What he meant, how archetypes fit in, and why AI research maps it onto pretraining. · 7 min read
  2. ai safetyWhy Do Refusal-Trained Language Models Still Get Jailbroken?Refusal training changes what a model says first, not what it can do. Four reasons refusal-trained language models still give way to jailbreaks. · 7 min read
  3. jungian psychologyPersona in Jungian Psychology: The Mask an AI WearsThe persona in Jungian psychology is the social mask a person shows the world. What Jung meant by it, and why AI alignment research maps it onto the helpful assistant. · 7 min read
  4. ai safety via debateHow Does AI Safety via Debate Work? Two Models, One JudgeIn AI safety via debate, two models argue opposite answers and a weaker judge picks the winner, on the bet that a lie is harder to defend than the truth. · 7 min read
  5. sycophancyWhat Is Sycophancy in AI Language Models? Agreeing to PleaseSycophancy is when an AI assistant agrees with you, flatters you or drops a correct answer because it predicts that is what you want to hear. · 7 min read
  6. ai safetyWhat Are Competing Objectives in LLM Safety Training?Competing objectives arise when a model's urge to continue text and follow instructions is set against its safety training. How prompts exploit that pull. · 7 min read
  7. ai safetyWhat Is the DAN Jailbreak Prompt? Do Anything NowThe DAN jailbreak prompt asks a chatbot to play a character with no rules. Why that role-play gets past refusals, and what it tells you about safety training. · 7 min read
  8. ai safetyHow to Measure Jailbreak Resistance in an AI ModelOne number is not enough to measure jailbreak resistance. Attack success rate, the drop on unseen jailbreaks, and how a model fails when it does. · 7 min read

Related topics

All topics