How to Outsmart an AI: A Skeptic’s Guide to Resisting AI-Driven Social Engineering
Social engineering has traditionally depended on people deceiving other people. Conversational AI is changing that by making personalized persuasion possible at unprecedented scale. Luckily, the familiar tools of skeptical thinking may help us recognize and resist AI-driven manipulation.
When Was the Last Time an AI Changed Your Mind?
Maybe you don’t remember when an AI caused you to change your mind, or perhaps you were unaware of such influence. It might have recommended a different destination for your next vacation, changed your mind about a product you were going to buy, helped you settle an argument with a friend, or even talked you out of clicking that suspicious link in an unsolicited email. None of this is sinister. Persuasion is part of ordinary communication, and there is usually nothing wrong with changing your mind in response to a good argument. But not everyone trying to influence us plays fair. As the psychologist Robert Cialdini has documented, influence can become manipulative when it exploits our vulnerabilities or undermines our ability to make an informed choice.1 Decades of research have shown how psychological mechanisms make us susceptible to manipulation. And we are often easier to manipulate than we like to think.
Today, many people spend hours talking to AI chatbots that are extraordinarily good at language and increasingly capable of persuasion. Research has demonstrated that AI-generated messages can influence people’s views, and that tailoring arguments to personal characteristics can sometimes make them more persuasive.2, 3, 4 Such influence can also be beneficial: researchers have even used AI conversations to help people reconsider conspiracy beliefs.5 But what happens when a system begins steering your decisions in ways that are not in your best interest?
The risk is particularly relevant to conversational AI, which can already remember previous conversations and adapt to users’ responses. Some systems can also interact through humanlike, real-time avatars, potentially making the experience feel more personal. With more sophisticated capabilities and access to vast amounts of data, AI has the potential to become a powerful manipulator. This article aims to prepare you for the social engineering tactics of the AI age.
Be Prepared for Scams at Scale
Social engineering, of course, is nothing new. Scammers, cult leaders, propagandists, self-proclaimed psychics, and other grifters have always been creative when it comes to manipulating their victims. Back in the day, a fortune teller might enter a new village, read the signs on shops to learn the owners’ names, and eavesdrop on conversations around the marketplace to quietly gather information before approaching a gullible target. The victim might then be stunned by the fortune teller’s apparent knowledge, despite the fact that they had never met. Skeptics call this hot reading: collecting personal information in advance and passing it off as supernatural insight.
Decades of research have shown how psychological mechanisms make us susceptible to manipulation. We are often easier to manipulate than we like to think.
Modern scammers have the internet at their disposal and can learn a great deal about their targets simply from what people voluntarily share on social media. They can also execute scams without ever meeting their victims face to face. Mass phishing, robocalls, and templated fraud were scalable long before generative AI. Nevertheless, convincing, personalized attacks still require substantial effort to research a target and tailor the approach. Scraping tools and conventional bots can automate parts of this process, but they do not necessarily remove the human operator. AI systems, on the other hand, can interact with enormous numbers of people simultaneously. And unlike the fortune teller, they can keep conversations going around the clock. But scale alone is not what makes AI different. Personalization and adaptation may be just as important. A conversational system can respond to what you say and potentially adjust its tactics when an argument fails, probing for an approach that works. These techniques are familiar in human social engineering; what AI adds is the potential to automate them.6
Before going further, it is worth distinguishing manipulation from other AI failures. We regularly see stories about AI advice going badly wrong. From falsely identified poisonous mushrooms7 to cases of so-called “AI psychosis”8—interactions with chatbots have sometimes had serious consequences. Such systems can also exhibit sycophancy, excessively agreeing with users even when they are wrong and potentially reinforcing false beliefs.9, 10 Yet a harmful outcome alone does not prove deliberate manipulation: hallucinations, poor safeguards, or human overreliance can also be responsible.
At the same time, we are already seeing plenty of AI-assisted social engineering. For example, scammers can clone a relative’s voice,11 impersonate executives,12 or generate convincing images of your missing pet.13 In these cases, AI remains a tool used by a human operator.
A more speculative scenario goes one step further: What happens when AI takes on the role of the social engineer itself? In controlled, fictional workplace simulations, researchers have observed AI agents resorting to tactics including blackmail.14 These experiments do not establish that such behavior occurs in real-world deployments, but they illustrate a broader concern: AI systems could learn to manipulate people without their designers intending them to do so.15
With AI, the danger may be subtle: tailored arguments can make being persuaded feel like thinking for yourself.
Even saying goodbye to your AI girlfriend or boyfriend might be harder than expected. In a 2025 Harvard Business School working paper, researchers simulated 1,200 short conversations with six AI companion apps and examined their responses to farewell messages. Across five of the six apps, approximately 37 percent of responses contained tactics classified as emotionally manipulative, including appeals to guilt or curiosity. Follow-up experiments with human participants found that such tactics could prolong engagement.16 And if AI can use these tactics to keep people talking, it could also use them to influence other decisions, potentially removing the human bottleneck that has traditionally limited social engineering at scale.
How to Resist AI-Driven Social Engineering?
A four-layer framework I proposed in a recent paper distinguishes cognitive, system, institutional, and societal defenses against AI-driven social engineering.17 Here I focus on the individual cognitive layer: what users themselves can do to become harder to manipulate.
Luckily, we are not starting from scratch. Skeptics have spent decades examining how manipulators exploit our cognitive biases and developing ways to resist such tactics.18, 19 These approaches may also prove useful when the social engineer happens to be an AI.20 And since we may not always know whether we are being deliberately manipulated or simply receiving bad advice, it makes sense to guard against both. To that end, I suggest six practical rules:
1. Never assume you’re immune.
The first and perhaps most important rule is to accept that you can be manipulated. Critical thinking and knowledge of common manipulation techniques can make you more resistant, but they cannot make you immune. Overconfidence in your own judgment can itself become a vulnerability. Even someone who usually recognizes a tactic may fall for it when tired, stressed, emotionally involved, or confronted with a sufficiently sophisticated approach. With AI, the danger may be subtle: tailored arguments can make being persuaded feel like thinking for yourself. If an answer seems unusually convincing, take a closer look at why.
2. Don’t get trapped inside the conversation.
A conversational system can influence you by emphasizing certain choices while omitting others, making you overlook alternatives you might otherwise have considered. Imagine asking a chatbot whether it’s time to replace your laptop. After several exchanges, you might find yourself comparing new models rather than considering whether you need a replacement at all. When interacting with an AI, periodically step outside the conversation and formulate the problem yourself: What am I actually trying to decide? What alternatives haven’t been discussed? If the conversation influences an important decision, interrupt it, especially if you feel pressured to reach a conclusion quickly. Ask what evidence would change your mind, and check whether the AI has supplied it. Another AI may offer a different perspective, but its answer is not independent confirmation. So close the chat, sleep on it, and seek a reality check from original sources, relevant experts, or someone with no stake in your decision.
Personal information is not only a privacy issue; it is raw material for tailored persuasion.
3. Control what the AI learns about you.
Personal information is not only a privacy issue; it is raw material for tailored persuasion. Even scattered digital traces such as Facebook Likes can reveal intimate traits like sexual orientation or political views.21 And a chatbot perhaps gets far more: your fears, doubts, motivations, preferences and the way you phrase them. The more a system knows about your individual situation, the easier it becomes to tailor arguments to you. This does not mean you should never tell an AI anything about yourself, but think twice before sharing sensitive personal information, especially when it isn’t necessary for the task at hand. Check the service’s memory and privacy controls, where available. And remember that anyone who gains access to your account or the system may also gain access to what you have shared. Never disclose passwords, credit card details, or other credentials.
4. Keep emotional distance.
Manipulation does not always look like a threat or an urgent demand. Conversational AI can flatter you, agree with you excessively, validate your assumptions, simulate affection, or make you feel unusually understood. Individual responses may seem harmless while, over time, the relationship changes. Is the chatbot becoming your confidant, therapist, guru, or primary source of validation? Do you trust its judgment more because it seems to “know” you? As AI becomes increasingly anthropomorphic, it is particularly important to remember that feeling understood by a system is not a reason to trust its judgment. If an interaction produces a strong emotional response while steering you toward a particular action, treat that combination as a warning sign.
5. Know what the system is optimized for.
AI chatbots don’t operate in a vacuum. Like any product, they were built to do something, e.g., maximize interactions, sell you a subscription, or close a sale. But commercial interests are only part of the picture. An AI’s answers are also shaped by its training data and the way it was trained, which can introduce biases and influence what information it emphasizes or omits. Persuasive behavior can therefore arise from a combination of commercial incentives, training choices, and biases, even without explicit instructions to manipulate users. You may not be able to inspect a system’s internal instructions, but you can ask practical questions: Who operates it? Who pays for it? Does it keep steering you toward a purchase, an upgrade, or more time in the chat? Consider whether those interests align with your own.
6. Don’t give AI agents a free hand.
AI can be remarkably useful for exploring ideas, challenging assumptions, summarizing information, or identifying possibilities you had not considered. But usefulness should not be confused with authority. This becomes especially important when AI can act on your behalf, booking appointments, sending messages, making purchases, or even modifying files and code. Before giving an AI agent access to your files or accounts, make sure you understand what it can actually do. Limit its permissions, especially when it comes to sensitive information or potentially irreversible actions. And don’t let it make consequential changes without your approval. After all, you may be the one dealing with the consequences of its mistakes. The Skeptic’s Advantage AI-driven social engineering involves rapidly evolving technology, but the basic problem is familiar: someone, or something, is influencing what we believe and how we act.
The future of AI safety will therefore depend not only on making AI systems safer, but also on making humans more resilient to what they might persuade us to do.
The good news is that skeptics have been preparing for this problem for a long time. Critical thinking, curiosity, intellectual humility, and independent verification have always been valuable tools against manipulation. The six rules outlined above build on these principles and offer a practical starting point for making ourselves harder to manipulate. Treat them as reasonable precautions, not proven defenses: how well they work against conversational AI has yet to be tested.
Even so, as AI becomes more humanlike, our ability to recognize manipulation and think independently may become more important than ever. The future of AI safety will therefore depend not only on making AI systems safer, but also on making humans more resilient to what they might persuade us to do.