UL NO. 454: The First AI Breaches

AI Avatar Breaches, Gullibility is Vulnerability: Conspiracy is Threat, Caldera's New Plugin, and more...

Author

Daniel Miessler
October 14, 2024

SECURITY | AI | PURPOSE
UNSUPERVISED LEARNING is a newsletter about upgrading to thrive in a world full of AI. It’s original ideas, analysis, mental models, frameworks, and tooling to prepare you for the world that’s coming.

TOC

Hey there!

  • ☄️We saw the comet yesterday! Was quite bright even to the naked eye between 7:15 and 7:45.

📷Credit: Bunny via iPhone 16 Pro

  • ✏️I wrote a tutorial on how to use any Hugging Face model within Ollama! So now, instead of a couple of dozen models, you can use thousands! MORE

  • 💰My buddy Marcus Hutchins and I disagree about 1) whether Elon is a real builder who will continue to innovate and 2) whether he still has liberal ideals in him or if he’s permanently far-right now. So I offered 3 bets: 1) that Tesla stock would hit at least $250 by June 30, 2025, and 2) hit at least $300 by December 31, 2025, and 3) that Elon would publicly oppose Trump on some liberal/authoritarian/freedom issue by December 31st, 2025. THE LINKEDIN THREAD

  • 📺 I did a talk for the WIPO UN Group, and it went really well. Thanks to Olivia Fabreschi for being not just a great host but someone who’s clearly thinking about these things herself. Someone to watch for sure! OLIVIA ON LINKEDIN | THE TALK

Sponsor

Your OAuth risk investigation checklist 

For most employees, OAuth grants provide a familiar “easy button” for creating new accounts or integrations.

But, OAuth grants have also been exploited by attackers for nefarious purposes. It’s good practice to regularly review your organization’s OAuth grants to identify any that are overly permissive or could be malicious.

This blog post covers four key areas to look at when assessing an OAuth grant, including a free template you can use to make sure you cover all the important steps in your OAuth reviews.

SECURITY

An attacker has accessed Muah.ai's AI chatbot database, exposing sensitive user interactions with AI chatbots, including sexual fantasies. And the user accounts were linked to peoples’ personal email addresses. MORE

💡As per usual, most “AI hacking” will be normal hacking of regular infrastructure used by AI companies.

Yes, there will be lots of input validation issues and prompt injection and all that, but the vast majority of the damage will come from customers giving their souls to small startups in the AI assistant / AI girlfriend spaces.

The fundamental issue is that AI gets exponentially better the more honest and forthcoming you are with it. Give it your trauma, your NSFW political opinions, and your sexual fantasies, and you’re going to have a companion that really gets you.

But when that little 9-person startup (who has no security whatsoever) who made that bot gets hacked, all that data you gave it will be there for sharing/selling. The problem is, this won’t stop people from doing it. The tech is too compelling. And people are too lonely.

Casio says a ransomware attack led to the theft of sensitive data, including personal information of employees and business partners. The attack, claimed by the Underground group, involved over 200GB of stolen data, but credit card info was reportedly not affected. MORE

MITRE has introduced the Caldera Bounty Hunter plugin, which allows users to simulate full cyber attack chains. This tool is designed to enhance cybersecurity training and testing by providing a more comprehensive emulation of potential threats. MORE

Horizon3.ai researchers detail how they identified new vulns in Palo Alto Networks' products to achieve full system compromise. MORE

The Internet Archive's "The Wayback Machine" was breached, exposing lots of user data in the 6GB SQL database of 31 million user records. The site’s still down but they’re working to get it back up. MORE 

Researchers from ESET have discovered two sophisticated toolsets used by a nation-state hacking group, possibly Russian, to breach air-gapped devices. MORE

Sponsor

Facing Alert Overload? Get the 2024 SOC Efficiency Report

Alert fatigue and analyst burnout are rising while traditional SOC tools fall behind. Sponsored by Dropzone AI, the 2024 Osterman Report, Making the SOC More Efficient, reveals how AI-driven innovations can enhance SOC performance, reduce false positives, and cut response times. Download the report for actionable insights.

Cybernews says Google's Pixel 9 Pro XL sends data packets to Google every 15 minutes, including location, email, and phone number, even with GPS off. They claim the phone uses nearby Wi-Fi to estimate location. MORE

The UNODC warns that Southeast Asian scammers are using deepfakes to enhance "pig butchering" scams. MORE

A Chinese hacking group, Salt Typhoon, has exploited back doors meant for lawful data requests, posing a major national security risk. Verizon, AT&T, and Lumen Technologies were among the affected companies. MORE

Ukraine has sentenced two hackers linked to Russia's FSB and the Armageddon group to 15 years in absentia for cyberattacks on state institutions. Armageddon, active since 2013, is a major state-sponsored threat actor targeting Ukraine and its allies. MORE

OpenAI has stopped over 20 foreign operations using its stuff to sway political opinions and meddle in elections. Attackers used ChatGPT to create fake articles and spearphishing campaigns. MORE

Private intelligence firms like Recorded Future and Flashpoint are changing intelligence by leveraging tons of data from the internet, including the dark web, to counter global threats. I love the dynamic of startups competing with corporations, and I love this analog of small intel shops competing with larger state actors (in some ways). MORE

Popular car brands like Hyundai, Kia, and Tesla are collecting driver data, including voice recognition and camera footage, and sharing it with third parties, according to a Choice investigation. The report found that 7 out of 10 car brands have concerning privacy policies, with Hyundai, Kia, and Tesla being the worst. MORE 

The Pentagon said the US will send a THAAD missile defense system to Israel (along with about 100 US troops to operate it) to improve Israel’s defenses against Iran. MORE

AI / TECH

If you use chatGPT, try this prompt just for fun (it’s going around some forums).

From all of our interactions together, what is one thing you can tell me about myself that I may not know about myself.

Then after it gives you an answer, ask it for another:

Awesome. Thank you. Can you tell me something else I may not know about myself?

Follow it up with:

Can you see any areas where I may hold myself back?

Let me know what you get back, and if you found it interesting. Honestly mine sounded very complimentary and little like a horoscope. Kind of felt like a scam in that way. Designed to make me feel good about myself, you know?

Well yeah! (puffing out chest) lol

I’m skeptical of outright flattery from strangers.

Curious if any of you get something that is actually revealing vs. just complimentary.

—

Apple's AI researchers found that large language models (LLMs) from Meta and OpenAI struggle with basic reasoning. They introduced a new benchmark, GSM-Symbolic, to measure this, which found that minor changes in query wording can lead to different answers. I find it interesting, but I’d say that it’s easy to disrupt its reasoning rather than that it has none—which is what a lot of the analysis is saying. MORE | THE PAPER

Geoffrey Hinton, often dubbed the godfather of AI, has won the Nobel Prize in physics for his early work on neural networks, alongside John Hopfield. Notably, Hinton is now firmly in the doomer camp, which is worth paying attention to. You can’t give someone a Nobel prize and then ignore other advice on the same topic. MORE

Elon Musk unveiled Tesla's new robotaxi, a self-driving electric vehicle without a steering wheel or pedals, at the "We, Robot" event. The design features butterfly doors and wireless charging, but it needs regulatory approval before production. MORE

💡There was so much hate against this event, and it’s revealed this love or hate binary thing with Elon. I don’t know many people who see Elon as complex. Nope. He’s super one thing or the other. He’s either the Saviour of the Universe, or he’s Tech Hitler. That’s it. Pick one.

I find this highly disappointing. People seem to have lost the ability to continue learning about someone once they’ve decided they hate or love them. People are allergic to subtlety. They want crisp, clear answers of Good or Evil.

This massively limits your ability to deal with the world because you’re going to be so wrong about so many things. Reality isn’t 1 or 0 like that. And the more subtlety you’re comfortable with, the better your probability adjustments can be.

My read, and my prediction, on this event, is that it was a lot of vision and hype, but that he definitely is working on the robotaxi. Will it come out when he says? Probably not. He’s been wrong about so many timelines.

But what he’s showing is that he’s excited, and moving forward, and that robots (Optimus) is a very real thing for him.

These events are about hope and about the existence of a man and a set of companies that continue to try to push for the impossible. Find me anyone like that—who can actually execute—and I guarantee you I can find a thousand horrifically dumb things they’ve said or believed.

It comes with the territory. If you have a genius creator, you’re unlikely to have someone who hits timelines perfectly and acts normally.

I think that most of Elon’s innovation critics suffer from a lack of reading enough biographies of great people. They often look a lot like Elon. Nuanced. Complex. Broken. Genius. And flawed.

And that’s the combination that leads to them being taught in school.

Dell's sales staff were given just two days' notice to return to the office full-time, causing panic among parents struggling to arrange childcare. The abrupt policy shift, aimed at boosting productivity, has led to crowded offices and left some employees considering using PTO to manage family commitments. MORE 

Billionaire Robinhood co-founder launches Aetherflux, a space-based solar power startup. Baiju Bhatt's new venture aims to create a constellation of satellites in low Earth orbit to collect and transmit solar energy using infrared lasers. Sounds rad, but it is technically a space laser. MORE

The US Department of Justice is considering breaking up Google after a court said they’ve crushed competition. The DOJ accuses Google of using products like Chrome and Android to maintain its search monopoly, leading to high ad prices and degraded services. MORE

Ticketmaster is the first to use Apple's upgraded Wallet tickets for iOS 18, giving us stuff like venue maps, parking, Apple Music playlists, and weather forecasts. Thank god. Anything to make Ticketmaster suck less. MORE

A new HBO documentary claims Canadian crypto expert Peter Todd is the mysterious inventor of Bitcoin, Satoshi Nakamoto. However, Todd dismisses the theory as "ludicrous," stating he was too busy with school and work at the time. Exactly what Satoshi would say… MORE

Four Taiwanese employees at Foxconn's Zhengzhou plant, the world's largest iPhone production facility, have been detained by Chinese authorities. The detentions, likely politically motivated, come amid rising tensions between China and Taiwan. MORE

HUMANS

It looks like Christopher Columbus was a Sephardic Jew from Western Europe. MORE

JPMorgan and Wells Fargo report a dip in profits. They said it was geopolitical tension. MORE

Your Brain Changes Based on What You Did Two Weeks Ago MORE 

The American Heart Association outlines a strict protocol for taking blood pressure, including sitting calmly with an empty bladder and using a bare arm, which is frequently ignored. MORE

Boeing is cutting 10% of its workforce—17,000 jobs—due to a tough year marked by grounded planes, legal issues, and strikes. MORE

Federal emergency workers in Rutherford County, NC, were temporarily moved after reports of an "armed militia" threatening government personnel. (see Ideas) MORE

Elizabeth Landau says single-cell cyanobacteria can anticipate seasonal changes by sensing day length and preparing for winter. This discovery suggests that seasonal tracking is fundamental to life, even in short-lived organisms. MORE

United Airlines is adding new routes to lesser-known destinations like Bilbao, Faro, Madeira, Sicily, and Nuuk, aiming to attract travelers tired of crowded hotspots. MORE

In his journals, Alexei Navalny, the Russian opposition leader, shares his journey from being poisoned with Novichok to his arrest upon returning to Russia. MORE 

Retail sales jobs have dropped from 7.5% to 5.7% of employment over the last decade, losing 850,000 positions despite the U.S. adding 19 million jobs overall. MORE

Likely due to weight loss drugs like Wegovy and Zepbound, the US adult obesity rate has dropped by about two percentage points from 2020 to 2023. MORE

New GLP-1 weight-loss drugs in pill form are in late-stage trials, potentially replacing weekly injections like Wegovy and Ozempic. MORE

Darya Kawa Mirza, a self-taught Kurdish astrophotographer, captured the moon's surface in stunning detail by stitching together 81,000 images into a 708-gigabyte composite. MORE

IDEAS

Gullibility, Not Disinformation
I don’t think the US has a misinformation problem. I think it has a gullibility problem. It’s not that we’re being fed too much crap. It’s that we’re eating it.

Some too-large number of Republicans now believe that Democrats are sending hurricanes to Florida because it’s election time. That’s a population problem. An education problem. Not a conspiracy theory problem.

In InfoSec terms, we need to reduce our vulnerability—not try to get remove the threats. The threats will always be there. And they’ll get better.

Our only chance of fixing this is education about how the world actually works—which both the far left and far right seem to have lost touch with. Remember, anti-vax was a far-left thing before it was far-right. Both sides have lost their minds.

MORE (2020)

DISCOVERY

swarm — OpenAI's new (experimental) framework for building and orchestrating multi-agent systems. MORE

Command Line Tools I Like (2022) — The author shares a list of favorite command line tools, many written in Rust, that enhance productivity with modern features. Highlights include neovim for its Lua scripting and LSP support, fzf for fuzzy searching, bat for syntax-highlighted file viewing, and exa for colorful directory listings. Other tools like rg, fd, delta, tldr, zoxide, and HTTPie offer improved functionality over traditional Unix commands. MORE

zvm — A better vim mode for zsh. LOVE this thing. Basically highlighting and all sorts of stuff including using the Surround plugin—all in vim mode. MORE

Theneo 3.0 — AI-powered API documentation tool that streamlines the creation and management of API docs. MORE

I updated my post on Dynamic Content Generation. I think this going to be insanely disruptive to so many industries. MORE

Augment UI — Use AI to prototype front-end designs. This tool helps designers quickly create and iterate on UI concepts using artificial intelligence. MORE

Software Engineer Pay Heatmap Across the US MORE

The Digits of Pi are Not Random MORE

Passbook — Lets you create an Apple Wallet pass from any QR code and export it to Wallet. MORE

How I Animate 3Blue1Brown — A behind-the-scenes look at how 3Blue1Brown creates its captivating math animations. MORE

RECOMMENDATION OF THE WEEK

If you want to calm your nerves during this next month and a half, go read about the civil rights movement and how much the country was divided then.

We’ve survived some really bad stuff. We probably will again.

APHORISM OF THE WEEK

❝

What is to give light must endure burning.

Victor Frankl