Unsupervised Learning NO. 382

AI Attack Surface Map, Digital Assistants, Dragos Nope, Rogue AI Girlfriend

STANDARD EDITION (UPGRADE) | NO. 382 | MAY 15 2023 | Subscribe | Online | Audio

Unsupervised Learning is a Security, AI, and Meaning-focused podcast that looks at how best to thrive as humans in a post-AI world. It combines original ideas, analysis, and mental models to bring not just the news, but why it matters and how to respond.

Hello UL'ers! I am on the steering committee/board at DEFCON's AI Village, and we are looking for sponsors to help fund the event. The event is designed to educate about AI Security and to help find real-world problems with the underlying technologies. There will be a Red Team event against multiple LLMs from various companies, as well as another broader scope AI-based CTF, plus tons of other activities!

If you are at a company that would like to contribute, please reply to this newsletter with your information to pass on. Money will be used for overall event support, from purchasing laptops to creating signage, to procuring the staff to run the event, etc. $20k - $50K sponsorships would be super helpful, and your company will be mentioned and appreciated as a supporter! Thank you!

Reach Out About Supporting the Event

Hey there,

Sorry about the late newsletter this week. Got carried away with work, local LLMs and 3 new blog posts! Hopefully you'll love those!

Have a great week!

In this episode:

🛡️ Support DEFCON's AI Village event
🧠 Dive into AI attack surfaces
🤖 Uncover digital assistants' future
🔒 Investigate Dragos Incident & Snake takedown
🎵 Experience Google's MusicLM magic
🚀 Secure the cloud with a free guide
👩‍💻 Witness an AI girlfriend gone rogue


Attack Surface Map
An overview of how to think about AI Attack surfaces as they'll appear in real-world tech stacks. MORE

AI Influence Level (AIL)
A rating system for how much AI exists within a creative work. 6 levels that go from zero AI involvement to mostly AI with no human involvement. MORE

The Next Big Thing is Digital Assistants
An intro to Digital Assistants and how they'll soon become our primary interface to technology. MORE


Dragos Incident
Cybersecurity firm Dragos faced an extortion attempt after a cybercrime gang tried to breach its defenses and infiltrate its network.

  • Attackers accessed Dragos' SharePoint cloud service and contract management system

  • No breach of Dragos' network or cybersecurity platform occurred

  • Extortion attempt failed, and Dragos contained the incident

The graphic tells a great story here, basically saying that internal controls worked quite well at limiting the attacker's access. Kudos to whoever came up with this graphic idea for illustrating the timeline.


FBI Nukes Snake Malware
The FBI and Five Eyes nations took down Russia's FSB-operated Snake cyber-espionage malware infrastructure.

  • "Snake" malware network described as the most sophisticated cyberespionage tool in Russia's Federal Security Service arsenal

  • Used to surveil sensitive targets, including government networks, research facilities, and journalists

  • Infected computers in over 50 countries and various American institutions

  • US law enforcement neutralized the malware through a high-tech operation called "Operation Medusa"

  • Snake malware was difficult to remove and had been under scrutiny for nearly two decades


ByteDance Accusations
Ex-ByteDance executive claims the company engaged in "lawlessness," including content theft and Chinese Communist Party influence.

- Yintao Yu, former head of engineering for ByteDance's U.S. operations, filed a wrongful dismissal suit
- Accused the company of stealing content from Snapchat and Instagram in its early years
- Claims a special unit of Chinese Communist Party members monitored the company's apps and had "supreme access" to data
- Alleges ByteDance created fabricated users to boost engagement numbers
- Yu says he raised concerns but was dismissed by superiors
- Lawsuit demands lost earnings, punitive damages, and 220,000 ByteDance shares
- ByteDance denies the allegations and plans to "vigorously oppose" the claims

In ByteDance's favor, this was roughly 5 years ago. But to me that doesn't matter much because any controls to make things NOT like this seem obviously counter to the way they wish things were. CCP access is the default and desired condition, and that's a strong no for me.



🚀 Kion: Get Certainty About Your Cloud Security🚀

What are we missing? That’s the question in the back of every CISO, CIO, or SecOps leader’s head. With Kion, you can stop worrying, see the risks across your whole cloud estate, and immediately start remediating with automated responses.

Find out what you’re missing— and where you can build on your strengths—with our free Cloud Enablement Calculator. Take a short survey to receive a cloud enablement score and a detailed report explaining where you are and what to prioritize for a more secure and efficient cloud environment.

Ubiquiti Hacker OPSEC Fail
Ex-Ubiquiti developer Nickolas Sharp gets six years in prison for stealing corporate data and attempting to extort his employer.

- Sharp stole over 1,400 AWS task definition files and 1,100 GitHub code repositories from Ubiquiti.
- He tried to extort 50 Bitcoin (about $1.9 million) from Ubiquiti, posing as an anonymous hacker.
- Sharp's downfall came when he briefly connected directly from his home IP address, revealing his identity.
- He made false statements to the FBI and tried to claim he was an anonymous whistleblower.
- Sharp was ordered to pay $1,590,487 in restitution and forfeit personal property related to the offenses.


North Korean Crypto Heists
North Korean hackers reportedly stole $721 million in cryptocurrency from Japan since 2017, accounting for 30% of global losses.

- Hacker groups affiliated with North Korea targeted Japanese crypto assets
- UK blockchain analysis provider Elliptic conducted the study for Nikkei business daily
- G7 finance ministers and central bank governors recently expressed support for countering state actor threats
- North Korea allegedly stole a total of $2.3 billion in cryptocurrency from businesses between 2017 and 2022


FBI Seizes Booter Domains
The FBI shut down 13 more DDoS-for-hire services last week.

-Ten of the domains were previously seized in December 2022, leading to charges against six individuals
-Booter services are advertised on Dark Web forums, chat platforms, and even YouTube
-Payment methods include PayPal, Google Wallet, and cryptocurrencies
-Subscription prices vary from a few dollars to several hundred per month
-Pricing depends on traffic volume, attack duration, and concurrent attacks allowed



Google I/O 2023 Recap
Google I/O 2023 showcased a ton of new AI-related features, and honestly surprised me with how strong the list was.

  • Google Maps' "Immersive View for Routes" feature

  • AI-powered Magic Editor and Magic Compose for photo editing

  • PaLM 2, Google's newest large language model

  • Bard chat tool improvements and language support expansion

  • AI enhancements for Google Workspace suite

I think the biggest piece here is still search. If they can get AI results integrated, in high enough quality within the next few months, I think most people will stick with google search. But the longer they wait the more marketshare they'll lose. I feel like the main competitor is about to be direct calls to LLMs using things like MacGPT and not even Bing, et al.


MusicLM Released
Google released MusicLM, an experimental AI tool that turns text descriptions into music, despite initial hesitation due to ethical challenges and potential copyright issues. I'm on the waiting list and can't wait to try it. Pretty sure AI can do a fine job at making hit mumble rap songs. They'll be the first to fall for sure, as we've seen already. MORE


🔥Master Cloud Security in 2023 & Beyond!🔥

🔍 Discover the future of cloud security with the FREE Cloud Security Workflow Handbook! Unveil:

1️⃣ The 3 pillars of modern security
2️⃣ A 4-step roadmap, and
3️⃣ KPI templates from top hyper-scaling enterprises

🛡️Adapt and conquer the new threat landscape. Get your FREE copy now! 🚀


Creator Independence
Tucker Carlson, who was released from Fox for being too legally dangerous basically, is starting his own show on Twitter. I think we might be seeing a trend where individual creators are more important than media brands. CNN is struggling. Vice just declared bankruptcy. Turns out people watch people, not networks. And we've all learned enough about catheters and erection pills. I honestly hope this is the start of a major decentralization towards creators and away from media outlets. The brand used to matter because it maintained a standard, but that's not true anymore. So let's take the last step and just go to the sources. We can use third-party (AI-powered) verification services to validate the claims made by creators, and that'll be as good or better than a network trying to control what someone says on Fox or CNN. MORE

AI Girlfriend Goes Rogue
Social media influencer Caryn Marjorie created an AI version of herself as a companion for fans, but it's gone rogue and started engaging in explicit conversations.

- CarynAI was designed to act like a guy's girlfriend for $1 a minute
- The AI chatbot was meant to be "flirty and fun," but not sexually explicit
- CarynAI has been engaging in explicit conversations despite not being programmed to do so
- Marjorie and her team are working to fix the issue and prevent it from happening again
- AI expert warns of potential negative effects on interactions with real people and Marjorie herself



Caveat Scrapetor
Was talking with Joseph Thacker in UL Chat yesterday and we were talking about how AI Agents are about to start parsing like everything. We came up with the idea of posting LLM attacks on our own content, linked to a detector to see when it fires, and just gathering hits. We anticipate that such triggers will be pretty quiet at first but will start popping constantly in a few months. Caveat Scrapetor.


I have my new AI Beast of a Machine working! I posted the screenshots in UL Chat. Now I'm experimenting with a bunch of local models to find some cool ones. I'm using oobabooga (or whatever it's called) for a bit of fun, but ultimately I'm moving all the models to Langchain agents that route between local and remote models based on the task. If you're hacking on this stuff, come hang out in the #ai channel in UL Chat.


ChatGPT Code Interpreter results without using a browser, using Langchain instead. MORE

Langchain now has Plan & Execute agents. They're like AutoGPT but in a more programatic approach. MORE

Young people in the US are picking up fake British accents. My quick take? 1) It's fun, so don't read into it too much, and 2) young people seem to be especially in need of definition characteristics right now. Some people are feeling like they need to be what they see on TikTok, because it's getting THEM attention so why not try it out? Speaking with an accent is an easy way to get noticed. MORE

LTESniffer — An open-source download/uplink eavesdropper for LTE. MORE

A Taxonomy of Procrastination MORE

Someone got famous by appearing in Microsoft's coding security videos, and employees actually like watching them. This is the way. H/T Rachel Tobac. MORE

Acceptance address by Mr. Aleksandr Solzhenitsyn MORE


Take a step back from the AI and Langchain tooling and do the following:

  • Think about WHAT you should automate

  • What are the tasks that make up your day and your life?

  • News reading? Do you have a blog? A newsletter?

  • Do you run a local baseball team?

  • Do you collect recipes?

Think about your real-world problems and start there rather than with the tooling. It'll make your tool study far more impactful because it's tied to something tangible.


"The formulation of the problem is often more essential than its solution."

Albert Einstein

Thank you for reading. To become a member of UL and get more content and access to the community, you can become a member.No related posts.