Smashing Security podcast #483: This AI helps thieves steal your iPhone

12 hours ago 6

JAMES BALL

At what stage was this surprise? You know, we locked 15 murderers in a room. You'll never believe what happened next. Sorry, you can tell I used to work at BuzzFeed.

Unknown

Smashing Security, episode 483.

JAMES BALL

This AI helps thieves steal your iPhone with Graham Cluley and special guest James Ball.

Unknown

Hello, hello, and welcome to Smashing Security episode 483. My name's Graham Cluley.

JAMES BALL

And I'm James Ball.

GRAHAM CLULEY

James, great to have you back on the show again. You've been keeping busy, out of mischief, I hope?

JAMES BALL

Horribly busy for an August, actually. I think when you're a freelancer, it's a much busier month than people realise, because when everyone else is on holiday, you're working.

GRAHAM CLULEY

Yeah, it's miserable, isn't it? Anyway, but at least we've had a little bit of rain, so that's good.

JAMES BALL

I'm hoping Finsbury Park will look less like, you know, the aftermath of a disaster movie and a bit more like a park, because it's usually lovely.

GRAHAM CLULEY

Yeah, less like the Kalahari, hopefully. Well, before we kick off, let's thank this week's wonderful sponsors, ThreatLocker, Intruder, and Vanta.

We'll be hearing more about them later on in the show. This week on Smashing Security.

We won't be talking about how the hacker who leaked footage of GTA 6 cashed out his CyberLeak cryptocurrency for about $270,000 just hours before the game's launch.

You'll hear no discussion of malware hidden inside a Chinese wallpaper app that encouraged users to disable their antivirus.

And we won't even mention how an exposed API key helped hackers steal 86 gigabytes of customer data from Manchester Airports Group.

So James, what are you going to be talking about this week?

JAMES BALL

Well, I'm looking to zoom us back a little bit from a whole bunch of the sort of cybersecurity rows going on around AI models and just look into a bit of what it's telling us and how significant it is, because I've had a couple of interesting conversations around that.

GRAHAM CLULEY

And I'm gonna be finding out how AI is helping to steal Apple iPhones. All this and much more coming up on this episode of Smashing Security. This episode is sponsored by Intruder.

Now, Joe, quick quiz. How often does your team ship code?

JOE

Multiple times a week, maybe more if someone's had too much coffee.

GRAHAM CLULEY

And how often do you get a proper pen test?

JOE

Oof, once a year, if we remember.

GRAHAM CLULEY

Well, that's the problem right there. Software moves weekly. Pen testing moves yearly. So most of what you ship never actually gets tested properly.

JOE

Which is exactly the gap Intruder's AI pen testing closes. You get the depth of a real manual pen test, but on demand, whenever you need it.

No scoping calls, no 6-week wait, and it costs a fraction of the traditional price.

GRAHAM CLULEY

It's built by Intruder's own certified pen testers, so the agents catch the complex stuff human testers can miss. And every finding is validated against your actual app.

Real issues, not noise. You get an audit-ready report within hours.

JOE

And it plugs straight into Intruder's full platform. Attack surface monitoring, cloud security, vulnerability management, all watching around the clock.

It flags what's exploitable, what to fix first, and how, so your team can act without waiting around for the security team.

GRAHAM CLULEY

Over 3,000 companies already trust Intruder with their attack surface.

JOE

You can kick off a pen test in minutes, and as a Smashing Security listener, get 25% off your first one.

GRAHAM CLULEY

So just head to intruder.io/smashing. That's intruder.io/smashing.

JOE

And thanks to Intruder for supporting the show.

GRAHAM CLULEY

Now, chums, we are just days away. I don't know how excited you are about this, James, but we are just days away from Apple announcing a new version of its iPhone.

It is widely expected that on Wednesday, September the 9th, Apple is going to announce not just the iPhone 18 Pro, presumably with about 17 cameras stuck on the back of it, but also the iPhone Ultra, the first foldable iPhone.

JAMES BALL

Yeah, I'm still not sold on foldable phones, but I am an absolute certified Apple fanboy.

I think I've got every bit of tech they've put out in the last decade except the VR headset.

So I am very much a mark for this, and if anyone is going to get me to buy a foldable phone, it's going to be Apple.

GRAHAM CLULEY

I mean, the thing with Apple is they're not always the first with a technology.

I think Samsung, and there's lots of Android phones which have been foldable for probably years by now, but Apple, sometimes its implementation can be better or bring something new, which makes everyone change in their wake, doesn't it?

JAMES BALL

They've had a good habit of waiting till a technology is actually ready so that it's not the finicky thing that only an early adopter could love, but everyone will just go, oh yeah, this is great.

And that does mean I do tend to sit and wait for them to do something, 'cause usually they've ironed out the kinks. There's been a few misses.

GRAHAM CLULEY

Hopefully there aren't too many kinks in their foldable iPhone.

But of course, it will mean that there will have never been a better time to have had your phone stolen, because if you're going to have it stolen, have it stolen just before Apple comes out with a brand new one, especially one which has something a little bit different about it, which you can get excited about.

You know, it's the silver lining on the cloud, isn't it?

JAMES BALL

It is. Have you ever had a phone stolen, Graham?

GRAHAM CLULEY

I've had a phone lost before. I've lost my iPhone in the back of a cab in Edinburgh in the past, and I was able to track it for a few days as the taxi driver—

GRAHAM CLULEY

I did eventually manage to get it back, which was quite miraculous. Thank you to that taxi driver. But it is a very stressful situation, isn't it?

It's very stressful because your whole life is running off one of these things.

JAMES BALL

Yeah, I got mugged for my phone once.

GRAHAM CLULEY

Oh my goodness.

JAMES BALL

There was a spate of muggings where people stopped stealing jewellery and stole phones hoping there were crypto wallets on them.

And clearly I had the look of someone who might have a crypto wallet on his phone because they didn't just steal the phone.

They kept beating me up to try and get the passwords and try and get into the banking apps.

And luckily for them, I keep my phone on such low battery that I gave them the passwords because they had a knife.

So they had the passwords, but the phone battery died and I'd managed to lock it.

I got home about 10 minutes after and locked it remotely before they could power it back up again, so they only got the phone.

But because I'd had to reset all the passwords and I didn't have the phone, I had about a month of admin work.

JAMES BALL

It was awful.

And there was a very stern woman on the bank call line who, when I said, well, no, I've had to give my security passwords out — oh, you're not supposed to give those out.

Yes, I know. As I told you, he had a knife on me when I gave it. You know, I wasn't just skipping through the streets saying, my banking secret word is X.

GRAHAM CLULEY

Awful. That is a truly horrendous situation. I wonder if some people will actually begin carrying around 2 phones with them, so if they do get stopped, you give them the rubbish one.

You give them the old Nokia brick. That's all I've got, I'm afraid, but you're welcome to it.

JAMES BALL

Honestly, the crime I was sort of okay with, but the endless — right, I ended up writing it up in the Sunday Times.

GRAHAM CLULEY

Oh, so you got something out of it at least.

JAMES BALL

So genuinely, I got about £650 for the Times article, which I worked out was my phone excess, the headphones, and worked out at being, for the time I'd spent, about half minimum wage on the calls.

So I sort of came out even-ish.

GRAHAM CLULEY

Well, it's a very stressful experience, clearly, both having your phone stolen and the aftermath as well. And I'll tell you something else which is a stressful experience.

Something stressful that happened in my life is, James, I want to tell you about a little game which my wife likes to play with me in bed.

JAMES BALL

I feel like we've got a new genre for this podcast now.

GRAHAM CLULEY

So I'm going to take you and our listeners a little insight under the covers, as it were. Let me give you the challenge, actually.

So the person who stole your iPhone from you, what word would you use to describe a person who steals something? They are a—

GRAHAM CLULEY

A thief. Yes, a thief. Now, my wife finds it very funny because of how I pronounce the word thief.

GRAHAM CLULEY

So as she often likes to point out, there's no TH at the end of thief.

And so I've approached this story with great trepidation because I know she will be listening and she'll be going, oh my God, you've done that on the podcast.

You said thief instead of — I don't know if I can do it now. Thief, right?

JAMES BALL

Does — is it for every — out of interest, you know, the leader of a tribe, a cheese.

GRAHAM CLULEY

And now I go, hang on.

JAMES BALL

Sorry, I feel like I'm just mocking a speech impediment here. That's completely fine.

GRAHAM CLULEY

A chief is easier. I think it's because of teeth, maybe.

I think that's why there are some words that if you do a regular podcast, you do find you get a lot of comments, not just from members of your family, but listeners as well.

Last week I couldn't pronounce subpoena properly. I've probably done it incorrectly again now.

JAMES BALL

No, that was right.

GRAHAM CLULEY

That's right. I've often had a problem with seizures, which apparently are—

GRAHAM CLULEY

Yes, yes. Like Julius Caesar, for instance.

JAMES BALL

Seizure salad.

GRAHAM CLULEY

Anyway, so that is a little game that she likes to give me. She likes to try and trick me into saying words in order to see that I'm still saying them incorrectly.

So I just want to say right up front that I may well say the word thief incorrectly during this section.

GRAHAM CLULEY

We've got that out of the way. Maybe I'll just say robber instead. So what happens when your iPhone actually gets stolen?

Is it normal that some kid just flogs it down the pub, or is there something more organised actually going on?

And the truth is that iPhone theft, despite the best efforts of Apple and the cops, it continues to be a big old problem.

And Boffin's threat intelligence firm, SOC Radar — and by the way, SOC Radar, what a magnificent name that is for a company.

JAMES BALL

It's kind of delightful, isn't it? Yeah.

GRAHAM CLULEY

SOC Radar. That's the sort of thing I need when I get out of bed in the morning. Something which just tells me.

JAMES BALL

At least you know what their freebies are gonna be, right?

GRAHAM CLULEY

So they've just shared their investigation into one iPhone crime outfit called AnonymousKit, and it is a criminal SaaS operation. So they are like a real company.

They promote themselves online. They offer customer support.

They've got a Telegram channel where they've got testimonials from happy customers, but their entire business is all about helping you make the most out of a stolen iPhone.

So if you are a robber of iPhones, you might turn to AnonymousKit.

GRAHAM CLULEY

And the problem they solve is that since 2013, so for over 10 years, Apple has had this feature built in called Activation Lock.

Activation Lock ties your iPhone to your Apple ID, so even if it's stolen or even if there's been a factory reset, if you tried to set up that phone as new, it will ask you — in fact, it will demand of you — that you log in before you can do anything, before you can change the settings.

And the old loophole with that used to be your passcode — so your 4-digit or 6-digit passcode.

So someone who stole your iPhone could reset your Apple ID password straight from Settings, and there was no old password required to do that.

They could turn off the Find My iPhone capability, reactivate your phone, go and sell it down the pub.

But these days, Apple's stolen device protection requires Face ID and Touch ID, not just the passcode.

Plus, there is a time delay built in, so if you're doing anything sensitive — like changing the Apple ID password when you're away from a familiar location — it'll make you wait a while, giving the true owner of the phone time to mark it as lost.

So that has meant that stolen iPhones on their own are as useful as a brick, as you discovered yourself, right? They need a password, they need something to get into your device.

And if they're—

JAMES BALL

Yes.

Although there's an interesting side market on this, before what you're going to come onto — so there's this sort of organised phone snatching, and they're often stolen and then buried in parks for a day or two.

Genuinely, literally just left in the soil in public parks for a day or two.

JAMES BALL

Which is to check if people are using Find My and are going to try and be vigilantes and sort of hunt it down, et cetera.

And then they're picked up about a day or two later en masse — people know which flowerbeds they've been left in, and they'll go and dig out 20 stolen phones.

And then they're cleared and they're shipped off to China where they are sort of jailbroken in some way and sold there.

So that's one mechanism, but as I understand it, it's not especially lucrative versus perhaps—

GRAHAM CLULEY

Perhaps the method I'm going to describe.

GRAHAM CLULEY

But it gives you an indication of just how valuable these things can be if they manage to get past all the security.

I mean, things like the iPhone Ultra — I think they're predicting it may cost as much as $1,900.

JAMES BALL

$2,000, I think. Yeah.

GRAHAM CLULEY

It's going to be an absolute fortune to have this bloody thing which you can bend. So enter AnonymousKit.

So this is a phishing as a service platform, so criminals pay a subscription to be part of the group.

They plug in details of a stolen phone and the platform does the rest — what it does is it tracks down the owners, it tricks them into handing over their passcode, their Apple ID — no flick knife required — and their 2FA code as well.

And the swines who stole the phone, they don't have to do anything. So it starts off simple enough, this particular attack.

So you've lost your phone, you get an email or text claiming to come from Apple saying, "Good news, we found your lost device," and there's a little map embedded in the HTML of the email saying this is its last known location.

So it looks really real to you and you think, okay, that's clever.

JAMES BALL

That is legitimately clever.

GRAHAM CLULEY

Yeah. But then it gets smarter because then you get a call from someone called Alice from Apple Support, and she introduces herself. She says her name is Alice Diaz.

Presumably she's got a brother called Buenas. And she says, I work for Apple Support, and that for quality assurance and security purposes, the call is being recorded.

So it all sounds legitimate.

GRAHAM CLULEY

It all sounds official. And she asks you to confirm that you are the owner of the phone, and she asks you to read out your passcode to verify your identity.

She says that someone brought their phone into an Apple Store, but she says, don't worry, a member of staff spotted it was in Lost Mode.

So someone came into the store, maybe with the phone saying, oh, I've got a problem, can't log in or whatever.

The person, the genius behind the desk has said, oh, it appears to be in Lost Mode. We're gonna retain this for security reasons. Maybe they suspected something.

And Alice says that they opened a recovery case as a result. So this is a service that Apple is giving you.

It's spotted that this phone has been stolen, spotted it didn't belong to the person who brought it in, and they want to verify who the true owner is.

And so they say, "We will send you, or you may already have received a text with a security link," and it's that link which the person is tricked into clicking into, which takes them to a web page which then asks for all of the information which is required to reset the phone.

So Apple ID password. The 6-digit 2-factor code. And there you go. You've handed it all over to the crooks.

JAMES BALL

And of course, typing in a 2-factor code feels about right, doesn't it? Yes.

GRAHAM CLULEY

We're doing it all the time.

JAMES BALL

Yeah. You sort of get told, we won't ask for it on the phone, et cetera, but typing it into a webpage, that's exactly what we do.

GRAHAM CLULEY

Now here's the thing. Alice Diaz, she's a smart cookie. She doesn't just speak English. She speaks Spanish and Portuguese as well.

But AnonymousKit doesn't employ huge swathes of people to make phone calls. They haven't got humans working for them at all.

Instead, subscribers to this service are renting an AI voice agent to trick you into handing over the information.

And when the experts at SOCRadar recovered some of the transcripts, they found it was pretty convincing.

So they could see where victims were reading out their numbers, and even if they paused midway through, the AI agent would actually come back to them and say, okay, yeah, I've got 1, 2, 5, what comes next?

And so you would think you were genuinely speaking to someone. And of course, voice AI these days is so much more convincing than it used to be.

Even if you're having an interactive conversation with a voice AI, there's not as much of a delay as there used to be. It does sound much more convincing.

JAMES BALL

Yes, it does. I thought I'd try and make that sound a bit robotic, but—

GRAHAM CLULEY

And the thing is, this is really cheap.

So the researchers said they saw evidence of hundreds of these Apple phone calls trying to phish the numbers and the passcodes from people trying to steal iPhones.

Each one of them was costing about 10 cents per call, and it's operating on scale as well.

So there are 168 different storefronts apparently out there using the same underlying code facility.

So it's been rebranded lots and lots of different places, lots of places criminals can go to, to actually affect something like this.

So you could be buying it from one person, but it's actually using the services of another criminal group as well.

JAMES BALL

And of course, once someone's in, they can often steal more than just the phone. Like if they've managed to unlock it, they can sometimes empty your bank accounts.

They can sort of move money with PayPal, they can buy things from online stores, your crypto wallet. Yes, if you've got a crypto wallet, you can really be in trouble.

So if it's 10 cents extra to attempt this, the economics of this are wildly in their favour.

GRAHAM CLULEY

I wonder what you have to dress like to suggest that you don't have a crypto wallet.

I'm just thinking of what you just said about having been stopped because these guys assumed you must be into cryptocurrency.

JAMES BALL

Yeah, yeah. I'd imagine either being a pensioner or maybe, you know, a woman who looks like she has a functioning social life. I think those two groups might be fairly safe.

They'll be fairly safe.

GRAHAM CLULEY

So this voice phishing traffic, 90% of it in this particular case, which SOC Radar uncovered, and by the way, they uncovered it because of sloppiness by the criminal gang.

They left some of their web server logs unprotected, and so they were able to see the transcripts of the conversations.

They were able to see some of the underlying infrastructure as well.

90% of the voice phishing traffic which they spotted was aimed at Brazil, but there have been other victims elsewhere in the world, including South Africa, Italy, India, and Kenya.

But I guess the overarching message which I have for listeners this week — so let's say this in a bold underlined font — is no legitimate Apple support engineer is ever going to call you up and ask you to read your passcode out loud down the phone or enter it onto a website.

If your phone is ever stolen and someone calls you to tell you it's been found but your identity needs verifying, be extremely cautious because it could just be a 10-cent attempt to try and make an awful lot more money out of you and all the information which is stored on your device.

JAMES BALL

And presumably, I want to check I know this right here. If you do get notified that your phone's been found, there's no reason that someone else would need to unlock it.

It can stay locked and on lost mode until it's back in your hands, and then you can take it out of lost mode then, right?

There's no time except when you've got your phone back with you that you would ever need to give this to someone. Is that right?

GRAHAM CLULEY

That is absolutely right. Now, there is always the danger, of course, because when your phone has been lost, you can send a message to it.

So if anyone picks it up, it'll say, you know, I am lost, please call Graham on this other number or contact me via this mechanism. Yeah.

There's always the danger that someone will say to you, come and meet me down this back alleyway and I'll hand the phone back to you.

And there they could have their flick knife or whatever.

GRAHAM CLULEY

In order to get all of your details. So, yeah, tread with caution, folks.

JOE

This week's episode is supported by Vanta.

GRAHAM CLULEY

Joe, what's your 2 AM security worry?

JOE

Honestly, whether I remembered to hit the record button.

GRAHAM CLULEY

What's your proper security worry? Like, do I have the right controls in place? Are my vendors secure? Nope.

JOE

I'm still worried we might not actually be recording.

GRAHAM CLULEY

Okay, look, how about the really scary one? How on earth do I dig myself out from under all of these ancient tools and manual processes?

JOE

Okay, fair enough. That does sound scary.

GRAHAM CLULEY

Well, enter Vanta. Vanta automates the manual misery so you can stop sweating over spreadsheets, chasing audit evidence, and filling in endless questionnaires.

JOE

That's right. Their trust management platform continuously monitors your systems, centralises your data, and uses AI to flag risks and keep you audit ready. All the time.

GRAHAM CLULEY

So whether you're chasing SOC 2, ISO 27001, GDPR, HIPAA, Vanta helps you move faster, scale confidently, and actually get back to sleep. So get started at vanta.com/smashing.

That's V-A-N-T-A.com/smashing. And listeners, you can get $1,000 off.

JOE

And thanks to Vanta for supporting the show.

GRAHAM CLULEY

Joe, you did hit record, didn't you?

GRAHAM CLULEY

Yeah, it was your job.

JOE

I thought it was you.

GRAHAM CLULEY

James, what have you got for us this week?

JAMES BALL

I've been tracking what feels like a never-ending saga of each of the big AI companies saying, oh no, we've realised we've accidentally hacked something.

And it's quite entertaining watching this sort of on Bluesky.

It started with OpenAI, I think maybe about a month ago now, saying that it had discovered that its agents had gone rogue, broken out of a sandbox, and hacked into Hugging Face.

And then sort of about a week or so later, Anthropic said that its AI models had broken out, but they'd broken into the systems of three different organisations.

GRAHAM CLULEY

They were trying to outdo OpenAI, weren't they? It felt like Anthropic's marketing department thought, oh, why didn't we think of that? What a great way to get us loads of publicity.

JAMES BALL

This genuinely started to feel like that because about a week later, Meta said, oh, by the way, we have an AI model and it hacked something too. Honest.

Sort of with mixed levels of convincingness, but to be fair, there look to have been actual incidents here with some quite sophisticated hacking.

And we're still seeing details come out on this. And let me stress that there are, of course, three separate hacking incidents here. They're involving agents.

And so spotting the reasoning is odd. We only have what different companies have released, and then it's filtered through different journalists' stories of this.

So this is my understanding at the moment, which may differ from other ones.

So I do apologise to any listeners if they think the details are off, but there's sort of one of the interesting elements that everyone's been talking about in the last week, which is that agents that were supposed to be sandboxed were collaborating with each other and communicating with each other.

JAMES BALL

Now, a lot of people seem to find that very exciting and very emergent, and it started off a whole new row of consciousness debates that I don't actually find very interesting.

The way reasoning models work is they actually — that scratchpad that they use to support their thinking acts as a scaffold.

It actually serves as part of their prompt and their engineering and encourages them to use different tools or access different agents.

And so it's built into the models, not just for a sort of audit trail, but for how they work, to constantly use scratchpads and chat.

And it's also built into them a lot to look for prompt and look for interaction.

And so it shouldn't be a surprise that if they find something where they can leave notes and where other agents are leaving notes, they communicate in that way.

That's essentially getting shocked that a tool is doing more or less what it's been designed to do. This isn't some evolution or emergent behaviour.

JAMES BALL

In the way some people are suggesting. What is kind of interesting is that they shouldn't have been able to be talking.

So the scratchpad that they were using was this thing called Artifactory, which is essentially a sort of bunch of tools, but they shouldn't have been able to access that.

And what seems to have happened is that the sandbox either had a flaw or was incorrectly configured.

JAMES BALL

The sandbox company is not clear who it is. You know, is this someone at OpenAI making a mistake? Is this something else?

But basically, first they could get into Artifactory and communicate with the other agents.

And then they found that they could use an exploit within Artifactory to browse the internet indirectly.

And so they had access to the open internet and thus could get information that the researchers had denied to them, that then suggested other routes out.

And then they managed to escalate their privileges in Artifactory and get admin control. And that's basically when it was spotted.

And it looks like the other two, the Meta hack and the Anthropic hack, were related and all used the same sandbox.

JAMES BALL

So it may have just been that there was a flaw with this sandbox.

I'm a little bit surprised by this because it's fairly basic that if you want to do anything like this, you just actually air gap and nothing else works.

It's sort of hammered into anyone who has been anywhere near a classified system or a secure system, that the only actually reliable thing, long before AI models, is an air gap.

You know, in a wireless world, air gap's actually kind of meaningless, as you know better than me.

But, you know, when we did Snowden 13 years ago, the rule that we had was if it's connected to the internet, it is not secure no matter what you have in the settings.

And so I'm completely baffled that they were relying on software safeguards for tools they had testing security.

GRAHAM CLULEY

My background is computer viruses, and so I've worked in computer virus labs or alongside virus labs for many years. And there would be a physical gap.

You know, not only were the networks not connected, there was no way electronically of getting from one to the other, but there would also be physical doors and locks as well.

Any disk which went into the virus lab, any floppy disk as it used to be way back when, would never come back out again. It would be destroyed.

GRAHAM CLULEY

And we even had different coloured cables for the different networks, so there's never a chance that someone would plug the wrong cable into the wrong computer.

JAMES BALL

It's almost like food hygiene places, like factories that do this.

People wear different colour aprons if you're working in the raw meat side of the factory or the cooked meat side of the factory.

And if you're not in the right colour, you literally cannot get in. You know, and that's for food safety.

GRAHAM CLULEY

Is the problem, James, that a lot of these AI companies have bubbled up fairly quickly or got into AI fairly recently and have grown at enormous pace?

And maybe they don't have that history.

They haven't built it into their psyche of how to do security properly, because the focus appears to always be, well, let's just see what we can do, and worry about cleaning it up afterwards.

In this particular case, we saw all the incredible headlines in the tabloid press, and not just tabloid to be honest, about some of this AI hacking other sites.

But what do you expect when you deliberately turn off all of the guardrails in order to test it inside a sandboxed environment, and then discover, well, the sandbox wasn't actually tight?

JAMES BALL

Well, this is absolutely the thing. And I do think almost all of the coverage I've seen of this has been terrible, because it's been everyone losing their minds about it.

And it's sort of, well, you designed the software to do this and it did it.

JAMES BALL

You know, good, but it was there to look for exploits. You gave it more exploits to look for than it should have.

JAMES BALL

Anyone I talked to from the security world — I think Kieran Martin's been quite interesting on this, sort of former head of National Cybersecurity Centre and various other things.

He's been essentially saying, well, yeah. Duh. Yes, exactly.

And when you start talking to people in the macro, there are actually reasons to think that this gives a defender's advantage.

You know, there'll be some very tricky transition stuff, but you talk to AI people and they're all, well, this is so interesting in this way and this way and this way.

You talk to security people, it's like, well, yeah, obviously this escaped. Of course this escaped. It was always going to escape.

You've got AI that is brilliant at finding exploits and zero days and escalation and doesn't get tired. And you've then given it a huge amount of compute.

JAMES BALL

At what stage was this a surprise? You know, we locked 15 murderers in a room — you'll never believe what happened next. Sorry, you can tell I used to work at BuzzFeed.

But it's like the world's least surprising breakout. You know, they broke out, gasp.

And the absolute lack of what would be regarded as very, very basic, very limited security measures that would go long before a regular data centre, a sort of run-of-the-mill Netflix customer data centre, let alone something before you had a, say, GCHQ data processing site.

Like, this is absolute rookie lack of precautions. And it's really surprised me that that element hasn't got into the coverage so much. And I find that quite interesting.

JAMES BALL

Because there's lots of obsession about lots of details on how the AI's operating, which are sort of academically interesting.

And I don't say any of this to be a sort of boring AI sceptic who goes, oh, it's spicy autocomplete. These are impressive models with impressive capabilities.

But we know that that's fine — this is a terrible way to test them, especially when it's security-focused tests.

And I do think this is a bit like trying to run a virology lab if you've never run one — if you've never run a high school chemistry lab — like they need to get some people in who can go, well, why the fuck was it connected to the internet, mate?

JAMES BALL

You know, I'm not a full-time cybersecurity bee. I am a general tech journalist and I know that, you know?

GRAHAM CLULEY

So why is it that the focus has so much been on, wow, these AIs are so clever — they're going to take over the world because of this, they're becoming sentient — rather than the AI companies are completely shambolic, how could they have screwed up so badly?

And why are all these AI companies so eager, it seems, to say, oh, our AI can do that too?

JAMES BALL

I think probably because AI is the sexy new thing and has the huge valuations and the IPOs, but it's been framed entirely as an AI story. And so people have gone to AI experts.

And if you are an AI expert and you know about how the models reason, et cetera, well, isn't it interesting that they spontaneously decided to communicate? No, it isn't.

Like, actually that was happening with, you know, what was the OpenCLAW and all of that?

It's not that, but you know, the hype train gets much more excited about talking about that because also you can then get on the — you know, is it becoming sentient? Is this AGI?

You know, is this the singularity?

JAMES BALL

And unfortunately, no one kind of goes, well, actually, we have experts in the actual mechanics of what's happening here. You know, we have cybersecurity experts.

Why don't we talk to some of them about it?

And I have seen a couple of cybersecurity journalists and a couple of cybersecurity commentators try and go, actually, I'm not sure this is as exciting as you think.

And I am worried about their lab practices or their research practices.

JAMES BALL

But I think because it's so through the AI lens, and that's the narrative lens everyone's putting on it, they're not talking to people who know the security stuff to go, actually, I'm not sure this is the most interesting question here.

The most interesting question here is, do they know how to research this stuff without starting some horrible— I mean, you know, I don't think they're about to activate Skynet.

I would worry that they're gonna set off something like WannaCry by accident and maybe do billions of pounds of damage or shut down NHS computers or something.

JAMES BALL

There's basics they're not doing here, or it seems that way to me. And I think those would be better questions to be asking them.

GRAHAM CLULEY

So maybe the headline shouldn't be so much, how has the AI become so clever? But rather, how have the humans become so dumb?

JAMES BALL

I think that's a very, very good way to frame it.

JOE

Yes. This episode of Smashing Security is supported by ThreatLocker. Agentic AI is beginning to change the tempo of cyberattacks.

GRAHAM CLULEY

That's right. We've seen research into autonomous ransomware, adaptive AI worms, and agents chaining tools without waiting for a human operator.

JOE

Which is all very interesting, just so long as it isn't your network they're experimenting on.

GRAHAM CLULEY

When enumeration, exploitation, and lateral movement happen at machine speed, relying on somebody to notice an alert and respond quickly begins to look rather optimistic.

Well, ThreatLocker puts default deny and least privilege between the agent and its next action.

So application allowlisting controls execution, ring-fencing restricts what trusted applications can access or launch, and privileged access management removes unnecessary elevation.

JOE

The attacker may be moving faster, but the controls are already in place. Agentic AI doesn't make established security principles obsolete.

It makes getting them right considerably more urgent.

GRAHAM CLULEY

So make sure that you are prepared for machine speed attacks with ThreatLocker. Visit threatlocker.com/smashing today to learn more and schedule your free demo.

JOE

That's threatlocker.com/smashing. And thanks to ThreatLocker for supporting the show.

GRAHAM CLULEY

And welcome back, and you join us at our favourite part of the show, the part of the show that we like to call Pick of the Week.

JAMES BALL

Pick of the Week. Pick of the Week.

GRAHAM CLULEY

Pick of the Week is the part of the show where everyone chooses something they like.

Could be a funny story, a book that they've read, a TV show, movie, a record, a podcast, a website, or an app. Whatever they wish. It doesn't have to be security related necessarily.

Well, my Pick of the Week this week is not security related. It is AI related, however. I don't know how we feel about that. Do we like AI? Do we hate it?

Is it turning our brains to mush? Is it stealing our jobs? Is it destroying the planet? The answer to all of those questions is yes, of course it is.

But one of the many concerns that people have is the risk that we will upload sensitive information to AI.

And obviously that will then get gobbled up into the AI hive mind, and who knows what will happen to it after that.

I don't really like the idea of that happening with sensitive information, which is why I prefer, if people are going to use AI — and there are legitimate reasons to use AI — I would prefer it if they're going to process sensitive information, that they use AI locally on their own computers rather than uploading it to some cloud server somewhere.

And my Pick of the Week is something which does just that. It is called Steno, which you can find at stenoai.co.

And this is a privacy-first tool that runs entirely on your own computer.

In my case, I've got a Mac Mini here running it, and it records, transcribes, and summarises my online meetings for me.

And what I like about it is that it's free, it's open source, doesn't upload anything to anybody, but at the end of a call, it'll make a transcript.

It does a neat little summary, so I still got the transcripts. I can see what was actually said. You even keep the actual audio file as well if you wish.

And you don't have an awkward bot joining your Teams call or your Google Meet call or anything like that. And you can even use your local AI to interrogate calls which you've had.

So if you say, I remember I was speaking to someone the other day about how I pronounce the word thief — hey, I really tried hard there — it would be able to tell me who I was discussing that with.

And so this is a tool I use on my Mac, but there is a Windows version on its way as well. Obviously it's going to put stenographers out of business, but I feel badly about that.

AI's going to put everyone out of business, frankly.

But I can't imagine I would've ever hired a stenographer to hide in a corner of my bedroom anyway to make notes as I have my calls. But anyway, stenoai.co.

It's free, it's open source, and it doesn't upload anything to the cloud. And in my experience, works really well. That is my Pick of the Week.

JAMES BALL

Sounds genuinely useful for my profession, that does.

JAMES BALL

Mine is much less useful, but I have a real long-running interest in how stuff works.

I always like if I can get behind the scenes and see a bit of manufacturing, or I spent 8 days on a container ship a couple of years ago, and rode from port to port and saw how all of that works, how they operate.

GRAHAM CLULEY

8 days, James.

JAMES BALL

Yeah, it was great.

GRAHAM CLULEY

Did you have mobile coverage? How did you cope? Because you like to be on your phone 56 hours a week.

JAMES BALL

Well, I had a lot of writing to do. I used it to finish my master's thesis. And we had more internet than I expected, actually.

JAMES BALL

Yeah, we were delivering cars up to Finland and paper back to Antwerp. And yeah, went through the Kiel Canal. So I love this logistics type stuff.

And the BBC have got a good show for people like me called Inside the Factory, and it's got a new season on.

I think it's on about season 10, and it's been cursed for most of its run by being mainly hosted by Gregg Wallace.

JAMES BALL

Yes, who I've not been able to stand for years, when he was a real creep to a colleague of mine when I was 21.

JAMES BALL

So, horrible man. And it's unfortunate, 'cause the other two hosts, Cherry Healy, and then there's a historian, Ruth Goodwin, who gives you bits of history, were always delightful.

And Greg Wallace is gone now. And so it's Paddy McGuinness who sort of, you know, hands up a little bit. But they go in quite interesting places.

You get to see how Quavers are made, or how lawnmowers are made, there's one on hardback books.

And as an author, it was really cool actually seeing how the printing works and how they sort of do all of that and how they make the beautiful covers.

And so there's a lovely little run of them on iPlayer. It's now 100% Greg Wallace free. And it's a very, very sort of charming educational bit of TV. So Inside the Factory.

GRAHAM CLULEY

Sounds like a great pick of the week. And that just about wraps up the show for this week. Thank you so much, James, for joining us.

I'm sure lots of our listeners would love to find out what you're up to and follow you online. What's the best way to do that?

JAMES BALL

I have a Substack and an email list at jamesrball.com. You can read my stuff in the New World magazine or the Eye, or I'm on Bluesky at jamesrball.com.

GRAHAM CLULEY

And of course, you can find me, Graham Cluley, on LinkedIn, Bluesky, Mastodon, Instagram, TikTok. The list goes on and on. And don't forget to ensure you never miss another episode.

Follow Smashing Security in your favourite podcast apps such as Apple Podcasts, Spotify, and Pocket Casts.

For episode show notes, sponsorship info, guest lists, and the entire back catalogue of over 480 episodes, check out smashingsecurity.com. Until next time, cheerio. Bye-bye.

GRAHAM CLULEY

You've been listening to Smashing Security with me, Graham Cluley.

And huge thanks, of course, to James Ball for joining us this week and to this episode's sponsors, ThreatLocker, Intruder, and Vanta. Be sure to check out their offerings.

We really appreciate those guys. And of course, talking to people we appreciate, we've got to talk about the patrons, right? Those people who've signed up for Smashing Security Plus.

Every week I will pick some of them out of the hat to have their names mocked, but also just to be thanked for supporting the show.

So kicking us off, Robert McCurdy, a name that can make milk coagulate. Benjamin Harouth, Stephen Castle with his lovely crenellations.

Big cheers to Jack Unverfirth, firm of grip if you ever need your pickle jar opened. And to the magnificently broody Dimitri, enormous thanks to you.

To Alexander Hooghuis, that is a surname so tall it requires its own oxygen tank, and to John Morris, Mark Norman, and the more shreds than marmalade, Mr. Bobby Hendrix.

And finally, Maya MacDonald, rounding things off in fine style. Those are just a few members of Smashing Security+, our Patreon group.

They all get episodes ad-free earlier than the general public, and they can have their names pulled out at random to be mocked.

As I said, if you'd like to join Smashing Security Plus, just head over to smashingsecurity.com/plus for all of the details. You can also support the show in other ways.

Of course, you can like and subscribe. You can leave a five-star review, and you can tell your friends about the podcast as well.

Go on, spread the word because every little bit helps, and it makes all the effort worthwhile. Well, until next week, where I hope you'll be tuning in again. Bye, bye-bye.

Read Entire Article