The first report was from China. It would have gone largely unnoticed.. but didn’t. The government probably leaked the clip only because it was about an American car going rogue. A smart car had picked up speed and gone on a rampage for 5.5 kms, annihilating everything and everyone on the road.
The footage was called
“Bone-chilling”, “Surprising” etc. by the world’s media outlets. But it didn’t
reach mainstream media, nor was it discussed as widely as it should have been.
Within a week, the incident was over in the world’s consciousness.
The next report came from Alaska.
This time, it was that a passenger could not get into her car in spite of using
the unlock passcode. The car had activated accident management protocol and
totalled the airbags. Anyone with a car knows that replacing the air bags is a
massively expensive thing. The lady made news, but only for 2-3 days. No one
was hurt.
The third incident was of the
Vietnam millionaire. His son’s car had crashed, but the airbags had NOT
deployed this time. Everyone inside the car was gone.
And those were just the ones that
got noticed.
February 2024
If Alisha was overawed, she was
not showing it. The Interpol Cyber Wing’s War room was lined with screens (what
else was she expecting?) and each screen had a head of national unit on it
right now.
There were 73 separate incidents
in the last 18 months – involving cars of a certain brand only.
She had written a paper, more on
a lark than anything else, in her college’s magazine, linking about 10 of these
crashes across countries.
That college magazine had been
read by Jeanie’s dad, who was with the Interpol.
She had received a call. The
caller introduced himself and asked her to explain her theory.
She used publicly available
information to make a quick case on the phone.
And a week later – this.
Next to her was Philip, the
genial head of the Cyber Unit, but the most feared cyber cop in the world. If
he was ruthless, there was no way of knowing that. But he had been known to use
every trick in the book to stop and punish everything from international
trafficking to international terror.
“A bit below your paygrade, don’t
you think? Car crashes?” She had made an effort to joke.
Philip smiled at her – the same
genial smile. “My dear, you had information on only 10 crashes. We now have 73
data points and are still not done compiling. It took a college student to
understand that the crashes are linked. What makes this my pay grade is not
what has already happened, but what might happen if we don’t stop it now. You’re
live in 5 minutes. Do you want to rehearse your opening?”
Philip always knew how to
communicate perfectly.
The Conference Begins
“Ladies and Gentlemen, thank you
for taking the time. You are all here because of this bright young lady –
Alisha. We now know that the hotshot luxury car company has been in at least 73
car crashes around the world in the last 18 months. I am sure that since the
meeting invite, some of you have found more data points in your own countries.
Yet, it was this college student who surmised that the crashes, though
unrelated in geography and time, were related in behaviour. Most of them had
one of 2 characteristics – the user has used the wrong opening code three
times, exactly 3 times, getting it right on the 4th effort, OR, the
user had disengaged automatic driving while cruising at more than 100 kmph. But
about 20 incidents are still outliers. We do not know what they had in common,
but it was something.
Alisha is the college student who
wrote that original paper. She is majoring in, no surprise, data analytics.
I would now like to invite her to
address us and share her thought process.”
Alisha spoke quietly and
confidently about how she started looking for patterns in data and went from
locations, time periods, make and model of car, colour of car, individual
feature present/missing in car, family size of user, and so on, until finally
hitting jackpot on user behaviour preceding the crash.
“When you think of it, its so
obvious! The crash was a response. So, the stimulus had to be there. What can
be more obvious than recent user behavior?” she smiled.
All the faces in all these large
screens nodded, taking assiduous notes.
“Since reading that paper, we
have done our own analysis, as you know.” Philip was back on the podium, “We
started by looking for incidents of unexplained crashes of cars with self-drive(auto-pilot)
feature. All of you helped immensely. We then removed incidents where the cause
was human and known. That left us with unexplained crashes. It took a lot of
legal wrangling to get a warrant for the central data of the car company, but
we finally managed it. When we analysed that data, we realised that all of these
cars were active on self-drive at the moment of crash. That is when we made the
connection between the self-drive feature and the crashes of the car. Alisha’s
paper had already told us to look for user behaviour immediately preceding the
crash. So, the long and short is, we know that the user did something, and
immediately afterwards, the self-drive activated, and then the car was made to
crash by the self-drive.”
What we also know, thanks to the
database from the company, is that this destructive behaviour was done by the
car every single time the trigger behaviour was done by the user. Which means
we know the causation is real.
We are all here today to answer
two questions:
A. What are the remaining 1-2
user behaviours that connect the remaining cases?
B. Who, or what, is responsible
for this? Is the car company sabotaging its own product? Or is it getting hacked?
Or does an active hacking organisation have a back door entry to the car
company’s systems?
Thank you.”
The Task Force
The Task Force had 10 country
heads of Interpol, Alisha, and Nishant. Nishant reported directly to Philip and
was widely considered the prodigal in the cyber sec unit.
The analytics tools had failed to
throw up anything that was common to the unexplained incidents.
But their bigger worry was
finding out who was behind this,
The Hunt Begins
Their work was neither glamorous
nor fun. It was hours and hours of staring at black blinking screens.
A whiteboard in the center of the
room listed all the variables they were testing against the common cause
hypothesis. So far, they had run through:
A. Registration plate numbers
B. First letter of registration
number
C. Names of owners
D. Where the car was before
malfunctioning
E. Whether drivers were left or
right handed
F. Music playing in the car
before the crash (the audio recorder records that)
G. Recording of the car dashcam before
the malfunction
H. Timing of the crash
I. Date of the crash
J. Month of the crash
K. Day of week of the crash
L. How many children the car
owners had..
.. You get the picture. It’s a
lot of fun when one is reading this in a detective novel. In that, one thing
leads to another and people come up with leads and inputs all the time. All
this team had was one frustration after another.
Until one day, Obja, the rep from
Egypt, came up with an idea that, like all great ideas, appears obvious post
facto:
“Look, boss, if the crash
happened in response to these stimuli, that has to be coded somewhere in the
car’s OS. Let’s run a simple test. Let’s repeat the stimuli in a car and see if
the behaviour is repeated? Then we know whether each car was individually
hacked or a malware injected into the OS?”
When the test was run, the car
crashed.
This was the team’s first
breakthrough. They now knew that they were looking for a malicious script in
the OS.
The hackers were smart. No one
was sitting around hacking cars. They had injected a piece of malware and were
now sitting and watching the show, so to speak.
The Elusive Code
If you haven’t already seen it, a
car’s code is a few million lines of code. Some of it is in assembly language
still.
The malicious script was a simple
If-Then command. This means that no AI was involved. If user does this, you do
this. The script could be absolutely anywhere – in any part of the OS.
The forensics team was enhanced
and the coffee machine lines got longer. It took them two whole weeks (for
scale, consider that every forensic engineer goes through a few thousand lines
of code per day using automated tools, and there were 15 of them working almost
non-stop) before they found the plug.
The plug was simple. It
instructed the car to speed at t-20 (20 kmph less than the top speed possible
for the vehicle) on loop. There was no termination line. Which means the car
was instructed to get to the top speed and then remain there for the rest of
its life.
When they got the full code out,
they smiled.
The three conditions that
triggered this script were all based on user behavior.
The three conditions were:
A. Where a user enters the wrong
passcode three times but gets it right on the fourth attempt.
B. If the user disengages self-drive
while cruising at a speed of 100kmph or above
C. Where the VR system of the car
hears the launch phrase “AI is crap.”
In spite of themselves, they all
laughed. So, this was the elusive “third condition” that their whiteboard had
been unable to get!
It was time to augment the team.
The Team
Suji was a cyber behavioral
specialist. His job was to look at the code and figure out what kind of group
or person was behind this sophisticated script.
The script was genius in its
simplicity. The three conditions were such that they would cause a few
accidents, but not enough to get widespread attention. And the best part was
that no one would think of linking these accidents to each other. The designer
of this script – person or group – had to have a very distinct personality.
Nathan was a grey hatter. His job
was to work out of his own house and to look for the kind of person or group
indicated by Suji. They were definitely
a new group, because no one had heard of this modus operandi before.
Nitesh and Alisha were to work
together on the toughest problem of all – the motivation.
What did the writers of the
script want? Why were they doing this?
Obja was the cyber forensic
expert whose job was to go through the server logs of the car company to
understand exactly when this script had been injected into the system. How long
before the first crash in 2020, was this done?
In theory, Obja’s job was
easiest. In practice, it was impossible.
The international organisations
had taken more than a year to put the pieces together. Server logs were
retained for 30 days on the drive and for 6 months in the backup drive. Which
means that the server logs were not going to show anything.
Obja still ran through them,
looking for indication of a modification to the script or something. Anything.
He got nothing.
Then, he moved to the code
backup. Every tech product has a back up of its code. This is so that, in case
of an issue after a tech upgrade, the customer’s code can be taken back to a
point at which it worked. This is called the restore point.
Being a luxury car company, the
offline backup of code was kept for 9 months. Code before that was not
available. The car company had been convinced to co-operate by Philip, who was
always very persuasive in such matters.
Obja dutifully looked through
this too. Nothing. Even the last restore point in the OS had this malicious
script. What was significant was that no change had been made to the script.
Which means whoever did the injection did it one time. They must have run a
test. And they never needed to come back to this script. From that point, the
show was on.
Suji was doing slightly better.
He now had a profile. The script was very simple. Which means the person
injecting it:
A. Had to know exactly where to
put it
B. Knew what to do so it doesn’t
come up in an audit or review at any time
C. Had access to the server to
make the injection.
So far, he was going with the
theory of lone wolf. The actor’s modus operandi prioritised stealth. Such a
person was not likely to use or even belong to a group. In fact, it was very
likely that s/he was a disgruntled engineer on the team. Event logs for the
event had not been disabled, meaning the person was not a hacker by habit.
Suji’s heart sank. This meant
that Nathan’s fishing may not be any use at all.
The next logical step would be to
check the backgrounds and actions of the thousands of engineers who had worked
on this car. This car was one of the first connected cars to enter the market.
It started slow – with just sending data about speed, location, use of systems
back to the central server.
Then, the cruise control was
added. That was their first foray into AI. Finally, in 2020, the autopilot
feature was launched. This allowed the user to sit back while the advanced
sensors did everything. It worked in all conditions except the most densely
populated areas in a few geographies. In the first world, the autopilot feature
was a dream come true.
The Breakthrough
It was so unexpected, it was
hilarious.
Alisha had this idea that she
wanted to hear all the voice recordings of the time before the first crash. She
wanted to understand why the hacker chose that particular catch phrase in his
script. The idea was wild – suppose a certain user used this catchphrase
regularly enough for the hacker to be sure that sooner or later, it would be
used. Suppose the entire death factory was to mask that one murder that the
hacker really wanted?
As motives go, this was as good
as any (considering they had no other motives on the table).
They started listening.
Nishant also started looking at
data points of the incidence of the other two user behaviours – forgetting the
password exactly thrice, and disengaging cruise control (the precursor to auto
pilot) at 100 kmph and above.
He found something curious. In
their category – these two were the least displayed behaviours. For example, if
100 people entered their passcode incorrectly, 70 of them would remember the
right passcode after 2 attempts – at the third attempt. 3 would put incorrect
passcode all 5 times. 10 would get it right in the fifth attempt. Only 1 user
was likely to get it right the fourth time. Only 1% of the users who forgot
their passcode were likely to remember it on the fourth attempt.
Likewise, cruise control was
disengaged at various speeds by users, but above 100 kmph was the least used
speed category.
So, the hacker wanted to minimize
the car crashes, but s/he still wanted them. Why? It made no sense.
Alisha’s work was not that easy.
The car company used to store the
voice commands on magnetic tapes that were stored at some cheap warehouse in
Arizona. She physically flew to the location with Manu, another team member.
And the room reminded her of a government office back room in any part of the
world. It was not dusty, but in every other respect, it was a govt office.
Stack upon stack of magnetic tape. Some stacks were labelled, most were just
dumped.
“What is this place?” Alisha
asked.
“The graveyard of code. This is
the graveyard of code. That way, there, you have the original OS of the car –
going back to the 1990s, when we first moved luxury car dashboards to
electronic display. This work was done by an Indian company for us then. We put
a screen to show stuff like speed, temperature etc. and the buyers went wild.”
Alisha’s eyes widened in
disbelief, “So, here you have the earliest version of code, going as far back
as the 1990s?”
“And all the voice commands ever
heard by our VR system since it was launched by us in 2016. Which is what you
are here to listen to.”
“Actually, what I am here for is
the frequency chart of a specific phrase and where that stands compared to the
most used phrases at the time. The time period we are looking at is 2018 – 2020
March or so.”
“I can give you that from 2019,
because that is when we put analytics on top of our VR. But before that is
nothing. Does that work?”
“That’d be a great start, yes.
Thank you!”
Manu retrieved the files and
loaded them on a machine in the records room. The dataset needed a specific
software which was only available on the company’s own machines.
They reached the same conclusion
as Nishit. “AI is crap” was one of the 5 least used phrases inside the car.
But Alisha had one more idea.
“This graveyard of code.. are the
graves marked? By year?”
“Nah. We might have some sort of
marking by version on some of the tapes, but I wouldn’t know which version came
in which year.”
“Ok, from which version do you
have this information?”
“Let me see… OS version
control….. hmm… wait…”
He pulled out a tape and started
working. Very soon, he said – this one, 12.0.1.345.4 – this was released on
February 12th, 2018. The next version we released was 12.0.1.346.0 –
and that was in October 2018.
So, that’s what we have. You are
welcome to the tapes here. Some of them have a number on top. Most of them
don’t. I have to be here while you work. So just go on there, pick up a tape
and bring it to me. Don’t try any hanky panky. All these files only open on our
proprietary software, so taking one away will not help you at all and will make
me very angry.”
Alisha smiled, “You do realise,
yes, that we are the Interpol?”
The man smiled back. It was
ceasefire time.
3 days later, Alisha and Manu had
put in a formal request for code of a certain version. They had done the
impossible! They had found the version in which the code appeared for the first
time. Just as the team had expected, the code was so simple it was pure genius.
It had needed zero modification since the first injection.
Now, they had to find out the
time range during which that OS version was in production.
The release log was not likely to
go that far back. 6 years is a long time.
The Dead End
The team was together after a
long time.
Nishant was the leader.
“Let’s sum up what we have so
far. We know that the accidents are caused by a malicious script in the OS of
the car.
We have a rough idea of the time
during which it could have been injected. We could be off by as much as 5-6 months.
We know that the person who wrote
this code had access to the analytics of the car company even before the
analytics layer was added. Which means that they had access to the raw data
which they could then put on a basic voice recognition engine and do some
private analysis.
In 2018, it was still possible
for some employees to put some private software on company laptops.
This was one such employee.
Also note that the script does
not generate any notifications. Which means that the hacker either did not care
to know when a crash happened, or could get to know without the need for a
notification. This can only mean that he or she is still on the team. It is one
of the people we have been meeting or interacting with.”
“Did we go over the list of
people who died? Did any of them have any connection with an engineer working
in this company? Family? Friends? Business feuds? School rivalry? You married
my girl how dare you? Or anything at all? Even neighbours?!”
“Nope. Nada. And believe me, we
LOOKED. Hard.”
“Since we removed the script 6
months ago, we know that the hacker, whoever he is, is not waiting around for
any more action. Now we have a sea of suspects, a little bit about the modus
operandi, but still no motive!” Suji concluded for everyone.
The Breakthrough – II
For some reason, Alisha kept
going back to the original code. “Why did he choose user behaviour for his
script? He could have chosen anything. But he chose a trigger by which the
driver would seal their own death warrant. And yet, he chose the behaviour
least likely to appear.
He wanted people to trigger their
own death, yet he did not want too many people to die.
Death was not the objective here.
Exposing the vulnerability of the car was. Exposing just how vulnerable the car
was – THAT was what this person wanted to do.”
Alisha scrambled to Nishant’s
office.
Nishant heard her out and gasped.
There was someone on the team who was desperately trying to tell the car
company that their cars had too much power under AI. That the very same AI
could be hacked to kill people.
But the company pushed ahead with
its AI development.
Who was that person?
The old timers were brought in.
In particular, people who had left the company in 2021 or thereabouts were
called in. Did they remember an engineer or project manager warning about the
need for safeguards in AI deployment? And he was ignored?
Two names popped up – Chris and
Sasha. Chris remained with the company, while Sasha had resigned and now worked
with children. They had married in 2019 and now lived close to the engineering
office. Chris was still part of the AI development team. He had been a
developer in 2018 and had slowly risen through the ranks.
When questioned, he confessed
readily enough.
“Yes, I wrote that script. I just
never expected it to go on for so long. I thought that with the first car crash
in China, they will be forced to sit up and do a code review. They did nothing.
Before injecting the script, for
6 months, I kept pleading with them to put a human override in the AI autopilot
feature being developed. I begged with them to have basic security protocol in
place for the AI engine that we were using in self-drive. You know what they
did? They used that budget to start recording what people were saying in their
cars! It was disgusting and voyeuristic.
I told them that with AI, we were
building systems that were, in turn, hackable. But because these were smart
engines, tracking a hack would be next to impossible. In most codes, we do not
check the code directly. They wouldn’t listen!
A prophet is not honoured in his
own country. I was ignored just because I was an engineer on their own team. If
I was one of those hot shot external consults, they would have paid attention.
Honest to God, I never thought it
would take them this long. I am sorry. For everything. But trust me, for the
100 odd people who have died because of me, thousands have been saved because
you found that script and removed it. If this is able to put some kind of
standards around how AI is secured in large scale implementations, I am happy
to spend the rest of my life in jail. Sasha and I have been expecting this.
That’s why we don’t have kids.”
The End
To be honest, Nishant did not
know whether he wanted to charge Chris or the CEO of the car company. The CEO
was going to ignore the next security warning too. Chris, on the other hand,
was just trying to scream his way to attention. Even that failed. And how.
It was a weary team that
congratulated itself that night. Weary, but oh, how victorious!
