Assistente AI
Trascrizione
00:00:510Tommaso Bianchi: What is going on, what's expressed.
00:05:750Tommaso Bianchi: I'm sorry. Okay, okay, there was the email. Yeah, this is the correct email.
00:12:810Tommaso Bianchi: So actually, I need to thank the old teaching Assistant Dennis. Maybe you know him. I don't know he's now like just finished the Phd.
00:22:110Tommaso Bianchi: So what we will see is like, how can we reverse something on the canvas? So you already saw what is the canvas? And
00:31:930Tommaso Bianchi: maybe you saw also, like the frame structure for the canvas.
00:35:990Tommaso Bianchi: And nowadays, actually, we find the same network internal network in cars. So every card
00:44:288Tommaso Bianchi: as different electronic parks parts all linked together with the canvas.
00:51:234Tommaso Bianchi: It is something that actually has no security overall, because there's just very few amounts of bits of data that you can send. And let's say, is very old technology. But it's very robust. So is also very safe to use in cars in the sense
01:12:744Tommaso Bianchi: yeah. So this is the conframe. If you want to check like whatever is written over the in the Pdf, you should have it on model. Yes.
01:26:699Tommaso Bianchi: stuff.
01:28:900Tommaso Bianchi: So whatever is happening
01:31:610Tommaso Bianchi: for the, for sending the messages so like the start of frame and some other like the end of frame, is not actually something that we will see here is not like we don't care about them is just about synchronization on the cam. And
01:45:760Tommaso Bianchi: let's say that what we want to see
01:48:770Tommaso Bianchi: is the can Id, that is pretty important.
01:52:00Tommaso Bianchi: And the data that is actually what what we need.
01:56:330Tommaso Bianchi: So, as you may know, Kennedy, is what defines like the the priority on the count. So whatever is lower as higher priority.
02:05:236Tommaso Bianchi: So, for example, if I send some messages with 0 continuously. I would just like, do a denial service on the account.
02:14:250Tommaso Bianchi: because every packet will like check every see you on the count. So the electronic component, before sending some messages, will check
02:24:710Tommaso Bianchi: weather, has the possibility to send it.
02:27:00Tommaso Bianchi: And check. What are other ids on the can
02:31:520Tommaso Bianchi: are trying to send a message.
02:34:770Tommaso Bianchi: So obviously.
02:36:460Tommaso Bianchi: if lower ideas, priorities 0 will always like get the overall priority on the account and get the like the possibility to send a message.
02:46:770Tommaso Bianchi: So to start first, st we need a virtual lab, so everyone can try to do something on on your laptop.
02:54:810Tommaso Bianchi: So if you have Linux good, if you don't have it, get it
03:01:91Tommaso Bianchi: we need. Can you choose and some other libraries that we will just download directly using this Icc, the simulator for the canvas. So there's this link.
03:10:670Tommaso Bianchi: But you will just write. I see SIM on Google, and you should find the the correct zombie Craig.
03:18:320Tommaso Bianchi: Link, if you want to do it now. Otherwise I would just
03:22:780Tommaso Bianchi: go straight with everything. Okay? Yeah, yeah. Yeah. You can have a virtual machine.
03:33:790Tommaso Bianchi: Okay, so what we need to do is following, like the instructions to to set up the the simulator. So it will say, Okay, download this library, these packages and libraries in order to use it. Obviously.
03:46:370Tommaso Bianchi: you need to install it.
03:49:440Tommaso Bianchi: okay, this is the clone for the repository.
03:52:520Tommaso Bianchi: And this is actually, we need to set up the virtual account environment. So in your Linux distribution, you can just set up account. Obviously, is virtual account. So
04:02:370Tommaso Bianchi: in different terminals, you can, for example, one, you will use it for reading data and one for sending data using the communities that you downloaded before.
04:11:770Tommaso Bianchi: And there's okay. These setup become sh bash script in order to do it like
04:20:480Tommaso Bianchi: automatically. If you don't want to
04:22:540Tommaso Bianchi: put every command on your own.
04:26:100Tommaso Bianchi: and if you try to check your network interfaces, then you will find that you have
04:32:520Tommaso Bianchi: this beacon interface. Now in your laptop. So okay, nothing to see just is up and running boom
04:41:760Tommaso Bianchi: in this.
04:43:151Tommaso Bianchi: Is everyone familiar with the the concept of virtual network? Or is it something that you've never seen before.
04:52:610Tommaso Bianchi: So basically, what we need to do in here is to create a virtual environment, right? I know you want to say.
05:05:900Tommaso Bianchi: inside your laptop, you will just create a virtual environment. So, for example, as you may know, to set up an environment for like, if you have to use some
05:15:650Tommaso Bianchi: python modules, and you need to like, do very dirty stuff. You just set up a new environment, put everything in there, and
05:22:920Tommaso Bianchi: when you will delete it, you will actually, just
05:26:10Tommaso Bianchi: like you have no repercussion on your laptop directly.
05:30:600Tommaso Bianchi: So the same for the network environment. You will have a virtual network inside your laptop. So there will be just
05:37:880Tommaso Bianchi: inside running something to say, like how to exchange data in this can essentially.
05:43:950Tommaso Bianchi: it will just simulate whatever 2 Khan network interface can do like to exchange data
05:52:796Tommaso Bianchi: this is the make command for the Icc. But actually, I noticed before that it changed. I can maybe show you like very quickly
06:03:320Tommaso Bianchi: now, it doesn't want to use
06:05:600Tommaso Bianchi: this directly, but you need to download. Where is it?
06:10:510Tommaso Bianchi: Amazing, this one. But just.
06:13:770Tommaso Bianchi: And you will use this command site.
06:20:870Tommaso Bianchi: So this is just to set up the environment. So when it it is all good to go.
06:27:600Tommaso Bianchi: we will use the the simulator. So
06:30:250Tommaso Bianchi: we have 2 parts of the simulator. One is this dashboard?
06:33:980Tommaso Bianchi: I see. Maybe I did it before. Yeah.
06:36:680Tommaso Bianchi: So I already have the interface running. Great.
06:47:440Tommaso Bianchi: yeah. Okay.
06:48:730Tommaso Bianchi: You see, this one.
06:50:200Tommaso Bianchi: I already sent some packets inside because I was doing the same thing here.
06:55:380Tommaso Bianchi: So we have this dashboard.
06:58:350Tommaso Bianchi: And we also need the Miami city.
07:04:235Tommaso Bianchi: We also need like the the control. So this year controls 0, yeah, this one.
07:17:460Tommaso Bianchi: So we have the dashboard where we can see what we are doing, for example, now, and just pressing the up arrow. And I am getting some speed
07:26:850Tommaso Bianchi: on the dashboard.
07:28:500Tommaso Bianchi: Otherwise we can open all the doors with
07:32:790Tommaso Bianchi: pressing right shift and left shift once, maybe the opposite. Okay, yeah.
07:38:970Tommaso Bianchi: You can see, I have all the doors open, and if I do the opposite they should. Okay, they should close.
07:45:650Tommaso Bianchi: and some other stuff just like pressing the left and right with the shift, maybe. Yeah, okay.
07:57:790Tommaso Bianchi: to get the arrow.
07:59:820Tommaso Bianchi: Okay, this pure basic things is a simulator not very like complex one. But still.
08:09:900Tommaso Bianchi: So in the Pdf, you also have the keys. If you want you can you? You have the possibility to use actual, an actual controller if you have one just to connect it to your laptop. But
08:20:390Tommaso Bianchi: we can use like, just the keys.
08:23:600Tommaso Bianchi: So for this the reversing part, so we can start using this simulator to understand how to reverse engineer some actions that you perform on the can.
08:33:700Tommaso Bianchi: So this is
08:37:690Tommaso Bianchi: Oh, just an example for the whatever is going on inside the the can network.
08:44:520Tommaso Bianchi: And we can also use the communities like, there's the conjun that is a common to just do random messages inside the virtual account.
08:55:170Tommaso Bianchi: and we can also use Candam to see whatever is running on the camp. So maybe if I
09:03:60Tommaso Bianchi: let's try to add one. So and
09:09:390Tommaso Bianchi: maybe I just need, yeah, okay, you can see that. Actually, now, thanks to the simulator that is running.
09:16:150Tommaso Bianchi: there's some messages like going on. So.
09:21:570Tommaso Bianchi: for example, now, and I am going up with the speed somewhere in this in this dump we have the speed of the canvas like the command to go up with the speed.
09:31:270Tommaso Bianchi: Okay, stop this.
09:34:740Tommaso Bianchi: Let's go back
09:36:180Tommaso Bianchi: so as you can see here. So the 1st column is the interface. In this case we only have beacon 0, but for the like.
09:45:500Tommaso Bianchi: the challenge. After this tutorial
09:48:700Tommaso Bianchi: you will have different canvas inside like to to look at. So is one big log file. I think with 5 different comes
09:57:936Tommaso Bianchi: you need like to separate the different messages from African and decode them. I see
10:02:830Tommaso Bianchi: the second column is the Id. So, for example.
10:06:650Tommaso Bianchi: if we take the 1st study, 1, 6, 6
10:09:320Tommaso Bianchi: is actually like with higher priority than I don't know. 2, 4, 4, the almost the last one.
10:17:250Tommaso Bianchi: And then we have the number of bytes that are in the data field and the data themselves in numbers.
10:27:200Tommaso Bianchi: Oh, sorry.
10:30:454Tommaso Bianchi: So yeah. Like, as I said, like, for example, here.
10:36:880Tommaso Bianchi: a different bytes that actually, we are using only the 1st 2.
10:41:660Tommaso Bianchi: But that depends because there are some messages that you can see. There's nothing after it.
10:50:550Tommaso Bianchi: and but that doesn't mean that all the zeros, after the 4 Byte are actually not used, maybe later in the, in another comment on the same Id, that depends on what actually is. Sending through the car.
11:06:375Tommaso Bianchi: Okay, so
11:09:540Tommaso Bianchi: to have a just a general overview of what you can do with communities. As I said before, these. We have some comments so conjun that it is to generate
11:21:160Tommaso Bianchi: count frames, that is, for testing. For example, it is just random messages.
11:28:570Tommaso Bianchi: and you can also collect them, and use wireshark to check whatever is going on in the car
11:37:240Tommaso Bianchi: tandem as a
11:40:210Tommaso Bianchi: as I show you before is to see whatever is inside the can. You can also use, for example, the dash ll
11:47:570Tommaso Bianchi: to log everything in a file, so you can just look at at everything you locked after it instead of just at the terminal continuously.
11:58:850Tommaso Bianchi: Oh, yeah, this is it.
12:01:530Tommaso Bianchi: Okay, player, that is to repeat whatever you logged before. So let's see.
12:08:530Tommaso Bianchi: I will just stop the simulator for a while.
12:12:350Tommaso Bianchi: Here, I will just start and dumb 00.
12:20:720Tommaso Bianchi: Welcome.
12:21:960Tommaso Bianchi: Yeah, there you go.
12:24:730Tommaso Bianchi: Okay?
12:25:860Tommaso Bianchi: So I will just Congen, please.
12:29:610Tommaso Bianchi: just to see if I need something else, or more than okay, just the
12:37:00Tommaso Bianchi: okay. Now it will generate some data
12:49:180Tommaso Bianchi: and things.
12:51:410Tommaso Bianchi: Yeah, okay, you can see here, we have some, some logs.
12:55:470Tommaso Bianchi: We can actually see them. In just a second Maurice. Okay?
13:04:15Tommaso Bianchi: opening them with wire shark that
13:08:270Tommaso Bianchi: there's a built in decoding for the count. So it's actually like nice to see everything. You see. Protocol
13:15:500Tommaso Bianchi: the idea, the less of the message. And down here
13:21:20Tommaso Bianchi: also the message itself, with the data.
13:23:410Tommaso Bianchi: This word so very easy.
13:28:910Tommaso Bianchi: But we can also reuse like this time. I will just use condom without yarn.
13:35:860Tommaso Bianchi: There's nothing inside, because nothing is sending data right now we can try to.
13:42:345Tommaso Bianchi: We use the same thing as before. This time we need to specify the interface and then use the log.
13:50:380Tommaso Bianchi: Was this wasn't the correct month.
13:57:90Tommaso Bianchi: A player become okay, just without specifying it.
14:05:170Tommaso Bianchi: And here, you can see, is replaying the logs that we collected before.
14:11:880Tommaso Bianchi: Let's see
14:14:833Tommaso Bianchi: and Kasnifer is a sort of can dump, but you can. Some specify, some filters you like, if you want to check only some ids or checking if the messages in the data and showing only the ones that are actually changing during the
14:31:520Tommaso Bianchi: like the the dom.
14:34:460Tommaso Bianchi: It is pretty useful, especially during like reverse engineering. Because you can see when some data like, maybe it's just flipped, for example, for 0 0 or ff for something I don't know.
14:46:147Tommaso Bianchi: For example, if you open or close the door and you can see when the did. We have some data changing invites.
14:53:410Tommaso Bianchi: because obviously, some some of these with devices, maybe 8 Byte, but always the same for
15:00:80Tommaso Bianchi: 1 min. You don't care about what is inside, because you perform some actions and nothing changed. So you can just discard them because they are not useful there. There's no information on them.
15:11:70Tommaso Bianchi: and consent just to send a specific like message. You will need to to know what you actually want to write.
15:17:860Tommaso Bianchi: So this is a little bit more involved
15:20:490Tommaso Bianchi: like, you need to specify the id, then the hashtag and the data.
15:28:170Tommaso Bianchi: And yeah, okay, so
15:31:160Tommaso Bianchi: we can start like sniffing the conf the the canvas while we are performing some actions.
15:39:410Tommaso Bianchi: I will just maybe show you also this part. So let's make sure, what's this? So
15:48:760Tommaso Bianchi: C is for like highlighting the changing bytes.
15:52:840Tommaso Bianchi: So, okay, now, there's nothing. But
15:55:780Tommaso Bianchi: if I do like before, just to show you that you can see
15:59:630Tommaso Bianchi: that, will, it will like, tell you, okay, these bytes are changing. The other ones are still the same.
16:05:360Tommaso Bianchi: And okay. Now it was very short log, so it just finished.
16:10:70Tommaso Bianchi: but after a while there will no changes in the bytes of of that Id. Then it would just remove them. Assume
16:18:470Tommaso Bianchi: so if we do, just if we play again with everything here
16:33:250Tommaso Bianchi: the other side. Sorry.
16:36:830Tommaso Bianchi: So now I'm just pressing as before.
16:41:580Tommaso Bianchi: We can see maybe something change is changing, but actually like, it's pretty hard in this way right now, because it's just only one action. But what we can do
16:50:310Tommaso Bianchi: you will do it like here is dumping everything so
16:58:310Tommaso Bianchi: you would just get the dump and log it in a file then perform your action.
17:04:680Tommaso Bianchi: and what you will do is
17:07:20Tommaso Bianchi: we're playing every time the the dump.
17:11:260Tommaso Bianchi: and this is like the operation. So you will do it the 1st time you will get us of the candamp log
17:19:454Tommaso Bianchi: plate. If you will see the action inside this
17:23:520Tommaso Bianchi: us, then you can remove the other apps, because, like there's it's just divide and conquer at the end. So
17:30:70Tommaso Bianchi: you continuously to iterate until you have like 3 or 4 ids, and you can check it manually. Essentially
17:36:50Tommaso Bianchi: so. It's quite a little bit too long to perform right now, maybe because you only have also the exercise, like the challenge to do before later. Sorry.
17:46:370Tommaso Bianchi: So you will just do this like
17:51:820Tommaso Bianchi: technique until you find, for example, when you press the right arrow, or when you open it, the the door.
17:58:340Tommaso Bianchi: and you can also try, then, to inject it. With consent. When you know which packet
18:05:213Tommaso Bianchi: you need to to perform the action, you can just pick the Id, take the data and just try to send that specific package and see the action.
18:13:910Tommaso Bianchi: So okay, some references.
18:16:250Tommaso Bianchi: And okay, some other stuff just for the for that.
18:22:850Tommaso Bianchi: With all the files that you should have in your in your zip there is also.
18:28:760Tommaso Bianchi: Now I lost everything. Let's made it here.
18:34:100Tommaso Bianchi: There's also this file, with all the directly, all the all the comments. So they are not like spread all over the Pdf. That we are just a spreadsheet with everything.
18:44:260Tommaso Bianchi: What you need to to do is take this. Well, see now he's opening it with.
18:50:750Tommaso Bianchi: I want just the tax editor
18:52:950Tommaso Bianchi: that maybe it would crush. I don't know.
18:55:960Tommaso Bianchi: You can see here you have a different cons. So con 2 come, 3 come 0 on one
19:03:90Tommaso Bianchi: and the data. So what you have to do is like 4 tasks, essentially the 1st one. It should be the easier one you need to find the did. That is an identifier on the count.
19:14:629Tommaso Bianchi: Get like some street that are let's say, is just
19:21:457Tommaso Bianchi: there's this string inside the the data field. So you should be able to find them
19:26:660Tommaso Bianchi: just decoding the the data itself.
19:30:940Tommaso Bianchi: What kind of radio it was listening during the this dump?
19:34:870Tommaso Bianchi: And the last one is reverse, which one is the steering and steering wheel angle sensor.
19:46:460Tommaso Bianchi: So you need to get which id, we have some specific behavior for the like the steering wheel. So
19:55:390Tommaso Bianchi: you can think about it otherwise later, if you
19:59:20Tommaso Bianchi: don't have a clue on how to find it, you can Google it obviously.
20:03:500Tommaso Bianchi: But there is some specific behavior on the data field for this. So you should be able to understand which one is the the angle sensor
20:12:850Tommaso Bianchi: just looking at the data and how it's changing.
20:16:410Tommaso Bianchi: So probably with, take a bit to try to do them, especially because you need
20:23:262Tommaso Bianchi: to decode everything inside the data. So you need to script your own python code for this essentially, or whatever you want to use. And if you want to use bash.
20:34:200Tommaso Bianchi: But yeah, that's it. I mean.
20:36:460Tommaso Bianchi: you can try to replicate the 1st part if you want, or going straight to this challenge.
20:44:431Tommaso Bianchi: because in the challenges. It says also something about street names. Yeah, mention that.
20:55:510Tommaso Bianchi: are we also supposed to find some string or not? Yeah, it's just you. You will decode from, like the the X to Ascii, and especially, we see in the data field the the name
21:08:920Tommaso Bianchi: deal.
21:11:840Tommaso Bianchi: Yeah.
21:13:110Tommaso Bianchi: yeah, just in this case, like, he's very much complex in a real world environment, like, you know, with the Rca. System and everything like.
21:26:400Tommaso Bianchi: there's still some messages. But there's all the authentication part before.
21:31:20Tommaso Bianchi: But yeah, I mean, it's actually the same. No, maybe
21:34:530Tommaso Bianchi: if you have access to the campus.
21:37:950Tommaso Bianchi: So yeah, I mean, they're actually.
21:40:260Tommaso Bianchi: I mean, yeah. But I'm not sure if now is a completely different canvas from what you can access, for example, from the Obd port, if you know, like, is a port used for diagnostic in your car. So when you go to the mechanically, we just plug in something and see everything.
21:56:640Tommaso Bianchi: I'm not sure if it is like
21:59:850Tommaso Bianchi: that kind of information is going through the same count or directory from the immobilizer system through the doors, or something.
22:08:530Tommaso Bianchi: because I didn't find anything like I tried to do this stuff. So
22:12:710Tommaso Bianchi: I'm thinking about the the fact that they hold the headline.
22:16:410Tommaso Bianchi: Oh, yeah, okay.
22:18:10Tommaso Bianchi: But it's not the Obd, maybe. Yeah, from there. Yeah. So maybe the yeah. The correct canvas for that. Yeah.
22:36:360Tommaso Bianchi: eventually.
22:38:10Tommaso Bianchi: Yeah.
22:42:570Tommaso Bianchi: Okay, one important thing regarding this simulator. So it it's a simulator. Indeed. Right. The point is that all the the frame formats or the interaction that you have with the campus?
22:59:980Tommaso Bianchi: Okay? So everything that you will be doing in the with the desktop you can do very, and
23:15:210Tommaso Bianchi: and on these these
23:17:500Tommaso Bianchi: nothing, only, for, like the packets, and then everything will exactly the same format will be the same.
23:24:650Tommaso Bianchi: The meeting would be the same thing about the understanding.
23:37:30Tommaso Bianchi: Okay, so you see, the the format is exactly the same. So the simulator is replicating certain behaviors. That's for sure. But it's completely
23:47:790Tommaso Bianchi: okay. Here is a little bit.
23:52:180Tommaso Bianchi: Maybe I caught like this is the steering wheel language sensor for real car.
23:58:30Tommaso Bianchi: This one is actually from the Obd, so is another protocol for diagnostic, where you can ask for some data, and this is to retrieve. For example, is when I get back the speed
24:08:550Tommaso Bianchi: I'm asking for the speed of the car, and I will get him back the detail.
24:15:180Tommaso Bianchi: So
24:19:390Tommaso Bianchi: you can also see this is a capture like the renewal.
24:24:90Tommaso Bianchi: Good God knows, my girlfriend, I use Surchart for this.
24:32:10Tommaso Bianchi: You asked for the permission right?
24:39:90Tommaso Bianchi: I also paid for the customer.
24:46:120Tommaso Bianchi: One thing that the simulator cannot do is to replicate the physical behavior of the campus. Right? So it's not something you can do in here. You don't have access to that level of info. So, for instance, if you wanted to- to amend the above.
25:03:570Tommaso Bianchi: you will not be able to do this in a simulator, or if you're able to do this, please, so much time trying to do that if you want to ride the physical simulator.
25:16:687Tommaso Bianchi: But then, yes, you don't have a basically, you don't have business and you don't have the
25:25:400Tommaso Bianchi: the file set machine implements, the control.
25:33:400Tommaso Bianchi: You don't have that here.
25:44:190Tommaso Bianchi: That's it. I mean they should, they should do the work. Now.
25:49:660Tommaso Bianchi: now the thing is, I think you have a machine which you can use, or maybe you can create some
25:58:120Tommaso Bianchi: if you need. I can trust someone with my laptop in case
26:03:650Tommaso Bianchi: please know. But yeah, if you need it.
26:08:450Tommaso Bianchi: But I think it would be interesting to try to stay now, and if you have any questions
26:21:150Tommaso Bianchi: we can ask questions.
26:24:430Tommaso Bianchi: I'm also here to help.
26:26:580Tommaso Bianchi: She needs to let me know it's a whole day.
26:37:291Tommaso Bianchi: I mean, if you can do everything in this hour.
27:00:387Tommaso Bianchi: Okay, let's do something different. If most of you don't have a machine that can use it. We move on with the lectures. And that's see, basically.
27:13:290Tommaso Bianchi: And you can try. And if you have any question you can ask them in the next lecture, or something like that.
27:28:350Tommaso Bianchi: Okay, let's do that.
30:30:740Alessandro Brighente: In the environment
30:33:910Alessandro Brighente: and try to solve the challenges. Of course, if you have any questions we are here to help you getting done with the
30:44:870Alessandro Brighente: with this stuff as well.
30:47:330Alessandro Brighente: Okay, so
30:52:170Alessandro Brighente: let's move on with the with something different then.
30:55:850Alessandro Brighente: So what we did up to now covers the the python. cars, right? We've seen autonomous systems in terms of
31:04:508Alessandro Brighente: only cars. But our autonomous system might also be drones, right? So we're talking about autonomous systems
31:14:970Alessandro Brighente: and
31:18:130Alessandro Brighente: and in terms of the systems, also in drones, we have a controller right? We have some sensors. We have some actuators, with the main difference that not only now we are moving into 2 dimensional space. We're not only driving along the street, but you're moving in a 3 dimensional space. Right? These devices are flying. So you have some additional degrees of freedom. Right? You you can do something more, both in terms of control, it makes it a bit more difficult to to actually control.
31:46:41Alessandro Brighente: How the drone behaves right now you have another dimension, a 3rd dimension, which is the the altitude of the drone.
31:51:850Alessandro Brighente: Okay, so
31:54:350Alessandro Brighente: why do we care about drone? And what do we refer to when we talk about drones? So here in the slide you have different examples of drones implementation, right? And the main difference that we've seen on the slide is between fixed wing rotors and copters. Right might be quadcopters to copters. It depends on the number of propellers that they have right? So a quadcopter, of course it has 4 propellers.
32:19:170Alessandro Brighente: 3 copter it has 3 might be 8 8
32:25:840Alessandro Brighente: fine. Well, hope that opens. Yeah, thanks.
32:28:920Alessandro Brighente: And so you will have 8 propellers right? But then the thing is, you see, these main difference in here. Right? So when we talk about these copters, we have propellers that rotate along this direction and put some trust towards the ground instead of here. You have propellers rotate in this direction, and the trust is not towards the ground. It's towards the back
32:53:170Alessandro Brighente: of the drone. Right? So the difference in here is that these drones are not able to hover over a given location. Right? You cannot put this drones above this this last thing here, and pretend them to be fixed, and to to stay in there as much as you want. These drones are just flying and passing by right
33:11:727Alessandro Brighente: the differences that we see are just in the implementation and the capabilities, the physical capabilities of the drones. Right? We're not talking about the capabilities in terms of of sensing or communication capabilities, control capabilities. Right? We're talking about something a bit different here. So usually, we focus on these kind of of drones, right? And mostly we focus on this kind of drones. These fixed wing rotor drones is something that are more.
33:39:980Alessandro Brighente: use the
33:41:790Alessandro Brighente: on a military grade, right? So military usually have these kind of drones. And indeed, these were the 1st kind of drones that were developed historically right. Historically, we have drones because of military purposes. But then someone realized, Hey, I can mount a camera on a drone and make it something funny instead of something that is causing a lot of problems lately. Right?
34:01:820Alessandro Brighente: Good. So we like the thing that the drones can be used for something funny. And how do they work right? So irrespectively on how the what is the proposed on the drone? Let's say, we need someone that controls drones or something that controls the drone. Right?
34:20:623Alessandro Brighente: So here you have an example on a possible operating mode. For the drone. Right? So you have your device in here, drone that needs to be guided somehow. By a remote pilot. Okay, so there is a communication channel between the drone and the pilot. Right? The pilot is on the ground, and it can send messages to the drone and tell the drone actually what to do. Right?
34:43:940Alessandro Brighente: So we talk about drones, or we talk about the amended area vehicles. Right? So the point of this vehicle is the fact that we have no pilot in in the vehicle itself right? There's no one sitting in the drone and providing controls to them. Right? It's very different from what we have what we what we've seen with cars. Right? We had the different levels of automation
35:07:639Alessandro Brighente: from side. This is something that we cannot
35:11:220Alessandro Brighente: having here, right, or at least not in the same way. Basically because there's no driver in a drone. Right? So
35:18:790Alessandro Brighente: we can have a mapping right? Of course, because we we can envision having a pilot in here. Right? So we can envision different levels of automation for the drone based on how much control the the ground pilot has over the actual right. So no control. It means that sorry, no automation. It means that the ground pilot is providing all the instructions to the drone
35:43:800Alessandro Brighente: and up the the level where we have full automation, where the maybe we have a grand final of this checking what the drone is doing right, what the drone is sensing to
35:54:100Alessandro Brighente: see that everything works fine. But we have no actual intervention from the controller. Okay, so in here, you see, one of the 1st security issues that we might have with drones. Right? So if an attacker can control this part in here the data exchange between the drone and the ground control station.
36:13:959Alessandro Brighente: It's a problem, right? Because basically someone can either report back in fake information to the ground control station or provide a fake instructions or malicious instruction to the drawing set right. And
36:28:680Alessandro Brighente: this communication, of course, is implemented by our protocols. It might be Wi-fi based protocols something like that. Where we have some levels of security, we might have authentication or integrity protection. But we need to be very careful when we design this kind of protocols, or when we deem this protocol to to be secure.
36:55:880Alessandro Brighente: and then second operating mode. Well, not necessarily. We want to have, as I mentioned before. Not necessarily. We want to have a ground conversation right? Maybe we want these drones to be fully autonomous and equivalently, from what we've seen with the vehicles, platoons. We can talk about multiple drones that communicate with one another that have a common goal, right? And can share
37:21:900Alessandro Brighente: instructions or yeah, control information. Okay? The difference in here is that we don't talk about platoons anymore. But we talk about swarms or fleets or drones. Right? So you have these. I think you might have seen these videos where we have hundreds of zones equipped with some light and stuff. And they do this fix in the air and create light figures. Right? That's exactly what
37:45:300Alessandro Brighente: what is happening in here. You have multiple drones that can share some information. And thanks to the information that they share, they have.
37:52:892Alessandro Brighente: They can
37:54:910Alessandro Brighente: understand the location, their mutual location in space and adhere to some predefined figures.
38:03:740Alessandro Brighente: Okay? So somehow, we have the same security issue that we had before right? Because these rules are communicating one another with some communication protocol should again provide authentication, integrity, protection. Blah, blah, right? And the these protocols sometimes are built a talk for
38:26:380Alessandro Brighente: Brand's amazing right? Themselves.
38:29:310Alessandro Brighente: Okay, and then we'll see how these protocols look like and how they work. Basically, okay, so from our perspective, these particular operating model for drones is very similar to what we've seen for autonomous cars. Right? So we can have this cooperative perception right? This cooperation between drones to reach information that they they can actually collect.
38:57:900Alessandro Brighente: And this is something we've already seen in in the introduction slides
39:05:510Alessandro Brighente: right? This is more or less the this come on the structure
39:11:680Alessandro Brighente: on the drone right? What? What are the main modules that we have in the drone irrespectively of the the complexity of the drone, or how big it is, or how small it is, or whatever right? That's the that's the main architecture that it has.
39:25:662Alessandro Brighente: So we see that, we have the battery in here. Right? Of course. Drones do not have a wide power supply, and we don't have wires connect to drones. But we have batteries right, which is one of the 1st
39:42:147Alessandro Brighente: constraints that we have in here. Right? So it's better. It's it's limited in terms of power, and we cannot waste so much energy for different operations. So if you check on the power profile of a drone.
39:58:840Alessandro Brighente: what you will see is that the most energy hungry component of the drone is the is the motors itself right? These motors that make the propeller spin. This is what consumes the most energy in a drone. Right? So you know that a lot of energy will be invested to having the drone actually flying, or to control maneuvers for the drone. So you understand that if you have a very complicated algorithm or very energy, hungry communications.
40:29:10Alessandro Brighente: it starts to get a problem right? Because if you are consuming a lot of energy for other for other tasks, and the drone is not able to find more, which is its main task. Right? Good. So this button in here should power all the other modules that you see in the figure, and it should power, active, sensory should power actuators, the motors that we were talking about should power, the communication module and processing unit.
41:00:100Alessandro Brighente: So this right here works like sensors, which might be productivity sensors. It might be cameras. It might be, whatever sensors you can use to to gather
41:10:150Alessandro Brighente: information from the environment. And thanks to the measurements that you gather in here, you pass this to the processing unit. Right performs this computation, and then decides on the action to to perform on the actuator a very simple example. We have there the proximity sensor for the drone. This proximity sensor is the equivalent of what we've seen for the lighter for the cars.
41:32:710Alessandro Brighente: Right? So you have this module that is actively sending some laser parts towards the ground and measuring how much time this laser pulse takes to get back to the drone. Based on this, you know, the the drone knows its altitude right, and if we want to have a fixed altitude for the drone, or if we don't know, we don't want the drone to fly at lower than a threshold value.
41:54:940Alessandro Brighente: Then this is something that the processing unit realize and provides information to the actuators to to compensate for this right. So, for instance, if we are too close to the ground, then the processing unit will tell the actuators the motors to put more thrust, such that the altitude of the drone increases.
42:13:880Alessandro Brighente: and then we have the communication module right? This is exactly what we've seen in the 2 previous slides for how drones should interact with the either the pilot or with the with other drones. Right? We have a plate of drones. Then, of course, communication model is talking with the communication module by talking with the control station in the ground.
42:35:970Alessandro Brighente: And of course you have these giant right? Very straightforward. This 5.
42:41:410Alessandro Brighente: And then in terms of control, right? We've
42:47:680Alessandro Brighente: we've seen for cars that we have controllers. Right? If you want to have autonomous vehicles, means that there's a part of the processing that deals with some of these physical measurements and how to to compensate for possible errors.
43:02:523Alessandro Brighente: And it works exactly the same also in movies, right? So we have this feedback Controller.
43:10:390Alessandro Brighente: usually. Pd controller, when you have just a proportional derivative part of the Controller. And you have these 3 module here, right? These 3 different controllers. So you have one that controls the position. So the altitude or the horizontal coordinates where the drone is in space.
43:31:00Alessandro Brighente: Then you have the velocity control which accounts for both vertical and horizontal spin. Right? How do the troll move in space. How fast it does that! And then you have the altitude control. Altitude means the rotations along the different axis. Right? So you have. You're drawing the 3D reference system right? And these pitch, roll and yaw define the angles along these 3 directions.
43:57:930Alessandro Brighente: Great. So you see that we have exactly what we've seen for controls in in clouds. Right? You can set some
44:06:804Alessandro Brighente: some target value, for instance, for for the altitude, and continuously check whether this altitude is changing
44:16:807Alessandro Brighente: as long as you have an error that is not 0 with respect to the current altitude and target altitude and adjust the behavior of the drone, the thrust that the drone based. Right? So this is something that you can do for position control. If you want to adjust the loss of control like you want the the drone to hover over a given location, right?
44:41:790Alessandro Brighente: But then, when the drone overs the given location you have
44:48:710Alessandro Brighente: weather is messing with you somehow. Right? So you have wind, for instance, wind is changing the location of the drone, because it's pushing the drone in a certain direction. So, thanks to this velocity control, you can check if the drone is moving, and you can see whether it's moving fast towards a given direction. You can estimate. Okay, these motion is due to external factors such as wind and I can compensate for that. So the Controller should compensate for these variations that the that the drone has
45:15:430Alessandro Brighente: right. So here you have a reference to these specific Controller, which is,
45:21:345Alessandro Brighente: very generous. Just to give an idea on the different modules that you will have on a drone, and how they they should interact with one another.
45:33:990Alessandro Brighente: Applications, of course. Why do we talk about drones? Right? So here you have some some information on where drones are applied, and how they are envisioned to to be applied right? Some
45:49:940Alessandro Brighente: so applications that maybe you didn't consider before, right? So the fact that we have drones. It's very nice for having videos of weddings or holidays or this kind of stuff. That's of course, not. The only application that we have right? So here you have.
46:09:880Alessandro Brighente: did I have the no so here, for instance, you have the example infrastructure. Why would we use drones for infrastructure. Well, because they help us monitoring critical infrastructures. For instance, right? So assume that you have a very big
46:27:48Alessandro Brighente: production environment, big factory with the big buildings. Right? It's not necessarily easy to climb on top of that and check whether some of these constructions are healthy or not right? That they have some structural problem. Well, you can use trunks to to move in there. And you have dedicated algorithm that that help you
46:49:314Alessandro Brighente: get this information and transportation. Why do we use drones for transportation? Well, because, for instance, if you have an emergency situation and you want to deploy a traffic sign or something like that in a specific location, and you can bring it there to a drone, right? It has its own Sd screen.
47:07:480Alessandro Brighente: and they will show these stingy, or you can monitor the speed of the cars. Thanks to thanks to drones, I'm not very happy about this application. But it's something you can definitely do right
47:19:470Alessandro Brighente: telecommunications imagine that.
47:23:653Alessandro Brighente: I don't know if you've ever experienced this problem. But if you are in a in a place that is too crowded right then. Your your calls, may not work right, or you may not have access to the Internet. So to solve this problem, you can have a Uav and bring it close to the to the location where you have this problem, and the Uav will work as a secondary station for those part of the traffic
47:50:452Alessandro Brighente: agriculture. Thanks to the sensing capabilities of drones, right? You can mount specific sensors that detect, for instance, the humidity of the soil or whether specific bugs affect your
48:04:220Alessandro Brighente: your field, and you can like this kind of stuff. And report this information back to the to a central controller that helps you distributing the resources that you need in order to to keep your plans healthy, and search and rescue right? This is very important for a situation where you are in the mountain, for instance. Situation where there's an avalanche. And you want to understand whether, when you have
48:28:850Alessandro Brighente: any issue with that right? I mean someone that face this problem, let's say before moving moving the the helicopter, you might just use a drone and sense whether there's
48:39:880Alessandro Brighente: something for someone beneath the snow, right? And so this is.
48:45:736Alessandro Brighente: yeah, just give you some hints on possible applications of drones. Well, now, what is interesting for us is, okay, what can we? How can we mess with drones? Right? What are the main components of the drones? Such that we can launch attacks against them? Right? So here you have the 6 targets that we can.
49:09:690Alessandro Brighente: that are of interest for an attacker.
49:14:130Alessandro Brighente: Right? And
49:16:250Alessandro Brighente: well, let's go through each of them. The 1st one is the drones hardware. Right? So. And talking about hardware, it means that we that, we consider the CPU and the processing unit. We talk about the sensor. We talk about the firmware right? All the low level components of the drone. Right? So.
49:32:670Alessandro Brighente: for instance, one of the I would say problems. But things
49:39:558Alessandro Brighente: with drones is that proprietary Pmwares are not released right? If I am buying a Dji drone, for instance, and I want to to mess with the software component of Dji. Well, 1st thing that I need to do is to reverse engineer the whole teamwork because
49:57:480Alessandro Brighente: I have no access to that right? Good. So how do I access the drone? Well, for instance, I may have the the drone in my hand. I have the hardware component. Attach some of these cable to jtag connectors right and directly talk with the with the hardware component without going through the whole protocol. I can directly talk with that. But then I need to reverse engineer, whatever the drone is trying to tell me
50:22:390Alessandro Brighente: if I don't have the correct format or the correct bound rate, or whatever physical information the drone is doing. I'm not able to decode anything out of that right
50:32:980Alessandro Brighente: sensors. Of course, sensors are a great target for us. Right? Let's give you an example that should be straightforward for you. Imagine what we have with Lidar. Right? We it works exactly the same for drones. You have a a sensor that is sending some signal towards the ground
50:52:830Alessandro Brighente: and measuring how much time it takes for the signal to be applied to estimate the altitude. Well, if we spoof this signal, you can imagine that we can change the perception of the drone towards its altitude. And that might be a problem. It might lead to the fact that an attacker can capture physically capture the drone.
51:10:880Alessandro Brighente: Good. So
51:13:670Alessandro Brighente: yeah, we'll see some some of these attacks and how they work again. In the sensors. Very often it's included the GPS right? Which is not the sensor, because it's not sensing anything but it. It relates to the to the to the drone hardware part. Why is that. So because basically, the the GPS sensor is telling the drone, okay, you are in this location in space. Right?
51:40:380Alessandro Brighente: So if you're moving away from the ground control station, and your battery is over. For instance, you might want to return back to the to the departure point. The take off point
51:51:710Alessandro Brighente: right? And in order to do that, you need to have access to the GPS information, otherwise you simply cannot get back to the initial point right? So attached to GPS is both in GPS causes these kind of problems, for instance.
52:06:163Alessandro Brighente: Then, drone, just as a package, right? All the non electronic devices.
52:14:690Alessandro Brighente: right? Which which somehow deal with the with the physical components of the drone that are not. specifically, it components. Right? So for instance. We may be talking about the the propellers right or the the motors on the drone.
52:31:00Alessandro Brighente: Right? So ideally, these these monitors are electric motors right? If we create an electromagnetic field that is powerful enough, we may block the rotation of these rotors right?
52:42:420Alessandro Brighente: Course. That's ideal. It's very difficult to achieve. But that's an idea. So we're talking about these these kind of of targets that are part of neutron itself.
52:54:620Alessandro Brighente: Then the ground control station, the ground control station as target might seem kind of weird, right? So we've seen these ground control station and we can imagine it like these this office
53:07:744Alessandro Brighente: which is close and and difficult to to access. Right? That's not actually the case, because a smartphone might be a ground conversation right? If you buy a Dji or whatever it seems like I'm responsible. It's not like that. If you buy a commercial drone and you want to interact with that. Usually you can download an application and start talking with that right
53:29:520Alessandro Brighente: and then makes your smartphone a ground control station. And then it means that if I download an application which has some security issues.
53:38:960Alessandro Brighente: then the drone will have some security issues.
53:42:530Alessandro Brighente: 1st principle, what is this Fpv channel? Usually. When we talk about commercial drones they are equipped with the camera. Right? That's the main
53:53:500Alessandro Brighente: game that you play with drones. Right? You connect them to your smartphone and you collect whatever image the the camera and drones capturing, you have it on your smartphone. Okay, so this is the 1st person view, son, because you are seeing here exactly what the drone sees. Right. If the drone is holding, flying around collecting information through the camera, and you're seeing exactly the thing
54:15:58Alessandro Brighente: these has. This is implemented through dedicated communication protocols. Right? The the data stream from the Uav. The smartphone, for instance, is implemented through a dedicated communication protocol, which has been found to have a severe security issues that we'll see. But that represents a farther target for impact.
54:40:271Alessandro Brighente: Then, of course, the pilot, right? So it might be a target. But that's not our target. The person that join the drone and cloud services right? Cloud services, because.
54:51:627Alessandro Brighente: the drone might generate a huge amount of data. Right? If you're collecting all the frames that the the camera captures. It's not something that necessarily, we want to transfer to the to the users smartphone something like that, right? Maybe we don't even need that
55:06:301Alessandro Brighente: so in this case drones can directly communicate with the Cloud Service Provider through Internet connections. Right? So you have this connection between the drone and a 3rd party server that might represent
55:21:660Alessandro Brighente: possible target for an attack writing. My, just want to to mess with the data under the drone is collecting. Because this data, then, is used by either other drones
55:32:630Alessandro Brighente: for the the ground control station.
55:37:810Alessandro Brighente: What could the be possible attackers? Right? So
55:46:180Alessandro Brighente: 1st thing. Now, we are talking about attackers that want to
55:51:168Alessandro Brighente: the target drones. Right? So you see, when we talk about drones, and when we talk about these kind of devices. The 1st thing might be okay. I'm interested in messing with these devices, or the other thing might be. I want to leverage these devices to cause security issues. Right? So in this case, we've been talking about an attacker, the targets, the drone itself. Right? So of course, this might be other civilians or military, with the different equipments available. And of course.
56:20:287Alessandro Brighente: but yeah, basically what you can.
56:23:949Alessandro Brighente: What you understand is that as a civilian and buying things. You you find online or on whatever website. Then you can implement attacks against drones using very simple devices. Right? So for instance, you might have software defined radios, right? These devices that are software controls. But basically have some radio frequency technology that you can
56:48:501Alessandro Brighente: fully controlled, and to use to interact with the the drone or to mess with the communication channel on the drone. Right, for instance, jamming is something that you can implement
56:57:235Alessandro Brighente: via software defined radius computer. Of course, you need a computer to control these kind of devices. But then you can use also lasers or magnets to to mess with the with sensors that the drone has right lasers, for instance. You might play with lasers, and in fact, the destination that the drone makes words its altitude, for instance, right again or a magnets to to mess with the with the.
57:23:270Alessandro Brighente: with the compass of the drone or some of these inertia measurement unit sensors depends on how they did, of course. But that's that's an idea. But definitely not to capture the drones code. But the other thing that is not mentioned in here and we'll see is that. Some of the devices that you have at home might directly be used to attract drones. Right? So, for instance, if we use speakers right for
57:51:870Alessandro Brighente: music, this kind of stuff right? If you have very loud speakers these might mess up with the the electromechanical sensors that are mounted inside the drones right, we will see a specific attack where you can use sound to to mess with the estimation that the drone makes works in state in space.
58:12:00Alessandro Brighente: Okay, so we don't need them. Very complicated talking devices in order to create to mess with drones.
58:24:319Alessandro Brighente: It's a joke, you know, the you want to catch butterflies. Why do you use this kind of stupid thing?
58:41:540Alessandro Brighente: Yeah, yes, yes. So this this has to do with the with what we discussed. Now, where? Where is the attacker? Right? Because, depending on the the
59:02:10Alessandro Brighente: the location of the attacker. Let's say, if I had the drone in my hand. I can do something if I don't have it, or I just see it in the skyline. Of course I have different capabilities. But yes, what you will see is that most of the attacks that have been proved effective against the sensors drones.
59:22:120Alessandro Brighente: Have a range of approximately one meter.
59:24:870Alessandro Brighente: Okay? So yeah, the worker.
59:30:280Alessandro Brighente: But yeah, you're limited in range
59:33:200Alessandro Brighente: unless we are talking about GPS, or we are talking about messing with with the controller, messing with the some of the information that we can directly provide to the drone. Right? But if, if, again, we are talking about these loudspeaker stuff that I was mentioning. Yes, it's nice works, and it's a mess for the drone. You you see it behaving crazy. But the the range is very little
00:00:310Alessandro Brighente: And yet again here we're talking about possible attacks that we have against drones. But if we want to secure how the drones behave, then we have a whole whole range of technologies that we can use. For instance, what we did is to to try to identify drones. Right? So you have a critical infrastructure where you have some drones deployed and military base, whatever
00:23:274Alessandro Brighente: and you have drones lying around, and you want these drones to play around because they're useful for whatever proposal you have. But then you don't want other drones to to get your critical infrastructure. You want to to leave them out. So the point is, how do you detect
00:38:820Alessandro Brighente: that no malicious drones are part of your network, whether malicious drones cannot get into your network. So here you have different strategies.
00:49:321Alessandro Brighente: And in here it's a bit easier if you want to to control this kind of stuff because you have networks, you have a laser, or
00:59:480Alessandro Brighente: oh, radar measurements which you can perform right? So usually defending the drone is a bit easier than than attacking the drone. But yeah, let's getting back to to the attacker again. We want to attack the drone and see what we can. Which kind of attacks can be effective and have a a range of sufficient not to be considered a threat for the drone. Right? So the
01:26:160Alessandro Brighente: the talking here is mainly 3 times right? So the 1st thing the 1st kind of attacker is, I have the drones in my hand. Right? How can I be an attacker if I have the join in my hand. Well, the the 1st thing is what we mentioned before right. If, for instance, the my target is the firmware of the drone, because I'm interested in intellectual property behind the water, whatever producer placing the drone.
01:53:40Alessandro Brighente: then this might be my main way to go right. I have a physical access to the drone. I can try to connect with the drone and reverse whatever the drone is trying to tell me.
02:06:380Alessandro Brighente: Good. So these enables me to modify some of the hardware parts of the drone, or to to mess with the firmware right trying to modify the firmware. If I have information with that, or find the way of doing that.
02:19:542Alessandro Brighente: Then the second attacker model is the physical, approximate one. Right? It means that I don't have a physical access to the drone, but I see it close by right. I see that the drone is there, and I can perform some interactions with the with the drone which means messing with its radio communication capabilities or mess with the Ftv channel or mess with some of these sensors of the drone
02:48:830Alessandro Brighente: is using, or I have a recent adversary. Right? So
02:53:682Alessandro Brighente: this is with the with the Internet of thing filing. Right? So it means that I have a drone that is connected to the Internet, I connect to to a network which is accessible worldwide. So from that instance, I may notice the connection of the drones to to a network, and then mess with that
03:13:330Alessandro Brighente: right, or again messing with the with the cloud service that the blue zone is using.
03:19:738Alessandro Brighente: And of course, the attacking party being measured through the CIA trial that you've seen many, many times at this point. Right? We're messing with the confidentiality, integrity, and availability on the drone. We focus mostly on the availability. Here. We talk about attacks against drones. Because it's what more interesting for us, right? See how the the drone behaves in in different working conditions.
03:49:790Alessandro Brighente: Okay? So give you, some hints on possible hardware attacks against wrong before going into the details on how that that should work. So
04:04:660Alessandro Brighente: when we deal with hardware attacks, it means that we have a piece of hardware. Say the compass right that is detecting the orientation on the drone towards the magnetic pole.
04:16:60Alessandro Brighente: And this compass, of course, is performing some physical measurement. But then it's associated with the software component that does something based on the compass value. Right? So if we want to mess up with the compass, we cannot just simply send random
04:32:678Alessandro Brighente: signals towards the compass. But we need to understand how the drone uses the compass in order to perform its operation right? So one of the the actions that the drone performs, based on the compass value is deciding whether to take off or not. Right. So, according to certain situations. Then, or according to certain compass value the the drone
04:57:290Alessandro Brighente: decides whether it's reasonable or not. Take off, and what they what they show in here is that, if you basically attack the electromagnetic field that the compass is using, then you can prevent the drop from
05:12:216Alessandro Brighente: from taking off right? And this is something that you don't need to to launch from very high distances. Right? If you know that the drone is taking off from this location, you can just mess with the magnetic field in this specific location, and this makes it convenient. So the question could be right, how do I know where the drone is taking off from. Well, one of the services that are coming up for drones is
05:36:483Alessandro Brighente: charging them via wireless power transfer. Right? So you have these. I think you're familiar with that. So you have your smartphone, and you can charge the smartphone without actually connecting a wire. Right?
05:50:480Alessandro Brighente: And the I should do this kind of stuff before sharing.
06:03:900Alessandro Brighente: Okay? So you have these base station that is providing an electromagnetic field to to the drone to charge the battery. So I know that the drone is going to take off from that base station, and I can mess with the electromagnetic field that it uses
06:19:200Alessandro Brighente: or another one in here attack on the stabilization algorithm that the drone implements through the camera right? So the camera is a sensor for the Uav. So, for instance, the the drone collects images right. It collects a lot of images from the ground, for instance, and detects some features on the ground right? And this is something that the drone uses for
06:43:540Alessandro Brighente: for stabilization algorithms. Right? So what does it mean? The drone wants to understand whether it's going towards a certain direction. The drone wants to understand whether it's moving around when it should be stable over a certain location, right? And these are all things that are part of the control software that the Uav should be implementing. So we can mess with this kind of stuff. Right? So it means that if the drone is detecting some features on the ground
07:13:168Alessandro Brighente: then we can mess somehow with these features right? So if it is detecting corners. We can project some light on the ground and basically prevent the drone from detecting these these corners or moving features around such that we, we let the drone move towards the direction that we that we control. So let's focus for a moment on this kind of of attack, right?
07:40:880Alessandro Brighente: So we said that the the drone uses the camera in order to understand the the direction which is moving, or whether it's stable over a certain geographical area.
07:55:700Alessandro Brighente: so in order to do this the drone uses something that is called optical flow, right? So when we talk about optical flow. We are talking about the apparent motion of features and objects. In a visual scene. Right? So what does it mean? It means that if I'm looking at the corner in there and I'm moving towards this direction, then, from my point of view, it's like the corner is moving towards the opposite direction. Right? And this is exactly what the drone is doing
08:22:890Alessandro Brighente: right. So, thanks to these optical flow algorithm.
08:27:100Alessandro Brighente: then the drone collects a series of successive images, the text feature and the text apparent motion of these features in the in different frames, right the different images that it collects.
08:40:250Alessandro Brighente: and again, based on the fact that we can define physics inform model of the how the drawing moves. Right? So we know that if the drone is going towards a certain direction at a certain speed. If different frames have been captured with a specific periodicity, right? There's I don't know half a second between the frame and the successive one. We know exactly how much features have moved in that image. Right?
09:09:330Alessandro Brighente: And thanks to this, John can understand whether it's moving towards the correct direction.
09:19:760Alessandro Brighente: This
09:23:760Alessandro Brighente: sure.
09:39:390Alessandro Brighente: sure what we're talking about. Let's say that the drone detects the feature in here, and it's moving towards this direction. Right? So I'm the successive image.
09:50:210Alessandro Brighente: of course, as long as try to take the same feature and expect this thing to be here right? Because it moved up
09:57:430Alessandro Brighente: opposite today. Good. This is something very important for the drones, because they need to understand where they are in space and how they're moving, and we cannot simply rely on a single sensor. Right? We cannot simply rely on the on the compass, or we cannot simply rely on
10:17:282Alessandro Brighente: on the on the GPS, we might have with some problems with that. So we use different information coming from different type of sensors.
10:28:366Alessandro Brighente: What does the device that implements the optical flow on the on the Uab look like? So this is the module that you might see in some some draws. Right? So basically, you have the camera. The camera is collecting some images. And then you have this processing
10:44:867Alessandro Brighente: component in here is implementing the the the computer vision algorithms that you need in order to detect features and to estimate the motion of these features in here. Right? So this camera is pointing towards the ground. So it's a down facing camera that is collecting images from the ground.
11:09:230Alessandro Brighente: okay, so these values in here is the offset that we expect from the location of the feature in the 1st image, and the same feature in a successive image, right. So based on the speed of the drone based on the the controllers, applied speed. We expect these features to move according to these venue here.
11:33:340Alessandro Brighente: Right? So there's 1 assumption in in this algorithm, right? So it's the assumption that the features that we are detecting are stationary, so we cannot. So, for instance, if we detect a feature from another vehicle that is moving the opposite direction, then that vehicle will have its own speed right now. Our computation will not match, because we don't know
11:55:870Alessandro Brighente: the speed of the other vehicle, and we cannot account for that. So the the assumption that you have in here is that the ground is sectional, and we detect features that are fixed in space that are not moving right? So if that is the case. We know exactly the drift that we expect from
12:10:140Alessandro Brighente: from these features.
12:13:200Alessandro Brighente: And then we need the some feature detection algorithm, right? Something that is rather easy to detect. As as mentioned before.
12:22:190Alessandro Brighente: the drone is battery powered.
12:25:190Alessandro Brighente: And
12:28:930Alessandro Brighente: yeah, and we cannot invest so much energy into implementing complicated algorithm. So we need the feature detection that are straightforward. So what are straightforward features that we can detect? Well, the the 1st feature that we can detect easily detect is edges right? So
12:53:390Alessandro Brighente: do you already talk about this?
12:57:310Alessandro Brighente: No, so when we detect edges right? And my, let's imagine a very simple example, right? So we have a a black and white features, and we look at the table from the above. Right? So what they see if I've had a very strong, light words. The table. I will see this thing in here as wide table.
13:17:630Alessandro Brighente: and then in here we'll see black right? So computing the difference of intensity, intensity of the pixel in here and in here I will see a huge difference, right? Because it's It's very well defined. And if the
13:35:600Alessandro Brighente: on the post yeah, it's a difference between 2 goes by. Pixels
13:43:700Alessandro Brighente: is greater than a certain threshold. Then for me, this is an edge right? And along these, as in here, we'll always see the same difference. So to me this is defines an edge.
13:55:260Alessandro Brighente: and the same reasoning can be applied to detect corners. Right? So what are corners if they're just 2 edges
14:04:290Alessandro Brighente: that are connected at a certain point. So in here I can perform the same computation. See that the the the mapping of the difference of fixes in here have a specific shape, and therefore this defines a nice right? And I will not see this difference in here, because it will have the the whole table.
14:23:10Alessandro Brighente: the color.
14:24:290Alessandro Brighente: Well, okay, this is exactly what the join is doing. The detecting. These kind of features is kind of straightforward. You're just computing the gradient of the
14:34:489Alessandro Brighente: you're just completely integrated. Right, it's just a difference between of intensity between successive pixels.
14:44:20Alessandro Brighente: Good. So in order to do that, we have dedicated algorithms. Right? So for instance, when we talk about the optical flow.
14:52:930Alessandro Brighente: we have the detection algorithm, and that provides us with the feature, like the location of the features. And then we have these other meter link here to compute the optical flow which is Lucas. And another method.
15:11:700Alessandro Brighente: Okay? So how does the kernel detector work exactly this thing as I mentioned here. So you just compute the derivative of the image to detect sudden changes in intensity. Right. So if you compute this derivative in here, you will see a sudden intensity change between these part of the
15:34:976Alessandro Brighente: of the image, and these are part of the image in here. Right? So if we have these in just one direction, then we have an edge. If we have the in 2 direction. We have a corner exactly this thing in here. Right? So this is very efficient. It requires a minimum effort right to compute this derivative. It's very simple. And so it doesn't
15:57:289Alessandro Brighente: consume too much energy for the drone to implement state. Of course, this is not the only algorithm that we can use automatic point detector is not the algorithm. If you check for more complicated and of course better in terms of performance algorithms. And in particular, if you check at how vehicles right cars
16:23:469Alessandro Brighente: implemented their simultaneous localization and mapping algorithm. They use exactly those algorithms to to detect features right? But, as we mentioned before. We cannot allow for complicated solutions in drones, because we don't simply have the resources to to implement that. So we we keep with the detection, because it has been proved to to be good enough for what the drone needs to do.
16:49:150Alessandro Brighente: And then so here you have a a description on how
16:58:510Alessandro Brighente: the Lucas Canada optical flow work. Right? So here you have the the whole description. But basically what you're doing here is to estimate the location of the the features. Right? So again, let's assume that you have
17:19:190Alessandro Brighente: your image, and you're detecting one feature in here.
17:22:630Alessandro Brighente: One feature in here, one feature in here, right? And then you want to see where this feature moved in the successive image. Right? So I don't know. Let's say that the drone is going towards this direction, and the features are moving in the other direction, like you might just see
17:39:200Alessandro Brighente: this one in here. Right? So you want to estimate the
17:46:333Alessandro Brighente: the the displacements of these features right where the the difference in terms of pixels from where it was before and where it is now. And in order to do this, you solve
17:58:510Alessandro Brighente: up on.
18:01:240Alessandro Brighente: It's a system of equation, right? So you have a certain variable that you want to determine which is this displacement. So you can write your system of equation and try to solve that if you try to do that. You will see that it's not straightforward to do that. But you need to solve a least squares algorithm right? So to perform some estimates in order to to get these
18:25:440Alessandro Brighente: this information in here.
18:30:290Alessandro Brighente: Okay, so
18:33:660Alessandro Brighente: this is how the the system. Look like, right? So here you have the locations of the different features. Right? So these Qn's are the features, and these are the location in terms of x and
18:51:22Alessandro Brighente: why, right so, and you can save it, you know
18:55:470Alessandro Brighente: that's the other way on. This is X, and this is Y, and this is the location of one image right? So if I have a
19:03:950Alessandro Brighente: Ix all over these featuring here.
19:08:780Alessandro Brighente: and Iy defines the the coordinate
19:12:390Alessandro Brighente: of these other features here, right? And of course, these move along a certain direction in time. T, right? So you have displacement at time. T. You detect them at another location.
19:26:530Alessandro Brighente: and you want to estimate the speed of the drone, thanks to the displacement of these features. Right? So you need to solve this system of the question here. This is like the speed of the drone in terms of X and Y coordinates right?
19:43:650Alessandro Brighente: And you have to solve something like this system in
19:55:530Alessandro Brighente: this system in here. Right? So you need to determine the D value out of these locations. So you will need to to invert this matrix in here matrix a, but the matrix can be inverted only if it is squared. If it is so, it means that it has the same number of rows and columns. But if that's not the case, you cannot simply invert the matrix right? You cannot multiply the inverse of the matrix.
20:20:240Alessandro Brighente: So you need an algorithm to solve this problem. And one of these algorithm is the the square algorithm.
20:29:390Alessandro Brighente: yeah, I I'd say that is
20:35:620Alessandro Brighente: it. For today, we'll see an example, how these works
20:39:900Alessandro Brighente: on Friday. Practical example of how to solve this problem.
20:44:600Alessandro Brighente: Thank you.
20:45:800Alessandro Brighente: Thank you for that.