Assistente AI
Trascrizione
00:01:390Alessandro Brighente: Oh.
00:02:930Alessandro Brighente: hey.
00:06:120Alessandro Brighente: okay. So back to the spoofing of lighters. Right? So
00:14:610Alessandro Brighente: Last time we discussed we have these sensing devices on cars, autonomous vehicles in general, which are called lidars. Right? It sends these laser pulses, and they get reflected to an object, and then we can measure how much time they take to to get back at us.
00:31:220Alessandro Brighente: And then we have
00:33:360Alessandro Brighente: sensing of the environment, right at least surfaces that we see
00:38:753Alessandro Brighente: so we've seen that we have some effects that we can achieve if we try to spoof these these signals, right? We need to send back to the lighter a signal that it expects with a certain with a certain power, and thanks to that, we can
00:55:560Alessandro Brighente: cause blinding right, we we make the lighter, not able to see objects that are in the surrounding or
01:03:330Alessandro Brighente: the vehicle that mounts them.
01:05:960Alessandro Brighente: Then we said that not only we can do blinding attachment we have blinding. It means that we point the light source towards the lighter, strong enough to separate the sensor right? We bring the sense of the separation region. Basically, we're not able to sense anything anymore. That's not the only thing we can. We can achieve. Right? We said that.
01:28:50Alessandro Brighente: hey, we can. We can have a strong signal, or we can capture signals, valid signals and then replay them towards the lighter to the lighter.
01:40:450Alessandro Brighente: such that we can create objects that do not exist. Right? So we have the spoofing by the name
01:47:604Alessandro Brighente: So
01:48:630Alessandro Brighente: so what happens? Right? So let's talk about the idea, scooping scenario. What would we like to achieve? Well, what we would like to do is, for instance, create an object in front of the car. Right? So ideally. We know the location of the car. We know, the location of the object that we want to spook. We know the shape of the object that we want to spook.
02:06:480Alessandro Brighente: and therefore we know how much time it should take for a legitimate signal to be reflected on. These object. Right? So we can do the the whole math and and basically replay the signal whenever it's needed.
02:24:570Alessandro Brighente: Good. So what are the different steps of these ideas moving process. So when the 1st thing, of course, is to have a tool that allows us to to perform the attack right, of course, we need something that is able to to measure
02:38:27Alessandro Brighente: or to receive a signal to capture the signal, to send the signal, and, of course, that it allows us to to control the delay that we want to impose on the reflected signal. Right? So how much or how fast, should we replay the signal back to the legitimate light, right? And of course it needs to be at the same wavelength of the legitimate light. And so
03:02:230Alessandro Brighente: great. So
03:04:470Alessandro Brighente: then, of course, we need to aim at the the victim, which is the lighter, and then, as soon as we receive a policy from the legitimate Lidar, we know how much time
03:19:470Alessandro Brighente: how long we want to wait before responding back to the lighter right? So we add our delay component to the signal, and then fire it back to the legitimate liar.
03:30:500Alessandro Brighente: Good so ideally this process create a unique dose, right? So for every laser pulse that we send from the ladder, and that gets reflected on the object and back to the ladder itself. We have a unique dose. And for this reason we said, Okay, we need to scan the whole 360 degrees
03:50:980Alessandro Brighente: on the vehicle, and we also need to control the the Asympte angle right the elevation angle to cover a sufficient space and get sufficient amount of points that are representative of the the environment.
04:04:394Alessandro Brighente: So ideally, this happens right? It depends on the the physical configuration of the dialer, and then the other component that we need to control. Is this the delay? Right? We said, that we want to create a certain object or hide a certain object, move them away from from the car. Right? Create some?
04:26:210Alessandro Brighente: some. Let's call them ghost objects right? And for this reason we need to compute
04:30:850Alessandro Brighente: the delay right? So ideally. The delay is just the time it takes from the laser pause to get from the ladder to the object and back.
04:38:960Alessandro Brighente: Well, but that's that's not actually true. Right? We we have some some computation that we need to to perform right? So we need to realize that there's a laser pulse incoming. We need to decide how much delay we want to add. And then we need to find it like this cost side, right? So we need to take that into account. So
05:01:558Alessandro Brighente: let's say that we want to to create an object right. A ghost object that is at the distant
05:07:300Alessandro Brighente: L.
05:08:780Alessandro Brighente: From the from the car. Right? You want to be in the object.
05:15:169Alessandro Brighente: And
05:16:680Alessandro Brighente: what we? You have some math in here just to to show you so basically, we said that the distance between the scooper and the victim Lidar is this Ls, which is lower equal than L,
05:30:620Alessandro Brighente: and then we want to generate a signal at a delay that appears L minus Ls. Further away than the spooker.
05:41:740Alessandro Brighente: and so we need to generate the entry time for for this distance. Right? So we have this L minus LS. Distance, and we know that the signal travels at the speed of light. So how much is the delay that we need to add on top of the signal in order to get to this distance while you have this thing in here, right? So from the physics point of view, this is correct. This is the way in which we describe the the inner working principle of the light. Right? Just a signal
06:07:650Alessandro Brighente: getting back and forth, and if you want to cover a certain distance well, we can do the math and get the delay
06:15:170Alessandro Brighente: good. So what are the limitations? If we just do this thing right? How?
06:22:150Alessandro Brighente: yeah, what are the shortcoming? So the the problems that we need to face when generating this kind of policy as well the 1st thing is that the attacker must be in the in the receiving angle of the of the victim. Lighter, right? So they need to face each other right? So what we said last time is that the lighter is rotating and has a specific receive window right? That can use.
06:48:70Alessandro Brighente: And this receive window is rotating with the lighter itself. So they did. The tracker.
10:12:370Alessandro Brighente: Okay.
10:15:250Alessandro Brighente: good. So the the same angle of the of the victim should face the attacker. And then the fact that if you remember the Lidar receives
10:25:660Alessandro Brighente: or accept signals only in a certain time window, right between the time that fires the laser pulse and and the receiving time we define these specific time window. So if we send the laser pulse back the spook one after this receiving window, then the lighter is not going to receive anything right. It's not going to to accept it as a embedded source.
10:55:240Alessandro Brighente: Good. So the other problem is, despite the fact that we said that ideally, whenever we send a laser pass, we just get a single dot from from the reflected object. That's not what happens in real life scenarios. Right. You are just firing a waveform. And this waveform is not confined in anything. Right? It's not.
11:18:920Alessandro Brighente: It's an electromagnetic waveform that is traveling in
11:23:960Alessandro Brighente: here, let's say right and actually, what you have is that these single waveform is is received by different objects, and therefore you might have reflections from different objects coming back to the lighter. Right so.
11:47:570Alessandro Brighente: And then you see both.
11:51:590Alessandro Brighente: except both.
11:53:560Alessandro Brighente: Let's say that these is your antenna right? And your antenna is sending signals right to with a certain pattern. And let's say that your button is done like this right or something similar.
12:08:390Alessandro Brighente: so you have the most power of the signal is limited towards this direction. She will have a strong signal in that direction. But it's not like you have just a single signal, right? You don't have just something very small that you point towards an object. And this is what happens. Similarly, also for the lighter right? So it means that the objects placed in here will all receive a part.
12:32:860Alessandro Brighente: And such waveform back to the line. Right? So you have something like this reflection. You have something like this reflection, and something like these reflections on so forth. Right? So you will have either multiple dots, considering a very small surface or multiple dots that are spread over larger areas. Right? It depends on the objects that you have on the front and how distant they are from from the source itself. But
13:03:155Alessandro Brighente: you get these reflections back right? So also from an attacker's perspective. This means that you need to take that into account. Right? So you can.
13:16:280Alessandro Brighente: scooping a single single object needs to adhere to this to this law in here. Right? Good. So
13:28:260Alessandro Brighente: this is something rather complicated from from a physics point of view. Right? So what happens if we neglect this thing? Right? So what happens if we forget about the fact that we have all this reflection, and we just want to scoop our object. And just and
13:44:360Alessandro Brighente: a single razor prospect. Right? What what would happen? Is it really that
13:49:770Alessandro Brighente: bad from an attacker's point of view?
13:56:340Alessandro Brighente: And so that's something that we'll see later on.
13:59:300Alessandro Brighente: And
14:01:260Alessandro Brighente: and the other thing is
14:03:940Alessandro Brighente: okay, we we are spoofing the signal. Right? So again, we said that we need to receive the signal. We need to realize that we want to send a Spoof signal back. And we need to to do our computations to
14:18:550Alessandro Brighente: to understand how long to take before sending back these pulse right? So we denote with this letter, as the time difference between the detection of the laser pulse and the flying back of the Spoof. The laser pulse was the the transmitter. Right? So
14:38:170Alessandro Brighente: our delay now is the thing that you see in there. Okay, so we have these these additional components, which is the processing time. Yes, right?
14:50:44Alessandro Brighente: Good. So we need to take them into account right? Something that you have also in
14:57:710Alessandro Brighente: illegitimate, the legitimate, either working
15:03:383Alessandro Brighente: but it's even worse when you want to compute something and do something
15:09:310Alessandro Brighente: to scoop the office. Right? You need to take this this processing into account. Good. So we can add these values.
16:00:410Alessandro Brighente: We're responsible.
16:01:240Alessandro Brighente: Scrolling.
16:02:390Alessandro Brighente: Yes.
16:17:20Alessandro Brighente: okay. So also, in this case, we have some commonalities between the the different spoofing strategy that we might implement right? So the the 1st thing, as we said for the for the blinding.
16:30:920Alessandro Brighente: we have something that is not visible to humans. Right again, we are generating some buses, firing them back to the laser, to the lighter. But it's something that is not visible to human eyes. Then we have the
16:47:810Alessandro Brighente: capability, the need to introduce multiple dots right? So this depends on the rotation speed on the lighter and on the receiving window. Right? So if we know that the lighter is rotating at the constant speed. We know that we have a certain time window in which we can fire these values back. it's something very different. So
17:10:180Alessandro Brighente: as soon as we get this period value, we know exactly when we can fire this these forces back.
17:18:500Alessandro Brighente: And then we have the the receiving angle right? The receiving angle is something that somehow limits the capabilities of the attacker. Right? We know that. We are only able to receive the signals back to the
17:35:40Alessandro Brighente: the lighter for a certain amount of time in a given specific receiving window. Right? So the smaller the receiving angle. The shorter is the the time that we have for us putting this value. And then the curve, the reception glass, right as we've seen
17:50:690Alessandro Brighente: last time we have this curved glass on top of the lighter, which reflects some of the lights that gets back to the lighter. So when we have to swoop objects in a specific direction. We need to take these inclination into account, this reflection on the the curved glass
18:06:870Alessandro Brighente: into account.
18:09:590Alessandro Brighente: Good. So what do we need that actually to to perform these this booking attacks. Right? What in terms of of
18:17:229Alessandro Brighente: the actual devices that that, we need so we need something that is able to detect these Lidar generated pulses. Right? Some photo detector of some kind
18:29:116Alessandro Brighente: and then photo detector is just this sensor that is able to to detect the presence of light and different wavelengths. Right? So in particular, it's also able to to sense something that is not visible and human eyes. And so it's useful for the lighter
18:47:560Alessandro Brighente: and good. So we have these incoming pulses that are coming from the legitimate lidar that captured by the photo detector, and they're converted into a series of 5 called pulses. Right? So we have these
19:05:220Alessandro Brighente: see
19:07:290Alessandro Brighente: repetitive time series of values right? And it's just 5 volts, and then zeros, and then 5 volts, and then zeros for reserve, certain amount of time.
19:16:750Alessandro Brighente: Good. So then, we have. These pulses are padded to a function generator, right? Which is triggered by the 1st received pulses. So as soon as the the laser pulse gets to the.
19:29:780Alessandro Brighente: to the attackers device. We have the triggering of this function generator, which what's the task of this function? Generator? Well, it waits for a predefined delay and then sends back a predefined number of copies to the transmitter.
19:47:880Alessandro Brighente: Okay, the transmitter means the actual device, the transceiver on the attacker's device is responsible to to finding back these laser passes.
19:59:70Alessandro Brighente: and we have the driver that lets the fire a signal. So the function generator is what is actually the the core of the attack, right? It needs to define how to modulate seamless, and how much to weigh in order to spoof objects at a certain distance.
20:19:600Alessandro Brighente: What is the results of these these attacks? Right? So again, neglecting these, this thing in here? What can we do?
20:31:200Alessandro Brighente: good. So you have 2 scenarios in here. You have. this is a room, but we have lighter, right? So the lighter is sensing some of the objects sensing the boundaries of rooms on the walls. Basically, right?
20:47:46Alessandro Brighente: So you see, the whole perimeter in here, and that's all. The objects are located inside the room, and then we have a scooter located in here. Right? It's just sending these fire pulses back. These laser pulses back to the liner, and is inducing some some fake dots right? So you see, these fake dots in here, so it's something like the lighter is able to sense things or objects that are located outside of the room, which makes no sense
21:16:700Alessandro Brighente: but due to the
21:18:860Alessandro Brighente: the spoof fake dots that the attacker generates. This is the actual map that the that the lidar gets right. So it senses something that is here. In this situation. Instead, you have that the attacker is also able to spoof objects that are
21:37:899Alessandro Brighente: closer to the lighter than its actual location. Right? So measuring these short term. Okay, so here is where you have the lighter. Here is the spoofer location, and here are the fake dots that the spoofer can actually take right? So the thing is ideally these lighters should send these.
22:03:73Alessandro Brighente: It's
22:05:550Alessandro Brighente: laser pulses towards this direction and gets these reflection right. Those from the board which are there. But then the attacker knows the very beginning of these laser passes towards this direction, and can shorten the delay that these Lidar is computing right? So it creates these playbooks in here.
22:25:930Alessandro Brighente: So in this scenario, we're creating a simple object. But but you realize the fact that if
22:32:900Alessandro Brighente: so not necessarily the number of points that you have in a lighter cloud is is so dense, right? You must just have some some small points that represent. I don't know a pedestrian or cone something on the road, and in this case, by inducing these.
22:51:550Alessandro Brighente: you can force, for instance, the car to suddenly break, because it thinks that something is on the road, and then it means that the people in the rear is going to to crash right? So you see, we are not creating weird, not weird, but complicated objects. Right? It's not something very.
23:14:180Alessandro Brighente: very strong from a dramatical point of view, in a sense, but still is something. Of course we have a more advanced lighter
23:24:740Alessandro Brighente: spoofing approaches.
23:28:143Alessandro Brighente: And they're based on the ways in which the lighter post processes these smaller points. Right. So you see also that in these situations what we've seen up to now, we're just
23:42:541Alessandro Brighente: detecting to the opposites. Right? We we are just checking the perimeter of the room we are checking, whether there's an object in the front. But we don't know anything about its actual shape. Right? That's something that we cannot actually know from the from the lighter point, cloud, etc. That's that's not sufficient.
24:03:870Alessandro Brighente: How do we hence
24:05:870Alessandro Brighente: get to the point where we where we detect the 3D object, and we can spook something better than just this is those right? So
24:17:650Alessandro Brighente: of course.
24:19:380Alessandro Brighente: We are going to use deep learning algorithms for this right? That's the thing. And I know you're not happy about this. But bear with me right?
24:31:350Alessandro Brighente: we are not going to develop deep learning algorithms. The point is
24:35:870Alessandro Brighente: all of these
24:39:260Alessandro Brighente: already deployed system, right? This autonomous system where you have the Apollo example in here. These are autonomous cars that are actually driving around in China, for instance, and they use a lot of computer vision and deep learning based algorithms to perform this stuff.
24:55:350Alessandro Brighente: Okay? So you see, what you're having here is, it's 1 of the examples.
25:00:790Alessandro Brighente: right? The point is, we have points in a 3D. Space that are not sufficiently representative enough to give us idea on the 3D. Space in the surrounding. But we need that
25:14:170Alessandro Brighente: good. So
25:15:775Alessandro Brighente: and the point for us, since we care about the the security of this stuff is okay, do we just want to create these sparse fake dots in here? Or do we want to create something more?
25:28:520Alessandro Brighente: Can we actually spoof an entire car? And can we actually spoof a group of cars? Well, what can we actually Spoof? How can we create a fake object that actually seem real right? And we cannot do that if we do not consider the post-processing of these this data that we collect the actually live.
25:51:870Alessandro Brighente: Okay? So we have different models for this.
25:56:400Alessandro Brighente: And
25:59:640Alessandro Brighente: we have basically 3 different categories. So the 1st one here is the bird's eye view
26:08:80Alessandro Brighente: that shouldn't be one and you based 3D object detection so you see that we have the point. Cloud is the things that we
26:20:420Alessandro Brighente: that we estimate, thanks to lighter right? And then we have these post processing modules that extract some some features right? And it provides them to these are coded feature maps. So we have the actual mapping of the features computed, thanks to the point cloud.
26:37:180Alessandro Brighente: And then we have our convolution, neural networks in here the confidence maps which brings out the results of the Cnn. And then the postpology model that tells us which objects have actually been detected right? Where are these objects, and what are these objects?
26:56:150Alessandro Brighente: The second implementation is this Boxer based 3D object detection. Right? We have a couple of well-defined models in here which are called Boxer net and point net which, basically, what they do is to map the point cloud that we get through the Lidar into these pillars. In here they create regions in space.
27:16:610Alessandro Brighente: and from the points located in each of these regions they apply these point nets to give to some significant features. Then we have the Cnn again, and then the final detection, the bounding boxes of the objects. And what these objects actually are.
27:33:206Alessandro Brighente: So you can imagine boxes as the 3D. Representation of pixels, right? So that the way I figured it out is like this Minecraft software, you have all the blocks right? And these are actually boxes. So basically, what you do is to segment your image in this way, create these these regions of the image and map all the the point clouds to these feature so these spaces of the image to to extract significant features.
28:02:340Alessandro Brighente: and then you have 3D. Pointwise 3D. Object detection.
28:08:350Alessandro Brighente: So in this case, we directly work on the point. Cloud, right? So you have this point, cloud, you give them to this structure, which is called point net. And then, we have a segmentation and
28:22:430Alessandro Brighente: feature, all the generation here a pooling layer and point metro, and then the final decision, base
28:30:630Alessandro Brighente: on the again the the bounding boxes for the objects and their nature.
28:37:540Alessandro Brighente: Okay. So you see that you have these 2 stations in here generate high quality region in the 3D. Space. Thanks to these point net, and then for each of the bounding boxes generating there extract features from these objects, and try to understand what these objects are
28:56:380Alessandro Brighente: good, so
28:57:790Alessandro Brighente: given that we know that these is how Li, there's
29:06:133Alessandro Brighente: how did
29:08:369Alessandro Brighente: the processing chain, if you want from lighter points to actual detection of objects work.
29:17:00Alessandro Brighente: How can we create these pain points? Right? So how can we create these big objects?
29:23:890Alessandro Brighente: Well.
29:26:390Alessandro Brighente: what we have access to is the lighter itself. Right? We can spoof values in the in the lighter domain. We can spoof these single pulses that get fired back to the light. But now the question is, how do we create these pulses? Such that the point cloud that we generate
29:45:430Alessandro Brighente: is
29:46:910Alessandro Brighente: deemed as an actual object by this whole chain in here. And how can we trick basically the computer vision algorithm into thinking that there's a spoofed bounding box that contains a car.
30:07:250Alessandro Brighente: And, of course, the way in which we can do this is through the other side of us, right? So we know that we can. We can create something in the future space right? And create these reflections from the lighter
30:22:860Alessandro Brighente: according to how the the classification algorithm works
30:30:890Alessandro Brighente: good. So
30:32:830Alessandro Brighente: so here you have a simple example on how that algorithm works. Right? So let's say that we have the legitimate point cloud.
30:43:200Alessandro Brighente: which is called X. And then here we consider the the Apollo pipeline. The bird's eye view with these features in here, which are coded, are coded features X.
30:56:300Alessandro Brighente: And then we have a function fee. That is the post-processing function
31:05:448Alessandro Brighente: that takes here, which is a feature map. And this of the point right? This here.
31:17:380Alessandro Brighente: and what we want to do is to basically design these deep values in here, the prime values in here, such that the decision taken by the algorithm is correct, right? It's something that we can control. It's something like the algorithm thinks that there's a non-existing object in here. Or it thinks that there's no object in that specific region. Okay?
31:49:430Alessandro Brighente: So in here, you have these, this function, which is this detection function, then you have the model right? In this case, you need to know how the model is built and how it behaves. And then here you have some constraints. Right? So you want to generate points that may sensor with respect to the attack capabilities that make sense with respect to the physics of the environment. Right? You cannot generate a random object, because if the object shape is so
32:15:410Alessandro Brighente: unusual, let's say then it's easy to detect that something is wrong.
32:23:440Alessandro Brighente: Good. So if you're interested in how these this attack is implemented on how the the whole computation works. You have the reference in here which is this is the 1st paper that proposed. These are the sensor attacks on the liar based perception. Okay, it's from 2,019. Not that old, though.
32:44:30Alessandro Brighente: and and it's very interesting. So what they did in here is to show that they were able to successfully attack these pipeline with just 60 cloud points
32:56:170Alessandro Brighente: and an 8 degree horizontal angle. Right? So you just need to to create 60 points in the whole cloud to spoof an entire vehicle. Right?
33:07:450Alessandro Brighente: Mean?
33:08:660Alessandro Brighente: This is
33:09:920Alessandro Brighente: facing. I don't need it's
33:13:60Alessandro Brighente: it's almost much right. It's how can they be able to do something like that? Right? So usually here you have some numbers. So a valid front vehicle, right? So I'm a vehicle, and I'm sensing a vehicle in the front. Usually the number of points that they need to represent, that the vehicle is 2,000 points and 15 degrees horizontal angle. So you see that it's
33:39:170Alessandro Brighente: almost twice
33:41:390Alessandro Brighente: and 60 compared to 2,000, right? So so what's happening in here?
33:46:330Alessandro Brighente: Well, the problem in here is that if we just consider
33:52:640Alessandro Brighente: the creation of these points without checking the physics of the environment.
33:58:980Alessandro Brighente: then we can create some, some false, positive right? So we cannot simply attack the the machine learning algorithm, create this in these examples
34:07:340Alessandro Brighente: and then deem them as value. So what do we need actually to create something that is valid.
34:19:30Alessandro Brighente: As I promised before, we are not going to see the details of the machine learning algorithm, because that's not the point. The thing is, how can we actually spoof these values? So how can we? How can we create something that is deemed as valid by the lighter? And therefore, how can we prevent this attack from jeopardizing the safety and the security of our product?
34:44:719Alessandro Brighente: Good? So where does this factor of only needing 60 points to super people come from.
34:52:540Alessandro Brighente: So we have 2 situations in here where we just need a smaller number of points in order to to represent a vehicle.
35:00:760Alessandro Brighente: So the 1st one is the occluded vehicle. Right? It means that I'm sensing a vehicle that is located behind the 1st object that I see.
35:12:100Alessandro Brighente: It's something like I'm in the car. I'm sensing this car in front of me. But together with that, I'm also sensing this device in here because some of the laser passes that I sent through. My lighters are reflected also by the object behind, but the object behind will only reflect the points that they see right. So
35:40:810Alessandro Brighente: if I am this card being here, then I have another card being here. Let's say that I have a truck speaker
35:49:654Alessandro Brighente: in front. Right? So of course I will censor this time here, because I will get these signals reflecting here, but at the same time I will also censor
35:59:370Alessandro Brighente: this thing in here, but I will only see these points right. I will not see other points, because they are behind the 1st speaker that is front of me. Still, these points might be sufficient for me to detect the presence of a vehicle. Okay? So in this sense, we know in this situation we are talking about occluded figures right? Something that we partially see.
36:21:420Alessandro Brighente: Why would we need the the other dots that we don't see from the car in front of us? Because from that car in front of us. We can estimate the
36:33:808Alessandro Brighente: so in this, in this sensing, sorry
36:37:460Alessandro Brighente: what you have with riders. It's not something like, okay, you only limit myself to the kind of phone. I'm sensing the environment, everything right? And then, based on that, I can take decisions the fact that I see these documents in here
36:53:00Alessandro Brighente: to me to present the fact. But then, what to do with the with that information is something that is different from what we've seen before, for instance, in terms of not just interested in this, in the, in the fact that there's a bank under me. But instead.
37:07:490Alessandro Brighente: Okay? And
37:10:470Alessandro Brighente: the the problem in here is that your algorithm, your partner, is detecting a car right? And in this situation it's not a big problem. I mean, it's sensing a car, whatever
37:22:700Alessandro Brighente: the problem is. What if I use these 60 points
37:28:380Alessandro Brighente: in here?
37:29:710Alessandro Brighente: Right? And we let the car think that it's closer to a vehicle that doesn't exist
37:34:170Alessandro Brighente: right? Because in in the end these 60 points, I mean, the card does not realize that there's an occlusion. It just sees these 60 points and attacks a bit from. So it would just break not to.
37:45:580Alessandro Brighente: You can actually there with a beacon in the phone.
37:48:400Alessandro Brighente: Okay? So this brings us back to the scenario. Right? We need, do we? We send something that does not exist?
37:57:340Alessandro Brighente: and it's a problem. We're still very grateful.
37:59:700Alessandro Brighente: Okay? So again, the problem is, we have these 2 scenarios in which a smaller number of points
38:08:150Alessandro Brighente: can use to spoof a vehicle that does not exist. Okay? So in this case we are talking about occlusion. And the other case, we are talking about the distant vehicle. Right? If we have a vehicle that is far away somehow. Then we would just see
38:23:890Alessandro Brighente: a smaller number of points. The number of days deposit will be reflected, is actually smaller, right? So the whole working principle of the Slider is similar to what you see with the with your eyes. Right? So you can. You can see it through the dots right. I can recognize by looking at the water bottle that behind that there's a laptop, right? I mean, I could do that also without knowing that it's my laptop.
38:48:250Alessandro Brighente: but I can see that there's a letter right? And at the same time, if I look on the other side of the street, I can see cars in there right? But they're just very small. Exactly the same thing happens with the point clouds in light. If I see something that is far away. I would just see a small number points with that. But at the same time I'm able to recognize that these small number of points represent a current.
39:15:350Alessandro Brighente: Maybe I'm just not creative enough. But
39:17:550Alessandro Brighente: what is bad about creating a car? That is a faraway thing.
39:22:790Alessandro Brighente: Oh, so this might. So what we've seen up to now is just the fact that I can break or speed up if I send something close to me. But if I need to to plan my whole.
39:40:800Alessandro Brighente: then it might be good.
39:42:550Alessandro Brighente: No, be good.
39:46:490Alessandro Brighente: Okay.
39:49:550Alessandro Brighente: So a break.
40:14:319Alessandro Brighente: Hmm.
40:16:160Alessandro Brighente: okay. So
40:19:910Alessandro Brighente: another consideration on the distributing. Right? So.
40:25:600Alessandro Brighente: okay.
40:26:630Alessandro Brighente: so this is very
40:28:380Alessandro Brighente: subtle, somehow, in the sense that
40:31:890Alessandro Brighente: when we say that we detect vehicles right through our computer vision, pipeline right? The output. We just have the information. There is a vehicle right? We don't have the information. There is a distant vehicle. There's an occupied vehicle. There's
40:45:260Alessandro Brighente: I don't know what kind of vehicles. Right? That's not the information you get. So, despite the fact that we might create a point cloud that looks like a distant vehicle or a point cloud that looks like an included vehicle.
40:59:550Alessandro Brighente: Well, the computer vision algorithm will will output is the presence of the vehicle. Right? So even though I might spoof a distant vehicle and spoof it here. Right. Use the point cloud of a distant vehicle in here. The vehicle would just detect the presence of the vehicle, it would know know that it's distant itself, right? So
41:18:410Alessandro Brighente: this has also impact on the on the Ccc scenario, for instance, that we we were discussing because
41:24:440Alessandro Brighente: the computer vision algorithm sees a vehicle right? And then it needs to take action based on that. Okay, so we we create this distinction because we know the situations in which we need a smaller number of point clouds points in our cloud in order to identify a vehicle. But the computer vision algorithm has no idea on the
41:46:90Alessandro Brighente: right, it would just detect the presence of a vehicle.
41:51:210Alessandro Brighente: Good. So in terms of occlusion. So what do we have in. In what kind of occlusions can we have?
42:01:63Alessandro Brighente: So we have 2 different patterns. The 1st one is inter occlusion, right? It means that we have an occluder and an occluding, and the occluder causes the Occlude to be partially visible right? And so what is the post-positive condition associated to that so post-positive.
46:40:30Alessandro Brighente: Right? Because
46:44:990Alessandro Brighente: this this thing about multitasking is something that I never understood cannot do that. And went saying, whatever So
46:54:920Alessandro Brighente: we know that these is something that an attacker can exploit right? Exactly. What does the adversarial machine learning algorithm do create points that have these specific shapes. Right?
47:08:220Alessandro Brighente: Good. So again, we don't care about how to attack the machine learning algorithm itself. We we care about the fact that this is a problem and something that the lighter cannot understand. Right? It doesn't know whether the the object is distance is a clue, that it's whatever it just sees a point cloud that is classified as an equal grade. So
47:32:830Alessandro Brighente: how do we deal with this? How do we
47:36:330Alessandro Brighente: counteract with these other side of the tax, meaning that we know that these situations might occur right. We have these very nice system here, which is called Kabul.
47:46:820Alessandro Brighente: which stands for occlusion over here, anomal detection, right? And what is the idea behind? So it's the fact that we can use this physics related phenomena in order to detect whether something has been spooked right, whether we are seeing something that makes sense with the physics or it does not make sense. Right? So we can actually detect Spoof, the fake vehicles
48:11:900Alessandro Brighente: right? So in order to perform this detection, we have these 2 building blocks in here. The 1st one is free. Space detects 3 States detection, and the second one is laser penetration, detection. Right? So you can imagine what these 2 building blocks mean. The 1st one is the second, whether there's free space, and whether the free space that we actually measure makes sense with the physics of vehicles, and
48:36:60Alessandro Brighente: actually, instead deals with the
48:40:240Alessandro Brighente: which kind of reflections we get from a specific area right? How can these?
48:49:819Alessandro Brighente: Then
48:51:450Alessandro Brighente: let's make an example. So if I'm sending laser towards the window, right? If the window is open, I will just have some reflections from the wall and the
49:03:500Alessandro Brighente: structure of the window, and I will have no reflection from the open window. Right? So that will make sense. If I know that I'm looking at an open window, right? So I can perform some some measurements on that as well. I will get no reflection.
50:33:40Alessandro Brighente: That's right. It's just the
50:35:210Alessandro Brighente: the actual object detected by the grid. So it tells us, okay, in these specific boxes there's a car, or in these specific boxes, whatever. Right? And inside these bounding box we have different reflection patterns. Right? It might be the case that part of the Resors just go up to the other side of the bounding box, or that I do not get any reflection from a specific
51:04:690Alessandro Brighente: portion of this one inbox
51:09:420Alessandro Brighente: right and
51:12:530Alessandro Brighente: And then what do we have? Great? So
51:15:370Alessandro Brighente: this thing in here is called the process, right? So basically, what happens is that if we define this bounding box, and you point our lighter towards that bounding box. We would just have reflections
51:27:930Alessandro Brighente: within these prostamine here, which is a
51:31:610Alessandro Brighente: 3D. Version of a cone if you want right? So it defines a structure in 3D. Space, right? So we need to build with these points. And how do we compute the free space in these points? Well, if I'm pointing at a certain direction and not get any response from that direction, I can leave that as free space right? There's nothing close enough to cause a reflection that I can sense in my detection time. So for me, that is free space.
52:00:590Alessandro Brighente: If, instead of in this box. I sense something right. There's this thing here might be a car, for instance. Then I get the reflection from from the command right. I get some some points that belong to my my cloud, and then tells me that there's an object in there.
52:16:820Alessandro Brighente: So in this case, what we have is that the the?
52:23:660Alessandro Brighente: It's really easy right to detect the free space in a certain direction. Right? We just simply don't get the response great. So how do we? How do we use this thing now?
52:41:290Alessandro Brighente: So if we think about these this bounding box now, and the space where the reflection are incoming from we can perform this simple separation that we said before, right, we have the free space, and we have the Occlude the space. This is the space where there's an object.
53:01:530Alessandro Brighente: So the 1st one, we can say that the the free space is embedded at the point level
53:08:730Alessandro Brighente: in the sense that we don't receive any.
54:50:300Alessandro Brighente: If you're detecting our religious policies and there's no targeting there, we can. We can see that basically, the the amount of 3 States
54:59:580Alessandro Brighente: is upper bounded by these. These value B in here. Right?
55:09:140Alessandro Brighente: it's distribution is of the value, right? So it means that the maximum free space, if they get in a certain bounding box has a maximum value, right? It will not get the
55:21:130Alessandro Brighente: with the sufficient probability higher than that.
55:27:570Alessandro Brighente: And instead, what happens when we have a fake. Vehicles. Well, these fake vehicles do not obey this occlusion. Button right? The the fake vehicles can have shapes that are not reasonable. Where's the towards the occlusion button? And what happens is that instead, for fake vehicles there the distribution is lower bounded by these value a in here. Right? So
55:57:20Alessandro Brighente: this is something that we get from from our
55:59:360Alessandro Brighente: right. So if we if we use actual lighters data, and we see the distribution right? We, we count how much space is included and how much space is free in different boundary boxes. We get these distributions. Okay? And so how do we? How do we use this distributions? Well.
56:19:867Alessandro Brighente: when we want to to detect whether a certain money boxer has
56:24:890Alessandro Brighente: contains spoof values
56:27:403Alessandro Brighente: we can compute our ratio right? So our ratio is this thing in here this ratio tells us. If we define if we divide the bounding box in certain number of cells in smaller pieces.
56:40:840Alessandro Brighente: and we count how many cells are free over the total number of cells that we have in this funding box. Well, this should be in a specific range, right in a specific range, that is the one defined by the legitimate statistic, if it does not belong that range, and with a certain probability we can say that that value has been
57:05:320Alessandro Brighente: right. So
57:08:700Alessandro Brighente: great, this is work. So as long as we have something like these suppression values A and E are in this relation. So we have a which is greater than d, then we have this chances over a probability to detect these through statistical means.
57:27:110Alessandro Brighente: Right? So what
57:28:750Alessandro Brighente: do this distribution look like? So
57:32:580Alessandro Brighente: we have a data set which is called it?
57:36:10Alessandro Brighente: Yeah and good. So what do we have in this data? Says, we have a lot of point clouds, violate lighters.
57:46:740Alessandro Brighente: Okay? And we have them for different situations. So what happens now we can perform our estimation right? So our estimation is over these values in here, right? And this is the distribution that we have for our F right the ratio that we said before. So the ratio of 3 space points over the total number of points that we have in our plot
58:14:230Alessandro Brighente: good. So this, again, is a distribution of these values ratio. And you see that it's actually at the bound. Right? It starts from 0. The ratio might be 0. And it can get up to to this point here, it cannot get higher than that.
58:31:340Alessandro Brighente: Now, what happens is that if we take the same data set right, run our adversary machine learning algorithms and generate the the fake point clouds that
58:44:170Alessandro Brighente: that do not care about the physics of this occlusion pattern well, what we observe is that
58:49:410Alessandro Brighente: state of the art algorithms.
58:52:210Alessandro Brighente: This is the distribution that we get right. So this F prime is the the range of values computed for the attackers generated points.
59:01:290Alessandro Brighente: And you see that we have this thing in here. So it's lower bounded by A, but the A value is higher than B. So why do we care about this? A value being higher than this? Because it means that the darkness distribution is confined in this region, whereas the D distribution is confined in this other region, and if these 2 distributions will also overlap, then we don't have
59:27:110Alessandro Brighente: undecidable situation, right situations in which we don't know exactly to which of the 2 cases the the
59:37:620Alessandro Brighente: our situation belongs to.
59:41:360Alessandro Brighente: Okay. So this prespace detection is nice. Right? So we just point our lighter towards a certain direction.
59:52:430Alessandro Brighente: Again, perform our computation performer the ratio, and see where it belongs to.
59:59:880Alessandro Brighente: I thought this seems a nice solution, right? It works
00:05:272Alessandro Brighente: the problem is that it's very time consuming right? So because you need to perform
00:10:640Alessandro Brighente: ray casting on all the possible bounding boxes in real time. So it means that you have your computer vision algorithm that detects a bounding box. And at the same time you should be able to send all these rays words, a specific bounding box where you detected an object and get the response as soon as possible, because you want to know now whether there's an object or not. And this is
00:32:790Alessandro Brighente: no
00:34:870Alessandro Brighente: possible right? I mean, it's possible. But it's very resource, intensive.
00:40:710Alessandro Brighente: And it takes a lot of resources to do this in real time.
00:44:60Alessandro Brighente: Good. So we know that this is nice and it works. But let's see whether there's an alternative to be something that is more likely. So that's why we have these laser penetration, detection stuff.
00:57:850Alessandro Brighente: Okay? And so what do we have? And how do we compute this laser penetration? Detection value.
01:05:260Alessandro Brighente: So the 1st thing to remember is that we are firing these voices right? And we're getting points back to these. This reflection.
01:14:640Alessandro Brighente: And
01:17:340Alessandro Brighente: what do these points represent? Actually they. They represent a boundary right. They represent a boundary between the outside object of the object and the inside of the object or the front and the back of the object. Right? So we just divide the space that we can see and the space that we cannot see
01:37:560Alessandro Brighente: good. So,
01:39:760Alessandro Brighente: how? Based on these? This idea define different spaces in the bounding boxes defined by our computer vision algorithm, right? So we can define these 3 different spaces that you see in here. So we have the 1st space, which is the space between the Lidar sensor and the bounding box, right? So the
02:10:340Alessandro Brighente: there
02:11:360Alessandro Brighente: and home thing
02:13:960Alessandro Brighente: to the among the object. We have our mounting box
02:18:190Alessandro Brighente: right? So we have 1st
02:21:630Alessandro Brighente: space in here, which is the space within the lighter and the object, the sensor object bounding box.
02:28:220Alessandro Brighente: Then we have the spacer inside the bounding box, right? Exactly the space that defines the object, and then.