AI Assistant
Transcript
00:19:450Paolo Guiotto: I don't know.
00:28:420Paolo Guiotto: Fantastic.
00:32:439Paolo Guiotto: Okay, good morning.
00:52:390Paolo Guiotto: The topic of today is orthogonal projection.
00:56:480Paolo Guiotto: That arises from an important applied problem, which is called the best approximation problem.
01:05:730Paolo Guiotto: So Let's introduce this.
01:10:450Paolo Guiotto: Best.
01:12:390Paolo Guiotto: Approximation.
01:16:350Paolo Guiotto: program.
01:19:660Paolo Guiotto: We have a vector space V.
01:23:30Paolo Guiotto: well, actually, a norm space V.
01:33:710Paolo Guiotto: a vector F.
01:35:970Paolo Guiotto: In this space V, and a subset U of the space, normally it is a subspace, so a vector space included in V, so U contended in V.
01:50:270Paolo Guiotto: sub… space, huh?
01:54:940Paolo Guiotto: So this means that, U is itself a vector… space, huh?
02:07:990Paolo Guiotto: contained.
02:13:30Paolo Guiotto: in… with the same, of course.
02:16:800Paolo Guiotto: operations of sum with same scalars, with same operation of product by scalar. So it's just a subset, which is linearly closed, no? When you take linear combination of vectors of you, you have a vector of you.
02:31:570Paolo Guiotto: Now, the problem is that you want to determine the best approximation of F
02:39:60Paolo Guiotto: in this subspace U, so the problem can be formalized in this way.
02:47:00Paolo Guiotto: determine…
02:49:780Paolo Guiotto: The best approximation means that you will assess the distance between F and U for U in this subspace, and you try to minimize this distance.
03:07:150Paolo Guiotto: Now, in general, this problem is quite complicated, and it has not a solution.
03:13:110Paolo Guiotto: in a general normal space, but in a Hilbert space, there is a nice theorem that shows that under a natural assumption on U, we have a unique solution, and moreover, this solution
03:27:360Paolo Guiotto: can be characterized by a certain, equation, okay? So… and the geometrical idea is, simple. So, if,
03:40:130Paolo Guiotto: V.
03:42:210Paolo Guiotto: Peace.
03:43:760Paolo Guiotto: Hilbert.
03:45:740Paolo Guiotto: space, huh?
03:48:340Paolo Guiotto: We can.
03:51:490Paolo Guiotto: Rouve.
03:53:770Paolo Guiotto: Existence and uniqueness.
04:03:760Paolo Guiotto: And also characterization.
04:09:690Paolo Guiotto: characterization.
04:18:680Paolo Guiotto: off.
04:19:839Paolo Guiotto: D.
04:21:390Paolo Guiotto: solution to… these… problem.
04:27:380Paolo Guiotto: Well, intuitively, and that's what is nice with the
04:31:340Paolo Guiotto: this structure with the scalar toad. Intuitively, we have, we can imagine the situation like this. Imagine as the space… the space H is R3, no? And the subset U, the subspace.
04:47:620Paolo Guiotto: is a plane in the three-dimensional space age. So, we can imagine, so if this is the zero, you will be sort of plane…
04:59:220Paolo Guiotto: contained in…
05:01:950Paolo Guiotto: Part 3, and we have a vector. Of course, the problem is trivial if the vector is already in U. It is non-trivial if the vector is out of U. So let's say that this is the vector F, and let's use the
05:16:580Paolo Guiotto: Geometical representation with arrows.
05:19:970Paolo Guiotto: Now, you are looking for the best possible approximation of F through a vector of U. So this best approximation will be a vector somewhere here. Let's say that if you pick a generic U here, you have this vector.
05:39:220Paolo Guiotto: This is, you.
05:41:860Paolo Guiotto: And the distance between these two is the quantity, so the norm of this vector here is F minus U. Its norm, norm of F minus U is the quantity that has to be minimized.
05:56:50Paolo Guiotto: So, if you look at the geometry, this should be done by the vector, which is about here, which is the orthogonal projection of F on the subspace U. We will give a name to this vector.
06:12:350Paolo Guiotto: we call PU of F.
06:14:790Paolo Guiotto: Which is this vector.
06:17:40Paolo Guiotto: And the main feature of this vector is that… this orthogonality condition. That means that this
06:24:100Paolo Guiotto: green arrow here is perpendicular to U. We can express this and say that this green arrow is… this green vector is F minus PUF. So what should happen is that when I take F minus PUF,
06:43:10Paolo Guiotto: F minus its orthogonal projection, this is perpendicular to every vector of the space U, to every u of capital U.
06:53:540Paolo Guiotto: Or, in other words, we have this equation, F minus PU. F scalar U equals 0 for every U in capital U.
07:06:140Paolo Guiotto: So this is what we expect, no? There exists a vector, which we will call PU of F, and we will name it orthogonal projection of F on U,
07:19:340Paolo Guiotto: which is a vector of U, and it is the vector at minimum distance to F. And this condition is characterized by this, let's say, equation for the vector PUF. So you have to look at this as an equation for PUF.
07:36:690Paolo Guiotto: Now, there will be some assumption. I said here, I already mentioned one important assumption, is that the space must be an infield space.
07:49:400Paolo Guiotto: And moreover, we need an assumption on the nature of the subspace U, because otherwise this problem might not have a solution, okay? So, let's say that even
08:05:740Paolo Guiotto: if, our space B is… a hill bed space.
08:15:740Paolo Guiotto: this… problem.
08:19:820Paolo Guiotto: might.
08:22:400Paolo Guiotto: Not.
08:23:880Paolo Guiotto: kind of… a solution.
08:27:280Paolo Guiotto: I want to show you with an example this.
08:30:910Paolo Guiotto: Now, the example we do is the following.
08:34:140Paolo Guiotto: So let's start with the space feed. We said that, basically, there is a unique prototype of a Hilbert space, which is a space L2.
08:44:680Paolo Guiotto: The simplest possible L2, of course, there is a finite-dimensional L2. In the final dimensions, this fact is true. Let's say, in finite dimensional, simplest version of L2 is what is called the little L2.
09:02:690Paolo Guiotto: What is this little L2? It is nothing but L2, the big L2, the standard L2, where the set on which we consider as domain of the function is the set of natural
09:16:160Paolo Guiotto: The sigma algebra is the sigma algebra of all the subsets of naturals, and the measure is the counting measure.
09:28:920Paolo Guiotto: In this case, what happens is that the counting measure is a measure that, I remind you that nu of e is the number of elements of E.
09:41:290Paolo Guiotto: So, in other words, it counts 1 every time that you have a point into E, no? The set E is a subset of naturals, so it can be either finite, and then you get a finite value, or an infinite subset of naturals, and in that case, you will get value equal plus infinity.
10:02:870Paolo Guiotto: the integral of, on… if I write integral on n of f with respect to this measure.
10:11:500Paolo Guiotto: I can always decompose this integral as the sum, since in this particular case, I can decompose the domain into this joint union of singletons, accountable this joint union of singletons, so I can write this is the sum for N
10:27:760Paolo Guiotto: going to 1 to… from m to infinity, I just integrate on this singleton n my function F, the mu. Well, now, the set is made of one single point, so the value of F is now constant on that domain, and it is F of N.
10:45:540Paolo Guiotto: And then you have the measure of the singleton, mu of n, which is always equal to 1. So, at the end, you see that the integral of this function is the sum of Fn.
11:00:880Paolo Guiotto: And in fact, a function F, a function F defined on n real, or C-valued, let's stay on the real case.
11:10:510Paolo Guiotto: is what is identified by a sequence of values FN with the N natural, because if you take the sequence made by the values taken on the integers from the… by this function, you get a sequence of numbers.
11:30:20Paolo Guiotto: So basically, the little l2 is, can be identified as a set of sequences that usually we write with an index, sequences with a natural index, such that
11:44:490Paolo Guiotto: the condition to be in L2… well, there is no measurability condition, because everything is measurable, so the set of the measurable class is made by every set, so every function is measurable, yeah.
11:57:900Paolo Guiotto: So there is no condition that comes, no restriction that comes from measurability. And then you should put integral on n of modulus F squared, then you find it.
12:11:670Paolo Guiotto: But what is this? Well, we just say that an integral is just the sum of the values, so this is the sum of n of modulus Fn squared, find it.
12:23:740Paolo Guiotto: So, this is,
12:26:200Paolo Guiotto: what is this little L2, which is a useful space to do, basically, examples. Sometimes can be used also to do the models, because it is a discrete set, it's not…
12:41:90Paolo Guiotto: A continuum set, so integrals reduces to… to sums, to discrete sums.
12:50:80Paolo Guiotto: And also, you see that, in some sense, XAT2 is an extension of an RB space.
12:58:600Paolo Guiotto: No? V is a set of, vectors with D components, and, well, of course, the sum of the squares of the components is 0 is 5, and the normal, the norm is the square root of that quantity.
13:12:780Paolo Guiotto: If you send this D to infinity, these arrays become sequences, infinite sequence of numbers, and the conditions are attributed to find it. The scalar product between these, these objects, so between a sequence FN and a sequence Gn, the scalar product.
13:35:30Paolo Guiotto: of this little L2 is just the sum, if we are in the real case, as we are here, FN times GN, okay? So this is the scalar problem.
13:47:570Paolo Guiotto: Okay, now, this is the, the environment.
13:52:00Paolo Guiotto: Now take this F and this U.
13:55:140Paolo Guiotto: take as F,
13:57:230Paolo Guiotto: this vector, the vector made by the sequence 1 over n, so let's start from 1, or if you want to start from 0 plus n plus 1, okay? So, since naturals, we always think they contain 0, we take 1 over n plus 1.
14:13:420Paolo Guiotto: Notice that this F is an element of little l2, because… Because we need to check that sum of n of models Fn squared is finite.
14:28:80Paolo Guiotto: But what is this sum? In this case, this is the sum for n going from 0 to infinity of Fn is 1 over n plus 1, so to the power 2, it becomes 1 over n plus 1.
14:41:10Paolo Guiotto: square, which is the harmonic sum, sum for N, going from 1 to infinity 1 over n squared, which is well known to be fine.
14:51:140Paolo Guiotto: So this is a vector of this space. Now, take the set U, Made by sequences, say, UN,
15:05:520Paolo Guiotto: with these conditions, that this sequence are, definitely equal to zero. So, let's say that,
15:12:740Paolo Guiotto: the UN is equal to 0 for n larger than some capital N that will depend on the sequence, so for which there exists an initial index, such that UN is equal to 0,
15:25:770Paolo Guiotto: starting from a certain capital N, no? So, elements of this U can be, for example, a vector like 1, 2, 3, 4, and then 0, 0, 0, forever. This is a vector of U.
15:40:130Paolo Guiotto: No?
15:41:420Paolo Guiotto: And in particular, you can see that the vector F is not in U, because the vector F has all components different from 0, so this F is not in U.
15:53:540Paolo Guiotto: So now, the problem of determining the elements of U at minimum distance to F makes sense. So what should be the minimum
16:05:370Paolo Guiotto: of distance norm of F minus U when U is in capital U.
16:13:110Paolo Guiotto: Well, now we showed that this minimum cannot exist. So in this case, the problem is not a solution, okay?
16:23:70Paolo Guiotto: The first remark is that if we take what always exists, you know that when you do a minimum on a set which is not a finite set, it is always dangerous, no? You cannot be sure that the minimum exists unless you have particular conditions, etc.
16:40:40Paolo Guiotto: So what, in any case, exists is the surrogate of minimum, which is the infamum, no? So we can talk about infamum, which is the best lower bound of these quantities. It is, let's say, the moral minimum, but the unique difference is that we are always sure that this exists.
17:01:30Paolo Guiotto: So, these… Always.
17:07:250Paolo Guiotto: exists.
17:09:819Paolo Guiotto: Now, what is the value of this? We can easily show that this infamum is zero.
17:17:540Paolo Guiotto: So, let's check.
17:23:849Paolo Guiotto: Dead.
17:25:190Paolo Guiotto: let's call… let's give a name to this number alpha. Alpha is equal to zero.
17:30:710Paolo Guiotto: So it is sufficient to show that I can find vectors in U,
17:36:980Paolo Guiotto: whose distance to F can be made arbitrarily small. And what are these vectors? Well, just take U, let's call, take, I don't know, take, U…
17:51:930Paolo Guiotto: So, I don't want to do a mess with the letters here. Now, because I'm using the letter N for indexes.
18:00:200Paolo Guiotto: in these arrays, perhaps I should have used another index. So now, let's use UK, the vector made by 1, 1 half, 1 third, 1 fourth, etc. When we arrive to 1 over K, then we have 0, 0, 0.
18:18:210Paolo Guiotto: Until the end of this array.
18:20:930Paolo Guiotto: Now, this is an array of type of vectors of U, no? It is definitely equal to zero, so this belongs to U.
18:29:640Paolo Guiotto: And what is the distance between F and this UK?
18:34:570Paolo Guiotto: Well, the distance, let's take the square. Since we are working on an inner product space, it is always convenient to work on the squares. So this is the integral
18:50:950Paolo Guiotto: integral over n of modulus F minus UK square, the nu. That is the sum
19:00:960Paolo Guiotto: from… for n going from 0 to infinity of, modules FN minus the nth component of UK. You understand this notation?
19:15:730Paolo Guiotto: Okay, so you have UK as an array, I think the F component of this array. F and, but F is the array, let's just write down here.
19:30:50Paolo Guiotto: is the array with components 1, 1 half, 1 third, etc. Of course, I arrive to 1 over K, then I continue 1 over K plus 1, 1 over k plus 2, etc.
19:41:120Paolo Guiotto: So when you do the difference, you see that they are all equal to zero. For N, going from 0 to this is position
19:50:630Paolo Guiotto: So, this is component 0, this is component 1, this is component 2, this is component K minus 1. So, actually, I have that what survives here is the sum for n going from k to plus infinity.
20:06:20Paolo Guiotto: After, position K, you see that the U is the… the nth component is 0.
20:12:900Paolo Guiotto: of k… of uk, while the nth component of F is non-zero, it's u over k plus 1. So, and then it is U over K plus 2, and the end, this becomes 1 over n plus 1 to power 2. So it is, in other words, the sum for n equal
20:31:200Paolo Guiotto: From k to infinity of 1 over n plus 1.
20:35:620Paolo Guiotto: Square.
20:36:630Paolo Guiotto: But this is the tail of the convergent series, huh? So when I make this K, and I send it to infinity.
20:46:490Paolo Guiotto: This, reminder goes to zero, because the total sum is funny, that's a tale of an infinite sum. When you say K to zero, you sound less silence, and you just finish.
21:01:70Paolo Guiotto: So this goes to zero.
21:04:810Paolo Guiotto: when K goes to plus infinity.
21:09:310Paolo Guiotto: So it means that the distance between F and UK can be small… orbitally small, can be made arbitrally smaller by choosing K sufficiently large.
21:19:210Paolo Guiotto: And this, in particular, means that when you take the infamum of the distances between F and U for a generic U in the subspace capital U, this will be, in particular, less or equal than the distance between F and that UK,
21:35:30Paolo Guiotto: Because that's the best lower bound of these distances, and since this one goes to zero.
21:42:610Paolo Guiotto: This is just a constant. You see, there is no K in that quantum. It's a constant. You're saying this constant is less than something that goes to zero, that constant cannot be anything less than zero in those things positive. It's an infinum of positive quantities.
21:59:860Paolo Guiotto: It is less or equal than something that goes to zero, that constant must be equal to zero. So we get that infamum
22:08:770Paolo Guiotto: Norm F minus U.
22:11:310Paolo Guiotto: on capital U is equal to 0. So this is the first point that says the influence that always exists, it is equal to zero.
22:21:130Paolo Guiotto: So, if maximum, sorry, if minimum would exist, it should be equal to the informum, and in particular, it should be zero. That's the… now the key point. So…
22:34:70Paolo Guiotto: If there is… minimum of distances between F and U, on capital U, that necessarily These… must be…
22:50:860Paolo Guiotto: equal to zero. But this means that since this one is a minimum.
22:56:320Paolo Guiotto: So it is achieved, no? It's not just the infinone that maybe is not achieved. This one is a minimum, it is achieved, so it means that there existed vector, U star, in this subspace, capital U, such that
23:12:670Paolo Guiotto: the distance between F and that spatial vector is the minimum value, no? It's attained, and it is equal to 0.
23:21:500Paolo Guiotto: But this means that that U star must be equal to F, or F must be equal to U star. That means that F must be in U.
23:32:160Paolo Guiotto: And, by construction, this is impossible, because U is made by sequences which are, zero, starting from a certain index. Not all of… not all the same… not the same index for all vectors, okay?
23:47:660Paolo Guiotto: for certain vectors, they might be zero after 1 million of components, for other after 3 components, for other after 2 billions, and so on, no? So it's not universal. But in any case, this is impossible.
24:06:640Paolo Guiotto: And so, this means that the minimum cannot exist. So, the conclusion is that… so there is no minimum.
24:16:870Paolo Guiotto: distance between F and U on this In this case.
24:22:200Paolo Guiotto: So what goes wrong? Well, actually, in this case, since our space is, in fact, an L2 space, an L2 space is complete. We said that all the LP spaces are always complete, so it is complete. It means that it is a Banach space.
24:39:870Paolo Guiotto: equipped with the natural norm here, which is that one induced by the scalar product.
24:45:760Paolo Guiotto: So this is a little bit space, okay?
24:48:910Paolo Guiotto: This is a little bit of… space, huh?
24:54:40Paolo Guiotto: So we have an inlet space, a very good space, so what should go wrong is either the vector F or the subspace U.
25:03:260Paolo Guiotto: Actually, the key point is that to ensure the existence of the minimum, we need a minimal assumption on U, which is not just… is not sufficient that U is just a linear subspace.
25:17:10Paolo Guiotto: Now, what is the key assumption? The key assumption is the following. So, to make… this.
25:28:770Paolo Guiotto: True.
25:31:290Paolo Guiotto: factor.
25:33:440Paolo Guiotto: we… need… technical…
25:44:110Paolo Guiotto: assumption.
25:45:890Paolo Guiotto: on you. And this is that it's a topological property, that is, you must be closed. So, since, perhaps you have not said what does it mean, so, in general.
26:01:20Paolo Guiotto: If V is a normal space, is a norm.
26:09:460Paolo Guiotto: space.
26:12:190Paolo Guiotto: a subspace… U contended in V is closed.
26:24:840Paolo Guiotto: Well, there are different, definitions, equivalent definitions. Here we use the, the… a definition that, it's immediately used in the theorem we are going to see, which is the counter characterization of this property. If…
26:44:540Paolo Guiotto: So, the property is that for every sequence of vectors, say, UN,
26:51:30Paolo Guiotto: Well, now, since probably you have still in mind that n is the index of the component, so this has no more to do with the previous example. So let's, in any case, use the letter K for the index. So this is a sequence of vectors in U, okay?
27:09:390Paolo Guiotto: Each UK is a vector of U, such that the sequence UK converges to some U in… according to the norm of the space, while closeness means that that limit U belongs to the subspace capital U. Then, U belongs to capital U.
27:29:650Paolo Guiotto: In other words, the subspace contains all possible limits of all possible convergent sequences taken in U, okay? So you cannot escape this U. If you have a sequence that converges somewhere, that somewhere must still be in you.
27:48:90Paolo Guiotto: Okay, now we are ready for the main theorem.
27:51:530Paolo Guiotto: This really deserves the name of theorem.
27:55:110Paolo Guiotto: that says, let V, say, keep with an inner product. This theorem holds for both cases, scalar and Hermitian product. B, Hilbert space.
28:16:440Paolo Guiotto: U containing the MV, B, A, closed, subspace, off.
28:30:110Paolo Guiotto: of V.
28:32:150Paolo Guiotto: Well, I have not, sorry.
28:35:80Paolo Guiotto: I have not said, but of course, if you go into this example here.
28:41:80Paolo Guiotto: you can see that this U is not closed.
28:44:640Paolo Guiotto: And basically, you have already the ingredients, because if you take the sequence UK that you see there, that's a sequence of vector of U. You see that this calculation we made here shows that the distance between UK and the F goes to zero, so the sequence is a sequence of vector of U.
29:02:700Paolo Guiotto: that converts to F, but that F is not in U. So in particular, U is not closed, because if it is closed for every sequence of that is not U, such that there is a limit, that unit must necessarily be into U, okay? Which is not the case here.
29:21:970Paolo Guiotto: So, this is why the example goes wrong here, because that U is not closed. So now, let V be a Hilbert space, U closed, it's a space of U. Then.
29:36:190Paolo Guiotto: for… every F in V
29:40:460Paolo Guiotto: there exists a unique element that we will, denote with this notation. PU of F exists a unique element of U.
29:51:160Paolo Guiotto: Such that,
29:52:990Paolo Guiotto: The distance between,
29:56:00Paolo Guiotto: F and this element, PUF, is less or equal than the distance between F and any other vector u of the subspace capital U for every little u in capital U.
30:11:490Paolo Guiotto: Which is an equivalent way of saying that That the user.
30:19:660Paolo Guiotto: Equivalently.
30:25:890Paolo Guiotto: There exists the minimum
30:29:140Paolo Guiotto: of distances between F and U when U is in capital U, and this is exactly the distance between F and that element, PU of F.
30:42:410Paolo Guiotto: Moreover, this element is uniquely characterized by the following equation, which is the orthogonality condition we wrote above.
30:50:740Paolo Guiotto: PU of F, is… characterized.
30:59:640Paolo Guiotto: Yeah, wow.
31:02:960Paolo Guiotto: It… Oz.
31:06:680Paolo Guiotto: the… unique.
31:10:460Paolo Guiotto: solution.
31:12:740Paolo Guiotto: of this condition, F minus its orthogonal projection, PF, scalar U is equal to 0 for every U in capital U.
31:25:640Paolo Guiotto: This element, PU of F, is called… orthogonal.
31:38:850Paolo Guiotto: rejection.
31:41:130Paolo Guiotto: of F on U.
31:45:150Paolo Guiotto: You will see that this is a really tremendous result, because, for example.
31:52:910Paolo Guiotto: Thanks to this, we can build a concept which seems to be really far from this setup, that we will see in the second part of this course, which is the conditional expectation that is a very important tool used in probability, okay?
32:12:590Paolo Guiotto: And it has several applications in approximation problems, so it's a really very important fact. And so, since it is very important, we will see also the proof, it is not a short proof.
32:29:180Paolo Guiotto: It's not particularly difficult. It shows nice ideas, here, so…
32:38:300Paolo Guiotto: It's a nice thing to see this profile. So the idea is that, first of all, we don't know that the minimum exists, so we start from the surrogate, so from the infamum. Let's call it alpha.
32:51:250Paolo Guiotto: let alpha be the infamum between these distances between F and U, U in capital U.
33:02:50Paolo Guiotto: Of course, if needed, I don't know yet, I do not remember the proof now. If there is a point where we need to specify that F should be taken out of you.
33:17:210Paolo Guiotto: If this is the case, we will see, because if F is in you, the solution is trivial. You take as F itself.
33:23:360Paolo Guiotto: And you are done, okay? So the problem is non-trivial when this F is not in U. I don't know if we need this, however, we will see later. In any case, that informum is finite, because you have at least one of this number, distance between FU as at least the vector 0, no? Cannot be empty, so there is at least one number.
33:43:420Paolo Guiotto: among these norms of F minus 2, and therefore, this infinible makes sense, and it is found.
33:52:100Paolo Guiotto: So, the goal is now to show that there exists an element in U that makes… that attains this alpha, okay? For which the distance between F and that element is exactly the value alpha. So, this is the first goal.
34:12:460Paolo Guiotto: show.
34:14:920Paolo Guiotto: that… There exists an element
34:19:310Paolo Guiotto: That will be PU of F, but I will not carry around this notation for the moment, because this is not yet uniqueness, it shows that existence, no? There exists an element, say, U… U star, in capital U,
34:33:150Paolo Guiotto: Such that the distance between F and U star is exactly equal to alpha. If I have done this, it means that that equilibrium is a minimum, because there is a vector that achieves the minimum value of alpha, okay?
34:50:639Paolo Guiotto: Now, how do we show that this vector u exists? Well, since that alpha is an infimum, I don't know if this is the case, but what I can say is the following. You remind that infamum has this property. That number is greater or equal than zero, by the way.
35:08:440Paolo Guiotto: So we have 0 somewhere, our alpha, which is the best lower bound between distances between F and some vector u.
35:16:940Paolo Guiotto: Now, a key property of infamum is that as soon as you take a number which is slightly bigger than alpha, this number is no more an infinum. So it means that there must be an element of the set that falls below that number. So this means that if you take alpha plus epsilon.
35:36:540Paolo Guiotto: This number is no more an infamum, so there should be, so for everyone, epsilon positive.
35:42:460Paolo Guiotto: there exists a vector U in capital U, such that the distance between F and U is less or equal than alpha plus epsilon. So here we have this distance between this F and this U.
35:58:140Paolo Guiotto: Of course, since you are doing an argument like, for every epsilon that is a U, that U in general will depend on epsilon, so let's write U epsilon for a second here.
36:09:840Paolo Guiotto: Now, this idea is also the base idea behind how to solve an optimization problem, variational problems, things like that. So, you start from the infamom.
36:23:210Paolo Guiotto: And now you build a minimizing sequence, so a sequence of vectors that should go to the candidate minimum, no?
36:31:960Paolo Guiotto: What is the sequence? What is the sequence here? The sequence is now created by taking epsilon equal 1 over n.
36:38:910Paolo Guiotto: So, I now… sorry, 1 over K, since I want to keep the notation. So, this means that for every K, there exists an… instead of calling it U1 over K, I will just call UK.
36:53:990Paolo Guiotto: in U, such that the distance between F and U k is less than that alpha plus 1 over K. And of course, since alpha is the infamum between distances… is the infimum of distances between F and U,
37:12:370Paolo Guiotto: That number, distance between F and UK, will be greater or equal than alpha.
37:17:910Paolo Guiotto: So you see that this UK is going to be what we call a minimizing sequence, because it is a sequence
37:25:900Paolo Guiotto: Okay, what we want to minimize is this quantity now. It is a sequence along which this quantity, when k goes infinity, at least this number goes to alpha, this is for true, okay? So it is clear that from this bound, this quantity goes to alpha.
37:43:800Paolo Guiotto: But now the point is that, what is your pain after doing?
37:49:170Paolo Guiotto: Because this says that the quantity norm of F minus k equals 12, but what do you do? Well, let's imagine with a figure what is going on here. So let's pick again…
38:02:470Paolo Guiotto: And that's what is nice with the Hilbert, the inner product structure, that you have some geometrical appeal here. So, imagine that you have your F,
38:14:800Paolo Guiotto: Somewhere here. And you are, you want to… the goal should be to identify this element, this magic element, which is the element at minimum distance.
38:26:870Paolo Guiotto: What you know is that, okay, this is… the distance here is what is the number half, no? It's the distance between F and that plane, U. What you can do is that you are not sure if you can take exactly that point, but what you can do, you can take points UK, somewhere down here in U, such that this distance between UK and that
38:51:840Paolo Guiotto: this is less or equal than alpha plus 1 over K.
38:57:90Paolo Guiotto: then you may imagine that these points, which are moving down here, they cannot go too far, no? Because this distance is going down too far. So you cannot expect that this point is here, and the next point is there.
39:12:190Paolo Guiotto: Because you would have a huge distance, no? A huge… yeah.
39:19:430Paolo Guiotto: a huge growth of the distance. But the point is now this. So, to show that this sequence QK is actually convergent, okay? So, new goal.
39:36:90Paolo Guiotto: show that… this minimizing sequence UK, is convergent.
39:45:370Paolo Guiotto: to some U star, that very likely is going to be that point I'm looking for here, okay?
39:53:210Paolo Guiotto: Now, how can I prove convergence? This is exactly the situation I told you when we started discussing limits of sequences in norm spaces.
40:04:10Paolo Guiotto: Here we have a sequence that arises because there exists a mechanism, no? So we don't know anything about this sequence. We want to prove that it converges to something. We don't know yet what is the something, because that's the goal of the proof. So exactly here, we have the case when we have a sequence.
40:23:280Paolo Guiotto: They want to show that it is convergent. We cannot use the limit, because the goal is to prove that that limit exists, you see? So how can I do that? And that's where the Cauchy property comes.
40:35:580Paolo Guiotto: So, if we prove that this UK is a Cauchy sequence, since we are in an environment, hidden but space, which is a Banach space, which is a space for which Cauchy sequences are convergent, we will have the desired convergence. So, let's show…
40:55:550Paolo Guiotto: that this UK is a Cauchy sequence.
41:01:460Paolo Guiotto: Now, to prove that it is a Cauchy sequence, I need to assess the distance between two vectors here. So let's say that we take UK minus UJ,
41:12:160Paolo Guiotto: Any two vectors, and we want to show that this quantity is… can be made small, provided these two indexes are big enough, okay?
41:20:620Paolo Guiotto: And now this here is where it comes. For the moment, you don't see any specific structure. They've not yet used the Scala product, right? So for the moment, this argument could be started in any normal space.
41:34:40Paolo Guiotto: But it is here where I can use this particular structure that comes with the product.
41:41:530Paolo Guiotto: I put a square because it is never a good idea to take norms in a product space, but better squares, so you take out the roots. And now you remind the identity we have seen yesterday, the parallelogram identity.
42:00:40Paolo Guiotto: what this says, you reminder, that it says norm of F plus G squared plus norm of F minus G squared equal to
42:08:620Paolo Guiotto: two times the square of norm of F plus the square of norm of G, okay? And this, in particular is where we use to get out this guy, the norm of the distance. So from this one.
42:24:120Paolo Guiotto: What is going on?
42:28:270Paolo Guiotto: Okay. Just it. So, the parallelogram identity…
42:37:110Paolo Guiotto: Remind, norm of F minus G squared equal norm of F, I carry that in the other side, so I have 2. That multiplies norm of UK
42:48:330Paolo Guiotto: Yeah, no. Yeah. Okay, let's write, but that's not yet the right thing to do. UK square plus, UJ square…
42:59:760Paolo Guiotto: minus a norm of UK plus UJ. This would be with the parallelogram identity.
43:06:560Paolo Guiotto: But I don't know anything about these norms. My information is on this quantity, you see, norm of F minus 2K. So I needed to introduce F inside that. So that's not the right thing, but the right thing is, say, take 2K minus QJ again.
43:27:870Paolo Guiotto: UJ… Now, spared. P8F inside, adding and subtracting, so you have UK minus F, minus UJ minus F squared.
43:47:840Paolo Guiotto: Okay, and now we apply the parallelogram identity to this.
43:52:280Paolo Guiotto: Okay? So here, we have the parallelogram.
43:56:940Paolo Guiotto: Identity, Which says it is 2 times
44:02:970Paolo Guiotto: the square of norm uk minus F.
44:07:800Paolo Guiotto: plus square of norm uj minus F, minus…
44:18:140Paolo Guiotto: the norm of the… now the sum between these two vectors, okay? So I have UK minus F plus…
44:28:320Paolo Guiotto: UJ minus F.
44:31:110Paolo Guiotto: square.
44:33:430Paolo Guiotto: Okay, so what they used is,
44:37:690Paolo Guiotto: Let's use different letters, norm of G plus H squared plus norm of G minus H squared equal to, norm of G squared plus norm of H squared.
44:54:230Paolo Guiotto: Okay?
44:56:730Paolo Guiotto: And now, let's see what… here we know something, because, these are the distances between UK and F and UJ and F. These are known. They are bounded above, respectively, by, before the square, by, alpha
45:14:370Paolo Guiotto: plus 1 over K, and this is bounded by,
45:22:400Paolo Guiotto: alpha plus 1 over J.
45:26:920Paolo Guiotto: So I can say that that distance, I need to bound to show that this is smaller. That distance is less or equal than to…
45:35:310Paolo Guiotto: Now, I have alpha plus 1 over K for the first to power 2. This square comes from this one.
45:44:670Paolo Guiotto: plus alpha plus 1 over J still to power 2, and again, I have that norm. Let's give a look to that. So let's first rewrite. This is norm of UK plus UJ minus 2F squared.
46:05:980Paolo Guiotto: Now, I do not have this, apparently.
46:08:700Paolo Guiotto: But, if you factorize these two, so factorize the 2, this means that you divide by 2 here, you multiply everything inside the norm by 2, you take out, it comes out a 2. 2 power 2, it is a 4. So we have this
46:25:640Paolo Guiotto: 4 in front of.
46:28:50Paolo Guiotto: And now, let's give a look to this quantity, the norm of UK plus UJ divided by 2 minus F.
46:37:820Paolo Guiotto: We can interpret this as the distance between a vector and F. And what is that vector?
46:44:60Paolo Guiotto: That vector is UK plus UJ, so it is the sum of two vectors of U divided by 2. So, in fact, it is a linear combination. 1 half UK plus 1 half UJ. So, since U is a linear subspace, this guy belongs to U.
47:03:200Paolo Guiotto: So this is in U, and I have here a distance between a vector of U and F.
47:10:250Paolo Guiotto: Wow.
47:11:440Paolo Guiotto: I don't know what is that distance, but what can be said is that that distance is greater or equal than alpha.
47:18:250Paolo Guiotto: So when I do the square, this will be greater than alpha squared. And this bound is good, why? Because remember that in front of that norm, we have F-.
47:30:270Paolo Guiotto: Okay? So, if I show that this is larger than…
47:35:370Paolo Guiotto: In that estimate, this would be smaller than, because there is the opposite. So I can continue now the chain. I have less or equal 2, that multiplies alpha plus 1 over k squared.
47:49:140Paolo Guiotto: plus alpha plus 1 over J squared, and then out I have minus 4 alpha squared.
47:57:810Paolo Guiotto: And now you will recognize that this quantity can be made small. Why? Because if you develop these squares, here you have alpha squared plus 2 alpha over k plus 1 over k squared, and similarly, here you have another alpha squared plus
48:14:290Paolo Guiotto: 2 alpha over J plus 1 over J squared.
48:19:860Paolo Guiotto: So you see that there is an alpha square here, another alpha square here, together they give 2 alpha squared. With this factor 2, they give 4 alpha squared. Minus that 4 alpha square we will see here, they cancel.
48:38:30Paolo Guiotto: So what remains, actually, at the end is equal, 2 times,
48:45:140Paolo Guiotto: two times these two sums, no? So, let's just write, because…
48:51:650Paolo Guiotto: We don't need to specify better, so 2 times 2 alpha over K plus 1 over k squared plus 2 alpha over J plus 1 over J squared.
49:05:850Paolo Guiotto: And now it is clear that, oh, this quantity is all provided K and J are B, no? Because 1 over K, minority go to 0, and K goes, again, the same for this quantity. So this quantity can be made
49:22:560Paolo Guiotto: less or equal than epsilon for every K and J larger than a certain capital N. Doesn't matter that we determine exactly.
49:34:630Paolo Guiotto: Okay, now we proved that the sequence is a Cauchy sequence, and therefore it is convergent, because we are in a complete space. So…
49:47:300Paolo Guiotto: V is… here, that space, particularly it is… Complete.
49:57:150Paolo Guiotto: which is the property that Cauchin sequences are convergent, there exists the limit
50:03:680Paolo Guiotto: in K of this sequence UK, let's give the name u star to this guy. Now, this guy is where? This is a limit of vectors UK that are in U, and we say that this U is supposed to be closed. So, this means that that limit is in U.
50:26:320Paolo Guiotto: Because… U is closed.
50:34:320Paolo Guiotto: Okay, so now we know that that U study is in, is in, U.
50:39:630Paolo Guiotto: And, to close this first part, the mind that the goal was to show that there exists a U star in U, such that the distance between alpha and U star is alpha. We now show this. And…
50:57:590Paolo Guiotto: Well, distance between F and U star.
51:02:580Paolo Guiotto: we never, perhaps, say it, but the norm is always a continuous function respect to itself, no? If you have a sequence which is convergent in that norm, the norm will converge as well. So this is the limit in K of norms of F minus UK,
51:21:430Paolo Guiotto: But these were less or equal than alpha plus 1 over K, and greater or equal than alpha, so we know that they converge to alpha, so this limit is alpha, and that's the conclusion.
51:36:230Paolo Guiotto: Okay, so what have we proved so far? We proved that
51:41:920Paolo Guiotto: there exists an element of you which minimizes distance, okay? So it is, basically, we are here. We proved the existence.
51:52:420Paolo Guiotto: Now we have to prove uniqueness, and finally, we will prove the characterization, okay?
51:59:70Paolo Guiotto: So, with this, We've… this… We… So, the… existence.
52:15:400Paolo Guiotto: Now, uniqueness.
52:22:300Paolo Guiotto: we have to show that if there are two U stars that makes this, they must be the same. So, suppose that… suppose…
52:34:180Paolo Guiotto: That, we have two vectors, U star, U, double star.
52:39:400Paolo Guiotto: in you, such that the distance between F and your star coincides with the distance between F
52:47:260Paolo Guiotto: And you double star.
52:49:620Paolo Guiotto: And it is equal to, in both cases, to alpha.
52:53:570Paolo Guiotto: The goal is to show that the two vectors coincide.
52:59:410Paolo Guiotto: U star equal U star, U double star.
53:04:320Paolo Guiotto: And how can we do that?
53:06:470Paolo Guiotto: Well, we assess the distance between these two vectors, so the norm of the difference, once again. So, norm of U star minus U double star.
53:16:350Paolo Guiotto: And now, what do you expect to use?
53:21:790Paolo Guiotto: The same trick, the parallelogram identity. We first introduce F, so as a ball, so we say U star minus F minus U double star.
53:34:520Paolo Guiotto: minus F.
53:38:90Paolo Guiotto: And now we apply the parallelogram, Identity.
53:43:440Paolo Guiotto: And exactly in the same manner, here we have 2 that multiplies the norm of U star minus F squared.
53:52:630Paolo Guiotto: Plus, norm of U double star.
53:55:840Paolo Guiotto: minus F squared.
53:58:600Paolo Guiotto: And then we have minus the sum exactly as before. This comes U star plus U double star minus 2F squared.
54:12:270Paolo Guiotto: We factorize the 2, so we multiply everything by 2, we carry outside of the norm, it becomes a 4, because there is a square. Yeah.
54:22:640Paolo Guiotto: And now, we know that, by assumption, this quantity is alpha, so squared.
54:29:760Paolo Guiotto: This quantity is, again, alpha squared.
54:33:10Paolo Guiotto: And about the last one is the same argument we used above. That's the distance between a vector of U and F.
54:39:930Paolo Guiotto: So, we don't know what exactly it is, but this is for sure, greater or equal than alpha. So, squared, it will be greater or equal than alpha squared, because there is a 4… well, let's keep the 4 there. So, we have that. Now, this becomes an inequality, because the first part is an equality, no? I have 2…
55:00:790Paolo Guiotto: alpha squared plus alpha squared, so 2 alpha squared. But now, since I say that the norm of that vector, U star plus double star over 2 minus alpha, is greater than alpha, this, the minus inverts, and this becomes an inequality, less or equal, minus 4 alpha squared, and all this is 0.
55:23:290Paolo Guiotto: So we get that the norm of U star minus U double star is less or equal than zero.
55:31:550Paolo Guiotto: It cannot be negative, it must be zero, and you have the conclusion. U star equals U double star.
55:40:150Paolo Guiotto: So now, with this, we have also existence.
55:45:380Paolo Guiotto: sorry, uniqueness. With this… Uniqueness.
55:55:20Paolo Guiotto: is oof.
55:59:110Paolo Guiotto: Finally, de-characterization with the orthogonality condition, okay?
56:07:620Paolo Guiotto: Final step?
56:15:960Paolo Guiotto: prove… orthogonality.
56:25:130Paolo Guiotto: characterization.
56:27:490Paolo Guiotto: Which is, we want to prove that F minus
56:32:390Paolo Guiotto: Well, let's keep that used data.
56:35:710Paolo Guiotto: Scalar U, this is always equal to 0 for every U in capital U.
56:45:820Paolo Guiotto: Okay, what do we know about this U-star? For the moment, we know that QStar is the minimum. So, we know the starting point is that
56:54:320Paolo Guiotto: we know that the distance between F and U star is less or equal than the distance between F and any other vector, U, for every U in capital U.
57:10:440Paolo Guiotto: Now, what we do is we square this.
57:16:660Paolo Guiotto: And, let's write this,
57:21:890Paolo Guiotto: by using the cosine theorem, no? You remind that we proved that if we have norm of G plus H,
57:32:520Paolo Guiotto: squared, this is exact formula, norm of G plus norm of H, both squared. Here, you see that for this, up to this point, we never used this specific product, this
57:46:560Paolo Guiotto: is an argument which is the same for the two products. Here, I have a little difference, but we do the calculation for the real product. The other case is similar. So we have two scarlet products between G and H. So this is what we do.
58:01:150Paolo Guiotto: So at left, we have norm of F,
58:05:330Paolo Guiotto: squared plus norm of U star. Of course, there is a plus, not a minus, no? It's like the square, no? Square of the sum, you have A plus B squared is always A squared plus B squared. It's not A minus B squared is A squared minus B squared, you see? It's a plus.
58:24:490Paolo Guiotto: But you have the product between the two, and that will be with minus. Minus 2F U star.
58:32:30Paolo Guiotto: At right, we have norm of F squared plus norm of u squared
58:42:380Paolo Guiotto: I'm not sure this is the right way, however, let's see.
58:45:960Paolo Guiotto: what… what happens? Minus 2… F, scholar, you.
58:52:790Paolo Guiotto: Of course, we cancel this guy, which is the same for both sides. We try to put together, we put together…
59:10:90Paolo Guiotto: The problem is that I am in the wrong direction.
59:20:770Paolo Guiotto: Okay, let's skip the identity in this way. I'm sorry, I will do a little mess, because probably I should have done at the beginning. So we get U star square.
59:31:980Paolo Guiotto: Minus… 2F scalar U star, is equal to norm of U square.
59:40:980Paolo Guiotto: minus 2F scalar U. Now, this U study is fixed while you use a generic vector of capital U.
59:50:950Paolo Guiotto: What we can always do is to write that U as U star plus, of course, what? Formally, it is the difference, U minus U star, no?
00:02:420Paolo Guiotto: But that vector, U minus U star, is still in U, because U is a linear space, so it's a difference of two vectors of U. It belongs to U. So we write this as U star, perhaps let's call it V, with V in U. And you may notice… yes.
00:21:940Paolo Guiotto: But there's some footing.
00:25:510Paolo Guiotto: There were, estrogen oil?
00:27:990Paolo Guiotto: Okay, yes. Sorry. So they should be let go, okay?
00:34:230Paolo Guiotto: So it is clear that this is fixed, that when you change U in carbon 2, you get all possible black and u or part of U, because this is an injection, no? So we can always represent things in this way. So let's plug this into this formula.
00:56:360Paolo Guiotto: I hope I'm not doing a mess.
01:00:300Paolo Guiotto: Nope.
01:05:660Paolo Guiotto: No, not enough.
01:20:00Paolo Guiotto: No.
01:27:190Paolo Guiotto: Because,
01:41:990Paolo Guiotto: Now, this would suggest… sorry, yeah? This is correct, but I don't see in this moment how
01:48:830Paolo Guiotto: to get there, but perhaps the idea should be that since we have the product between these two, F minus 2 star and U, this should be this middle term of the squares, so let's compute this. F minus U,
02:06:800Paolo Guiotto: Star… Let me see, bless you.
02:11:240Paolo Guiotto: Let's do this.
02:14:880Paolo Guiotto: Okay, square.
02:18:350Paolo Guiotto: So,
02:21:530Paolo Guiotto: Let's see what happens here, because I'm trying to recreate that scalar product, okay? So if I do this, I have norm of F minus U star, again, with the cosine theorem.
02:33:780Paolo Guiotto: squared plus normal u squared, and then I have the double product plus 2F minus U star scalar U.
02:43:630Paolo Guiotto: This seems to be a bit more promising, because this is equal to… Alpha…
02:52:880Paolo Guiotto: And this is… we can always imagine the alpha square, sorry. This is F minus U star minus U, right? So it is the distance between F and the vector that belongs to U.
03:06:200Paolo Guiotto: So this quantity is a generic distance, and therefore it will be less or equal than alpha, so the left-hand side will be less or equal than alpha squared. So from this, I get that alpha squared is greater or equal than alpha squared plus norm of U squared plus 2F
03:26:350Paolo Guiotto: Minus U star U.
03:30:380Paolo Guiotto: We simplify the alpha square here and here, so it remains zero. Yes, this is the right way.
03:37:360Paolo Guiotto: So, we have… we find this, too. Well, let's write in red.
03:44:480Paolo Guiotto: 2. Scalar product between F minus U star and U.
03:49:440Paolo Guiotto: plus the square of norm of U is less or equal than 0. This holds for every U in capital U.
03:59:520Paolo Guiotto: It is not yet the conclusion, but now you will see that we easily get the conclusion from this one.
04:07:20Paolo Guiotto: Now, the first thing is that,
04:11:470Paolo Guiotto: The argument is that this is a linear term, and U, this is quadratic timing. When U is small, this term is negligible with respect to this one. So, if this term must be negative, this must be negative. Now, we can formally do this in this way. Replace U,
04:31:810Paolo Guiotto: by, I don't know, TU, or if you prefer, which is small, epsilon U.
04:38:390Paolo Guiotto: So, replacing that formula, instead of vector u, epsilon U, what you get is 2F minus U star, scalar, epsilon U,
04:49:40Paolo Guiotto: Plus, when you put the epsilon into the normal, it comes out as an epsilon to power 2, so epsilon squared normal U,
04:59:500Paolo Guiotto: Less or equal than zero, right?
05:02:750Paolo Guiotto: Now, since here we are working with the… in any case, even if the product is a median, epsilon is supposed to be positive, for example, so we can carry outside this epsilon linearly, so it comes in front of the product, and we divide by epsilon, which is positive.
05:19:910Paolo Guiotto: So, divide everything by epsilon, and we get 2. F minus U star, U plus epsilon, norm of u squared, it is still less or equal than zero. Now, this also for every u in capital U, and for every epsilon positive.
05:41:500Paolo Guiotto: You see? With this trick of replacing U into, this is what is called the anohomogeneity argument.
05:49:480Paolo Guiotto: We arrived this, but now we can let epsilon to zero.
05:53:770Paolo Guiotto: So if we send epsilon to 0, you see that this gives this part here, and therefore, we get that 2F minus U star, color U must be less or equal than 0. Now, we simplify the 2 also.
06:09:740Paolo Guiotto: We are almost done, because this sold for every U in capital U.
06:14:670Paolo Guiotto: But if you now replace that U with minus U, which is still a vector of U, no? So we are still working with vectors of U, replace U with the minus U,
06:25:700Paolo Guiotto: This becomes F minus U star. color with minus U is less or equal than zero, and it is clear that for every U in U, minus U is in U, and vice versa.
06:37:770Paolo Guiotto: Because U is a linear space. So that's for every U in capital U.
06:42:760Paolo Guiotto: But now again, you take out the minus in front of the product, so you put in front, and you multiply by minus 1 everything. What you get is then that F minus U star, scholar U, is now positive.
06:59:390Paolo Guiotto: for every U in capital U. Now, we've put together these two. This says that F minus U star scalar U is negative for every U. This says that F minus U star, scholar U is positive for every U. The only possibility is that this quantity is zero.
07:17:230Paolo Guiotto: So from this, we get, finally, that F minus U star.
07:23:30Paolo Guiotto: U is 0 for every u in capital U, and the proof is finally over.
07:35:410Paolo Guiotto: Okay.
07:37:200Paolo Guiotto: So that's this important factor, and let's show immediately an application of this. This is a standard kind of problem. I take this, which is an example note, example 14,
07:56:730Paolo Guiotto: 1 tree.
08:00:30Paolo Guiotto: Let's say, how it's written, determine… the… best.
08:07:780Paolo Guiotto: approximation…
08:14:690Paolo Guiotto: The best approximation Il.
08:19:130Paolo Guiotto: L2.
08:21:510Paolo Guiotto: 0, 1… Oh.
08:25:290Paolo Guiotto: X squared.
08:27:210Paolo Guiotto: Through… First, degree.
08:36:319Paolo Guiotto: polynomial.
08:41:630Paolo Guiotto: This is how a simple approximation problem could be stated, no? You have a specific function, and you want to find out the best possible approximation
08:52:69Paolo Guiotto: into certain class of functions here, first-degree polynomials, okay, in a certain space. So let's try to translate all this into the machinery of the orthogonal projection.
09:04:450Paolo Guiotto: So, first of all, we have a space. So, here, the space V is L201,
09:16:870Paolo Guiotto: I keep the…
09:18:950Paolo Guiotto: It is not… if not differently said, we assume that this is the real case. It keeps with
09:28:29Paolo Guiotto: This color product, F scalar G in L2 is the integral from 0 to 1 of F of X times G of X dx.
09:40:740Paolo Guiotto: This space, with this color product, is an… is a Hilbert space.
09:45:850Paolo Guiotto: So, V with the L2.
09:49:370Paolo Guiotto: standard Scala product is a Hilbert.
09:56:980Paolo Guiotto: space
09:58:310Paolo Guiotto: This is a known fact. We know that the LP spaces are all Barnac spaces. Particularly, the P equals 2 is special, because differently, no other LP has an inner product, only L2.
10:12:360Paolo Guiotto: has, the… this comes from, this color plot.
10:16:410Paolo Guiotto: Then, we have a vector, which is X squared. That's what is F.
10:23:700Paolo Guiotto: let… F… VX squared, which is, of course, a vector of the space, V,
10:32:480Paolo Guiotto: And what is the subspace U? Here it says first-degree polynomials. So this means that our U, for this case, is made of functions of type A plus BX,
10:47:130Paolo Guiotto: where A and B are any real coefficients. So that's you.
10:53:850Paolo Guiotto: all these functions are contained into L2. Now, they are continuous functions, so they are integral on any finite interval, so…
11:06:970Paolo Guiotto: weight that I have to put powers, yeah.
11:10:610Paolo Guiotto: And so, this is definitely a subset, and also a subspace, as a linear subspace of, V, no?
11:22:100Paolo Guiotto: Because if you sum first-degree functions, you still have a first degree… if you have first degree… if you sum first degree… if you do a linear combination of first-degree polyometers, you'll get a first-degree polyometer. So, it's clearly a linear subspace contained in D. So, is…
11:42:60Paolo Guiotto: linear.
11:44:450Paolo Guiotto: subspace.
11:47:880Paolo Guiotto: Now, you know that to apply the projection theorem, we need to know that it is closed.
11:52:950Paolo Guiotto: Okay?
11:54:150Paolo Guiotto: Here, there is a very standard fact, because this particular space, as you can see, all vectors are a combination of two particular vectors, no? You can see that this means that alpha A plus BX, a first-degree polynomial, can be seen as alpha times 1 plus B times X.
12:17:100Paolo Guiotto: Where these are two special vectors.
12:19:990Paolo Guiotto: So, in other words, in linear algebra, this would be a basis for the space, no? Because all vectors are linear combinations of these two spatial vectors, no? So, this is, U…
12:36:830Paolo Guiotto: is D.
12:39:150Paolo Guiotto: linear. Space… generated… When we say generated, it means through linear combinations, by two vectors, 1 and X.
12:54:340Paolo Guiotto: Okay, so we write… we use this notation, we… Right, huh?
13:02:200Paolo Guiotto: That U is the span
13:06:510Paolo Guiotto: of these two vectors, 1 and X.
13:11:990Paolo Guiotto: Now, the key point is that, as you may imagine, this means that, basically, if I'm too proficient AB, so that set is identified by pairs of points of numbers AB. So, points in the Cartesian plane.
13:31:810Paolo Guiotto: Now, as permeating, there is a correspondence between elements here and points of R2. And this is what makes, basically, through the fact that five-dimensional spaces are always like a fine-dimensional RD, which is always true.
13:51:290Paolo Guiotto: Okay, so this… what's up, Jess?
13:57:670Paolo Guiotto: Martha is… Okay. So, general factor…
14:04:660Paolo Guiotto: I do not prove it's boring. If you are curious, you can read the proof in the notes, is that any
14:14:970Paolo Guiotto: find it.
14:17:610Paolo Guiotto: dimensional.
14:19:980Paolo Guiotto: subspace.
14:23:480Paolo Guiotto: Off.
14:25:90Paolo Guiotto: Annie.
14:27:780Paolo Guiotto: Normed.
14:31:10Paolo Guiotto: space.
14:33:790Paolo Guiotto: V is closed.
14:39:430Paolo Guiotto: This is not the case. In general, it's not true that if the space is in finally dimensional, it is closed. An example of this is, again, once again, the example we have seen here in the beginning. This subspace, U,
14:58:600Paolo Guiotto: is a subspace of delta space little l2.
15:02:960Paolo Guiotto: But it is not closed, we have seen, no? And if you have to write a basis for this.
15:10:510Paolo Guiotto: How can you write… how can you write… you see that you don't… you cannot have a fine basis for this, because a vector is identified by a certain number of components, but definitely they are equal to zero. So, if you have, if you take, let's say, let's do with two vectors.
15:30:230Paolo Guiotto: Take two backdrops into this view.
15:32:780Paolo Guiotto: has all components equal to zero after a certain N1, no? So the first 100 components are different, you know, they're all components from 101 to infinity are zero. And the second vector, what's going on?
15:54:860Paolo Guiotto: It's tough. Well, it's better if I stay here.
15:58:800Paolo Guiotto: And the second vector, maybe is 0, after 200 components. But this means that if you do linear combinations, you get vectors with zero components after the index 200, so you will never be able to generate all this space.
16:17:180Paolo Guiotto: So this is not a finite dimensional space.
16:20:600Paolo Guiotto: So we are not in this case, and that's why it is not closed.
16:26:520Paolo Guiotto: In any case, a five-dimensional space is always closed, so in particular.
16:36:930Paolo Guiotto: Our EU is closed.
16:41:260Paolo Guiotto: Okay, so, we are exactly in that, in that setup.
16:49:480Paolo Guiotto: So, what have we to find? The problem says, determine the best approximation in L2 of X squared through first degree polynomial. So, the problem…
17:03:710Paolo Guiotto: asks… to determine… What?
17:09:940Paolo Guiotto: To determine the minimum.
17:13:610Paolo Guiotto: of L2 norms, I don't like particularly this thing I'm going to write, because I'm going to say, you want to find the best distance between X squared and the first degree polynomial, AX plus B.
17:29:510Paolo Guiotto: over all possible choices on AB. So, A, B, and R.
17:37:280Paolo Guiotto: So, why I don't like? Because X squared is a function, so normally you don't put the variable into the norm, okay?
17:46:680Paolo Guiotto: So, by the way, I should use something like the dash here to say that's the function, the square, the other, but that would be a little bit more incomprehensible. So let's keep the variable notation, even if it is a little bit…
18:03:600Paolo Guiotto: not a correct notation, huh? So what I want to find is this. I want to find the polynomial AX plus B that yields the best approximation of X squared in L2 norm. And that's exactly the problem of searching for the minimum between X squared and U,
18:24:80Paolo Guiotto: In L2, when U belongs to that subspace capital U. So the solution is…
18:33:820Paolo Guiotto: Since, the, projection, TRM.
18:40:770Paolo Guiotto: applies.
18:45:310Paolo Guiotto: Anger.
18:47:840Paolo Guiotto: So, the solution… Is… The particular polynomial, let's say, alpha star plus
18:57:620Paolo Guiotto: Well, I said I'm doing a mess with the order of this. A star X plus B star is the projection of X squared on U.
19:10:280Paolo Guiotto: But now, of course, the point is, how do we compute these coefficients? Alpha star, A star, and this star. So, how…
19:23:50Paolo Guiotto: Can… Sweet.
19:25:390Paolo Guiotto: compute… A star and B star.
19:34:130Paolo Guiotto: Well, actually, here in this particular problem, there would be another method, which is proceeding directly, and finding, at the end, a function of A and B, which is a function of two-year variables you can minimize with the tools of calculus, of second-year calculus. However, let's use what we learned here.
19:53:400Paolo Guiotto: So what we know is that this function that we are looking for is the orthogonal projection. So this function is characterized by the orthogonality property. So what we have is that…
20:07:230Paolo Guiotto: A star X plus B star is D.
20:15:180Paolo Guiotto: unique… A unique element of you, such that…
20:23:820Paolo Guiotto: This color product between F, F is X square.
20:27:900Paolo Guiotto: And this, the orthogonal projection, so A star X plus B star, scalar product with U is equal to 0 for every U in capital U.
20:43:710Paolo Guiotto: Now, this means that…
20:46:920Paolo Guiotto: Equivalently, the scarlet product is the integral of the product. So, integral between 0, 1 of X squared minus A star X plus B
20:59:150Paolo Guiotto: So you see, this is the function you have here, times this function, which is a generic function in the subspace U, so a generic first degree polynomial, this must be zero.
21:11:310Paolo Guiotto: Now, here there is another nice remark for this case. Now, we may notice that since U is generated by
21:20:690Paolo Guiotto: two vectors.
21:23:530Paolo Guiotto: It is the span of two particular functions, the function constant equal to 1, and the function X.
21:30:920Paolo Guiotto: You may notice that if this identity is verified, for every U, it must be verified by these two functions. So, since U is this, in particular.
21:46:310Paolo Guiotto: Star will imply that
21:50:50Paolo Guiotto: When I do the scalar product between X squared minus A star X plus B star, and One…
21:59:890Paolo Guiotto: In L2, I will get 0. And similarly, when I do the scalar product between X squared minus A star X plus B star and X, I will get, again, 0.
22:14:520Paolo Guiotto: And these two conditions must be verified together, no? Because, for every U, no?
22:20:700Paolo Guiotto: Actually, this is an if and only if.
22:24:250Paolo Guiotto: Why?
22:25:460Paolo Guiotto: And that's because every other U is a linear combination of these two, no? So, if these two are zero.
22:35:210Paolo Guiotto: You can multiply by alpha and beta.
22:39:890Paolo Guiotto: You carry the coefficient inside here and there.
22:44:700Paolo Guiotto: and you sum the two identities, and you get X squared minus A star X plus B star, scalar, alpha,
22:55:940Paolo Guiotto: plus beta. X equals 0 for every alpha-beta, but this is a generic polynomial of the space.
23:06:240Paolo Guiotto: So, the point is that.
23:09:160Paolo Guiotto: That orthogonality, basically, is verified once it is verified on the generator of the space. If they are finitely many, this yields a finite number of conditions to be verified. So, in other words, it yields a system.
23:24:700Paolo Guiotto: A system where we have to find the two numbers A star and B star. Now, let's solve the system.
23:32:360Paolo Guiotto: So we have that. The first line is integral 01 of x squared minus a star X plus B star.
23:43:680Paolo Guiotto: times 1, so DX equals 0. And the second is integral x squared minus A star X plus B star.
23:55:200Paolo Guiotto: times X, DX, equal 0.
23:59:420Paolo Guiotto: Now, we source for these two, we do a little bit of calculation, so we have…
24:04:630Paolo Guiotto: Integral of X squared is X cubed over 3. To be evaluated between 0, 1, so it remains 1 third.
24:13:680Paolo Guiotto: minus A star integral of X is X squared over 2, evaluated between 0, 1, so you get 1 half.
24:23:650Paolo Guiotto: minus B star integral 0 to 1 of 1 is 1. So this must be equal to 0.
24:30:700Paolo Guiotto: The second is similar, so this is, again, 1 3rd. No, sorry, sorry, sorry, sorry.
24:39:630Paolo Guiotto: Now we have the integral of X cubed.
24:42:730Paolo Guiotto: which is X power 4 divided 4. Between 0, 1, you get 1 fourth here, minus A star, I have X times X, integral of X squared, X cubed divided 3, between 0, 1, 1 3rd.
24:56:400Paolo Guiotto: minus B star, now we have the integral of X, so X squared divided to 1 half.
25:02:770Paolo Guiotto: This equals 0.
25:04:770Paolo Guiotto: And this is now a 2x2 linear system that you can easily solve. I don't,
25:10:40Paolo Guiotto: continue the solution. You will get A star something, B star something else, okay? And you have the solution of this problem.
25:22:230Paolo Guiotto: So, there are many exercises similar to this one at the end of the chapter, and you can find also in some of the exams. So, do…
25:34:510Paolo Guiotto: Exercises.
25:36:610Paolo Guiotto: 14… 3… 1, 2, 3… they are all similar to this one.
25:45:70Paolo Guiotto: I will do the number 5, which is still an exercise on orthogonal projection, but a bit different.
25:54:140Paolo Guiotto: It's a little bit more… Theoretical, let's say.
25:59:750Paolo Guiotto: Here it says we have the space V equal L2R.
26:04:200Paolo Guiotto: with the usual, Scala… real Scala product. When we say usual, we mean, this. F, Scala, G in L2, here it…
26:15:830Paolo Guiotto: It will be the integral on R of F times G.
26:21:270Paolo Guiotto: Then we have a set U, which is defined in this way.
26:25:940Paolo Guiotto: It is the set of functions of V, so of L2, such that F of minus X, it is equal to F
26:36:120Paolo Guiotto: almost every… X in R. So these are basically even functions.
26:44:460Paolo Guiotto: Almost everywhere.
26:46:890Paolo Guiotto: Now… Question 1U is closed.
26:55:290Paolo Guiotto: Question 2… Check that the orthogonal projection, yeah, Let's see.
27:07:880Paolo Guiotto: Not back at all. Wow. Right now, or…
27:13:270Paolo Guiotto: Okay, so at least the situation is not in itsiamo.
27:19:410Paolo Guiotto: At least the situation is not,
27:23:780Paolo Guiotto: Too bad. What was going on?
27:31:530Paolo Guiotto: Okay, but this is… says that…
27:34:710Paolo Guiotto: It is still recording, so the recording should be working.
27:39:950Paolo Guiotto: Now, let's start with the screen.
27:46:110Paolo Guiotto: Okay, so I hope you hear me. Can you give me a feedback?
27:53:540Paolo Guiotto: just writing the… It's…
28:02:360Paolo Guiotto: Okay.
28:04:70Paolo Guiotto: this.
28:05:100Paolo Guiotto: Problems.
28:11:290Paolo Guiotto: Responding, so… We don't know if,
28:18:880Paolo Guiotto: So, in any case, it seems that it is working.
28:24:950Paolo Guiotto: Okay, so… show that the auto projection on you is 1FF F of X.
28:37:210Paolo Guiotto: plus F… It's a preliminary…
28:45:980Paolo Guiotto: Good question.
28:52:150Paolo Guiotto: Question
28:56:910Paolo Guiotto: While using the overall objection.