AI Assistant
Transcript
00:05:40Paolo Guiotto: Okay.
00:06:540Paolo Guiotto: Good morning, Mr.
00:28:30Paolo Guiotto: Okay, at the end of last class, we introduced…
00:33:770Paolo Guiotto: It's the definition of multivariate random variable.
00:39:140Paolo Guiotto: Well, a multivariate random variable is just an array of random variables.
00:45:570Paolo Guiotto: There are two equivalent definitions.
00:53:90Paolo Guiotto: random variable with values in Rn is a function from omega to Rn.
01:00:420Paolo Guiotto: Whose components are themselves random variables? Real, scalar, numerical.
01:06:210Paolo Guiotto: or equivalently, It's not trivial, this step, but we accept.
01:15:330Paolo Guiotto: that, the same property that holds for scalar random variable, that is, X belongs to set E, is an event.
01:26:580Paolo Guiotto: Of the sigma algebra F of the events for every Borel set of the real line, this is for scar, happens, for, subsets of, Borel sets over N,
01:41:130Paolo Guiotto: So, when we take an element of the sigma algebra generated by OpenSet. So, in any case, the set of omega such that X of omega belongs to E, is an event, whatever is E in the Borel sigma algebra.
02:00:840Paolo Guiotto: Okay, well, what can be said in general about multivariate random variables? So, some few, general…
02:13:210Paolo Guiotto: Jenny route… Definitions… for… Multivariate.
02:30:970Paolo Guiotto: random variable.
02:33:630Paolo Guiotto: Well, exactly as, for Scala Randall variable, we have, the concept of low, or, distribution.
02:43:540Paolo Guiotto: So, normally, it's, is, it's called lobe of X.
02:49:510Paolo Guiotto: So, if X is a multivariate random variable from omega to Rn, So, omega, F, P… is a probability.
03:05:390Paolo Guiotto: space.
03:07:440Paolo Guiotto: So we define the law, The… low.
03:17:30Paolo Guiotto: of X.
03:19:710Paolo Guiotto: Is a probability measure.
03:24:370Paolo Guiotto: Mobility Manager.
03:27:520Paolo Guiotto: on… RN… with the Borel Sigma Algebra.
03:34:840Paolo Guiotto: Awful.
03:35:950Paolo Guiotto: RN… And the measure is defined in this way, mu X of E, exactly as for…
03:45:20Paolo Guiotto: the scalar case, it is the probability that X belongs to the set E. So, let's say, formally, this is the probability of the set of omegas in the sample space of capital omega, such that X of omega belongs to E.
04:04:400Paolo Guiotto: Or if you want, again, let's put in parentheses all these notations, which are never used in probability. In probability, you will always read things like this, which is more immediate as notation, even if it is a little bit improper, no?
04:20:600Paolo Guiotto: Another way, this is the anti-image of the set E.
04:28:250Paolo Guiotto: But again, you will never see this kind of notations improbability, but only the first one.
04:34:70Paolo Guiotto: Now, what can be said about this law? Well, it is useful… it is useful to, notice, that,
04:46:800Paolo Guiotto: So the idea is that if you know the law, you know everything about the variable X, because you know the probabilities
04:54:240Paolo Guiotto: of X is in some set, which are the important quantities, or you have also the expected values. So, for example, we have the formula that the expected value of
05:08:490Paolo Guiotto: a function of the variable X, so here phi will be a function of n variables, X1.
05:18:460Paolo Guiotto: XN, so it will be defined on RN.
05:22:920Paolo Guiotto: real valued.
05:26:590Paolo Guiotto: So, the expected value is exactly the same as it happens for the scalar case. We have seen the proof of this formula. It's the same here. So we have the integral on Rn of PX1.
05:44:680Paolo Guiotto: Excel.
05:46:730Paolo Guiotto: respect to that measure, the mu X, invaluable 61, Xn.
05:57:610Paolo Guiotto: So… Another important concept could be the CDF, but here you see it's a bit…
06:07:310Paolo Guiotto: A bit more,
06:12:930Paolo Guiotto: let's say, cumbersome object, because the CDF is a function, FX. Now, remind that, if, if X… if X is a random variable.
06:27:700Paolo Guiotto: So when we say random variable, we mean a scalar variable, okay? So X is a function from omega to R.
06:36:00Paolo Guiotto: the CDF, FX,
06:39:110Paolo Guiotto: is an object, is a function defined on the real line with values 0, 1, and it is defined as FX of X, the probability that capital X is less or equal than little X.
06:55:510Paolo Guiotto: So it's a spatial, say, case of, the…
07:01:930Paolo Guiotto: low, evaluated, on this, set, the offline 1 minus infinite equal to X.
07:09:230Paolo Guiotto: It is a more immediate object with respect to the law, because a law is a measure, and the CDF is a function, so it's an object that we handle more easily, and we have seen that to each FX, there is a random variable whose distribution is that FX. We know that the key properties
07:33:130Paolo Guiotto: properties.
07:37:970Paolo Guiotto: I.
07:39:280Paolo Guiotto: Number one, this FX is increasing. Number two, the FX at minus infinity is 0. At plus infinity is 1.
07:51:270Paolo Guiotto: And number three, the CDF is, continos.
07:59:260Paolo Guiotto: from… the… Right.
08:05:980Paolo Guiotto: And since it is monotonic increasingly, it will have limits from left.
08:11:710Paolo Guiotto: And, because of the monotonicity. Well, that's property usually written, even if it is actually a consequence of these three. These three are sufficient to collateralize, the FX, so FX…
08:24:750Paolo Guiotto: Oz.
08:26:490Paolo Guiotto: Left.
08:28:450Paolo Guiotto: limits.
08:33:220Paolo Guiotto: at every X in R.
08:36:900Paolo Guiotto: And they are less than equal as the value.
08:39:690Paolo Guiotto: of the function. In, French literature, there is, at least,
08:46:460Paolo Guiotto: now, nowadays, it's quite old as age, but since in France,
08:54:920Paolo Guiotto: there has always been a strong school of probabilists. So there were basically two important schools of probabilistic in the world. One was in France, and the other one was in Russia, historically.
09:12:290Paolo Guiotto: And so, they, French mathematicians, they invented a lot of acronyms for this. They call this… you can find some books this…
09:22:180Paolo Guiotto: this, acronym, CADLEG.
09:26:990Paolo Guiotto: which is shortened for Continuity Adruat Limitagos. You see? So if sometimes you see this somewhere, it means exactly this thing. Continuity from the right, and limit from the left.
09:43:920Paolo Guiotto: Okay, no. And, so, if, F.
09:50:480Paolo Guiotto: verifies…
09:55:00Paolo Guiotto: 1, 2, and 3, then F is the CDF of some random variable. So, this interiorly characterizes a random variable.
10:06:720Paolo Guiotto: Now, what happens if we take a multivariate random variable? If now… X is multivariate.
10:22:470Paolo Guiotto: random variable. So X is defined on omega with values in R, and…
10:32:160Paolo Guiotto: What could be the analogous of this thing? Well.
10:36:360Paolo Guiotto: Here you have fx of little x is the probability that the variable is less or equal than x.
10:42:50Paolo Guiotto: Here, the variable is an array, so we may define similarly by taking the probability that the first component of the array, X1, is less than some X1, the second component less than some X2, and so on. So, FX turns out to be a function of
11:01:110Paolo Guiotto: a number of variables, which is the same of the number of components, so X1 to Excel.
11:09:00Paolo Guiotto: It is defined on RN.
11:12:780Paolo Guiotto: With the steel values in 0, 1.
11:16:310Paolo Guiotto: And the definition is F sub X of X1.
11:21:470Paolo Guiotto: XN…
11:23:210Paolo Guiotto: is, by definition, the probability that the first component, capital X1, is less than X1. The second component, X2, is less than X2, and so on.
11:34:800Paolo Guiotto: Until we finish X and less than a little X again.
11:40:460Paolo Guiotto: Now, the key properties of this thing are a little bit more,
11:46:710Paolo Guiotto: involvement. For example, we cannot say it is increasing, because,
11:51:500Paolo Guiotto: the argument of this function is now a vector, and you cannot say that a vector X is less than the vector y, no? There is not this kind of ordering.
12:03:350Paolo Guiotto: However.
12:05:480Paolo Guiotto: we can easily understand that this could be true in the sense that if we increase the components of the array, we automatically will increase the value of F. So, number one,
12:22:890Paolo Guiotto: These are the, let's say, key properties.
12:26:80Paolo Guiotto: Characteristic, let's say, proposition.
12:28:950Paolo Guiotto: We will not prove this, it's just, similar.
12:33:740Paolo Guiotto: So, FX verifies… Number one.
12:42:270Paolo Guiotto: if, say, the array X1.
12:48:860Paolo Guiotto: XN.
12:50:710Paolo Guiotto: is less or equal than the array Y1.
12:54:330Paolo Guiotto: YN… in the… Sansa… back, too.
13:03:860Paolo Guiotto: X1 is less or equal than Y1.
13:08:240Paolo Guiotto: X2 is less or equal than Y2, so component by component, they are…
13:16:70Paolo Guiotto: ordered. This is called the lexicographic ordering. So, we have that F…
13:24:400Paolo Guiotto: capital X of X1. Xn will be, of course, less or equal than F capital X, Y1,
13:34:840Paolo Guiotto: Why yang, no?
13:37:690Paolo Guiotto: So, in particular, if you look at these effects as a function of 1,
13:43:350Paolo Guiotto: component, keeping fixed all the other variables, this would be an increasing function, okay? Second.
13:54:70Paolo Guiotto: Now, it is… what is the left-right limit? If we do the limit from the left, we may say decreasing each of the coordinates, so the limit…
14:09:370Paolo Guiotto: for X1 going to X1 from the right, X2 Going… well, sorry, let's…
14:19:70Paolo Guiotto: I'm doing a mess here. Y1 to X1 from the right, Y2 going to X2 from the right, and so on. YN going to…
14:31:330Paolo Guiotto: XN from the right of the, F.
14:36:520Paolo Guiotto: Sub X, Y1, YN… we will get the value of FX at point, X1, Xena.
14:51:760Paolo Guiotto: And, number 3. So, what is the analogous of f of minus infinity equals 0, f of plus infinity equals 1?
15:00:920Paolo Guiotto: Of course, will be… when we send all the coordinates to minus infinity, we get 0, because intuitively, this is the probability that X1 is less than minus infinity, X2 less than minus infinity. All of them less than minus infinity, that would be nothing, so the probability will be 0. So the limit when…
15:20:110Paolo Guiotto: X1 goes to minus infinity, X2 goes to minus infinity, and so on. Xn goes to minus infinity. Not only one of them, all of them. FX of X1
15:35:350Paolo Guiotto: X, N… This is equal to 0, and the limit when…
15:44:150Paolo Guiotto: X1 goes to plus infinity, etc, XN,
15:49:670Paolo Guiotto: goes to plus infinity. So not the array… I'm not saying the array goes to plus infinity, because this could be, with some components going to plus infinity, some other being constant.
16:03:240Paolo Guiotto: All the components must go to plus infinity or minus infinity, X1, X, and this will be equal to 1.
16:14:850Paolo Guiotto: Now, the proof of these properties, not, particularly important.
16:25:20Paolo Guiotto: Let me say that, in particular.
16:34:890Paolo Guiotto: We noticed that… If we have the CDF of the multivariate random variable.
16:45:370Paolo Guiotto: So let's say that we have the CDF of the array made of these X1XN, so…
16:54:980Paolo Guiotto: Note that if X, capital X, is
16:59:250Paolo Guiotto: made of components X1 to XN,
17:02:700Paolo Guiotto: Now, each of these components is itself a random variable, so it will have a certain CDF.
17:11:10Paolo Guiotto: You understand that if you want to extract from the CDF,
17:16:170Paolo Guiotto: of the full array, the CDF of one single component. You can do this because if you look at this formula, imagine that you want to extract the CDF of X1, which is the probability that X1 is less or equal than little X1. You want all of that.
17:34:800Paolo Guiotto: Well, it is sufficient to put X2, X3, Xn equal to plus infinitive.
17:39:610Paolo Guiotto: Intuitively. Okay, so let's say better. To compute the limit when X1, X2, X2, Xn goes to plus infinity. So we have that CDF of X1 at point X1, which is the probability
17:56:510Paolo Guiotto: that X is less or equal than X1. Now, of course, you need to do a proof of this factor. It is also the limit when X2 goes to plus infinity, X3 goes to plus infinity, and so on. Xn goes to plus infinity.
18:15:930Paolo Guiotto: of the probability that X is less… X1 is less or equal than little x1, X2 is less or equal than little x2, and so on. Xn is less or equal than little xn.
18:30:270Paolo Guiotto: And here, the idea is that this is because of the continuity from below. When we increase this X2, X3, XN,
18:40:540Paolo Guiotto: the set, this set here, this event, is increasing to what? To the union, and the union is exactly the event where capital X1, sorry, capital X1 is less than little X1.
18:55:330Paolo Guiotto: Okay? So this is, because of the continuity from… Below.
19:05:300Paolo Guiotto: But that's exactly the limit when X2 goes to plus infinity, etc. Xn goes to plus infinity of the
19:16:930Paolo Guiotto: CDF of the vector X, evaluated at point X1, X2, XN.
19:27:200Paolo Guiotto: So, informally, informally.
19:35:250Paolo Guiotto: we write that DCDF of X1 at point little x1 is the CDF of the array X at point X1, and all the other components are equal to plus infinity.
19:51:10Paolo Guiotto: And similarly, we can do for the second component, the CDF of X2,
19:58:20Paolo Guiotto: will be F sub X of plus infinity X2 plus infinity, etc, plus infinity, and so on.
20:10:410Paolo Guiotto: So, from the CDF of the array, we can deduce all the CDFs of the components. So… from…
20:24:380Paolo Guiotto: F… X.
20:27:440Paolo Guiotto: We… can get.
20:33:60Paolo Guiotto: FXJ, or J equal 1 to N. That's why
20:40:140Paolo Guiotto: Another, another jargon here used is that this is the… this… this… Is… Also… called the… the joint…
20:59:540Paolo Guiotto: CDF.
21:02:110Paolo Guiotto: Why these are called the marginals.
21:05:810Paolo Guiotto: Margins, you know?
21:09:960Paolo Guiotto: CDF.
21:12:460Paolo Guiotto: So, you can get from the joint all the marginals.
21:17:340Paolo Guiotto: But the vice versa, it's not, let's say, possible, in the sense that if you have only the distribution of the marginals, you cannot get the distribution of the array, because this depends on how these marginals are related. We will see better, later.
21:37:180Paolo Guiotto: this factor. This is possible only if we know that certain specific random variable, it is possible to have this relation one-to-one.
21:49:740Paolo Guiotto: Another concept is the concept of density.
21:56:880Paolo Guiotto: And, absolutely continuous multivariate.
22:04:270Paolo Guiotto: random variables.
22:07:150Paolo Guiotto: So… We say that, definition we say, bets.
22:19:500Paolo Guiotto: X.
22:20:880Paolo Guiotto: is absolutely… continuous.
22:26:360Paolo Guiotto: If, as for the Scala case, there is a density, so if there exists a function.
22:34:530Paolo Guiotto: that we still call F sub X, which is now a function of n variables, the same number of variables as the size of the array X, defined on Rn.
22:49:380Paolo Guiotto: Which is positive valued.
22:54:130Paolo Guiotto: Such that, well, the, law of X, huh?
23:02:320Paolo Guiotto: on a set E is the integral on E of the density FX, X1, XN… With respect to…
23:13:980Paolo Guiotto: Dealer-backed measure.
23:15:830Paolo Guiotto: So, informally, we say…
23:24:410Paolo Guiotto: Since, not the… the… if you look at the definition of flow of X,
23:30:760Paolo Guiotto: low of X is the probability that X belongs to E, but we know that the measure of E can be always written as the integral of E on E of 1 with respect to the measure mu X. So, if this integral, this integral here, is
23:51:90Paolo Guiotto: this integral, so this is the integral on e of 1 d mu X, it means that D mu X
24:01:270Paolo Guiotto: is the function FX, X1, Xn.
24:08:80Paolo Guiotto: times a die-Lebag measure, DX1, DXN.
24:13:620Paolo Guiotto: So, informally, we write also this in this way.
24:18:860Paolo Guiotto: Now, in particular, this, function FX, huh?
24:25:80Paolo Guiotto: Which is, positive.
24:27:820Paolo Guiotto: If you take the set E equal the full space, since 1 is equal to the probability of Rn, now this is the probability that X belongs to Rn, this would be, of course, a sure event.
24:46:420Paolo Guiotto: This means that this integral on our end of the density must be equal to 1.
24:52:930Paolo Guiotto: So A density will be a function, a positive function, whose integral is equal to 1, such that you can express these probabilities. So these are, again, the probability that X belongs to E.
25:09:160Paolo Guiotto: As this integral.
25:13:260Paolo Guiotto: This function, FX is, also called FX is called D.
25:22:660Paolo Guiotto: is called… joint.
25:29:130Paolo Guiotto: Density.
25:35:920Paolo Guiotto: From this formula that we mentioned above.
25:42:450Paolo Guiotto: the formula that computes the expected values of any function of the array X. It is the integral on our n of the numerical function phi times the measure mu X. Since the measure mu X is FX times the Lebec measure, we get this.
26:03:380Paolo Guiotto: In this case…
26:09:390Paolo Guiotto: the expected value
26:11:890Paolo Guiotto: of any function phi of the random variable x, which is the integral on Rn of the function phi, little x1, xn.
26:24:920Paolo Guiotto: In the measure, the law of the variable X,
26:30:730Paolo Guiotto: Since the law is F times the bad measure, this integral becomes the integral on Rn of the function phi X1, XN
26:45:180Paolo Guiotto: F, X,
26:47:130Paolo Guiotto: X1, XL, DX1… be upset.
26:58:700Paolo Guiotto: Okay.
27:00:90Paolo Guiotto: Exactly as in the case of CDF,
27:08:100Paolo Guiotto: where we here determine the CDF of each of the components. If we have a density for the array, we have a density for each of the components, so it turns out that
27:23:930Paolo Guiotto: proposition… if, excellent.
27:30:730Paolo Guiotto: is… Well, let's say if X equal X1XN.
27:38:970Paolo Guiotto: is absolutely… continu Then, each XJ… is absolutely continuous.
27:53:230Paolo Guiotto: So, all the components are absolutely continuous, and…
28:00:290Paolo Guiotto: the density of F, of XJ,
28:06:990Paolo Guiotto: is obtained by integrating the density, the joint density, of X1.
28:14:540Paolo Guiotto: Excel.
28:16:370Paolo Guiotto: But respect to all the variables exact to the variable XJ, So, in DX1…
28:23:540Paolo Guiotto: DXJ minus 1, DXJ plus 1, DXN.
28:31:940Paolo Guiotto: So this will be an integration on Rn minus 1.
28:37:520Paolo Guiotto: And this is because, if we look at the FXJ,
28:46:520Paolo Guiotto: the CDF of the variable XJ,
28:51:260Paolo Guiotto: Well, we said above that,
28:54:690Paolo Guiotto: there is this formula. Well, let's do for X1, it's the same for the other two simplified denotations. So, X2 for X1. So, for example, the CDF of X1
29:07:800Paolo Guiotto: He is, F, the joints, the F,
29:13:790Paolo Guiotto: Evaluated on X1 and plus infinity, plus infinity, etc, plus infinity.
29:20:840Paolo Guiotto: But now, since there is,
29:24:970Paolo Guiotto: there is a, this is the probability that X is less than X1, and the other are free, no? Because X2 less than plus infinity means X2 is in R, etc. So this is, if you want the mu X,
29:43:340Paolo Guiotto: Of the set minus infinity.
29:47:470Paolo Guiotto: to X1, then we have, copies of R, N minus 1 copies of R.
29:55:430Paolo Guiotto: These are N minus 1 cognas.
29:59:230Paolo Guiotto: So this will be the integral on that domain, minus infinity 2X1 times R, R, etc, and minus 1 times
30:12:110Paolo Guiotto: Since the, we are assuming that X is absolutely continuous, so there is a joint density, this means that the probabilities are computed through the density FX.
30:27:610Paolo Guiotto: from this formula. So, I… I put here FX,
30:33:20Paolo Guiotto: X1, X2, etc, XN in the LeBac measure, DX1, DXN.
30:47:70Paolo Guiotto: But this means that if we use now the Fubini theorem.
30:54:920Paolo Guiotto: We can split this multiple integration into N integrations, and actually, we will split into two integrations. One is the integration in the first variable.
31:07:620Paolo Guiotto: So I have the integration for the first variable… I'm sorry, it's better if I call Y1 the variable for X, because I'm using X1 for that. So this… let's call this Y1.
31:22:940Paolo Guiotto: BY1.
31:24:780Paolo Guiotto: So this will be for the first variable.
31:29:740Paolo Guiotto: And then there will be an integral in the other variables, dx2, EXN.
31:36:110Paolo Guiotto: Of course, of the density, FXY1X2.
31:41:950Paolo Guiotto: XN…
31:44:500Paolo Guiotto: Now, what is the range? For X2, Xn, the range is this one, so I have R times R, N minus 1 times, because this is the range where there are X2, the array from X2 to Xn, belongs here. The first variable is here.
32:04:780Paolo Guiotto: Because the full array, Y1, X2, Xn, is in that set. So for Y1, I have the integration from minus infinity to X1.
32:15:40Paolo Guiotto: And for, yeah, that's, I have an integration on R.
32:19:210Paolo Guiotto: So, I can see this as integral from minus infinity to X1 of this integral on Rn minus 1 of this function, FXY1X2, XN
32:35:190Paolo Guiotto: DX2.
32:36:680Paolo Guiotto: the Excel…
32:39:550Paolo Guiotto: in the DY1 variable. So, if you call this function for a moment, say, well, let's give a name F1 of Y1,
32:53:550Paolo Guiotto: I'm saying that the CDF of X1 evaluated at point little X1 is the integral from minus infinity to X1 of this function f1, y1 in DY1.
33:10:670Paolo Guiotto: And this exactly means that the function the CBF affects one… is differentiable.
33:23:210Paolo Guiotto: Almost everywhere, and… The derivative of this…
33:30:890Paolo Guiotto: the derivative with respect to X1 of this function, is exactly…
33:38:380Paolo Guiotto: the function you have inside the integral, so the F1 evaluated at point x1.
33:45:660Paolo Guiotto: So this is saying that X1 is absolutely continuous.
33:53:270Paolo Guiotto: And the density of X1 at point X1 is exactly that function F1X1.
34:02:510Paolo Guiotto: Which is, by definition, that integral. The integral on Rn minus 1 of FX, so the joint density, evaluated at point X1
34:17:460Paolo Guiotto: X2. Xena?
34:23:30Paolo Guiotto: And the integration is on X2XL.
34:29:429Paolo Guiotto: And this explains, why we have,
34:34:30Paolo Guiotto: We do have this formula. So, we can compute the density of each of the components of the array by integrating the joint density with respect to all the other variables.
34:47:860Paolo Guiotto: Now, you may wonder, again, it is possible… so, basically, the message here, knowing the joint density, I know the density of the margins, of the components.
34:58:260Paolo Guiotto: Is it possible to do the vice versa? No, generally it's not possible. We will see better later this problem, because this depends on how these variables are, let's say, for the moment, it's vague term, related, okay?
35:14:760Paolo Guiotto: In one special case, it is possible to do, let's say.
35:22:780Paolo Guiotto: to reconstruct the joint density from the marginal, this is an important case, which is the case of independent random variables we will see later. So, where the joint density is just the D product, algebraic product of the marginals.
35:39:570Paolo Guiotto: Okay, so these were just a few definitions, and so just to have a language. Now, let's see an important, very fundamental example of a multivariate, a random variable, which is the Gaussian multivariate.
35:59:50Paolo Guiotto: So… quotient.
36:15:500Paolo Guiotto: This is, very important.
36:20:370Paolo Guiotto: in application, it's just an extension of the univalid, one-dimensional random variable. So, we remind that.
36:40:920Paolo Guiotto: a random variable.
36:43:360Paolo Guiotto: X is normal with the mean M.
36:47:740Paolo Guiotto: And the variance sigma squared.
36:50:910Paolo Guiotto: Beef.
36:52:850Paolo Guiotto: X is… Absolutely continuous with… density.
37:03:890Paolo Guiotto: FX of little x, so this is a scalar variable.
37:09:300Paolo Guiotto: Equal 1 over root of 2 pi.
37:13:600Paolo Guiotto: Sigma square.
37:15:280Paolo Guiotto: E minus X minus the mean square divided 2 sigma squared.
37:24:680Paolo Guiotto: X in R.
37:30:160Paolo Guiotto: Now, what is the analogous of this thing for an array?
37:36:560Paolo Guiotto: So, we have this definition.
37:41:970Paolo Guiotto: So, let… Excellent.
37:49:500Paolo Guiotto: from omega to RN, B. A multivariate.
38:03:170Paolo Guiotto: random variable.
38:05:740Paolo Guiotto: So, we say that…
38:13:760Paolo Guiotto: X.
38:15:910Paolo Guiotto: is Goshen.
38:22:560Paolo Guiotto: with… Mean… M…
38:30:350Paolo Guiotto: Now, remember that for, for Scala, Gaussians, for Gaussian random variables, we have this, that the expected value of X is the value M, so that's why we call M the mean, because it is the mean value of the random variable.
38:48:520Paolo Guiotto: And the variance of X is exactly so, the expected value of X.
38:55:140Paolo Guiotto: minus, if you want the mean square, this is exactly sigma squared. So, that's why we call sigma squared the values, no?
39:04:260Paolo Guiotto: Now…
39:05:410Paolo Guiotto: Starting with the mean. If X is now an array, you have to expect that the expected value of an array, what should be? Well, you may imagine it is an array I will define as a vector whose components are the expected values of each of the components.
39:22:520Paolo Guiotto: So, it's, the expected value of X is the factor made of the expectations of each of the components. So, in particular.
39:35:700Paolo Guiotto: be a vector of the same size of the, of the, of the array. So M is a vector in Rn.
39:48:300Paolo Guiotto: Now, about the deep variants, this is more complex, and, so let's, let's do, because the point is, okay, so what should be now should become sigma squared, that's sigma square.
40:03:380Paolo Guiotto: You also imagine that if you want to have a density now.
40:09:320Paolo Guiotto: This new density will be a function of a vector, X minus n, so now what is X minus n squared, for example? N is a vector, X is attracted to be a vector of the same size, so X minus n is… that's the difference between two vectors, but what is the square?
40:30:130Paolo Guiotto: So, this is not defined, and
40:34:340Paolo Guiotto: It should be, it should be a scholar if you want to use the exponential, because that's a numerical function.
40:43:280Paolo Guiotto: So a natural way to think is to transform that square into the product of the vector by itself. So X minus n dot X minus N is like X minus… if we are in the measurement, it is X minus N squared. So that X minus N could be X squared, could be transformed into X minus n dot of the X minus n.
41:06:130Paolo Guiotto: But what about the minor atmosphere? Now, this is the reciprocal of something that here becomes a matrix. So… and this is, so it's Gaussian with mean M, and…
41:22:850Paolo Guiotto: covariance.
41:28:70Paolo Guiotto: Of course, these two are called mean and covariance, because they will turn out to be the mean and the covariance of this array. And covariance C…
41:40:950Paolo Guiotto: where BC must be.
41:45:30Paolo Guiotto: C is.
41:48:690Paolo Guiotto: N by N… symmetric…
42:02:230Paolo Guiotto: And also, it must be, as you will see in a second, positive.
42:08:430Paolo Guiotto: Positive. Definite.
42:12:540Paolo Guiotto: metrics.
42:16:710Paolo Guiotto: Now, if you do not remind this, the meaning of this, well, this means just that C is applied to a vector V,
42:29:340Paolo Guiotto: scalar product with V. So, I have my matrix. I multiply line by column with the vector V,
42:37:150Paolo Guiotto: And I do… I get a vector, I do the scalar product with V. So this is informally CV square.
42:45:50Paolo Guiotto: Okay, so if these were numbers, this would be CB squared, and so C positive would mean that this quantity is positive for every, V in RM.
42:59:250Paolo Guiotto: Now, since we say positive definite, actually we want slightly more than this. We want that this quantity to be always positive, strictly.
43:08:180Paolo Guiotto: Except the unique case when this is not possible, and it is the case when the vector V is 0. So we can say this is positive for every vector V of R, and except the vector of 0.
43:21:390Paolo Guiotto: So this is what it means, positive definite metrics.
43:25:660Paolo Guiotto: Okay, so we say that X is Gaussian with mean m and covariance C, If… X is absolutely continuous.
43:38:600Paolo Guiotto: with density.
43:43:740Paolo Guiotto: And now let's see how this formula for the density of one-dimensional, Gaussian, for the Scalar Gaussian, becomes. So, FX
43:57:40Paolo Guiotto: Well, instead of writing X1XM, okay, which would create a mess in the notation, let's keep X, but keep in mind that here X is a vector, okay? So I write FX of X, let's write down here, X is in Rn now.
44:16:270Paolo Guiotto: So keep in mind that this is a vector. Equal the scaling factor. Now, this is for dimension 1,
44:23:290Paolo Guiotto: For dimension n, it's a sort of product of this Kalinic factor n times. So you have 1 over the root… imagine that you multiply something like this n times, you first get 2 pi to power n, so 2 pi to power n.
44:42:370Paolo Guiotto: Now, what is sigma square here? What replaces this sigma square is the determinant of the matrix C,
44:52:740Paolo Guiotto: quantity, which is… I will return in a second, because of this
44:59:850Paolo Guiotto: factor. It is… it is… the determinant, of course, is well-defined, but the point is that it is, different from zero, because the metrics verifying this condition is, in particular, inventible.
45:13:830Paolo Guiotto: And, as we'll see in a moment, so the determinant cannot be zero, so there is no danger we are dividing by 0.
45:20:420Paolo Guiotto: Then we have the exponential E minus 1 half, that's the one half that you have here, this part. And now the part X minus M squared divided sigma squared becomes…
45:33:670Paolo Guiotto: the inverse of the matrix C applied to vector X minus the mean M,
45:40:560Paolo Guiotto: So X minus M is a vector. I apply to the matrix C minus 1.
45:46:230Paolo Guiotto: I went into the second one. If we say it a second ago, it is embeddable under these assumptions, so C minus 1 is well-defined. I get the vector, and I do the scalar product with X minus M again. So this is what becomes that X minus M squared divided by sigma squared.
46:06:720Paolo Guiotto: Now, let's just fix this point, remark.
46:11:480Paolo Guiotto: Since, C is positive definite.
46:17:440Paolo Guiotto: C is invertible.
46:23:780Paolo Guiotto: So, in particular, there exists the inverse matrix, and the determinant of C will be different from zero.
46:33:120Paolo Guiotto: That's why, because we just, since it is an N by N matrix, we just need to… we can just reduce to verifying that, as a map, RN to RN, this is injective.
46:50:30Paolo Guiotto: Okay? So this is, equivalent…
46:55:520Paolo Guiotto: to say that C is injective.
47:03:850Paolo Guiotto: as linear map.
47:10:470Paolo Guiotto: from RN to RN.
47:16:230Paolo Guiotto: This is a factor.
47:18:990Paolo Guiotto: that comes from linear algebra. And indeed, this is true, because, indeed.
47:26:990Paolo Guiotto: If, let's say, to track injectivity for linear maps, it is sufficient to prove that when the metric C multiplied by vector V is 0, then necessarily that vector V must be equal to zero, okay?
47:43:730Paolo Guiotto: So, if CV is equal to zero with
47:49:920Paolo Guiotto: be different from zero, so let's say that it is not injective. Then, multiplying by V, so doing CV's color V,
47:59:660Paolo Guiotto: Since V is different from 0, and the matrix is positive, definitely this quantity would be positive.
48:06:350Paolo Guiotto: Because C is positive definite, and we reminded here what does it mean, no? CV dot V is strictly positive for every vector except vector 0. So if we assume that CV is 0,
48:20:50Paolo Guiotto: with V different from 0, we would have CV scala V positive, but at the same time, since this is 0, we would have this is equal to zero scala V, which is 0, and that would be impossible.
48:35:30Paolo Guiotto: So, this explains why, under these assumptions, these metrics
48:43:560Paolo Guiotto: is invertible, and therefore this formula makes sense. There is no danger.
48:49:170Paolo Guiotto: Another important thing that we have to remind that will be very helpful in calculations, so let's put this fact here, because we will use it.
49:01:390Paolo Guiotto: is that this matrix C, which is N by N symmetric
49:08:170Paolo Guiotto: you know that symmetric mattresses are diagonalizable, okay? So, since C is symmetric.
49:21:300Paolo Guiotto: This is an important fact to keep in mind. Then, C is… Bye, yeah.
49:38:180Paolo Guiotto: So, this means that, That is…
49:43:260Paolo Guiotto: There exists a matrix, which is actually a change of base, let's say, T. Now, this matrix is an orthogonal matrix, orthogonal.
49:59:400Paolo Guiotto: metrics, huh?
50:02:30Paolo Guiotto: Orthogonal means that, T times the transposed of T,
50:10:560Paolo Guiotto: is equal to the identity of the space. Here we are in dimension N, so the identity of Rn. Now, notice that in particular, this identity means that the orthogonal matrix is also the inverse matrix of T, okay?
50:29:340Paolo Guiotto: In particular, so… in particular, The orthogonal matrix is also the inverse matrix of it.
50:43:490Paolo Guiotto: So there existed this orthogonal matrix, such that if you do T, C, T transposed, you get a matrix which is diagonal, let's say D,
50:56:650Paolo Guiotto: diagonal… metrics.
51:01:650Paolo Guiotto: So this means that the matrix D is a matrix where all the elements are zero, except at most those on the diagonals, which are also… they are important numbers, they are the eigenvalues of the matrix C. So let's call them… here we will use this notation.
51:20:420Paolo Guiotto: Well, let's use for a moment lambda 1, lambda 2, etc, lambda N, and 0, 0 here.
51:30:00Paolo Guiotto: So where… these numbers, lambda J, R, the… Now you can… values… of sleep.
51:47:90Paolo Guiotto: Okay, so this is just because C is symmetric.
51:50:790Paolo Guiotto: If I now add the fact that C is positive definite.
51:56:110Paolo Guiotto: Okay, so this is just related to the symmetry of the matrix.
52:02:160Paolo Guiotto: If, moreover.
52:08:490Paolo Guiotto: C is positive definite.
52:12:720Paolo Guiotto: Then we have that these eigenvalues easily are positive. The lambda J are positive for every j.
52:23:170Paolo Guiotto: And therefore, we call them, for a reason that it will be clear later, since they are positive, we're baptized as a sigma J square, so they will be the squares of some numbers, okay?
52:37:950Paolo Guiotto: So this is just a notation, a convenient notation, because it will turn out that these sigma J are… have a meaning very similar to the old sigma J that… the old little sigma squared that we have here.
52:51:290Paolo Guiotto: And this because… This… Because…
53:01:10Paolo Guiotto: You know that, since the lambda J are the eigenvalues, there are eigenvectors, so there are vectors different from zero.
53:10:390Paolo Guiotto: So this is because there exists a vector VJ such that C of VJ is equal to lambda J, VJ different from 0, no? This is the
53:22:280Paolo Guiotto: definition of the eigenvalues. There is just an eigenvector associated to that value, lambda J, for which the metrics transform into, basically, itself, modulo a factor.
53:35:990Paolo Guiotto: Now, from this, if we multiply in the scalar product by VJ, we get that CVJ dot VJ would be equal to lambda J vJ dot J.
53:48:890Paolo Guiotto: But at right, we have lambda J,
53:52:950Paolo Guiotto: the scalar product of a vector by itself is the square of the norm of the vector, okay? So, norm of VJ
54:03:410Paolo Guiotto: square. Here, the dot is just the Euclidean dot, okay? So, remind that U dot V here is some UJVJ, so say UIVI, do not…
54:18:440Paolo Guiotto: Use the same index from 1 to n, no? So when you do the product of a vector by itself, you get the sum of the squares, which is the square of the Euclidean norm.
54:28:490Paolo Guiotto: But on the other side, you have CVJ dot VJ, which is supposed to be positive, because the vector VJ
54:36:300Paolo Guiotto: is different from 0 and C is positive. So we have that lambda J times this square of normal VJ would be a positive number, and this is possible if and only if the lambda J is a positive number.
54:53:610Paolo Guiotto: Okay?
54:55:610Paolo Guiotto: So… Let's, summarize.
55:00:370Paolo Guiotto: So… 4… D.
55:06:610Paolo Guiotto: Covariance metrics.
55:12:770Paolo Guiotto: seat of a motion.
55:18:210Paolo Guiotto: distribution…
55:24:240Paolo Guiotto: We have, huh?
55:28:790Paolo Guiotto: that T will exist the matrix T, such that T times T transposed is the identity, such that TCT transposed is the diagonal matrix with the elements sigma 1 squared
55:47:80Paolo Guiotto: sigma n squared on the diagonal. We write it just in this way, okay? So this stands for the diagonal matrix with the death diagonal.
55:59:480Paolo Guiotto: This is an important fact to keep in mind. Now.
56:04:900Paolo Guiotto: To do some calculation… oh, I forgot to…
56:09:260Paolo Guiotto: To introduce a notation for this, let's add here. We write…
56:17:650Paolo Guiotto: for this kind, exactly, we use the same annotation, exactly as here, for scalar variables, we say X is normal, mil m.
56:28:690Paolo Guiotto: variance sigma squared. For a multivariate, we use a notation which is pretty much the same. We say normal, mean m, and we put the variance matrix C here, because you just need, of course, these two
56:42:240Paolo Guiotto: ingredients to describe, or to fully describe, the Gaussian random variable.
56:49:500Paolo Guiotto: Now, let's do some, are the work, so calculations.
56:55:980Paolo Guiotto: So we proved this, fact, proposition.
57:00:120Paolo Guiotto: If X is multivariate Gaussian, so mean M covariance matrix C, Then,
57:11:300Paolo Guiotto: the expected value of X, which is, say, X is the array made of X1, XN.
57:21:590Paolo Guiotto: And so this will be, by definition, the array of the expectations of the components, expectation of X1, expectation of X2, etc, expectation of X
57:37:990Paolo Guiotto: N… well, this array is just the vector n.
57:43:570Paolo Guiotto: And you remind that, We introduced, last time, the definition of covariance.
57:53:950Paolo Guiotto: The covariance matrix, so given a multivariate, generic multivariate random variable made by components X1, XN, the covariance matrix of that multivariate is the matrix made by the covariances of these components.
58:12:760Paolo Guiotto: Well, it turns out, of course, that the covariance matrix of X is the matrix C that we have into the Gaussian.
58:20:520Paolo Guiotto: So that's… this is why we call mean the mean and the covariance the covariance. So, the covariance matrix… so let's say that the covariance matrix of font, covariance between X psi
58:36:460Paolo Guiotto: And XJ is exactly CIJ, where CIJ is the element at line I, column J, of the covariance matrix of the distribution.
58:54:260Paolo Guiotto: I hope it is clear what I'm saying. Now, the covariance matrix is an object that we can define for every array of random variables.
59:04:470Paolo Guiotto: And it is written here. It is a matrix made by… the entries are the covariances between each combinations of each pair of
59:17:910Paolo Guiotto: components, Xi XJ, okay? So this is an object.
59:21:960Paolo Guiotto: Now, I defined the covariant… the Gaussian distribution with the mean M and the covariance method C as a random variable whose density is this thing, okay?
59:35:790Paolo Guiotto: Now, what we verify is that that matrix C turns out to be the covariance matrix of this array, and that vector M turns out to be the mean value of that distribution. So that's why we call, of course, them mean and variance of the… mean and covariance of the distribution.
59:56:680Paolo Guiotto: So, since to do this calculation, it's elementary, but not, let's say, 100%, let's say, trivial.
00:10:360Paolo Guiotto: Perhaps, if… I don't know, but let's do. It's some… artwork that,
00:18:720Paolo Guiotto: Perhaps it is worth. Now, since the expectation is the array of the expectation, so each of these components is, no, expectation of, I should say, expectation of XJE,
00:35:610Paolo Guiotto: Is, the, is, the,
00:43:60Paolo Guiotto: Yeah, for example, let's do… let's take the question in this way. Now, it's the expected value of the XJ. So, since we know that this array X has a joint density
00:56:230Paolo Guiotto: FX, we just say that, in general, whenever you have a joint density, you have also marginal densities, so each of the components is itself an absolutely continuous random variable. So, if I want to compute this expected value.
01:15:410Paolo Guiotto: I should say, okay, this is the integral on R, because XJ is a scalar variable, of the, let's say, X, let's call XJ the variable, times the density of XJ
01:32:400Paolo Guiotto: in the XJ, so I should compute this. So the question is, what is this density?
01:37:950Paolo Guiotto: So, how can I compute the density of FXJ?
01:42:570Paolo Guiotto: of XJ. So the density of X day.
01:46:670Paolo Guiotto: Let's see if we can do…
01:50:600Paolo Guiotto: evaluated the point XJ is the integral
01:57:630Paolo Guiotto: is the integral… but then we see some other method to do this calculation. However, is the integral on Rn minus 1 of… I should take the joint density, I should integrate in all variables except the variable XJ, so…
02:17:680Paolo Guiotto: Let's do for the… it is the same, just for convenience of notations, let's do for the first index. So let's take the first component, X1, in such a way that we have to integrate on X2, XN minus 1,
02:35:580Paolo Guiotto: the joint density, DX2, DXN minus 1.
02:43:60Paolo Guiotto: Now, this is… this quantity here is 1 over root of 2 pi to the n, the determinant of the covariance matrix C, E minus 1 half C minus 1.
02:59:240Paolo Guiotto: So let's, give a name. This is the vector X, so X minus M dot X
03:09:170Paolo Guiotto: minus M, but the integration is not on the full array X, but on X2, XN, DX2DXN.
03:21:440Paolo Guiotto: Now, another interesting thing to notice here is that this calculation at the end is simple, but rather tricky, because you have to handle with these vectors, etc.
03:37:470Paolo Guiotto: So we can do, but later, we will see with a different tool that we have still to see, which is basically the Fourier transform, by the way.
03:47:560Paolo Guiotto: that this same calculation could be made in a very, very easy way with the use of Fourier transform. So this is to say that I will show you how to do a calculation. For example, in this case.
03:59:770Paolo Guiotto: that uses the density, but that maybe is not the best way to do this calculation, so there are maybe other ways to compute the same thing, and that's pretty normal. Okay, so, the first point is that
04:16:850Paolo Guiotto: This, this, this, calculation, is not…
04:29:260Paolo Guiotto: So I would like to extract the, the, the, this, how this depends on X1.
04:37:820Paolo Guiotto: So it will be a little bit messy because of, of this,
04:46:880Paolo Guiotto: Okay, but let's take, let's imagine that we take the density, the full density, as function of,
05:06:660Paolo Guiotto: Maybe it's better if I keep,
05:09:570Paolo Guiotto: I think… why I'm thinking here? Because,
05:15:650Paolo Guiotto: I want to profit off this diagonal structure, but this diagonal structure, I can diagonalize when I involve all the variables. So, let's work with this integral. So, basically.
05:28:700Paolo Guiotto: Mmm, it is, better, instead of using this formula.
05:35:570Paolo Guiotto: If I do this, you remind that this is integral of XJ. Now, the J, the density of the J variable is itself an integral on Rn minus 1 of the full density.
05:51:970Paolo Guiotto: respect to all the variables, X1, XN, except the integrated in all variables except the J variable. So if I take the first variable, so let's keep the first index here, 1,
06:10:540Paolo Guiotto: 1, so in such a way that I have one here, EX1 down here.
06:16:640Paolo Guiotto: So this will be an integration in X2.
06:19:770Paolo Guiotto: DXN, because this quantity here
06:27:50Paolo Guiotto: You see, this is the integration, the integral of the joint density with respect to the variables X2, Xn, and this is exactly… I'm doing the opposite way. It's this formula here.
06:42:440Paolo Guiotto: where I use the integral to represent the density. So in this wave.
06:50:180Paolo Guiotto: This becomes an integral on Rn, of X1,
06:56:980Paolo Guiotto: Versus the joint density, X1, Xena.
07:01:590Paolo Guiotto: integrated in DX1, DXN.
07:07:450Paolo Guiotto: Okay.
07:09:370Paolo Guiotto: Now, here, it is where I…
07:13:890Paolo Guiotto: I use the formula for the density, 1 over root of 2 pi, to power n, determinant C,
07:25:640Paolo Guiotto: E minus 1 half C minus 1, X minus M, dot X minus M,
07:34:520Paolo Guiotto: DX, let's say DX stands for… DX1 for DXN.
07:41:520Paolo Guiotto: Now, to compute this integral, I want to diagonalize C. I want to do a change of variable that diagonalizes this matrix C. What is the change of variable? So I want to make here to appear something like a writing.
07:57:130Paolo Guiotto: In, in, in blue. I do not have these two matrices, okay?
08:03:220Paolo Guiotto: But I could create these two matrices. Let's see how. Because, well, if you want, take the… this formula here.
08:17:130Paolo Guiotto: And, so we have… So…
08:23:20Paolo Guiotto: I want to write this in diagonal form. I know that TC, T transposed is a diagonal matrix, where this is diagonal.
08:35:660Paolo Guiotto: Okay, so I want to write C minus 1.
08:41:470Paolo Guiotto: So, first, I extract C. How can I do? Well, remind that T times T transposed is the identity. So, T transposed is the inverse matrix of T. So, if I multiply by T transposed, left and right side, this thing disappear, is the identity.
09:01:510Paolo Guiotto: So I will get CT transposed equal T transposed D.
09:07:670Paolo Guiotto: And if now I multiply everything by T at right.
09:13:550Paolo Guiotto: Here, usually, T transpose T, but this is also the identity, because if you want,
09:22:130Paolo Guiotto: you can multiply both sides by T transpose, you have T transposed t, times T transport…
09:31:740Paolo Guiotto: is T transposed, and since TT transpose is impactable, you can multiply by the inverse, which is T, and this becomes the identity. So again, T transpose T, or TT transpose, they are both equal to the identity.
09:46:319Paolo Guiotto: T and T transpose are each diverse of the other, so when you multiply, you get the identity. So this simplifies to the identity.
09:56:440Paolo Guiotto: And finally, we get C is T transposed DT.
10:01:650Paolo Guiotto: So, now I can take the minus 1. C minus 1 is the minus 1 of this product, T transpose DT.
10:14:930Paolo Guiotto: Now, you know what is the inverse of a product? All these matrix are invertible, because T, T transpose are invertible, D is the diagonal matrix with positive elements on the diagonal, that's invertible. It's, again, a diagonal matrix with the reciprocals on the diagonal.
10:34:170Paolo Guiotto: When you do the product, this is not the product of the inverses, you have always to invert the other, no? So this is T minus 1,
10:41:960Paolo Guiotto: D minus 1, then there would be T transpose minus 1.
10:46:790Paolo Guiotto: But the inverse of T is T transpose, so D minus 1, and the inverse of T transpose D is T, no? So this is T.
10:56:310Paolo Guiotto: So at the end, we have this formula. C minus 1 can be written as T transpose DT. And that's what I now plug into that integral. So let's go…
11:08:560Paolo Guiotto: Back to the integral. I have integral on Rn of X1E minus 1 half, which is the spelling factor I will write later. C minus 1 is E.
11:24:60Paolo Guiotto: T transposed DT applied to vector X minus m dot X minus M.
11:36:590Paolo Guiotto: Then we have index, and put here the scaling factor, which is root of 2 pi to the n determinant of the matrix C.
11:49:660Paolo Guiotto: Okay, now… This T transposed here.
11:55:280Paolo Guiotto: we can carry on the other factor. You know that when you switch a matrix from one factor to the other of the scalar product, this matrix is transposed, no? So you have, you have…
12:07:250Paolo Guiotto: T transposed, I don't… M, but let's say, V dot W,
12:16:570Paolo Guiotto: When you move that matrix, T transposed, to the other factor, it becomes T, right? The transpose of… it would be the double transport, so it is
12:27:400Paolo Guiotto: V dot TW.
12:31:160Paolo Guiotto: So, here we have… Integral on our N.
12:36:110Paolo Guiotto: of X1E minus 1 half the diagonal matrix, where perhaps I forgot at E-1, yeah.
12:46:150Paolo Guiotto: D minus 1. That is a D minus 1.
12:50:20Paolo Guiotto: So, D minus 1… T… X minus M, dot, we say, the TX minus M.
13:01:550Paolo Guiotto: Then we have dx divided root of 2 pi to the n determinant of the matrix C.
13:10:20Paolo Guiotto: Okay, now it's time to change variable, because if I change variable, and I call this vector here Y,
13:20:940Paolo Guiotto: So I set Y equal T times XM minus M.
13:26:380Paolo Guiotto: Remind this is a transformation Rn to RN, okay?
13:31:700Paolo Guiotto: So, what happens when I change the integral? Now, the domain, since this map is basically… is TX,
13:42:560Paolo Guiotto: minus TM, so it's in a fine map. The important part of this one, which is the detection, this is a constant.
13:53:520Paolo Guiotto: added to this, and this is an injection, okay, so it's a good change of our form. So what happens? When I take an X in RN, and I multiply this, we get all vectors of RN, because that's an injection. So the integration domain remains RN.
14:14:800Paolo Guiotto: Now, X1 is what? X1… I should extract X1 from this. Well, I can do, because T minus 1Y is X minus M, so X is equal to T minus 1Y.
14:32:210Paolo Guiotto: plus M. So, in other words, T minus 1 reminds that the inverse of T is the transposed matrix of T, so this is T transposed Y plus M.
14:46:860Paolo Guiotto: So, X1 is the first component of this factor, so let's write this. P transposed Y plus M,
14:56:490Paolo Guiotto: First company.
15:00:640Paolo Guiotto: Perhaps I write below, because it's a bit complex. So, integral on Rn of the first component of PY plus M,
15:13:400Paolo Guiotto: So this means, First, the… component.
15:21:880Paolo Guiotto: of vector, TY plus M.
15:25:300Paolo Guiotto: Because I have to take here X1.
15:29:00Paolo Guiotto: Okay?
15:31:50Paolo Guiotto: Then I have an E minus 1 half.
15:35:800Paolo Guiotto: Since I set Y equal T times X minus M, that becomes Y, so D minus 1
15:44:230Paolo Guiotto: Y dot Y.
15:47:440Paolo Guiotto: Then I have that. What happens to DX at DY?
15:52:460Paolo Guiotto: Now, since X is this, so X is,
16:02:160Paolo Guiotto: T transpose the Y plus M. What is DX?
16:07:370Paolo Guiotto: Now here, this is a multi-dimensional change of variable.
16:11:250Paolo Guiotto: So you remind that there is the modules of the determinant, of the Jacobian matrix of the change of variable, so the derivative of the change of variable. This is P0, because the change of variable is linear, so the derivative of that transformation is just the matrix we transported.
16:30:150Paolo Guiotto: So, what I have right there is the models of the determinant.
16:35:930Paolo Guiotto: of T transpose them, times the DY.
16:39:900Paolo Guiotto: And this comes down here.
16:42:180Paolo Guiotto: model's determinant of T transposed.
16:46:520Paolo Guiotto: DY, and then there is the scaling factor root of…
16:51:10Paolo Guiotto: 2 pi to the n determinant of C.
16:59:830Paolo Guiotto: Okay, we are almost over.
17:02:760Paolo Guiotto: Why?
17:04:730Paolo Guiotto: Because,
17:07:830Paolo Guiotto: First, what is this determinant? T is orthogonal matrix. I know that T transposed, T is the identity, right?
17:17:690Paolo Guiotto: So, if I compute the determinant to both sides, I have the determinant of T transpose the T is equal to the determinant of the identity.
17:28:670Paolo Guiotto: And that is equal to 1.
17:31:550Paolo Guiotto: But you know what is the determinant of… the product of two square matrices? It is the product of the determinants. So this is the determinant of T transposed times determinant of T.
17:46:490Paolo Guiotto: But the determinant of T transpose and the determinant of T are the same, so this is determinant of T
17:53:360Paolo Guiotto: Square.
17:56:800Paolo Guiotto: So if this determinant squared is 1, it means that the determinant of t is either plus or minus 1.
18:06:240Paolo Guiotto: In any case, that's why, because basically that T is a rotation in the space, so the rotations, they are always the tangent, plus 1 or minus 1. So that number is 1 at the end. Okay, so we eliminate it.
18:21:590Paolo Guiotto: Then, what else?
18:25:520Paolo Guiotto: Here, we have basically transformed, if you, if you forget this part… look at the, the factor e to minus 1 calculus 1Y dot Y. That's the factor that you have in any quotient. We may say.
18:44:600Paolo Guiotto: It is, it is…
18:49:890Paolo Guiotto: This part of the motion, there is a big. You see, we dash zero, and the C is replaced by D, okay? Look at this, no?
19:01:130Paolo Guiotto: So, if I have the same matrix down here, this would be the motion. But there, I have that the scaling factor is root of 2 pi to the n, that's good, determinant of C.
19:19:60Paolo Guiotto: If I have determinant of B, I am done, because that could be a Gaussian with, say, mean 0 and the variance matrix, now the diagonal matrix T. Now, the point is that, of course, the determinant of C is equal to the determinant of T. Why?
19:38:840Paolo Guiotto: Because determinant of C, you remind that the relation between C and D, we can get from this formula. C is T transpose DT. So, it is like determinant
19:52:30Paolo Guiotto: of, T transposed, DT.
20:00:120Paolo Guiotto: Of course, I mean, all these properties are linear algebra properties, so probably you already know, but if you are not completely familiar with that, let's quickly review this. Now, there is a remarkable property of determinant.
20:16:430Paolo Guiotto: In this case, we can use the property we said above. So, this is the determinant of the product of three square matrices. So, this is determinant of T transposed, determinant of the determinant of T.
20:34:590Paolo Guiotto: But we already said that the determinant of P times the determinant of T transport is 1. So this product is equal to 1, and that's equal to the determinant of T.
20:47:230Paolo Guiotto: So, at the end, we transformed our calculation into this one. So… The expected value.
20:55:810Paolo Guiotto: of, we say, the X1,
20:59:420Paolo Guiotto: Now, what changes if I do with XJ? The unique point where you see the 1 is here, you would have with the J component, so let's, let's write in general. It is the integral on Rn of this TY plus M
21:16:170Paolo Guiotto: J component, E minus 1 half D minus 1Y dot Y.
21:23:170Paolo Guiotto: DY over the root of 2 pi to the n
21:28:510Paolo Guiotto: determinant of D. So, basically, we transformed into the calculation with respect to a very particular Gaussian, because the fact that this is diagonal
21:40:810Paolo Guiotto: D minus 1Y dot Y.
21:44:780Paolo Guiotto: Now, this, the metrics remind that we said it is the matrix, since the elements on the diagonal, the values of the covariance are positive, we call them sigma 1 squared, sigma n squared.
22:02:280Paolo Guiotto: So the inverse of this matrix is nothing but the same diagonal matrix, but with the reciprocals on the diagonal. So we have 1 over sigma 1 squared, and here 1 over sigma n squared, 0, 0 here.
22:19:560Paolo Guiotto: So when you do D minus 1Y dot Y, you are doing the diagonal matrix with
22:28:350Paolo Guiotto: On the diagonal, 1 over sigma, 1 square, 1 over sigma n.
22:33:820Paolo Guiotto: Square, 00 here.
22:36:950Paolo Guiotto: times vector Y1.
22:39:580Paolo Guiotto: YN… dot Y1… Why y'all?
22:46:510Paolo Guiotto: But if you do the first product, that's easy, because you have lots of zeros. So you have these times by 1.
22:55:80Paolo Guiotto: plus 0 minus 22, plus 0 minus 23, plus 0, etc, by hand. So, when you do the first component of that vector, we'll be just…
23:04:90Paolo Guiotto: 1 over sigma 1 squared Y1, and the second will be 1 over sigma 2 squared Y2, and the third will be 1 over Y… sigma 3 squared Y3. The last one, 1 over sigma n squared YN.
23:20:640Paolo Guiotto: Scala product with Y1, white men.
23:25:790Paolo Guiotto: So, at the end, we get the sum.
23:28:870Paolo Guiotto: J goes from 1 to N of YJ squared divided by sigma j squared.
23:38:680Paolo Guiotto: And if you now plug back this into this integral, You get them.
23:46:50Paolo Guiotto: That the expected value of XJ… is the integral on Rn, TY plus MJ component.
23:59:960Paolo Guiotto: Then we have the exponential minus 1 half tam j over 1, going from 1 to 1, YJ squared plus sigma
24:16:500Paolo Guiotto: J square.
24:18:720Paolo Guiotto: DY1. DYN.
24:22:920Paolo Guiotto: Divided by the square in facto, which is the root of… 2 pi with the N… Determinante.
24:32:510Paolo Guiotto: Determinant, determinant of, the matrix,
24:37:390Paolo Guiotto: D is what? If you look at that matrix, the determinant is nothing but the prototype, the sigmas. So that's the sigma…
24:48:210Paolo Guiotto: 1 square sigma N squared.
24:52:260Paolo Guiotto: So, at the end, since you see that this density here, it's basically a product of densities, of one-dimensional densities.
25:03:80Paolo Guiotto: one exponential of minus YJ squared divided to sigma j squared. We give to this the corresponding factor, which is 2 pi
25:15:630Paolo Guiotto: root of 2 pi sigma j squared.
25:18:930Paolo Guiotto: You see that these are one-dimensional motions.
25:24:40Paolo Guiotto: So, in particular, this means that…
25:30:860Paolo Guiotto: So, what is the J component of this?
25:34:110Paolo Guiotto: Well, the jade component of M will be MJ.
25:38:160Paolo Guiotto: The J component of T, is what?
25:42:820Paolo Guiotto: So, you have matrix T on vector Y. You want the jade component.
25:48:360Paolo Guiotto: of that vector, you obtain that by doing the jade line of T times the vector Y. So let's say it's the scalar product of the… I write this, TJ dot Y, where this is the J
26:06:800Paolo Guiotto: Line.
26:08:800Paolo Guiotto: of E.
26:12:30Paolo Guiotto: So this is, basically what?
26:15:900Paolo Guiotto: this quantity, TJ, TY plus M, J component is, yeah, J component of M plus this product, TJ dot Y,
26:28:400Paolo Guiotto: So it is something like MJ plus… this is a Scala product, so let's say that we have some, TIJYI for I going from 1 to N.
26:43:120Paolo Guiotto: Something like this.
26:46:660Paolo Guiotto: And now we can, hopefully finish pickups.
26:51:620Paolo Guiotto: So, first we split the integral. The integral of MJ
26:58:730Paolo Guiotto: in that product, I don't copy J1 from N of this, so… this thing, huh?
27:09:600Paolo Guiotto: DY1DYN.
27:14:380Paolo Guiotto: Now, this is a Gaussian density, so this… we can even split the integral, we can carry outside the constant. It is MJ times the product J going from 1 to n of one variable integrals.
27:30:200Paolo Guiotto: E minus YJ squared divided 2 sigma j squared over root of 2 pi.
27:37:760Paolo Guiotto: sigma J squared, DYJ,
27:41:590Paolo Guiotto: But these are integrals of the Gaussian distribution. All these values are 1, because that's a probability distribution. So this is a product of 1s, and this yields MJ.
27:54:480Paolo Guiotto: While… when I do the integral, on Rn of the other component, so TJ scalar product with Y,
28:06:220Paolo Guiotto: And again, in the product density, J1 to N of blah blah blah, so the same thing also here.
28:17:800Paolo Guiotto: DY1.
28:20:180Paolo Guiotto: DYN.
28:23:140Paolo Guiotto: Now, since this scalar product is a linear combination of Ys, so I can say this is sum on I of TIJYI,
28:36:770Paolo Guiotto: I carry outside the sum. I carry out the T, the capital D, because these are scalars, so this, at the end, is the sum of i of T, iJ integrals on Rn.
28:49:830Paolo Guiotto: of some YI versus that product density, DY1, DYN.
28:58:310Paolo Guiotto: But that integrals are zero, because…
29:01:740Paolo Guiotto: You can always split, you see that this is involved, you can split this into the block, saying that you have product.
29:11:500Paolo Guiotto: Of integrals.
29:14:200Paolo Guiotto: So for J different from I, I am just integrating on R the density, E minus Y
29:23:690Paolo Guiotto: J squared divided by 2 sigma J squared over root of 2 pi sigma J squared dy j.
29:33:960Paolo Guiotto: There is just one integral which is different. It's that one for the height coordinate, so I have integral on R of Yi.
29:44:50Paolo Guiotto: E minus Yi squared divided 2 sigma i squared over root of 2 pi.
29:52:40Paolo Guiotto: sigma I DYI.
29:55:230Paolo Guiotto: Now, these integrals are all equal to 1 because of the… these are Gaussian densities, so they are probability distributions, and this integral, instead, is equal to 0.
30:08:880Paolo Guiotto: Because this is the integral of a NOD function.
30:12:420Paolo Guiotto: You see? You can compute this one directly, because you have a derivative here, but you don't need… this is odd…
30:21:740Paolo Guiotto: function.
30:24:400Paolo Guiotto: on symmetric… integral. So, integral… of an odd function of a symmetric interval. This integral is equal to zero.
30:39:700Paolo Guiotto: So at the end, this is a big sum of zeros, so we get zero.
30:44:230Paolo Guiotto: So, only speculation to obtain that the expected value of X, what is, XI is equal to MI.
30:56:430Paolo Guiotto: Okay? Now, you can repeat, because,
31:00:580Paolo Guiotto: It's, the same if you do the expense, if you do the covariance, the covariance of Xi
31:12:440Paolo Guiotto: Yxixj.
31:16:60Paolo Guiotto: you… just to set the calculation, you have to do expected value of the covariances, by definition, Xi minus the mean of Xi, so we say now that this is MI, XJ minus MJ, right?
31:33:700Paolo Guiotto: Because we know that this is the expectation of MI, and this is the expectation of MJ.
31:44:160Paolo Guiotto: Now, transforming this into the integral, respecting the density, this becomes the integral on RN of Xi minus MI times XJ minus MJ.
32:02:700Paolo Guiotto: of, E minus 1 half C minus 1, X minus N.
32:09:260Paolo Guiotto: dot X minus M.
32:11:860Paolo Guiotto: divided the root of 2 pi to the n determinant of matrix C, in the X.
32:22:440Paolo Guiotto: Well, we could. First of all… first of all, eliminate DMI MJ. Call vector Y equal X minus M. We could have done before, even.
32:34:480Paolo Guiotto: So, it's just a translation that makes this equal Xi minus mi becomes Yi, this is YJ, this is YY, and DX is equal DY, because it's just a translation. So, this becomes integral in Rn.
32:51:280Paolo Guiotto: of Yiyj e to minus 1 half C minus 1Y dot Y divided the root of 2 pi to the n determinant of CDY.
33:08:930Paolo Guiotto: And now we proceed as above.
33:11:890Paolo Guiotto: So, we introduced that change of variable that diagonalizes the covariance. So the change of variable is the same, so let's just copy what we did.
33:22:660Paolo Guiotto: At a certain point, we set the Y equal, so since we already eliminated the mean, the change of variable would be just Y equal TX. If you put Y equal TX here, well, Y has been already involved, so let's say Z equal TY,
33:43:420Paolo Guiotto: No, it's the contrary.
33:47:840Paolo Guiotto: No, it's correct.
33:50:910Paolo Guiotto: Okay, in any case, you get… let's see what should be the same. You get the integral on our n, because this is a linear transformation invertible of our N, so we will get T
34:02:820Paolo Guiotto: Y, D, T, Z, so, so it should be…
34:09:460Paolo Guiotto: The change of variable is,
34:12:280Paolo Guiotto: That's not particularly… so it is Y equals TZ, this is the change of variable. Y.
34:20:180Paolo Guiotto: So this becomes YI…
34:23:720Paolo Guiotto: is the ith component of vector Y. Y becomes TZ, this will be the ith component of vector Z, and this will be the j-th component of the vector TZ.
34:37:90Paolo Guiotto: So we have E minus 1 half inverse of C, Y is TZ dot TZ,
34:45:190Paolo Guiotto: divided by root of 2 pi, 2DN, determinant of C.
34:52:380Paolo Guiotto: Now, DZ to DY is the same, because to pass from DZ to DY, you have the matrix T, so the change of variable, so it should be Z equals T minus 1Y, but T minus 1 is T transposed.
35:08:670Paolo Guiotto: And DZ is a modulus of the determinant of the derivative of the transformation, which is the matrix T perpendicular, times DY. But that is equal to 1. So DZ is equal to DY.
35:23:460Paolo Guiotto: Now, you carry this side, so this becomes the transpose, and at the end, this is the matrix D minus 1, as above. So you have integral on Rn.
35:35:350Paolo Guiotto: BZ.
35:36:840Paolo Guiotto: ith component, T1, TZJ component. Here we have E minus 1 half the inverse of the diagonal matrix, so that matrix with 1 over sigma squares on the diagonal, z dot Z over root of 2 pi.
35:56:110Paolo Guiotto: to the N, and as above, this is the determinant of the matrix D. It's the same.
36:01:810Paolo Guiotto: Visa.
36:03:350Paolo Guiotto: And now you have a calculation which is similar to this one we have seen here, with the T.
36:11:410Paolo Guiotto: Where we are here to compute the integral of the jade component of TY. It's similar, no?
36:17:860Paolo Guiotto: So you proceed in the same way, so please try to complete. So, TZ, ith component will be the ith line of T scalar product with Z. So, something like sum, you have to introduce indexes, so let's say K of E.
36:37:220Paolo Guiotto: IKZK, and similarly for the jade component.
36:42:990Paolo Guiotto: So if you do the calculation, you use the factorization, you should get that this, at the end, becomes the element C, I, J, E of covariance.
37:00:630Paolo Guiotto: metrics.
37:04:160Paolo Guiotto: Capital C.
37:06:680Paolo Guiotto: Okay, try to finish this.
37:09:730Paolo Guiotto: It's,
37:15:330Paolo Guiotto: Let me see if I can review…
37:21:450Paolo Guiotto: Yes, you should try, for the next time. Do exercises 4, 3, 1… Tsu… to lead?
37:42:700Paolo Guiotto: I'm not sure on the floor.
37:45:280Paolo Guiotto: So let's limit to these three, okay? So we will start solving these three exercises. These are exercises on the joint density, on the joint CDF, things like this, and some probability to be computed.
38:00:10Paolo Guiotto: Okay.
38:04:170Paolo Guiotto: And let's stop.