Based on this correlation matrix which of the following statements is not accurate

3.02.3.5.3(i) Correlation matrix

The correlation matrix is a (K × K) square and symmetrical matrix whose ij entry is the correlation between the columns i and j of X. Large values in this matrix indicate serious collinearity between the variables involved. However, the nonexistence of extreme correlations does not imply lack of collinearity. The regressor variables for a multiple regression can be highly multicollinear even though no pairwise correlations are large.97 For instance, one of the variables may be approximated by a linear function of four other variables without any two of the variables being highly correlated. Hence, pairwise correlations are of limited use as a collinearity diagnostics. The examination of the eigenvalues and eigenvectors of the correlation matrix provides a better means for detecting multicollinearity. This is the basis of the condition number.

Filtering and modeling

Correlation matrices find wide application in the area of signal processing, for signal modeling, filtering, etc. As an example, we mention a spiking filter used when an impulsive signal is smeared by the propagation medium, as in seismic signal processing. Shaping the measured signal x(n) is achieved by the filter h(n) such that xn⊗hn=δn. The filter coefficients h are then computed using:

(30)Rxh=x*(0)[100⋅⋅⋅0]

Models of signals are widely used in areas of spectral analysis, diagnostics, etc. An example of a model for measuring signal x is the autoregressive (AR) model:

(31)xi=−∑k=1pakxi−k+wi

The vector of the model's parameters is then computed using:

(32)Ra=σ2I

with

I=[ 100…0]T

See ADAPTIVE FILTERS for application.

An example of the adaptive line enhancement (ALE) procedure is shown in Figure 8. This can be applied in situations where a composite signal which includes both broadband and narrowband signals is to be decomposed. The objective can be to extract either the narrowband or the broadband component. The narrowband signal can be an approximate harmonic signal, like those found in rotating machinery vibrations. The method is based on the fact that the narrowband signals tend to have longer autocorrelation durations than the broadband ones.

For the composite signal:

x=x1+x2

where τ1 and τ2 denote the respective autocorrelation lags beyond which the autocorrelations of x1 and x2 become negligible. Then τ2 < τ1.

The delay d shown in Figure 8 is chosen such that τ2 < d < τ1. The delayed signal component x2n−d is not correlated with x2(n), while the delayed component x1n−d is still correlated with x1(n). The adaptive filter adjusts its weights so as to cancel x1, the component which is correlated with x1n− d. The output of the filter will be an estimate x1 of x1. The subtraction results in x2, or if desired, x1 can be used for further processing. The method has been applied to composite vibration signals in order to enhance periodic components mixed with broadband random ones.

An adaptive noise cancellation (ANC) scheme is shown in Figure 9. This is a more general case of the earlier application. It can be used to cancel an interfering signal e(t) from a composite signal st+et, in those cases where another reference signal e1(t) is available.

The requirement is that e1(t) must be highly correlated with e(t). (Full correlation is obviously described in the hypothetical case shown in Figure 9.) The adaptive filter adjusts its weight so as to generate an estimate e(t) of the interfering signal. The scheme is widely used for acoustic noise cancellation, where the interfering signal is often either periodic or mostly harmonic. The correlated reference signal can often be obtained from vibration measurements, as in cases where harmonic vibrations generate that part of the acoustic signal which needs to be attenuated/cancelled. ANC has also found applications in vibration base diagnostics of rotating machinery (see DIAGNOSTICS AND CONDITION MONITORING, BASIC CONCEPTS).

Dependence and Interdependence

The correlation matrix treats all variables on the same basis in which case any analysis may be described as interdependence analysis. Sometimes, however, the variables do not have the same status. In such cases, our interest may be in how some variables depend upon others; this arises particularly when there is a temporal ordering of the variables. We may then be interested in knowing how the later ones in the sequence depend on those which come earlier. For example, this might enable us to make predictions about the values of the later variables. The conceptually simplest, and best-known, example of dependence analysis is regression analysis where we have a set of variables whose values we wish to use to predict some other variable. For example, this might be from scores obtained in a job selection exercise where we may wish to predict the performance of a candidate on the job. Such predictions are essential in judging the aptitude of potential pilots, for example, when it is too costly or dangerous to test their ability in actual flight.

Illustration

■

The correlation matrix of three data sets

x ={2, 6, 3, 1, 8} ; y ={6, 2, 0, − 3, 9};z ={1, 2, 1, 2, − 5};

lhs=MatrixForm[N[Correlation[Transpose[{x, y, z}]]]]

rhs=MatrixForm[N[{Correlationx,x,Correlationxy,Correlationx,z,Correlationx,x,Correlationyy,Correlationy,z,Correlationz,x,Correlationzy,Correlationz,z}]]

lhs == rhs

7 Correlation Matrices and Generalizations

Statisticians have long ago developed alternative standardization procedures with desirable properties by extending Eq. (2) to p vectors to handle multivariate situations. The correlation matrix {rij} is a multivariate descriptive statistics between two or more variables which is free from units of measurement. That is, it is invariant under any linear transformation.

Computation of the correlation matrix is accomplished by the command “cor” as:

It is illustrated for our selected variables as:

Note that the correlation matrix is symmetric, rij = rji, because it measures only a linear dependence between the pairs of variables. The R function “cor.test” allows formal testing of the null hypothesis that the population correlation is zero against various alternatives. I prefer the “rcorr” function of the “Hmisc” package over “cor” because “rcorr” reports three matrices: (i) Pearson or Spearman correlation matrix with pairwise deletion of missing data, (ii) the largest number of data points available for each pair, and (iii) matrix of p-values.

7.1 New Asymmetric Generalized Correlation Matrix

Zheng et al. (2012) have recently developed generalized measures of correlation (GMC) by using Nadaraya–Watson nonparametric Kernel regressions designed to overcome the linearity assumption of Pearson's standard correlations, ρX,Y. Vinod (2013) developed a weak but potentially useful “kernel-causality” defined by using computer intensive data-driven kernel-based conditional densities: f(Y |X) and f(X|Y). He defines δ = GMC(X|Y ) −GMC(Y |X). When δ < 0 we know from the properties of GMCs that X better predicts Y than vice versa.

Using better prediction as an indicator of causation (by analogy with Granger causality), I define that “X kernel causes Y” if δ < 0. The qualifier “kernel” in kernel causality should remind us that this causality is subject to fallacies, such as when the models are misspecified.

Ordinary correlations among p variables implies a symmetric p × p correlation matrix, where positive correlation means that the two variables move in the same direction. Since the GMC's are always positive, they lose important information regarding the direction of the relation. Hence, let us consider new generalized correlation coefficients ρ*(Y |X) based on signed square roots of the GMC’s. We simply assign the sign of the simple correlation to that of the generalized one by defining a sign function, sign(ρXY), equaling −1 if (ρXY < 0), and 1 if (ρXY ≥ 0).

Now the off-diagonal elements of our new correlation matrix are as follows:

(21)ρ*(Y|X)=sign (ρXY)√[GMC(Y| X)],

Unlike the usual correlation matrix, this one is not symmetric. That is, ρ*(Y |X)≠ρ*(X|Y ). When we have p variables, letting i,j = 1,2,…,p, the (i,j) location of the p × p matrix of generalized correlation coefficients (in population) contains ρ*(Xi|Xj)=ρij*, where the row variable Xi is the “effect” and the column variable Xj is the predictor or the “cause.” The p × p matrix of generalized sample correlation coefficients is denoted as: {rij*}.

Let us consider a larger set of six ratio scale variables from the cars data. The following R code selects them, computes the matrix of simple correlations, and plots them is a color scheme to show ellipses as well as numbers using the R package “corrplot” by Wei (2013).

In the following code, the object “ca” contains the correlation matrix.

names(mtcars)

attach(mtcars)

mtx=cbind(mpg, disp, hp, drat, wt, qsec)

ca=cor(mtx)

require(corrplot)

corrplot.mixed(ca, upper="number", lower="ellipse")

These correlations (subject to the strong assumption of a linear relation) range from a high of 0.89 for “disp” and “wt” to a low of 0.091 between “qsec” and “drat.” Figure 2 plots these symmetric correlations with color coding. We display rij numbers above the diagonal. Instead of reporting the symmetric numbers below the the diagonal, the command “corrplot.mixed” displays appropriate ellipses representing rij numbers below the diagonal, where negatively sloped ellipses indicate negative correlations.

Figure 2. Color-coded simple correlations for cars data.

A table of associated p-values (omitted for brevity) suggests very few p-values exceeding 0.05, implying that most coefficients are statistically significant. However, we do have nonrejection of ρqsec,drat = 0 and ρqsec,wt = 0, implying that these few relationships are statistically insignificant at the usual 5% level. Is this all we can learn from a bivariate analysis?

Since the new 6×6 matrix containing all new generalized correlation pairs will obviously have ones along the diagonal, we need to compute the GMC(Y |X) as R2 of 30 nonparametric nonlinear Kernel regressions giving proper attention to bandwidths (Hayfield and Racine, 2008). The R code for computing my new correlations is as follows:

gmcmtx=function(mym){

# mym is a data matrix with n rows and p columns

# some NAs may be present in the matrix

p=NCOL(mym)

#print(c("p=",p))

out1=matrix(1,p,p)# out1 has asymmetric correlations

for (i in 1:p){

x=mym[,i]

for (j in 1:p){

if (j>i){ y=mym[,j]

ava.x=which(!is.na(x))#ava means available

ava.y=which(!is.na(y))#ava means non-missing

ava.both=intersect(ava.x,ava.y)

newx=x[ava.both]#delete NAs from x

newy=y[ava.both]#delete NAs from y

c1=cor(newx,newy)

sig=sign(c1) #get sign of r(x,y)

#bandwidths for non parametric regressions

bw=npregbw(formula=newx~newy,tol=0.1, ftol=0.1)

mod.1=npreg(bws=bw, gradients=FALSE, residuals=TRUE)

corxy= sqrt(mod.1$R2)*sig #sign times r*(x|y)

out1[i,j]=corxy # r(i,j) has xi given xj as the cause

 bw2=npregbw(formula=newy~newx,tol=0.1, ftol=0.1)

mod.2=npreg(bws=bw2, gradients=FALSE, residuals=TRUE)

coryx= sqrt(mod.2$R2)*sig #sign times r*(y|x)

out1[j,i]=coryx

}#end i loop

}#end j loop

}#endif

return(out1)}

We need to supply this function with the data matrix of six variables and then plot them by the following code.

require(np)

cg=gmcmtx(mtx)

colnames(cg)=colnames(mtx)

rownames(cg)=colnames(mtx)

require(xtable)

print(xtable(cg, digits=3))

The interpretation of new generalized correlations is straightforward. If |rij*|>|rji*|, it is more likely that the row variable Xi is the “effect” and the column variable Xj is the “cause,” or at least Xj is the better predictor of Xi, than vice versa. For example, letting i = 1,j = 2 the entries in Table 2 show 0.951=|r12*|>|r 21*|=0.894. This suggests that “disp” better predicts “mpg” than vice versa.

Table 2. Table of Asymmetric Generalized Correlations Among Car Variables

	mpg	disp	hp	drat	wt	qsec
mpg	1.000	−0.951	−0.938	0.685	−0.916	0.738
disp	−0.894	1.000	0.931	−0.770	0.901	−0.761
hp	−0.853	0.817	1.000	−0.554	0.693	−0.927
drat	0.688	−0.946	−0.744	1.000	−0.750	0.549
wt	−0.917	0.968	0.920	−0.730	1.000	−0.772
qsec	0.751	−0.609	−0.754	0.230	−0.188	1.000

Now, the following code creates a color-coded asymmetric plot by calling the “corrplot” function as follows.

require(corrplot)

corrplot(cg, method="ellipse")

Figure 3 plots our new generalized asymmetric correlation coefficients defined in Eq. (21) with color coding similar to Fig. 2.

Figure 3. Color-coded generalized asymmetric correlations for cars data.

Finally, we claim that asymmetric correlations in Table 2 contain useful causation information in their asymmetry itself. We have noted that when the asymmetry satisfies |rij*|>|rji*|, the variable Xj is more likely to be the cause than the variable Xi. Hence, the new table and Fig. 3 represent useful supplements to the traditional table and Fig. 2. This has applications in all sciences including newer exploratory techniques using “Big Data.”

Problem 7.4 Multiple correlation coefficient for two explanatory variables

Estimate the multiple correlation coefficient R1(2.3) between a variable ξ1 and two variables ξ2 and ξ3.

Solution: For correlation matrices R and R11 we can write R =1R12R13R121R23R13R231 and R11 =1R23R231. In these expressions, the symmetry Rij = Rji of paired correlation coefficients is used. After substitution into Eq. (7.22) and some rearrangement we get

(7.23)R123=R122+R132−2R12R13R231−R232

Equation (7.23) shows that paired correlation coefficients can not reach any value in the range –1 < Rij < 1, but they are mutually bounded by the condition R1(2.3) ≤ 1.

If R23 = 0, the explanatory variables are mutually uncorrelated, and

(7.24a)R12,32=R122=R132

Conclusion: The multiple correlation coefficient may be estimated as the function of paired correlation coefficients. When the explanatory variables ξ2,…, ξm are mutually uncorrelated, the square of the multiple correlation coefficient is equal to the sum of squares of paired correlation coefficients.

In some cases the centred random variables or normalized random variables are used. For centred random variables ξc,j = ξj – μj, j = 1,…,m and for normalized random variablesξN.j=ξj−μjσj,j=1,…,m. The regression defined by Eq. (7.17) may be expressed with the use of centred random variables in the form

(7.24b)Eξ1/ xc*=c1TC*−1x c*=∑i=1m−1ai xc.i+1

It can be seen that centring does not change the estimates of the regression coefficients, but the intercept term is equal to zero. With normalized random variables, the regression E(ξ1/xc*) takes the form

(7.25)EξN1/xN*=RTR*−1xN*= aTDxc*σ1=bTxN*

where R is a vector of size ((m – 1) × 1) containing paired correlation coefficients ρ(ξ1,ξj), j = 2, …, m, and R* is the correlation matrix of the vector of explanatory variables of size (m – 1) × (m – 1), D denotes a diagonal transformation matrix with elements σj, j = 2,… m, on the main diagonal. The coefficients bj = R*−1 R are called the normalized regression coefficients. From Eq. (7.25) it follows that a relationship exists between non-normalized (aj) and normalized (bj) regression coefficients

(7.26)aj−1=bj−1σ1σj,j=2,…,m

The normalization changes the magnitude of the regression coefficients. The advantage of normalized regression coefficients is the fact that they concern directly the paired correlation coefficients and are easier to interpret.

B.2 2-DPI Process (6)

The spin-correlation matrix (B.6), which is a part of the state (33) needed for studying Coulombic entanglement in a (ep, ea) pair generated in the 2-DPI process (6) taking place in an atom T, has been derived at several places (Chandra & Ghosh, 2004b, 2006a, 2013). The following expression is taken from Chandra and Ghosh (2013, eq. (6.5))

(B.6)ρ2-DPIS0;S1+*;S2+;u ˆp;uˆaμp,μa;μp′,μa′=−1 S0+2S1+*+S2++μp′+ μa′2S1+*+1∑ sμμ1μ2−1s +μ2s+11/21/2sμp−μp′μ11/21/2sμa−μa′μ21/21/2sS1+*S1+*S0 1/21/2sS1+*S1+*S2+Dμ1μsωp*Dμ2,−μsωa*.

For neither of the two 6-j symbols present in this expression to vanish identically, they must satisfy the following two respective triangular conditions

(B.7a)ΔS0S1+*12⟹S1+*−1 2≤S0≤S1+*+12,ie,S0=S1+*±12

and

(B.7b)ΔS2+S1+*12⟹S2+−12≤S1 +*≤S2++12,ie,S2+=S1+*±12

Discussion given on pages 53–62 in Section 3.1.1.1.2 is based upon the density matrix (B.6).

EXAMPLE 1

Suppose that the correlation matrix among four variables, x1, x2, x3, and x4, is given by

R=[10aa01a−aaa10a−a01],

where 2a2 ≤ 1. The SMC of x1 can be computed as

1−det(R)/det[ 1a−aa10−a0 1]=1−(1−2a2)21−2a2=2a2,

provided that 2a2 ≠ 1. Similarly, it can be shown that the SMC's of all the four variables are equal to 2a2.

The following factor loading matrix Λ and the unique variance matrix Ψ,

Λ=[1001aaa−a],andΨ=[00000000001−2a200001−2a2],

on the other hand, satisfy the factor analysis model, (26). The communalities of the four variables can be computed as (1, 1, 2a2, 2a2). Thus, the SMC's are equal to the communalities for variables 3 and 4, while the SMC's are smaller than (or equal to) the communalities for variables 1 and 2. Since the communalities of variables 1 and 2 are unity, factors f1 and f2 can be rotated to coincide with them. Since the SMC's are equal to the squared length of the projection of x3 and x4 onto the factor space which are now spanned by x1 and x2, it can be easily seen that the SMC's of variables x3 and x4 coincide with their communalities. In terms of the factor analysis model, we can write

(37)ψ1=ψ2=0,andψ3=ψ4=1−2a2.

This result covers Case 2 in (31). It also covers Theorem 3 of Roff (1936, p. 5), which states that SMC(j) is equal to the communality of variable xj, if variables contain m (m < p) statistically independent variables each with unit communality (where p is the number of variables and m is the number of common factors).

In (31) it is important to consider the case in which ψi = 0 (i ≠ j) does not hold. In such a case, rji = 0 (i ≠ j) should be true. Since rji is the (j, i)th element of R−, it follows from (8) of Property 3, that

(38)(Q X(j)xj)′(QX(i)xi)=0(i≠j),

provided that Rj/(j) ≠ 1, which implies that the diagonal matrix D as defined by (13) is nonsingular. (38) implies that the anti-image of variable xj is uncorrelated with that of variable xi. It is interesting to note that (38) is closely related to Theorem 4 of Guttman (1953), which states that if a common-factor space of dimensionality m is determinate for an infinitely large universe of content, then there is no other determinate common factor space. In this case, the communalities are uniquely determined and are equal to the corresponding total norms, and in addition the common-factor scores are the total image scores, and the unique factor scores are the total ant-images. If (38) holds for any combination of i and j, then anti-image variable Qx(j)xj behaves like the unique factor, ej, corresponding to the jth variable, xj.

2.6.1 Symmetric Matrices

Figure 2.1 shows, in schematic form, various special matrices of interest to multivariate analysis. The first property for categorizing types of matrices concerns whether they are square (m = n) or rectangular. In turn, rectangular matrices can be either vertical (m > n) or horizontal (m < n).

As we shall show in later chapters, square matrices play an important role in multivariate analysis. In particular, the notion of matrix symmetry is important. Earlier, a symmetric matrix was defined as a square matrix that satisfies the relation

A=A′or,equivalently,(aij)=(aji)

That is, a symmetric matrix is a square matrix that is equal to its transpose. For example,

A= [32420−54−51];A ′=[32420−54−51]

Symmetric matrices, such as correlation matrices and covariance matrices, are quite common in multivariate analysis, and we shall come across them repeatedly in later chapters.7

A few properties related to symmetry in matrices are of interest to point out:

The product of any (not necessarily symmetric) matrix and its transpose is symmetric; that is, both AA′ and A′A are symmetric matrices.

If A is any square (not necessarily symmetric) matrix, then A + A′ is symmetric.

If A is symmetric and k is a scalar, then kA is a symmetric matrix.

The sum of any number of symmetric matrices is also symmetric.

The product of two symmetric matrices is not necessarily symmetric.

Later chapters will discuss still other characteristics of symmetric matrices and the special role that they play in such topics as matrix eigenstructures and quadratic forms.

Parameter estimates and distributional results

The ordinary least-squares parameter estimates βˆ are given by:

As described above, we estimate the error correlation matrixV using the ReML method. The error covariance matrix is then given by σˆ2V (Eqn. 8.39). The covariance of the parameter estimates is:

A t-statistic can then be formed by dividing a contrast of the estimated parameters cT βˆ by its estimated standard deviation:

where σ2 is estimated using Eqn. 8.39.

The key difference, in relation to the spherical case, i.e. when the error is IID, is that the correlation matrix V enters the denominator of the t-value. This gives us a more accurate t-statistic. However, because of V, the denominator of Eqn. 8.42 is not the square root of a χ2-distribution. (The denominator would be exactly χ2 distributed, when V describes a spherical distribution.) This means that Eqn. 8.42 is not t-distributed and we cannot simply make inferences by comparing with a null distribution with trace(RV) degrees of freedom.

Instead, one approximates the denominator with a χ2-distribution (Eqn. 8.42). T is then approximated by a t-distribution. The approximation proposed (Worsley and Friston, 1995) is the Satterthwaite approximation (see also Yandell, 1997), which is based on fitting the first two moments of the denominator distribution with a χ2 distribution. The degrees of freedom of the approximating χ2-distribution are called the effective degrees of freedom and are given by:

8.43v=2E(σˆ2)2Var(σ ˆ2)=trace(RV)2tr ace(RVRV)

See Appendix 8.2 for a derivation of this Satterthwaite approximation.

Similarly, the null distribution of an F-statistic in the presence of serial correlations can be approximated. In this case, both the numerator and denominator of the F-value are approximated by a χ2-distribution.