In this study, we consider a bias reduction of the conditional maximum likelihood estimators for the unknown parameters of a Gaussian second-order moving average (MA(2)) model. In many cases, we use the maximum likelihood estimator because the estimator is consistent. However, when the sample size n is small, the error is large because it has a bias of O(n−1). Furthermore, the exact form of the maximum likelihood estimator for moving average models is slightly complicated even for Gaussian models. We sometimes rely on simpler maximum likelihood estimation methods. As one of the methods, we focus on the conditional maximum likelihood estimator and examine the bias of the conditional maximum likelihood estimator for a Gaussian MA(2) model. Moreover, we propose new estimators for the unknown parameters of the Gaussian MA(2) model based on the bias of the conditional maximum likelihood estimators. By performing simulations, we investigate properties of this bias, as well as the asymptotic variance of the conditional maximum likelihood estimators for the unknown parameters. Finally, we confirm the validity of the new estimators through this simulation study.
Gaussian second-order moving average modelconditional maximum likelihood estimatorsbias reductionasymptotic expansion62M15Introduction
Estimators of unknown parameters must be consistent. The consistency is ensured when we have large samples. The estimators might have a bias if the sample size is small. In recent years, some computational methods has been developed to compute estimates for unknown parameters. However, an analytical solution for the bias enables us to know a relationship between unknown parameter and the bias. It means that the bias changes depending on unknown parameters. Many analytical evaluations for the bias for a class of nonlinear estimators in models with i.i.d. samples have been conducted for many years. Tanaka (1983) [14] provided asymptotic expansions of the least square estimator for the first-order autoregressive process AR(1) and computed its bias. Tanaka (1984) [15] also gave asymptotic expansions of the maximum likelihood estimators for autoregressive moving average (ARMA) models, including AR(1), AR(2), MA(1), and MA(2), and also computed their bias. Cordeiro and Klein (1994) [8] derived the bias of the maximum likelihood estimators for ARMA models in another way although the result for MA(2) was not shown. Cheang and Reinsel (2000) [7] developed a way to reduce the bias of AR models using the restricted maximum likelihood estimation.
Practically, we often rely on the conditional maximum likelihood estimation for reducing the computational cost of the maximum likelihood estimation and for a prediction of an unobserved variable which is the next value of the observed data (see Section 2 for the definition of the conditional maximum likelihood estimation). The conditional maximum likelihood estimation is often referred as the quasi-maximum likelihood estimation (QMLE). Statistical properties of the conditional maximum likelihood estimation have been discussed in some literatures (see [3] and [4] by Bao and Ullah, for example). Giummolè and Vidoni (2010) [9] showed the bias of the conditional maximum likelihood estimator for a Gaussian MA(1) model in a process of obtaining improved coverage probabilities for ARMA models. However, the bias of the estimator for a Gaussian first-order moving average (MA(1)) model was slightly strange. Hence, Kurosawa, Noguchi, and Honda (2017) [12] corrected the bias and deduced a simple expression for the bias using a method by Barndorff-Nielsen and Cox (1994) [5]. We also should note the recent remarkable results by Y. Bao (2016) [1] and (2018) [2]. We shall discuss his results in Remark 3.4 below.
In this study, we show the bias of the conditional maximum likelihood estimators of unknown parameters for a Gaussian second-order moving average (MA(2)) model followed by the method in [12]. In Section 2, we introduce a Gaussian MA(2) model and the conditional maximum likelihood function. In Section 3, we derive both the bias and the mean squared errors (MSEs) of the conditional maximum likelihood estimator for a Gaussian MA(2) model, and then propose new estimators based on the O(n−1) term in the bias of the conditional maximum likelihood estimators. Moreover, we show that the proposed estimators are less biased and have the lower MSEs than those of the conditional maximum likelihood estimators. In Section 4, we conduct a simple simulation study to verify our results. Furthermore, we apply our method to GNP in United States of America as an illustrative example of our method in Section 6.
A Gaussian MA(2) model and the conditional maximum likelihood estimator
Let {Yt} be a Gaussian MA(1) model (see, e.g., [6, 10]) defined by
Yt=μ+εt+ρεt−1,εt∼i.i.d.N(0,σ2)(t≥1),
where |ρ|<1. Kurosawa, Noguchi, and Honda (2017) [12] computed the bias of the conditional maximum likelihood estimator under the condition that
ε0=0
for the Gaussian MA(1) model. Assumption (2) is a useful condition for not only an estimation problem but also a prediction problem, since εT (T≥1) can be written by a linear combination of Y1,…,YT. Then, the best linear unbiased estimator YˆT+h of YT+h (h>00$]]>) given S={Y1=y1,…,YT=yT} is described using a finite linear combination of ε1,…,εT (see, e.g., [12]). They gave the following:
The bias of the conditional maximum likelihood estimators of the unknown parameters given (2) for the Gaussian MA(1) model isE[μˆ−μ]=o(n−1),E[σˆ−σ]=−5σ4n+o(n−1),E[ρˆ−ρ]=2ρ−1n+o(n−1).
In this study, we consider a Gaussian MA(2) model defined by
Yt=μ+εt+ρ1εt−1+ρ2εt−2(t≥1),
where εt∼i.i.d.N(0,σ2) and θ=(μ,σ,ρ1,ρ2)⊤ is a vector consisting of the unknown parameters. Although the MA(2) model has the property of stationarity regardless of the values of ρ1 and ρ2, we assume the invertibility, which means
ρ1−ρ2<1,ρ1+ρ2>−1,−1<ρ2<1,-1,\hspace{1em}-1<{\rho _{2}}<1,\]]]>
to identify the model uniquely. Otherwise, the maximum likelihood function takes the same value at different points.
If we consider an estimation problem for the maximum likelihood function with Y1,…,Yn, then the likelihood function can be expressed using the infinite number of εs. To avoid the problem of infinite number of εs, we solve the conditional maximum likelihood function given
ε0=ε−1=0
for the Gaussian MA(2) model. In this case, Y1,…,Yn can be transformed using the finite number of εs. The conditional log-likelihood function using the finite number of εs for the Gaussian MA(2) model given (4) is expressed as
L(θ;y)=−n2log(2π)−n2log(σ2)−∑t=1n{εt(θ;y)}22σ2.
The likelihood function with (4) is referred as the conditional likelihood function (see [11, p. 653] and [17]). We use the following lemma to compute the bias of the unknown parameters in the Gaussian MA (2) model.
LetY=(Y1,…,Yn)⊤be a vector of random variables generated by a Gaussian MA(2) model in (3). Assume that (4) holds. Then, we haveεt(θ;Y)=εt=∑k=0t−1∑l=0t−k−1λ1lλ2t−k−l−1(Yk+1−μ)(t≥1),whereλ1=−ρ1+Δ2,λ2=−ρ1−Δ2,Δ=ρ12−4ρ2.
The proof will be in the Appendix. We know that
εt=1(1−λ1L)(1−λ2L)(Yt−μ)=∑k=0∞∑l=0kλ1lλ2k−l(Yt−k−μ)=∑k=−∞t−1∑l=0t−k−1λ1lλ2t−k−l−1(Yk+1−μ)
since the process is invertible. The lemma suggests that the coefficients of Yt(t≤0) are zero when ε0=ε−1=0.
Since the conditional likelihood function is expressed as a function of independent samples ε1,…,εn, we can apply the following lemma in [5] to the conditional log-likelihood function. We apply Lemma 2.3 for i.i.d. random variables ε, not Y. The high-order differentiations of the conditional log-likelihood function (5) are required to obtain the bias and the MSEs of the maximum likelihood estimators and Lemma 2.2 will be used for the calculations. An asymptotic expansion of the bias of the maximum likelihood estimator is given by the following: (See Barndorff-Nielsen and Cox (1994) [<xref ref-type="bibr" rid="j_vmsta187_ref_005">5</xref>, p. 150]).
Letθ=(θ1,…,θd)⊤be a vector of unknown parameters for a random variable Z, andθˆ=(θˆ1,…,θˆd)⊤be a vector of the maximum likelihood estimators of θ for a vector of random samplesZ=(Z1,…,Zn)⊤. Then, the bias ofθˆr(1≤r≤d)is given byEZ[θˆr−θr]=12∑s=1d∑t=1d∑u=1diθrθsiθtθu(νθsθtθu+2νθsθt,θu)+O(n−3/2)(r=1,…,d),whereL(θ;Z)is the log-likelihood function forZ=(Z1,…,Zn)⊤,lθs=∂L(θ;Z)∂θs,lθsθt=∂2L(θ;Z)∂θs∂θt,lθsθtθu=∂3L(θ;Z)∂θs∂θt∂θu,iθrθs=In(θ)−1r,s,νθsθtθu=EZ[lθsθtθu],νθsθt,θu=EZ[lθsθtlθu],andIn(θ)is the Fisher information matrix for Z.
Main results
In this section, we compute the biases of estimators for the unknown parameters of the Gaussian MA(2) model using the conditional maximum likelihood estimate, and also propose new estimators for these parameters. Before obtaining the results on bias, we observe the MSEs. The MSEs appear in the diagonal elements in the covariance matrix by
E[(θˆ−θ)(θˆ−θ)⊤]=In(θ)−1+o(n−1),
where In(θ) is the Fisher’s information matrix. It can be simplified by applying asymptotic properties
limn→∞nIn(θ)−1=J(θ).
Therefore,
E[(θˆ−θ)(θˆ−θ)⊤]=J(θ)n+o(n−1).
The elements in the asymptotic covariance matrix of the conditional maximum likelihood estimators of the unknown parameters under (4) for the Gaussian MA(2) model in (3) is given byE[(μˆ−μ)(σˆ−σ)]=E[(μˆ−μ)(ρˆ1−ρ1)]=E[(μˆ−μ)(ρˆ2−ρ2)]=E[(σˆ−σ)(ρˆ1−ρ1)]=E[(σˆ−σ)(ρˆ2−ρ2)]=o(n−1)andE[(μˆ−μ)2]=σ2(ρ1+ρ2+1)2n+o(n−1),E[(σˆ−σ)2]=σ22n+o(n−1),E[(ρˆ1−ρ1)2]=1−ρ22n+o(n−1),E[(ρˆ2−ρ2)2]=1−ρ22n+o(n−1),E[(ρˆ1−ρ1)(ρˆ2−ρ2)]=ρ1(1−ρ2)n+o(n−1).
The proof is given in Section 5.1. By applying Lemma 2.3 to (5), we obtain the bias of the conditional maximum likelihood estimators.
The bias of the conditional maximum likelihood estimators of the unknown parameters under (4) for the Gaussian MA(2) model in (3) is given byE[μˆ−μ]=o(n−1),E[σˆ−σ]=−7σ4n+o(n−1),E[ρˆ1−ρ1]=ρ1+ρ2−1n+o(n−1),E[ρˆ2−ρ2]=3ρ2−1n+o(n−1).
The proof is given in Section 5.2. We observe that the bias of the conditional maximum likelihood estimators for the Gaussian MA(2) model is the same as that for the full maximum likelihood estimators for a Gaussian MA(2) model (see Tanaka (1984) [15]). Although (11) looks different from the result by Tanaka, we can deduce the same result for the bias of σˆ2 (see (14)).
We note that MA(2) is reduced to MA(1) if we put ρ2=0 in (3). However, the bias for ρ(=ρ1) in Theorem 2.1 is not obtained even if we put ρ2=0 in (11). The results in Theorem 3.2 with ρ2=0 are obtained from a solution of the maximum likelihood estimate in four dimensions. The bias of MA(1) in Theorem 2.1 can be considered as the bias of MA(2) given ρ2=0 in Theorem 3.2. This implies that Theorem 3.2 is not result under the assumption with algebraic relationships among unknown parameters such as ρ2=0, ρ2=1, Δ=ρ12−4ρ2=0, and so on. Namely, we do not assume any algebraic relationships among the unknown parameters in advance. Thus, Δ≠0 which is equivalent to λ1≠λ2 is used in the proof of Theorem 3.2 and Propositions and Lemmas in the appendix except for Lemma 2.2.
We have recently found the notable results by Y. Bao (2016) [1] although we originally proved Theorem 3.2. The abstract in [1] claims that the bias of the conditional Gaussian likelihood estimation with nonnormal errors is derived. The gap between the “Gaussian” and the “nonnormal errors” imply that he used likelihood function (5) even if the errors follow a nonnormal distribution. For the calculation of the bias, we require the values of the skewness γ1 and the kurtosis γ2. Namely, he derived the bias regarding the likelihood function as the Gaussian likelihood function without the conditions γ1=0 and γ2=3 for a normal distribution. He gave the bias of various models including MA(2) with a matrix representation which was originally studied by Corderio and Klein (1994) [8], while we use the roots of the characteristic function for the derivation of the bias. We purely focus on the Gaussian MA(2) model with the conditional likelihood function and evaluate the corrected bias. Furthermore, we propose a new estimator below based on the corrected bias and discuss the corrected estimators under a pure Gaussian MA(2) model in detail using the (estimated) bias and the MSE in the simulation study.
Using (10), (11), and (12), we propose the following new estimators for the Gaussian MA(2) model:
σ˜=σˆ+7σˆ4n,ρ˜1=ρˆ1−ρˆ1+ρˆ2−1n,ρ˜2=ρˆ2−3ρˆ2−1n.
As we can see, the proposed estimators are asymptotically equal to the usual estimators. We consider the MSEs of the new estimators:
E[σˆ]=1−74nσ+o(n−1),E[σˆ2]=1−3nσ2+o(n−1).
Thus, we have
E[(σ˜−σ)2]=σ22n+o(n−1).
In other words, the MSEs of σ˜ and σˆ are asymptotically the same as well:
E[ρˆ12]=ρ12+1−ρ22n+2ρ1ρ1+ρ2−1n+o(n−1),E[ρˆ22]=ρ22+1−ρ22n+2ρ23ρ2−1n+o(n−1),E[ρˆ1ρˆ2]=ρ1ρ2+ρ23ρ1+ρ2−1n+o(n−1).
Therefore, we have
E[(ρ˜1−ρ1)2]=1−ρ22n+o(n−1),E[(ρ˜2−ρ2)2]=1−ρ22n+o(n−1).
In other words, the MSEs of ρ˜1 and ρˆ1 are asymptotically the same, as are the MSEs of ρ˜2 and ρˆ2.
Simulation study
In this section, we conduct a simulation study in order to verify Theorems 3.1 and 3.2 and evaluate the validity of the new estimators. Let μ=1 and σ=1. For a fixed n and a fixed ρ, we generate y=(y1,…,yn)⊤ 30,000 times from the Gaussian MA(2) model. For each y, we calculate a vector of the conditional maximum likelihood estimators θˆ=(μˆ,σˆ,ρˆ1,ρˆ2)⊤. Using the 30,000 replications, we calculate the estimated bias and MSEs using Monte Carlo simulations. In Subsetions 4.2 and 4.3, we also compute the full maximum likelihood estimators θˆMLE to compare it with our estimators.
Evaluation of asymptotic variances
We evaluate how much the MSEs of the conditional maximum likelihood estimators change depending on the true values of the unknown parameters. Table 1 shows the estimated MSEs and the values of J(θ)/n obtained in Theorem 3.1 of the conditional maximum likelihood estimators for each unknown parameter.
Comparisons of the estimated MSEs and J(θ)/n for each unknown parameter (upper: the estimated MSE, lower: J(θ)/n)
(ρ1,ρ2)=(0.25,−0.25)
(ρ1,ρ2)=(−0.40,−0.59)
n
μˆ
σˆ
ρˆ1
ρˆ2
n
μˆ
σˆ
ρˆ1
ρˆ2
50
0.02046
0.01133
0.03102
0.03684
50
0.01412
0.01146
0.04705
0.02783
0.02000
0.01000
0.01875
0.01875
0.01312
0.01000
0.01304
0.01304
100
0.01020
0.00526
0.01136
0.01233
100
0.00691
0.00578
0.01566
0.00944
0.01000
0.00500
0.00938
0.00938
0.00656
0.00500
0.00652
0.00652
150
0.00675
0.00346
0.00706
0.00739
150
0.00453
0.00386
0.00921
0.00564
0.00667
0.00333
0.00625
0.00625
0.00437
0.00333
0.00435
0.00435
(ρ1,ρ2)=(0.15,−0.55)
(ρ1,ρ2)=(0.15,0.55)
n
μˆ
σˆ
ρˆ1
ρˆ2
n
μˆ
σˆ
ρˆ1
ρˆ2
50
0.00786
0.01148
0.03108
0.03334
50
0.05849
0.01099
0.01943
0.02612
0.00720
0.01000
0.01395
0.01395
0.05780
0.01000
0.01395
0.01395
100
0.00379
0.00526
0.01051
0.01123
100
0.02907
0.00520
0.00814
0.00937
0.00360
0.00500
0.00698
0.00698
0.02890
0.00500
0.00698
0.00698
150
0.00248
0.00346
0.00597
0.00620
150
0.01932
0.00343
0.00511
0.00559
0.00240
0.00333
0.00465
0.00465
0.01927
0.00333
0.00465
0.00465
We conducted simulations under the four settings. The top-left table is prepared for checking of performance under the condition that the true parameters are within the invertibility condition. On the other hand, the top-right table is close to the boundary of the invertibility condition. The two bottom tables are made for checking the symmetry of ρ2. It is clearly observed from Table 1 for all the settings that the estimated MSEs decrease when the sample size n is large. The estimated MSE of σˆ does not depend on the values of ρ1 and ρ2, which coincides with the result that J(θ)/n of σˆ in Theorem 3.1 does not include ρ1 and ρ2 in the expression. Since J(θ)/ns of ρˆ1 and ρˆ2 depend on the value of ρ2 but are independent of the value of ρ1, we expect that the estimated MSEs of ρˆ1 and ρˆ2 on (ρ1,ρ2)=(0.15,−0.55) and (ρ1,ρ2)=(0.15,0.55) are close, but the results show different values in the small sample size n=50. This result may be the influence of o(n−1). We compare n times the estimated MSEs and J(θ) on (ρ1,ρ2)=(0.15,−0.55) and (ρ1,ρ2)=(0.15,0.55) of n=50 and n=1000 to verify the influence by o(n−1) in Table 2.
Comparisons of n times the estimated MSEs and J(θ) of n=50 and n=1000
(ρ1,ρ2)=(0.15,−0.55)
μˆ
σˆ
ρˆ1
ρˆ2
n×the estimated MSE
n=50
0.39307
0.57401
1.55412
1.66715
n=1000
0.36061
0.51210
0.72833
0.71769
J(θ)
0.36000
0.50000
0.69750
0.69750
(ρ1,ρ2)=(0.15,0.55)
μˆ
σˆ
ρˆ1
ρˆ2
n×the estimated MSE
n=50
2.92462
0.54941
0.97164
1.30611
n=1000
2.88205
0.51144
0.69979
0.71517
J(θ)
2.89000
0.50000
0.69750
0.69750
Generally, n times the estimated MSEs converge to J(θ) if n is large. The result shows that the estimated MSE of ρˆ1 is close to ρˆ2 for n=1000. Next, we present the behavior of the estimated MSEs for ρ1=0.25 and ρ2=−0.25 when the sample size is small in Table 3.
Behavior of the estimated MSEs when the sample size is small
n
μˆ
σˆ
ρˆ1
ρˆ2
10
0.14485
0.09400
0.41133
0.36819
11
0.12914
0.08262
0.34886
0.33276
12
0.11453
0.07419
0.29700
0.29798
13
0.10000
0.06659
0.26646
0.27767
14
0.08955
0.06034
0.23022
0.25188
15
0.08262
0.05502
0.21125
0.23577
16
0.07426
0.05051
0.18974
0.21314
17
0.06889
0.04616
0.17351
0.19723
18
0.06413
0.04295
0.15869
0.18569
19
0.05969
0.04028
0.14624
0.17146
20
0.05529
0.03762
0.13552
0.16067
The estimated MSE becomes smaller as the sample size becomes larger.
Estimator of <italic>σ</italic>
We express the bias of σˆ as
E[σˆ−σ]=e1+e2,e1=−7σ4n,e2=o(n−1),
which implies that
is the bias of the conditional maximum likelihood estimator,
is the bias of the conditional maximum likelihood estimator without the term O(n−1).
Evaluation of the estimated bias of σˆ
(ρ1,ρ2)=(0.25,−0.25)
(ρ1,ρ2)=(0.15,0.55)
n
e1+e2
e2
n
e1+e2
e2
50
−0.03579
−0.00079
50
−0.02814
0.00686
100
−0.01635
0.00115
100
−0.01322
0.00428
150
−0.01035
0.00132
150
−0.00825
0.00342
The bias of σˆ does not depend on the value of ρ1 and ρ2, which coincide with (10). Moreover, |e2| is smaller than |e1+e2| because of the exclusion of the term O(n−1). Next, we compare the bias and MSEs of σˆ and the proposed estimator σ˜ for ρ1=0.25 and ρ2=−0.25. We also compute the full maximum likelihood estimator σˆMLE.
Comparison of the bias and MSEs of σˆ and σ˜
Bias
MSE
n
σˆ
σˆMLE
σ˜
n
σˆ
σˆMLE
σ˜
50
−0.03579
−0.04193
−0.00204
50
0.01133
0.01195
0.01077
100
−0.01635
−0.01820
0.00087
100
0.00526
0.00531
0.00517
150
−0.01035
−0.01142
0.00120
150
0.00346
0.00347
0.00343
The estimated bias of σ˜ is lower than those of σˆ and σ˜MLE. On the other hand, there is no difference among the three estimated MSEs. This result certainly is in accordance with the discussion in Section 3.
We express the bias of θˆi(i=3,4) as
E[θˆi−θi]=e1+e2,e1=O(n−1),e2=o(n−1),
where e1=(ρ1+ρ2−1)/n for i=3, e1=(3ρ2−1)/n for i=4. Then, e1+e2 and e2 imply the bias of the conditional maximum likelihood estimator and the bias of the conditional maximum likelihood estimator without the term O(n−1), respectively.
Evaluation of the estimated bias of ρˆ1 and ρˆ2
(ρ1,ρ2)=(0.25,−0.25)
(ρ1,ρ2)=(0.40,−0.59)
ρˆ1
ρˆ2
ρˆ1
ρˆ2
n
e1+e2
e2
e1+e2
e2
n
e1+e2
e2
e1+e2
e2
50
−0.03591
−0.01591
−0.04913
−0.01413
50
−0.13333
−0.10953
−0.00976
0.04564
100
−0.01319
−0.00319
−0.01969
−0.00219
100
−0.07309
−0.06119
0.01046
0.03816
150
−0.00847
−0.00181
−0.01271
−0.00104
150
−0.05414
−0.04621
0.01191
0.03037
(ρ1,ρ2)=(0.15,−0.55)
(ρ1,ρ2)=(0.15,0.55)
ρˆ1
ρˆ2
ρˆ1
ρˆ2
n
e1+e2
e2
e1+e2
e2
n
e1+e2
e2
e1+e2
e2
50
−0.06690
−0.03890
−0.06690
−0.01390
50
−0.00862
−0.00262
0.00769
−0.00531
100
−0.02493
−0.01093
−0.02765
−0.00115
100
−0.00305
−0.00005
0.00247
−0.00403
150
−0.01475
−0.00542
−0.01663
0.00104
150
−0.00250
−0.00050
0.00095
−0.00338
Except for the case where ρ1 and ρ2 are close to the boundary condition (ρ1,ρ2)=(0.40,−0.59), |e2| is smaller than |e1+e2|. The bias of ρˆ1 depends on the value of ρ1 and ρ2. The bias of ρˆ2 depends on the value of ρ2 but not ρ1, which coincides with (12). Next, we compare the bias between ρˆ1 and ρ˜1 and between ρˆ2 and ρ˜2. We also compute the full maximum likelihood estimators ρˆiMLE.
Comparisons of estimated bias for the estimator of ρ1 and ρ2
(ρ1,ρ2)=(0.25,−0.25)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
−0.03591
−0.03166
−0.01421
−0.04913
−0.05512
−0.01118
100
−0.01319
−0.01087
−0.00286
−0.01969
−0.02115
−0.00160
150
−0.00847
−0.00716
−0.00166
−0.01271
−0.01338
−0.00079
(ρ1,ρ2)=(0.40,−0.59)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
−0.13333
−0.08952
−0.10667
−0.00976
−0.05341
0.04623
100
−0.07309
−0.03134
−0.06057
0.01046
−0.01752
0.03784
150
−0.05414
−0.01796
−0.04592
0.01191
−0.01095
0.03014
(ρ1,ρ2)=(0.15,−0.55)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
−0.06690
−0.06346
−0.03622
−0.06690
−0.09484
−0.00988
100
−0.02493
−0.01971
−0.01041
−0.02765
−0.03755
−0.00032
150
−0.01475
−0.01109
−0.00521
−0.01663
−0.02199
0.00137
(ρ1,ρ2)=(0.15,0.55)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
−0.00862
−0.00733
−0.00260
0.00769
0.02588
−0.00577
100
−0.00305
−0.00265
−0.00004
0.00247
0.00854
−0.00410
150
−0.00250
−0.00223
−0.00049
0.00095
0.00473
−0.00340
The biases of the proposed estimators ρ˜1 and ρ˜2 are less than those of ρˆ1 and ρˆ2, respectively, except for the case as mentioned the above. Moreover, the calibration of ρ˜1 and ρ˜2 depends on the values of ρˆ1 and ρˆ2. Next, we compare the MSEs between ρˆ1 and ρ˜1, and between ρˆ2 and ρ˜2.
Comparisons of the estimated MSEs for the estimators of ρ1 and ρ2
(ρ1,ρ2)=(0.25,−0.25)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
0.03102
0.03332
0.02822
0.03684
0.04005
0.03055
100
0.01136
0.01157
0.01090
0.01233
0.01264
0.01124
150
0.00706
0.00711
0.00686
0.00739
0.00748
0.00695
(ρ1,ρ2)=(0.40,−0.59)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
0.04705
0.03905
0.03866
0.02783
0.03581
0.02665
100
0.01566
0.01050
0.01364
0.00944
0.01016
0.01021
150
0.00921
0.00579
0.00826
0.00564
0.00569
0.00619
(ρ1,ρ2)=(0.15,−0.55)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
0.03108
0.03492
0.02635
0.03334
0.04263
0.02561
100
0.01051
0.01124
0.00972
0.01123
0.01288
0.00985
150
0.00597
0.00613
0.00568
0.00620
0.00668
0.00569
(ρ1,ρ2)=(0.15,0.55)
n
ρˆ1
ρˆ1MLE
ρ˜1
ρˆ2
ρˆ2MLE
ρ˜2
50
0.01943
0.01993
0.01848
0.02612
0.03126
0.02306
100
0.00814
0.00818
0.00795
0.00937
0.00993
0.00883
150
0.00511
0.00513
0.00503
0.00559
0.00577
0.00538
There is no difference between the estimated MSEs of ρˆi and ρ˜i(i=1,2), which certainly coincides with the discussion in Section 3.
Proof of theorems
We show our main Theorems 3.1 and 3.2 using lemmas and propositions in Appendix A.
Proof of Theorem <xref rid="j_vmsta187_stat_004">3.1</xref>
The second derivatives are given by the inverse of the Fisher information matrix and the components of the matrix can be obtained by the expectation of the second derivative of the log-likelihood functions. The expectations of the components are given in Proposition A.4, and then the Fisher information matrix is
In(θ)=iμμ0000iσσ0000iρ1ρ1iρ1ρ200iρ1ρ2iρ2ρ2,
where
iμμ=−E[lμμ]=1σ2∑t=1ndt−12,iσσ=−E[lσσ]=2nσ2,iρ1ρ1=−E[lρ1ρ1]=∑t=1n∑k=1t−1(φ1(k))2,iρ2ρ2=−E[lρ2ρ2]=∑t=1n∑k=1t−2(φ1(k))2,iρ1ρ2=−E[lρ1ρ2]=∑t=1n∑k=2t−1φ1(k)φ1(k−1).
The functions φ1 and dt are defined in (A.3) and (A.5), respectively. Thus, the inverse matrix is given by
In(θ)−1=iμμ0000iσσ0000iρ1ρ1iρ1ρ200iρ1ρ2iρ2ρ2,
where
iμμ=1iμμ=σ2∑t=1ndt−12,iσσ=1iσσ=σ22n,iρ1ρ1=M−1iρ2ρ2,iρ1ρ2=−M−1iρ1ρ2,iρ2ρ2=M−1iρ1ρ1,M=iρ1ρ1iρ2ρ2−(iρ1ρ2)2.
We shall compute the limiting values for iμμ, iσσ, iρ1ρ1, iρ1ρ2, and iρ2ρ2. Using the different expression (B.4) for φ1, we have
dt−1=∑t=1tφ1(k)=11+ρ1+ρ2+ρ2φ1(t)−φ1(t+1)1+ρ1+ρ2.
We consider the summation from t=1 to n of dt−12 to get iμμ. The second term in dt−1 is not a main term for the summation as n→∞. Therefore,
iμμ=σ2(1+ρ1+ρ2)2n+o(n−1).
Similarly, we have
∑k=1t−1(φ1(k))2=1Δ2λ12−λ12t1−λ12−2ρ2−ρ2t1−ρ2+λ22−λ22t1−λ22
and
∑k=2t−1φ1(k)φ1(k−1)=1Δ2λ12−λ12t−11−λ12+ρ11−ρ2t1−ρ2+λ22−λ22t−11−λ22,
and then
iρ1ρ1=iρ2ρ2=1Δ2λ121−λ12−2ρ21−ρ2+λ221−λ22n+o(n)=−1+ρ2(1−ρ12+2ρ2+ρ22)(1−ρ2)n+o(n)
and
iρ1ρ2=1Δ2λ11−λ12+ρ11−ρ2+λ21−λ22n+o(n)=−ρ1(1−ρ12+2ρ2+ρ22)(1−ρ2)n+o(n).
Thus,
M=iρ1ρ1iρ2ρ2−(iρ1ρ2)2=−1(1−ρ12+2ρ2+ρ22)(1−ρ2)2n2+o(n2).
Therefore, we obtain iρ1ρ1=iρ2ρ2=1−ρ22n+o(n−1),iρ1ρ2=ρ1(1−ρ2)n+o(n−1).
Proof of Theorem <xref rid="j_vmsta187_stat_005">3.2</xref>
Eq. (9) is trivial by Lemma 2.3 and Proposition A.6. We show (10) using Lemma A.7. The lemma can be reduced to
E[θˆr−θr]=12iθrθr∑t=1d∑u=1diθtθu(νθrθtθu+2νθrθt,θu)+O(n−3/2)
if iθrθs=0 for all s≠r. We see iσμ=iσρ1=iσρ1=0 from (15), and then we can apply (19) for the bias σˆ. The components in the sum (19) are iμμ(νσμμ+2νσμ,μ)=σ2∑t=1ndt−12−2σ3∑t=1ndt−12=−2σ,iσσ(νσσσ+2νσσ,σ)=σ22n−2nσ3=−1σ,iρ1ρ1(νσρ1ρ1+2νσρ1,ρ1)=M−1iρ2ρ2−2σiρ1ρ1=−2iρ1ρ1iρ2ρ2Mσ,iρ2ρ2(νσρ2ρ2+2νσρ2,ρ2)=M−1iρ1ρ1−2σiρ2ρ2=−2iρ1ρ1iρ2ρ2Mσ,iρ1ρ2(νσρ1ρ2+2νσρ1,ρ2)+iρ2ρ1(νσρ2ρ1+2νσρ2,ρ1)=2iρ1ρ2(νσρ1ρ2+νσρ1,ρ2+νσρ2,ρ1)=−2M−1iρ1ρ2−2σiρ1ρ2=4(iρ1ρ2)2Mσ. The summation of (22), (23), and (24) is
−4iρ1ρ1iρ2ρ2Mσ+4(iρ1ρ2)2Mσ=−4σ.
By adding (20) and (21) on the above, we obtain (10).
Next, we show (11) and (12) using Proposition A.8. The 10 components of the equation in Lemma 2.3 which are used for the calculation of the biases of ρ1 and ρ2 are given by merely substituting the results in Proposition A.8:
iμμ(νρ1μμ+2νρ1μ,μ)=−2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k)∑s=1nds−12,iσσ(νρ1σσ+2νρ1σ,σ)=0,iρ1ρ1(νρ1ρ1ρ1+2νρ1ρ1,ρ1)=M−1iρ2ρ2∑t=1n(S1,t+2T0,0,t),2iρ1ρ2(νρ1ρ1ρ2+νρ1ρ1,ρ2+νρ1ρ2,ρ1)=−2M−1iρ1ρ2∑t=1n(S2,t+T0,1,t+T1,0,t),iρ2ρ2(νρ1ρ2ρ2+2νρ1ρ2,ρ2)=M−1iρ1ρ1∑t=1n(S3,t+2T1,1,t),
and
iμμ(νρ2μμ+2νρ2μ,μ)=−2∑t=1n∑k=1t−2dt−1dt−k−2φ1(k)∑s=1nds−12,iσσ(νρ2σσ+2νρ2σ,σ)=0,iρ1ρ1(νρ2ρ1ρ1+2νρ2ρ1,ρ1)=M−1iρ2ρ2∑t=1n(S0,t−1+2T1,0,t),2iρ1ρ2(νρ1ρ2ρ2+νρ1ρ2,ρ2+νρ2ρ2,ρ1)=−2M−1iρ1ρ2∑t=1n(S1,t−1+T1,1,t+T0,0,t−1),iρ2ρ2(νρ2ρ2ρ2+2νρ2ρ2,ρ2)=M−1iρ1ρ1∑t=1n(S2,t−1+2T0,1,t),
where Sp,q and Tp,q,t are defined in (A.26) and (A.27), respectively. Now, we want to know the limiting values of the right-hand side of the above equations and to evaluate the sum of the first 5 equations and the remaining 5 equations. Let U and V be the sum of the first 5 equations and that of the remaining 5 equations, respectively. We see that
∑t=1n∑k=1t−1dt−1dt−k−1φ1(k)=1(ρ1+ρ2+1)3n+o(n),∑t=1n∑k=1t−2dt−1dt−k−2φ1(k)=1(ρ1+ρ2+1)3n+o(n)
by (16). What regards the other summations, the values are constructed by Sp,q and Tp,q,t. The components of Sp,q and Tp,q,t are given by using (B.4) and (B.5) which are a sort of geometric series. We consider the summations of Sp,q and Tp,q,t with respect to t from 1 to n as n→∞. Since the components are expressed as a sort of geometric series, the following limiting evaluations are useful.
∑t=1n∑k=1tαk=α1−αn+o(n),∑t=1n∑k=1t(k+1)αk+1=α2(2−α)(1−α)2n+o(n),∑t=1n∑k=1t∑m=1t−kαkβm=αβ(1−α)(1−β)n+o(n)
for any |α|<1 and |β|<1. Then, the summations of Sp,t and Tp,q,t with respect to t are
∑t=1nS1,t=2ρ1(ρ2+1)(ρ2−1){ρ12−(ρ2+1)2}2n+o(n),∑t=1nS2,t=2{ρ12−ρ2(ρ2+1)2}(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nS3,t=2ρ1{−ρ12+2ρ2(ρ2+1)}(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nS0,t−1=2{1+ρ2(2−ρ12+ρ2)}(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nS1,t−1=2ρ1(ρ2+1)(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nS2,t−1=2{ρ12−ρ2(ρ2+1)2}(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nT0,0,t=−2ρ1(ρ2+1)(ρ2−1){ρ12−(ρ2+1)2}2n+o(n),∑t=1nT0,1,t=−2{ρ12−ρ2(ρ2+1)2}(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nT1,0,t=ρ12+(ρ2+1)2(ρ2−1){ρ12−(ρ2+1)2}2n+o(n),∑t=1nT1,1,t=ρ1(1+ρ12−2ρ2−3ρ22)(ρ2−1)2{ρ12−(ρ2+1)2}2n+o(n),∑t=1nT0,0,t−1=−2ρ1(ρ2+1)(ρ2−1){ρ12−(ρ2+1)2}2n+o(n).
Now, we are ready to use Lemma 2.3. Using the above evaluations, we have
U=−2ρ1+ρ2+1+4ρ1(ρ2+1)2−2ρ1{ρ12+(ρ2+1)2}{ρ12−(ρ2+1)2}2+o(1),V=−2ρ1+ρ2+1+4ρ1(ρ2+1){ρ12−ρ2(ρ2+1)2}+2ρ12(1+ρ12−2ρ2−3ρ22)(1−ρ2){ρ12−(ρ2+1)2}2+o(1).
By (17) and (18), the biases of the unknown parameters ρˆ1 and ρˆ2 are
E[ρˆ1−ρ1]=12iρ1ρ1U+12iρ1ρ2V=ρ1+ρ2−1n+o(n−1)
and
E[ρˆ2−ρ2]=12iρ1ρ2U+12iρ2ρ2V=3ρ2−1n+o(n−1).
Therefore, we obtain (11) and (12).
Practical examples
We provide a practical example using quarterly U.S. GNP from 1947(1) to 2002(3), n=223 observations. The original data which are provided by the Federal Reserve Bank of St. Louis [16] are introduced after being adjusted as a good example for MA(2) in [13] when the data are transformed to compute the GNP rate from Xt by
Yt=∇log(Xt),
where Xt is the U.S. GNP. The adjusted data which are different from the original can be obtained from the web site by the author of the book.
We do not know true values for unknown parameters in the MA(2) but we assume that the GNP rate follows
Yt=0.0083+εt+0.3028εt−1+0.2036εt−2,
where the coefficients are calculated by the (full) maximum likelihood estimation using n=223−1=222 observations because we need to take the difference by (25) and confirm whether the bias using the model with the last 20 samples is reduced by our method or not.
MLE and conditional MLE with a correction for U.S. GNP using MA(2)
μ
σ
ρ1
ρ2
TRUE (n=222)
0.0083
0.0094
0.3028
0.2036
MLE (n=20)
0.0071
0.0060
0.1446
0.1370
QMLE (n=20)
0.0072
0.0060
0.1564
0.1321
corrected MLE
–
0.0065
0.1824
0.1680
corrected QMLE
–
0.0065
0.1939
0.1639
Bias for MLE
−0.0012
−0.0035
−0.1582
−0.0666
Bias for QMLE
−0.0012
−0.0035
−0.1464
−0.0714
Bias for corrected MLE
–
−0.0029
−0.1204
−0.0356
Bias for corrected QMLE
–
−0.0029
−0.1089
−0.0397
For an unknown parameter θ(θ=μ,σ,ρ1,ρ2), MLE and QMLE in Table 9 correspond to the maximum likelihood estimate θˆMLE and the conditional maximum likelihood estimate (quasi-maximum likelihood estimate) θˆ using n=20 observations. The corrected MLE and corrected QMLE correspond to the result by Tanaka (1984) [15] and θ˜ defined in (13), respectively. The reason why there are dashes (–) in the table for μ is that the correction is not required for μ. The four biases in the table from the bottom correspond to those for MLE, QMLE, corrected MLE, and corrected QMLE. We note that the corrected MLE is expected to be the best when we use n=20 observations because the estimate uses the full maximum likelihood estimation. The both corrections work well, namely the bias for true model (26) become small against the models without the corrections.
Lemmas
We show the derivatives of the conditional log-likelihood function and expectations. Let
λ1+λ2=−ρ1,λ1−λ2=Δ,λ1λ2=ρ2
and
λi2+ρ1λi+ρ2=0
for i=1,2. Furthermore, we define the following functions: φ1(k)=∑l=0k−1λ1lλ2k−1−l,φ2(k)=−2∂φ1(k)∂ρ1, and
dt=∑k=1t+1φ1(k).
These functions reduce calculation costs. We list the lemmas and propositions in the following. Their proofs are provided in Appendix B except for Lemmas A.3 and A.5 because Lemmas A.3 and A.5 just list the derivatives of the log-likelihood function and they are easily obtained by the chain rule for (A.6) and (A.7).
The first derivatives of the conditional log-likelihood function (5) for the Gaussian MA(2) model are given by lμ=1σ2∑t=1ndt−1εt,lσ=−nσ+1σ3∑t=1nεt2,lρ1=−1σ2∑t=1nεt∂εt∂ρ1,lρ2=−1σ2∑t=1nεt∂εt∂ρ2.
The first derivatives ofεtin (6) are given by∂εt∂μ=−dt−1,∂εt∂ρ1=−∑k=1t−1φ1(k)εt−k=−∑k=1t−1φ1(t−k)εk,∂εt∂ρ2=∂εt−1∂ρ1=−∑k=1t−2φ1(k)εt−k−1=−∑k=1t−2φ1(t−k−1)εk.
Lemma A.1 implies that ∂εt/∂ρ1 does not depend on εt, but is expressed as a linear combination of ε1,…,εt−1. Furthermore, ∂εt/∂ρ2 does not depend on εt−1 and εt, but depends on ε1,…,εt−2.
The second derivatives ofεtin (6) are given by∂2εt∂μ∂ρ1=∂dt−1∂ρ1=−∑k=1t−1φ1(k)dt−k−1,∂2εt∂μ∂ρ2=∂2εt−1∂μ∂ρ1=∂dt−1∂ρ2=−∑k=1t−2φ1(k)dt−k−2,∂2εt∂ρ12=∑k=1t−2φ2(k+1)εt−k−1=∑k=1t−2φ2(t−k)εk,∂2εt∂ρ1∂ρ2=∂2εt−1∂ρ12=∑k=1t−3φ2(k+1)εt−k−2=∑k=1t−3φ2(t−k−1)εk,∂2εt∂ρ22=∂2εt−2∂ρ12=∑k=1t−4φ2(k+1)εt−k−3=∑k=1t−4φ2(t−k−2)εk.
Similarly, ∂2εt/∂ρ12 is expressed as a linear combination of ε1,…,εt−2, ∂2εt/∂ρ1∂ρ2 is a linear combination of ε1,…,εt−3, and ∂2εt/∂ρ22 is that of ε1,…,εt−4.
The second derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given bylμμ=−1σ2∑t=1ndt−12,lμσ=−2σ3∑t=1ndt−1εt,lμρ1=1σ2∑t=1nεt∂dt−1∂ρ1+dt−1∂εt∂ρ1,lμρ2=1σ2∑t=1nεt∂dt−1∂ρ2+dt−1∂εt∂ρ2,lσσ=nσ2−3σ4∑t=1nεt2,lσρ1=2σ3∑t=1nεt∂εt∂ρ1,lσρ2=2σ3∑t=1nεt∂εt∂ρ2,lρ1ρ1=−1σ2∑t=1n∂εt∂ρ12+εt∂2εt∂ρ12,lρ1ρ2=−1σ2∑t=1n∂εt∂ρ1∂εt∂ρ2+εt∂2εt∂ρ1∂ρ2,lρ2ρ2=−1σ2∑t=1n∂εt∂ρ22+εt∂2εt∂ρ22.
Since ∂εt/∂ρ1 is a linear combination of εk (k=1,…,t−1), and ∂εt/∂ρ2 is a linear combination of εl (l=1,…,t−2) by Lemma A.1, their expectations equal 0. Lemma A.3 is easily obtained by the first derivatives (A.6) and (A.7) with the chain rule.
The expectations of the second derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given byE[lμμ]=−1σ2∑t=1ndt−12,E[lσσ]=−2nσ2,E[lμσ]=E[lμρ1]=E[lμρ2]=E[lσρ1]=E[lσρ2]=0,E[lρ1ρ1]=−∑t=1n∑k=1t−1(φ1(k))2,E[lρ1ρ2]=−∑t=1n∑k=2t−1φ1(k)φ1(k−1),E[lρ2ρ2]=−∑t=1n∑k=1t−2(φ1(k))2.
The third derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given bylμμμ=0,lμσσ=6σ4∑t=1ndt−1εt,lμρ1ρ1=1σ2∑t=1ndt−1∂2εt∂ρ12+εt∂2dt−1∂ρ12+2∂εt∂ρ1∂dt−1∂ρ1,lμρ2ρ2=1σ2∑t=1ndt−1∂2εt∂ρ22+εt∂2dt−1∂ρ22+2∂εt∂ρ2∂dt−1∂ρ2,lμρ1ρ2=1σ2∑t=1n∂dt−1∂ρ1∂εt∂ρ2+∂εt∂ρ1∂dt−1∂ρ2+εt∂2dt−1∂ρ1∂ρ2+dt−1∂2εt∂ρ1∂ρ2,lσμμ=2σ3∑t=1ndt−12,lσσσ=−2nσ3+12σ5∑t=1nεt2,lσρ1ρ1=2σ3∑t=1n∂εt∂ρ12+εt∂2εt∂ρ12,lσρ2ρ2=2σ3∑t=1n∂εt∂ρ22+εt∂2εt∂ρ22,lσρ1ρ2=2σ3∑t=1n∂εt∂ρ1∂εt∂ρ2+εt∂2εt∂ρ1∂ρ2,lρ1μμ=−2σ2∑t=1ndt−1∂dt−1∂ρ1=2σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k),lρ1σσ=−6σ4∑t=1nεt∂εt∂ρ1,lρ1ρ1ρ1=−1σ2∑t=1n3∂2εt∂ρ12∂εt∂ρ1+εt∂3εt∂ρ13,lρ1ρ2ρ2=−1σ2∑t=1n∂εt∂ρ1∂2εt∂ρ22+2∂εt∂ρ2∂2εt∂ρ1∂ρ2+εt∂3εt∂ρ1∂ρ22,lρ1ρ1ρ2=−1σ2∑t=1n∂εt∂ρ2∂2εt∂ρ12+2∂εt∂ρ1∂2εt∂ρ1∂ρ2+εt∂3εt∂ρ12∂ρ2,lρ2μμ=−2σ2∑t=1ndt−1∂dt−1∂ρ2=2σ2∑t=1n∑k=1t−2dt−1dt−k−2φ1(k),lρ2σσ=−6σ4∑t=1nεt∂εt∂ρ2,lρ2ρ2ρ2=−1σ2∑t=1n3∂2εt∂ρ22∂εt∂ρ2+εt∂3εt∂ρ23.
We easily obtain the third derivatives in Lemma A.5 by Lemma A.3 and the chain rule. The following three propositions are derived from the expectations of the third derivatives and the products of the first and second derivatives of the conditional log-likelihood function (5). The three propositions correspond to the unknown parameters μ, σ, ρ1 and ρ2, respectively.
The expectations of the third derivatives and the products of the first and second derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given byνμμμ=νμσσ=νμρ1ρ1=νμρ2ρ2=νμρ1ρ2=0,νμμ,μ=νμσ,σ=νμρ1,ρ1=νμρ1,ρ2=νμρ2,ρ1=νμρ2,ρ2=0.
The expectations of the third derivatives and the products of the first and second derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given byνσμμ=2σ3∑t=1ndt−12,νσσσ=10nσ3,νσρ1ρ1=2σiρ1ρ1,νσρ2ρ2=2σiρ2ρ2,νσρ1ρ2=2σiρ1ρ2,νσμ,μ=−2σ3∑t=1ndt−12,νσσ,σ=−6nσ3,νσρ1,ρ1=−2σiρ1ρ1,νσρ2,ρ2=−2σiρ2ρ2,νσρ1,ρ2=−2σiρ1ρ2,νσρ2,ρ1=−2σiρ1ρ2.
The expectations of the third derivatives and the products of the first and second derivatives of the conditional log-likelihood function (5) for the GaussianMA(2)model are given byνρ1μμ=2σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k),νρ2μμ=2σ2∑t=1n∑k=1t−2dt−1dt−k−2φ1(k),νρ1σσ=0,νρ2σσ=0,νρ1ρ1ρ1=3∑t=1nS1,t,νρ2ρ1ρ1=∑t=1n(S0,t−1+2S2,t),νρ1ρ2ρ2=∑t=1n(S3,t+2S1,t−1),νρ2ρ2ρ2=3∑t=1nS2,t−1,νρ1μ,μ=−2σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k),νρ2μ,μ=−2σ2∑t=1n∑k=1t−2dt−1dt−k−2φ1(k),νρ1σ,σ=0,νρ2σ,σ=0,νρ1ρ1,ρ1=∑t=1n(T0,0,t−S1,t),νρ2ρ1,ρ1=∑t=1n(T1,0,t−S2,t),νρ1ρ2,ρ2=∑t=1n(T1,1,t−S1,t−1),νρ2ρ2,ρ2=∑t=1n(T0,1,t−1−S2,t−1),νρ1ρ1,ρ2=∑t=1n(T0,1,t−S0,t−1),νρ2ρ2,ρ1=∑t=1n(T0,0,t−1−S3,t),whereSp,t=∑k=1t−p−1φ1(k+p)φ2(k+1),Tp,q,t=−∑k=1t−p−1∑m=1t−k−1−p−qφ1(k)φ1(k+m+p+q)φ1(m)−∑k=1t−1∑m=1t−k−q−1φ1(k)φ1(k+m−p+q)φ1(m).
Proof of lemmas and propositionsProof of Lemma <xref rid="j_vmsta187_stat_002">2.2</xref>
We show that the solution of
εt=Yt−μ−ρ1εt−1−ρ2εt−2(t≥1),ε−1=ε0=0
coincides with (6) by a mathematical induction. For t=−1 and 0, the values of the both sides of (6) are 0. Then, (6) holds when t=−1, 0.
We assume that
εs−2=∑k=0s−3∑l=0s−k−3λ1lλ2s−k−l−3(Yk+1−μ),εs−1=∑k=0s−2∑l=0s−k−2λ1lλ2s−k−l−2(Yk+1−μ)
hold for s≥1. By (A.1) and (A.2), we have
εs=(Ys−μ)−ρ1εs−1−ρ2εs−2=(Ys−μ)−ρ1∑k=0s−2∑l=0s−k−2λ1lλ2s−k−l−2(Yk+1−μ)−ρ2∑k=0s−2∑l=0s−k−3λ1lλ2s−k−l−3(Yk+1−μ)=(Ys−μ)+∑k=0s−2∑l=0s−k−3λ1lλ2s−k−l−1−ρ1λ1s−k−2(Yk+1−μ)=(Ys−μ)+∑k=0s−2∑l=0s−k−1λ1lλ2s−k−l−1(Yk+1−μ).
Therefore, (6) holds for all t≥−1. □
Proof of Lemma <xref rid="j_vmsta187_stat_008">A.1</xref>
First, we show (A.8). We see that εt can be written using φ1 as
εt=∑k=0t−1φ1(t−k)(Yk+1−μ).
Thus,
∂εt∂μ=−∑k=0t−1φ1(t−k).
Therefore, (A.8) follows by (A.5).
Next, we show (A.9). We have, by (A.4) and (B.1),
−2∂εt∂ρ1=∑k=0t−1φ2(t−k)(Yk+1−μ)=∑k=0t−1φ2(t−k)(εk+1+ρ1εk+ρ2εk−1)=∑k=1t−2(φ2(t−k+1)+ρ1φ2(t−k)+ρ2φ2(t−k−1))εk+(ρ1φ2(1)+φ2(2))εt−1+φ2(1)εt
since ε−1=ε0=0. We note that φ2(2)=−2(∂λ1∂ρ1+∂λ2∂ρ1)=2 and φ2(1)=0. Thus, the function does not contain εt in the linear combination. We know by (A.2) that
φ1(k+1)+ρ1φ1(k)+ρ2φ1(k−1)=0.
Therefore,
φ2(k+1)+ρ1φ2(k)+ρ2φ2(k−1)=2φ1(k),
and then the coefficient of εk(1≤k≤t−1) in (B.2) is 2φ1(t−k).
Finally, we prove (A.10). By (B.3), we have
∂φ1(k+1)∂ρ2+ρ1∂φ1(k)∂ρ2+ρ2∂φ1(k−1)∂ρ2=−φ1(k−1),
and then
∂εt∂ρ2=∑k=1t−2∂φ1(t−k+1)∂ρ2+ρ1∂φ1(t−k)∂ρ2+ρ2∂φ1(t−k−1)∂ρ2εk+ρ1∂φ1(1)∂ρ2+∂φ1(2)∂ρ2εt−1+∂φ1(1)∂ρ2εt=−∑k=1t−2φ1(t−k−1)εk
by the same way as in (B.2).
Proof of Lemma <xref rid="j_vmsta187_stat_009">A.2</xref>
First, we show (A.11). Eqs. (A.8) and (A.9) yield
∂dt−1∂ρ1=−∂2εt∂ρ1∂μ=∑k=1t−1φ1(k)∂εt−k∂μ=−∑k=1t−1φ1(k)dt−k−1.
By applying the same way to (A.10), we have
∂dt−1∂ρ2=−∂2εt∂ρ2∂μ=−∂∂μ∂εt−1∂ρ1=∂dt−2∂ρ1.
Then, we obtain (A.12).
Next, we show (A.13). First, we express φ2 in a different form. By (A.1), we have
∂Δ∂ρ1=ρ1ρ12−4ρ2=ρ1Δ,∂λ1∂ρ1=−Δ+ρ12Δ=−λ1Δ,∂λ2∂ρ1=−Δ+ρ12Δ=λ2Δ.
We note that we do not have any algebraic relationship among unknown parameters (μ,σ,ρ1,ρ2) such as Δ=ρ12−4ρ2=0, μ=0, etc., which makes a different problem. Using these derivatives, we have φ1(k)=λ1k−λ2kΔ,φ2(k)=2Δ2ρ1φ1(k)+k(λ1k+λ2k). This implies
2∑l=1kφ1(l)φ1(k+1−l)=φ2(k+1)
and then, we get with (A.4)
∂∂ρ1∂εt∂ρ1=−∑k=1t−1∂φ1(k)∂ρ1εt−k−φ1(k)∑l=1t−k−1φ1(l)εt−k−l=∑s=1t−2−∂φ1(s+1)∂ρ1+∑k=1sφ1(k)φ1(s+1−k)εt−s−1=∑s=1t−2φ2(s+1)εt−s−1.
This gives (A.13), and we also obtain (A.14) and (A.15) by (A.10).
Proof of Proposition <xref rid="j_vmsta187_stat_011">A.4</xref>
We show only (A.16) because the remaining equations are trivial by Lemma A.3. The reason why they trivial is that ∂dt−1∂ρ1 in lμρ1 is not a function of εs(1≤s≤t). As for lσρ1, it contains ∂εt∂ρ1, but ∂εt∂ρ1 is a linear combination of ε1,…,εt−1 by (A.9). The expectation of εtεs(1≤s≤t−1) vanishes. So, we only consider (A.16) carefully.
E[lρ1ρ2]=−1σ2E∑t=1n∂εt∂ρ1∂εt∂ρ2=−1σ2E∑t=1n∑k=1t−1∑m=1t−2φ1(k)εt−kφ1(m)εt−m−1=−1σ2E∑t=1n∑k=2t−1φ1(k)φ1(k−1)εt−k2=−∑t=1n∑k=2t−1φ1(k)φ1(k−1).
Proof of Proposition <xref rid="j_vmsta187_stat_013">A.6</xref>
The log-likelihood functions appearing in the first line in Proposition A.6 are linear functions of εk. Thus, the expectations vanish because the expectation for εk is 0. For the expectations in the second line, we see that the first or the cubic power of εk appears in the product of the first and the second derivatives of the log-likelihood functions as in
νμμ,μ=E[lμμlμ]=−1σ4E∑t=1n∑s=1ndt−12εsds−1=0,νμσ,σ=−2σ6E∑t=1n∑s=1ndt−1εtεs2=0.
We also see that the functions in the expectations of the following values are expressed as the product of three linear combinations of εk. At least, the first power or the cubic power of εk is included in the expression. Therefore, we have
νμρ1,ρ1=−1σ4E∑t=1n∑s=1nεt∂dt−1∂ρ1εs∂εs∂ρ1+∑t=1n∑s=1ndt−1∂εt∂ρ1εs∂εs∂ρ1=0,νμρ1,ρ2=νμρ2,ρ1=νμρ2,ρ2=0.
Proof of Proposition <xref rid="j_vmsta187_stat_014">A.7</xref>
Unlike Proposition A.6, the product of the first and the second derivative of the log-likelihood function is expressed as the sum of the second or the fourth power of εk. The expectations are obtained by presenting courteously the power of εk as
νσμ,μ=−2σ5E∑t=1n∑s=1ndt−1εtds−1εs=−2σ5E∑t=1ndt−12εt2=−2σ3∑t=1ndt−12,νσσ,σ=−n2σ3+n·nσ2σ5+3n·nσ2σ5−3σ7E∑t=1n∑s=1nεt2εs2=3n2σ3−3σ3n(n+2)=−6nσ3.
Similarly, the following expectations are given by the second and the forth moments. The resulting values are expressed using iρ1ρ1, iρ1ρ2, and iρ2ρ2 which are components in the Fisher information matrix.
νσρ1,ρ1=−2σ5E∑t=1n∑s=1nεt∂εt∂ρ1εs∂εs∂ρ1=−2σ5E∑t=1nεt2∂εt∂ρ12=−2σ5E∑t=1n∑k=1t−1∑m=1t−1εt2φ1(t−k)εkφ1(t−m)εm=−2σ5∑t=1n∑k=1t−1(φ1(t−k))2σ4=−2σ∑t=1n∑k=1t−1(φ1(k))2=−2σiρ1ρ1,νσρ2,ρ2=−2σ∑t=1n∑k=1t−2(φ1(k))2=−2σiρ2ρ2,νσρ1,ρ2=−2σ5E∑t=1n∑s=1nεt∂εt∂ρ1εs∂εs∂ρ2=−2σ5E∑t=1nεt2∂εt∂ρ1∂εt∂ρ2=−2σ5E∑t=1n∑k=1t−1∑m=1t−2εt2φ1(t−k)εkφ1(t−m−1)εm=−2σ5E∑t=1n∑k=1t−2εt2φ1(t−k)φ1(t−k−1)εk2=−2σ∑t=1n∑k=2t−1φ1(k)φ1(k−1)=−2σiρ1ρ2,νσρ2,ρ1=−2σ5E∑t=1n∑s=1nεt∂εt∂ρ2εs∂εs∂ρ1=−2σiρ1ρ2.
Proof of Proposition <xref rid="j_vmsta187_stat_015">A.8</xref>
Eqs. (A.17) and (A.18) are trivial by the expressions of lρ1μμ, lρ2μμ, lρ1σσ, and lρ2σσ. First, we show (A.21). The expectation of lρ1μ,μ is expressed as
νρ1μ,μ=1σ4E∑t=1n∑s=1nεt∂dt−1∂ρ1ds−1εs+1σ4E∑t=1n∑s=1ndt−1∂εt∂ρ1ds−1εs.
The first term of the right-hand side in (B.6) is given by
1σ4E∑t=1n∑s=1nεt∂dt−1∂ρ1ds−1εs=1σ2∑t=1ndt−1∂dt−1∂ρ1=−1σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k).
Similarly, the second term in (B.6) is given by
1σ4E∑t=1n∑s=1ndt−1∂εt∂ρ1ds−1εs=−1σ4E∑t=1n∑s=1n∑k=1t−1dt−1φ1(t−k)εkds−1εs=−1σ4E∑t=1n∑k=1t−1dt−1dk−1φ1(t−k)εk2=−1σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k).
Hence, we obtain
νρ1μ,μ=−2σ2∑t=1n∑k=1t−1dt−1dt−k−1φ1(k).
By the same process, we have
νρ2μ,μ=−2σ2∑t=1n∑k=1t−2dt−1dt−k−2φ1(k).
Eq. (A.22) is trivial. Next, we show (B.7), (B.9), and (B.10) to prove (A.19), (A.20), (A.23), (A.24), and (A.25). We have
E∂2εt−p+1∂ρ12∂εt∂ρ1=−E∑k=1t−p−1∑m=1t−1φ2(t−p−k+1)εkφ1(t−m)εm=−σ2∑k=1t−p−1φ1(t−k)φ2(t−p−k+1)=−σ2∑k=1t−p−1φ1(k+p)φ2(k+1)=−σ2Sp,t.
Moreover, we have
E∑s=1n∂εt−p∂ρ1∂εt∂ρ1εs∂εs−q∂ρ1=−E∑s=1n∑k=1t−p−1∑m=1t−1∑l=1s−q−1φ1(t−p−k)εkφ1(t−m)εmεsφ1(s−q−l)εl
by (A.9) for p,q≥0. We consider the right-hand side of (B.8). First, we fix k and m. For l(1≤l≤s−q−1), s must not be l. The summation appears only when s=k and l=m, or l=k and s=m. When s=k and l=m, we have
E∑k=1t−p−1∑m=1k−q−1φ1(t−p−k)φ1(t−m)φ1(k−q−m)εk2εm2=σ4∑k=1t−p−1∑m=1k−q−1φ1(t−p−k)φ1(t−k+q+m)φ1(m)=σ4∑k=1t−p−1∑m=1t−k−1−p−qφ1(k)φ1(k+m+p+q)φ1(m).
On the other hand, when l=k and s=m, we have
E∑m=1t−1∑k=1m−q−1φ1(t−p−k)φ1(t−m)φ1(m−q−k)εm2εk2=σ4∑m=1t−1∑k=1m−q−1φ1(t−m)φ1(t−m+k−p+q)φ1(k)=σ4∑k=1t−1∑m=1t−k−q−1φ1(k)φ1(k+m−p+q)φ1(m).
Therefore,
E∑t=1n∑s=1n∂εt−p∂ρ1∂εt∂ρ1εs∂εs−q∂ρ1=σ4∑t=1nTp,q,t.
Moreover, we consider the following expectation:
E∑t=1n∑s=1nεt∂2εt−p∂ρ1εs∂εs−q∂ρ1=−E∑t=1n∑s=1n∑k=1t−p−2∑m=1s−q−1εtφ2(t−p−k)εkεsφ1(s−q−m)εm.
Similarly, we fix t and k. Then, the summation appears only when s=t and m=k.
E∑t=1n∑k=1t−p−2φ2(t−p−k)φ1(t−q−k)εt2εk2=σ4∑t=1n∑k=1t−p−2φ1(k+p−q+1)φ2(k+1).
Therefore,
E∑t=1n∑s=1nεt∂2εt−p∂ρ1εs∂εs−q∂ρ1=−σ4∑t=1nSp−q+1,t−q.
We complete the proof of the lemma by (B.7), (B.9), and (B.10).
Acknowledgement
The authors would like to thank anonymous referees for giving us suitable suggestions and comments. They gave us not only technical comments, but also practical insights. By their comments, we revised a lot of equations and added Remark 3.3 and Section 6. The paper was dramatically improved by their comments. The author also thanks Dr. Tao Zou (ANU) for giving us advices about the data generation process we use and for having discussions with us. This work was supported by JSPS KAKENHI Grant Numbers JP18K11200.
ReferencesBao, Y.: Finite-sample bias of the conditional Gaussian maximum likelihood estimator in ARMA models. Adv. Econom.36, 207–244 (2016)Bao, Y.: A general result on the estimation bias of ARMA models. J. Stat. Plan. Inference197, 107–125 (2018). MR3799026. https://doi.org/10.1016/j.jspi.2018.01.001Bao, Y., Ullah, A.: The second-order bias and mean squared error of estimators in time-series models. J. Econom.140, 650–669 (2007). MR2408921. https://doi.org/10.1016/j.jeconom.2006.07.007Bao, Y., Ullah, A.: Expectation of quadratic forms in normal and nonnormal variables with applications. J. Stat. Plan. Inference140, 1193–1205 (2010). MR2581122. https://doi.org/10.1016/j.jspi.2009.11.002Barndorff-Nielsen, O.E., Cox, D.R.: Inference and Asymptotics. Chapman & Hall/CRC (1994). MR1317097. https://doi.org/10.1007/978-1-4899-3210-5Brockwell, P.J., Davis, R.A.: Time Series: Theory and Methods. Springer (1991). MR2839251Cheang, W.K., Reinsel, G.C.: Bias reduction of autoregressive estimates in time series regression model through restricted maximum likelihood. J. Am. Stat. Assoc.95, 1173–1184 (2000). MR1804241. https://doi.org/10.2307/2669758Cordeiro, G.M., Klein, R.: Bias correction in ARMA models. Stat. Probab. Lett.19, 169–176 (1994). MR1278646. https://doi.org/10.1016/0167-7152(94)90100-7Giummolé, F., Vidoni, P.: Improved prediction limits for a general class of Gaussian models. J. Time Ser. Anal.31, 483–493 (2010). MR2732602. https://doi.org/10.1111/j.1467-9892.2010.00680.xHamilton, J.D.: Time Series Analysis. Princeton University Press (1994). MR1278033Hillmer, S.C., Tiao, G.C.: Likelihood function of stationary multiple autoregressive moving average models. J. Am. Stat. Assoc.74(367), 652–660 (1979). MR0548261Kurosawa, T., Noguchi, K., Honda, F.: Bias reduction of the maximum likelihood estimator for a conditional Gaussian MA(1) model. Commun. Stat., Theory Methods31(17), 8588–8602 (2017). MR3680779. https://doi.org/10.1080/03610926.2016.1185119Shumway, R.H., Stoffer, D.S.: Time Series Analysis and Its Applications With R Examples, 4th edn. Springer (2017)Tanaka, K.: Asymptotic expansions associated with the AR(1) model with unknown mean. Econometrica51, 1221–1231 (1983). MR0710228. https://doi.org/10.2307/1912060Tanaka, K.: An asymptotic expansion associated with the maximum likelihood estimators in ARMA models. J. R. Stat. Soc. B46, 58–67 (1984). MR0745216 The Federal Reserve Bank of St. Louis. http://research.stlouisfed.org/Wilson, G.T.: The estimation of parameters in multivariate time series models. J. R. Stat. Soc.35(367), 76–85 (1973). MR0339421