Minimax interpolation of sequences with stationary increments and cointegrated sequences

We consider the problem of optimal estimation of the linear functional $A_N{\xi}=\sum_{k=0}^Na(k)\xi(k)$ depending on the unknown values of a stochastic sequence $\xi(m)$ with stationary increments from observations of the sequence $\xi(m)+\eta (m)$ at points of the set $\mathbb{Z}\setminus\{0,1,2,\ldots,N\}$, where $\eta(m)$ is a stationary sequence uncorrelated with $\xi(m)$. We propose formulas for calculating the mean square error and the spectral characteristic of the optimal linear estimate of the functional in the case of spectral certainty, where spectral densities of the sequences are exactly known. We also consider the problem for a class of cointegrated sequences. We propose relations that determine the least favorable spectral densities and the minimax spectral characteristics in the case of spectral uncertainty, where spectral densities are not exactly known while a set of admissible spectral densities is specified.


Introduction
In this paper, we investigate the problem of estimating the missed observations of stochastic sequences with stationary increments. Kolmogorov [13], Wiener [27], and Yaglom [29,30] developed effective methods of estimation of the unknown values of stationary sequences and processes. Later on Yaglom [28] and Pinsker [21] introduced and investigated stochastic processes with stationary increments of order n. Properties of these and other processes generalizing the concept of stationarity are described in the books by Yaglom [29,30]. The stationary and related stochastic sequences are widely used in econometrics and in financial time series analysis. Examples of these sequences are autoregressive sequences (AR), moving-average sequences (MA), and autoregressive moving-average sequences (ARMA). Time series with trends are described by integrated ARMA sequences (ARIMA) and seasonal time series, which are examples of stochastic sequences with stationary increments. These models are properly described in the book by Box, Jenkins, and Reinsel [2]. Granger [8] introduced a concept of cointegrated sequences, namely, the integrated sequences such that some linear combination of them has a lower order of integration. Cointegrated sequences are described in more details in the paper by Engle and Granger [5]. We also refer to the papers [3,4,9,12] for recent developments.
Traditional methods of finding solutions to extrapolation, interpolation, and filtering problems for stationary and related stochastic processes are developed under the basic assumption that the spectral densities of the considered stochastic processes are exactly known. However, in most practical situations, complete information on the spectral densities of the processes is not available. Investigators can apply the traditional methods considering the estimated spectral densities instead of the true ones. However, as it was shown by Vastola and Poor [26] with the help of some examples, this approach can result in significant increasing of the value of the error of estimate. Therefore, it is reasonable to derive estimates that are optimal for all densities from a certain class of spectral densities. These estimates are called minimax-robust since they minimize the maximum of the mean-square errors for all spectral densities from a set of admissible spectral densities simultaneously. This approach to study the problem of extrapolation of stationary stochastic processes was introduced by Grenander [10]. Franke [6] investigated the minimax extrapolation and interpolation problems for stationary sequences applying the convex optimization methods. In the book by Moklyachuk [20], the minimax-robust estimates of the linear functionals of stationary sequences and processes are presented. See also the survey paper [18], The classical and minimax-robust problems of interpolation, extrapolation, and filtering of the functional of stochastic sequences with stationary increments are investigated in the papers by Luz and Moklyachuk [14][15][16][17]19]. Particularly, the cointegrated sequences are investigated in the papers [14,15]. The classical extrapolation problem in the case where both the signal and the noise processes are not stationary was investigated by Bell [1].
In the present paper, we consider the problem of estimation of the linear functional which depends on the unknown values of the sequence ξ(k) with stationary nth increments based on observations of the sequence ξ(k) + η(k) at points m ∈ Z \ {0, 1, 2, . . . , N }. The sequence η(k) is assumed to be stationary and uncorrelated with ξ(k).

Stationary increment stochastic sequences. Spectral representation
In this section, we present the main results of the spectral theory of stochastic sequences with nth stationary increments. For more details, we refer to the books by Yaglom [29,30].
Definition 1. For a given stochastic sequence {ξ(m), m ∈ Z}, the sequence where B µ is the backward shift operator with step µ ∈ Z such that B µ ξ(m) = ξ(m − µ), is called a stochastic nth increment sequence with step µ ∈ Z.
Definition 2. The stochastic nth increment sequence ξ (n) (m, µ) generated by a stochastic sequence {ξ(m), m ∈ Z} is wide sense stationary if the mathematical expectations exist for all m 0 , µ, m, µ 1 , µ 2 and do not depend on m 0 . The function c (n) (µ) is called the mean value of the nth increment sequence, and the function D (n) (m, µ 1 , µ 2 ) is called the structural function of the stationary nth increment sequence (or the structural function of nth order of the stochastic sequence {ξ(m), m ∈ Z}). Theorem 1. The mean value c (n) (µ) and the structural function D (n) (m, µ 1 , µ 2 ) of the stochastic stationary nth increment sequence ξ (n) (m, µ) can be represented in the following forms: where c is a constant, F (λ) is a left-continuous nondecreasing bounded function with F (−π) = 0. The constant c and the function F (λ) are determined uniquely by the increment sequence ξ (n) (m, µ).
Representation (3) and the Karhunen theorem [7] give us a spectral representation of the stationary nth increment sequence ξ (n) (m, µ): where Z ξ (n) (λ) is a random process with uncorrelated increments on [−π, π) with respect to the spectral function F (λ): We will use the spectral representation (4) for deriving the optimal linear estimates of unknown values of stochastic sequences with stationary increments.
Interpolation problem for the sequences ξ(m) and η(m) is considered as the problem of the mean-square optimal estimation of the linear functional which depends on the unknown values of the stochastic sequence ξ(m) at points m = 0, 1, . . . , N based on observations of the sequence ζ(m) = ξ(m) + η(m) at points of the set Z \ {0, 1, 2 . . . , N }.
Suppose that the spectral densities f (λ) and g(λ) satisfy the minimality condition Under this condition, the mean-square error of the estimate of the functional A N ξ is not equal to zero [24]. The functional A N ξ admits the representation where The coefficients v µ,N (k), k = −µn, −µn + 1, . . . , −1, and b µ,N (k), k = 0, 1, 2, . . . , N , are calculated by the formulas (see [15]) where by [x] ′ we denote the least integer number among the numbers that are greater than or equal to x, the coefficients {d µ (k) : k ≥ 0} are determined by the relationship and a N = (a(0), a(1), a(2), . . . , a(N )) ′ is a vector of dimension (N + 1).
The functional H N ξ from representation (6) has finite variance, and the functional V N ζ depends on the known observations of the stochastic sequence ζ(k) at the points k = −µn, −µn + 1, . . . , −1. Therefore, optimal estimates A N ξ and H N ξ of the functionals A N ξ and H N ξ and the mean-square errors ∆(f, g; A N ξ) = E|A N ξ − A N ξ| 2 and ∆(f, g; H N ξ) = E|H N ξ − H N ξ| 2 of the estimates A N ξ and H N ξ satisfy the following relations: Thus, the interpolation problem for the functional A N ξ is equivalent to the interpolation problem for the functional H N ξ. This problem can be solved by applying the Hilbert space projection method proposed by Kolmogorov [13]. The optimal linear estimate A N ξ of the functional A N ξ can be represented in the form where h µ (λ) is the spectral characteristic of the optimal estimate H N ξ.
µ ) be the closed linear subspace generated by elements {ξ (n) (k, µ) + η (n) (k, µ) : k ≤ −1} of the Hilbert space H = L 2 (Ω, F , P) of random variables γ with zero mean value and finite variance, Eγ = 0, E|γ| 2 < ∞, with the inner product (γ 1 ; Let us also define the subspaces L 0− 2 (p) and L N + 2 (p) of the Hilbert space L 2 (p) with the inner product (x 1 ; x 2 ) = π −π x 1 (λ)x 2 (λ)p(λ)dλ that are generated by the functions {e iλk (1 − e −iλµ ) n (iλ) −n : k ≤ −1} and {e iλk (1 − e −iλµ ) n (iλ) −n : k ≥ N + 1}, respectively, where the function is the spectral density of the sequence ζ(m), m ∈ Z [15]. The optimal estimate H N ξ of the functional H N ξ is the projection of the element H N ξ of the Hilbert space H = L 2 (Ω, F , P) onto the subspace The following conditions characterize the estimate H N ξ: The functional H N ξ in the space H admits the spectral representation Making use of the described representation and condition 2), we derive the following equation for determining the spectral characteristic h µ (λ): Thus, the spectral characteristic h µ (λ) can be represented as follows: where c µ (k), k = 0, 1, 2, . . . , N + µn, are unknown coefficients we have to determine. Condition 1) implies that the spectral characteristic h µ (λ) satisfies the following equations: The derived equations are represented as a system of N + µn + 1 linear equations: where the coefficients {a µ,N (m) : 0 ≤ m ≤ N + µn} are calculated by the formula (14) and the Fourier coefficients {T µ k,j , P µ k,j : 0 ≤ k, j ≤ N + µn} are calculated by the formulas Denote by [D µ N a N ] +µn the vector of dimension (N + µn + 1) constructed by adding µn zeros to the vector D µ N a N of dimension (N + 1). Using these definitions, system (12)- (13) can be represented in the matrix form are vectors of dimension (N + µn + 1); and P µ N and T µ N are matrices of dimension (N + µn + 1) × (N + µn + 1) with elements (P µ N ) l,k = P µ l,k and (T µ N ) l,k = T µ l,k , 0 ≤ l, k ≤ N + µn. Thus, the coefficients c µ (k), 0 ≤ k ≤ N + µn, are determined by the formula The existence of the invertible matrix (P µ N ) −1 was shown in [25] under condition (5). The spectral char- The value of the mean-square errors of the estimates A N ξ and H N ξ can be calculated by the formula where Q N is the matrix of dimension (N + 1) × (N + 1) with the coefficients We can summarize the derived results in the form of the following theorem.

Theorem 2.
Let {ξ(m), m ∈ Z} be a stochastic sequence with stationary nth increments ξ (n) (m, µ), and let {η(m), m ∈ Z} be a stationary stochastic sequence uncorrelated with ξ(m). Let the spectral densities f (λ) and g(λ) of the sequences satisfy the minimality condition (5). The optimal linear estimate A N ξ of the functional A N ξ, which depends on the values ξ(m), 0 ≤ m ≤ N , based on the observations of the sequence ξ(m) + η(m) at points of the set Z \ {0, 1, 2, . . . , N } is calculated by formula (10). The spectral characteristic h µ (λ) and the value of the mean-square error ∆(f, g; A N ξ) of the optimal estimate A N ξ are calculated by formulas (11) and (15), respectively.

Corollary 1.
Let the spectral density f (λ) of the sequence ξ(m) satisfy the minimality condition The optimal linear estimate A N ξ of the functional A N ξ of unknown values ξ(m), 0 ≤ m ≤ N , based on observations of the sequence ξ(m) at the points m ∈ Z \ {0, 1, 2, . . . , N } can be calculated by the formula The spectral characteristic h ξ µ (λ) and the mean-square error ∆(f ; A N ξ) of the optimal estimate A N ξ can be calculated by the formulas where F µ N is the matrix of dimension (N + µn + 1) × (N + µn + 1) with elements In the case of estimation of an unobserved value ξ(p), 0 ≤ p ≤ N , the following statement holds true.

The value of the mean-square error of the estimate is calculated by the formula
Let us find the estimate A 1 ξ of the value of the functional A 1 ξ = 2ξ(0) + ξ(1) based on observations of the sequence ξ(m) at the points m ∈ Z\{0, 1}. Let φ = 1/2. In this case, v 1,1 (−1) = −2, The value of the mean-square error of the estimate is ∆(f, g; A 1 ξ) = 88 17 .

Interpolation of cointegrated sequences
Consider two integrated sequences {ξ(m), m ∈ Z} and {ζ(m), m ∈ Z} with absolutely continuous spectral functions F (λ) and P (λ) and the corresponding spectral densities f (λ) and p(λ). The interpolation problem for cointegrated sequences consists in mean-square optimal linear estimation of the functional of unknown values of the stochastic sequence ξ(m) based on observations of the stochastic sequence ζ(m) at the points m ∈ Z \ {0, 1, 2, . . . , N }. To solve the problem, we can use the results obtained in the previous sections.
Suppose that the spectral density p(λ) of the sequence ζ(m) satisfies the minimality condition Let the matrices P µ,β N , T µ,β N , Q β N be defined by the Fourier coefficients of the functions in the same way as the matrices P µ N , T µ N , Q N were defined. Theorem 2 implies the following formula for calculating the spectral characteristic h β µ,N (λ) of the optimal estimate of the functional A N ξ: The value of the mean-square error of the estimate A N ξ is calculated by the formula The described results are presented as the following theorem.  (22) and (23), respectively.

Minimax-robust method of interpolation
Formulas for calculating values of the mean-square error ∆(h(f, g); f, g) = ∆(f, g; A N ξ) = E|A N ξ − A N ξ| 2 and the spectral characteristics of the optimal estimates of the functional A N ξ based on observations of the sequence ξ(m) + η(m) can be applied under the condition that the spectral densities f (λ) and g(λ) of the stochastic sequences ξ(m) and η(m) are known. However, these formulas often cannot be used in many practical situations since the exact values of the densities are not available. In this situation, the minimax-robust method can be applied. It consists in finding the estimate that provides a minimum of the mean-square errors for all spectral densities from a given set D = D f × D g of admissible spectral densities simultaneously.

Definition 4.
For a given class of spectral densities D = D f × D g , spectral densities f 0 (λ) ∈ D f and g 0 (λ) ∈ D g are called the least favorable densities in the class D for the optimal linear interpolation of the functional A N ξ if the following relation holds: Definition 5. For a given class of spectral densities D = D f × D g , the spectral characteristic h 0 (λ) of the optimal linear estimate of the functional A N ξ is called minimax-robust if the following conditions are satisfied:

Lemma 1.
The spectral densities f 0 ∈ D f and g 0 ∈ D g that satisfy the minimality condition (5) are the least favorable in the class D for the optimal linear interpolation of the functional A N ξ based on observations of the sequence ξ(m) + η(m) at the points m ∈ Z \ {0, 1, 2, . . . , N } if the matrices (P µ N ) 0 , (T µ N ) 0 , (Q N ) 0 whose elements are defined by the Fourier coefficients of the functions where p 0 (λ) = f 0 (λ) + λ 2n g 0 (λ), determine a solution to the constrained optimization problem The minimax-robust spectral characteristic The presented statements follow from the introduced definitions and Theorem 2.
The minimax-robust spectral characteristic h 0 and the least favorable spectral densities (f 0 , g 0 ) form a saddle point of the function ∆(h; f, g) on the set H D × D. The saddle-point inequalities and (f 0 , g 0 ) is a solution to the constrained optimization problem This constrained optimization problem is equivalent to the unconstrained optimization problem where ∆(f, g|D f × D g ) is the indicator function of the set D f × D g : A solution (f 0 , g 0 ) to the unconstrained optimization problem is determined by the condition 0 ∈ ∂∆ D (f 0 , g 0 ), which is a necessary and sufficient condition that the pair (f 0 , g 0 ) belongs to the set of minimums of the convex functional ∆ D (f, g) [11,22,23]. By ∂∆ D (f, g) we denote the subdifferential of the functional ∆ D (f, g) at the point (f, g) = (f 0 , g 0 ), that is, the set of all linear continuous functionals Λ on the space L 1 × L 1 that satisfy the inequality In the case of estimating the cointegrated sequences, we have the following optimization problem of finding the least favorable spectral densities: A solution (f 0 , p 0 ) to this optimization problem is characterized by the condition 0 ∈ ∂∆ D (f 0 , p 0 ). The derived representations of the linear functionals ∆(h µ (f 0 , g 0 ); f, g) and ∆(h β µ (f 0 , p 0 ); f, p) allow us to calculate derivatives and subdifferentials in the space L 1 × L 1 . Therefore, the complexity of the optimization problems (26) and (27) If the spectral densities f 0 ∈ D − 0,f , g 0 ∈ D − 0,g and the functions where p 0 (λ) = f 0 (λ) + λ 2n g 0 (λ), are bounded, then the linear functional ∆(h µ (f 0 , g 0 ); f, g) is continuous and bounded in the space L 1 × L 1 . The condition 0 ∈ ∂∆ D (f 0 , g 0 ) implies that the spectral densities f 0 ∈ D − 0,f and g 0 ∈ D − 0,g are determined by the relations where the constants The derived statements allow us to formulate the following theorems.

The least favorable spectral densities in the class
Consider the problem of minimax-robust interpolation of the functional A N ξ based on observations of the sequence ξ(m)+η(m) at the points of m ∈ Z\{0, 1, 2, . . . , N } in the case where the spectral densities f (λ) and g(λ) belong to the set D = D 2ε1 × D 1ε2 , where are ε-neighborhoods of the given spectral densities f 1 (λ) and g 1 (λ) in the spaces L 2 and L 1 , respectively. Suppose that the spectral densities f 1 (λ) and g 1 (λ) are bounded and the functions h µ,f (f 0 , g 0 ) and h µ,g (f 0 , g 0 ) calculated by formulas (28) and (29) with spectral densities f 0 ∈ D 2ε1 and g 0 ∈ D 1ε2 are bounded as well. The condition 0 ∈ ∂∆ D (f 0 , g 0 ) implies the following relations for determining the least favorable spectral densities: where the function |γ(λ)| ≤ 1 and γ(λ) = sign(g(λ) − g 1 (λ)) if g(λ) = g 1 (λ); α 1 , α 2 are two constants to be found using the equations Now we can present the following theorems, which describe the least favorable spectral densities in the class D = D 2ε1 × D 1ε2 . Theorem 8. Suppose that the spectral densities f 0 (λ) ∈ D 2ε1 and g 0 (λ) ∈ D 1ε2 satisfy the minimality condition (5), the functions h µ,f (f 0 , g 0 ) and h µ,g (f 0 , g 0 ), calculated by formulas (28) and (29), are bounded. The spectral densities f 0 (λ) and g 0 (λ) determined by equations (34)-(36) are the least favorable spectral densities in the class D = D 2ε1 × D 1ε2 for the linear interpolation of the functional A N ξ if they give a solution to constrained optimization problem (25). The function h µ (f 0 , g 0 ), calculated by formula (11) is the minimax-robust spectral characteristic of the optimal estimate of the functional A N ξ.
Theorem 9. Suppose that the spectral density f (λ) is known, the spectral density g 0 (λ) ∈ D 1ε2 , and they satisfy the minimality condition (5). Suppose also that the function h µ,g (f, g 0 ) calculated by formula (29) is bounded. Then the spectral density is the least favorable in the class D 1ε2 for the linear interpolation of the functional A N ξ if a pair (f, g 0 ) provides a solution to constrained optimization problem (25).
The function h µ (f, g 0 ), calculated by formula (11) is the minimax-robust spectral characteristic of the optimal estimate of the functional A N ξ.
Theorem 10. Suppose that the spectral density g(λ) is known, the spectral density f 0 (λ) ∈ D 2ε1 , and they satisfy the minimality condition (5). Suppose also that the function h µ,f (f 0 , g), calculated by formula (28), is bounded. The spectral density f 0 (λ) determined by the equation 2 and the condition π −π |f 0 (λ) − f 1 (λ)| 2 dλ = 2πε 1 is the least favorable spectral density in the class D 2ε1 for the linear interpolation of the functional A N ξ if a pair (f 0 , g) provides a solution to constrained optimization problem (25). The function h µ (f 0 , g) calculated by formula (11) is the minimax-robust spectral characteristic of the optimal estimate of the functional A N ξ.
The function h µ (f 0 , p 0 ) calculated by formula (22) is the minimax-robust spectral characteristic of the optimal estimate of the functional A N ξ.

Conclusions
In the article, the problem of the mean-square optimal linear estimation of the functional A N ξ = N k=0 a(k)ξ(k), which depends of unknown values of the sequence ξ(m) with nth stationary increments based on observations of the sequence ξ(m) + η(m) at the points m ∈ Z \ {0, 1, 2, . . . , N }, is considered in the case of observations with the stationary noise η(m) uncorrelated with ξ(m). The classical and minimaxrobust methods of interpolation are applied in the case of spectral certainty and in the case spectral uncertainty. Particularly, in the case of spectral certainty, formulas for calculating the spectral characteristics and the value of the mean-square error of the optimal estimate are found. The derived results are applied to interpolation problem for a class of cointegrated sequences. In the case spectral uncertainty, where spectral densities are not known exactly, whereas some sets of admissible spectral densities are given, formulas that determine the least favorable spectral densities and the minimax-robust spectral characteristics are derived for some special sets of admissible spectral densities.