A law of the iterated logarithm for small counts in Karlin’s occupancy scheme

In the Karlin inﬁnite occupancy scheme, balls are thrown independently into an inﬁnite array of boxes 1, 2 , . . . , with probability p k of hitting the box k . For j, n ∈ N , denote by K ∗ j ( n ) the number of boxes containing exactly j balls provided that n balls have been thrown. We call small counts the variables K ∗ j ( n ), with j ﬁxed. Our main result is a law of the iterated logarithm (LIL) for the small counts as the number of balls thrown becomes large. Its proof exploits a Poissonization technique and is based on a new LIL for inﬁnite sums of independent indicators P k ≥ 1 1 A k ( t ) as t → ∞ , where the family of events ( A k ( t )) t ≥ 0 is not necessarily monotone in t . The latter LIL is an extension of a LIL obtained recently by Buraczewski, Iksanov and Kotelnikova (2023+) in the situation that ( A k ( t )) t ≥ 0 forms a nondecreasing family of events.


Introduction 1.Definition of the model
Let (p k ) k∈N be a discrete probability distribution, with p k > 0 for infinitely many k.The infinite occupancy scheme is defined by independent allocation of balls over an infinite array boxes 1, 2, . .., with probability p k of hitting the box k.The scheme is usually called the Karlin occupancy scheme because of Karlin's seminal work [10].A survey of the literature on the infinite occupancy up to 2007 is given in [7].An incomplete list of very recent contributions includes [2,3,4,5].Among other things, the authors of [7] discuss applications of the scheme to ecology, database query optimization and literature.Another portion of possible applications can be found in Section 1.1 of [8].
There are deterministic and Poissonized versions of Karlin's occupancy scheme.In a deterministic version the nth ball is thrown at time n ∈ N.For j, n ∈ N, denote by K j (n) and K * j (n) the number of boxes hit by at least j balls and exactly j balls, respectively, up to and including time n.Observe that K 1 (n) is the number of occupied boxes at time n.Sometimes the variables K * j (n), with j fixed, are referred to as small counts.To define the other version of the scheme we need an additional notation.Let (S k ) k∈N denote a random walk with independent jumps having an exponential distribution of unit mean.The counting process π := (π(t)) t≥0 given by π(t) := #{k ∈ N : S k ≤ t} for t ≥ 0 is a Poisson process on [0, ∞) of unit intensity.
In a Poissonized version of Karlin's occupancy scheme the nth ball is thrown at time S n , n ∈ N, and it is assumed that the allocation process is independent of (S k ) k∈N , hence of π.Thus, in the time interval [0, t] there are π(t) balls thrown in the Poissonized version and ⌊t⌋ balls thrown in the deterministic version.While the occupancy counts of distinct boxes are dependent in the deterministic version, these are independent in the Poissonized version.The latter fact is a principal advantage of the Poissonized version.It is justified by the thinning property of Poisson processes.For j ∈ N and t ≥ 0, denote by K j (t) and K * j (t) the number of boxes containing at least j balls and exactly j balls, respectively, in the Poissonized scheme at time t.The random variables are the infinite sums of independent indicators.As a consequence, their analysis is much simpler than that of K j (n) and K * j (n) which are infinite sums of dependent indicators.
The function ρ is said to belong to the de Haan class Π if, for all λ > 0, for some ℓ slowly varying at ∞.The function ℓ is called auxiliary.According to Theorem 3.7.4 in [1], the class Π is a subclass of the class of slowly varying functions.Further detailed information regarding the class Π is given in Section 3 of [1] and in [6].Denote by Π ℓ, ∞ the subclass of the de Haan class Π with the auxiliary functions ℓ satisfying lim t→∞ ℓ(t) = ∞.
In the case α ∈ (0, 1], according to Theorems 3, 5 and 5' in [10], both K * j (t) and K * j (n), centered by their means and normalized by their standard deviations, converge in distribution to a random variable with the standard normal distribution.In the case ρ ∈ Π ℓ, ∞ , Corollary 1.6 in [9] provides functional central limit theorems for K * j (t) and K * j (n), properly scaled.Our purpose is to prove laws of the iterated logarithm (LILs) for K * j (t) as t → ∞ and K * j (n) as n → ∞.While doing so, we treat the three cases separately: α = 0, α ∈ (0, 1) and α = 1.The reason is that the forms of the LILs are slightly or essentially different in these cases.If ρ is slowly varying at ∞ and satisfies an additional assumption, then the actual limit relation is either a law of the single logarithm or a LIL.However, to keep the presentation simple we prefer to call LILs all the limit relations involving upper or lower limits which appear in the paper.
for some β > 0 and l slowly varying at ∞, then, for each j ∈ N, and for some σ > 0 and λ ∈ (0, 1), then, for each j ∈ N, and In both cases and Remark 1.2.Treatment of the situations in which ρ is slowly varying at ∞, yet ρ / ∈ Π is beyond our reach.To reveal complications arising in this case we only mention that even the large-time asymptotics of t → Var K * j (t) is not known.To find the asymptotic, a second-order relation for ρ like (2) seems to be indispensable.If α ∈ (0, 1], then the regular variation of ρ alone ensures that, for all λ > 0, Thus, no extra conditions are needed in this case. Remark 1.3.Our present proof only works provided that, for some a > 0, ρ(t) = O((ℓ(t)) a ) as t → ∞.In view of this, Theorem 1.1 does not cover the diverging slowly varying functions ℓ which grow slower than any positive power of the logarithm, for instance, ℓ(t) ∼ log log t as t → ∞.Indeed, it can be checked that lim t→∞ ℓ(t) = ∞ entails lim t→∞ (ρ(t)/ log t) = ∞, whence trivially, for all a > 0, lim t→∞ (ρ(t)/(ℓ(t)) a ) = ∞.
Theorem 1.4.Assume that, for some α ∈ (0, 1) and some L slowly varying at +∞, Then, for each j ∈ N, and and where Γ is the Euler gamma function and Theorem 1.5.Assume that, for some L slowly varying at +∞, Then, for each j ≥ 2, relation (11) holds, and Assume that, for each small enough γ > 0, where L(t) := ∞ t y −1 L(y)dy, being well-defined for large t, is a function slowly varying at ∞ and satisfying Then relation (11) holds with j = 1.If (18) does not hold, then and In any event Theorems 1.1, 1.4 and 1.5 will be deduced in Section 4 from the LIL for infinite sums of independent indicators given in Theorem 2.6.
Finally, we present LILs for the variables K * j (n).Theorem 1.6.Under the assumptions of Theorems 1.1, 1.4 or 1.5, for j ∈ N, all the LILs stated there hold true with K * j (n), EK * j (n) and Var K * j (n) replacing K * j (t), EK * j (t) and Var K * j (t), and n → ∞ replacing t → ∞.A transfer of results available for the Poissonized version to the deterministic version is called de-Poissonization.Theorem 1.6 will be deduced in Section 4 from Theorems 1.1, 1.4 and 1.5 with the help of a de-Poissonization technique.

LIL for infinite sums of independent indicators
Let (A 1 (t)) t≥0 , (A 2 (t)) t≥0 , . . .be independent families of events defined on a common probability space (Ω, F , P). Assume that k≥1 P(A k (t)) < ∞, for each t ≥ 0, and then put Since, for t ≥ 0, b(t) := EX(t) = k≥1 P(A k (t)) < ∞, we infer X(t) < ∞ almost surely (a.s.) and further Under the assumption that, for each k ∈ N and 0 ≤ s < t, A k (s) ⊆ A k (t) a LIL for X(t) can be found in Theorem 1.6 of [3].As an application, LILs for K j (t) were proved in that paper, see Theorems 3.1, 3.3 and 3.4 therein.According to (1), the variable K * j (t) is a particular instance of X(t).However, for each k ∈ N, the corresponding events (A k (t)) t≥0 are not monotone in t, which shows that a LIL for K * j (t) cannot be deduced from Theorem 1.6 of [3].This serves a motivation for the present section.Here, dropping the monotonicity assumption we provide sufficient conditions under which a LIL for X(t) holds.
Remark 2.3.The assumption imposed in [3] that, for each k ∈ N, the family (A k (t)) t≥0 is nondecreasing in t simplifies significantly the analysis of (43).Indeed, for any θ > 0, we then infer sup v∈ In the absence of the monotonicity assumption, it is necessary to find some monotone majorant for |X(t + v) − X(t)| which is sufficiently close to the true supremum.
(A5) For each n large enough, there exists A > 1 and a partition and, for all ε > 0, ) is a summable sequence.
Remark 2.5.A sufficient condition for (A5) is that f is eventually strictly increasing and eventually continuous.Indeed, one can then choose a partition that satisfies, for large n, ) is indeed summable.Assuming (A1) and (A3), fix any γ > 0 and put for large n ∈ N with µ as given in (25).Here, with q as given in (26), (B21) For sufficiently large t > 0 and each ς > 0, let R ς (t) denote a set of positive integers satisfying the following two conditions: for each ς > 0 and each γ > 0, both close to 0 there exists (B22) For sufficiently large t > 0, let R 0 (t) denote a set of positive integers satisfying the following two conditions: for each γ > 0 close to 0 there exists Now we are ready to present a LIL for infinite sums of independent indicators.
3 Proof of Theorem 2.6

Auxiliary results
We start with a simple inequality which will be used in the last part of the proof of Proposition 3.8.Lemma 3.1.Suppose (A2).Then, for ϑ ∈ R and t > s ≥ 0, The equality stems from the fact that Φ k only takes nonnegative integer values.Hence, for ϑ ∈ R and 0 ≤ s < t, For each B ≥ 0 and each D > 1, put g 1, B (t) := (B + 1) log log t, t > e and g D (t) := (D − 1) log t, t > 1.
(31) Lemma 3.2 does two things.First, it explains the choice of the sequences (t n ) and (v n ) and the functions g 1, q̺ and g µ̺ (even though (t n ) is not present in Lemma 3.2 explicitly, it is of crucial importance for defining the sequence (s n ).)Second, it secures a successful application of the Borel-Cantelli lemma in the proof of Proposition 3.8.
For k ∈ N and t ≥ 0, put X * (t) := X(t) − EX(t) and η k (t and that η 1 (t), η 2 (t), . . .are independent centered random variables.Lemma 3.3 provides a uniform bound for higher moments of the increments of X * .The bound serves a starting point of the chaining argument in the spirit of Lemma 3.4.A result of an application of Lemma 3.4 to the present setting is given in Lemma 3.5.
for a positive constant D r which does not depend on t and s.
Proof.In view of the representation the variable X * (t) − X * (s) is an infinite sum of independent centered random variables with finite moment of order 2r.Invoking Rosenthal's inequality (Theorem 3 in [12]) in the case r ≥ 2 we infer In the case r = 1, the inequality trivially holds with C 1 = 1 as is seen from In view of (A2), for r ∈ N and 0 ≤ s < t, Here, we have used for the second and (A2a) for the third.The argument for the case 0 ≤ t < s is analogous.
Combining fragments together we conclude that (35) holds with D r := 2 2r C r .
The next result is borrowed from Lemma 2 in [11].
Proof.We first show that the assumption of Lemma 3.4 holds with This in combination with Lemma 3.3 yields thereby proving that the assumption of Lemma 3.4 does indeed hold.Hence, inequality (36) follows from Lemma 3.4 and the definition of t n :
Putting θ = 2(1 + κ)h ̺ (a(s n )) and then invoking Lemma 3.2(a) we obtain According to (40), for some n 0 ∈ N large enough.An application of the Borel-Cantelli lemma completes the proof of Lemma 3.9.
Next, in order to prove (39) it suffices to show that and Since ( 42) is a consequence of (41), we are left with proving (41).
Proof of (41).Let t n = t 0, n < . . .< t j, n = t n+1 be a partition defined in (A5).With this at hand, write By Markov's inequality and Lemma 3.5, for any r > 0 and all ε > 0, By Lemma 3.2(b), there exists an integer r ≥ 2 such that the right-hand side forms a sequence which is summable in n.Hence, an application of the Borel-Cantelli lemma yields Next, we work towards proving that According to (A2), for any 0 By (A1) and (A5), Finally, for all ε > 0, having utilized Markov's inequality for the second inequality, Lemma 3.1 for the third and (A5) for the fourth.Invoking (A5) once again we conclude that the right-hand side is summable in n.Hence, an application of the Borel-Cantelli lemma yields lim The proofs of both (41) and Proposition 3.8 are complete.
4 Proofs related to Karlin's occupancy scheme

Auxiliary results
For ease of reference, we state two known results.The former is an obvious extension of Theorem 1.5.3 in [1].The latter is Lemma 6.2 in [3].
Lemma 4.1.Let f be a function which varies regularly at ∞ of positive index and g a positive nondecreasing function with lim t→∞ g(t) = ∞.Then there exists a nondecreasing function h satisfying f (g(t)) ∼ h(t) as t → ∞.
4.2 Asymptotic behavior of EK j (t) and Var K j (t) Given next is a collection of results on the asymptotics of EK j (t) and Var K j (t) taken from Lemma 6.5 of [3].Recall that Π ℓ, ∞ denotes the subclass of the de Haan class Π with the auxiliary functions ℓ, see (2), satisfying lim t→∞ ℓ(t) = ∞. and Assume that ρ(t) ∼ t α L(t) as t → ∞ for some α ∈ (0, 1] and some L slowly varying at ∞.If α ∈ (0, 1) and j ∈ N or α = 1 and j ≥ 2, then, as t → ∞, and 4.3 Asymptotic behavior of EK * j (t) and Var K * j (t) For j ∈ N, the asymptotics of t → EK * j (t) as stated in Theorems 1.1, 1.4 and 1.5 can be found in Lemma 6.5 of [3].Next, we show that, for j ∈ N, the functions t → Var K * j (t) exhibit the asymptotics given in the aforementioned theorems.Lemma 4.4.Assume that ρ ∈ Π ℓ, ∞ .Then, for each j ∈ N, Assume that ρ(t) ∼ t α L(t) as t → ∞ for some α ∈ (0, 1] and some L slowly varying at ∞.If α ∈ (0, 1) and j ∈ N or α = 1 and j ≥ 2, then, as t → ∞, with c j, α as defined in (15) and (17).
According to formula (6) in [7], Note that (55) does not require even regular variation assumption on ρ.
Assume now α ∈ (0, 1) and j ∈ N or α = 1 and j ≥ 2. Then invoking (55) and either (13) or ( 16) we obtain We are left with showing that the constants c j, α are positive for α ∈ (0, 1) and j ∈ N and α = 1 and j ≥ 2 or equivalently This is a consequence of Here, the last inequality is justified with the help of mathematical induction.The proof of Lemma 4.4 is complete.
4.4 Proof of Theorems 1.1, 1.4 and 1.5 For k ∈ N and t ≥ 0, denote by π k (t) the number of balls in box k at time t in the Poissonized version.It has already been mentioned in Section 1.1 that the thinning property of Poisson processes implies that the processes (π 1 (t)) t≥0 , (π 2 (t)) t≥0 , . . .are independent.Moreover, for k ∈ N, (π k (t)) t≥0 is a Poisson process with intensity p k .As a consequence, both K * j (t) and K j (t) can be represented as the sums of independent indicators Hence, it is reasonable to prove the desired LILs for the small counts by applying Theorem 2.6.
Proof of Theorems 1.1, 1.4 and 1.5.We first prove Theorem 1.5 in the case j = 1.This setting is much simpler than the others, for the LIL for can be derived from the already available LILs for K 1 (t) and K 2 (t).The statements of Theorem 1.5 concerning the function L has already been justified in the proof of Lemma 4.4.According to (51) and (54), Var K * 1 (t) ∼ Var K 1 (t) ∼ t L(t) as t → ∞.Invoking the latter relation, (50) and (19) we conclude that, Var K 2 (t) ∼ 2 −1 tL(t) = o(Var K 1 (t)) as t → ∞.By Theorem 3.4 and Remark 1.7 in [3], Armed with this, the claim of Theorem 1.5 in the case j = 1 is secured by Theorem 3.4 in [3].Indeed, the theorem states that depending on whether relation (18) holds or not, the right-hand side is either equal to 2 1/2 (−2 1/2 ) or is not larger than 2 1/2 (not smaller than −2 1/2 ) a.s.
In the remaining part of the proof we treat simultaneously Theorems 1.1 and 1.4 and the case j ≥ 2 of Theorem 1.5.It has already been announced that our plan is to derive the LILs from Theorem 2.6.Hence, now we work towards checking the conditions of the aforementioned theorem in the present setting.Condition (A1) holds according to (52) in conjunction with lim t→∞ ℓ(t) = ∞, (53) and (54).
for all k, j ∈ N and t ≥ 0, see Remark 2.2.The corresponding function f is given by We bring out the dependence on j and α to distinguish the so defined functions for the different settings.By the same reasoning, we write a j, α instead of a, where a(t) = Var K * j (t) for t ≥ 0. Condition (A3).Assume first that ρ ∈ Π ℓ, ∞ .According to (48) and (52), for each j ∈ N, f j, 0 (t) ∼ 2ρ(t) and a j, 0 (t) ∼ Cℓ(t) as t → ∞, respectively.Here and hereafter, C denotes a constant whose value is of importance and may vary from formula to formula.Under (3), invoking (46) we conclude that (A3) holds with µ = 1/β + 1.Under (6), using (47) we infer µ = 1.Thus, we have to check the additional conditions pertaining to the case µ = 1.First, the function f j,α is continuous.Second, q = 1/λ − 1 and L(t) ≡ 1 for all t ≥ 0 by another appeal to (47).
Assume now that α ∈ (0, 1) and j ∈ N or α = 1 and j ≥ 2.Then, according to (53), a 0;j, α can be chosen as a monotone equivalent of t → c j, α t α L(t) which exists by Lemma 4.1.
Thus, in all settings (A4) holds according to Remark 2.4.Condition (A5) holds according to Remark 2.5, for f j, α is continuous and strictly increasing.Condition (B1) holds in view of which shows that a j, α is a continuous function.Condition (B22).For t > 1, put c(t) := t/ log t and d(t) := t log t and then By Lemma 4.5, in all settings relation (30) which is the second part of (B22) holds.
Passing to the first part of (B22), we are going to refer to the table below which contains all the necessary information.In the first line, we list the values of µ which have already been found while checking (A3).Recall that the definitions of w n (γ, µ) and τ n can be found right after formula (29) and in (29), respectively.
We conclude that in all the settings τ n+1 /τ n diverges to ∞ superexponentially fast, whereas log τ n only grows polynomially fast.Hence, for large enough n, c(τ n+1 ) > d(τ n ), which justifies the first part of (B22).The proofs of Theorems 1.1, 1.4 and 1.5 are complete.

Proofs of Theorem 1.6
We start with some preparatory work.It is known, see, for instance, Lemma 1 in [7], that for any probability distribution (p k ) k∈N and j ∈ N, However, we are not aware of a counterpart of this relation for variances.Proposition 4.6 fills up this gap.Recall that ρ ∈ Π ℓ, ∞ means that ρ ∈ Π and that its auxiliary function ℓ, see (2), satisfies lim t→∞ ℓ(t) = ∞.
Proof of Proposition 4.6.We start by noting that, in view of (52), ( 53) or (54), for j ∈ N, In the case α = 1 this is secured by lim t→∞ L(t) = 0, which follows from the definition of L. For k, j, n ∈ N, the event {the box k contains exactly j balls out of n} will be denoted by A k (j, n).Then It is enough to prove that lim Proof of (68).For k, j, n ∈ N, In view of this and (64), the numerator in (68) is equal to According to the penultimate inequality in the proof of Lemma 2.13 in [9], for large enough n and any j ≤ n, for some positive constants A j and B j .Therefore, as n → ∞, since under our assumptions lim n→∞ Var K The proof of (68) is complete.
Proof of (69).For k, i, j, n ∈ N, We shall use an appropriate decomposition of C j To analyze C (1) j we argue as in the proof of Lemma 1 on p. 152 in [7].Invoking an expansion (x − y) m = x m + O(mx m−1 y), m → ∞, which holds for positive x and y, x > y, with x = (1−p i )(1−p k ), y = p i p k and m = n−2j, we infer Next, we intend to show that the contributions of F j (i, k, n), G j (i, k, n) and C (2) j (i, k, n) to the sum are negligible in comparison to Var K * j (n) as n → ∞.Analysis of G j .With Lemma 4.7 at hand, we obtain j (i, k, n).Further, invoking Lemma 4.7 yields Here, the latter asymptotic relation is a consequence of (67).The argument for F (2) j is analogous, and we omit details.

Analysis of C
(2) j .Notice that n j − n−j j = O(n j−1 ) as n → ∞.Hence, mimicking the argument used for the analysis of F (1) j we conclude that Combining all the fragments together we arrive at (69).
With Proposition 4.6 at hand, we are ready to prove the LIL stated in Theorem 1.6.We argue along the lines of the proof of Theorem 3.7 in [3].
Proof of Theorem 1.6.The deterministic and Poissonized schemes discussed in Section 1.1 are not necessarily defined on a common probability space.Our plan is to deduce LILs for K * j (n) from the corresponding LILs for K * j (t).To this end, we need to couple the two schemes.Let X 1 , X 2 , . . .be independent random variables with distribution (p k ) k∈N , which are independent of a Poisson process π and particularly its arrival sequence (S n ) n∈N .For all j, n ∈ N and t ≥ 0, we define coupled versions of K j (n), K * j (n), K j (t) and K * j (t) as follows, keeping the notation for the variables unchanged: K j (n) = # of distinct values that the variables X 1 , X 2 , . . ., X n take at least j times, K * j (n) = # of distinct values that the variables X 1 , X 2 , . . ., X n take exactly j times, K j (t) = # of distinct values that the variables X 1 , X 2 , . . ., X π(t) take at least j times, K * j (t) = # of distinct values that the variables X 1 , X 2 , . . ., X π(t) take exactly j times, To justify the construction, observe that the variable X i can be thought of as the index of a box hit by the ith ball.The most important conclusion of the preceding discussion is that, for all j, n ∈ N, K * j (n) = K * j (S n ) a.s.(for the coupled variables).We prove the result in several steps.

k≥1½
{the box k contains at least j balls at time t} and K * j (t) = k≥1 ½ {the box k contains exactly j balls at time t} with s = 0 and (A2c) together imply that b