Contextual Bandit Based Adaptive Numerology for Initial Access in 5G NR Networks

Abstract

Massive machine type communication (mMTC) is a key use case that is expected to be supported by 5G NR. One of the key challenges in 5G NR communications for mMTC use case is to enhance the initial access procedure to overcome the problem of congestion and overloading from mMTC devices. The intermittent blockages in the mmWave frequency range of 5G further degrade the random access performance significantly. 5G NR introduces the concept of numerology where multiple types of sub-carrier spacing is allowed. To circumvent the problem of random access performance enhancement, we propose a blockage aware adaptive numerology to maximize the number of devices that can be connected to the cellular network. In this paper, the problem of selecting optimal numerology for random access is modelled as a contextual multi-arm bandit framework while taking the blockages and their mobility pattern in the cell at a given time into account. We consider the upper confidence bound (UCB) action selection method to take the uncertainty of rewards of numerologies at a given instance into account. The proposed adaptive numerology solution maximizes the number of devices joining the network per RACH occasion and also minimizes the average joining time of a device when compared to fixed numerology schemes.

Appendix A  Convergence Bound for Linear UCB

Considering the linear estimate of the coefficients as given in Eq. 16, the convergence of the reward can be explained as follows. With at least a probability of $1-\delta$, the convergence bound given in (A1) holds good for any $\delta > 0$ and $\varvec{x}_{t,a} \in \mathbb {R}^d$,

$$\begin{aligned} \left| \varvec{x}_{t,a}^{'} \hat{\varvec{\theta }}_a-\mathbb {E}[N_i^a\vert x_t]\right| \le \gamma \sqrt{\varvec{x}_{t,a}^{'}\left( \varvec{H}_a^{'}\varvec{H}_a+\varvec{I}_d\right) ^{-1}\varvec{x}_{t,a}} \end{aligned}$$

(A1)

where $\gamma =1+\sqrt{ln(2/\delta )/2}$ is a constant. In other words,

$$\begin{aligned} P\left( \left| \varvec{x}_{t,a}^{'} \hat{\varvec{\theta }}_a-\mathbb {E}[N_i^a\vert x_t]\right| \ge \gamma \sqrt{\varvec{x}_{t,a}^{'}\left( \varvec{H}_a^{'}\varvec{H}_a+\varvec{I}_d\right) ^{-1}\varvec{x}_{t,a}}\right) \le \delta \end{aligned}$$

(A2)

The parameter $\gamma$ controls the degree of exploration of numerologies with low experience. In other words, as $\gamma$ increases, the exploration of new arms of less experienced arms is more encouraged. The above convergence has been proven in [27] for the first time and then applied to contextual bandits in [24].