Modeling price dynamics and risk Forecasting in Tehran Stock Exchange: Conditional Variance Heteroscedasticity Hidden Markov Models

Volatility and risk measurement are essential parameters in risk management programs that can affect economic activities and public confidence in the stock market. Also, these two are the keys in the studies that connect the stock market, economic growth


Introduction
One of the main functions of the stock market is to distribute risks arising from economic activities among many people.Thus, understanding the volatility behavior of [stock] prices is important for valuing financial assets and implementing risk-hedging strategies (Evgenidis, 2018).One accepted fact in the empirical research conducted in the financial and stock markets is that volatility varies over time (Bentes, 2021).Many models that model timevarying volatility have been developed in the last decade, which fall into two major groups.In group (1), there are GARCH-type models (Engel, 1982), and in group (2), there are stochastic volatility (SV) models (Taylor, 1986), Markov-Switching (MS), and hidden Markov (HM).Koopman et al. (2016) introduce the group of SV and Markov models as parameter-driven models.This latter group is flexible in volatility modeling.Mayer and Yu (2000), Berg et al. (2004), andShephard (2005) consider the strength of group (2) models in comparison with GARCH-type models to be the existence of an excess white noise process in the equation of state.In these models, volatility is a hidden variable and follows a stochastic process (Brooks & Prokopczuk, 2013) and Lin et al., 2020).Some research in financial literature has shown that group (2) models are more favorable than GARCH models, and it is better to model volatility as a hidden random process (Yu, 2002, Hansen, Huang, andShek, 2012;Wang et al., 20016).Considering the structural transmission of volatility during economic and financial cycles, some researchers have recommended volatilityswitching models in volatility forecasting and risk assessment and management.For example, Marcucci (2005) compared the forecasting ability of ensemble MS-GARCH models with single-regime GARCH models.His findings prove that MS-GARCH models are significantly superior to the models of the second group.Chang ( 2012) applied an MS-EGARCH model to predict future prices.He found that this model can well portray the stylized facts in the financial markets.Also, Herrera et al. (2018) confirm that the MS-GARCH model provides more accurate forecasts than simple GARCH-type models for predicting future price volatility.In general, regime-switching models that incorporate the effect of structural changes in economic conditions improve the ability to model the characteristics of financial time series.Also, Lin et al. ( 2020) have compared the efficiency of MS and HM volatility models in forecasting volatility in WTI crude oil prices and China Dacoin.They have shown that HM models are more effective in predicting both types of crude oil volatility.
In the literature of the financial field, less attention has been paid to parameter-driven models, especially hidden Markov models in risk management and evaluation using quantitative criteria such as Value at Risk (VaR) and Conditional Value at Risk (CVaR).Because GARCH models do not include the structural instabilities of the stock market due to regime change, it will lead to biased estimates of volatility and risk.The innovation of this paper, apart from the Bayesian estimation method, is the use of data from the financial market of Iran, which is a developing and relatively new market with the main feature that an economy is subject to heavy and multilateral and unilateral sanctions by economic powers.The characteristics of such a market have yet to be discovered, and its deeper investigation using financial econometric methods can help advance financial science.
The following structure of this research will be as follows: In the second part, a literature review will be presented; in the third part, the method of estimations will be included; in the fourth part, the empirical findings will be discussed; and in the final part, the conclusion will be presented.

Literature Review
Risk is an integral part of financial markets.In the field of risk measurement in financial literature, there are generally three common measurement methods used in research in the order of volatility, value at risk (VaR), and conditional value at risk (CVaR).Engel (2006) says, "Uncertainty can change people's behavior in macroeconomics.Although such an effect (in macroeconomics) can be real, it is not large compared to other effects.In finance, uncertainty and risk determine the main characteristics of what will happen in the future", therefore; the special task of volatility is to measure uncertainty in financial markets, and for this reason, its effect is very important, but although this measure is related to risk, it is not the same.Because risk is associated with adverse future outcomes of the market, but volatility is also associated with positive outcomes.However, those like Sharpe (1966) have used volatility as a suitable proxy variable to measure risk in the definition of Sharpe's ratio.Also, Markowitz (1952) and Tobin (1958) have defined Variance as a measure of risk.This method of measuring risk is correct when the probability distribution of returns is normal.In this case, there is no difference between risk and volatility (Poon, 2005).
The VaR measurement method, first introduced by Morgan (1994), has been developed as one of the most common approaches in financial markets to manage market risk.VaR defines the maximum losses an investor can face for a given tolerance level over a specific time.Although Basel II and III recommend VaR, financial institutions have widely adopted it.
However, in measuring risk, it should be noted that the risk depends on the properties of the stochastic process generating asset price data (heavy-tailed or light-tailed of the process, whether the process is peaked or not peaked, the presence or absence of jumps in the process), market type, time horizon (shortterm, medium-term and long-term), and the macroeconomic environment (Chan & Grant, 2016).
According to the opinion of the Banking International Settlements (BIS) Committee, the VaR criterion cannot provide a good representation of rare events in the tails of the return distribution (Chen et al., 2019).Also, Artzner et al. (1999) argue that VaR does not satisfy the sub-collectivity requirement and therefore is not a consistent measure of risk.
The CVaR criterion measures risk conservatively but consistently (Chen et al., 2019).Financial studies using VaR and CVaR to measure risk generally focus on declining stock prices (i.e., downside risk).Therefore, investors in the market inevitably lose when stock prices fall due to sudden negative news.
The effectiveness of VaR and CVaR methods in evaluating market risk depends on the methods of modeling and predicting volatility.Research shows parameter-driven group models perform better than GARCH group models in modeling and predicting turbulence (Hansen et al., 2012).2020) compared different types of GARCH models in predicting the risk of asset prices using 12 out-of-sample forecasting criteria.They concluded that the best risk prediction model for oil prices is the HM-EGARCH single-regime model.

Lin et al. (
Rastgar and Hemati (2022) analyzed the sensitivity of three different methods in calculating value at risk.They conclude that estimation methods using loss function, Garch-Copola methods, Extreme value theory, and dynamic conditional correlation were ranked first to third in VaR calculation, respectively.Behzadi (2020), for modeling and measuring portfolio risk, investigates the effect of financial stylized facts in GARCH models.He concludes that when using the GARCH family, the GARCH Golsten-Jaganathahan-Runkel model performs better than the other two methods, which shows the importance of considering leverage effects in risk modeling for stock returns in the Tehran Stock Exchange.
In an article, Farhadian et al. (2020) investigated random volatility models and Markov-Garch switching.Their conclusion shows that MS-GJRGARCH models perform better than other models for calculating risk, which shows the importance of the change of state system in risk modeling.
In a research, Raei et al. (2020) evaluate the CVaR criterion using fixed variance and variable variance methods in the Iranian stock market.They conclude that to assess and manage risk with the CVaR criterion, the use of time-varying variance methods shows better performance.Saranj and NoorAhmadi (2016) in research compare different Frein and GARCH value models in estimating VaR and CVR criteria.They conclude that based on the McNeil and Frey method and the MCS ranking test, Freinder's value methods for estimating VaR and CVaR criteria perform better than GARCH methods.Najafi et al. (2017) investigated this issue in research on how to optimize the stock portfolio.They show that using the CVaR criterion optimizes the portfolio interval more efficiently.

Research Methodology
For the current research, the daily data of the total price index of the Tehran Stock Exchange was used in the period from September 27, 2010, to July 22, 2022.To calculate data yield, we use the logarithmic yield formula as follows: In the rest of this section, econometric methods are presented according to the research objectives.

Definition of parameter-driven and observation-driven models
Different volatility modeling differences can be understood through observation-driven and parameter-driven models introduced by Cox (1981) and reviewed by Koopman et al. (2016).If is a parameter at time t and εt is a random variable observed at time t, in an observation-driven model, we will have: where Φ is a measurable function.In the parameter-driven model, the relationship is defined as follows: (3) Where ηt is a new and specific change at time t.Also, Φ* is a measurable function.In the latter case, the model has a hidden structure that cannot generally be directly related to the observations.
In the current research, the performance of parameter-driven Volatility models -Hidden Markov models (HM) that do not have an observationoriented driven form will be investigated in evaluating and forecasting the risk of the Total price index of the Tehran stock market.

Hidden Markov model
A hidden Markov model consists of a binary stochastic process, one of which is a hidden Markov chain to express the behavior of the hidden states, and the other is a sequence of observations specified by the currently hidden states of a Markov chain (Rabiner & Juang, 1986).A hidden Markov model can be expressed as follows: 1) Probability distribution of the initial state , 2) Probability of transition between states or regimes

3) Distribution of observations (signals) in terms of regimes
N is the number of states shown with Zt at time t in the above relations.
Therefore, a hidden Markov model is a triple of the

Symmetrical model
The general form of a GARCH model introduced by Bullersleaf ( 1986) is as follows: (8) For confirmation of the positive conditional variance, a limitation to be imposed on the coefficients of this model is: Also, the sufficient condition for the validity of the GARCH process is: 1

Asymmetric EGARCH model
Usually, the impact of uncertainty on investors' decisions in asset markets could be more symmetrical.In other words, investors will give more weight to the occurrence of a potential future loss than an identical potential profit when making their decisions in buying and selling assets.The facts observed in stock markets show that negative and positive return shocks do not have the same effect on volatility, which means an asymmetric effect of uncertainty on investors' decisions.This asymmetry is sometimes described as the leverage effect and other times as the risk premium effect.To choose a suitable model for measuring and evaluating volatility in Tehran Stock Exchange, it is necessary to check the existence of the leverage effect.Ignoring the leverage effect in the stock market (if any) leads to a fundamental bias in predicting future market prices (Hall & White, 1987).In this regard, generalizations such as the EGARCH model by Nelson (1991), GJR-GARCH by Glosten et al. (1993), or the TGARCH model by Zakoian (1994) can be mentioned, which include the asymmetric relationship of returns and changes in variance.In this research, to investigate the existence of the leverage effect in the Tehran Stock Exchange, the EGARCH model has been used in the hidden Markov model.
This model was presented by Nelson (1991) to correct some of the weaknesses of the GARCH model.The general form of this model is as follows:


The i  coefficient must be negative in the above expression if there are leverage effects.In this case, the occurrence of a negative shock equal to one unit will have an effect equal to (1 )   the volatility, and if the shock is positive, the amount of this effect will be equal to (1

Combination of volatility models with Markov functions of regime transition
The dynamics of asset return behavior depend on whether the variance behavior is mechanistic or evolutionary.An evolutionary model describes volatility behavior concerning variance regimes.What regime the variance is in at any point in time can be considered a random or definite matter depending on the information of the researchers.In the present research, it is assumed that it is impossible to definitively determine what regime the variance is in at any time.In such a situation, the t y conditional distribution can be defined as follows: (12) The above relation   .Also, Markov properties After introducing the dynamics of return behavior and how it is related to the variable variance in time and different regimes according to Haas, Mittnik, and Paollela (2004), the conditional variance 2 , st  t r return from a GARCH regime change model provided that the Z s variable is in regime s will be defined as follows: (12) In the above relations, the   .f functional form of the variance is conditional and, depending on the specification of the GARCH model and the parameter vector of the S  variable regime of q, can be equal to one or two.
After the model's specification, the likelihood function is formed to estimate the parameters.Of course, because the Z t state variable is unobservable, this method's problem of estimating coefficients is highly nonstandard.If we show the vector of parameters as a set its likelihood function will be as follows: (13) is the probability density function of if there is 1 t G  a filter and the  parameter vector?For the GARCH regime change model, density r t is defined as follows: (14) In the above relationship , 1 1 1 ( , ) is the Hamiltonian filter of the probability of state I at time t-1.
The above description is volatility modeling using the Markov state change model (MS-GARCH).When the S t states are considered hidden variables, their equations should be obtained using Markov's MCMC.This issue will lead to the reduction of volatility measurement error.The present study estimates

Evaluation of models based on out-of-sample forecasting
The out-of-sample prediction error calculation criteria of root mean square error (RMSE) and mean absolute error (MAE) have been used to choose the appropriate model for predicting volatility and risk.To make the comparison between the models more appropriate, forecasting a future period for different two-regime HM-GARCH models that include both asymmetric effects in the form of the EGARCH model and heavy-tailed marginal distribution of the ttype is examined.As expected, we will use the model in which the criteria obtained the lowest amount for predicting volatility and assessing risk.

The risk backtests
To evaluate the back test of VaRs accuracy, it is necessary to calculate the empirical failure rate of the estimates.The failure rate calculates the ratio of times returns exceed the estimated VaRs to the total number of observations.If the calculated ratio is equal to the predetermined VaR level (i.e., α=1% and α=5%), the model is said to be correctly specified.If the failure rate is higher than the desired quantile, it can be concluded that the model underestimates the risk.
The Failure rate for the downside risk of a long trading position (denoted by FRVaRs) is calculated as the percentage of negative returns smaller than the left-hand quantile.Also, the failure rate for the upside risk of a short trading position (denoted by FRVaRd) is defined as the ratio of returns more significant than the right-hand quantile of VaR.FRVaRs and FRVaRd are defined as follows: (15) -The conjugate prior probability distribution function is the prior distribution that produces the posterior distribution function of its cognate parameters.These types of priors reduce the computational complexity of the moments of the posterior distribution of the parameters.

,
In this relation, and are estimated values for downside and upside risk at time t for a given confidence interval.Also, T is the number of observations and is the indicator function, which is defined as follows: Several formal tests are based on the above criteria for backtesting VaR estimates.In the present study, the unconditional coverage test (LRuc), proposed by Kupice (1995), and the dynamic quantile test (DQ) introduced by Engel and Manganelli ( 2004) are used.Both tests examine the null hypothesis H0 ∶F R=α.Good VaR model performance should be accompanied by unconditional exact coverage, i.e., statistically expected failure rate to equal the prescribed significance level.
In the following, a number of the most recent researches that have dealt with volatility modeling and risk assessment with VaR and CVaR criteria in different markets using HM-GARCH models have been introduced.

Data Analyses
graph (1) shows the empirical distribution of total price index data.Graph (1) shows that the return behavior of the total index of the Tehran Stock Exchange is almost symmetrical (around the average).In the case of substantial returns (negative or positive), it is almost similar to other financial markets.As indicated by arrows A and B, the substantial and very negative return values shown in the tails of the distribution occur in this market more than would be expected from a normal variable.This stretch above the tails can be seen as a sign of different variance regimes.
In general, the dispersion of stock market returns shows that the risk and danger in this market are not uniformly distributed, but it shows themselves densely in limited (sequential) events.This distribution of return data shows that the stock market is heavily influenced by unpredictable human events.Modeling price dynamics and risk Forecasting in Tehran… figure 1

. Empirical distribution of total index return data
Analysis of historical stock market data in a box plot (graph ( 2)) shows that the number of outliers (beyond the branches of the box plot) is much higher than expected from a normally distributed variable.

figure 2. Box plot of yield data and outliers in sample data
Table (2) shows the skewness coefficient of the price return data and the central and dispersion indices.This coefficient measures the relative frequency of occurrence of large returns in a specific direction (right or left).One of the realities of the Tehran stock market is that this market has experienced negative and positive values of large or limited returns almost equally (with a slight deviation towards large positive values of returns because the skewness coefficient of these data is positive but less than 0.5 (The value of this coefficient is 0.1768) this deviation cannot be considered very important).Such a behavior of the return distribution in the stock market is expected according to the utility theory.Because it means that the investor in this market prefers a slight chance of getting a significant return to a big chance of getting a small return; nevertheless, due to the almost insignificant importance of this coefficient in the stock market, traces of the irrational behavior of market participants can be guessed.

Inferential data analysis
Table (3) shows the evaluation results of two-regime symmetric (HM-GARCH) and asymmetric (HM-EGARCH) hidden Markov volatility models in light of the marginal distribution of returns in each regime using RMSE and MAE criteria.Based on the results presented in Table (3), the HM-EGARCH model with normal distribution in both regimes has the lowest error rate based on RMSE and AME criteria.Therefore, based on this result, this model will be the most suitable for predicting future risks.
After selecting the appropriate model for predicting risk and volatility, the parameters of the selected HM-EGARCH-Normal models were estimated for the return of the total index of the Tehran Stock Exchange, and the results are reported in Table (4).These results show that the volatility in the return of the total index contains two regimes characterized by regime one and regime two.According to the findings in Table ( 4), the unconditional volatility (indicated by Uncon Vol) is higher in regime 2. Therefore, the severe volatility regime will be regime two, and the mild volatility regime will be regime one.
In the Tehran stock market, the leverage effect is confirmed in both regimes.Based on these results, the effect of a negative shock on Tehran's stock market volatility will be 0.732 in regime 1 and 0.1683 in regime 2. Therefore, negative shocks in a mild regime will have a stronger effect on volatility. The coefficient shows the stability of volatility waves in both regimes.According to the results, the stability of volatility waves in regime 1 is equal to 0.237, and in regime two is equal to 0.941.As a result, in the Tehran Stock Exchange, volatility stability is more in the volatility regime.

Table 4. Estimation of regime models for stock returns
The results of probability distribution between the states show that in Tehran Stock Exchange, the probability of transferring volatility from regime 1 to regime 2 equals 0.0486, and the probability of transferring from regime 2 to regime 1 is also about 0.021.This situation shows that the transition of the market to calm and volatile conditions depend on rare events and completely random events such as sanctions.5), it is expected that the volatility in the stock exchange will gradually increase and reach the highest level in predicted days on the eighth day.

VaR and CVaR calculation results
After choosing the appropriate model among the HM-GARCH models for predicting volatility, It is used to predict Value at Risk (VaR) and Conditional Value at Risk (CVaR).In the current research, based on the selected HM-EGARCH_ Normal model, the VaR and CVaR criteria have been calculated depending on the occurrence of severe volatility (regime 2) or mild volatility (regime 1) for the ten days of the beginning of August 2022 in the quantile of 5%.Based on the findings presented in Table ( 6), in the case of a mild volatility regime (regime 1) in the Tehran Stock Exchange based on the VaR criterion, the highest expected loss (equal to -0.136 per monetary unit invested in this market) with the probability occurrence of 5% is related to the tenth day and the lowest loss (equal to -0.115 per monetary unit invested in this market) will be related to the first, sixth and seventh days.However, in case of the realization of the extreme volatility regime (regime 2), the maximum loss in the time frame of this study's forecasts (equal to -2.395 per currency unit of investment) is expected to occur on the tenth day, and the lowest expected loss will occur on the first day of trading.The risk assessment in Tehran Stock Exchange using the CVaR criterion is similar in terms of the time of occurrence of the maximum and minimum losses in the case of regime 1.However, in the case of regime 2, the maximum loss is predicted to occur in the fifth month and the minimum in the first month of the year.The results of the Kupiec test for the HM-EGARCH-Normal model show that the hypothesis H0 ∶F R=α is not rejected.As a result, it can be said that the model correctly calculates the value at risk of the Tehran Stock Exchange market.The method presented by Kupiec (1995) can test for overor underestimation of a VaR model.However, it does not consider whether the exceptions are scattered or occur in clusters.In other words, the Kupiec approach to the VaR backtest ignores the autocorrelation in the tails of the return distribution and the VaR values.Engel and Manganelli's (2004) test has been used in the current research to overcome this weakness.This test is known as the dynamic quantile (DQ) test.As Table (8) shows, the estimated VaR values in the 5% and 1% quantiles, neither in the downward (short position) nor in the ascending (long position) state, do not reject the null hypothesis (H0 ∶F R=α).As a result, we can accept the efficiency of the HM-EGARCH-Normal method in risk estimation.

Discussion and Conclusion
The results of this research show that to accurately assess risk in the Tehran Stock Exchange and control its negative consequences, the structure of volatility assessment models should be modified in such a way that the changes in the dynamics of developments in this market include the result of events such as business cycles (which determined with known phases/regimes as stagnation, crisis, improvement, and expansion) and political (such as the agreement or non-agreement of the JCPOA).Ignoring non-linear developments in the volatility dynamics of the Tehran stock market will lead to an incorrect assessment of the expected loss in this market.Also, this finding is significant in risk management programs.Because reliable forecasts of stock price volatility are an important input for monetary and financial authorities whose task is to stabilize growth and promote productivity growth.Also, the results showed that the marginal normal distribution is sufficient in describing the volatility developments of the Tehran stock market.
In addition, this research showed that in the framework of hidden Markov models, the most suitable model for predicting the return and volatility of the Tehran market is the asymmetric EGARCH model with normal distribution.This issue precisely indicates that adding heavy tail distributions such as t distribution instead of normal distribution will not increase the efficiency of random volatility models in predicting the future return and volatility of the Tehran Stock Exchange.Also, adding the leverage effect from the EGARCH model to the HM model significantly improves the out-of-sample prediction efficiency of the Tehran Stock Exchange volatility.Considering that the data of this study covered from the second month of 1986 to the end of 2021 (a relatively long-time trend), the change of stock market dynamics according to the political cycles in Iran is an expected issue and HM models because this change by considering the dynamics over time by regularizing the volatility changes can reflect this characteristic of crude oil time series data.
Future research can achieve newer results by developing the models of this research.For example, future research can extend the HN model with different GARCH models for each regime (for example, the HM-EGARCH-GARCH model that uses the EGARCH model for regime one and the GARCH model for regime 2).Also, methods such as the Bayesian factor and other criteria can be used to evaluate different HM models.Although these generalizations have many computational difficulties, they are an interesting area for future research.
the Bayesian approach and the MCMC algorithm.For this purpose, based on the Bayesian rule, it is necessary to use the   f  prior probability density function in combination with the likelihood function of relation (13) to obtain the functional form of the posterior probability distribution is uncertain because the prior probability distribution functions are not conjugate.1,so it is necessary to use Gibbs or Metropolis-Hastings sampling algorithms for estimation.
Graph (3)  shows the probability of the working days of the Tehran Stock Exchange being in the high volatility regime or regime (2).

figure 3 .
figure 3. The smoothed probability of the market working days being in a highly volatile regime Based on the findings presented in this graph, almost all the days after 20198 until the beginning of August 2022, with a probability of more than 0.8, are in regime (2) or severe volatile regimes.This finding follows graph (4), which shows the yield volatility in the time interval from the first working day of 2019 to August 1, 2022.

figure 4 .
figure 4. volatile waves corresponding to the volatile regime or regime (2) Table (5) shows the volatility prediction of the total index of the Tehran Stock Exchange based on HM-EGARCH_ Normal models for the first ten days of August 2022.Based on the results presented in Table (5), it is expected that the volatility in the stock exchange will gradually increase and reach the highest level in predicted days on the eighth day.

Table ( 7
) shows the results of Kupiec's test for the HM-EGARCH-Normal model.