Iranian Journal of Finance

Abstract This study aims to demonstrate the performance of algorithmic trading strategies compared to traditional trading methods in artificial financial markets. This research uses a hybrid model based on agent-based modeling and machine learning methods to simulate agents' behavior in an artificial financial market. This model includes two categories, traditional agents and intelligent agents. Traditional agents are divided into three groups: liquidity providers, liquidity consumers, and noise traders. Intelligent agents are trained using deep learning techniques and recurrent neural networks. Based on the developed algorithms, the agent-based model simulates both categories of traditional and trained agents in an artificial financial market. Sensitivity analysis tests were used to test the validity and reliability of the model, and the values of the fat-tailed distribution of returns, volatility clustering, autocorrelation of returns, long memory in order flow, concave price impact, and extreme price events are calculated in the model and compared with the standardized values. Historical data was used to predict stock prices, and model simulations were used to generate trading signals and update the limited order book. The results of executing the model show the ability of intelligent agents to trade in artificial financial markets compared to traditional agents.

Abstract Although theoretical and empirical literature regarding the stylized facts shows evidence of their correlations to herding behavior in financial markets, the causes of such phenomena are still unknown. Using an agent-based model strengthened by the competition co-evolution algorithm (STGP) technique, this study provides laboratory evidence on capital market dynamics and analyses the behavioral foundations of stylized facts such as fat tails, leverage effects, and volatility clustering. The simulated stock markets consist of two groups; the “Best agents”, which are a small portion of artificial agents, and the “Residual agents”, which are the main group of artificial agents. The best performance in terms of breeding fitness returns is the main feature of the “Best agents”. More, the size of the “Best Agents” group is specified as 2.5%, 5%, 10% &20% of the total population size. An agent-based model consists of two portions, a two thousand population of trader agents that each has its decision-making strategy, and a virtual market that creates the trading strategies. Then the model evolved step by step using a feed with real quotes of the financial instruments by Adaptive Modeler. A training period is considered 2500 bars (started in November 2003), and the test period started in December 2013. The observation shows that the herding behavior in the price series created by the “Residual agents” is less than the “Best agents” series. Therefore, the greater diversity of trade strategies as the genetic differences of artificial agents leads to less herding. The observations exhibit that the volatility clustering, leverage effects, and nonlinear dependence are more likely to experience in the price series generated by “Best gents”. Furthermore, observations indicate that if the population is well diversified in terms of trading strategies, the efficiency of the market increases.