International Journal of Technology and Applied Science

E-ISSN: 2230-9004   •   Impact Factor: 9.914

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 17 Issue 11 (November 2026) Submit your research before the last 3 days of this month to publish your research paper in the current issue.

Scalable Stock Market Prediction Using MapReduce-Based Algorithms: A Distributed Computing Approach

Author(s) Dr. Thallapally Sameera Pallavi
Country India
Abstract The amount of data that is generated by stock markets is growing in volume, velocity, and variety, making it difficult for a single machine to process in a timely manner. In this paper, we explore how feature engineering and model estimation for stock market prediction can be made scalable using the MapReduce programming model as a means of parallelization. The study briefly summarizes the existing literature on machine learning for market forecasting and on distributed computing for analytics, and introduces the proposed MapReduce-based pipeline: mappers calculate technical indicators and partial sufficient statistics for every partition of ticker data; a reducer sums up the statistics to obtain an estimate of a ridge-regularised direction classifier. The pipeline was tested on 500 to 8000 synthetic equity series, with a planted weak autoregressive signal, with true multiprocess execution on a two-core machine. In the distributed model, the centralised coefficients were reproduced within 4.4 × 10⁻¹³, thus proving that the approach does not alter the result of the computation, rather that it merely speeds up the calculation. The run times were reduced by 1.31 to 1.71, compared to sequential execution, and out-of-sample directionality accuracy was close to 51.3 per cent, compared to 50.7 per cent for the maximum class. The paper covers the overheads, data leakage, backtest overfitting, and comparison of Hadoop MapReduce and in-memory engines.
Keywords stock market prediction; MapReduce; distributed computing; big data analytics; Apache Hadoop; machine learning; time series forecasting.
Published In Volume 12, Issue 6, June 2021
Published On 2021-06-05
DOI https://doi.org/10.71097/IJTAS.v12.i6.1454

Share this