When forecasting time series, it is important to classify them according to linearity behavior; the linear time series remains at the forefront of academic and applied research. It has often been found that simple linear time series models usually leave certain aspects of economic and financial data unexplained. The dynamic behavior of most of the time series in our real life, with its autoregressive and inherited moving average terms, pose the challenge to forecast nonlinear times series that contain inherited moving average terms using computational intelligence methodologies such as neural networks. It is rare to find studies that concentrate on forecasting nonlinear times series that contain moving average terms. In this study, we demonstrate that the common neural networks are not efficient for recognizing the behavior of nonlinear or dynamic time series which has moving average terms and hence low forecasting capability. This leads to the importance of formulating new models of neural networks such as Deep Learning neural networks with or without hybrid methodologies such as Fuzzy Logic.