Recurrent Neural Networks, particularly LSTMs and GRUs, have revolutionized how machine learning models handle sequential data, leading to breakthroughs in numerous fields.

Natural Language Processing is a vast field focused on enabling computers to understand, interpret, and generate human language. RNNs are foundational to many NLP tasks.

Sentiment Analysis Example:

• Problem: Given a piece of text (e.g., a movie review, a tweet, a product feedback comment), determine the underlying sentiment expressed (e.g., positive, negative, neutral).
• Why RNNs are Suitable: The sentiment of a sentence is not just about individual words, but about their sequence and context. For example, "This movie was not bad" carries a positive sentiment due to the sequence "not bad," whereas "bad" alone is negative. An MLP would struggle with this nuance as it processes words independently.

RNN Approach:

1. Word Embeddings: Each word in the input text is first converted into a numerical vector representation called a "word embedding." These embeddings capture semantic meaning and relationships between words (e.g., "king" and "queen" might be close in the embedding space).
2. Sequential Input: The sequence of word embeddings is fed into an RNN (LSTM or GRU) layer, one word (embedding) at a time.
3. Contextual Understanding: As the RNN processes each word, its hidden state accumulates a contextual understanding of the sentence up to that point. It "remembers" previous words and their influence.
4. Final Prediction: After processing the entire sentence, the final hidden state (or a combination of hidden states) is passed to a dense output layer (often with a Sigmoid activation for binary sentiment: positive/negative, or Softmax for multi-class: positive/negative/neutral). This output layer then predicts the overall sentiment.

RNNs are also used in:
- Machine Translation: Translating text from one language to another.
- Speech Recognition: Converting spoken words into text.
- Text Generation: Creating new, coherent text (e.g., story writing, chatbots).
- Named Entity Recognition: Identifying specific entities (people, organizations, locations) in text.

13.2.2 Time Series Forecasting (Conceptual)

Time series data consists of observations recorded at successive time points. Forecasting involves predicting future values based on past observations.

Problem: Predict the stock price of a company for the next week based on historical prices, predict energy consumption, or forecast weather patterns.

Why RNNs are Suitable: Time series data inherently has dependencies over time. Future values are often influenced by a sequence of past values, not just the single immediately preceding value.

RNN Approach:

1. Sequential Input: A sequence of historical time series values (e.g., past 30 days' stock prices) is fed into the RNN.
2. Pattern Recognition: The RNN (LSTM or GRU) learns to recognize temporal patterns, trends, seasonality, and long-term dependencies within the sequence. It can capture how a rise in price 10 days ago might still influence today's price.
3. Prediction: Based on the learned patterns and the current hidden state, the RNN outputs a prediction for the next value in the sequence (e.g., tomorrow's stock price). For multi-step forecasting, this output can then be fed back as an input for the next prediction step.

Conceptual Example: To predict tomorrow's temperature, an RNN might consider the temperatures of the past 7 days, the day of the week, and perhaps even historical weather patterns, learning how these sequential factors contribute to the next day's forecast.

While more complex models exist for time series (e.g., Transformer models), RNNs laid the groundwork for deep learning in this domain due to their ability to process and "remember" sequential information.