PyTorch Sentiment Analysis with Recurrent Neural Networks (RNN) using SST Dataset

This assignment aims to implement Recurrent Neural Networks (RNN) using PyTorch for sentiment analysis. Sentiment analysis involves classifying sentences (input) into specific sentiments (output labels), such as positive, negative, and neutral. We will utilize the SST (Stanford Sentiment Treebank) dataset, a benchmark for this task.

We begin by downloading and preprocessing the SST dataset from the torchtext package. This includes building a vocabulary and splitting the data into training, validation, and test sets. The provided code for this step can remain unchanged.

import copy
import torch
from torch import nn
from torch import optim
import torchtext
from torchtext import data
from torchtext import datasets

TEXT = data.Field(sequential=True, batch_first=True, lower=True)
LABEL = data.LabelField()

# load data splits
train_data, val_data, test_data = datasets.SST.splits(TEXT, LABEL)

# build dictionary
TEXT.build_vocab(train_data)
LABEL.build_vocab(train_data)

# hyperparameters
vocab_size = len(TEXT.vocab)
label_size = len(LABEL.vocab)
padding_idx = TEXT.vocab.stoi['<pad>']
embedding_dim = 128
hidden_dim = 128

# build iterators
train_iter, val_iter, test_iter = data.BucketIterator.splits(
    (train_data, val_data, test_data), 
    batch_size=32)

1. Defining Training and Evaluation Functions

The following code defines functions for training and evaluating the model:

def train(model, iterator, optimizer, criterion):
    model.train()
    epoch_loss = 0
    epoch_acc = 0
    
    for batch in iterator:
        optimizer.zero_grad()
        
        text = batch.text
        label = batch.label
        
        predictions = model(text)
        loss = criterion(predictions, label)
        
        acc = calculate_accuracy(predictions, label)
        
        loss.backward()
        optimizer.step()
        
        epoch_loss += loss.item()
        epoch_acc += acc.item()
        
    return epoch_loss / len(iterator), epoch_acc / len(iterator)


def evaluate(model, iterator, criterion):
    model.eval()
    epoch_loss = 0
    epoch_acc = 0
    
    with torch.no_grad():
        for batch in iterator:
            text = batch.text
            label = batch.label
            
            predictions = model(text)
            loss = criterion(predictions, label)
            
            acc = calculate_accuracy(predictions, label)
            
            epoch_loss += loss.item()
            epoch_acc += acc.item()
        
    return epoch_loss / len(iterator), epoch_acc / len(iterator)


def calculate_accuracy(predictions, label):
    _, predicted = torch.max(predictions, 1)
    correct = (predicted == label).float()
    accuracy = correct.sum() / len(correct)
    return accuracy

Explanation:

• train function:
  • Sets the model to training mode (model.train()).
  • Iterates through batches in the data iterator.
  • Clears accumulated gradients (optimizer.zero_grad()).
  • Feeds the input text to the model and obtains predictions.
  • Calculates the loss and accuracy.
  • Performs backpropagation (loss.backward()).
  • Updates model parameters using the optimizer (optimizer.step()).
  • Accumulates epoch-level loss and accuracy.
• evaluate function:
  • Sets the model to evaluation mode (model.eval()).
  • Disables gradient computation (with torch.no_grad():).
  • Iterates through batches in the iterator.
  • Calculates loss and accuracy without performing backpropagation.
  • Accumulates epoch-level loss and accuracy.
• calculate_accuracy function:
  • Finds the predicted class based on the highest probability.
  • Compares predictions with true labels to calculate accuracy.

Important: Before using these functions, you need to define, initialize, and provide your model, optimizer, and loss criterion. These functions will then be used to train and evaluate the model on the SST dataset.