Skip to content

README.md

Latest commit

 

History

History
167 lines (120 loc) · 4.04 KB

README.md

File metadata and controls

167 lines (120 loc) · 4.04 KB

Hyperparameter Tuning Script Documentation

This README provides an overview of the hyperparameter tuning script.

Table of Contents

  1. Prerequisites
  2. Script Overview
  3. Setup and Usage
  4. Configuration Details
  5. Output and Results
  6. Edits and Customizations
  7. FAQs

Prerequisites

  • Python 3.8 or later
  • Dependencies can be installed using the env.yaml file:
conda env create -f env.yaml
conda activate rllib_env_2.2.0

Script Overview

The script performs hyperparameter tuning on a Multi-Layer Perceptron (MLP) model. Key functionalities include:

  • Loading datasets from CSV files.
  • Defining and training an MLP model.
  • Conducting hyperparameter tuning with Ray Tune using the ASHAScheduler for early stopping.
  • Logging and saving the best model and hyperparameters.

Setup and Usage

Clone the Repository

Ensure the script and required files are in the same directory.

git clone https://github.com/IITH-Compilers/IR2Vec-Classification.git
cd IR2Vec-Classification

Dataset Preparation

Prepare the training, validation, and test datasets. Set the paths in python code:

python hyperparameter_tuning.py

Run the Script

Execute the script:

python hyperparameter_tuning.py

Configuration Details

Hyperparameter Search Space

The script explores the following hyperparameter configurations:

  • Number of Layers (num_layers): Random integer between 3 and 8.
  • Units Per Layer (units_per_layer): Random choice from [64, 128, 256, 512] for each layer.
  • Dropout (dropout): Uniformly sampled between 0.0 and 0.3.
  • Normalize Input (normalize_input): Boolean (True or False).
  • Activation Function (activation): Choice of ReLU, LeakyReLU, Tanh, or SiLU.
  • Optimizer (optimizer): Currently set to Adam.
  • Learning Rate (lr): Log-uniform sampling between 1e-4 and 1e-2.
  • Batch Size (batch_size): Choice of [32, 64, 128, 256, 512, 1024].
  • Epochs (epochs): Set to 5000.

Scheduler

The ASHAScheduler is used for early stopping based on validation accuracy.

Resource Allocation

  • CPU: 10 cores per trial
  • GPU: 0.125 per trial

Output and Results

  • Checkpoints: Saved periodically during training in the directory: 
    /Pramana/IR2Vec/tuned_models_ir2vec/tmp/ray_results
  • Best Model and Hyperparameters: The best model and hyperparameters are logged and saved in JSON format.

Example output:

{
  "best_config": {
    "num_layers": 5,
    "units_per_layer": [256, 128, 128, 64, 64],
    "dropout": 0.1,
    "normalize_input": true,
    "activation": "ReLU",
    "optimizer": "Adam",
    "lr": 0.001,
    "batch_size": 128,
    "epochs": 5000
  },
  "best_results": {
    "val_accuracy": 0.85,
    "train_accuracy": 0.88
  }
}

Edits and Customizations

Paths to Datasets

Update paths to your dataset files:

train_dataset_path = "/path/to/training.csv"
val_dataset_path = "/path/to/val.csv"
test_dataset_path = "/path/to/testing.csv"

Input and Output Dimensions

Update the input_dim and num_classes to match your dataset:

input_dim = 56  # Number of features
num_classes = 98  # Number of classes

Logging and Temporary Directory

Modify logging and temporary directory settings if needed:

ray.init(_temp_dir="/custom/path/to/tmp")

FAQs

  1. How do I use a specific GPU for training? Set the CUDA visibility environment variable to the index of the GPU you want to use:

    CUDA_VISIBLE_DEVICES=0 python hyperparameter_tuning.py

  2. What should I do if I encounter CUDA/Torch errors? Check the compatibility of your Torch and CUDA versions and adjust the Torch version accordingly.

  3. How do I manage logs and outputs for different runs? Change the log and CSV file names before running the script to match your configuration.