Page Comparison

In computational research, GPU acceleration has become indispensable, especially for handling complex tasks like deep learning. The script we present below provides an illustrative example of how deep learning computations can be offloaded CoESRA’s GPU node gives researcher’s the opportunity to access that computational power remotely.

...

The CoESRA virtual desktop can process deep learning computations assigned to GPUs for faster processing on the CoESRA virtual desktop. This script example first creates a synthetic dataset and , then utilises it to train a Deep Neural Network (DNN), showcasing the . The computational demands and GPU usage typical of real-world applicationsis visually showcased via the nvtop task monitor.

...

This script has been tailored to generate a significant computational load illustrating the CoESRA virtual desktop GPU node’s computation capabilities.

...

This script takes approximately 35min to run in the CoESRA virtual environment’s GPU node using default settings. By comparison, it takes more than 46hrs to run the script using a CPU node with default settings in CoESRA’s virtual environment (see screenshot below).

...

If you would like to trial the CoESRA GPU node, run the below python script in VSCode within the CoESRA virtual desktop.

import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout

# Check the availability of GPU
if tf.config.experimental.list_physical_devices('GPU'):
    print("GPU is available")
else:
    print("GPU not available, using CPU")

# Generate a synthetic dataset for demonstration purposes
X = np.random.rand(300000000, 10)  # Input data with 10 features
y = np.random.rand(300000000, 1)   # Corresponding output data

# Initialize a Sequential DNN model
model = Sequential()

# Input layer: this layer has 20 neurons (units of computation) 
# and receives input data with 10 features
model.add(Dense(20, input_dim=10, activation='relu'))  

# Hidden layers: layers that process the data between input and output
# Each of these layers has 4096 neurons
model.add(Dense(4096, activation='relu')) 
model.add(Dropout(0.5))  # Introducing some randomness by "dropping out" some neurons to prevent overfitting
model.add(Dense(4096, activation='relu')) 
model.add(Dense(4096, activation='relu'))
model.add(Dense(4096, activation='relu'))

# Output layer: produces the final prediction. Only 1 neuron as our synthetic 
# data has a single output value per sample
model.add(Dense(1, activation='linear'))

# Set up the model for training. Here:
# - 'mean_squared_error' is the criteria to measure how well the model is performing (lower is better).
# - 'adam' is a type of optimisation algorithm that adjusts neuron weights to improve the model's predictions.
model.compile(loss='mean_squared_error', optimizer='adam')

# Train the model using the synthetic data. 
# The model will process the data in batches of 1024 samples at a time, for 1 full cycle (epoch).
model.fit(X, y, epochs=1, batch_size=1024)

print("Training complete!")

...

Computational Insights and Real-World Relevance:

1. Data Size:
   The synthetic dataset (X = np.random.rand(300000000, 10)) contains 300 million samples. This magnitude is similar to those encountered when analysing satellite images, where each pixel might represent data points to process and study.

2. Model Complexity:
   The DNN model (model.add(Dense(4096, activation='relu'))) is intricate, with multiple layers and a large number of processing units (neurons). Such model designs are parallel to those employed in medical research, particularly in diagnostic imaging like MRI, where high-resolution images demand detailed neural networks to detect abnormalities.

3. Model Reliability:
   The Dropout step (model.add(Dropout(0.5))) is an approach to enhance the model's robustness, a technique often applied in financial modelling. When predicting stock market movements, for instance, it's essential to ensure the model doesn't over-rely on specific trends.

4. Training Strategy:
   The script employs batch training (model.fit(X, y, epochs=1, batch_size=1024)). This segmented data processing technique can be likened to genomic research. When decoding vast stretches of DNA sequences, data is often broken down into manageable segments, akin to batches, for detailed analysis.

To offer a visual representation of GPU utilisation, please refer to the screenshot below, capturing the real-time metrics using nvtop:

...

Figure 1: GPU utilisation as captured by nvtop during the execution of the script.

Conclusion: The script, tailored for a significant computational load, illustrates the essential role of GPUs in training intricate deep learning models. Through synthetic data and a complex neural network architecture, we've demonstrated the GPU's capability within the CoESRA virtual desktop environment, mirroring real-world computational demands found in cutting-edge research.

Simplified Terminologies:

...

Neurons: Think of neurons as miniature decision-making boxes in the neural network. In the human brain, neurons process the information they receive and pass it on. Similarly, in a Deep Neural Network (DNN), each neuron takes in data, processes it (with some mathematical operations), and then sends this processed data forward. The collective work of all the neurons allows the network to make meaningful predictions or classifications. The depth and breadth of neurons determine the complexity and capacity of the network.

...

Dropout: Imagine you're training a team, and you occasionally ask some members to sit out a practice. This ensures the team doesn’t become overly reliant on any specific member. Dropout in deep learning mirrors this concept. It temporarily "turns off" or ignores certain neurons during training, making the network more robust. This technique helps prevent overfitting, a situation where the model becomes so finely tuned to the nuances of the training data that it performs poorly on unfamiliar data.

...