In computational research, GPU acceleration has become indispensable, especially for handling complex tasks like deep learning. The script we present below provides an illustrative example of how deep learning computations can be offloaded to GPUs for faster processing on the CoESRA virtual desktop. This script creates a synthetic dataset and utilises it to train a Deep Neural Network (DNN), showcasing the computational demands and GPU usage typical of real-world applications.
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout
# Check the availability of GPU
if tf.config.experimental.list_physical_devices('GPU'):
print("GPU is available")
else:
print("GPU not available, using CPU")
# Generate a synthetic dataset for demonstration purposes
X = np.random.rand(300000000, 10) # Input data with 10 features
y = np.random.rand(300000000, 1) # Corresponding output data
# Initialize a Sequential DNN model
model = Sequential()
# Input layer: this layer has 20 neurons (units of computation)
# and receives input data with 10 features
model.add(Dense(20, input_dim=10, activation='relu'))
# Hidden layers: layers that process the data between input and output
# Each of these layers has 4096 neurons
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5)) # Introducing some randomness by "dropping out" some neurons to prevent overfitting
model.add(Dense(4096, activation='relu'))
model.add(Dense(4096, activation='relu'))
model.add(Dense(4096, activation='relu'))
# Output layer: produces the final prediction. Only 1 neuron as our synthetic
# data has a single output value per sample
model.add(Dense(1, activation='linear'))
# Set up the model for training. Here:
# - 'mean_squared_error' is the criteria to measure how well the model is performing (lower is better).
# - 'adam' is a type of optimisation algorithm that adjusts neuron weights to improve the model's predictions.
model.compile(loss='mean_squared_error', optimizer='adam')
# Train the model using the synthetic data.
# The model will process the data in batches of 1024 samples at a time, for 1 full cycle (epoch).
model.fit(X, y, epochs=1, batch_size=1024)
print("Training complete!")
Computational Insights and Real-World Relevance:
Data Size:
The synthetic dataset (X = np.random.rand(300000000, 10)) contains 300 million samples. This magnitude is similar to those encountered when analysing satellite images, where each pixel might represent data points to process and study.Model Complexity:
The DNN model (model.add(Dense(4096, activation='relu'))) is intricate, with multiple layers and a large number of processing units (neurons). Such model designs are parallel to those employed in medical research, particularly in diagnostic imaging like MRI, where high-resolution images demand detailed neural networks to detect abnormalities.Model Reliability:
The Dropout step (model.add(Dropout(0.5))) is an approach to enhance the model's robustness, a technique often applied in financial modelling. When predicting stock market movements, for instance, it's essential to ensure the model doesn't over-rely on specific trends.Training Strategy:
The script employs batch training (model.fit(X, y, epochs=1, batch_size=1024)). This segmented data processing technique can be likened to genomic research. When decoding vast stretches of DNA sequences, data is often broken down into manageable segments, akin to batches, for detailed analysis.
To offer a visual representation of GPU utilisation, please refer to the screenshot below, capturing the real-time metrics using nvtop:
Figure 1: GPU utilisation as captured by nvtop during the execution of the script.
Conclusion: The script, tailored for a significant computational load, illustrates the essential role of GPUs in training intricate deep learning models. Through synthetic data and a complex neural network architecture, we've demonstrated the GPU's capability within the CoESRA virtual desktop environment, mirroring real-world computational demands found in cutting-edge research.
Simplified Terminologies:
Neurons: Think of neurons as miniature decision-making boxes in the neural network. In the human brain, neurons process the information they receive and pass it on. Similarly, in a Deep Neural Network (DNN), each neuron takes in data, processes it (with some mathematical operations), and then sends this processed data forward. The collective work of all the neurons allows the network to make meaningful predictions or classifications. The depth and breadth of neurons determine the complexity and capacity of the network.
Dropout: Imagine you're training a team, and you occasionally ask some members to sit out a practice. This ensures the team doesn’t become overly reliant on any specific member. Dropout in deep learning mirrors this concept. It temporarily "turns off" or ignores certain neurons during training, making the network more robust. This technique helps prevent overfitting, a situation where the model becomes so finely tuned to the nuances of the training data that it performs poorly on unfamiliar data.
Batch Size: When training a model, it's often impractical or inefficient to feed all the data at once, especially when the dataset is large. Instead, the data is divided into smaller chunks or "batches." Batch size refers to the number of data samples processed in one forward and backward pass of the neural network. Consider it like reading a long book: rather than reading it all at once, you'd read a few chapters (a batch) at a time, gradually completing the entire book.
0 Comments