Learning when to stop: in life and when training deep networks presented at CYSEC 2020

by Ileana Buhan,

URL : https://www.youtube.com/watch?v=8DGRTlzKiM0

Summary : Today, deep neural networks represent a common option when conducting the profiled side-channel analysis. Such techniques commonly do not require pre-processing, and yet, they can break targets that are even protected with countermeasures. Unfortunately, it is usually far from trivial to find neural network hyper-parameters that would result in such top-performing attacks. The hyper-parameter leading the training process is the number of epochs during which the training happens. If the training is too short, the network does not reach its full capacity, while if the training is too long, the network over fits, and consequently, is not able to generalize to unseen examples. Finding the right moment to stop the training process is particularly difficult for side-channel analysis as there are no clear connections between machine learning and side-channel metrics that govern the training and attack phases, respectively. In this work, we tackle the problem of determining the correct epoch to stop the training in deep learning-based side-channel analysis. First, we explore how information is propagated through the hidden layers of a neural network, which allows us to monitor how training is evolving. Second, we demonstrate that the amount of information transferred to the output layer can be measured and used as a reference metric to determine the epoch at which the network offers optimal generalization. To validate the proposed methodology, we provide extensive experimental results that confirm the effectiveness of our metric of choice for avoiding overfitting in the profiled side-channel analysis.On a personal note, I will share the three most important lessons that helped me evolve as a person and in my career.