Using graph embeddings to explore deep neural network
architectures
Supervised by Arlindo L. Oliveira and authored by José
CarreiraConvolutional neural networks and vision transformers
represent the state of the art in artificial neural network
(ANN) models for vision problems, such as classification,
segmentation, and object detection. Many different
architectures exist, that exhibit significant variations in
performance, complexity and training cost. Using the
appropriate transformations, it is possible to generate graph
(or hypergraph) representations of deep neural network
architectures, and these representations can be embedded into
appropriate spaces that may be more amenable to performance
quantification. This dissertation will explore the idea that
graph embeddings of deep neural network architectures (and,
possibly, weights) can be used to explore the architecture
space in more effective ways than is possible today.
Requisites: The student should have significant programming
experience, and practical knowledge of machine learning
languages and environments, such as PyTorch or TensorFlow.
Notes: The work will be be developed in cooperation with
research groups from the University of Tokyo and the Hong Kong
Polytechnic, which have significant expertise in the graph
embedding techniques that will be used in this work. The
selected student will have access to the facilities of
INESC-ID and the MLKD group (https://mlkd.idss.inesc-id.pt/),
including computing facilities that include four DELL
PowerEdge C41402 servers, eight NVIDIA 32GB Tesla V100S and
eight NVIDIA 64GB Tesla A100, among other computing
servers.
Deep neural network architectures for dual process
computation
Supervised by Arlindo L. Oliveira and authored by Guilherme
CostaDual process theories have been used to explain the different
modes of behavior of the human brain when processing
information. These theories became popular with the work of
Kaheman, Thinking Fast and Slow, but they are based on decades
of experimental evidence that the human brain works in two
different modes. System 1 processes large amounts of visual
and sensory information, efficiently and unconsciously. For
instance, face and object recognition, speech processing and
many other automatic functions are performed effortlessly by
the human brain, using system 1. Other tasks require conscious
effort, like answering complex riddles, executing non-trivial
arithmetic operations of planning unfamiliar tasks. These
tasks are performed by system 2. Existing systems, like
convolutional neural networks, for vision, or transformers,
for natural language processing, behave very much like system
1 in the human brain: they perform fast, high-throughput,
processing of high-dimensional information, in an unconscious
way. This dissertation will be focused on the design of deep
neural network architectures that can be used to emulate the
dual process computation that characterizes the human brain
and also on the relation of dual process architectures and
consciousness. Requisites: The student should have significant
programming experience, and practical knowledge of machine
learning languages and environments, such as PyTorch or
TensorFlow. He/she should also have interest in developing the
understanding of neuroscience and human psychology. Notes: The
selected student will have access to the facilities of
INESC-ID and the MLKD group (https://mlkd.idss.inesc-id.pt/),
including computing facilities that include four DELL
PowerEdge C41402 servers, eight NVIDIA 32GB Tesla V100S and
eight NVIDIA 64GB Tesla A100, among other computing
servers.
Stenosis detection in coronary X-ray angiographies
Supervised by Arlindo L. Oliveira and Miguel Menezes and
authored by Tomás Nunes
Automatic processing of images from coronary X-ray
angiographies using deep learning techniques has been
explored, and the results show that it is possible to perform
high-quality segmentation of relevant coronary arteries.
Building on top of existing segmentation methods, based on
deep convolutional neural networks, this dissertation will be
focused on the estimation of the value of the instantaneous
wave-free ratio (iFR) and/or the Fractional Flow Reserve (FFR)
index from segmented images. The objective is to develop a
methodology that can estimate the value of the iFR using
non-invasive procedures and that has sufficient sensitivity to
avoid the need for invasive measurement methods, such as the
insertion of a guidewire with a pressure sensor inserted
through a coronary catheter. Estimating the iFR and the FFR
indexes is a difficult task, since imaging data, even after
segmentation, will provide insufficient information, in many
cases. Exploration of the possible tradeoffs between positive
predictive value and recall will play an essential role in the
identification of the best approach. Co-supervisors: Miguel
Nobre Menezes (20%), João Lourenço Silva (40%) Requisites: The
student should have significant programming experience, and
practical knowledge of machine learning languages and
environments, such as PyTorch or TensorFlow. He/she should
also have interest in developing the understanding of medical
image processing and cardiology. Notes: This work will be
developed in cooperation with the school of department of
cardiology of the School of Nedicine of the University of
Lisbon. The selected student will have access to the
facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include four DELL PowerEdge C41402 servers,
eight NVIDIA 32GB Tesla V100S and eight NVIDIA 64GB Tesla
A100, among other computing servers.
Operation log monitoring using machine learningSupervised by Arlindo L. Oliveira and Fernando Silva authored
by José VelezTraditional monitoring techniques may no longer be able to
handle the complexity of modern applications, infrastructures
and environments. These do not make the best use of the
massive amounts of data being generated, thus several alarms
are created that are not necessarily indicative of a new
incident. The main objective of this thesis is to improve the
monitoring and alarm generation by applying different Machine
Learning algorithms and techniques with the rich and vast
amount of data, to accurately detect complex problems even if
they are outside the boundaries of the monitored software,
which is common in modern architectures such as the Micro
Service. The proposed work is framed within a critical IT
application inside an international organization, in order to
provide business and research value by solving a real world
modern problem. The case study in question, consists in
developing a monitoring solution using state of the art
production Machine Learning (ML) algorithms, based on the
modern Artificial Intelligence for IT Operations (AIOps)
Platforms, to detect anomalies and generate reliable alarms
for complex faults in HERMES, a critical application of
EDP.
Deep learning when data is scarceSupervised by Arlindo Manuel Limede de Oliveira and authored
by Ana Pimenta AlvesCurrent deep learning models require enormous amounts of data
to be trained. Recent studies by DeepMind show that even
models like GPT-3, which is trained with 300 billion tokens,
may still be “significantly undertrained”. Simply gathering
more data to keep increasing the models’ performance is not
biologically reasonable (as humans don’t need such quantities
of data to learn), is not possible for some tasks (where
obtaining more data is very expensive) and widens the gap
between the researchers with the most resources and the rest
of the community. There are several approaches that try to
avoid this data requirement: few shot learning,
self-supervision, using pre-trained models, and loss
smoothing. The objective of this dissertation is to compare
these approaches and analyze in particular their relative
performance per dataset size. The selected student will have
access to the facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines. The work
will be developed using the Pytorch or TensorFlow programming
platforms for machine learning and the Observable platform for
data processing. The selected student will work within the
scope of the Magellan project, and have access to the sources
of data and financial resources made available by the project.
Data fusion and object recognition from sensor dataSupervised by Arlindo Manuel Limede de Oliveira and authored
by Francisco HonórioWith the increased opportunities for digitalization, cities
will need to ensure that the conditions of public spaces are
adequate for their functions. The use of sensor data (sound
and images) to provide information about the quality of public
spaces, ensuring safety and accessibility to all users,
represents an important tendency for smart cities. Smart
cities will use models for spaces, trained using data obtained
from sensors and used to provide information about the
characteristics of the spaces. The objective of this
dissertation is to develop algorithms to collect, identify,
and integrate sensor data and to create and train machine
learning models that can process audio and image data to
provide relevant information about public spaces. Models such
as Mask R-CNN, Faster R-CNN, and YOLO will be assessed and
trained using existing image databases, to perform real-time
object detection. Once trained, data from pilot sites will be
used to test the performance of the models. This project will
be developed within the scope of project Magellan, developed
in cooperation with Schréder and other research institutions.
The selected student will have access to the facilities of
INESC-ID and the MLKD group (https://mlkd.idss.inesc-id.pt/),
including computing facilities that include two DELL PowerEdge
C41402 servers and eight NVIDIA 32GB Tesla V100S, among other
machines. The work will be developed using the Pytorch or
TensorFlow programming platforms for machine learing and the
Observable platform for data processing. The selected student
will work within the scope of the Magellan project, and have
access to the sources of data and financial resources made
available by the project.
Using contrastive learning to learn representations from
texts and imagesSupervised by Arlindo Manuel Limede de Oliveira and authored
by Pedro HenriquesConvolutional neural networks and transformer based
architectures have shown the ability to perform complex
classification and inference tasks, for images and texts.
Still, most of the existing systems rely on the use of massive
datasets of annotated data, such as ImageNet. This restricts
the applicability of the technology to areas where such
massive datasets exist, or imposes large labeling costs. Joint
learning from texts and images. Recently, an approach based on
contrastive language-image pre-training (CLIP) has
demonstrated the ability to learn from unlabeled data, and to
generate systems that are competitive with those trained on
labeled data. The objective of this dissertation is to apply a
CLIP-like approach to data available in Portuguese media, and
to assess the quality of the derived system in a set of tasks,
such as image classification, image captioning, and multimodal
question answering. The selected student will have access to
the facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines. The work
will be developed using the Pytorch or TensorFlow
programming.
Finding interesting regions in whole slide images using deep
learningSupervised by: Arlindo Manuel Limede de Oliveira and Jonas
Almeida. Developed in cooperation with the Data Science &
Engineering Research Group of the Nacional Cancer Institute.
Authored by: Martim AfonsoDigital pathology, the analysis of visual information
generated from digitized specimen slides, is a rapidly growing
are. Whole-Slide Imaging (WSI) enables medical samples to be
processed and used in diagnostic medicine, leading to more
efficient and scalable processes made possible by deep
learning techniques. Whole-slide, very high-resolution,
images, need to be handled using special techniques that
enable specialists to rapidly focus on the Regions of Interest
(RoI) and identify relevant features for the task at hand. The
efficient manipulation of WSIs, in a clinical setting,
requires efficient system-level protocols and effective
feature detection algorithm that, combined, save time and
increase the productivity of physicians by automating the
triage of WSI for RoIs. The objective of this dissertation is
to develop effective deep learning techniques for the
identification of the regions of interest in whole-slide
images, and to integrate the resulting systems with
client-based viewing software developed by NIH researchers.
The system integration may or may not be included in the
dissertation work, depending on the results obtained during
the RoI identification phase. The selected student will have
access to the facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines. The work
will be developed using the Pytorch or TensorFlow programming
platforms for machine learing and the Observable platform for
data processing.
Deep learning on chaos game representation of genetic
sequencesSupervised by: Arlindo Manuel Limede de Oliveira and Susana
Vinga. Developed in cooperation with the Data Science &
Engineering Research Group of the Nacional Cancer Institute,
headed by Jonas Almeida. Authored by: Vincente SilvestreDeep learning (DL) has been applied with success to areas as
diverse as computer vision, natural language processing and
protein folding. The ability of deep learning architectures to
derive the appropriate features for classification and
inference enabled these systems to reach unparalleled
performance. However the successful application of deep
learning depends on the existence of an appropriate canonical
representation with built-in structure, in one, two or more
dimensions. The objective of this dissertation is to study the
application of deep learning techniques to genomic data, using
chaos game representation (CGR), an iterated function that
generates bijective maps between symbolic sequences and
cartesian spaces.. The dissertation will study the application
of standard deep learning architectures, such as ResNet of
EfficientNet to the inference of genotype-phenotype
correlation from the chaos game representation of genetic
sequences and mutation signatures. Genetic sequence data for
specific conditions and related information will be selected
from the Cancer Genome Atlas (TCGA), Polygenic Score Catalog
(PGS) databases and Genome-Wide Studies at NCI, and used to
test the DL+CGR approach. The selected student will have
access to the facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines. The work
will be developed using the Pytorch or TensorFlow programming
platforms.
Using self-supervised contrastive learning to improve medical
image analysisSupervised by Arlindo Manuel Limede de Oliveira and authored
by Miguel Rasquinho FerreiraDeep learning architectures, which include convolutional
neural networks and vision transformers, have made it possible
to achieve human-like performance in several medical image
analysis tasks. However, some fields, including medical image
analysis, are limited by the lack of labeled data.
Furthermore, the use of extensive amounts of labeled data to
train deep neural network architectures leads to behaviors and
peculiar characteristics of the classifiers that do not have
parallel in human vision. Self-supervised contrastive learning
is a technique that can be effectively used to train systems
when labeled data does not exist or is sparse. Furthermore,
systems trained using self-supervised contrastive learning
have the potential to exhibit behaviors that are more similar
to the behavior of the primate’s vision system. The objective
of this dissertation is to apply self-supervised contrastive
learning techniques to problems in medical image, namely the
detection of stroke in computed-tomography images of human
brains. The results will be assessed both in term of the
performance attained and the similarity of the features
derived to features that are present in the visual system of
primate brains. The selected student will have access to the
facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines. The work
will be developed using the Pytorch or TensorFlow programming
platforms.
Analysis of visual sensor data for monitoring of open spaces
A Modular Architecture for Model-Based Deep Reinforcement
LearningSupervised by Arlindo L. Oliveira and authored by João
NovoThe objective of this dissertation is to process data
received from distributed sensor arrays in order to infer the
level and characteristics of space usage in urban
environments. The final objective is to derive detailed person
and vehicle data from data obtained by light and sound
sensors. The student will be integrated in a team developing a
large scale project managed by Schréder Hyperion. Project
Magellan – Localizable, interoperable, cyber-safe, resilient,
distributed autonomous and connected urban infrastructure -
has as its objective the development of a new paradigm of
urban infrastructure, resilient, robust, interconnected, open
and interoperable that will support future smart cities.
Knowledge of machine learning, analytics and programming.
Interest in data processing and interest in learning data
analysis methods. The selected student will have access to the
facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include two DELL PowerEdge C41402 servers and
eight NVIDIA 32GB Tesla V100S, among other machines.