Using sequences of coronary angiograms to quantify the
severity of stenosis
Supervised by Arlindo L. Oliveira and authored by Mariana
Serrão
Automatic processing of images from coronary X-ray
angiographies using deep learning techniques has been
explored, but the ability to estimate accurately physiological
indexes such as the instantaneous wave-free ratio (iFR) and/or
the Fractional Flow Reserve (FFR) has not yet been
demonstrated.. The objective of this dissertation is to
develop a methodology that can estimate the value of the iFR
from sequences of angiographies that has sufficient precision
to avoid the need for invasive measurement methods, such as
the commonly used insertion of a guidewire with a pressure
sensor inserted through a coronary catheter. The approach that
will be used in the application of deep learning techniques to
frame sequences, using additional information from the
sequence that cannot be obtained from single frame analysis.
Biologically inspired CNNs for Medical Imaging tasks
Supervised by Arlindo L. Oliveira and Tiago Marques and
authored by Daniela Carvalho
Medical image data poses several challenges for computer
vision algorithms: it spans multiple imaging modalities and
biological tissues, it contains several sources of noise and
variation, and there is a scarcity of available labeled
datasets. Some recent advances in computer vision models, such
as the use of vision transformers and self-supervised learning
have showed promising results in dealing with some of these
challenges. However, it has not been tested whether the use of
biologically inspired computations, another recent advanced in
computer vision with considerable improvements in robustness,
also translates to gains in medical imaging tasks. The goal of
this project is to adapt the VOneNet family, a hybrid CNN with
a front-end inspired and constrained by the primate primary
visual cortex (V1), to multiple computer vision neural network
architectures used for medical imaging tasks and to test their
performance in a wide range of related benchmarks.
Using Large Language Models do solve the Abstraction and
Reasoning Challenge
Supervised by Arlindo L. Oliveira and authored by Guilherme
Costa
Current deep learning models, while adept at specific tasks,
often struggle with human-like adaptability to new and varied
challenges. This research delves into the creation of
artificial intelligence systems that can mimic the
generalization capabilities of human intelligence,
particularly through the use of the Abstraction and Reasoning
Corpus (ARC). ARC is a compilation of reasoning tasks that are
deeply rooted in Knowledge Priors, which are essential human
skills for effective problem-solving, such as counting. The
proposed solution involves integrating a Large Language Model
(LLM) with several DreamCoders, forming a Mixture of Experts
(MoE) framework. In this framework, the LLM acts as a
classifier, pinpointing the specific skills required for each
ARC task. Following this identification, the problem is
delegated to a specialized DreamCoder, each trained solely to
tackle tasks within the identified skill set.
Representation learning of animal behavior
Supervised by Arlindo L. Oliveira and Adrien Jouary and
authored by Gonçalo Goulart Oliveira
Over the past decade, several methods have been developed that
allow high-throughput automated quantification of animal
behavior. Advances in computer vision make it possible to
automatically track multiple body points. And continuous
movements can be decomposed into a sequence of meaningful
elementary units. In this project, we aim to build a latent
variable model of a large dataset of zebrafish larva behavior.
The behavior of each larva consists of a sequence of
stereotypical tail movements. The model will be trained to
perform prediction of future action. Once the model is trained
we will explore transfer learning by using the representation
from the model to detect the effect of drug treatment. For
this, we will use a dataset of the larva behavior in response
to 10 pharmacological compounds at different concentrations.
Our goal is to learn the internal state of the animal using
this approach, which could be useful for studying the brain
and improving the detection of drug-induced behavioral
changes. Our approach holds promise for neuroscience and
preclinical research, as careful measurements of animal
behavior have proven to be an important complement to modern
techniques for recording and manipulating neural circuits.
Marques, J.C., Lackner, S., Félix, R. and Orger, M.B., 2018.
Structure of the zebrafish locomotor repertoire revealed with
unsupervised behavioral clustering. Current Biology, 28(2),
pp.181-195. // Wiltschko, A.B., Tsukahara, T., Zeine, A.,
Anyoha, R., Gillis, W.F., Markowitz, J.E., Peterson, R.E.,
Katon, J., Johnson, M.J. and Datta, S.R., 2020. Revealing the
structure of pharmacobehavioral space through motion
sequencing. Nature neuroscience, 23(11), pp.1433-1443. //
Oord, A.V.D., Li, Y. and Vinyals, O., 2018. Representation
learning with contrastive predictive coding. arXiv preprint
arXiv:1807.03748.
Using biological features to improve deep neural network
models for vision
Supervised by Arlindo L. Oliveira and Tiago Marques and
authored by Lucas Alergy
Convolutional neural networks and vision transformers
represent the state of the art in artificial neural network
(ANN) models for vision problems, such as classification,
segmentation, and object detection. However, the performance
of these models still falls behind human performance in many
problems and is highly susceptible to image variation,
lighting conditions, and deliberate attacks. Recent results
have shown that it is possible to draw inspiration from the
architecture and function of the visual cortex to improve the
performance of ANNs and to make these systems more robust to a
wide range of image perturbations. The objective of this
dissertation is to study how structural and functional
characteristics of the primate visual pathways can be used to
derive new layers and optimization goals in deep neural
networks that contribute to improving their robustness and
performance in image classification tasks. Efficient coding
algorithms, used in the retina and the primary visual cortex,
and different connection patterns between layers are some of
the approaches that will be tested. The novel models will be
assessed both in terms of their performance in existing
computer vision benchmarks and on how well their internal
components and behavioral output match those of real primate
brains using the Brain-Score platform. The work will be
co-supervised by Tiago Marques, currently at the Champalimaud
Foundation. The selected student will have access to the
facilities of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include four DELL PowerEdge C41402 servers,
eight NVIDIA 32GB Tesla V100S and eight NVIDIA 64GB Tesla
A100, among other computing servers
(https://mlkd.idss.inesc-id.pt/cluster)
Modelos de causalidade para determinação do impacto de acções
comerciais
Supervised by Arlindo L. Oliveira and Filipa Marques and
authored by Miguel Vicente
Nesta tese, irão ser desenvolvidos modelos de causalidade para
quantificar o impacto comercial das diferentes acções que
resultam do processo de geração de leads pelos modelos
analíticos. O objectivo é quantificar o impacto dos modelos de
análise de dados no negócio, determinando quais acções
específicas tiveram impacto no resultado final, usando para
tal modelos de causalidade. Serão usados dados reais de
cliente e de histórico de vendas da Fidelidade e analisadas as
consequências da diferentes acções de geração de leads a
partir da analítica. Serão também desenvolvidas métricas para
avaliar o impacto dos modelos de scores no resultado final do
negócio. Requisitos: o candidato deverá ter conhecimentos e
interesse em análise de dados, aprendizagem automática e
mecanismos de causalidade. A frequência de disciplinas destas
áreas é recomendada. Notas: Esta tese será desenvolvida em
parceria com a Fidelidade e co-orientada pela Dra. Filipa
Marques.
Using large language models to interact with personal
information systems
Supervised by Arlindo L. Oliveira and authored by João Amoroso
Large language models, such as ChatGPT and GPT-4 have shown
remarkable abilities to interact in natural language. However,
they cannot be used to access and learn from personal data,
stored in email records, note taking systems or photos and
videos. The objective of this dissertation is to design a
system that uses large language model as the interface for
personal data, using APIs and enabling the user to query,
relate and retrieve information stored in different
sub-systems, such as mailboxes, Google records and note taking
platforms such as Obsidian. The resulting system should be
able to emulate the behavior of an intelligent assistant that
has access to all stored personal data and, ultimately, to
answer questions about that data in a way similar to the user
that owns the data. Requisites: The student should have
significant programming experience, and practical knowledge of
machine learning languages and environments, such as PyTorch
or TensorFlow. He/she should also have interest in developing
the understanding of large language models and LLM APIs.
Notes: The selected student will have access to the facilities
of INESC-ID and the MLKD group
(https://mlkd.idss.inesc-id.pt/), including computing
facilities that include four DELL PowerEdge C41402 servers,
eight NVIDIA 32GB Tesla V100S and eight NVIDIA 64GB Tesla
A100, among other computing servers
(https://mlkd.idss.inesc-id.pt/cluster)
Accurate prediction of stroke outcome from computed tomography
scans
Supervised by Arlindo L. Oliveira and authored by João
Teixeira
Accurately predicting the functional outcome of stroke
patients remains a problem with medical relevance that cannot
yet be adequately solved by automated means. Although it is
known that brain computed tomography (CT) scans contain
relevant information, their practical usefulness in predicting
this variable remains an open question. The objective of this
thesis is to develop algorithms to determine whether brain CT
scans (with and without contrast) could be automatically
analysed using deep learning models, to improve the prediction
of the three months post stroke functional outcome, as
measured by the modified ranking score. The selected student
will study the application of deep learning architectures,
including convolutional neural networks and vision
transformers, to the this problem. One intermediate variable
that will be studied as a possible predictor for the
functional outcome is occlusion, the existence of blocked
arteries that lead to the death of brain regions. Requisites:
The student should have significant programming experience,
and practical knowledge of machine learning languages and
environments, such as PyTorch or TensorFlow. The student
should have an interest in becoming familiar with the
biological and medical phenomena involved in stroke. Notes:
This work will be developed in close cooperation with the
neurology department of the Santa Maria Hospital. The selected
student will have access to the facilities of INESC-ID and the
MLKD group (https://mlkd.idss.inesc-id.pt/), including
computing facilities that include four DELL PowerEdge C41402
servers, eight NVIDIA 32GB Tesla V100S and eight NVIDIA 64GB
Tesla A100, among other computing servers.