29set2026
09:00 Doctoral defense Room 85 of IC2
Topic on
Representation Learning of Visual Features Through Self-Supervision
Student
Thalles Santos Silva
Advisor / Teacher
Hélio Pedrini - Co-advisor: Gerberth Adín Ramírez Rivera
Brief summary
The paradigm shift from supervised to self-supervised learning represents a fundamental milestone in the field of machine learning. This new approach has enabled the development of scalable foundation models that learn directly from data, without the need for human-generated labels. In the field of Computer Vision, although current self-supervision models compete directly with supervised models, their architectures still carry implicit biases that limit their adoption in cutting-edge applications. To overcome these limitations, this thesis presents a progression of solutions that rethink how visual features are learned, extracted, stored, and organized. Firstly, to mitigate spatial decimation caused by convolutional architectures, we introduced the Contextualized Local Visual Embeddings (CLoVE) method, which adapts the multi-headed self-attention mechanism and avoids pooling operations that aggregate local vectors into a single global vector, preserving the informational granularity necessary for dense prediction tasks. Next, to address the optimization instabilities in prototype-based clustering methods, we propose Consistent Assignment of Views over Random Partitions (CARP), a divide-and-conquer algorithm that regularizes training through stochasticity, ensuring stable end-to-end learning without relying on complex heuristics. Motivated by the computational and optimization costs of maintaining prototypes, we present a non-parametric alternative to self-supervised clustering methods. We propose Memory Augmented Self-Supervised Learning (MaSSL), which replaces rigid prototypes with a memory module, allowing the neural network model to learn directly from past experiences by contrasting current signals with previously experienced concepts. Based on these same principles, we formulated the Self-Organizing Prototypes (SOP) to solve the overgrouping problems prevalent in standard methods. Unlike current approaches, SOP delegates assignment responsibilities to multiple in-memory representations. Finally, to address the representational inefficiencies of passive First-In-First-Out (FIFO) queues that feed existing non-parametric memory systems, we introduced Complexity-Adaptive Memory (CAM), which implements a Refine-or-Insert dynamic routing policy capable of autonomously compressing redundant features and reallocating memory capacity based on the implicit difficulty of each concept. Through extensive qualitative and quantitative evaluations, we demonstrate that the proposed methods overcome the identified limitations, consistently improving representational power compared to the state of the art. In summary, the solutions presented here provide a robust and scalable foundation for the next generation of self-supervised visual methods.
Examination Board
Headlines:
| Hélio Pedrini | IC / UNICAMP |
| Alceu de Souza Britto Junior | CCET/PUCPR |
| Ronaldo Cristiano Prati | CMCC / UFABC |
| Fabricio Aparecido Breve | IGCE / UNESP |
| Levy Boccato | FEEC / UNICAMP |
Substitutes:
| Esther Luna Colombini | IC / UNICAMP |
| Moacir Antonelli Ponti | ICMC / USP |
| Edimilson Batista dos Santos | DCOMP / UFSJ |