Poster
in
Workshop: Methods and Opportunities at Small Scale (MOSS)
Geometry of Rank Constraints in Shallow Polynomial Neural Networks
Param Mody · Maksym Zubkov
Keywords: [ optimisation ] [ algebraic geometry ] [ machine learning theory ]
We study shallow quadratic and cubic polynomial neural networks of width 2. In this setting, the ambient space is the space of symmetric polynomials, which is finite-dimensional. We consider four target functions that correspond to rank-2 and rank-3 symmetric matrices, and rank-2 and rank-3 symmetric tensors. We compare the learning dynamics when the target function lies within versus outside the function space (neuromanifold), and we analyze the patterns of critical points in both the parameter space and the corresponding functional space.