Abstract: Deep learning in general, and Large Language Models specifically, have extreme difficulties in reliably solving hard reasoning problems, even when provided with millions of solved instances for training. Even then, introducing side-constraints is usually difficult, when possible. In this talk, I will introduce a (deep) neuro-symbolic architecture that we initially designed to efficiently learn how to play one-player decision NP-complete games that reside in 2D or 3D space such as Sudoku, Futoshiki or, more seriously, computational protein design. Thanks to a dedicated loss function, this architecture is able to efficiently end-to-end learn how to predict an explicit probabilistic (or deterministic) model, conditioned by the provided 2D/3D information. This explicit representation can a posteriori be sampled, optimized or constrained, without retraining but with a possibly high inference compute cost, related to the NP-complete nature of the underlying problems. 

Thanks to the extreme flexibility of deep learning, this architecture can learn from arbitrary raw data and therefore also tackle « Decision-focused learning » (DFL) problems. At the cost of some scalability, it can even learn from partially observed solutions. On logical games, DFL graph optimization problems, and on more challenging protein design problems, we observe that this architecture is able to produce solutions for instances where pure Deep learning auto-regressive models struggle. Applied to hard protein design instances, a famous autoregressive model eventually scored its solutions better than its own. This difference translates in practice, being confirmed by experimental characterization of synthesized designed proteins.

Bio: Thomas Schiex is a Director of Research at INRAE (France), affiliated with the MathNum (Mathematics and Digital Sciences) department. His research focuses on artificial intelligence, automated reasoning, constraint optimization, and deep learning, with a heavy emphasis on computational biology and structural protein design. He is a pioneer in Cost Function Networks and one of the creators of the toulbar2 exact discrete graphical model optimization solver. For his seminal contributions to the field, he was elected a Fellow of the European Association for Artificial Intelligence (EurAI) in 2016 and a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI) in 2020.

Abstract: TBA

Bio: TBA