This is a "Giving Back" repository where NeSy models are implemented from scratch in notebook style as a tutorial.
Toy Problem: MNIST Addition (two or multi-digit)
The following benchmarks are typically used in the literature but are not considered for evaluation in this repository, as our focus is on learning architectures.
- Sudoko Solving
- Visual Sudoko
- Hand-Written Formula
- Leaf and Scene Identification
CTSketch, 2025
Data-Efficient Learning with Neural Programs, NeurIPS, 2024
What Planning Problems Can A Relational Neural Network Solve?
Differentiable Tree Machine, ICML, 2023
Learning Modulo Theories, 2023
Logic Tensor Network, Elsevier AI, 2022
DreamCoder, ACM-SIGPLAN, 2021
(LTN, DeepProbLog, Scallop, PyReason, NeurASP, ProbLog, Answer Set solvers) and graph stacks (PyG, DGL, Neo4j).
- NeuroSymbolic AI (NSAI) aims to combine symbolic reasoning with deep learning to improve trust, interpretability, and robustness, moving beyond the limitations of purely sub-symbolic models.
- Key challenges include variable grounding, commonsense reasoning, and extracting first-order logic (FOL) from neural networks—difficult due to universal quantifiers violating the universal approximation theorem in bounded continuous spaces.
- Logic Tensor Networks (LTNs) provide a framework where constants and variables are embedded as latent vectors and predicates are approximated via neural networks, enabling differentiable reasoning over FOL-like structures.
- NSAI supports explainability, expert knowledge integration, and bridges statistical learning with knowledge representation and reasoning (KRR).
- Localist vs Distributed representations: Localist assigns discrete symbols to concepts (e.g., one-hot), while distributed representations encode concepts across many parameters, aiding generalization but limiting interpretability.
- Symbolic knowledge can be encoded in networks via initialization (compilers), loss constraints, or factor graph conversions, making reasoning differentiable and data-efficient.
- NSAI design combines gradient descent, modularity, symbolic languages, and constraint satisfaction; open problems include FOL/higher-order logic extraction, goal-directed reasoning, and incorporating human-in-the-loop for validating learned representations.
- A key challenge is how distributed representations in neural networks can support structured reasoning and symbolic manipulation.
- Reasoning and planning typically require symbolic abstractions, but learning such symbols from perceptual stimuli (e.g., images, audio) is non-trivial. Techniques like VQ-VAE, discrete probabilistic latent spaces, and deep hashing attempt to learn discrete representations for planning, but often fail to ensure non-ambiguity and composability.
- End-to-end learning of symbolic representations introduces additional difficulties for gradient-based optimization, including issues with gradient discontinuity, temperature annealing, and straight-through estimators (STE).
- Desired properties of symbolic representations include: non-ambiguity, purity, usefulness, symbol composability, manipulation composability, and generalization.
- Challenges:
- (a) Gradient-based optimization: Approaches such as STE, annealing schedules, and perturbation-based finite differences are used, but remain unstable or brittle.
- (b) Loss function choice: Fuzzy framework etc.
- (c) Supervision paradigms: Vary from fully supervised to unsupervised, including weak or partial signals (e.g., monotonicity constraints, ontologies, or class equivalence).
- (d) Knowledge injection strategies: Include constraints on representations, downstream task constraints (e.g., LatPlan), architectural constraints, semantic regularization, symbolic conditioning, and integration of unstructured knowledge.