Advanced Optimization Algorithms Research
157 papers on this topic
Delving into Muon and Beyond: Deep Analysis and Extensions
The Muon optimizer has recently attracted considerable attention for its strong empirical performance and use of orthogonalized updates on matrix-shaped parameters, yet its underlying mechanisms...
Optimization Learning
This article introduces the concept of optimization learning, a methodology to design optimization proxies that learn the input/output mapping of parametric optimization problems. These...
LLM-Guided Evolution of Symbolic Acceleration Inequality Families for MILP
Integer programming (IP) is central to many combinatorial optimization tasks but remains challenging due to its NP-hard nature. A practical way to improve IP solvers is to manually design...
Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants
Recent advancements in deep learning optimization have introduced new algorithms, such as Schedule-Free optimizers, AdEMAMix, MARS and Lion which modify traditional momentum mechanisms. In a...
Towards graph neural networks for provably solving convex optimization problems
Recently, message-passing graph neural networks (MPNNs) have shown potential for solving combinatorial and continuous optimization problems due to their ability to capture variable-constraint...
Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programming
Cutting planes (cuts) play an important role in solving mixed-integer linear programs (MILPs), as they significantly tighten the dual bounds and improve the solving performance. A key problem for...
DualSchool: How Reliable are LLMs for Optimization Education?
Consider the following task taught in introductory optimization courses which addresses challenges articulated by the community at the intersection of (generative) AI and OR: generate the dual of...
Solving Mixed Integer Programs Using Neural Networks
Mixed Integer Programming (MIP) solvers rely on an array of sophisticated heuristics developed with decades of research to solve large-scale MIP instances encountered in practice. Machine learning...
Learning to Compare Nodes in Branch and Bound with Graph Neural Networks
Branch-and-bound approaches in integer programming require ordering portions of the space to explore next, a problem known as node comparison. We propose a new siamese graph neural network model...
On the Convergence of Adam and Beyond
Several recently proposed stochastic optimization methods that have been successfully used in training deep networks such as RMSProp, Adam, Adadelta, Nadam are based on using gradient updates...
Neural Horizon Model Predictive Control -- Increasing Computational Efficiency with Neural Networks
The expansion in automation of increasingly fast applications and low-power edge devices poses a particular challenge for optimization based control algorithms, like model predictive control. Our...
An improved column-generation-based matheuristic for learning classification trees
Decision trees are highly interpretable models for solving classification problems in machine learning (ML). The standard ML algorithms for training decision trees are fast but generate suboptimal...
Combining Lipschitz and RBF Surrogate Models for High-dimensional Computationally Expensive Problems
Standard evolutionary optimization algorithms assume that the evaluation of the objective and constraint functions is straightforward and computationally cheap. However, in many real-world...
Exact Combinatorial Optimization with Temporo-Attentional Graph Neural Networks
Combinatorial optimization finds an optimal solution within a discrete set of variables and constraints. The field has seen tremendous progress both in research and industry. With the success of...
A Survey of Optimization Methods from a Machine Learning Perspective
Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much...
An Enhanced Grey Wolf Optimizer with Elite Inheritance and Balance Search Mechanisms
The Grey Wolf Optimizer (GWO) is recognized as a novel meta-heuristic algorithm inspired by the social leadership hierarchy and hunting mechanism of grey wolves. It is well-known for its simple...
Numerical Methods for Convex Multistage Stochastic Optimization
Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP)....
Structured Pruning of Neural Networks for Constraints Learning
In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity across diverse applications, including cancer treatment, algorithmic...