Awesome
LLM4Opt
Collection on Large Language Models for Optimization (LLM4Opt)
🔥 Applying Large language models (LLMs) for diverse optimization tasks (Opt) is an emerging research area. This is a collection of references and papers of LLM4Opt. The Papers are sorted by time (first publicly available).
Any suggestions and pull requests are welcomed!
It is far from a comprehensive list. If you want to update the list:
- Fork, Add, and Merge
- Report an issue
- Contact Fei Liu (fliu36-c@my.cityu.edu.hk)
The sharing principle of these references here is for research. If any authors do not want their paper to be listed here, please feel free to contact us.
Paper distribution from A Systematic Survey on Large Language Models for Algorithm Design, 2024
Overview
Platform
| Project | Description |
|---|---|
| EoH (Evolution of Heuristics) | optimization, mathematics, machine learning, etc |
| OpenELM | robots, image, programming puzzles, etc |
Course
| Course | Description |
|---|---|
| 2024 Fall, LLM Agents | LLM basics and LLM for agents |
Tutorial&Workshop
| Event | Link |
|---|---|
| NeurIPS 2023 Workshop: Foundation Models for Decision Making | Link |
| AAAI 2024 Workshop: Public Sector LLMs: Algorithmic and Sociotechnical Design | Link |
| GECCO 2024 Workshop: Large Language Models for and with Evolutionary Computation (LLMfwEC) | Link |
| GECCO 2024 Workshop: EGML-EC — 3rd GECCO workshop on Enhancing Generative Machine Learning with Evolutionary Computation (EGML-EC) 2024 | Link |
| GECCO 2024 Tutorial: Using Large Language Models for Evolutionary Search | Tutorial Link, Tutorial Report Link |
| PPSN 2024 Tutorial: Large Language Models as Tools for Metaheuristic Design: Exploring Challenges and Opportunities | Link |
| KDD 2024 Tutorial: NL2Code-Reasoning and Planning with LLM for Code Development | Link, Link |
| ICML 2024 Workshop: AI for Math | Link |
| NeurIPS 2024 Workshop: Multimodal Algorithmic Reasoning (MAR) | Link |
| NeurIPS 2024 Workshop: The 4th Workshop on Mathematical Reasoning and AI | Link |
Competition
| Event | Link |
|---|---|
| AAAI 2024 Global Competition on Math Problem Solving and Reasoning | Link |
| ICML 2024 Challenges on Automated Math Reasoning | Link |
| IEEE CIS 2024 Technical Challenge FLAME – ‘Fusing Large lAnguage Models with computational intElligence’ | Link |
SpecialIssues
| Event | Link |
|---|---|
| IEEE TETCI, Special Issue on Neural Architecture Search and Large Machine Learning Models | Link |
| ACM TELO, Special Issue on Integrating Evolutionary Algorithms and Large Language Models | Link |
| IEEE TEVC, Special Issue on Evolutionary Computation Meets Large Language Models | link |
| Engineering, Special Issue on Applications of ChatGPT | link |
Papers
Review
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap | Arxiv, Jan 2024 | [code] | paper |
| A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond | Arxiv, Mar. 2024 | [code] | paper |
| When Large Language Model Meets Optimization | Arxiv, May 2024 | [code] | paper |
| A Systematic Survey on Large Language Models for Algorithm Design | Arxiv, Oct 2024 | [code] | paper |
Position paper
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| The Era OF Semantic Decoding | Arxiv, Mar. 2024 | [code] | paper |
| Leveraging Foundational Models for Black-Box Optimization: Benefits, Challenges, and Future Directions | ICML 2024, May 2024 | [code] | paper |
Algorithm/Heuristic/Function Search
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Hypothesis Search: Inductive Reasoning with Language Models | Arxiv Sep 2023, ICLR 2024 | [code] | paper |
| ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search | Arxiv Oct 2023, ICLR 2024 | [code] | paper |
| Algorithm Evolution using Large Language Model | Arxiv Nov 2023 | code | paper |
| Mathematical discoveries from program search with large language models | Nature | code | paper |
| Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model | ICML 2024 (Oral) | code | paper |
| ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution | Arxiv Feb 2024, NeurIPS 2024 | code | paper |
| Discovering More Effective Tensor Network Structure Search Algorithms via Large Language Models (LLMs) | [code] | paper | |
| AutoSAT: Automatically Optimize SAT Solvers via Large Language Models | Arxiv Feb 2024 | [code] | paper |
| Large Language Models tO Enhance Bayesian Optimization | Arxiv Feb 2024, ICLR 2024 | [code] | paper |
| On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks | Arxiv Feb 2024 | [code] | paper |
| How Can LLM Guide RL? A Value-Based Approach | Arxiv Feb 2024 | code | paper |
| LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation | Arxiv Mar 2024 | code | paper |
| Evolve Cost-aware Acquisition Functions Using Large Language Models | PPSN 2024 | [code] | paper |
| Benchmarking ChatGPT on Algorithmic Reasoning | Arxiv April 2024 | [code] | paper |
| How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems | Arxiv March 2024 | [code] | paper |
| LLM-ABR: Designing Adaptive Bitrate Algorithms via Large Language Models | Arxiv April 2024 | [code] | paper |
| Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems | Arxiv April 2024 | [code] | paper |
| GLHF: General Learned Evolutionary Algorithm Via Hyper Functions | Arxiv | [code] | paper |
| tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) | ICML 2024 | [code] | paper |
| LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics | Arxiv May 2024 | code | paper |
| Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models | PPSN 2024 | code | paper |
| Discovering Preference Optimization Algorithms with and for Large Language Models | Arxiv June 2024 | code | paper |
| OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code | Arxiv May 2024 | code | paper |
| In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics | Arxiv Oct 2024 | code | paper |
LLM as Optimizer
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Large Language Models as Optimizers | Arxiv, Sep 2023 | code | paper |
| Large language model for multi-objective evolutionary optimization | Arxiv, Oct. 2023 | code | paper |
| Large Language Models as Evolutionary Optimizers | Arxiv, Oct. 2023 | [code] | paper |
| Using Large Language Models for Hyperparameter Optimization | NeurIPS 2023 | [code] | paper |
| Large Language Models As Evolution Strategies | Arxiv, Feb. 2024 | [code] | paper |
| Large Language Model-based Evolutionary Optimizer: Reasoning with Elitism | Arxiv, Mar. 2024 | [code] | paper |
| Large Language Model-Aided Evolutionary Search for Constrained Multiobjective Optimization | Arxiv, May 9, 2024 | [code] | paper |
| Towards Optimizing with Large Language Model | KDD 2024 | [code] | paper |
LLM as/for surrogate model
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study | Arxiv, June 2024 | code | paper |
| Large Language Model-assisted Surrogate Modelling for Engineering Optimization | CAI, 2024 | [code] | paper |
| LLM Performance Predictors are good initializers for Architecture Search | Arxiv, Aug 2024 | [code] | paper |
Code Generation
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| L2MAC: LARGE LANGUAGE MODEL AUTOMATIC COMPUTER FOR EXTENSIVE CODE GENERATION | Arxiv Oct 2023, ICLR 2024 | [code] | paper |
Prompt Opt
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers | Arxiv Sep 2023, ICLR 2024 | [code] | paper |
| PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization | Arxiv Oct 2023, ICLR 2024 | [code] | paper |
Machine Learning
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Evolution through Large Models | Arxiv, June 2022 | code | paper |
| Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph | Arxiv, July 2023, ICLR 2024 | [code] | paper |
| Evoprompting: Language models for code-level neural architecture search | NeuIPS 2023 | [code] | paper |
| Eureka: Human-Level Reward Design via Coding Large Language Models | ICLR 2024 | code | paper |
| Language Model Decoding as Direct Metrics Optimization | Arxiv, Oct 2023, ICLR 2024 | [code] | paper |
| Label-free Node Classification on Graphs with Large Language Models (LLMS) | Arxiv, Oct 2023, ICLR 2024 | [code] | paper |
| L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks | GECCO 2024, Jan 2024 | [code] | paper |
| Data-driven Discovery with Large Generative Models | Arxiv, Feb. 2024 | [code] | paper |
| Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network | Arxiv, Feb. 2024 | [code] | paper |
| LLM Guided Evolution-The Automation of Models Advancing Models | Arxiv, Mar. 2024 | [code] | paper |
| Identify Critical Nodes in Complex Network with Large Language Models | Arxiv, Mar. 2024 | [code] | paper |
| Evolving Interpretable Visual Classifiers with Large Language Models | Arxiv, April 2024 | [code] | paper |
Science
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Exploring evolution-aware & -free protein language models as protein function predictors | NeurIPS 2022 | [code] | paper |
| A Prompt-Engineered Large Language Model | Arxiv, Jan 2024 | [code] | paper |
| LLM-SR: Scientific Equation Discovery via Programming with Large Language Models | Arxiv, April 2024 | [code] | paper |
| Large Language Model Agent as a Mechanical Designer | Arxiv, April 2024 | [code] | paper |
| LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery | Arxiv, May 2024 | code | paper |
| Taming Large Language Model For Conversational Protein Design | Openreview, 2024 | [code] | paper |
| AutoTurb: Using Large Language Models for Automatic Algebraic Model Discovery of Turbulence Closure | Arxiv, Oct. 2024 | [code] | paper |
Industry
| Title | Publication with Date | Code | Paper |
|---|---|---|---|
| Large Language Models for Supply Chain Optimization | Arxiv, July 2023 | [code] | paper |
| How Can Large Language Models Help Humans in Design and Manufacturing | Arxiv, July 2023 | [code] | paper |
| LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation | Arxiv, Dec. 2023 | [code] | paper |
Related Collections
| Collection | Link |
|---|---|
| Awesome LLM | Link |
| Foundation Models for Combinatorial Optimization | Link |
| LLM for Planning | Link |
| MOO-ML-Papers | Link |