Awesome

LLM4Opt

Collection on Large Language Models for Optimization (LLM4Opt)

🔥 Applying Large language models (LLMs) for diverse optimization tasks (Opt) is an emerging research area. This is a collection of references and papers of LLM4Opt. The Papers are sorted by time (first publicly available).

Any suggestions and pull requests are welcomed!

It is far from a comprehensive list. If you want to update the list:

Fork, Add, and Merge
Report an issue
Contact Fei Liu (fliu36-c@my.cityu.edu.hk)

The sharing principle of these references here is for research. If any authors do not want their paper to be listed here, please feel free to contact us.

Overview

Platform

Project	Description
EoH (Evolution of Heuristics)	optimization, mathematics, machine learning, etc
OpenELM	robots, image, programming puzzles, etc

Course

Course	Description
2024 Fall, LLM Agents	LLM basics and LLM for agents

Tutorial&Workshop

Event	Link
NeurIPS 2023 Workshop: Foundation Models for Decision Making	Link
AAAI 2024 Workshop: Public Sector LLMs: Algorithmic and Sociotechnical Design	Link
GECCO 2024 Workshop: Large Language Models for and with Evolutionary Computation (LLMfwEC)	Link
GECCO 2024 Workshop: EGML-EC — 3rd GECCO workshop on Enhancing Generative Machine Learning with Evolutionary Computation (EGML-EC) 2024	Link
GECCO 2024 Tutorial: Using Large Language Models for Evolutionary Search	Tutorial Link, Tutorial Report Link
PPSN 2024 Tutorial: Large Language Models as Tools for Metaheuristic Design: Exploring Challenges and Opportunities	Link
KDD 2024 Tutorial: NL2Code-Reasoning and Planning with LLM for Code Development	Link, Link
ICML 2024 Workshop: AI for Math	Link
NeurIPS 2024 Workshop: Multimodal Algorithmic Reasoning (MAR)	Link
NeurIPS 2024 Workshop: The 4th Workshop on Mathematical Reasoning and AI	Link

Competition

Event	Link
AAAI 2024 Global Competition on Math Problem Solving and Reasoning	Link
ICML 2024 Challenges on Automated Math Reasoning	Link

SpecialIssues

Event	Link
IEEE TETCI, Special Issue on Neural Architecture Search and Large Machine Learning Models	Link
ACM TELO, Special Issue on Integrating Evolutionary Algorithms and Large Language Models	Link
IEEE TEVC, Special Issue on Evolutionary Computation Meets Large Language Models	link

Papers

Review

Title	Publication with Date	Code	Paper
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap	Arxiv, Jan 2024	[code]	paper
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond	Arxiv, Mar. 2024	[code]	paper
When Large Language Model Meets Optimization	Arxiv, May 2024	[code]	paper

Position paper

Title	Publication with Date	Code	Paper
The Era OF Semantic Decoding	Arxiv, Mar. 2024	[code]	paper
Leveraging Foundational Models for Black-Box Optimization: Benefits, Challenges, and Future Directions	ICML 2024, May 2024	[code]	paper

Algorithm/Heuristic/Function Search

Title	Publication with Date	Code	Paper
Hypothesis Search: Inductive Reasoning with Language Models	Arxiv Sep 2023, ICLR 2024	[code]	paper
ToolChain: Efficient Action Space Navigation in Large Language Models with A Search	Arxiv Oct 2023, ICLR 2024	[code]	paper
Algorithm Evolution using Large Language Model	Arxiv Nov 2023	code	paper
Mathematical discoveries from program search with large language models	Nature	code	paper
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model	ICML 2024 (Oral)	code	paper
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution	Arxiv Feb 2024	code	paper
Discovering More Effective Tensor Network Structure Search Algorithms via Large Language Models (LLMs)		[code]	paper
AutoSAT: Automatically Optimize SAT Solvers via Large Language Models	Arxiv Feb 2024	[code]	paper
Large Language Models tO Enhance Bayesian Optimization	Arxiv Feb 2024, ICLR 2024	[code]	paper
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks	Arxiv Feb 2024	[code]	paper
How Can LLM Guide RL? A Value-Based Approach	Arxiv Feb 2024	code	paper
Evolve Cost-aware Acquisition Functions Using Large Language Models	PPSN 2024	[code]	paper
Benchmarking ChatGPT on Algorithmic Reasoning	Arxiv April 2024	[code]	paper
How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems	Arxiv March 2024	[code]	paper
LLM-ABR: Designing Adaptive Bitrate Algorithms via Large Language Models	Arxiv April 2024	[code]	paper
Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems	Arxiv April 2024	[code]	paper
GLHF: General Learned Evolutionary Algorithm Via Hyper Functions	Arxiv	[code]	paper
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)	ICML 2024	[code]	paper
LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics	Arxiv May 2024	code	paper
Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models	PPSN 2024	code	paper
Discovering Preference Optimization Algorithms with and for Large Language Models	Arxiv June 2024	code	paper
OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code	Arxiv May 2024	code	paper

LLM as Optimizer

Title	Publication with Date	Code	Paper
Large Language Models as Optimizers	Arxiv, Sep 2023	code	paper
Large language model for multi-objective evolutionary optimization	Arxiv, Oct. 2023	code	paper
Large Language Models as Evolutionary Optimizers	Arxiv, Oct. 2023	[code]	paper
Using Large Language Models for Hyperparameter Optimization	NeurIPS 2023	[code]	paper
Large Language Models As Evolution Strategies	Arxiv, Feb. 2024	[code]	paper
Large Language Model-based Evolutionary Optimizer: Reasoning with Elitism	Arxiv, Mar. 2024	[code]	paper
Large Language Model-Aided Evolutionary Search for Constrained Multiobjective Optimization	Arxiv, May 9, 2024	[code]	paper
Towards Optimizing with Large Language Model	KDD 2024	[code]	paper

LLM as/for surrogate model

Title	Publication with Date	Code	Paper
Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study	Arxiv, June 2024	code	paper
Large Language Model-assisted Surrogate Modelling for Engineering Optimization	CAI, 2024	[code]	paper
LLM Performance Predictors are good initializers for Architecture Search	Arxiv, Aug 2024	[code]	paper

Code Generation

Title	Publication with Date	Code	Paper
L2MAC: LARGE LANGUAGE MODEL AUTOMATIC COMPUTER FOR EXTENSIVE CODE GENERATION	Arxiv Oct 2023, ICLR 2024	[code]	paper

Prompt Opt

Title	Publication with Date	Code	Paper
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers	Arxiv Sep 2023, ICLR 2024	[code]	paper
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization	Arxiv Oct 2023, ICLR 2024	[code]	paper

Machine Learning

Title	Publication with Date	Code	Paper
Evolution through Large Models	Arxiv, June 2022	code	paper
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph	Arxiv, July 2023, ICLR 2024	[code]	paper
Evoprompting: Language models for code-level neural architecture search	NeuIPS 2023	[code]	paper
Eureka: Human-Level Reward Design via Coding Large Language Models	ICLR 2024	code	paper
Language Model Decoding as Direct Metrics Optimization	Arxiv, Oct 2023, ICLR 2024	[code]	paper
Label-free Node Classification on Graphs with Large Language Models (LLMS)	Arxiv, Oct 2023, ICLR 2024	[code]	paper
L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks	GECCO 2024, Jan 2024	[code]	paper
Data-driven Discovery with Large Generative Models	Arxiv, Feb. 2024	[code]	paper
Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network	Arxiv, Feb. 2024	[code]	paper
LLM Guided Evolution-The Automation of Models Advancing Models	Arxiv, Mar. 2024	[code]	paper
Identify Critical Nodes in Complex Network with Large Language Models	Arxiv, Mar. 2024	[code]	paper
Evolving Interpretable Visual Classifiers with Large Language Models	Arxiv, April 2024	[code]	paper

Science

Title	Publication with Date	Code	Paper
Exploring evolution-aware & -free protein language models as protein function predictors	NeurIPS 2022	[code]	paper
A Prompt-Engineered Large Language Model	Arxiv, Jan 2024	[code]	paper
LLM-SR: Scientific Equation Discovery via Programming with Large Language Models	Arxiv, April 2024	[code]	paper
Large Language Model Agent as a Mechanical Designer	Arxiv, April 2024	[code]	paper
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery	Arxiv, May 2024	code	paper
Taming Large Language Model For Conversational Protein Design	Openreview, 2024	[code]	paper

Industry

Title	Publication with Date	Code	Paper
Large Language Models for Supply Chain Optimization	Arxiv, July 2023	[code]	paper
How Can Large Language Models Help Humans in Design and Manufacturing	Arxiv, July 2023	[code]	paper
LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation	Arxiv, Dec. 2023	[code]	paper

Related Collections

Collection	Link
Awesome LLM	Link
Foundation Models for Combinatorial Optimization	Link
LLM for Planning	Link
MOO-ML-Papers	Link