Awesome

Awesome-DevAI

Getting started

Many developers have concerns about how LLMs will impact development. What AI won’t replace in your programming describes why the expertise and intuition of programmers will remain important.

A GitHub survey found that 92% of developers are already using AI tools at work or in their personal time. Developers are the first group to adopt AI at work. Here’s why that matters. provides some insight into how software development is already evolving due to LLMs.

It can be hard to figure out what LLM to use while coding. What LLM to use? A perspective from the DevAI space walks you through the most popular commercial and open-source models.

Researchers use benchmarks to evaluate and compare the relative performance of LLMs. An introduction to code LLM benchmarks for software engineers gives you an overview of the current code benchmarks, so you can use them to figure out which LLMs are worth trying.

It can be deceptively challenging to use LLMs while coding. How to use a Large Language Model while coding helps you avoid common mistakes and shares some ideas for how to think about using LLMs while coding.

When you use an LLM while coding, you might want to adjust the preset. Is it still necessary to tune an LLM preset? Tracing the history of temperature, penalty, and sampling schemes gives you the confidence to adjust the model preset when needed.

If you find an open-source code LLM useful, you will likely want to deploy it for your entire team. How to deploy an open-source code LLM for your dev team provides a guide on how to make this happen.

Codex, a GPT language model fine-tuned on publicly available code from GitHub, is the model behind GitHub Copilot. Evaluating Large Language Models Trained on Code introduced it to the world, while also discussing the potential broader impacts of deploying powerful code generation technologies.

Many developers have found Github Copilot to be quite useful. Copilot Internals looks at its source code and shares how the extension sends valuable information from surrounding code to the Codex model.

When you do or don't hit tab after a GitHub Copilot suggestion, this signals what it should do more or less of in the future. It’s time to collect data on how you build software makes you aware of this as you use LLMs when coding.

Software isn’t created in one dramatic step. Large sequence models for software development activities explains how Google is using the process of software development as the source of training data for its ML models that assist developers.

There is a lot of great research on code LLMs. Large Language Models for Software Engineering: Survey and Open Problems gives an overview of the latest as of October 2023.

amplified.dev is a living document that begins to sketch a vision for a future where developers are amplified, not automated. It is focused on how systemic issues some platform development teams have seen in how engineering organizations are adopting, using, and evolving AI software development systems.

It's worth it to learn more about code LLM research too. codefuse-ai/Awesome-Code-LLM and huybery/Awesome-Code-LLM are curated list of best code LLM research and is a great place to read more about the latest.