← ML Research Wiki / 2302.11382

A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Jules White [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Quchen Fu [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Sam Hays Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Michael Sandborn [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Carlos Olea [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Henry Gilbert [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Ashraf Elnashar [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Jesse Spencer-Smith [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA, Douglas C Schmidt [email protected] Department of Computer Science Vanderbilt University Tennessee NashvilleTNUSA (2023)

Paper Information
arXiv ID
2302.11382
Venue
arXiv.org
Domain
natural language processing
Contents

Abstract

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT.Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output.Prompts are also a form of programming that can customize the outputs and interactions with an LLM.This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs.Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs.This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks.First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains.Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations.Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

Summary

This paper presents a catalog of prompt engineering techniques for conversational large language models (LLMs) like ChatGPT, aimed at enhancing their usability in software development contexts. The authors compare prompt patterns to software patterns, establishing a framework for documenting reusable solutions that can improve LLM interactions. The paper categorizes various prompt patterns, including those for input semantics, output customization, error identification, prompt improvement, interaction, and context control. The document outlines the intent, structure, examples, and consequences of each pattern, emphasizing the importance of effective prompt design, adaptability, and user engagement. It also reflects on limitations and proposes avenues for further research and refinement in prompt patterns, particularly as LLM capabilities evolve.

Methods

This paper employs the following methods:

  • Prompt Engineering
  • Pattern Recognition

Models Used

  • ChatGPT

Datasets

The following datasets were used in this research:

  • None specified

Evaluation Metrics

  • None specified

Results

  • Framework for documenting patterns for structuring prompts
  • Catalog of 16 distinct prompt patterns to solve common problems in LLM interaction
  • Strategies for combining multiple prompt patterns to enhance outputs

Limitations

The authors identified the following limitations:

  • Potential confusion with ambiguous prompt definitions
  • The necessity for users to understand and evaluate generated outputs
  • Dependency on user input for context management

Technical Requirements

  • Number of GPUs: None specified
  • GPU Type: None specified

Keywords

prompt patterns prompt engineering large language models ChatGPT software patterns prompt design

Papers Using Similar Methods

External Resources

Funding: Not specified
References: 54
Influential Citations: 63