Prompt Engineering: Techniques and Best Practices for LLMs

Prompt engineering is the art of crafting inputs to Large Language Models to get desired outputs. Effective prompts significantly improve LLM performance.

Prompt Engineering Fundamentals

Basic Prompt Structure

from openai import OpenAI

client = OpenAI()

# Simple prompt
response = client.chat.completions.create(
    model='gpt-4',
    messages=[
        {'role': 'user', 'content': 'What is Python?'}
    ]
)

# Structured prompt with role
response = client.chat.completions.create(
    model='gpt-4',
    messages=[
        {
            'role': 'system',
            'content': 'You are an expert Python programmer with 10 years of experience.'
        },
        {
            'role': 'user',
            'content': 'Explain Python decorators to a beginner.'
        }
    ]
)

# Multi-turn conversation
messages = [
    {'role': 'system', 'content': 'You are a helpful assistant.'},
    {'role': 'user', 'content': 'What is machine learning?'},
    {'role': 'assistant', 'content': 'Machine learning is...'},
    {'role': 'user', 'content': 'Can you give an example?'}
]

response = client.chat.completions.create(
    model='gpt-4',
    messages=messages
)

Prompt Engineering Techniques

1. Zero-Shot Prompting

# No examples provided
prompt = """
Classify the sentiment of this review:
"The product is amazing and works perfectly!"

Sentiment:
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)
# Output: Positive

2. Few-Shot Prompting

# Provide examples
prompt = """
Classify the sentiment of reviews:

Review: "Great product, highly recommend!"
Sentiment: Positive

Review: "Terrible quality, waste of money"
Sentiment: Negative

Review: "It's okay, nothing special"
Sentiment: Neutral

Review: "Best purchase ever made!"
Sentiment:
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)
# Output: Positive

3. Chain-of-Thought Prompting

# Ask model to think step by step
prompt = """
Solve this problem step by step:

Q: A store has 50 apples. It sells 30 apples and receives 20 more.
How many apples does it have now?

Let me think through this:
1. Starting amount: 50 apples
2. After selling 30: 50 - 30 = 20 apples
3. After receiving 20: 20 + 20 = 40 apples

Answer: 40 apples

Now solve this:
Q: If a book costs $15 and you buy 3 books with a 10% discount,
how much do you spend?

Let me think through this:
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)

4. Role-Based Prompting

# Assign specific role/persona
prompts = {
    'expert': """
You are a world-renowned expert in machine learning with 20 years of experience.
Explain neural networks to a beginner.
""",
    
    'teacher': """
You are an experienced teacher who specializes in making complex topics simple.
Explain quantum computing to a 12-year-old.
""",
    
    'critic': """
You are a critical reviewer. Analyze this code for potential issues:
[code snippet]
""",
    
    'translator': """
You are a professional translator fluent in English and Spanish.
Translate this text: [text]
"""
}

for role, prompt in prompts.items():
    response = client.chat.completions.create(
        model='gpt-4',
        messages=[{'role': 'user', 'content': prompt}]
    )

5. Instruction-Based Prompting

# Clear, detailed instructions
prompt = """
Task: Summarize the following article

Instructions:
1. Keep summary under 100 words
2. Use bullet points for key points
3. Maintain original meaning
4. Use simple language
5. Include main conclusions

Article:
[article text]

Summary:
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)

6. Constraint-Based Prompting

# Specify constraints
prompt = """
Write a Python function with these constraints:
- Function name: calculate_average
- Input: list of numbers
- Output: average of numbers
- Must handle empty list
- Must include docstring
- Must include type hints
- Maximum 10 lines of code

Function:
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)

7. Format-Specific Prompting

# Specify output format
prompts = {
    'json': """
Extract information from this text and return as JSON:
[text]

Format:
{
  "name": "...",
  "age": "...",
  "email": "..."
}
""",
    
    'markdown': """
Create a markdown document with:
- Title
- Introduction
- 3 main sections
- Conclusion
- References

Topic: Python Best Practices
""",
    
    'code': """
Write Python code that:
- Reads a CSV file
- Filters rows where age > 18
- Saves to new CSV

Requirements:
- Use pandas
- Include error handling
- Add comments
""",
    
    'table': """
Create a comparison table of:
- Python vs JavaScript
- Columns: Feature, Python, JavaScript
- Include 5 features
"""
}

Advanced Techniques

1. Retrieval-Augmented Generation (RAG)

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter

# Load and split documents
loader = TextLoader('document.txt')
documents = loader.load()
splitter = CharacterTextSplitter(chunk_size=1000)
docs = splitter.split_documents(documents)

# Create embeddings and vector store
embeddings = OpenAIEmbeddings()
vector_store = FAISS.from_documents(docs, embeddings)

# Retrieve relevant context
query = "What is machine learning?"
relevant_docs = vector_store.similarity_search(query, k=3)

# Build prompt with context
context = "\n".join([doc.page_content for doc in relevant_docs])
prompt = f"""
Based on this context:
{context}

Answer this question: {query}
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)

2. Tree-of-Thought Prompting

# Explore multiple reasoning paths
prompt = """
Problem: How to improve code performance?

Let me think about different approaches:

Approach 1: Profiling
- Use cProfile to identify bottlenecks
- Analyze function call frequency
- Focus on hot spots

Approach 2: Optimization
- Use efficient algorithms
- Optimize data structures
- Reduce memory usage

Approach 3: Parallelization
- Use multiprocessing
- Implement async operations
- Distribute workload

Which approach would you recommend and why?
"""

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': prompt}]
)

3. Self-Consistency Prompting

# Get multiple responses and find consensus
def get_multiple_responses(prompt, num_responses=3):
    responses = []
    for _ in range(num_responses):
        response = client.chat.completions.create(
            model='gpt-4',
            messages=[{'role': 'user', 'content': prompt}],
            temperature=0.7  # Higher temperature for diversity
        )
        responses.append(response.choices[0].message.content)
    return responses

# Use multiple responses
prompt = "What is the best way to learn Python?"
responses = get_multiple_responses(prompt, num_responses=3)

# Analyze responses
for i, response in enumerate(responses, 1):
    print(f"Response {i}: {response}\n")

4. Prompt Chaining

# Break complex task into steps
def prompt_chain():
    # Step 1: Understand the problem
    step1_prompt = "Explain what web scraping is"
    response1 = client.chat.completions.create(
        model='gpt-4',
        messages=[{'role': 'user', 'content': step1_prompt}]
    )
    explanation = response1.choices[0].message.content
    
    # Step 2: Provide example
    step2_prompt = f"""
Based on this explanation: {explanation}

Provide a Python example of web scraping using BeautifulSoup
"""
    response2 = client.chat.completions.create(
        model='gpt-4',
        messages=[{'role': 'user', 'content': step2_prompt}]
    )
    example = response2.choices[0].message.content
    
    # Step 3: Add best practices
    step3_prompt = f"""
Based on this example: {example}

Add best practices and error handling
"""
    response3 = client.chat.completions.create(
        model='gpt-4',
        messages=[{'role': 'user', 'content': step3_prompt}]
    )
    
    return response3.choices[0].message.content

result = prompt_chain()

Prompt Optimization

Parameter Tuning

# Temperature: Controls randomness
# 0.0 = deterministic, 1.0 = random
response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': 'Write a poem'}],
    temperature=0.8  # Creative
)

# Max tokens: Limit response length
response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': 'Summarize this article'}],
    max_tokens=100  # Short summary
)

# Top-p: Nucleus sampling
response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': 'Generate ideas'}],
    top_p=0.9  # Diverse but coherent
)

# Frequency penalty: Reduce repetition
response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': 'List 10 ideas'}],
    frequency_penalty=0.5  # Avoid repetition
)

def refine_prompt(initial_prompt, feedback):
    """Iteratively improve prompt based on feedback"""
    
    refinement_prompt = f"""
Original prompt: {initial_prompt}

Feedback: {feedback}

Improved prompt:
"""
    
    response = client.chat.completions.create(
        model='gpt-4',
        messages=[{'role': 'user', 'content': refinement_prompt}]
    )
    
    return response.choices[0].message.content

# Example usage
prompt = "Explain machine learning"
feedback = "Too technical, use simpler language"
improved = refine_prompt(prompt, feedback)

Best Practices

Be specific: Clear, detailed prompts get better results
Provide context: Include relevant background information
Use examples: Few-shot learning improves performance
Specify format: Tell model how to structure output
Set constraints: Limit length, style, or complexity
Iterate: Refine prompts based on results
Test variations: Try different approaches
Monitor costs: Track API usage and costs

Common Pitfalls

Bad Practice:

# Don't: Vague prompts
"Tell me about Python"

# Don't: No context
"Summarize this"

# Don't: Conflicting instructions
"Write a short essay that is very detailed"

# Don't: Assume understanding
"Do the thing"

Good Practice:

# Do: Specific prompts
"Explain Python's key features for someone with no programming experience"

# Do: Provide context
"Summarize this article in 100 words: [article]"

# Do: Clear instructions
"Write a concise essay (200-300 words) about machine learning"

# Do: Explicit requirements
"Write Python code that reads a CSV file and calculates the average age"

Conclusion

Prompt engineering is a critical skill for working with LLMs. Master various techniques, understand model parameters, and iterate on prompts to achieve desired results. As LLMs evolve, prompt engineering techniques will continue to develop, making continuous learning essential.

Prompt Engineering: Techniques and Best Practices for LLMs

Prompt Engineering: Techniques and Best Practices for LLMs

Prompt Engineering Fundamentals

Basic Prompt Structure

Prompt Engineering Techniques

1. Zero-Shot Prompting

2. Few-Shot Prompting

3. Chain-of-Thought Prompting

4. Role-Based Prompting

5. Instruction-Based Prompting

6. Constraint-Based Prompting

7. Format-Specific Prompting

Advanced Techniques

1. Retrieval-Augmented Generation (RAG)

2. Tree-of-Thought Prompting

3. Self-Consistency Prompting

4. Prompt Chaining

Prompt Optimization

Parameter Tuning

Iterative Refinement

Best Practices

Common Pitfalls

Conclusion

Comments