User Guide

Given—

• a prompt string

• a list of possible completion strings

• and a language model

—CAPPr picks the completion which is most likely to follow prompt according to the language model.

Here’s a quick example:

from cappr.openai.classify import predict

prompt = """
Tweet about a movie: "Oppenheimer was pretty good. But 3 hrs...cmon Nolan."
This tweet contains the following criticism:
""".strip("\n")

completions = ("bad message", "too long", "unfunny")

pred = predict(prompt, completions, model="text-ada-001")
print(pred)
# too long

There are three factors which influence the performance of CAPPr: the language model, the prompt-completion format, and the prior.

• Select a language model
  • Hugging Face
  • Llama CPP
  • OpenAI
• Select a prompt-completion format
  • Prompt-completion formats
  • Quirks
  • Wrangle step-by-step completions
  • A note on few-shot prompts
  • Footnotes
  • References
• (Optional) Supply a prior
  • Examples
• Examples
• Why predict probabilities?
  • Examples
  • References
• A note on workflow
  • Gather data
  • Split data into train and test
  • Iterate on the training dataset
  • If necessary, bring out the big guns
  • Evaluate on the test dataset once