Cost Optimization

You pay per token, so the cheapest request is the shortest one that still gets the answer. A few habits that keep the bill down:

Recommendations:

• Implement caching - Cache responses for identical or similar queries.
• Use compression techniques - Summarize long documents before sending them as context.
• Optimize prompt templates - Shorter, more focused prompts use fewer tokens while often producing better results.
• Adjust temperature settings - Lower temperature values (0.1-0.4) typically produce more concise responses.
• Implement token limits - Set appropriate max_tokens values to prevent unnecessarily long responses.
• Send only the context the question needs - Every token in the prompt is billed, including the system message and any history you resend.

Example: Cost-Efficient API Call

response = openai.ChatCompletion.create(
  model="openai/gpt-oss-20b",  # cheapest input price on the board
  messages=[
      {"role": "system", "content": "You are a concise assistant that gives brief, accurate answers."},
      {"role": "user", "content": "Explain quantum computing"}
  ],
  temperature=0.3,         # Lower temperature for more focused output
  max_tokens=150,          # Limit response length
  presence_penalty=0.6     # Discourage repetition
)