Reliability & Error Handling

Implement robust error handling to ensure your application remains reliable.

Recommendations:

• Implement exponential backoff - When rate limited, retry with progressively longer wait times.
• Handle timeouts gracefully - Set appropriate timeouts and provide fallback responses.
• Validate responses - Check that responses meet your application’s requirements before using them.
• Monitor API usage - Implement logging and alerts for unusual patterns.
• Implement circuit breakers - Temporarily disable features that repeatedly fail rather than continuing to send failing requests.

Example: Robust Error Handling

import time
import random
from openai import OpenAI
from openai.error import RateLimitError, APIError, Timeout

client = OpenAI(api_key="your_api_key", base_url="https://api.oraicle.me/v1")

def get_completion_with_retry(prompt, max_retries=5):
    retries = 0
    while retries <= max_retries:
        try:
            response = client.chat.completions.create(
                model="deepseek-ai/DeepSeek-R1",
                messages=[
                    {"role": "user", "content": prompt}
                ],
                timeout=10  # 10 second timeout
            )
            return response.choices[0].message.content
        except RateLimitError:
            # Exponential backoff with jitter
            sleep_time = (2 ** retries) + random.random()
            time.sleep(sleep_time)
            retries += 1
        except (APIError, Timeout) as e:
            if retries >= max_retries:
                return "Sorry, I'm having trouble processing your request right now."
            time.sleep(1)
            retries += 1

    # Fallback response if all retries fail
    return "I apologize, but I'm currently unavailable. Please try again later."