fix: improve AI provider error handling and retries

[arshdeepkaur-code]

Description

Implemented improvements to AI provider error handling and retry behavior.
Changes include:
-Improved handling of transient failures and provider errors
-Enhanced retry behavior for better reliability
-Added better logging and error tracking for debugging

Related Issue

Fixes #476

Type of change

• Bug fix
• New feature / enhancement
• Documentation update
• Test addition
• Refactor

Checklist

• I have read CONTRIBUTING.md
• My branch is up to date with main
• I have run pytest -v and all tests pass
• I have not introduced duplicate issues or features
• My PR title follows the format: feat/fix/docs/test: short description
• I have added tests for new features (Level 2 and 3 issues)
• No hardcoded secrets or API keys in my code
• This PR is linked to a GSSoC 2026 issue

Screenshots (if frontend change)

Test evidence

pytest -v
# paste output

[imDarshanGK]

Looks aligned with Issue #476.

Please add/update tests covering retry behavior and error scenarios (429, 5xx, timeout, and connection errors).

Thanks!