refactor(python-sdk): Improve request handling, pagination, and error handling

[colesmcintosh]

This pull request introduces a series of methodical improvements to the Firecrawl Python SDK, aimed at fortifying its resilience and clarity. Key enhancements include:

• Generic Response Classes: Implementation of generic response data and crawl status classes, which encapsulate response details and streamline data management across the SDK.
• Robust Retry Mechanism: Introduction of an exponential backoff strategy for request retries, significantly improving the SDK’s ability to gracefully handle transient failures.
• Refined Pagination: Enhanced pagination handling for crawl status requests, ensuring efficient and reliable traversal through paginated data sets.
• Simplified Error Handling: Consolidation of error management and response parsing logic, reducing complexity while improving maintainability.
• Type Hints and Structural Improvements: Addition of comprehensive type hints alongside structural refinements, enhancing code clarity and facilitating future development.

These modifications collectively enhance the robustness, readability, and overall maintainability of the Firecrawl SDK, ensuring a more seamless developer experience.

[fumeoss]

Summary

• Introduces generic response classes for better encapsulation of API response data.
• Implements a robust retry mechanism using exponential backoff for request failures.
• Enhances pagination handling for crawl status requests improving data retrieval.
• Simplifies error handling and response parsing, making the codebase cleaner and more maintainable.
• Adds type hints and structural improvements to increase code clarity and facilitate future development.

🚨 1 passed, 1 failed

🟢 Automated Test: Verify Generic Response Container Error Handling

🔴 Test Exponential Backoff Retry Logic for FirecrawlApp