Skip to content

Exceptions

exceptions

Exception hierarchy and structured error envelope for the Scrape.do Async API surface

Defines the typed exceptions raised by the ScrapeDoAsyncAPIClient, AsyncScrapeDoAsyncAPIClient, and polling helpers. In addition, defines AsyncScrapeDoErrorMessage, a pydantic model wrapping the {Error, Code} JSON body returned by q.scrape.do on non-2xx responses

AsyncScrapeDoErrorMessage

Bases: BaseModel

Structured representation of a Scrape.do Async API error body

Tolerant Parsing

  • Server-side additions are silently ignored (extra="ignore")

  • Missing fields fall back to their defaults

Different From The Sync Envelope

  • The synchronous gateway's ScrapeDoJSONErrorMessage uses a richer schema (StatusCode, Message, PossibleCauses, ErrorType, etc.)

  • Error carries the human-readable description, Code carries the HTTP status code

  • Both models follow the same try_from_response parsing contract so callers can route either envelope to typed exceptions without special-casing

Attributes:

Name Type Description
error Optional[str]

human-readable description of what went wrong
code Optional[int]

HTTP status code echoed in the body

try_from_response(raw_resp) classmethod

Parse a Scrape.do Async API error envelope, or return None

Return Value

  • If raw_resp has a parsable JSON body containing at least one Error and Code AND the pydantic validation of this model succeeds, this method returns a validated AsyncScrapeDoErrorMessage instance

  • Otherwise, it returns None

  • This method never raises

Parameters:

Name Type Description Default
raw_resp Response

The raw httpx.Response object received from q.scrape.do

 required

Returns:

Type Description
Optional[AsyncScrapeDoErrorMessage]

An AsyncScrapeDoErrorMessage instance when the body parses as a JSON dict containing at least one of Error / Code and pydantic validation succeeds. None otherwise. Never raises

__str__()

Human-readable rendering for inclusion in exception messages

Returns:

Type Description
str

A multi-line string with both fields labeled

AsyncAPIError

Bases: ScrapeDoError

Base exception for every error raised by the Scrape.do Async API surface

Routing

  • Catching this class guarantees every Async-API-originated failure is handled

  • It inherits from ScrapeDoError, so a except ScrapeDoError block still catches it

AsyncAPIResponseError

Bases: AsyncAPIError

Base for non-2xx HTTP responses returned by q.scrape.do

Error Parsing

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPIUnparsableResponseError

Bases: AsyncAPIError

Raised when the SDK can't parse a successful response sent by q.scrape.do

This can either happen because the response sent by q.scrape.do is not parsable JSON, or because one of the endpoints returned an unexpected JSON schema and caused one of the custom pydantic response models to raise a ValidationError

Additional Exception Information

This exception is always chained to either a JSONDecodeError or a ValidationError exception to provide additional information on what caused the error

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do

AsyncAPIBadRequestError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns HTTP 400

Indicates the job request was rejected at schema validation

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPIAuthError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns HTTP 401

Indicates the X-Token header is missing or invalid

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPINotFoundError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns HTTP 404

Indicates the requested JobID / TaskID does not exist or has already expired

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPINotAcceptableError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns HTTP 406

Indicates the request is structurally valid but cannot be served in the current state (e.g. canceling a job that already completed)

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPIRateLimitError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns HTTP 429

Indicates the account has exhausted its Async-API concurrency allowance (roughly 30% of the main plan concurrency)

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

AsyncAPIServerError

Bases: AsyncAPIResponseError

Raised when q.scrape.do returns a 5xx status code

Indicates a gateway or proxy pool failure on Scrape.do's side

Attributes:

Name Type Description
raw_response Response

The raw httpx.Response object received from q.scrape.do
raw_status_code int

The HTTP status code (shortcut for raw_response.status_code)
error_info Optional[AsyncScrapeDoErrorMessage]

Parsed envelope or None when the body wasn't a recognizable Async API error

JobLifecycleError

Bases: AsyncAPIError

Base for terminal-state errors surfaced by polling helpers

Raised by wait_for_job and submit_and_wait when the job reaches a terminal status the caller didn't expect

Attributes:

Name Type Description
job JobDetails

The JobDetails snapshot at the terminal status that triggered this error

JobFailedError

Bases: JobLifecycleError

Raised when a polled job reaches terminal status error

Attributes:

Name Type Description
job JobDetails

The JobDetails snapshot at the terminal status that triggered this error
Per-Task Errors

After the polling helper raises, callers can still inspect each task's error_message via the corresponding TaskDetails

JobCanceledError

Bases: JobLifecycleError

Raised when a polled job reaches terminal status canceled

Attributes:

Name Type Description
job JobDetails

The JobDetails snapshot at the terminal status that triggered this error

TaskLifecycleError

Bases: AsyncAPIError

Base for terminal-state errors surfaced by TaskDetails

Raised by TaskDetails.raise_for_status when the task has reached a terminal failure status the caller didn't expect

Includes the task.error_message in the exception's error message if it exists.

Attributes:

Name Type Description
task TaskDetails

The TaskDetails snapshot at the terminal status that triggered this error

TaskFailedError

Bases: TaskLifecycleError

Raised when a TaskDetails snapshot has terminal status error

Attributes:

Name Type Description
task TaskDetails

The TaskDetails snapshot at the terminal status that triggered this error

TaskCanceledError

Bases: TaskLifecycleError

Raised when a TaskDetails snapshot has terminal status canceled

Attributes:

Name Type Description
task TaskDetails

The TaskDetails snapshot at the terminal status that triggered this error

JobTimeoutError

Bases: AsyncAPIError

Raised when the PollingStrategy.__call__, or a user-defined PollingFunction has chosen to stop polling for a job before it reached a terminal status

Attributes:

Name Type Description
job_id str

The JobID that was being polled
last_status str

The last non-terminal status observed before timing out
elapsed float

Total seconds spent polling before timeout
attempts int

Number of get_job calls made before timeout