Skip to content

Constants

constants

Defines constants with valid parameter values expected by the Scrape.do API and runtime resources bundled with the package (e.g. Scrape.do's CA cert).

Attributes:

Name Type Description
_SUPER_SUPPORTED_COUNTRIES set[str]

The complete list of ISO 3166-1 alpha-2 country codes supported when super=True.
_DATACENTER_SUPPORTED_COUNTRIES set[str]

The restricted list of ISO 3166-1 alpha-2 country codes supported when super=False.
_ZIPCODE_FORMATS dict[str, Pattern]

Pre-compiled regular expressions mapping lowercase country codes to their strict regional postal code formats.
_SUPER_ONLY_COUNTRIES set[str]

ISO 3166-1 alpha-2 country codes supported only when super=True
_ZIPCODE_ALLOWED_COUNTRIES set[str]

Set of country codes for which the postal_code parameter is allowed
_ZIPCODE_NOT_ALLOWED_COUNTRIES set[str]

Set of country codes for which the postal_code parameter is not allowed
SCRAPE_DO_CA_PATH str

Filesystem path to Scrape.do's bundled CA certificate, used by DEFAULT_PROXY_SSL_CONTEXT and exposed for third-party tooling that needs to trust the same root.
DEFAULT_PROXY_SSL_CONTEXT SSLContext

Default SSL context used by the proxy-mode clients. Loads system CAs plus Scrape.do's bundled CA so HTTPS targets validate through Scrape.do's MITM step.
_EXPECTED_ERROR_KEYS set[str]

Defines the list of keys expected to exist in the JSON body of an error response returned by the Scrape.do API.

DEFAULT_PROXY_SSL_CONTEXT module-attribute 

DEFAULT_PROXY_SSL_CONTEXT = (
    _build_default_proxy_ssl_context()
)

SSL context loaded with system CAs plus Scrape.do's bundled CA.

Used as the default verify value on ScrapeDoProxyClient and AsyncScrapeDoProxyClient so HTTPS targets validate correctly through Scrape.do's MITM step without forcing users to disable TLS verification.

Overriding The Default

  • verify=True → system CAs only (for users who've installed Scrape.do's CA into their OS keychain).

  • verify=False → disable TLS verification entirely (discouraged).

  • verify=<path> or verify=<ssl.SSLContext> → custom certificate bundle or context for mutual-TLS / corporate CAs.

Shared Instance

  • This context is process-shared and treated as immutable

  • Safe to assign as the default in client constructors

  • Mutating it (loading additional locations, changing verify-mode, etc.) affects every proxy client instance constructed afterwards.

Strict X.509 Validation Relaxation

  • Scrape.do's bundled CA is a self-signed root that omits the Authority Key Identifier X.509 extension.

  • RFC 5280 makes that extension optional for self-signed roots, but OpenSSL's VERIFY_X509_STRICT flag rejects certs without it — and Python 3.13's ssl.create_default_context enables that flag by default.

  • To allow chain validation against the bundled CA, this context clears VERIFY_X509_STRICT while leaving every other verification check (chain, hostname, expiry, signature, etc.) intact.

SCRAPE_DO_CA_PATH module-attribute 

SCRAPE_DO_CA_PATH = str(
    files("scrape_do.data") / "scrapedo_ca.crt"
)

Filesystem path to Scrape.do's bundled CA certificate.

The cert is shipped under the scrape_do.data package so it travels with the wheel; resolved at import time via importlib.resources.files().

Use Cases

  • Default verify source for the proxy-mode clients (see DEFAULT_PROXY_SSL_CONTEXT).

  • Configuring third-party tooling (e.g. Selenium / Playwright) to trust Scrape.do's MITM cert when using proxy mode.

  • Mirroring the SDK's default proxy-mode TLS behavior in custom HTTP clients.