DownloadTaskStatus
Summarizes the execution state and outputs of a single download operation, including current status, any error, and final or intermediate results.
Fields:
| Name | Type | Description |
|---|---|---|
| Error | String | Optional. Request execution error |
| TaskState | DownloadTaskStates | Optional. Task state |
| Result | DownloadInfo | Optional. Download result |
| IntermedResults | Array of DownloadInfo | Optional. Intermediate requests download results stack |
DownloadTaskStates
Lifecycle states a download task can transition through from creation to completion or deletion.
Enumeration values:
| Name | Description |
|---|---|
| Handled | Task is handled and its results are available |
| AccessDeniedForRobots | Access to a URL is denied by robots.txt |
| AllRequestGatesExhausted | All request gateways (proxy and host IP addresses) were exhausted but no data was received |
| InProgress | Task is in progress |
| Created | Task has not been started yet |
| Deleted | Task has been deleted |
DownloadInfo
Captures request/response details for a download attempt, including HTTP metadata, headers, cookies, and payload.
Fields:
| Name | Type | Description |
|---|---|---|
| Method | String | Required. HTTP method |
| Url | String | Required. Request URL |
| IsSuccess | Bool | Required. Was the request successful |
| HttpStatusCode | Int | Required. HTTP status code |
| ReasonPhrase | String | Required. HTTP reason phrase |
| RequestHeaders | Array of HttpHeader | Required. HTTP headers sent with the request |
| ResponseHeaders | Array of HttpHeader | Required. HTTP headers received in the response |
| RequestCookies | Array of Cookie | Required. Cookies sent with the request |
| ResponseCookies | Array of Cookie | Required. Cookies received in the response |
| RequestDateUtc | DateTime | Required. Request date and time in UTC |
| DownloadTimeSec | Double | Required. Download time in seconds |
| ViaProxy | Bool | Required. Is the request made via a proxy |
| WaitTimeSec | Double | Required. What was the delay (in seconds) before the request was executed (crawl latency, etc.) |
| CrawlDelaySec | Int | Required. A delay in seconds applied to the request |
HttpHeader
Represents a single HTTP header with a name and one or more values.
Fields:
| Name | Type | Description |
|---|---|---|
| Name | String | Required. Header name |
| Values | Array of String | Required. Header values |
Cookie
Represents an HTTP cookie as sent via Set-Cookie/ Cookie headers, including attributes.
Fields:
| Name | Type | Description |
|---|---|---|
| Name | String | Required. Name |
| Value | String | Required. Value |
| Domain | String | Required. Domain |
| Path | String | Required. Path |
| HttpOnly | Bool | Required. HttpOnly |
| Secure | Bool | Required. Secure |
| Expires | DateTime | Optional. Expires |