Fetches the next batch of scraped data using a data cursor from CrawlMdr, returning JSON documents and an updated cursor when more data remains.
Arguments
| Name |
Type |
Description |
| dataCursor |
CrawlMdrDataCursor |
Required. Cursor from CrawlMdrResult for fetching data batches. |
| downloadTasksCount |
Int |
Required. The count refers to the number of download tasks to be processed in this request. In most cases, one download task corresponds to one document. However, if a table was handled, it will return as many documents as were in the table (for each download task). Additionally, if there are multiple data objects on each level, the document count will be a multiplication of all counts on each level |
| path |
String |
Required. Path to a level in the MDR tree. It should start with / and contain at least one step. Each step is separated by /. Path must not end with / |
Return Type
Returns a CrawlMdrData object containing the scraped data and a cursor for the next batch.
| Name |
Type |
Description |
| Data |
Array of String |
Required. Array of scraped data objects in JSON format. |
| DataCursor |
CrawlMdrDataCursor |
Optional. Cursor for fetching the next batch of data (null if no more data). |
CrawlMdrDataCursor
| Name |
Type |
Description |
| JobId |
String |
Required. Job Id |
| NextCursor |
String |
Optional. Cursor for fetching the next batch of scraped data (null if done) |