For the complete documentation index, see llms.txt. This page is also available as Markdown.

Create a crawling task description

Basic Information

Interface Description: Create a crawling task for the Web Scraper API through the interface.

Text request address: https://scraperapi.thordata.com/builder YouTube audio/video request address: https://scraperapi.thordata.com/video_builder

Request Method: POST

Request parameters

Header:

Parameter name
Type
Required
Describe

token

string

Yes

Please log in to the Dashboard-MENU-My Account to view your token parameter value

Authorization

string

Yes

Please log in to the Dashboard to view your Authorization token parameter value. Example: "Authorization: Bearer *********************"

Body:

Parameter name
Type
Required
Describe

spider_name

string

Yes

Facebook:

spider_name facebook.com

Amazon:

spider_name amazon.com

YouTube:

spider_name youtube.com

spider_id

string

Yes

Amazon:

spider_id amazon_product_by-url, amazon_product_by-best-sellers, etc.

YouTube:

spider_id youtube_video_by-url, youtube_audio_by-url, etc.

For more ids, please see the parameter description.

spider_parameters

text

Yes

Amazon:

url crawling link, zip_code zip code, sort_by sorting method, etc.

YouTube:

url crawling link.

For more parameters, please see the parameter description.

spider_errors

string

Yes

true

file_name

string

Yes

Customize the name of the downloaded file.

{{TasksID}}, {{VideoID}}, etc.

Response example

If the returned code value is not 200, please check the error code description list and modify it.

Parameter name
Type
Required
Describe

code

number

Yes

The returned code value.

data

object

No

The returned details.

task_id

number

No

The id of the task.

Last updated