Parameter Description

Understanding Thordata's Universal Scraping API - Scraping Parameters

Token: This parameter defines the API token used for scraping. It plays a decisive role in your successful scraping.

Parameter	Name	Function
token	Token	API token for authentication during scraping

URL: This parameter defines the target website URL for scraping, with google.com as default. You can also change it to other search engine links.

Parameter	Name	Function
url	URL	Target website URL for scraping

Output Format: This parameter defines the output format of scraped results. Options include HTML and PNG formats, with HTML as default.

Parameter	Name	Function	Parameter Values
format	Output Format	Defines output format of scraped results	format=HTML format=PNG

JS Rendering: JS rendering can handle dynamically loaded content and Single Page Applications (SPA), supporting more interactive and complex pages and rendering requirements. We recommend enabling this.

Parameter	Name	Function	Parameter Values
js_render	JS Rendering	Determines whether to get JS-rendered webpage content	js_render=True js_render=False

Block Website Resources: This parameter can restrict unnecessary resource scraping to speed up the process.

Parameter	Name	Function	Parameter Values
block_resources	Block Resources	Determines whether to fetch certain unnecessary resources (multiple selectable)	block_resources=Images block_resources=JavaScript

country: This parameter defines which country/region's proxy to use during scraping, with no proxy country/region's as default.

Parameter	Name	Function	Parameter Values
country	Country/Region	Specifies proxy country/region to use for scraping	country=Random country=Any country code