# Query Parameters

####  Web Unlocker Query Parameters

Learn about the crawling parameters of Thordata's General Crawling API.

**`Token`:**\
This parameter defines the API token used for crawling and plays a decisive role in the success of your crawling.

| Parameter | Name  | Function                                   |
| --------- | ----- | ------------------------------------------ |
| token     | Token | API token for verification during crawling |

**`URL`:**\
This parameter defines the target website link for crawling, with a default value of google.com. You can also change it to links of other search engines.

| Parameter | Name | Function                        |
| --------- | ---- | ------------------------------- |
| url       | URL  | Target website URL for scraping |

**Example Request:**

Example with `token`:<mark style="color:blue;">Token ,</mark>`url`:<mark style="color:blue;"><https://www.google.com></mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True"
```

<details>

<summary><code>js_render</code> <strong>,JS Rendering(Optional)</strong></summary>

JS rendering can handle dynamically loaded content and Single - Page Applications (SPAs). It supports pages with more complex interactions and rendering requirements, so it is recommended that you enable it.

**Example Request:**

Example with `js_render`:<mark style="color:blue;">True</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True"
```

</details>

<details>

<summary><code>type</code> <strong>,Format(Optional)</strong></summary>

This parameter defines the output format of the crawling results. The options include HTML and PNG formats, with the default being HTML.

**Example Request:**

Example with `type`:<mark style="color:blue;">png</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=png" -d "js_render=True"
```

</details>

<details>

<summary><code>block_resources</code><strong>,</strong>Block<strong>(Optional)</strong></summary>

This parameter can restrict the crawling of unnecessary resources, thereby accelerating the crawling speed.

**Example Request:**

Example with `block_resources`:<mark style="color:blue;">script</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True" -d "block_resources=script"
```

</details>

<details>

<summary><code>country</code><strong>,</strong>Proxies<strong>(Optional)</strong></summary>

This parameter defines the country/region of the proxy used for crawling, with the default value being no proxy.

**Example Request:**

Example with `country`:<mark style="color:blue;">al</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True" -d "country=al"
```

</details>

<details>

<summary><code>clean_content</code><strong>,</strong>Remove<strong>(Optional)</strong></summary>

This parameter is used to remove `JS` and `CSS` content from the returned results.

**Example Request:**

Example of a parameter with `clean_content: JS,CSS`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "clean_content=js,css" \
 -d "header=False"
```

</details>

<details>

<summary><code>wait</code><strong>,</strong>Wait for (ms)<strong>(Optional)</strong></summary>

Wait for the page to load content within the specified time (in milliseconds). Maximum value: 100,000 ms\
**Example Request:**

Example of a parameter with `wait:1000`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "wait=10000"
```

</details>

<details>

<summary><code>wait_for</code> <strong>,</strong>Wait for selector<strong>(Optional)</strong></summary>

Wait for the CSS selector to load in the DOM. `wait_for` overrides `wait` (ignoring fixed time) if both are used. Maximum wait time is 30 seconds; content is returned automatically on timeout.

**Example Request:**

Example with `wait_for:.content`  parameter

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "wait=10000" \
 -d "wait_for=.content"
```

</details>

<details>

<summary><code>headers</code><strong>,</strong>Custom Headers<strong>(Optional)</strong></summary>

Pass custom request headers to the target website. When there are multiple headers, please separate them with an English comma ',';\
HTTP headers are key-value pairs separated by a colon (:). The parameters sent should be in JSON format.

**Example request:**

If you want to send `User-Agent` and `Content-Type`, it should be as follows:

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=token" \
 -d 'headers=[{"name":"name1","value":"value1"}]' \
 -d 'cookies=[{"name":"name2","value":"value2"}]'
```

</details>

<details>

<summary><code>cookies</code><strong>,</strong>Cookies<strong>(Optional)</strong></summary>

Pass custom cookies to the target website. When there are multiple cookies, please separate them with a semicolon ';'.

Cookie is a small piece of data stored by a website on a user's device through a web browser. Cookies allow websites to retain user information, such as login status, preferences, or tracking details, thereby improving and personalizing the browsing experience.

When making a request, you can add cookies in two ways:

1. **In the `headers` parameter**: Send them as part of the `Cookie` header in the following format:\
   （e.g.：`headers:{"name":"Cookie","value":"cookie_name_1=cookie_value_1"}`)
2. **Using the dedicated `cookies` parameter**: Pass them directly in this format:\
   （e.g.：{"`"Cookie`":"`cookie_name_1=cookie_value_1`"}）

**Example Request:**\
Using the dedicated `cookies` parameter: Pass them directly in this format:\
`cookie_name_1=cookie_value_1; cookie_name_2=cookie_value_2`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=token" \
 -d 'headers=[{"name":"name1","value":"value1"}]' \
 -d 'cookies=[{"name":"name2","value":"value2"}]'
```

</details>

For assistance, contact [support@thordata.com](https://mailto:support@thordata.com/).