# Query Parameters

####  Web Unlocker Query Parameters

Learn about the crawling parameters of Thordata's General Crawling API.

**`Token`:**\
This parameter defines the API token used for crawling and plays a decisive role in the success of your crawling.

| Parameter | Name  | Function                                   |
| --------- | ----- | ------------------------------------------ |
| token     | Token | API token for verification during crawling |

**`URL`:**\
This parameter defines the target website link for crawling, with a default value of google.com. You can also change it to links of other search engines.

| Parameter | Name | Function                        |
| --------- | ---- | ------------------------------- |
| url       | URL  | Target website URL for scraping |

**Example Request:**

Example with `token`:<mark style="color:blue;">Token ,</mark>`url`:<mark style="color:blue;"><https://www.google.com></mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True"
```

<details>

<summary><code>js_render</code> <strong>,JS Rendering(Optional)</strong></summary>

JS rendering can handle dynamically loaded content and Single - Page Applications (SPAs). It supports pages with more complex interactions and rendering requirements, so it is recommended that you enable it.

**Example Request:**

Example with `js_render`:<mark style="color:blue;">True</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True"
```

</details>

<details>

<summary><code>type</code> <strong>,Format(Optional)</strong></summary>

This parameter defines the output format of the crawling results. The options include HTML and PNG formats, with the default being HTML.

**Example Request:**

Example with `type`:<mark style="color:blue;">png</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=png" -d "js_render=True"
```

</details>

<details>

<summary><code>block_resources</code><strong>,</strong>Block<strong>(Optional)</strong></summary>

This parameter can restrict the crawling of unnecessary resources, thereby accelerating the crawling speed.

**Example Request:**

Example with `block_resources`:<mark style="color:blue;">script</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True" -d "block_resources=script"
```

</details>

<details>

<summary><code>country</code><strong>,</strong>Proxies<strong>(Optional)</strong></summary>

This parameter defines the country/region of the proxy used for crawling, with the default value being no proxy.

**Example Request:**

Example with `country`:<mark style="color:blue;">al</mark> parameter

```sh
curl -X POST https://webunlocker.thordata.com/request -H "Authorization: Bearer Token" -H "Content-Type: application/x-www-form-urlencoded" -d "url=https://www.google.com" -d "type=html" -d "js_render=True" -d "country=al"
```

</details>

<details>

<summary><code>clean_content</code><strong>,</strong>Remove<strong>(Optional)</strong></summary>

This parameter is used to remove `JS` and `CSS` content from the returned results.

**Example Request:**

Example of a parameter with `clean_content: JS,CSS`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "clean_content=js,css" \
 -d "header=False"
```

</details>

<details>

<summary><code>wait</code><strong>,</strong>Wait for (ms)<strong>(Optional)</strong></summary>

Wait for the page to load content within the specified time (in milliseconds). Maximum value: 100,000 ms\
**Example Request:**

Example of a parameter with `wait:1000`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "wait=10000"
```

</details>

<details>

<summary><code>wait_for</code> <strong>,</strong>Wait for selector<strong>(Optional)</strong></summary>

Wait for the CSS selector to load in the DOM. `wait_for` overrides `wait` (ignoring fixed time) if both are used. Maximum wait time is 30 seconds; content is returned automatically on timeout.

**Example Request:**

Example with `wait_for:.content`  parameter

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=True" \
 -d "wait=10000" \
 -d "wait_for=.content"
```

</details>

<details>

<summary><code>headers</code><strong>,</strong>Custom Headers<strong>(Optional)</strong></summary>

Pass custom request headers to the target website. When there are multiple headers, please separate them with an English comma ',';\
HTTP headers are key-value pairs separated by a colon (:). The parameters sent should be in JSON format.

**Example request:**

If you want to send `User-Agent` and `Content-Type`, it should be as follows:

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=token" \
 -d 'headers=[{"name":"name1","value":"value1"}]' \
 -d 'cookies=[{"name":"name2","value":"value2"}]'
```

</details>

<details>

<summary><code>cookies</code><strong>,</strong>Cookies<strong>(Optional)</strong></summary>

Pass custom cookies to the target website. When there are multiple cookies, please separate them with a semicolon ';'.

Cookie is a small piece of data stored by a website on a user's device through a web browser. Cookies allow websites to retain user information, such as login status, preferences, or tracking details, thereby improving and personalizing the browsing experience.

When making a request, you can add cookies in two ways:

1. **In the `headers` parameter**: Send them as part of the `Cookie` header in the following format:\
   （e.g.：`headers:{"name":"Cookie","value":"cookie_name_1=cookie_value_1"}`)
2. **Using the dedicated `cookies` parameter**: Pass them directly in this format:\
   （e.g.：{"`"Cookie`":"`cookie_name_1=cookie_value_1`"}）

**Example Request:**\
Using the dedicated `cookies` parameter: Pass them directly in this format:\
`cookie_name_1=cookie_value_1; cookie_name_2=cookie_value_2`

```sh
curl -X POST https://webunlocker.thordata.com/request \
 -H "Authorization: Bearer token" \
 -H "Content-Type: application/x-www-form-urlencoded" \
 -d "url=https://www.google.com" \
 -d "type=html" \
 -d "js_render=token" \
 -d 'headers=[{"name":"name1","value":"value1"}]' \
 -d 'cookies=[{"name":"name2","value":"value2"}]'
```

</details>

For assistance, contact [support@thordata.com](https://mailto:support@thordata.com/).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://doc.thordata.com/doc/scraping/web-unlocker/query-parameters.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.