Webhook Integration

Webhook Integration Feature Description： This feature is used to send notifications when specific events occur in a designated Web Scraper API crawling task. It allows you to dynamically add data to the notification payload.

Integration Feature Name: Used to define the name of the integration task. For easier management, it is recommended that you customize the name.

Event Types: Creation Succeeded: Successfully created a new scraping task. Creation Failed: Failed to create a scraping task. Running: The scraping task is actively scraping. Task Succeeded: The scraping task successfully returns data. Task Failed: The scraping task returns no content.

Send Notification: You can send notifications using HTTP request operations. This operation sends an HTTP POST request with a JSON payload to a specified URL. The payload is defined using a payload template, which is a JSON-like syntax that allows you to use variables enclosed in double curly braces {{variable}}. This allows dynamic injection of data into the payload when the webhook is triggered.

1. Payload: The payload template is a JSON-like string that allows you to customize the payload structure and inject dynamic data that is only known at the time of the Webhook call. Besides variables, the string must also be valid JSON format. Variables must be enclosed in double curly braces and can only use the predefined variables listed in the "Available Variables" section. Using any other variables will result in validation errors. The syntax for variables is: {{oneOfAvailableVariables}}. Variables support dot notation for accessing nested properties: {{variable.property}}.

① Default Payload:

{
"userId": "{{userId}}",
"createdAt": "{{createdAt}}",
"eventType": "{{eventType}}",
"eventData": "{{eventData}}",
"resource": "{{resource}}"
}

② Available Variables: userId: string, the Thordata user ID that owns this webhook. createdAt: string, the ISO string format of the time when the webhook event was triggered. eventType: string, the type of the event that triggered the webhook. eventData: Object, the data associated with the event that triggered the webhook. resource: Object, the resource related to the triggered event. globals: Object, global context data available. Contains dateISO (the date the webhook event was triggered in ISO 8601 format) and dateUnix (the date of the event in Unix timestamp format, in seconds). Customizable variables, for example: "text": "My user id is {{userId}}"

③ Example:

{
"createdAt": "2025-06-07T06:33:14Z",
"eventData": {
"actorId": "H5PMBR7kx5OMBBwS",
"actorRunId": "n6QWqyUOraU4AAvO"
},
"eventType": "ACTOR.RUN.SUCCEEDED",
"resource": {
"apiCode": 400,
"apiErrorMsg": "",
"apiKey": "289f5c53e47d616********************",
"apiResponseTime": 28000,
"apiResult": "",
"apiUrl": "",
"audioUrl": "",
"createdAt": "2025-06-07T14:33:14+08:00",
"csvUrl": "https://th-scrapers-1303252866.cos.na-siliconvalley.myqcloud.com/scrapers/thordata/2025/06/07/2e9ddee0582a40a9****************.csv",
"errorNumber": 1,
"fileSize": 1765,
"finishedAt": "2025-06-07T14:33:44+08:00",
"flow": 1765,
"jsonUrl": "https://th-scrapers-1303252866.cos.na-siliconvalley.myqcloud.com/scrapers/thordata/2025/06/07/2e9ddee0582a40a9****************.json",
"prodect_id": 3,
"subtitleUrl": "",
"successRate": 0,
"taskId": "2e9ddee0582a40a9b1f459831b1419e1",
"type": 0,
"url": "{"spider_errors":true,"spider_id":"3","spider_parameters":[{"url":"https://www.amazon.com/dp/B0DPMHRZ1W?th=1\u0026psc=1"}]}",
"userId": 3406,
"userName": "Thor8444****",
"videoUrl": ""
},
"userId": 3406
}

1. Header Template: The headers template is a JSON-like string in which you can add extra information to the default headers of the webhook request. You can pass variables (including string interpolation and available variables) just like in the payload template. The generated headers must be a valid JSON object, and the values must be strings. Note that the following keys are hardcoded and will always be overwritten: Available Variables: Host: Request URL. Content-Type: application/json. Webhook: Tasks value. Webhook-Dispatch-Id: Tasks id Request-Origin.

If you need further assistance, please contact us via email at [e-mail].