# Configuring a Data Connector

## Preliminaries

Before diving into how to configure a Data Connector, there are a few things worth keeping in mind when working with them.

* **Multiple Projects**\
  Data Connectors are created and configured on a per-project basis. If you want multiple projects to send data to the same Endpoint URL, you have to create one Data Connector per project.
* **Multiple Data Connectors**\
  All Data Connectors in a project will receive events from all the devices in that project. It is recommended to filter on event types when possible.
* **Access Control**\
  To create, update, or delete a Data Connector, your DT Studio User or [Service Account](https://docs.developer.disruptive-technologies.com/service-accounts/introduction-to-service-accounts) must have the [role](https://docs.developer.disruptive-technologies.com/service-accounts/managing-access-rights#roles-and-permissions) of Project Developer or higher.

## Enable and Disable

A Data Connector can be enabled or disabled at any time. A Data Connector that is disabled will not forward events to its Endpoint URL until it is re-enabled. Any events that were generated while the Data Connector was disabled will not be forwarded once the Data Connector is re-enabled.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then toggle the **Data Connector enabled** switch. Remember to click **Update Data Connector**.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-Me_97K37twhou8DfVpR%2F-Me_A5VF6vayWHvGFWot%2Fimage.png?alt=media\&token=dca8d794-9640-4d96-beb3-b385dff18465)
{% endtab %}

{% tab title="REST API" %}
Send a **PATCH** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>`

A request body with the following parameter should be included.

```javascript
{
    "status": "ACTIVE"
}
```

The `status` field of a Data Connector can be set to either `"ACTIVE"` or `"USER_DISABLED"`. If the Data Connector gets automatically disabled by DT Cloud, the status will be set to `SYSTEM_DISABLED`. See the [Auto-Disabling of Data Connectors](#auto-disabling-of-data-connectors) section for more information.

#### Example Usage

Using cURL with a Service Account for authentication, the following example disables the speficied Data Connector by setting `status` to `"USER_DISABLED"`.

```bash
curl -X PATCH "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>" \
    -d '{"status": "USER_DISABLED"}'
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be updated by calling the following resource method.

* [disruptive.DataConnector.update\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#update-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example disables the specified Data Connector by setting `status` to `"USER_DISABLED"`.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Disabled Data Connector by updating the status.
dcon = dt.DataConnector.update_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
    status='USER_DISABLED',
)

# Print the updated Data Connector.
print(dcon)
```

The `status` field of a Data Connector can be set to either `"ACTIVE"` or `"USER_DISABLED"`. If the Data Connector gets automatically disabled by DT Cloud, the status will be set to `SYSTEM_DISABLED`. See the [Auto-Disabling of Data Connectors](#auto-disabling-of-data-connectors) section for more information.
{% endtab %}
{% endtabs %}

{% hint style="danger" %}
Disabled Data Connectors have no at-least-once guarantee.

When a Data Connector is disabled, undelivered events and events generated after being disabled will not be sent. Re-enabling the Data Connector will **not** backfill data from the period it was disabled and must be fetched programmatically using our REST API.
{% endhint %}

#### Auto-Disabling of Data Connectors

A Data Connector will be automatically disabled by DT Cloud if it has failed consistently for an extended period of time (currently 5 days) without any success. A push-attempt from a Data Connector is considered a failure if the receiver responded with a non-2xx status code, or didn’t respond at all. A Data Connector that is disabled by DT Cloud will get the status `SYSTEM_DISABLED`.

## Filtering on Event Types

By default, all [event types](https://docs.developer.disruptive-technologies.com/concepts/events#types) are forwarded by a Data Connector. If you want to avoid unnecessary traffic on your endpoint, events can be filtered by type.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then toggle the **Forward All Events** followed by a selection. Remember to click **Update Data Connector**.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-MRKrlVUZ9ojvBDMCX98%2F-MRKsR153QutvaKaSV7E%2Fforward_specific.png?alt=media\&token=cb0b59e9-f8af-4152-babc-20f3014b3383)
{% endtab %}

{% tab title="REST API" %}
Send a **PATCH** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>`

A request body with the following parameter should be included.

```javascript
{
    "events": ["touch", "temperature", ...]
}
```

The events array takes one- or several [event types](https://docs.developer.disruptive-technologies.com/concepts/events#types).

#### Example Usage

Using cURL with a Service Account for authentication, the following example sets a filter so that only temperature- and humidity events are forwarded.

```bash
curl -X PATCH "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>" \
    -d '{"events": ["temperature", "humidity"]}'
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be updated by calling the following resource method.

* [disruptive.DataConnector.update\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#update-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example sets a filter so that only temperature- and humidity events are forwarded.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Update the Data Connector to only forward specified events.
dcon = dt.DataConnector.update_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
    event_types=[
        dt.events.TEMPERATURE,
        dt.events.HUMIDITY,
    ],
)

# Print the updated Data Connector.
print(dcon)
```

{% endtab %}
{% endtabs %}

## Including Labels

A Data Connector can optionally forward device labels along with each event. By default, no device labels will be included, but you can specify a list of label keys that should be included in each request. See the [Request Contents](https://docs.developer.disruptive-technologies.com/receiving-events#request-contents) section for more details about how these device labels are included in the requests.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then append the label keys to the list. Remember to click **Update Data Connector**.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-MRKsvFPOzNq6BaAqSme%2F-MRKwFGIes8Bkm0bEdlp%2Finclude_labels.png?alt=media\&token=e022830e-d339-4269-a909-91ccde150ad6)
{% endtab %}

{% tab title="REST API" %}
Send a **PATCH** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>`

A request body with the following parameter should be included.

```javascript
{
    "labels": ["name", "floor", ...]
}
```

#### Example Usage

Using cURL with a Service Account for authentication, the following example configures the specified Data Connector to include device labels called `"floor"` and `"customer-ID"`.

```bash
curl -X PATCH "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>" \
    -d '{"labels": ["name", "floor"]}'
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be updated by calling the following resource method.

* [disruptive.DataConnector.update\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#update-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example configured a Data Connector to include device labels called `"name"` and `"floor"`.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Update the Data Connector to include device labels.
dcon = dt.DataConnector.update_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
    labels=['name', 'floor'],
)

# Print the updated Data Connector.
print(dcon)
```

{% endtab %}
{% endtabs %}

#### DT Studio Specific Labels

DT Studio uses labels with the keys `name` and `description` as the display name and the description of a device. It is recommended that an integration uses these labels in the same way when displaying or updating a device.

## Adding HTTP Headers

Custom HTTP headers can be included with each event forwarded by a Data Connector.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then add a key- and value pairs for your custom headers. Remember to click **Update Data Connector**.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-MRKwSivHdnOdBQTupmx%2F-MRKxLfebWy6oY-fVROL%2Fcustom_headers.png?alt=media\&token=c6e577bd-4b30-45ce-b4a3-5947c6710040)
{% endtab %}

{% tab title="REST API" %}
Send a **PATCH** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>`

A request body with the following parameter should be included.

```javascript
{
    "httpConfig": {
        "headers": {
            "name-1": "value-1",
            "name-2": "value-2",
            ...
        }
    }
}
```

#### Example Usage

Using cURL with a Service Account for authentication, the following example adds the header `"my-header"` with value `"my-value"` to every event forwarded by a Data Connector.

```bash
curl -X PATCH "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>" \
    -d '{"httpConfig": {"headers": {"my-header": "my-value"}}}'
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be updated by calling the following resource method.

* [disruptive.DataConnector.update\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#update-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example adds the header `"my-header"` with value `"my-value"` to every forwarded event.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Update the Data Connector to include custom HTTP headers.
dcon = dt.DataConnector.update_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
    config=dt.DataConnector.HttpPushConfig(
        headers={
           'my-header': 'my-value',
        },
    ),
)

# Print the updated Data Connector.
print(dcon)
```

{% endtab %}
{% endtabs %}

## Signing Events

For increased security in a production integration, we recommend configuring a Signature Secret which can be used to[ verify the signed events](https://docs.developer.disruptive-technologies.com/receiving-events#verifying-signed-events) on the receiving end.

* The HTTP header of the webhook request includes a [JSON Web Token (JWT)](https://jwt.io/) that is signed with the Signature Secret.
* The JWT payload contains the checksum of the request body.

This allows you to validate the origin and the content integrity of the received request for a more secure integration. The process is two-fold and must be configured for both the Data Connector and the receiving [endpoint](https://docs.developer.disruptive-technologies.com/receiving-events#verifying-signed-events).

#### Signing Events Sent by the Data Connector

Adding a signature secret to your Data Connector will automatically sign each forwarded event.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then add a strong and unique signature secret. Remember to click **Update Data Connector**.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-MT5hP89O9yBgzrwyBcW%2F-MT5kF_zzQkxMbZu3sME%2Fdcon-sign-secret.png?alt=media\&token=d1c08186-7667-4211-a59e-3ad055bebf49)
{% endtab %}

{% tab title="REST API" %}
Send a **PATCH** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>`

A request body with the following parameter should be included.

```javascript
{
    "httpConfig": {
        "signatureSecret": "some-good-secret"
    }
}
```

#### Example Usage

Using cURL with a Service Account for authentication, the following example adds a signature secret to the specified Data Connector.

```bash
curl -X PATCH "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>" \
    -d '{"httpConfig": {"signatureSecret": "some-good-secret"}}
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be updated by calling the following resource method.

* [disruptive.DataConnector.update\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#update-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example adds a signature secret to the specified Data Connector.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Update the Data Connector with a signature secret.
dcon = dt.DataConnector.update_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
    config=dt.DataConnector.HttpPushConfig(
        signature_secret='<YOUR_SECURE_SECRET>',
    ),
)

# Print the updated Data Connector.
print(dcon)
```

{% endtab %}
{% endtabs %}

#### Verifying Signed Events

To handle incoming events that have been signed by a signature secret, see [Verifying Signed Events](https://docs.developer.disruptive-technologies.com/receiving-events#verifying-signed-events).

## Synchronize

When synchronizing a Data Connector, the most recent event of each event type of each device currently in the project will be published to the Data Connector. This can be useful when a new Data Connector has been set up, and you need to get an up-to-date snapshot of each device in the project.

{% tabs %}
{% tab title="DT Studio" %}
In [DT Studio](https://studio.disruptive-technologies.com/), click on **API Integrations -> Data Connectors**. Select the desired Data Connector, then click **Synchronize Data Connector** towards the bottom of the page.

![](https://3704330445-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MR5PbKbz-q3w3qIO6MH%2F-Me_g82RhPb8vIjqMRKT%2F-Me_gNinQZQF5YzDF4Xm%2Fimage.png?alt=media\&token=04a1dd4b-bd0b-45cf-8f1b-9aa902355235)
{% endtab %}

{% tab title="REST API" %}
Send a **POST** request to:

`https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR>:sync`

#### Example Usage

Using cURL with a Service Account for authentication, the following example syncs the specified Data Connector.

```bash
curl -X POST "https://api.d21s.com/v2/projects/<PROJECT_ID>/dataconnectors/<DATA_CONNECTOR_ID>:sync" \
    -u "<SERVICE_ACCOUNT_KEY_ID>":"<SERVICE_ACCOUNT_SECRET>"
```

{% endtab %}

{% tab title="Python API" %}
Once the package is installed and authenticated as described in the [Python API Reference](https://developer.disruptive-technologies.com/api/libraries/python/index.html), a Data Connector can be synced by calling the following resource method.

* [disruptive.DataConnector.sync\_data\_connector()](https://developer.disruptive-technologies.com/api/libraries/python/client/resources/data_connector.html#sync-data-connector)

#### Example Usage

Using our Python API with Service Account credentials for authentication, the following example syncs the specified Data Connector.

```python
import disruptive as dt

# Authenticate the package using Service Account credentials.
dt.default_auth = dt.Auth.service_account(
    key_id='<SERVICE_ACCOUNT_KEY_ID>',
    secret='<SERVICE_ACCOUNT_SECRET>',
    email='<SERVICE_ACCOUNT_EMAIL>',
)

# Synchronize the specified Data Connector.
dt.DataConnector.sync_data_connector(
    data_connector_id='<DATA_CONNECTOR_ID>',
    project_id='<PROJECT_ID>',
)
```

{% endtab %}
{% endtabs %}