Data Collection Client Configuration

Devices with granted access can automatically upload data to projects according to data collection rules. Collection rules apply to all devices using a shared configuration. If you need to divide collection rules by project, please refer to About Automatic Data Collection and Diagnostics to use the rule engine to meet your requirements.

About Data Collection Client Configuration

🤖 Permission: Only Organization Administrators can edit collection rules, which apply uniformly to all organization devices

Data collection configuration is managed by organization administrators and applies to devices with installed data collection clients within the organization. These devices automatically monitor data based on collection rules, and all organization devices share one set of collection configurations.

If you need to divide collection rules by project, please refer to About Automatic Data Collection and Diagnostics to use the rule engine to meet your requirements.

During collection configuration, you need to define key information such as device ID location, device monitoring directory, client initialization monitoring time range, collection directory, rule trigger topics, and client auto-update mechanism. After device access is granted, it will actively pull the organization's configured collection rule information and run dynamically on the machine end accordingly.

Entry

On the "Devices" tab of the organization management page, click the [Edit Collection Rules] button to enter the data collection client configuration editing interface. YAML language is used for editing here.

After completing the configuration, click [Save], and the configuration will take effect immediately for all organization devices with granted collection client access.

Collection Rule Format Details

Collection rules primarily configure 5 modules:

Module Name	Function Description
Data Collector Settings (collector)	Whether to delete device-end data after collection is complete
Storage Settings (mod)	Device ID storage location; monitoring directory; client initialization monitoring time range; collection directory
Device Event Properties (device)	Event attribute values
Rule Trigger Topics (topic)	Rule trigger topics
Update Settings (updater)	Whether to enable auto-update for the current program

Example template as follows:

collector:
  delete_after_interval_in_hours: 48 # Default value is -1, meaning no deletion

mod:
  # mod name, default is 'default', supports monitoring files in specified device directories
  # For custom monitoring methods, please contact coScene
  name: 'default'

  conf:
    # Enable switch, true/false, enabled by default
    enabled: true

    # Assuming there's a file /home/coscene/example.yaml on the machine end with content:
    # serial_num: 1234
    #
    # Then during machine registration, it will read the serial_num field value (1234) from example.yaml as the machine's unique identifier
    sn_file: /home/coscene/example.yaml
    sn_field: serial_num

    # Device monitoring directories, used as rule monitoring directories in projects
    listen_dirs:
      - /home/bag/

    # Files won't be monitored/collected when the time difference between current time and file update time exceeds {skip_period_hours}
    skip_period_hours: 2

    # Device collection directories, used as specified directories for project data collection tasks and rule collection
    collect_dirs:
      - /home/bag/
      - /home/log/

# Assuming there's a file /home/coscene/device.yaml on the machine end with content:
# soft_version: v1.0
#
# After device rule triggering, it will read soft_version: v1.0 from device.yaml as the generated event's attribute value
# To view the event statistics panel, please contact coScene
device:
  extra_files:
    - /home/coscene/device1.yaml
    - /home/coscene/device2.yaml

# Topics serve as options for rule trigger topics in projects to narrow down rule matching scope
# Assuming there's an error_code topic
topics:
  - error_code

# Auto update
updater:
  # Auto update switch, true/false, disabled by default
  enabled: false

Let's go through each function and its usage:

Data Collector Settings (collector)

This module is used to configure whether to delete device-end data after collection is complete, effectively managing device storage. Configurable parameters include:

• delete_after_upload: Default value is false. If set to true, device-end data will be deleted after upload.

• delete_after_interval_in_hours: Default value is -1, meaning no deletion; if set to a specific duration (e.g., 48), it will delete data older than that duration.

• scan_interval_in_secs: Default value is 60, meaning the folder is scanned every 60 seconds.

# Data collector settings
collector:
  delete_after_upload: true # Default value is false
  delete_after_interval_in_hours: 48 # Default value is -1, meaning no deletion; otherwise will delete data older than 48 hours
  scan_interval_in_secs: 60 # Default value: 60, scans folder every 60s

Storage Settings (mod)

Mainly responsible for configuring device-end data storage location information:

• name: Default name is default, supports monitoring files in specified device directories. For custom monitoring methods, please contact coScene.

• conf: Enable switch, true/false, enabled by default.

• sn_file: Assuming there's a corresponding file on the machine end (e.g., /home/coscene/example.yaml) containing a unique device identifier (e.g., serial_num: 1234), during machine registration, it will read the specified field value (1234) as the machine's unique identifier.

• sn_field: Corresponding identifier field name.

• listen_dirs: Device monitoring directories, used as rule monitoring directories in projects.

• collect_dirs: Device collection directories, used as specified directories for project data collection tasks and rule collection.

mod:
  # mod name, default is 'default', supports monitoring files in specified device directories
  # For custom monitoring methods, please contact coScene
  name: 'default'

  conf:
    # Enable switch, true/false, enabled by default
    enabled: true

    # Assuming there's a file /home/coscene/example.yaml on the machine end with content:
    # serial_num: 1234
    #
    # Then during machine registration, it will read the serial_num field value (1234) from example.yaml as the machine's unique identifier
    sn_file: /home/coscene/example.yaml
    sn_field: serial_num

    # Device monitoring directories, used as rule monitoring directories in projects
    listen_dirs:
      - /home/bag/

    # Files won't be monitored/collected when the time difference between current time and file update time exceeds {skip_period_hours}
    skip_period_hours: 2

    # Device collection directories, used as specified directories for project data collection tasks and rule collection
    collect_dirs:
      - /home/bag/
      - /home/log/

Device Event Properties (device)

Assuming there's a specific file on the machine end (e.g., /home/coscene/device.yaml with content soft_version: v1.0), after device rule triggering, it will read the specified content (e.g., soft_version: v1.0) as the generated event's attribute value. Additional related files can be configured through extra_files.

# Assuming there's a file /home/coscene/device.yaml on the machine end with content:
# soft_version: v1.0
#
# After device rule triggering, it will read soft_version: v1.0 from device.yaml as the generated event's attribute value
# To view the event statistics panel, please contact coScene
device:
  extra_files:
    - /home/coscene/device1.yaml
    - /home/coscene/device2.yaml

Rule Trigger Topics (topic)

Topics serve as options for rule trigger topics in projects, helping narrow down rule matching scope and improve monitoring efficiency. For example, assuming there's an error_code topic:

# Topics serve as options for rule trigger topics in projects to narrow down rule matching scope
# Assuming there's an error_code topic
topics:
  - error_code

Update Settings (updater)

Used to configure whether to auto-update the data collection client. Default is false (no auto-update). If changed to true, the device will automatically update when connected to the internet after we release a new version of the data collection client.

# Auto update
updater:
  # Auto update switch, true/false, disabled by default
  enabled: false

---

Through these detailed steps and instructions, administrators can comprehensively and accurately configure the device data collection client, ensuring efficient operation of device data collection.

If you have any questions, please feel free to contact us for support.