Use Connectors for Data Integration

XTrace SDK provides a flexible way to integrate with various data sources through connectors. Connectors allow you to load, process, and retrieve data from different formats and storage systems seamlessly.

Local File Connector

Local file connectors are used to load data from files stored on your local filesystem. The XTrace SDK supports various file formats, including text files, markdown, JSON, html, and more. You can use these connectors to preprocess and load data into the XTrace system. For example, to load a folder of files from your local filesystem, you can use LocalDiskConnector class:

from xtrace_sdk.connectors.local_disk import LocalDiskConnector
from xtrace_sdk.data_loaders import DataLoaderBase

# Initialize the local disk connector
connector = LocalDiskConnector(path='path/to/your/local/files')

# Initialize a data loader from executrion context
data_loader = DataLoaderBase._from_path('data/exec_context', XTraceStorage(api_key='your_api_key'))

# preprocess data from the local files
collection = connector.load_data()
index, db = data_loader.load_data_from_memory(collection)

# Dump the processed data into the XTrace system
data_loader.dump_db(db, index=index, kb_id="your_kb_id", org_id="your_org_id", meta_data=meta_data)

Google Drive Connector

TODO

Telegram Connector

TODO