Use Connectors for Data Integration
=================================================


XTrace SDK provides a flexible way to integrate with various data sources through connectors. Connectors allow you to load, process, and retrieve data from different formats and storage systems seamlessly.

Local File Connector
-------------------------------------------------
Local file connectors are used to load data from files stored on your local filesystem. The XTrace SDK supports various file formats, including text files, markdown, JSON, html, and more. You can use these connectors to preprocess and load data into the XTrace system.
For example, to load a folder of files from your local filesystem, you can use `LocalDiskConnector` class:

.. code-block:: python

    from xtrace_sdk.connectors.local_disk import LocalDiskConnector
    from xtrace_sdk.data_loaders import DataLoaderBase

    # Initialize the local disk connector
    connector = LocalDiskConnector(path='path/to/your/local/files')

    # Initialize a data loader from executrion context
    data_loader = DataLoaderBase._from_path('data/exec_context', XTraceStorage(api_key='your_api_key'))

    # preprocess data from the local files
    collection = connector.load_data()
    index, db = data_loader.load_data_from_memory(collection)

    # Dump the processed data into the XTrace system
    data_loader.dump_db(db, index=index, kb_id="your_kb_id", org_id="your_org_id", meta_data=meta_data)


Google Drive Connector
-------------------------------------------------
TODO 


Telegram Connector
-------------------------------------------------
TODO