Using XTrace Managed Service

Here is how to use the XTrace managed service. You can use the XTrace managed service by creating instances of the XTrace Integration and passing in your API key. Then use the XTrace Integration with data loaders and retriever to load and use your context data.

Use XTrace Storage with Data loaders

from xtrace_sdk.integrations.xtrace.storage import XTraceStorage
from xtrace_sdk.data_loaders.base import DataLoaderBase
from xtrace_sdk.crypto.paillier_client import PaillierClient
from xtrace_sdk.crypto.encryption.aes import AESClient
from xtrace_sdk.utils.embedding import Embedding
from xtrace_sdk.connectors.local_disk_connector import LocalDiskConnector


storage = XTraceStorage(api_key='...',org_id = '...')
kb_id = '...'
paillier_client = PaillierClient(embd_len=512,key_len=1024)
aes_client = AESClient("this is a super safe AES Key")
embedding = Embedding()

# Now you can use the XTrace managed service
data_loader = DataLoaderBase(embedding, aes_client, paillier_client, storage)
connector = LocalDiskConnector("path/to/your/data/files")
collection = connector.load_data()

# Load data from the collection into an encrypted index and database
index,db = data_loader.load_data_from_memory(collection)
meta_data = [{'key':'value'},...]

#stores encrypted data and index on XTrace server
await data_loader.dump_db(db,index=index,kb_id=kb_id,meta_data=meta_data)

Use XTrace Compute with Retrievers

from xtrace_sdk.integrations.xtrace.compute import XTraceCompute

compute = XTraceCompute(api_key='...',org_id = '...')

retriever = SimpleRetriever(embeddding,aes_client,paillier_client,compute)
q = "what is XTrace"

# Now you can search over ciphers stored on XTrace server with an encrypted query
ids = await retriever.nn_search_for_ids(q,k=3,meta_filter={"filter1":"value"},kb_id=kb_id)
contexts = await retriever.retrieve_and_decrypt(ids,kb_id=kb_id)

XTrace Configuration

  • Note kb_id and org_id are required to store and retrieve data on the XTrace managed service. You can get the kb_id and org_id from the XTrace dashboard.

  • XTrace managed storage service supports metadata. You can pass it as a list of dictionaries. The key of the dictionary is the metadata key and the value is the metadata value. The metadata will be stored on the XTrace managed storage service and can be used for filtering when searching for data.