Inference

XTrace SDK also provides packaged inference capabilities for multiple inference service providers

  1. OpenAI

  2. Anthropic

  3. Phala

  4. Ollama

For end to end privacy we recommand users to use locally set up Ollama for inference service. If that is not feasible, user can still achieve privacy via GPU TEE using Phala’s inference service. If inference privacy is not a concern, users can use OpenAI or Anthropic’s inference service.

Use OpenAI as Inference Provider

from xtrace_sdk.inference.client import InferenceClient

inference = InferenceClient(inference_provider="OpenAI", model_name="o1", api_key="your_api_key")
inference.query("how many r are there in the word strawberry")

For supported models, refer to the documentation of OpenAI.

Use Phala as Inference Provider through Redpill

from xtrace_sdk.inference.client import InferenceClient

inference = InferenceClient(inference_provider="redpill", model_name="DeepSeek: R1 Distill 70B", api_key="your_api_key")
inference.query("how many r are there in the word strawberry")

TODO: Add supported models and more details about Phala’s inference service here.