How do I download a Topic's related data?

Modeling
1

Hello community :waving_hand:
I have a client who puts a lot of effort into getting clean tables with useful logic into Omni.

For an integration with an external tool, I’d like to be able to programmatically ingest/retrieve/download/dump the dataset related to a Topic using the API.

Has anyone done something like this before, or can point me in the right direction on how to get started?

Thanks in advance!

2

If I understand correctly, Carlos you would want to use the Query API (or the corresponding Python SDK) to query and retrieve the data from that data?

Would that suffice for you? We can double check that API is enabled for you if you let us know the instance (perhaps over slack/email).

3

Thanks for the quick answer! :raising_hands: Awesome, I already have those resources on my radar—just not 100% sure how to actually use them yet.

Is the Topic materialized somewhere so I can craft a query to pull that dataset? I was thinking maybe I could write a query expression that searches by Topic—is that the right approach?

(I’m still waiting to get the access credentials. As soon as I have them, I can share the instance with you so we can check it out together.)

4

Yes in the links above there are good examples for how to query the data. You can add fields, filters, topics, sorts, etc to the request.

Currently the topic is not materialized anywhere, our semantic layer writes the query once we receive the request. However there are 2 potential options here:

  1. Dashboard/topic cubing is in development right now see demo. This opens up the option of materializing the topic in our server side cache.
  2. You could materialize the query as a new model in your transformation layer using our dbt bi-directional integration (see docs).

Perfect, when you receive credentials, email support@omni.co , and we’ll ensure the correct API endpoints are enabled for you.