How does the 50K limit on rows work?

Omni currently has a 50k row limit, meaning we can display up to 50,000 rows in your workbook. By default, Omni only displays the first 1,000 rows unless you adjust the limit in your workbook tab.

However, this does not mean Omni only calculates your results based on the first 50k rows in your data. Omni uses all of the data in your query to create the output first, but we only display up to the first 50,000 output rows. If you’re familiar with SQL, it’s like adding a “LIMIT 50000” to the end of your SQL query.

For example, let’s say you’re querying a table with a billion rows (looking at you, ads data :eyes:):

  • If you run a COUNT(*) on that table, Omni will return 1,000,000,000 → Omni is still looking at all billion rows.
  • If you run a COUNT(*) grouped by date for 30 days, Omni will return only 30 rows, but the COUNT(*) will still take into account all rows in the table (so if January 1 has 1 million rows, the row for January 1 will still show 1,000,000) → Omni is still looking at all billion rows.

You might be used to returning all of your data, then constructing a visualization only on a subset of that data. This workflow is slightly different: if you think about want you want your chart to look like and return the values and aggregations you want to see there, we find that this typically doesn’t even hit the 1000 row default limit.

In other words, you’re using your (already powerful) database as the engine for transforming the larger volume of data. Then, plug that data into Omni to get some really cool visualizations! :sunglasses: