Hi,
We have a requirement to create reports using BigQuery data. If we use API to access BigQuery the data is very huge and we are unable to process it or we need to call the integration multiple times to retrieve the data. Instead, we are trying to configure BigQuery as a data source.
Do we have any plugins or a way to connect to Google BigQuery? Please let us know.
Thanks,
Bhargavi.
Discussion posts and replies are publicly visible
Can you provide more context on how you want to report on this data? Do you actually have to pull in all of the data into Appian or can you structure your API call to query only the data that needs to be dispayed on the report? Integrations are the only way to connect to BigQuery, but I'm curious if you can structure your integration calls to be more efficient and only return the information you plan to display.
We can structure the API call by retrieving the specific fields that are required to generate reports but still 1 API call isn't sufficient to retrieve the entire data we may need to call the API again(many times) using page token to retrieve the data completely. We are worried if this causes any performance issues.
But why do you need to retrieve all the data? There aren't many useful reasons to return thousands of rows of data, because you just can't display that much information in a way that makes sense to users. How are you trying to display the data on your report? I think there may be options but we'll need more context on how how you plan to use the data.
We are planning to generate statistics using charts
I'm not that familiar with BigQuery, but are there APIs / queries that you can run that would aggregate the data and just return the aggregated results? Then you could use those results directly in charts / grids. In general this will be much more efficient because the query will be done directly on the source system rather than pulling the data into Appian and manipulating it.
Sure Lewis will try that approach. Thank you!