Skip to main content
bhargavim0001
bhargavim0001
January 18, 2022
Question

Connect to Google BigQuery

  • January 18, 2022
  • 14 replies
  • 0 views

Hi, 

We have a requirement to create reports using BigQuery data. If we use API to access BigQuery the data is very huge and we are unable to process it or we need to call the integration multiple times to retrieve the data. Instead, we are trying to configure BigQuery as a data source.

Do we have any plugins or a way to connect to Google BigQuery? Please let us know.

Thanks,

Bhargavi.

14 replies

richardm900458
richardm900458
January 18, 2022

Hello Bhargavi,
as of now, i have no perfect formula for you.
Whats the amount of data we are talking about? MB or even GB size?
Did you try something like a record based on Web-api?
Any change to use DB constructs like ETL etc?

bhargavim0001
bhargavim0001Author
January 19, 2022

Hi Richard,

As of now, we are using Google BigQuery APIs in Appian integration.

The data is more than 50MB and it may vary(>50MB) based on response.

peter.lewis
peter.lewis
Employee
January 19, 2022

Can you provide more context on how you want to report on this data? Do you actually have to pull in all of the data into Appian or can you structure your API call to query only the data that needs to be dispayed on the report? Integrations are the only way to connect to BigQuery, but I'm curious if you can structure your integration calls to be more efficient and only return the information you plan to display.

bhargavim0001
bhargavim0001Author
January 21, 2022

We can structure the API call by retrieving the specific fields that are required to generate reports but still 1 API call isn't sufficient to retrieve the entire data we may need to call the API again(many times) using page token to retrieve the data completely. We are worried if this causes any performance issues.

peter.lewis
peter.lewis
Employee
January 21, 2022

But why do you need to retrieve all the data? There aren't many useful reasons to return thousands of rows of data, because you just can't display that much information in a way that makes sense to users. How are you trying to display the data on your report? I think there may be options but we'll need more context on how how you plan to use the data.

S
sakthidevit0001
April 3, 2023

Hi,

We have a similar requirement to report BQ data in Appian using API.

I am new to Appian. Can you please share any documents/links which will help to establish Appian - GCP BQ connection.

Thank you

peter.lewis
peter.lewis
Employee
April 3, 2023

Appian's integration connector works great for any connection over REST APIs. Here's some basic info on building the integration object: https://docs.appian.com/suite/help/latest/Create_an_Integration.html

Keep in mind thought that the majority of the work will likely be working with the BigQuery API docs to identify what configurations you need to send for authentication, parameters, etc.

S
sakthidevit0001
April 5, 2023

Thank you Peter. 

abhayd3663
abhayd3663
April 4, 2023

Are you sure BigQuery can be used as a DataSource? AFAIK BigQuery is a REST based web service.

S
sakthidevit0001
April 5, 2023

Hi Abhay,

We are exploring all the options - trying more like a POC.

We have our view ready to utilize in Bigquery, hence trying the integration with BQ.

Also we are very new to Appian. Any suggestions here will be useful to us.

Thank you,

Sakthidevi

abhayd3663
abhayd3663
April 5, 2023

#1. Appian can be integrated easily with any REST services.

#2. Appian offers Pre-Built Connected Systems to make it easier to connect to some of the most common system

Amazon Machine Learning
Appian RPA
Blue Prism
DocuSign
Google Cloud Natural Language
Google Cloud Translation
Google Cloud Vision
Google Drive
Google reCAPTCHA
Microsoft Azure LUIS
Microsoft Dynamics 365
Salesforce
Sharepoint

https://docs.appian.com/suite/help/23.1/Connected_System_Object.html#connected-system-types

#3. Appian supported Databases - docs.appian.com/.../System_Requirements.html

Terms & ConditionsAccessibility statement