Connect to Google BigQuery

RE: Connect to Google BigQuery

Abhay Dalsaniya — Wed, 05 Apr 2023 14:43:30 GMT

#1. Appian can be integrated easily with any REST services.

#2. Appian offers Pre-Built Connected Systems to make it easier to connect to some of the most common system

Amazon Machine Learning
Appian RPA
Blue Prism
DocuSign
Google Cloud Natural Language
Google Cloud Translation
Google Cloud Vision
Google Drive
Google reCAPTCHA
Microsoft Azure LUIS
Microsoft Dynamics 365
Salesforce
Sharepoint

https://docs.appian.com/suite/help/23.1/Connected_System_Object.html#connected-system-types

#3. Appian supported Databases - docs.appian.com/.../System_Requirements.html

RE: Connect to Google BigQuery

sakthit0001 — Wed, 05 Apr 2023 08:14:50 GMT

Hi Abhay,

We are exploring all the options - trying more like a POC.

We have our view ready to utilize in Bigquery, hence trying the integration with BQ.

Also we are very new to Appian. Any suggestions here will be useful to us.

Thank you,

Sakthidevi

RE: Connect to Google BigQuery

sakthit0001 — Wed, 05 Apr 2023 07:51:32 GMT

Thank you Peter.

RE: Connect to Google BigQuery

Abhay Dalsaniya — Tue, 04 Apr 2023 18:18:16 GMT

Are you sure BigQuery can be used as a DataSource? AFAIK BigQuery is a REST based web service.

RE: Connect to Google BigQuery

Peter Lewis — Mon, 03 Apr 2023 19:27:52 GMT

Appian's integration connector works great for any connection over REST APIs. Here's some basic info on building the integration object: https://docs.appian.com/suite/help/latest/Create_an_Integration.html

Keep in mind thought that the majority of the work will likely be working with the BigQuery API docs to identify what configurations you need to send for authentication, parameters, etc.

RE: Connect to Google BigQuery

sakthit0001 — Mon, 03 Apr 2023 14:17:53 GMT

Hi,

We have a similar requirement to report BQ data in Appian using API.

I am new to Appian. Can you please share any documents/links which will help to establish Appian - GCP BQ connection.

Thank you

RE: Connect to Google BigQuery

Bhargavi — Sat, 22 Jan 2022 04:53:49 GMT

Sure Lewis will try that approach. Thank you!

RE: Connect to Google BigQuery

Peter Lewis — Fri, 21 Jan 2022 23:10:11 GMT

I'm not that familiar with BigQuery, but are there APIs / queries that you can run that would aggregate the data and just return the aggregated results? Then you could use those results directly in charts / grids. In general this will be much more efficient because the query will be done directly on the source system rather than pulling the data into Appian and manipulating it.

RE: Connect to Google BigQuery

Bhargavi — Fri, 21 Jan 2022 14:21:06 GMT

We are planning to generate statistics using charts

RE: Connect to Google BigQuery

Peter Lewis — Fri, 21 Jan 2022 13:48:55 GMT

But why do you need to retrieve all the data? There aren't many useful reasons to return thousands of rows of data, because you just can't display that much information in a way that makes sense to users. How are you trying to display the data on your report? I think there may be options but we'll need more context on how how you plan to use the data.

RE: Connect to Google BigQuery

Bhargavi — Fri, 21 Jan 2022 11:45:21 GMT

We can structure the API call by retrieving the specific fields that are required to generate reports but still 1 API call isn't sufficient to retrieve the entire data we may need to call the API again(many times) using page token to retrieve the data completely. We are worried if this causes any performance issues.

RE: Connect to Google BigQuery

Peter Lewis — Wed, 19 Jan 2022 17:52:48 GMT

Can you provide more context on how you want to report on this data? Do you actually have to pull in all of the data into Appian or can you structure your API call to query only the data that needs to be dispayed on the report? Integrations are the only way to connect to BigQuery, but I'm curious if you can structure your integration calls to be more efficient and only return the information you plan to display.

RE: Connect to Google BigQuery

Bhargavi — Wed, 19 Jan 2022 09:38:21 GMT

Hi Richard,

As of now, we are using Google BigQuery APIs in Appian integration.

The data is more than 50MB and it may vary(>50MB) based on response.

RE: Connect to Google BigQuery

Richard Michaelis — Tue, 18 Jan 2022 12:15:08 GMT

Hello Bhargavi,
as of now, i have no perfect formula for you.
Whats the amount of data we are talking about? MB or even GB size?
Did you try something like a record based on Web-api?
Any change to use DB constructs like ETL etc?