I have a non-synced REST API that I am using to fetch data via an integration. This endpoint supports paging. In both the record data source and the integration, when I put a a!pagingInfo(x, y), in the pagingInfo test inputs, the data increments through pages correctly according to the values I give for x and y.
But, in the data model test listing and the record list URL, the first 50 records appear (as expected) with the correct record total (more than 1,000), but when I click the next button, the next page is blank and says "No items available". I can click the previous button, and the first 50 records display again. The Paging Info drop down on the data model page is mapped to the correct rule input on the record data source.
The Appian version is 23.2.
Here is my code for the dataSubset in the record data source:
a!dataSubset( startIndex: ri!pagingInfo.startIndex, batchSize: ri!pagingInfo.batchSize, totalCount: local!integrationResponse.result.body.total_items, data: a!forEach( local!integrationResponse.result.body.items, cast( 'type!{urn:com:appian:types:YYY}YYY_MyRecordType', fv!item ) ), identifiers: a!forEach( local!integrationResponse.result.body.items, fv!item.id ) ),
I cannot seem to find anything wrong. What am I missing?
UPDATE:
The problem was that I needed to convert start index into a page number.Much appreciation for the tremendous effort to help from @Stefan Helzle
Discussion posts and replies are publicly visible
I do not see any obvious problems. Can you share more of your code? Especially I am interested in that local!integrationResponse. Starting with the integration object itself, you need to validate each and every step, making sure that the correct values go in and come out.
Two tips:
- You can cast a list of items without a foreach. Just cast to a!listType(<YOUR_RECORD_TYPE>).
- You can extract all values form a list of items without a foreach -> "local!integrationResponse.result.body.items.id"
I think I eliminated caching as the possible problem because when I set the number of records for the record listing to 1000, it lists all of those records.
Do you use it in a grid to display the data? Can you share that code?
Like the definition of the columns?
{ a!gridColumn( label: "Col1", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col1}col1', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col1}col1'] ), a!gridColumn( label: "Col2", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col2}col2', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col2}col2'] ), a!gridColumn( label: "Col3", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col3}col3', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col3}col3'] ), a!gridColumn( label: "Col4", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col4}col4', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col4}col4'] ), a!gridColumn( label: "Col5", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col5}col5', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col5}col5'] ), a!gridColumn( label: "Col6", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col6}col6', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col6}col6'] ), a!gridColumn( label: "Col7", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col7}col7', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col7}col7'] ), a!gridColumn( label: "Col8", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col8}col8', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col8}col8'] ), a!gridColumn( label: "Col9", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col9}col9', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col9}col9'] ), a!gridColumn( label: "Col10", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col10}col10', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col10}col10'] ), a!gridColumn( label: "Col11", sortField: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col11}col11', value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY_MyRecord.fields.{col11}col11'] ) }
:-) I need to see all the code, except the columns!
The grid needs to interact with your expression to read paged data from that API. There are a few way to do so and I am interested in the way how you did it.
I did not create an interface. I am just using the grid record list on the data record. I did not write the code for it.
How do I give you the code of the record list on the data record? I don't see any buttons that let me see the whole thing, just the columns. As far as I can see, Appian does not give a way to see that, but hopefully I am wrong.
OK. I understand. Then, there is no code.
Can you share the record source configuration? I mean, assuming that the API supports paging, the integration successfully applies paging, and the wrapper expression as well, the only spot left, is the source config.
Hm ... again, I do not see any obvious issues.
Can you try to create an interface and add a grid to it that uses this record? The data preview in the record has some limitations.
I created the interface, and it pulls however many records are specified in page size, but then next pages say no records. If I put 10, it pulls 10 records from the REST service for the first page. If I put 300, it pulls 300 records from the REST service. In both cases, all of the next pages give blank results.
{ a!gridField( label: "Read-only Grid", labelPosition: "ABOVE", data: 'recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY Record', pageSize: 300, columns: { a!gridColumn( label: "YYY Column", value: fv!row['recordType!{4d44effc-c462-4c70-b1aa-87905a73c395}YYY Record.fields.{some_column}some_column'] ) }, validations: {} ) }
Can you try to enable HTTP logging in the integration, do a few calls and check that the data is as expected?
I enabled logging for the integration. If I do to system logs, where do I find the logs related to these requests?
As per the documentation in integration_req_resp_activity.log
I see that now. Unfortunately, I don't have high enough access to browse that directory.
OK. That makes things complicated.
There is a plugin that allows you to write message to the tomcat-stdout log file. But you need admin permissions to install plugins.
To make sure there is no caching, please modify the code in the source expression like this:
a!refreshVariable( value: local!integrationResponse: rule!YYY_searchRecords( pagingInfo: ri!pagingInfo, searchText: ri!searchText ), refreshAlways: true ),
I am a bit confused on where exactly to put that. Where am I supposed to put it precisely?
I did an additional test as a troubleshooting measure:
I configured the data record source to put the paging info properties into one of the columns of the record to visually validate that the data record source was receiving correct paging info values. And they are valid. So that eliminates that as an issue.
Problem solved. After finding someone who has access to the logs, I quickly found the problem. The paging info has a start index giving the 1-based offset in the total number of records, while the endpoint has an offset that is actually a page number, so I needed to convert start index to a page number based on the batch size.
I am glad you got this resolved. I have been in similar situations before and validating each and every step in that chain is the route to success.
Hi Stefan Helzle Ryan Daniels , I'm facing a similar issue, In my case, the pagination itself is not visible. The total number of records are 110. If I set Rows to Display Per Page as 100, it is returning all 100 rows of data but pagination is not workingCould you please help me on this. Please find the below screenshots:1. 2. 3.
Can you check the total count:
a!queryRecordType( recordType: XXXXX, fields: {}, pagingInfo: a!pagingInfo( startIndex: 1, batchSize: 100 ), fetchTotalCount: true() ).totalCount
Yogi Patel I guess there is no issues in totalCount. The length() is returning the exact total count. However, I tried using queryRecordType as you suggested, I'm now facing below issue