Hi. I'm a novice so as much help as possible would be appreciated! I have managed to work out how to extract the data I want via an API, however, there's too much of it and I get a memory error when setting the batch size to -1. Smaller batches work fine, so:1. How do I loop the code below so that it returns smaller batches (1000)?2. How do I merge these batches back into one file as would happen in a single call to the endpoint?Thanks in advance,
Adrian
a!localVariables( local!entities: a!queryEntity( entity: cons!TDW_REVIEWERS_ANSWER_DS, query: a!query( selection: a!querySelection( columns: { a!queryColumn(field: "respondedDate"), a!queryColumn(field: "schemeId"), a!queryColumn(field: "answer"), a!queryColumn(field: "comment"), a!queryColumn(field: "reviewer") } ), logicalExpression: a!queryLogicalExpression( operator: "AND", filters: { a!queryFilter( field: "schemeId", operator: ">=", value: 70000 ), a!queryFilter(field: "comment", operator: ">", value: " "), a!queryFilter( field: "comment", operator: "<>", value: "Test" ) }, ignoreFiltersWithEmptyValues: true ), pagingInfo: a!pagingInfo(startIndex: 1, batchSize: - 1) ), fetchTotalCount: true ).data, a!httpResponse( headers: { a!httpHeader( name: "Content-Type", value: "application/json" ) }, body: a!toJson(value: local!entities) ) )
Discussion posts and replies are publicly visible
Hi, how many records you have in your table? i would have tried execute stored procedure function to get the data, but again it totally depends upon the use case and the amount of the data.
At the moment I think it's about 5000.I need to be able to get the data into PowerBI which I do for other tables using PowerQuery and API endpoints.Thanks.
for 5000 records , you can use queryRecordType but for more than 5000 records you can use function executestoredprocedure(). But you would need to write a procedure in your db.
Thanks but with my limited knowledge, they are not going to be possible.