How much to Query when displaying a few rows at a time in a read-only grid?

You can use the manual paging option in the grid field configuration.
Refer the tutorial given in (Manual paging section)- docs.appian.com/.../read-only-grid-configuration.html

Basically, create a local!pagingInfo with start Index 1 and batch size of 25. In the load section write a query entity to query first 25 records and display them in the grid. Use the pagingSaveInto in the a!gridField to save the updated local!pagingInfo and then re-query next datasubset based on this pagingInfo

I would reconsider the design as paging through 100,000 rows with a page size of 25 would require 4,000 pages, which probably won't be a good user experience. You might want to consider adding some sort of searching or filtering functionality if you have to be able to display this data in Appian (or find an alternative design to meet your requirement).

Manual paging shouldn't be necessary in this case - the standard method of adding the query to the data parameter and using fv!pagingInfo should allow you to query 25 items at a time.

Here's an example: 

a!gridField(
    label: "Read-only Grid",
    labelPosition: "ABOVE",
    data: a!queryEntity(
      entity: cons!EMPLOYEE_ENTITY,
      query: a!query(
        selection: a!querySelection(
          columns: {
            a!queryColumn(
              field: "id"
            ),
            a!queryColumn(
              field: "firstName"
            ),
            a!queryColumn(
              field: "lastName"
            ),
            a!queryColumn(
              field: "department"
            ),
            a!queryColumn(
              field: "startDate"
            )
          }
        ),
        pagingInfo: fv!pagingInfo
      ),
      fetchTotalCount: true
    ),
    columns: {
      a!gridColumn(
        label: "First Name",
        sortField: "firstName",
        value: fv!row.firstName
      ),
      a!gridColumn(
        label: "Last Name",
        sortField: "lastName",
        value: fv!row.lastName
      ),
      a!gridColumn(
        label: "Department",
        sortField: "department",
        value: fv!row.department
      ),
      a!gridColumn(
        label: "Start Date",
        sortField: "startDate",
        value: fv!row.startDate,
        align: "END"
      )
    },
    pagesize: 25,
    initialsorts: {
      a!sortInfo(
        field: "lastName",
        ascending: true
      )
    }
)

dannyh0001 said:

However, since the read-only grid does not have a totalCount parameter, I am unsure of how I can tell the read-only grid that there are still more rows to be queried and displayed.

Just to be clear here, I believe you're misunderstanding how the [new style] read-only grid works. 

It's correct that it doesn't have a "totalCount" parameter any more, but it's incorrect that this means that it doesn't account for this. 

In the 19.2+ style grid, you would use the DATA parameter for this - at which point you would either pass in a valid query that returns a dataSubset, or a local variable storing a datasubset.  Both of these will contain a "totalCount" within the data subset, and the Grid component uses that in order to determine the current max size.  Note that your query will need to have "fetch total count" turned on.