Skip to content

groupBy on a BigInt fails (ElectricSQL Collection) #1085

@mhornbacher

Description

@mhornbacher
  • I've validated the bug against the latest version of DB packages

Describe the bug

When grouping on a BigInt column, it seems that the JSON serializer fails here:

const keyString = JSON.stringify(key)

Stack Trace
chunk-7KYJVCJM.js?v=bd10d8e8:1981 Uncaught TypeError: Do not know how to serialize a BigInt
    at JSON.stringify (<anonymous>)
    at chunk-NR67FSGJ.js?v=bd10d8e8:2738:32
    at chunk-NR67FSGJ.js?v=bd10d8e8:1975:50
    at Array.map (<anonymous>)
    at _MultiSet.map (chunk-NR67FSGJ.js?v=bd10d8e8:1975:19)
    at MapOperator.inner (chunk-NR67FSGJ.js?v=bd10d8e8:2269:23)
    at MapOperator.run (chunk-NR67FSGJ.js?v=bd10d8e8:2181:33)
    at D2.step (chunk-NR67FSGJ.js?v=bd10d8e8:2221:10)
    at D2.run (chunk-NR67FSGJ.js?v=bd10d8e8:2229:12)
    at CollectionConfigBuilder.maybeRunGraph (chunk-NR67FSGJ.js?v=bd10d8e8:12917:27)
    at CollectionConfigBuilder.executeGraphRun (chunk-NR67FSGJ.js?v=bd10d8e8:13049:10)
    at run (chunk-NR67FSGJ.js?v=bd10d8e8:12996:23)
    at Scheduler.schedule (chunk-NR67FSGJ.js?v=bd10d8e8:8646:7)
    at CollectionConfigBuilder.scheduleGraphRun (chunk-NR67FSGJ.js?v=bd10d8e8:12992:32)
    at CollectionSubscriber.sendChangesToPipeline (chunk-NR67FSGJ.js?v=bd10d8e8:12594:34)
    at sendChanges (chunk-NR67FSGJ.js?v=bd10d8e8:12600:12)
    at CollectionSubscription.callbackWithSentKeysTracking (chunk-NR67FSGJ.js?v=bd10d8e8:6628:7)
    at CollectionSubscription.emitEvents (chunk-NR67FSGJ.js?v=bd10d8e8:6761:12)
    at CollectionChangesManager.emitEvents (chunk-NR67FSGJ.js?v=bd10d8e8:7069:20)
    at CollectionStateManager.commitPendingTransactions (chunk-NR67FSGJ.js?v=bd10d8e8:6154:22)
    at commit (chunk-NR67FSGJ.js?v=bd10d8e8:7448:24)
    at @tanstack_electric-db-collection.js?v=bd10d8e8:1558:15
    at chunk-7KYJVCJM.js?v=bd10d8e8:1978:17
    at Array.map (<anonymous>)
    at chunk-7KYJVCJM.js?v=bd10d8e8:1976:61

To Reproduce

  1. Create a table with a bigint key and a FK pointing to it.
  2. Use a subQuery to get the related count
Example

It is a bit complex to setup a minimal reproduction, if its needed I can but below is an example pulled from our code. The district_id field is a BigInt in Postgres.

Store

import { DistrictStatus, States } from "@acme/db/constants";
import { electricCollectionOptions } from "@tanstack/electric-db-collection";
import { createCollection } from "@tanstack/react-db";
import z from "zod";

const districtSchema = z.object({
  id: z.number(),
  name: z.string(),
  state: z.enum([...States]),
  salesforce_id: z.string(),
  status: z.enum([...DistrictStatus]),
  updated_at: z.string(),
});

export const districtCollection = createCollection(
  electricCollectionOptions({
    shapeOptions: {
      url: `${window.location.origin}/api/sync/admin`,
      params: {
        table: "district",
        columns: Object.keys(districtSchema.shape),
      },
      liveSse: true,
    },
    schema: districtSchema,
    getKey: (item) => Number(item.id),
  })
);

const schoolSchema = z.object({
  id: z.number(),
  district_id: z.number(),
  name: z.string(),
  nces_id: z.string().optional(),
  salesforce_id: z.string().optional(),
  status: z.enum([...DistrictStatus]),
  updated_at: z.string(),
});

export const schoolCollection = createCollection(
  electricCollectionOptions({
    shapeOptions: {
      url: `${window.location.origin}/api/sync/admin`,
      params: {
        table: "district_school",
        columns: Object.keys(schoolSchema.shape),
      },
      liveSse: true,
    },
    schema: schoolSchema,
    getKey: (item) => Number(item.id),
  })
);

Component

import { count, eq, useLiveQuery } from "@tanstack/react-db";
import { createFileRoute } from "@tanstack/react-router";
import { useState } from "react";
import { useFormatter } from "use-intl";
import { Pagination } from "@/components/ui/general/pagination";
import {
  Cell,
  Column,
  Row,
  Table,
  TableBody,
  TableHeader,
} from "@/components/ui/general/table";
import { districtCollection, schoolCollection } from "@/stores/district";

export const Route = createFileRoute("/_app/admin/districts/")({
  component: RouteComponent,
});

function RouteComponent() {
  const pageSize = 35;
  const [page, setPage] = useState(1);
  const format = useFormatter();

  const { data: total } = useLiveQuery((q) =>
    q
      .from({ district: districtCollection })
      .select(({ district }) => ({ total: count(district.id) }))
  );

  const { data } = useLiveQuery(
    (q) => {
      const schoolCounts = q
        .from({ school: schoolCollection })
        .groupBy(({ school }) => school.district_id)
        .select(({ school }) => ({
          district_id: school.district_id,
          count: count(school.id),
        }));
      return q
        .from({ district: districtCollection })
        .join({ schools: schoolCounts }, ({ district, schools }) =>
          eq(district.id, schools.district_id)
        )
        .orderBy(({ district }) => district.name, "asc")
        .offset((page - 1) * pageSize)
        .limit(pageSize)
        .select(({ district, schools }) => ({
          id: district.id,
          name: district.name,
          schools: schools?.count ?? 0,
          status: district.status,
          updatedAt: district.updated_at,
          salesforce_id: district.salesforce_id,
          updated_at: district.updated_at,
        }));
    },
    [page]
  );

  console.log("total", { total, data });

  // const { data } = useSuspenseQuery(api.admin.districts.get.queryOptions());

  return (
    <>
      <div className="h-[80svh] overflow-auto">
        <Table className="">
          <TableHeader>
            <Column isRowHeader>{"Salesforce ID"}</Column>
            <Column>{"Name"}</Column>
            <Column>{"Schools"}</Column>
            <Column>{"Status"}</Column>
            <Column>{"Updated At"}</Column>
          </TableHeader>
          <TableBody items={data}>
            {(item) => (
              <Row href={`/admin/districts/${item.id}`}>
                <Cell className="font-mono">{item.salesforce_id}</Cell>
                <Cell>{item.name}</Cell>
                <Cell>{item.schools}</Cell>
                <Cell>{item.status}</Cell>
                <Cell>
                  {format.dateTime(new Date(item.updated_at), {
                    dateStyle: "short",
                    timeStyle: "short",
                  })}
                </Cell>
              </Row>
            )}
          </TableBody>
        </Table>
      </div>

      <Pagination
        currentPage={page}
        onPageChange={setPage}
        totalPages={Math.ceil((total?.[0]?.total ?? 0) / pageSize)}
      />
    </>
  );
}

Expected behavior
Aggregation should work

Desktop (please complete the following information):

  • OS: MacOS 26
  • Browser Chrome
  • Version 143.0.7499.41

Additional context

  • Tanstack Start application. Rendered in data-only mode.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions