blog

Pre-generating filters on static sites

Working with data on a static site can be tricky. In a scenario where data doesn't change too regularly, it would be ideal to be able to build the data into the static site. But how does that change if users are allowed to filter the dataset?

I recently started rebuilding LottoStats, a project that uses scheduled functions to scrape various UK lottery results and display them on the LottoStats website. The site uses Next.JS and Supabase and my goal was to pre-generate all pages at build time. I also wanted users to be able to filter the lottery games displayed on the results table.

Keeping everything static was easy when displaying all the data and not permitting any user interaction, however when it came to adding filters, I immediately started reaching for a client side fetch to get the filtered results. This went against my goal of all pages being pre-generated, so I looked for an alternative.

I started by considering how a server rendered site handles filtering. Filters are likely encoded in the page URL, extracted by the server, and used in a database call.

My initial mental model of a page with filters applied was that it was a subset of an "all results" page. However, by re-considering I realised that a page with filters applied could be thought of as a page in its own right. By thinking of each filtered page as a separate page I would be able to generate a static page for every combination of filters and then use routing to fake filters being toggled.

For LottoStats there are four filters each of which can either be on or off, leading to 16 possible combinations.

Next.JS helps with the implementation by giving us getStaticPaths to create pages based on URL paths. For only 16 combinations, the paths can be hardcoded using the filter query string as the param. For sites with many more combinations a catch-all getStaticPaths is likely to work better.

Next.JS passes the query string used for getStaticPaths over to getStaticProps via context automatically. getStaticProps can then use the query string to fetch the required data from the database during build.

The result of this is a set of pages with each page representing a specific combination of filters.

Now we have a set of pre-generated filtered result pages, attention can turn to getting the correct page in front of the user during runtime. Filters are typically presented to the user as a collection of buttons, so we can deal with routing in the onClick handler.

When the user clicks a filter toggle, the name of the toggle they clicked is passed into the onClick handler. We can find the current path using the Next.JS useRouter hook. By comparing the toggle that was clicked with the current path we can determine what the user is trying to achieve.

There are a few possibilities,

  • there are currently no filters applied and the user wants to add one
  • there are filters applied and the user wants to add another
  • there are filters applied and the user wants to remove one
  • all the filters are applied and the user wants to remove one

Note that for this method, I've made the filters a toggle with it only being possible to add or remove one filter at a time.

We can use a conditional to create a suitable filter string for each of these possibilities. The structure of the filter string should take the same format as the pathnames used to generate pages in getStaticPaths.

This filter string can then be passed into the Next.JS useRouter push method to direct the user to the page that matches the filters that the user is looking for.

This method of pre-generating filter pages excels where site speed is a concern. By removing a database call, filters appear to be applied very quickly. It also has the side-effect of lodging the filter query into the browser history ensuring that filters will remain in effect should the user navigate back to the filter page, somewhere SPA sites often fall short.

I had no hard requirement to keep the whole LottoStats site static, but it was fun working out a way to implement it. Check out the LottoStats source on Github.