Methodology

This is an experimental project from Cast from Clay. We are testing different ways of working with data, and integrating AI into our workflows. In this case, we want to create an easier way of engaging with the data Parliament publishes on APPGs.

Currently, the data is provided either in 1,000 page PDFs or in similarly long HTML files. It's really hard to track how things are changing over time, and to identify how different funders may be funding different things.

So, we built this dashboard. The process we followed was:

  • Received the data from the Parliamentary website and compiled this using Python into JSON files in a structure we needed.
  • Used OpenAI o3 for deep understanding and reasoning to categorise the 3,000+ APPG data entries against our desired categories.
  • Created a full-stack Next.js application in a JAMstack approach, with all data statically generated, not requiring any database or external servers. Next.js powers a highly performant, fast loading dashboard, enabling rapid searching, sorting and filtering. We used Tailwind to speed up styling and make it easier to maintain. Similarly we used TypeScript to catch mistakes earlier, making the platform easier to maintain and more reliable.

The data coming from Parliament is not well structured. Through this project we have given it structure, but there is a risk that through that parsing process some weirdnesses or inaccuracies occur. For example, we know that some APPGs have changed their name. We decided not to 'fix' that and align differently named APPGs, on the basis that that is in-line with how Parliament publishes the data.

You might also notice that we have taken data from June in 2023 and 2025 and May in 2024. This is because new editions of the Register of APPGs are published "every 6 weeks or so." We have tried to take from as close to the same point in the year across the 3 years we looked at, and in 2024 the closest Register was published on 30th May. The full dates of the Registers we have used are:

  • 2023: June 28th
  • 2024: May 30th
  • 2025: June 18th

We conceptualised the product, retrieved the data, designed, developed, and launched this dashboard within 6 weeks. If you are looking for a digital products partner, let us know (hello@castfromclay.co.uk) what you're thinking of building, and we'll tell you within 24 hours how long it will take us to build you an MVP.

The kinds of things we could build:

  • Do you want to track climate commitments made by hundreds of companies?
  • Are you trying to monitor agricultural regulations across 50 US states?
  • Are you an energy company with 50,000 business customers, and you want to showcase your contribution to energy security?
  • Are you monitoring progress made in health policy in every EU member nation?
  • Do you have thousands of academics at your university, and you want to identify research from them that may have policy implications?
  • Are you a retailer with 2,000 suppliers and 10,000 employees who wants to tell each MP about your economic contribution to their constituency?
  • Are you an NGO searching for new funders?