2025
-
Access S3Tables data with Grafana and Amazon Athena
Amazon S3Tables is a new way to manage Iceberg tables on AWS. Accessing the data using the Athena Console is straight forward, but making it play nice with Grafana takes a little extra work. In this blog we explain how to utilise your S3Tables Catalogs with the Grafana Athena Data Source
-
Airwaves to Data Lake in 60 Seconds!
Amazon Data Firehose can now send your data directly to Iceberg tables stored in AWS S3Tables Table Buckets! This blog post combines the Vector log shipper with its Firehose sink component to get you building data lake ready tables in no time.
-
Capture and Upload Data with Vector
Log shippers like Vector are traditionally used to transform and upload application log files to platforms like Datadog. This blog will demonstrate how they can be used as powerful data integration tools in their own right.
-
Uncovering Event Patterns in SQL with Match_Recognize
MATCH_RECOGNIZE is a powerful way to discover complex event patterns in SQL. In this blog we use Trino with DuckDB to demonstrate how to detect price cycle durations in WA Fuelwatch data
-
Big JSON documents with DuckDB
Working with large JSON documents can be challenging - especially if their size makes it difficult to understand the underlying schema. Read on to discover how DuckDB can help you cut through the noise in no time at all.
2024
-
Using Amazon Data Firehose to populate Iceberg Tables
A new destination of Amazon Data Firehose makes it easier than ever to start a data lake on AWS
-
The Case for Evidence
An introduction to Evidence.dev, code first Business Intelligence: and how to make the most from it when producing visualisations for your data project
-
Bringing the Duck(DB) to the Data Lake
A walkthrough of building a pipeline to transform semi-structured JSON data into an embeddable DuckDB database, used in an Evidence.dev visualisation
-
Using Pants to Package Your Python Lambda
A brief introduction to the Pants build system, and how it provides an alternative to other ways to develop, test and package Lambda functions, such as AWS SAM, AWS CDK, etc.
-
Aggregate Time Series Data to a Single Multi-Series Grafana Graph
Time Series data sets often involve many distinct measurements over a period of time. Showing multiple measurements on a single graph can often introduce performance issues, if multiple queries are used to retrieve data. This blog demonstrates a way to let query engines do the work and reduce the burden to Grafana from downsampling and executing multiple queries.