Posts

OSI layers for the data ecosystem
Blueprint pillars for a data mesh architecture.
Georg Heiler, Martin Windisch, ChatGPT
Last updated on Jan 18, 2023 7 min read
OSI layers for the data ecosystem
Making BigData small again (and green)
Towards simpler and perhaps more energy efficient data platforms with increased developer productivity.
Georg Heiler
Last updated on Apr 3, 2022 9 min read
Making BigData small again (and green)
Comparing SQL-based streaming approaches
Comparing established and up-and-coming streaming approaches for an integrated real-time data model
Georg Heiler
Last updated on Apr 25, 2022 26 min read
Comparing SQL-based streaming approaches
SFTP sensor
Way too many data pipelines still work with SFTP file transfer. Even a modern data orchestrator needs to interface here well.
Georg Heiler, Sandy Ryza
Last updated on Apr 30, 2022 5 min read
SFTP sensor
Connector goodness from Airbyte E2E lineage
Simplify data ingestion with the plentiful connectors of Airbyte without compromising on data lineage
Georg Heiler, Sandy Ryza
Last updated on Mar 17, 2022 3 min read
Connector goodness from Airbyte E2E lineage
Scalable data pipelines from dagster with pyspark
Getting started with simple dagster pipelines.
Georg Heiler, Sandy Ryza
Last updated on Mar 17, 2022 5 min read
Scalable data pipelines from dagster with pyspark
Tame your notebooks
Include jupyter notebooks into reliable data pipelines.
Georg Heiler, Sandy Ryza
Last updated on Mar 17, 2022 4 min read
Tame your notebooks
Fully-fledged example with resources
A full example E2E
Georg Heiler, Sandy Ryza
Last updated on Apr 3, 2022 5 min read
Fully-fledged example with resources
Turning the data pipeline inside out
Flip the data pipeline to get a better notion about the things we actually care about: data assets not transformations.
Georg Heiler, Sandy Ryza
Last updated on Mar 17, 2022 7 min read
Turning the data pipeline inside out
From hello-world to simple pipelines
Getting started with simple dagster pipelines.
Georg Heiler, Sandy Ryza
Last updated on Mar 17, 2022 6 min read
From hello-world to simple pipelines