Data Engineering
Data engineering builds and operates the pipelines, storage, and validation systems that make data usable.
Follow databases, streaming and batch pipelines, lakehouses, and data quality. Coverage looks at how teams move, store, transform, and maintain data for analytics and AI.
Latest coverage
Articles
NewsValkey 9.2 adds forkless snapshots and replication compression
NewsDuckDB 2.0.1 LTS restores per-match updates in SQL
NewsDatabricks details interactive architectures for 70 industries
NewsMotherDuck launches experimental Excel add-in preview
NewsClickHouse previews Arrow Flight connector for Power BI
NewsDatabricks opens Genie ZeroOps public beta with pull-request workflow
NewsPolars brings distributed queries to existing Ray clusters
NewsApache Airflow 3.4.0 adds bounded task loops and AI agent workflows
NewsMotherDuck says its Azure service is live in two regions
NewsClickHouse launches managed observability service in public beta
NewsDatabricks Connects AWS HealthLake Data Through OpenSharing
NewsAWS documents SageMaker AI data preparation workflows
Videos
55:14Gen AI data chain at scale
Dmytro Spodarets·Nov 17, 2023
53:52LlamaIndex: How to use LLMs to Interface with Multiple Data Sources
Dmytro Spodarets·Aug 7, 2023
32:35Rise in the use of synthetic data for regulated industries
Dmytro Spodarets·Jul 31, 2023
55:02Why you should move to a Lakehouse
Dmytro Spodarets·May 29, 2023
47:37Webinar "The promising role of synthetic data to enable responsible innovation"
Dmytro Spodarets·Mar 2, 2023
0:00Webinar "Pachyderm in production: lessons learned" (RU)
Dmytro Spodarets·Oct 1, 2021
55:46Webinar "Re-usable pipelines for ML projects with DVC" (RU)
Dmytro Spodarets·Sep 17, 2021