It’s going to have a very big hill to climb if it’s playing in a space where duckdb already has a hold. Duck has probably been my favourite technology find in the last few years. Awesome tech.
Cool! But as a data engineer I don't know when I would ever use this. Getting data into a centralized place so it can be joined and queried easily is like prio 1 for any data team.
I'm sure SREs will really love me doing expensive adhoc queries against production postgres /s
I've yet to work in enterprises big enough to have multi cloud data warehouses though, maybe it's more useful in that setting?
As a consultant data engineer (ish), I think it has potential. You're right that any company doing data analytics is gonna be prioritizing a single source of truth and a unified platform, but each one will choose a different set of tools, which I'll have to learn, install, and even teach, for each new client. If I can use this to both explore AND implement stuff for clients regardless of their underlying database, that would be a pretty significant win.
Coincidentally, I wrote an article today on how I use it for similar scenarios. It can fetch from S3, multiple databases at once, and so on.
And you get all the benefits of a database when you need to join or postprocess data from multiple sources.
https://rushter.com/blog/clickhouse-data-processing/
I’ll still check this out though.
I'm sure SREs will really love me doing expensive adhoc queries against production postgres /s
I've yet to work in enterprises big enough to have multi cloud data warehouses though, maybe it's more useful in that setting?