Any numbers on insertion times? #2

Activity

1. alexey-milovidov commented on Feb 1, 2025

   

   alexey-milovidov

   on Feb 1, 2025

   Member

   More actions

   Let me copy the answer from the README:

   The benchmark does not record data loading times. While it was one of the initial goals, many systems require a finicky multi-step data preparation process, which makes them difficult to compare.

   And the comments from the author, @tom-clickhouse:

   We were fighting a lot getting each system to ingest the JSON data properly, each data store, including ClickHouse had issues with different subsets of the data (parsing issues, special characters issues etc), therefore for each system individually, we take some extra steps in the data loading scripts, that makes it tricky to fairly compare ingest time.
   We are still planning to add ingest time metric though, we need to split the data preparations steps for this etc.

   here is the data massaging extra step for PostgreSQL for example. For DuckDB we split the 1million doc files into smaller chunks as if even just one doc has an issue, the whole file is discarded

   😄React with 😄1Reacted by Raj Karri
2. RajKarri commented on Feb 1, 2025

   

   RajKarri

   on Feb 1, 2025

   Author

   More actions

   Yeah. Makes sense. It's hard to set these things up before running queries. Please share if any other studies on clickhouse insert speeds (may not be related to this one). Thanks.
3. rschu1ze commented on Mar 11, 2025

   

   rschu1ze

   on Mar 11, 2025

   Member

   More actions

   Please share if any other studies on clickhouse insert speeds (may not be related to this one). Thanks.

   https://clickhouse.com/blog/clickhouse-input-format-matchup-which-is-fastest-most-efficient