Skip to content

Broken part with force deduplicate_insert_select and LWU #102103

@PedroTadim

Description

@PedroTadim

Describe the bug

Not sure if this is a real issue. It took me some time to reduce the input. I think it's a race condition.

How to reproduce

With --ignore-error run issue.sql

Error message and/or stacktrace

Stack trace:

2026.04.08 15:45:42.230995 [ 1118031 ] {96af32fa-fd13-41cf-8549-83eccbd0725b} <Error> default.t0 (f8b34dd4-38de-4802-a38f-518f80729c0f): Part /var/lib/clickhouse/store/f8b/f8b34dd4-38de-4802-a38f-518f80729c0f/patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27/ is broken and needs manual correction. Reason: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/store/f8b/f8b34dd4-38de-4802-a38f-518f80729c0f/patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27/partition.dat: , errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. contrib/llvm-project/libcxx/include/__exception/exception.h:113:14: Poco::Exception::Exception(String&&, int) @ 0x00000000234e4fb3
1. src/Common/Exception.cpp:139:7: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000137dffa9
2. src/Common/Exception.h:171:100: DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000c79b1d4
3. src/Common/Exception.h:57:54: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000c79acd1
4. src/Common/ErrnoException.h:42:11: DB::ErrnoException::ErrnoException<String const&>(int, int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x00000000138f3f87
5. src/Common/ErrnoException.h:57:18: void DB::ErrnoException::throwFromPath<String const&>(int, String const&, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x00000000138f3e42
6. src/IO/OpenedFile.cpp:34:9: ? @ 0x0000000016f29230
7. src/IO/OpenedFile.cpp:42:9: DB::OpenedFile::getFD() const @ 0x0000000016f29262
8. src/IO/ReadBufferFromFile.h:101:20: DB::ReadBufferFromFilePReadWithDescriptorsCache::ReadBufferFromFilePReadWithDescriptorsCache(String const&, unsigned long, int, char*, unsigned long, std::optional<unsigned long>, std::shared_ptr<DB::IThrottler>) @ 0x0000000016f22417
9. contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:773:30: DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, bool)::$_1::operator()(unsigned long, unsigned long, int) const @ 0x0000000016f21a6c
10. src/Disks/IO/createReadBufferFromFileBase.cpp:239:15: DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, bool) @ 0x0000000016f2132b
11. src/Disks/DiskLocal.cpp:372:12: DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x0000000017e920f1
12. src/Storages/MergeTree/DataPartStorageOnDiskFull.cpp:130:31: DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001bcc181b
13. src/Storages/MergeTree/IMergeTreeDataPart.cpp:3134:37: DB::IMergeTreeDataPart::readFile(String const&) const @ 0x000000001bcf072d
14. src/Storages/MergeTree/MergeTreePartition.cpp:407:22: DB::MergeTreePartition::load(DB::IMergeTreeDataPart const&) @ 0x000000001bfb5357
15. src/Storages/MergeTree/IMergeTreeDataPart.cpp:1666:19: DB::IMergeTreeDataPart::loadPartitionAndMinMaxIndex() @ 0x000000001bd05532
16. src/Storages/MergeTree/IMergeTreeDataPart.cpp:1187:9: DB::IMergeTreeDataPart::loadColumnsChecksumsIndexes(bool, bool, bool) @ 0x000000001bd00efd
17. src/Storages/MergeTree/MergeTreeData.cpp:2016:19: DB::MergeTreeData::loadDataPart(DB::MergeTreePartInfo const&, String const&, std::shared_ptr<DB::IDisk> const&, DB::MergeTreeDataPartState, DB::SharedMutex&) @ 0x000000001bde79a5
18. src/Storages/MergeTree/MergeTreeData.cpp:2158:20: DB::MergeTreeData::loadDataPartWithRetries(DB::MergeTreePartInfo const&, String const&, std::shared_ptr<DB::IDisk> const&, DB::MergeTreeDataPartState, DB::SharedMutex&, unsigned long, unsigned long, unsigned long) @ 0x000000001bded51f
19. src/Storages/MergeTree/MergeTreeData.cpp:2222:28: void std::__function::__policy_func<void ()>::__call_func[abi:se210105]<DB::MergeTreeData::loadDataPartsFromDisk(std::vector<std::shared_ptr<DB::MergeTreeData::PartLoadingTree::Node>, std::allocator<std::shared_ptr<DB::MergeTreeData::PartLoadingTree::Node>>>&)::$_0>(std::__function::__policy_storage const*) @ 0x000000001be99c4e
20. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000014d4fa4a
21. src/Common/threadPoolCallbackRunner.h:222:13: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::function<void ()>>::enqueueAndGiveOwnership(std::function<void ()>&&, Priority, std::optional<unsigned long>)::'lambda'()::operator()() @ 0x0000000014d4f821
22. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000013974f39
23. contrib/llvm-project/libcxx/include/__type_traits/invoke.h:0: ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001397c1a0
24. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000013972343
25. contrib/llvm-project/libcxx/include/__type_traits/invoke.h:0: void* std::__thread_proxy[abi:se210105]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x000000001397988e
26. start_thread @ 0x000000000009caa4
27. __GI___clone3 @ 0x0000000000129c6c
 (version 26.4.1.653 (official build))
2026.04.08 15:45:42.231206 [ 1118031 ] {96af32fa-fd13-41cf-8549-83eccbd0725b} <Error> default.t0 (f8b34dd4-38de-4802-a38f-518f80729c0f): while loading part patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27 on path store/f8b/f8b34dd4-38de-4802-a38f-518f80729c0f/patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/store/f8b/f8b34dd4-38de-4802-a38f-518f80729c0f/patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27/partition.dat: , errno: 2, strerror: No such file or directory: Part contains files: columns.txt (125 bytes), default_compression_codec.txt (10 bytes), data.cmrk4 (0 bytes), count.txt (1 bytes), serialization.json (132 bytes), checksums.txt (172 bytes), data.bin (0 bytes), columns_substreams.txt (316 bytes), metadata_version.txt (1 bytes): Part is empty. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):

0. contrib/llvm-project/libcxx/include/__exception/exception.h:113:14: Poco::Exception::Exception(String&&, int) @ 0x00000000234e4fb3
1. src/Common/Exception.cpp:139:7: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000137dffa9
2. src/Common/Exception.h:171:100: DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000c79b1d4
3. src/Common/Exception.h:57:54: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000c79acd1
4. src/Common/ErrnoException.h:42:11: DB::ErrnoException::ErrnoException<String const&>(int, int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x00000000138f3f87
5. src/Common/ErrnoException.h:57:18: void DB::ErrnoException::throwFromPath<String const&>(int, String const&, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x00000000138f3e42
6. src/IO/OpenedFile.cpp:34:9: ? @ 0x0000000016f29230
7. src/IO/OpenedFile.cpp:42:9: DB::OpenedFile::getFD() const @ 0x0000000016f29262
8. src/IO/ReadBufferFromFile.h:101:20: DB::ReadBufferFromFilePReadWithDescriptorsCache::ReadBufferFromFilePReadWithDescriptorsCache(String const&, unsigned long, int, char*, unsigned long, std::optional<unsigned long>, std::shared_ptr<DB::IThrottler>) @ 0x0000000016f22417
9. contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:773:30: DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, bool)::$_1::operator()(unsigned long, unsigned long, int) const @ 0x0000000016f21a6c
10. src/Disks/IO/createReadBufferFromFileBase.cpp:239:15: DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, bool) @ 0x0000000016f2132b
11. src/Disks/DiskLocal.cpp:372:12: DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x0000000017e920f1
12. src/Storages/MergeTree/DataPartStorageOnDiskFull.cpp:130:31: DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001bcc181b
13. src/Storages/MergeTree/IMergeTreeDataPart.cpp:3134:37: DB::IMergeTreeDataPart::readFile(String const&) const @ 0x000000001bcf072d
14. src/Storages/MergeTree/MergeTreePartition.cpp:407:22: DB::MergeTreePartition::load(DB::IMergeTreeDataPart const&) @ 0x000000001bfb5357
15. src/Storages/MergeTree/IMergeTreeDataPart.cpp:1666:19: DB::IMergeTreeDataPart::loadPartitionAndMinMaxIndex() @ 0x000000001bd05532
16. src/Storages/MergeTree/IMergeTreeDataPart.cpp:1187:9: DB::IMergeTreeDataPart::loadColumnsChecksumsIndexes(bool, bool, bool) @ 0x000000001bd00efd
17. src/Storages/MergeTree/MergeTreeData.cpp:2016:19: DB::MergeTreeData::loadDataPart(DB::MergeTreePartInfo const&, String const&, std::shared_ptr<DB::IDisk> const&, DB::MergeTreeDataPartState, DB::SharedMutex&) @ 0x000000001bde79a5
18. src/Storages/MergeTree/MergeTreeData.cpp:2158:20: DB::MergeTreeData::loadDataPartWithRetries(DB::MergeTreePartInfo const&, String const&, std::shared_ptr<DB::IDisk> const&, DB::MergeTreeDataPartState, DB::SharedMutex&, unsigned long, unsigned long, unsigned long) @ 0x000000001bded51f
19. src/Storages/MergeTree/MergeTreeData.cpp:2222:28: void std::__function::__policy_func<void ()>::__call_func[abi:se210105]<DB::MergeTreeData::loadDataPartsFromDisk(std::vector<std::shared_ptr<DB::MergeTreeData::PartLoadingTree::Node>, std::allocator<std::shared_ptr<DB::MergeTreeData::PartLoadingTree::Node>>>&)::$_0>(std::__function::__policy_storage const*) @ 0x000000001be99c4e
20. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000014d4fa4a
21. src/Common/threadPoolCallbackRunner.h:222:13: DB::ThreadPoolCallbackRunnerLocal<void, ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>, std::function<void ()>>::enqueueAndGiveOwnership(std::function<void ()>&&, Priority, std::optional<unsigned long>)::'lambda'()::operator()() @ 0x0000000014d4f821
22. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000013974f39
23. contrib/llvm-project/libcxx/include/__type_traits/invoke.h:0: ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*>(void (ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::ThreadFromThreadPool*&&)::'lambda'()::operator()() @ 0x000000001397c1a0
24. contrib/llvm-project/libcxx/include/__functional/function.h:508:12: ? @ 0x0000000013972343
25. contrib/llvm-project/libcxx/include/__type_traits/invoke.h:0: void* std::__thread_proxy[abi:se210105]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x000000001397988e
26. start_thread @ 0x000000000009caa4
27. __GI___clone3 @ 0x0000000000129c6c
 (version 26.4.1.653 (official build))
2026.04.08 15:45:42.231289 [ 1118031 ] {96af32fa-fd13-41cf-8549-83eccbd0725b} <Error> default.t0 (f8b34dd4-38de-4802-a38f-518f80729c0f): Detaching broken part /var/lib/clickhouse/store/f8b/f8b34dd4-38de-4802-a38f-518f80729c0f/patch-f18f7271629a324b0d26b6ad0b83a6c2-all_28_28_1_27 (size: 757.00 B). If it happened after update, it is likely because of backward incompatibility. You need to resolve this manually

Metadata

Metadata

Assignees

Labels

fuzzProblem found by one of the fuzzerspotential bugTo be reviewed by developers and confirmed/rejected.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions