Working towards returning KernelArgumentHolder/PolymorphicVal's from fusion execution. #3890
Conversation
…nelArgumentHolder::createKernelArgumentHolder in favor of a KernelArgumentHolder constructor.
…sion_cache_utils and fusion_kernel_runtime still.
…nto polymorphic_kernel_arg
…IDIA/Fuser into polymorphic_kernel_arg
…. Need to disambiguate KernelExecutor::Compile next.
Co-authored-by: Gao, Xiang <[email protected]>
|
!test |
csarofeen
force-pushed
the
polymorphic_outs_step_1
branch
from
e7bc2cf to
42a46c1
Compare
|
!test |
csrc/fusion_segmenter.cpp
Outdated
| @@ -3937,7 +3937,7 @@ SchedulerType SegmentCandidateFinder::deriveSchedulerType( | |||
| "Can not find a scheduler to schedule fusion segment"); | |||
| return scheduler_type; | |||
| } | |||
|
|
|||
| // Update the constructor implementation | |||
There was a problem hiding this comment.
Erroneous change.
csrc/runtime/executor.h
Outdated
| @@ -81,6 +81,51 @@ class KernelExecutor : public ExecutorAbstract { | |||
| CompileParams compile_params, | |||
| SchedulerType sceduler_type = SchedulerType::None); | |||
|
|
|||
| // Initializer list version that creates KernelArgumentHolder from tensors | |||
There was a problem hiding this comment.
Double check these constructors are still necessary. They get cleaned up in: step 2 (#3916) so shouldn't be a big deal, but worth checking.
There was a problem hiding this comment.
Removed one, will cleanup in followup.
There was a problem hiding this comment.
Actually, can remove all.
| @@ -22,20 +22,48 @@ | |||
|
|
|||
| namespace nvfuser { | |||
|
|
|||
| namespace detail { | |||
| template <typename T> | |||
There was a problem hiding this comment.
Double check this is still needed and we otherwise would recurse into the constructor of KernelArgumentHolder.
There was a problem hiding this comment.
@zasdfgbnm I would value a review of KernelArgumentHolder from you. I could use your advice here, please.
csrc/scheduler/runtime_info.h
Outdated
| @@ -52,10 +52,30 @@ class SchedulerRuntimeInfo : public NonCopyable { | |||
| const std::vector<TensorView*>& all_tvs = {}, | |||
| std::optional<PrimDataType> forced_index_type = std::nullopt); | |||
|
|
|||
| SchedulerRuntimeInfo( | |||
There was a problem hiding this comment.
Double check this is still needed.
csrc/utils.cpp
Outdated
| @@ -100,6 +102,40 @@ int8_t getCommonDeviceCUDA( | |||
| return found_device ? index : (int8_t)0; | |||
| } | |||
|
|
|||
| int8_t getCommonDeviceCUDA( | |||
There was a problem hiding this comment.
Check if this can be cleaned up now or if there's still calls into the c10::IValue variant.
| @@ -0,0 +1,586 @@ | |||
| # SPDX-FileCopyrightText: Copyright (c) 2024-present NVIDIA CORPORATION & AFFILIATES. | |||
There was a problem hiding this comment.
Remove from this branch, it's in another branch/PR.
| DataType array_type = | ||
| ArrayType{std::make_shared<DataType>(DataType::Float), 2}; | ||
| auto array = IrBuilder::create<Val>(array_type); | ||
| // DataType array_type = |
There was a problem hiding this comment.
@zasdfgbnm this was something I couldn't figure out how to get to work with KernelArgumentHolder, could you please advise what the right push incantation is for it?
|
!test |
No description provided.