alpaka: Alpaka objects do not have an "empty, invalid state"
Alpaka objects do not have an “empty, invalid state”, which prevents their direct use in various situations.
lifetime different than the containing object
One such case is holding a memory buffer that does not need to live as long as the holding object, but still longer than individual methods:
// the definition or templating on Device, Queue, Acc, etc. are left out for clarity
class Producer {
public:
Producer() : ... // Note: the buffers cannot be initialised here, as their size is not known yet
{
}
// method called by the framework to begin an asynchronous operation
void async(Queue const& queue, Input const& input) {
// allocate host and device buffers
// Note: the buffers cannot be declared here, because they need to live longer that this function scope
auto input_size = input.size();
auto output_size = estimate_size(input_size);
h_input_ = input.buffer();
d_input_ = alpaka::allocBuf<float>(device_, input_size);
h_output_ = alpaka::allocBuf<float>(host_, output_size);
d_input_ = alpaka::allocBuf<float>(device_, output_size);
// async copy the input to the device
alpaka::memcpy(queue, d_input_, h_input_, input_size);
// launch a kernel working on device data
auto workDiv = ...; // e.g. based on input_size, the device properties, etc.
alpaka::enqueue(queue, alpaka::createTaskKernel<Acc>(workDiv, kernel, alpaka::getPtrNative(d_input_), input_size, alpaka::getPtrNative(d_output_), output_size);
// async copy the output from the device
alpaka::memcpy(queue, h_output_, d_output_, output_size);
// do not wait for the copy and kernel to be done
}
// method called by the framework only after the asynchronous operation has complete
void then(Output & output) {
// release the handle to the input host buffer
h_input_.reset(); // <-- currently not possible
// free the device buffers
d_input_.reset(); // <-- currently not possible
d_output_.reset(); // <-- currently not possible
// move the result to the output data structure
output.buffer = std::move(h_output_));
}
private:
alpaka::DevCpu host_;
Device device_;
HostBuffer<float> h_input_;
HostBuffer<float> h_output_;
DeviceBuffer<float> d_input_;
DeviceBuffer<float> d_output_;
};
(this is just a mock-up example I made up on the fly to illustrate the argument; apologies for any errors or inconsistencies)
reference to output parameters
An other use cases are APIs that will return a value by actually taking an output parameter by reference.
One example we have run into is TBB’s concurrent_queue::try_pop(https://spec.oneapi.io/versions/latest/elements/oneTBB/source/containers/concurrent_queue_cls/safe_member_functions.html#popping-elements)
(but I’ve see the same pattern in other parallel libraries), that takes an argument by reference and assigns the popped value (if any) to it:
bool try_pop( value_type& value );
We are storing alpaka::Event
s in a concurrent_queue
in order to reuse them (creating CUDA events is expensive, while reusing them is cheaper), and we would like to do something like
tbb::concurrent_queue<TEvent> events_;
...
TEvent get_event() {
TEvent event; // empty event
if (events_.try_pop(event)) {
// use the popped event
} else {
// create a new event
event = ...;
}
return event;
}
(again, this is a very simplified mock-up)
As has been suggested elsewhere, it should be possible to address these use cases by wrapping (almost) all Alpaka objects in an std::optional
.
But if every object needs to be declared as optional to be used, maybe the current API is not a good match for our use case 😦
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 20 (19 by maintainers)
It may also be worth noting that the Microsoft standard library developers recommend using
std::optional
for the use case presented in the initial issue comment: Source.I talked with Andrea a few weeks ago about this issue. One possible solution that he proposed, which would keep alpaka’s programming pattern unchanged, was:
alpaka::Buf::invalid_state()
which creates and returns an invalid buffer (or a free function likealpaka::make_invalid<Buf>()
;alpaka::Buf::is_valid()
or a free functionalpaka::is_valid<Buf>(Buf const&)
to check if a buffer (or an object, more general) is valid.With this solution:
Any comment/idea about this proposal?