alpaka: Alpaka objects do not have an "empty, invalid state"

Alpaka objects do not have an “empty, invalid state”, which prevents their direct use in various situations.

---

lifetime different than the containing object

One such case is holding a memory buffer that does not need to live as long as the holding object, but still longer than individual methods:

//  the definition or templating on Device, Queue, Acc, etc. are left out for clarity
class Producer {
public:
  Producer() : ...  // Note: the buffers cannot be initialised here, as their size is not known yet
  {
  }

  // method called by the framework to begin an asynchronous operation 
  void async(Queue const& queue, Input const& input) {

    // allocate host and device buffers
    // Note: the buffers cannot be declared here, because they need to live longer that this function scope
    auto input_size = input.size();
    auto output_size = estimate_size(input_size);
    h_input_ = input.buffer();
    d_input_ = alpaka::allocBuf<float>(device_, input_size);
    h_output_ = alpaka::allocBuf<float>(host_, output_size);
    d_input_ = alpaka::allocBuf<float>(device_, output_size);

    // async copy the input to the device
    alpaka::memcpy(queue, d_input_, h_input_, input_size);

    // launch a kernel working on device data
    auto workDiv = ...;  // e.g. based on input_size, the device properties, etc.
    alpaka::enqueue(queue, alpaka::createTaskKernel<Acc>(workDiv, kernel, alpaka::getPtrNative(d_input_), input_size, alpaka::getPtrNative(d_output_), output_size);

    // async copy the output from the device
    alpaka::memcpy(queue, h_output_, d_output_, output_size);

    // do not wait for the copy and kernel to be done
  }

  // method called by the framework only after the asynchronous operation has complete
  void then(Output & output) {
    // release the handle to the input host buffer
    h_input_.reset();  // <-- currently not possible

    // free the device buffers
    d_input_.reset();   // <-- currently not possible
    d_output_.reset();  // <-- currently not possible

    // move the result to the output data structure
    output.buffer = std::move(h_output_));
  }

private:
  alpaka::DevCpu host_;
  Device device_;

  HostBuffer<float> h_input_;
  HostBuffer<float> h_output_;
  DeviceBuffer<float> d_input_;
  DeviceBuffer<float> d_output_;
};

(this is just a mock-up example I made up on the fly to illustrate the argument; apologies for any errors or inconsistencies)

---

reference to output parameters

An other use cases are APIs that will return a value by actually taking an output parameter by reference. One example we have run into is TBB’s concurrent_queue::try_pop(https://spec.oneapi.io/versions/latest/elements/oneTBB/source/containers/concurrent_queue_cls/safe_member_functions.html#popping-elements) (but I’ve see the same pattern in other parallel libraries), that takes an argument by reference and assigns the popped value (if any) to it:

bool try_pop( value_type& value );

We are storing alpaka::Events in a concurrent_queue in order to reuse them (creating CUDA events is expensive, while reusing them is cheaper), and we would like to do something like

tbb::concurrent_queue<TEvent> events_;
...

TEvent get_event() {
  TEvent event;  // empty event
  if (events_.try_pop(event)) {
    // use the popped event
  } else {
    // create a new event
    event = ...;
  }
  return event;
}

(again, this is a very simplified mock-up)

---

As has been suggested elsewhere, it should be possible to address these use cases by wrapping (almost) all Alpaka objects in an std::optional. But if every object needs to be declared as optional to be used, maybe the current API is not a good match for our use case 😦