alpaka: Alpaka objects do not have an "empty, invalid state"

Alpaka objects do not have an “empty, invalid state”, which prevents their direct use in various situations.


lifetime different than the containing object

One such case is holding a memory buffer that does not need to live as long as the holding object, but still longer than individual methods:

//  the definition or templating on Device, Queue, Acc, etc. are left out for clarity
class Producer {
public:
  Producer() : ...  // Note: the buffers cannot be initialised here, as their size is not known yet
  {
  }

  // method called by the framework to begin an asynchronous operation 
  void async(Queue const& queue, Input const& input) {

    // allocate host and device buffers
    // Note: the buffers cannot be declared here, because they need to live longer that this function scope
    auto input_size = input.size();
    auto output_size = estimate_size(input_size);
    h_input_ = input.buffer();
    d_input_ = alpaka::allocBuf<float>(device_, input_size);
    h_output_ = alpaka::allocBuf<float>(host_, output_size);
    d_input_ = alpaka::allocBuf<float>(device_, output_size);

    // async copy the input to the device
    alpaka::memcpy(queue, d_input_, h_input_, input_size);

    // launch a kernel working on device data
    auto workDiv = ...;  // e.g. based on input_size, the device properties, etc.
    alpaka::enqueue(queue, alpaka::createTaskKernel<Acc>(workDiv, kernel, alpaka::getPtrNative(d_input_), input_size, alpaka::getPtrNative(d_output_), output_size);

    // async copy the output from the device
    alpaka::memcpy(queue, h_output_, d_output_, output_size);

    // do not wait for the copy and kernel to be done
  }

  // method called by the framework only after the asynchronous operation has complete
  void then(Output & output) {
    // release the handle to the input host buffer
    h_input_.reset();  // <-- currently not possible

    // free the device buffers
    d_input_.reset();   // <-- currently not possible
    d_output_.reset();  // <-- currently not possible

    // move the result to the output data structure
    output.buffer = std::move(h_output_));
  }

private:
  alpaka::DevCpu host_;
  Device device_;

  HostBuffer<float> h_input_;
  HostBuffer<float> h_output_;
  DeviceBuffer<float> d_input_;
  DeviceBuffer<float> d_output_;
};

(this is just a mock-up example I made up on the fly to illustrate the argument; apologies for any errors or inconsistencies)


reference to output parameters

An other use cases are APIs that will return a value by actually taking an output parameter by reference. One example we have run into is TBB’s concurrent_queue::try_pop(https://spec.oneapi.io/versions/latest/elements/oneTBB/source/containers/concurrent_queue_cls/safe_member_functions.html#popping-elements) (but I’ve see the same pattern in other parallel libraries), that takes an argument by reference and assigns the popped value (if any) to it:

bool try_pop( value_type& value );

We are storing alpaka::Events in a concurrent_queue in order to reuse them (creating CUDA events is expensive, while reusing them is cheaper), and we would like to do something like

tbb::concurrent_queue<TEvent> events_;
...

TEvent get_event() {
  TEvent event;  // empty event
  if (events_.try_pop(event)) {
    // use the popped event
  } else {
    // create a new event
    event = ...;
  }
  return event;
}

(again, this is a very simplified mock-up)


As has been suggested elsewhere, it should be possible to address these use cases by wrapping (almost) all Alpaka objects in an std::optional. But if every object needs to be declared as optional to be used, maybe the current API is not a good match for our use case 😦

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 20 (19 by maintainers)

Most upvoted comments

It may also be worth noting that the Microsoft standard library developers recommend using std::optional for the use case presented in the initial issue comment: Source.

I talked with Andrea a few weeks ago about this issue. One possible solution that he proposed, which would keep alpaka’s programming pattern unchanged, was:

  • no addition of a default constructor for invalid objects;
  • addition of a function like alpaka::Buf::invalid_state() which creates and returns an invalid buffer (or a free function like alpaka::make_invalid<Buf>();
  • addition of a method like alpaka::Buf::is_valid() or a free function alpaka::is_valid<Buf>(Buf const&) to check if a buffer (or an object, more general) is valid.

With this solution:

  • no invalid objects are constructed by default;
  • Alpaka internally assumes that all the objects are valid, as usual;
  • the user can decide to explicitly construct invalid objects, but then he has to check whether such objects are valid or not.

Any comment/idea about this proposal?