converting std::queue to a native python type

[2bndy5]

I have a library with a class that exposes a std::queue member in the public API. I've been reading the docs about converting STL containers, but most end with a "examine our test cases' src code for detailed example of..." The test cases end up looking very much like the brief example snippets in the docs, or they're specifically designed to identify regressions.

So, before I go off the into deep end trying various methods of converting std::queue object to a python type, I figured I'd ask:
Has this been done before?
How would I go about doing this?

The resulting python type does not need to be a live reference to the C++ side, but it would help future proof the binding that I'm working on.

[Skylion007]

If you just want to cast a queue to a list, you can just add a typecaster or extend the typecaster in stl.h:

pybind11/include/pybind11/stl.h

Line 175 in ffa3468

template <typename Type, typename Alloc> struct type_caster<std::deque<Type, Alloc>>

Interestingly, we have a caster for std::deque, but not std::queue. Apparently, stack and queue are usually implemented as restricted container adapters of deque. I suppose we should consider adding them:

pybind11/include/pybind11/stl.h

Line 175 in ffa3468

template <typename Type, typename Alloc> struct type_caster<std::deque<Type, Alloc>>

[2bndy5]

Yeah, I was considering making a feature request, but I don't know the src code well enough to make a proposal. I'm not sure if there's also a standards compliance reason for the lack of caster for std::queue.

BTW, that was an awesome response time! I'll try this suggestion and report back.

[2bndy5]

I hit another more advanced problem. The type of elements in the queue are also wrapped:

// in src.cpp
std::queue<RF24NetworkFrame> external_queue;

Where I had to wrap RF24NetworkFrame (into a derived class named RF24NetworkWrapper) because the frame's buffer can accept bytes or bytearray python types which don't easily translate to a uint8_t* buffer.

Using

namespace pybind11 {
namespace detail {

    template<typename Type, typename Alloc>
    struct type_caster<std::queue<Type, Alloc>> : list_caster<std::queue<Type, Alloc>, Type>
    {
    };
} // namespace detail
} // namespace pybind11

// later in binding
.def_readwrite("external_queue", &RF24NetworkWrapper::external_queue)

produced in error saying

      In file included from /mnt/path/to/pyRF24/src/pyrf24/pyRF24Network.cpp:5:
      /mnt/path/to/pyRF24/pybind11/include/pybind11/stl.h: In instantiation of ‘static pybind11::handle pybind11::detail::list_caster<Type, Value>::cast(T&&, pybind11::return_value_policy, pybind11::handle) [with T = const std::queue<RF24NetworkFrame>&; Type = std::queue<RF24NetworkFrame>; Value = RF24NetworkFrame]’:
      /mnt/path/to/pyRF24/pybind11/include/pybind11/pybind11.h:232:43:   required from ‘void pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = RF24Network; D = std::queue<RF24NetworkFrame>; Extra = {char [138]}; type_ = RF24NetworkWrapper; options = {}]::<lambda(const type&)>; Return = const std::queue<RF24NetworkFrame>&; Args = {const RF24NetworkWrapper&}; Extra = {pybind11::is_method}]’
      /mnt/path/to/pyRF24/pybind11/include/pybind11/pybind11.h:100:9:   required from ‘pybind11::cpp_function::cpp_function(Func&&, const Extra& ...) [with Func = pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = RF24Network; D = std::queue<RF24NetworkFrame>; Extra = {char [138]}; type_ = RF24NetworkWrapper; options = {}]::<lambda(const type&)>; Extra = {pybind11::is_method}; <template-parameter-1-3> = void]’
      /mnt/path/to/pyRF24/pybind11/include/pybind11/pybind11.h:1482:22:   required from ‘pybind11::class_<type_, options>& pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = RF24Network; D = std::queue<RF24NetworkFrame>; Extra = {char [138]}; type_ = RF24NetworkWrapper; options = {}]’
      /mnt/path/to/pyRF24/src/pyrf24/pyRF24Network.cpp:462:17:   required from here
      /mnt/path/to/pyRF24/pybind11/include/pybind11/stl.h:160:9: error: no matching function for call to ‘begin(const std::queue<RF24NetworkFrame>&)’
        160 |         for (auto &&value : src) {
            |         ^~~
      In file included from /usr/include/c++/9/bits/forward_list.h:35,
                       from /usr/include/c++/9/forward_list:38,
                       from /mnt/path/to/pyRF24/pybind11/include/pybind11/detail/common.h:240,
                       from /mnt/path/to/pyRF24/pybind11/include/pybind11/pytypes.h:12,
                       from /mnt/path/to/pyRF24/pybind11/include/pybind11/cast.h:13,
                       from /mnt/path/to/pyRF24/pybind11/include/pybind11/attr.h:13,
                       from /mnt/path/to/pyRF24/pybind11/include/pybind11/pybind11.h:13,
                       from /mnt/path/to/pyRF24/src/pyrf24/pyRF24Network.cpp:1:
      /usr/include/c++/9/initializer_list:89:5: note: candidate: ‘template<class _Tp> constexpr const _Tp* std::begin(std::initializer_list<_Tp>)’
         89 |     begin(initializer_list<_Tp> __ils) noexcept
            |     ^~~~~
      /usr/include/c++/9/initializer_list:89:5: note:   template argument deduction/substitution failed:

I'm marking the initial response as the answer to the original question posed, but I've stumbled into other problems remotely related to this question's goal.