How can I track the number of bytes consumed by from_cbor, (and from_bson, etc)?

[mccolljr]

I expected this to be simple, but apparently it is not?

class cbor_parser final
{
public:
    explicit cbor_parser(std::function<void(nlohmann::json)> cb) : callback_{cb} {}

    void feed(std::span<std::uint8_t> data)
    {
        // extend the scratch buffer with new data
        scratch_.insert(scratch_.back(), data.begin(), data.end());
        
        // process objects in the buffer
        while (!scratch_.empty())
        {
            try
            {
                // THIS IS PSEUDOCODE! How can I do this? I have not been able to make this work,
                // because `from_cbor` makes internal copies of the iterators if I use the iterator-based signature, 
                // and I can't find a way to pass a custom `input_adapter` implementation
                auto some_adapter = my_counting_adapter(scratch_.begin(), scratch_.end());
                auto next_value = json::from_cbor(some_adapter, /* not strict! */ false);
                scratch_.erase(scratch_.begin(), scratch_.begin() + some_adapter.bytes_consumed());
                
                // notify the consumer that a value was parsed from the stream
                callback_(next_value);
            }
            catch (const nlohmann::json::parse_error& e)
            {
                if (e.id == 110 && e.byte == scratch_.size())
                {
                    // we'll try again once we get more data
                    return;
                }

                std::rethrow_exception(std::current_exception());
            }
        }
    }

private:
    std::function<void(nlohmann::json)> callback_;
    std::deque<std::uint8_t>            scratch_;
}

[nlohmann]

Maybe this helps: #5038 (reply in thread)

[mccolljr]

Hi @nlohmann, thank you for the reply. I've actually already encountered this class (and the related other input adapter classes). The input_iterator_adapter class you shared takes ownership of the passed in iterators. It would be nice if it would take them by reference so I could introspect the begin iterator after the parsing call completes, but it doesn't at the moment.

Ideally, I am looking for a way to supply my own input adapter class. Unfortunately, I have not found any part of the public API that accepts this. The "best" solution I have found so far is to implement a custom iterator type that wraps a std::shared_ptr to a the real underlying iterator type, and then inspecting the inner iterator after parsing completes. I would like to avoid this kind of double-indirection in the production solution. Are you aware of any way to do so?