How does bookmarking work/How do I keep track of how far I've read while replaying from Batch? Will you also index by date? It can take a long time to replay a lot of data; do you have any numbers on the read rates you support per topic?
> How does bookmarking work/How do I keep track of how far I've read while replaying from Batch?
We do not have any bookmarking functionality built (yet) as we currently expect folks to just tweak their search query. Each one of the events has a new id attached to it that you can query and reference during search.
> Will you also index by date?
We do! Every event has a microsecond timestamp attached to it.
> It can take a long time to replay a lot of data; do you have any numbers on the read rates you support per topic?
We've done some initial replay throughput tests and have been able to reach ~10k/s outbound via HTTP - of course, this is all _highly_ dependent on where you're located. We expect that for folks who need super high throughput, we'll probably need to be closer to them - we fully expect to have to peer with some of our customers and optimize for throughput by doing gRPC and ... batching :)
So far, we've done most of our testing on inbound and we are currently able to sustain ~50k/s (with ~5KB event size). Our inbound is able to scale horizontally and so can go waaaaay beyond 50k/s if needed.
We have a ton of service instrumentation so we've got good visibility around throughput (and thus should know well in advance as to when we're starting to hit limits).
