Add Sigstore Rekor staging log to staging log list

[Hayden-IO]

No description provided.

[rgdd]

Any information to share about how this staging log is used and it's expected
lifetime? Mainly asking because the requested qps is 10% of the list's capacity
(and we don't yet have a higher qps list I can direct this request towards).

So just double-checking that the qps is intentional before giving an approve
tick, and I'm also trying to figure out if you're rolling this kinda staging
logs often or not so that we're not congesting this list if that's the case.

(And much appreciated for filing the 1qps request, helps us pipe clean!)

[Hayden-IO]

Ah, I might have misunderstood - I had assumed that there could be a 1-1 mapping between witness deployments and the configuration, as in the test witnesses would read from the test configuration, the new staging witnesses would read from the staging configuration, etc. If that's not true and we're recommending that witnesses read from all available lists, then this won't be needed. Our staging log will be roughly at 1-2 QPS.

[rgdd]

Ah, I might have misunderstood - I had assumed that there could be a 1-1 mapping between witness deployments and the configuration, as in the test witnesses would read from the test configuration, the new staging witnesses would read from the staging configuration, etc. If that's not true and we're recommending that witnesses read from all available lists, then this won't be needed.

Nah, to me it sounds like you're understanding is spot on. I.e., staging witnesses would typically configure staging log list(s). While testing witnesses (which are more 'yolo style') only configure the testing list.

Now we just happened to have some staging witnesses that also configure the testing lists. Once, e.g., Glasklar adds their staging witness it would probably not configure the testing log list.

Our staging log will be roughly at 1-2 QPS.

The qpd is currently set do 1 qps. Does that mean you would like to increase it? Note that exceeding the limit may result in witnesses counting that as 'abuse'.

How about the lifecycle for the staging log, how often are you rolling new ones?

[Hayden-IO]

Sorry for the delayed response!

Now we just happened to have some staging witnesses that also configure the testing lists. Once, e.g., Glasklar adds their staging witness it would probably not configure the testing log list.

Once there are more staging witnesses up that use the staging log list, we'll do the same and remove our staging log from the testing log list.

The qpd is currently set do 1 qps. Does that mean you would like to increase it? Note that exceeding the limit may result in witnesses counting that as 'abuse'.

Ah no, I was thinking QPS for requests to the log, not log to the witness QPS. We're <1 QPS, we batch every ~2s. So no, we would not go over 1 QPS. We could increase the batching time if needed.

How about the lifecycle for the staging log, how often are you rolling new ones?

Targeting every 6 months, with a brief period (~2 weeks) of overlap where both logs are live.

[rgdd]

Thanks good info!

We could increase the batching time if needed.

How high could you go? I'd be comfortable to let this into the list on a 5s batching interval right away. Lower than that I'd like to first finish discussing with Al and Filippo since we're new at this.

(Concretely what we're discussing is whether we fill up this list and then create another 10qps list, or if we should create a higher qps list right away.

For context, I've mostly been thinking about the 10 qps list as serverless and other lower-frequency logs with a little bit of latency. So that the list doesn't become full by 10 logs. And when we do add 1 qps logs, then we ought to make sure that it's a good spending of witness resources. I do think that's the case for sigstore's log though, but before I let this or anything else with 1qps in I'd like to sync properly with Al and Filippo since we haven't had this discussion properly yet.)

[Hayden-IO]

We could increase to 5s I think, need to double check request timeouts across clients. However we have tried to match staging and production as much as possible, effectively treating staging as our preprod environment, so I’d prefer if we can keep it as is.

Definitely don’t want to monopolize available quota, so let me know the preference! Happy to wait until you’ve had discussions, as long as the current witnesses continue to support the test log list.

[rgdd]

Update: we're working on getting a higher qps list added, please hold on :-).

#37

[rgdd]

Now https://staging.witness-network.org/log-list-100qps-40klogs.1 is available, and shortly there should be at least one witness pulling from this list. Can you redirect this participation requests to the higher perf list:

https://github.com/transparency-dev/witness-network/blob/main/lists/staging/log-list-100qps-40klogs.1

Thanks!

[Hayden-IO]

Done!

[rgdd]

Thanks, and sorry for the delay. This was helpful pipe cleaning!