Prevent WebHooks from firing recursively

About

A user can create a webhook that is configured so that the trigger and webhook URL create an infinite response loop.

An example of this is https://gitlab.com/gitlab-org/security/gitlab/issues/61. https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/216 applied a method of blocking webhooks from firing for this particular scenario. However, we are still exposed to recursive webhooks being created, for example, for issue events, and others.

We would like to have blanket protection against all forms of (accidental) webhook loops.

For more context see: https://gitlab.com/gitlab-org/gitlab/-/issues/33786#note_536653486.

Proposal

Add a header to all outgoing webhook requests, the header would contain a count (initialised as 1). Before executing the webhook the same header would be checked, the webhook blocked from firing if the count was above a certain threshold, otherwise, the count in the header would be incremented for the outgoing request again.

We may be able to utilise SafeRequestStore to pass the relevant values to WebhookService.

We could revert https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/216.