ratelimit

Rolling-window commit-rate budget for the Vulcan sync tier.

HuggingFace caps direct commits at 100 / hour per repo. Vulcan’s sync tier maintains a rolling-window counter (default ceiling: 90 commits/hour, leaving a 10-commit safety margin) and refuses to create_commit while the window is full. The budget is persisted to disk so multiple sync invocations (cron jobs, daemon restarts) share the same accounting.

This module also exposes sleep_with_backoff(), an exponential- backoff helper that honours Retry-After headers on HTTP 429 / 503 responses (whichever the sync tier classifies as rate-limit-like).

class stringforge.vulcan.ratelimit.CommitBudget(state_path, *, budget=90, window_s=3600.0)

Bases: object

Process-safe wrapper around a CommitWindow persisted to disk. Multiple processes calling reserve() against the same state file see a consistent view modulo flush latency. Within a single process the wrapper is thread-safe.

Note: this is advisory rate limiting at the process boundary. The state file ({state_path}) is written atomically per-process, but two processes hitting the same file concurrently can race; the budget can therefore overshoot by up to (number of concurrent writers - 1) commits per window. The cluster best-practice (see section “Best practices for cluster runs” in the docs) is to run a SINGLE head-node sync process, which makes this advisory-only guarantee tight in practice.

class stringforge.vulcan.ratelimit.CommitWindow(budget=90, window_s=3600.0, timestamps=<factory>)

Bases: object

Rolling-window record of recent commit timestamps for a single repo. Persisted as a JSON file under the staging directory.

budget

Maximum commits permitted within window_s.

window_s

Window length in seconds.

timestamps

Monotonic-clock timestamps of commits recorded so far. Older entries are pruned lazily on every reserve() call.

n_recent(now=None)

Count commits inside the current rolling window.

Parameters:

now (Optional[float]) – Wall-clock override for tests. Defaults to time.time().

Returns:

• int – Number of timestamps within the last window_s

• seconds.

Return type:

int

remaining(now=None)

Compute the number of commits still available within the current rolling window.

Parameters:

now (Optional[float]) – Wall-clock override for tests.

Returns:

int – Non-negative slot count.

Return type:

int

reserve(now=None)

Attempt to record a new commit at now.

Parameters:

now (Optional[float]) – Wall-clock override for tests.

Returns:

• bool – True if the slot was recorded; False if the

• budget is exhausted.

Return type:

bool

seconds_until_slot(now=None)

Compute the wall-clock delay before a new slot opens.

Parameters:

now (Optional[float]) – Wall-clock override for tests.

Returns:

• float – 0.0 if a slot is currently free; otherwise the

• seconds until the oldest recorded commit ages out of the

• window.

Return type:

float

stringforge.vulcan.ratelimit.DEFAULT_BUDGET: int = 90

Default budget consumed by a single sync run. 10-commit margin below the HF cap leaves head-room for ad-hoc operations the user may perform manually (e.g. README edits).

stringforge.vulcan.ratelimit.DEFAULT_WINDOW_S: float = 3600.0

Default window over which the budget applies (seconds).

stringforge.vulcan.ratelimit.HF_COMMITS_PER_HOUR_CAP: int = 100

Hard ceiling enforced by HuggingFace. Our default budget sits a safety margin below this.

stringforge.vulcan.ratelimit.INITIAL_BACKOFF_S: float = 1.0

Initial backoff for the exponential-backoff helper, in seconds.

stringforge.vulcan.ratelimit.MAX_BACKOFF_S: float = 300.0

Cap on a single backoff sleep. Five minutes is enough to ride out most transient HF outages; anything longer should error.

stringforge.vulcan.ratelimit.parse_retry_after(value)

Parse an HTTP Retry-After header value into seconds.

The header may be either an integer (seconds) or an HTTP-date. Only the integer form is honoured here – HF returns seconds. Unparseable values return None.

Return type:

Optional[float]

stringforge.vulcan.ratelimit.sleep_with_backoff(attempt, *, retry_after_s=None, initial=1.0, cap=300.0, sleep_fn=<built-in function sleep>)

Sleep for an exponentially-backed-off interval, honouring Retry-After when provided.

Parameters:

• attempt (int) – 0 for the first retry, 1 for the next, …

• retry_after_s (Optional[float]) – Server-reported retry-after (HF returns this on 429). When supplied, the sleep duration is the maximum of this and the exponential value – never less than what the server asked for.

• initial (float) – Initial backoff (seconds).

• cap (float) – Maximum sleep duration (seconds).

• sleep_fn (Callable[[float], None]) – Override for tests.

Returns:

Actual sleep duration in seconds.

Return type:

float