Module robots_policy 

Robots-policy guard port — recon/production URL permission check (T111). Robots-policy guard port — closes guide failure mode #9.

RobotsPolicyGuard is the consumer-owned port trait that both recon and production paths use to ask “may I fetch this URL?”. The RobotsPolicy value type lives in the domain layer and is shared; only the guard implementation (the part that actually fetches + parses robots.txt) is swappable.

Default adapters shipped by stygian-graph:

• PermissiveRobotsGuard — returns Allow for every URL. Used when the operator has explicitly opted in to IgnoreSilently or the guard has not been wired up. Safe default for unit tests.
• (Reserved for future adapters) — CachedRobotsGuard backed by a real robots.txt fetch + parse.

Structs

PermissiveRobotsGuard

No-op guard: returns RobotsDecision::Allow for every URL.

Enums

ForbidReason

Reason captured in the RobotsDecision::Forbid variant.

PolicyOutcome

The reducer’s output — what the caller should do for this URL.

Traits

RobotsPolicyGuard

Port trait: ask a guard whether a URL is permitted under the configured RobotsPolicy.

Functions

apply_policy

Apply a RobotsPolicy to a RobotsDecision. The single source of truth for how a decision maps to a concrete action.

permissive_guard

Wrap a PermissiveRobotsGuard so the result is always a fresh Arc — convenient for callers that need a stable port-owned type.

validate_guard_pair

Validate that the supplied RobotsPolicy + RobotsDecision combination is internally consistent.