Multiple instances run concurrently; new instances may be created dynamically during execution. A completion condition evaluated after each instance determines when to stop. Neither the total count nor the stopping point is knowable before execution starts.
A web scraping AI begins with a single seed URL for a competitive intelligence task. It spawns a Research Agent for the seed page. That agent discovers 8 more URLs. Each spawns its own Research Agent. Each of those agents may discover further URLs. New agents are created as links are found — mid-execution, with no predetermined count.
The process does not terminate at a fixed instance count. Instead, a Quality Gate evaluates after each agent completes: "Are there still pending URLs AND has the quality threshold not been met?" If both conditions hold, the URL Discoverer is triggered again. The process terminates when no new links are pending AND the accumulated research meets the quality threshold — a condition that is unknowable at the start of execution.
| Metric | Signal |
|---|---|
| Total instances spawned per run | Tracks cost growth; spikes signal either problem complexity or runaway loop conditions |
| Quality Gate trigger count | Number of cycles before termination — high counts indicate either a difficult problem or a poorly calibrated quality function |
| URL deduplication rate | Fraction of discovered URLs already in the visited set — high deduplication rate signals circular link structure |
| Output quality score | Evaluated against a held-out test set — monitors whether the Quality Gate termination condition is well-calibrated |
| Node | What it does | What it receives | What it produces |
|---|---|---|---|
| URL Discoverer | Reads the current URL queue. Emits one activation per pending URL. XOR split: if queue is empty, routes to Research Output. XOR join: merges back-edge from Quality Gate with initial start. | URL queue (seed URL on first run; discovered URLs on subsequent runs) | URL activations for Research Agents; or termination signal |
| Research Agent | Fetches and analyzes a single URL. Extracts structured information. Appends newly discovered URLs to the shared queue. Writes findings to the research accumulator. | Single URL + research scope | Structured findings + newly discovered URLs |
| Quality Gate | Evaluates completion condition: queue empty AND quality threshold met? XOR split: routes back to URL Discoverer if more work exists; routes to Research Output if done. | Research accumulator state + URL queue depth | Continue signal or completion signal |
| Research Output | Consolidates all agent findings into the final research report. Runs only after the Quality Gate confirms termination. | Full research accumulator | Competitive intelligence report |
| Origin of Value | Where it appears | How it is captured |
|---|---|---|
| Future Cashflow | Research Output quality | Dynamic discovery reaches content that a fixed-count approach would miss. Depth of coverage scales with the problem's actual complexity, not a preset bound. |
| Conditional Action | Each Research Agent instance | Cost is unbounded without a hard cap. Each URL discovered generates potential new agents. Without a max-instances guard, a single run can consume unbounded resources. |
| Risk Exposure | Quality Gate (termination logic) | A Quality Gate that never returns true causes infinite execution. A Quality Gate that returns true too early produces a low-quality output. Both failure modes are silent — the process completes either way. |
Hard cap is mandatory. The mi max parameter (e.g., max: 100) must be enforced at the infrastructure level, not only in the Quality Gate logic. Without a hard cap, a single corrupted URL that keeps generating new URLs causes runaway execution. The cap is not an approximation — it is a safety constraint.
Page A links to Page B; Page B links to Page A. Both agents discover each other's URL and re-queue it. The queue never empties, and the Quality Gate loops forever. Fix: maintain a visited-URL set; the URL Discoverer filters out already-processed URLs before emitting activations.
The Quality Gate uses a simple word-count heuristic for quality. A few verbose but low-value pages satisfy the threshold early. The process terminates with shallow coverage. Fix: use a multi-dimensional quality signal (coverage breadth, source diversity, key entity presence) rather than a single proxy metric.
A single page containing 500 outbound links triggers 500 new Research Agents simultaneously. Resource exhaustion follows immediately. Fix: the URL Discoverer should enforce a per-cycle spawn limit; newly discovered URLs beyond the limit are queued for the next cycle.
Multiple Research Agents write to the shared URL queue and research accumulator simultaneously. Without atomic operations, two agents may both enqueue the same URL or overwrite each other's findings. Fix: use a queue with deduplication semantics (e.g., a set-backed queue); use append-only writes with unique keys for the research accumulator.
| Variant | Modification | When to use |
|---|---|---|
| Budget-Gated Discovery | Quality Gate incorporates a token/cost budget as an additional termination criterion | Cost predictability matters; the process must terminate within a compute budget regardless of quality |
| Depth-Limited Crawl | URL Discoverer tracks link depth; stops spawning beyond depth D | Link graphs decay in relevance with depth; beyond 3 hops from seed, quality drops sharply |
| Priority Queue Discovery | URL queue is priority-ordered by predicted relevance score | High-relevance content should be processed first; if the budget runs out, the most important work is already done |
| Pattern | Relationship |
|---|---|
| 60.63 MI Runtime | More constrained variant — count is fixed before execution; no mid-run spawning |
| 60.62 MI Design-Time | Most constrained variant — count is a design-time constant |
| 10.15 Evaluator-Optimizer | Quality Gate with back-edge is structurally similar; use Evaluator-Optimizer when the feedback refines a single output rather than spawning new work items |
| 20.21 Tool-Use Loop | Single-agent iterative variant — when one agent discovers and processes sequentially rather than spawning concurrent instances |