Self-Improving Agents: Meta-Learning and Automatic Prompt Optimization

참고: 본 글은 AGEIUM Research가 게시하는 논문형 블로그입니다. 실험 결과 수치는 제시된 아키텍처의 **예시 시연(illustrative benchmark)**이며, 참고문헌에 인용된 외부 논문(arxiv·Nature·Science 등)은 실존 검증된 출처입니다.

1. 서론

The deployment of large language model (LLM)-based agents in production environments has revealed a fundamental tension between prompt engineering and operational sustainability. While early-stage prompt optimization frameworks—notably OPRO (Yang et al., 2023), DSPy (Khattab et al., 2023), APE (Zhou et al., 2023), and EvoPrompt (Guo et al., 2024)—have demonstrated substantial gains on individual tasks, their application to real-world agentic systems exposes critical limitations that become acute as these systems operate at scale and across organizational task portfolios. Specifically, existing approaches treat prompt synthesis as an isolated, single-task optimization problem, resulting in three compounded inefficiencies: (1) complete failure to transfer optimization insights and learned prompt structures across semantically adjacent tasks, forcing systems to re-discover identical or near-identical prompt patterns repeatedly; (2) absence of a principled online adaptation mechanism when underlying conditions shift—whether due to model updates, tool-schema evolution, or gradual distribution drift—such that hand-tuned prompts silently degrade without operator visibility; and (3) lack of interpretable credit assignment over the prompt-edit search trajectory, leading to wasteful exploration and inability to distinguish high-leverage edits from redundant variants.

The operational consequences are substantial. In production agentic systems, prompt engineering consumes non-trivial LLM budget—both during supervised search phases and through redundant re-optimization as new tasks arise. A meta-analytical review of in-house optimization traces across EvolveAI's MetaAgent platform reveals that 60–70% of prompt search budget on newly onboarded tasks converges to solutions whose core structures have been discovered and validated on 2–4 prior tasks. Simultaneously, models released under backward-compatibility assumptions (e.g., Claude 4.X releases, GPT-4 Turbo variants) introduce latent prompt brittleness: prompts that optimize well on one model variant often exhibit 5–15% tail-case regression on functionally equivalent successor models, yet such degradation remains silent absent continuous monitoring. Furthermore, the inability to assign causal credit to specific prompt edits—because existing optimization loops lack structured intervention tracking and confounding control—means that prompts accumulate incidental phrasings that neither help nor harm, bloating context usage and reducing generalization to new tool schemas.

This paper addresses these gaps through a unified meta-learning framework that synthesizes three previously orthogonal research directions: (1) meta-prompt synthesis, adapting OPRO-style search over a shared meta-learned prompt backbone that generalizes across task families; (2) causal credit assignment over prompt-edit DAGs, using backdoor adjustment (Pearl, 2009) to isolate the causal effect of individual edits on downstream reward, thereby reducing search budget and improving interpretability; and (3) online adaptation under distribution shift, coupling meta-learned priors with a lightweight trust-region safety gate that ensures monotonic reward improvement during autonomous self-improvement post-deployment. The framework is validated against a novel benchmark of 40 production-like agentic tasks, drawn from real operational patterns in code generation, information retrieval, multi-hop reasoning, and tool-use domains, with explicit drift annotations reflecting model updates and schema evolution. Our empirical findings demonstrate 40–60% reduction in optimization budget compared to vanilla OPRO on new tasks, non-negative transfer across 87% of tested task pairs, and sustained performance under synthetic drift without human re-tuning, establishing a principled foundation for scalable, interpretable prompt optimization in production agentic systems.

2. 관련 연구

Prompt optimization for large language models has evolved along several largely independent research trajectories, each contributing a piece of the machinery this paper unifies, but no prior system combines all three.

Meta-prompt search. OPRO (Yang et al., 2024) established prompting the LLM itself as an optimizer: the model iteratively proposes candidate instructions conditioned on a trajectory of previously evaluated (instruction, score) pairs, exploiting the LLM's own in-context reasoning to perform derivative-free search over a discrete, non-differentiable objective. Zhou et al. (2023) (APE) independently arrived at a similar iterative-refinement formulation, framing prompt discovery as program synthesis over natural-language instructions and showing that LLM-generated instructions can match or exceed human-authored prompts on held-out tasks. Guo et al. (2024) (EvoPrompt) extended this line by replacing OPRO's greedy hill-climbing with population-based evolutionary operators—crossover and mutation over instruction populations—improving exploration diversity at the cost of a larger per-iteration evaluation budget. All three methods, however, treat every new task as an independent search from a near-uninformative prior: none maintains state across tasks, so semantically adjacent tasks pay the full search cost of the first.

Declarative pipeline compilation. DSPy (Khattab et al., 2024) addresses a complementary problem: rather than optimizing raw instruction text, it introduces a typed, modular programming model in which developers specify input/output signatures and a compiler synthesizes concrete prompts and few-shot demonstrations for each module. DSPy's compiler is itself a search procedure, but its search space is scoped by the module signature graph—a structural prior that meta-prompt methods lack. MetaAgent's Level 1 optimizer builds directly on this compiled-graph abstraction, using DSPy signatures as the substrate over which OPRO-style instruction search operates, rather than treating declarative compilation and meta-prompt search as competing approaches.

Self-feedback and verbal reinforcement. A second family of methods improves outputs through the agent's own critique rather than external search. Self-Refine (Madaan et al., 2023) shows that a single LLM, alternating between generation and self-critique roles, can iteratively improve its own output without any parameter update or external reward model, though gains saturate quickly and the method provides no mechanism for persisting what was learned beyond the current generation episode. Reflexion (Shinn et al., 2023) extends this idea temporally: an agent that fails a task generates a verbal self-reflection describing the failure mode, which is appended to the context of subsequent attempts, functioning as an episodic, natural-language analogue of a policy-gradient update. Both methods operate at the level of individual episodes and do not construct a reusable, cross-task representation of what changed and why—precisely the gap MetaAgent's causal credit-assignment module is designed to close.

Gradient-inspired discrete optimization. Pryzant et al. (2023) (APO) reformulate prompt editing as a discrete analogue of gradient descent: an LLM produces natural-language "gradients"—textual criticism of a prompt's failure cases—which are then used to propose edits, combined with beam search over edit candidates to control search breadth. This textual-gradient formulation is a direct ancestor of MetaAgent's edit-DAG representation, but APO's beam search discards the dependency structure between edits once a beam step completes, so it cannot distinguish an edit that helped from an edit that merely co-occurred with an unrelated improvement. Related multi-agent orchestration systems, such as AutoGen and MetaGPT, extend the underlying credit-assignment problem to compositions of multiple agents and tools; a full causal treatment of multi-agent credit assignment falls outside the single-agent scope of the present formulation and is discussed as future work in Section 7.

Positioning. No prior system combines (i) a persistent, cross-task meta-prior over prompt-search trajectories, (ii) a causal—rather than correlational—account of which edits within a trajectory produced the observed reward change, and (iii) a safety mechanism for committing edits autonomously post-deployment without human review. OPRO, APE, and EvoPrompt supply search mechanics but no cross-task memory; DSPy supplies structural priors but no meta-learning; Self-Refine and Reflexion supply within-episode feedback but no persistent, causally-validated transfer; APO supplies edit-level granularity but no DAG-structured causal attribution. Section 4 develops the mechanism that closes all three gaps simultaneously by grounding the meta-learner's warm start in a causal-credit ledger accumulated via backdoor adjustment (Pearl, 2009) over Level 1 edit trajectories.

3. 배경

Recent advances in language model optimization have converged on three distinct paradigms, each addressing limitations in how autonomous agents acquire and refine their reasoning capabilities. Model-Agnostic Meta-Learning (MAML; Finn et al., 2017) established the theoretical foundation for rapid adaptation through gradient-based second-order updates, enabling systems to generalize from few demonstration trajectories. This framework has proven particularly valuable in few-shot learning contexts where data scarcity limits conventional supervised training. However, MAML's application to discrete, non-differentiable domains—such as prompt text optimization—requires careful adaptation of its core machinery.

Prompt optimization as a meta-level search problem emerged as a practical alternative to gradient-based tuning. Large Language Models as Optimizers (OPRO; Yang et al., 2023) introduced a significant shift: rather than treating prompts as fixed hyperparameters, OPRO frames prompt discovery as a sequential optimization problem where the language model itself iteratively refines instructions based on validation feedback. This approach sidesteps gradient computation entirely, making it applicable to black-box objectives and non-differentiable transformations. The core insight—that LLMs can reason about their own prompt effectiveness and propose meaningful improvements—has become foundational to recent agentic systems.

Complementing this optimization view, declarative program synthesis via DSPy (Khattab et al., 2023) introduced structured, compositional abstractions for building and refining language model pipelines. Rather than hand-crafting complex prompts, DSPy users define modular predictors with input/output type signatures, and a compiler automatically synthesizes high-quality prompts and chain-of-thought decompositions. This declarative approach significantly reduces the manual tuning burden and creates a natural interface for programmatic optimization: the DSPy compiler itself becomes a differentiable-in-spirit substrate for meta-level improvements.

A critical gap in existing approaches, however, lies in credit assignment under asynchronous, non-markovian feedback. When a multi-step prompt-optimization trajectory generates a single reward signal (e.g., final task performance), isolating which individual prompt edits contributed to that reward remains poorly understood. Standard gradient attribution (via backpropagation) is inapplicable to discrete prompt spaces. Causal inference methods—specifically Pearl's do-calculus (Pearl, 2009) and backdoor adjustment (Rotnitzky & Robins, 1995)—provide a principled alternative: by constructing a causal DAG over the edit trace, we can decompose total reward into individual causal effects of each edit, even in the absence of gradient signals.

Furthermore, the interaction between local task optimization (Level 1) and meta-learner updates (Level 2) introduces a second-order coordination problem. MAML's theoretical guarantees assume access to clean gradient signals from individual tasks; in contrast, agentic systems operate under noisy, sparse, or delayed reward structures. Extending MAML to handle credit-assigned reward deltas—rather than raw task loss—requires careful treatment of the meta-update objective to avoid reward hacking or overfitting to spurious edit patterns.

4. 방법론

MetaAgent operationalizes prompt optimization through a principled two-level hierarchy that decomposes the problem of autonomous agent improvement into task-local synthesis and meta-learner generalization. The architecture combines three orthogonal mechanisms—declarative program specification via DSPy, iterative meta-prompt search via OPRO, and causal credit attribution—to achieve both immediate task performance gains and transfer learning across unseen task distributions.

The Level 1 task-local optimizer operates within a single task trajectory and performs two coupled optimizations. First, it uses DSPy's declarative program model to convert an agentic task specification (input signature, expected output constraints, intermediate module structure) into a structured compute graph. This graph specifies typed input-output contracts at each node; for example, a code-review agent might decompose as Signature(code_snippet: str, context: str) → Signature(identified_issues: List[Issue], severity: Enum{critical|high|medium|low}). Unlike free-form prompt engineering, this declarative representation enables DSPy's compiler to automatically generate grounded prompts from the signature semantics and automatically detect when outputs violate typed constraints, triggering resampling or backtracking. Within this compiled graph, we apply OPRO (Optimization by Prompting) to systematically improve instruction text. OPRO treats the LLM as a black-box optimizer: at each iteration, the meta-optimizer (running in a single forward pass) generates candidate instruction variants by prompting a separate LLM with the current instruction, exemplars of past failures, and summary statistics of task accuracy. These candidates are evaluated against a held-out task batch, and the top-performing instruction is propagated to the next iteration. In MetaAgent, we extend OPRO with local search: rather than replacing the entire instruction globally, we partition the edit space into fine-grained operations (e.g., add a reasoning step, remove redundant context, specialize to code-review domain) and search over combinations of micro-edits, which reduces search depth by ~40% empirically.

The causal credit-assignment module addresses a critical failure mode: when task performance improves, which intermediate edits were responsible? Standard reinforcement-learning credit-assignment (e.g., n-step returns) breaks down because edits are discrete, non-differentiable, and highly correlated. Instead, we apply Pearl's do-calculus to construct a causal DAG over the edit trace: each node represents an edit operation (e.g., instruction rewrite, few-shot example injection, output parser strictness), and edges encode functional dependencies (e.g., the output format parser depends on the instruction that determined output template). When reward (task accuracy) changes between successive edits, we compute the causal effect P(Accuracy | do(Edit_i=applied)) using backdoor adjustment: the confounding paths (e.g., both edits improving due to underlying model capability) are blocked by conditioning on edit precedence timestamps. This yields a per-edit contribution score; edits with positive causal effect are prioritized in future meta-prompt generations, while edits with zero or negative effect are pruned. This mechanism is essential for navigating the exponential edit space without reverting to reward averaging, which conflates signal noise with true causality.

The Level 2 meta-learner applies Model-Agnostic Meta-Learning (MAML) principles to prompt optimization. Traditional MAML trains a shared initialization that allows fast few-shot adaptation to new tasks via one or two gradient steps; the meta-learner optimizes for tasks that will be solvable by any individual task's optimizer. In MetaAgent, we adapt this idea to prompt space: the meta-learner maintains a task-distribution-aware initialization for the optimization trajectory (e.g., an initial instruction set, a seed few-shot pool, default parser configurations). At the inner loop (single-task adaptation), the Level 1 optimizer refines this initialization over its improvement trajectory, accumulating edits and credit assignments. At the outer loop (meta-update), the meta-learner updates the shared initialization using the query-set task accuracy. This is formalized as a bilevel optimization:

θ* ← arg min_θ Σ_(τ~T) L_val,τ(θ − α · ∇_θ L_train,τ(θ))

where θ denotes the meta-learned initialization state (seed instruction embedding, few-shot pool, edit-priority prior), τ ~ T is a task sampled from the task distribution, α is the Level 1 inner-loop adaptation budget—a fixed number of OPRO-style edit-search iterations rather than a literal scalar learning rate, since the prompt-edit space is discrete—L_train,τ(θ) is the inner-loop training loss (held-out train-batch accuracy after Level 1 optimization from initialization θ), and L_val,τ is the outer-loop meta-objective evaluated on a disjoint query set.

Because prompt edits are discrete and non-differentiable, ∇_θ L_train,τ(θ) cannot be obtained via backpropagation. MetaAgent replaces the gradient with a causal score-function estimator built directly from the credit-assignment mechanism above: each accepted edit e_i in a Level 1 trajectory contributes a causal effect c_i = P(Accuracy | do(e_i = applied)) − P(Accuracy | do(e_i = withheld)), backdoor-adjusted over edit-precedence timestamps. The resulting vector of causal effects functions as a REINFORCE-style finite-difference proxy for ∇_θ L_train,τ(θ): the outer-loop update moves θ toward initializations that historically produce high-causal-effect edits and away from initializations whose edits carry near-zero or negative causal contribution, so noisy or spurious edits that merely correlate with a reward increase are down-weighted rather than reinforced.

Concretely, the Level 2 meta-learner maintains a bank of task-cluster centroids in a task-embedding space derived from the DSPy signature graph together with a summary of the task's few-shot examples. A new task τ is mapped to its nearest centroid(s) by cosine similarity, and that cluster's accumulated state—seed instruction template, few-shot pool, and edit-priority prior, each weighted by historical causal effect—is retrieved as θ_0 for τ's Level 1 optimizer, rather than initializing from a generic template. After the Level 1 trajectory completes, the outer-loop update adjusts both the retrieved cluster's centroid and its associated θ using the causal-effect-weighted proxy gradient: clusters whose retrieved priors led to low search budget and high transfer quality are reinforced, while clusters that produced negative transfer—edits whose causal effect on the new task was negative—are down-weighted or split into finer-grained sub-clusters.

Because the meta-learner continues to adapt online after deployment—updating cluster priors as new tasks and drift events arrive—an unconstrained outer-loop update risks catastrophic forgetting or regression on already-served production tasks. We therefore gate every online meta-update behind a trust-region safety check, adapted conceptually from trust-region policy optimization to the discrete prompt-edit setting: a proposed update to θ (or to a cluster's prior) is committed only if (i) held-out validation accuracy on a rotating sample of previously-served production tasks does not drop by more than a tolerance ε (ε = 2% in our deployment default), and (ii) the embedding-space distance between the proposed and current prior does not exceed a radius δ calibrated per task family. An update violating either constraint is rejected and the previous prior is retained; three consecutive rejections escalate to a human-visible drift alert rather than silently degrading. This gate is what allows MetaAgent to self-improve continuously post-deployment without the monotonic-improvement guarantee assumed by classical MAML, which is unavailable in the discrete, non-stationary prompt-optimization setting addressed here.

5. 실험

We evaluate MetaAgent on the 40-task production-pattern benchmark introduced in Section 1, spanning four domains—code generation (12 tasks), information retrieval (9 tasks), multi-hop reasoning (11 tasks), and tool-use/API orchestration (8 tasks)—with explicit drift annotations marking scheduled model-version swaps (successive Claude 4.X minor releases) and tool-schema revisions at fixed points in each task's session history. Tasks are grouped into 9 semantic clusters by domain experts, independently of the meta-learner's own clustering, to check whether MetaAgent's automatically discovered clusters agree with a human task taxonomy.

Baselines. Vanilla OPRO (no cross-task memory, cold-start every task), the DSPy compiler alone (structural prior, no iterative meta-prompt search), EvoPrompt (population search, no memory), and two MetaAgent ablations: L1-only (Level 1 optimizer with causal credit assignment but no Level 2 meta-learner, i.e., cold-start each task) and L1+L2-no-causal (meta-learner active, but edit prioritization uses raw reward averaging instead of backdoor-adjusted causal effects).

Metrics. (1) Optimization budget—the number of LLM optimizer calls required to reach 95% of each method's own asymptotic accuracy on a task, reported relative to cold-start OPRO; (2) Transfer non-negativity—the fraction of ordered task pairs (τ_i → τ_j) for which warm-starting τ_j from τ_i's cluster prior does not reduce final accuracy relative to a cold start; (3) Drift-robustness—accuracy retention measured five sessions after a scheduled model-version swap or schema revision, with no human-initiated re-tuning.

The full system reaches a 43% relative optimization budget—a 57% reduction, within the 40–60% range reported in the abstract—relative to cold-start OPRO, 87% non-negative transfer across tested task pairs, and 92% accuracy retention five sessions after a drift event, versus 61% for cold-start OPRO's naive re-discovery and 81% for the meta-learner without causal weighting. The gap between L1+L2-no-causal (68% budget, 81% drift retention) and the full system (43% budget, 92% drift retention) isolates the causal credit-assignment module's contribution: reward-averaging-based prioritization propagates roughly a quarter of accepted edits with zero or negative true causal effect into the meta-prior, diluting it with noise that the backdoor-adjusted estimator filters out.

Disabling the trust-region safety gate causes mean accuracy on already-served production tasks to regress by 4.1% within the first 10 online meta-updates following a drift event, with 3 of 40 tasks declining by more than 15%—consistent with catastrophic-forgetting-style overwriting of a previously reliable prior. With the gate enabled, no task in the benchmark regresses beyond the ε = 2% tolerance, at the cost of a 6% slower convergence to the new drift-adapted optimum, a trade-off we consider acceptable for autonomous post-deployment operation without human review.

6. EvolveAI MetaAgent 플랫폼 전략

EvolveAI operationalizes MetaAgent as usage-metered infrastructure rather than a one-time consulting engagement, aligning platform revenue with the budget savings it produces. Three pricing primitives compose the commercial offering:

• Savings-share billing: customers are charged a percentage of the LLM optimization budget MetaAgent measurably avoids relative to a cold-start OPRO baseline on the same task, computed from the optimization-budget metric introduced in Section 5—aligning incentive directly with the paper's core empirical claim rather than a flat per-seat or per-call fee.
• Cluster-prior marketplace: task-cluster priors accumulated across a customer's own task history remain private by default, but customers may opt in to contribute anonymized, causal-effect-weighted priors to a shared cross-tenant pool, receiving a fee credit proportional to downstream reuse. This creates a two-sided flywheel: contributors amortize their own Level 1 search cost across the platform, while later-arriving customers on semantically similar task families inherit warm starts without paying the full discovery cost.
• Enterprise/regulated tier: for customers requiring on-premise or single-tenant deployment (financial services, healthcare), the meta-learner and its causal-credit ledger stay entirely within the tenant boundary; the trust-region tolerance ε and radius δ are exposed as configurable compliance parameters, and the causal-effect DAG is exportable as an audit artifact explaining why any specific production prompt reached its current state—a requirement increasingly common in regulated-AI change-management processes.

Integration is designed to sit alongside, not replace, existing agent tooling: a DSPy-compatible compiler plugin lets teams already using DSPy's signature model adopt MetaAgent's Level 1/Level 2 optimization without rewriting module definitions; adapters accept prompts previously tuned via vanilla OPRO or EvoPrompt as a warm Level 1 trajectory rather than discarding prior optimization work; and framework connectors (LangChain/LangGraph, AutoGen, CrewAI) expose the causal credit-assignment API so that multi-agent orchestration layers can query which upstream prompt edit is causally responsible for a downstream failure—extending the single-agent attribution mechanism of Section 4 to composed agent pipelines as a practical, if not yet formally verified, integration point.

7. 결론

This paper introduced a unified meta-learning framework for autonomous prompt optimization that closes three gaps left open by prior single-task search methods: persistent cross-task transfer via a two-level (task-local/meta) hierarchy, causal—rather than correlational—credit assignment over discrete edit trajectories using backdoor-adjusted do-calculus (Pearl, 2009), and a trust-region safety gate that permits continuous online adaptation post-deployment without human-in-the-loop review. On a 40-task production-pattern benchmark, the full system reduces optimization budget by roughly 57% relative to cold-start OPRO, achieves non-negative transfer on 87% of tested task pairs, and retains 92% of pre-drift accuracy five sessions after a scheduled model or schema change, with ablations isolating the causal credit-assignment module and the trust-region gate as the two mechanisms jointly responsible for the gap between MetaAgent and its non-causal, non-gated variants.

Three limitations point to concrete future work. First, the causal DAG construction in Section 4 assumes edit-precedence timestamps are sufficient to block confounding paths between concurrent edits; multi-agent settings where several agents propose edits to a shared prompt concurrently, as in AutoGen- or MetaGPT-style orchestration, violate this assumption and require extending the model to a multi-agent structural causal model with explicit inter-agent confounders. Second, the trust-region radius δ is currently calibrated per task family via offline grid search; learning δ online as a function of observed drift severity, rather than treating it as a fixed hyperparameter, would likely tighten the accuracy/convergence-speed trade-off reported in Section 5. Third, the present formulation treats prompt text and few-shot examples as the sole edit surface; extending the edit-DAG representation to cover tool-schema definitions and retrieval-index configuration would let the same causal credit-assignment machinery attribute performance changes to a wider range of agent configuration decisions, not prompt text alone. Closing these three gaps—multi-agent causal attribution, adaptive trust regions, and edit-surface generalization—is, in our view, the most direct path toward a fully autonomous, safely self-improving agentic optimization layer.

참고문헌

1. Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., Chen, X. (2024). Large Language Models as Optimizers. arXiv:2309.03409
2. Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., Miller, H., Zaharia, M., Potts, C. (2024). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv:2310.03714
3. Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., Ba, J. (2023). Large Language Models Are Human-Level Prompt Engineers. arXiv:2211.01910
4. Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., Yang, Y. (2024). Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers. arXiv:2309.08532
5. Finn, C., Abbeel, P., Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400
6. Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366
7. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., Clark, P. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651
8. Pryzant, R., Iter, D., Li, J., Lee, Y. T., Zhu, C., Zeng, M. (2023). Automatic Prompt Optimization with Gradient Descent and Beam Search. arXiv:2305.03495
9. Pearl, J. (2009). Causality: Models, Reasoning, and Inference (2nd ed.). doi:10.1017/CBO9780511803161