Agentic AI: NCP-AAI Exam Premium File as PDF & Test Engine

What is a key limitation of Chain-of-Thought (CoT) prompting when using smaller language models for reasoning tasks?

• A. CoT prompting requires relatively large models; smaller models may produce reasoning chains that appear logical but are actually incorrect, leading to poorer performance.
• B. CoT prompting simplifies error analysis for small models, making it easy to identify and correct mistakes at each reasoning step.
• C. CoT prompting ensures step-by-step outputs, enabling even small models to solve complex problems reliably.
• D. CoT prompting consistently improves the logical accuracy of outputs for both small and large language models.

When evaluating coordination failures in a multi-agent system managing distributed manufacturing workflows, which analysis approach best identifies state management and planning synchronization issues?

• A. Monitor agent outputs individually to confirm local correctness and examine results of specific workflow steps.
• B. Assess synchronization methods during design reviews and use simulations to evaluate coordination across representative workflow scenarios.
• C. Deploy distributed state tracing across agents, analyze transition timing, study communication overhead, and verify synchronization accuracy.
• D. Track workflow throughput and task completions to measure performance trends and highlight workflow outcomes.

You are evaluating your RAG pipeline. You notice that the LLM-as-a-Judge consistently assigns high similarity scores to responses that contain irrelevant information.
What should you investigate as the most likely potential cause with the least development effort?

• A. The temperature setting used by the LLM during response generation.
• B. The quality of the synthetic questions used for evaluation.
• C. The size of the knowledge base used to power the RAG pipeline.
• D. The prompt used to instruct the LLM-as-a-Judge to assess the response.

You are tasked with comparing two agentic AI systems - System A and System B - both designed to generate marketing copy.
You've run identical prompts and have recorded the generated outputs.
To objectively assess which system is performing better, what is the most appropriate approach?

• A. Implement a human-in-the-loop to subjectively rate each output on a scale of 1 to 5 based on the user's personal preference.
• B. Implement a benchmark pipeline that automatically compares the generated outputs using metrics like relevance, creativity, and grammatical correctness.
• C. Measure the click-through rate for each system's marketing copy as the primary indicator of performance.
• D. Gather ratings from a panel of users, with each rating marketing copy on a 1 to 5 scale for overall impression of relevance, creativity, and grammatical correctness.