MCP Server Architecture and Internal Components

What is MCP 

MCP (Model Context Protocol) is a way for AI models (like me) to interact with external tools or systems in a structured way.

MCP Components:

• Host: Think of this as the "middleman" or platform that runs the AI model and manages communication.
• Client: This is the user-facing application (like your chat interface) that sends requests to the host.
• Server: This is the tool or service that provides extra functionality (like a database, calculator, or image generator). The host talks to the server when the client needs something special.

Communication mechanism between MCP Components:

• MCP Client→ sends a request (e.g., "generate an image") to the Host.
• What it is: A software component (usually code) that runs in the user-facing application (like a chat UI, IDE plugin, or CLI).
• Role: Sends structured requests to the MCP Host and receives responses.
• Not an agent by itself—it’s typically implemented in Python, JavaScript, or other languages as part of the app.
• Think of it as: The “bridge” between the user and the MCP Host.
• Client → Host: Protocol => JSON-RPC 2.0 over WebSocket or HTTP
• MCP Host→ interprets the request and decides if it needs help from a Server.
• What it is: A runtime environment (code) that runs the AI model and orchestrates communication with MCP Servers.
• Role: Receives requests from the Client, interprets them, and calls MCP Servers when tools or external data are needed.
• Not an agent in isolation—it’s usually part of the AI platform (like OpenAI’s MCP implementation).
• Think of it as: The “brain” that routes requests and aggregates results.
• Host ↔ Server: Protocol =>  JSON-RPC 2.0 over WebSocket or HTTP
• MCP Server → does the job (e.g., creates the image) and sends the result back to the Host.
• What it is: A service or process (code) that exposes tools/resources via the MCP protocol.
• Role: Provides access to external systems (DB, APIs, file system) through standardized MCP endpoints.
• Implemented as: Python, Node.js, or any language that supports MCP spec.
• Think of it as: The “toolbox” behind the Host.
• Server ↔ tools: Protocol => HTTP, SQL, File I/O to backend
• Host → returns the result to the MCP Client

Protocols & Message Flow:

• Requestor: The human or external app that initiates a request (e.g., you in a chat UI).
• Requestor → MCP Client: UI calls or app method invocations (or HTTP if separate service).
• MCP Client: The user-facing application (chat app, IDE extension, etc.). It sends requests to the Host.
• MCP Client → MCP Host: JSON-RPC 2.0 over WebSocket or HTTP.
• MCP Host: The orchestrator/platform that runs the AI model and speaks MCP. It routes requests to MCP servers.
• MCP Host → MCP Server(s): JSON-RPC 2.0 over WebSocket or HTTP.
• MCP Server: One or more services that expose tools/resources via MCP (e.g., file system, DB, APIs).
• MCP Server → Tool/DB/API: HTTP/REST, SQL, File I/O, etc.
• Tools / DB Access: The actual backends the MCP server uses (databases, APIs, search, compute, etc.).

MCP and Architecture View:

• Layer 1: User Layer
• Component: Requestor (User or external app)
• Role: Initiates commands or queries.
• Layer 2: Interface Layer
• Component: MCP Client
• Role: Converts user input into structured MCP requests.
• Protocol: JSON-RPC over WebSocket/HTTP to Host.
• Layer 3: Orchestration Layer
• Component: MCP Host
• Role: Runs the AI model, interprets requests, and routes to MCP Servers.
• Protocol: JSON-RPC over WebSocket/HTTP to Servers.
• Layer 4: Integration Layer
• Components: MCP Servers (API Tools, DB Adapter, File System)
• Role: Wrap external resources and expose them via MCP.
• Protocol: HTTP, SQL, File I/O to backends.
• Layer 5: Backend Layer
• Components: External APIs, Databases, File Storage

Background about MCP server protocol as RPC:

Remote Procedure Call (RPC):

Remote Procedure Call (RPC) is a method that enables a program to execute a function on another computer in a network as if it were local. The client sends the request (with arguments) to the server, the server executes the function, and the result is sent back. RPC hides the details of networking,

  Step by Step - RPC Works

• Client Calls Stub: The client calls a local procedure (stub) as if it were normal.
• Marshalling: The stub packs (marshals) all input parameters into a message.
• Send to Server: The message is sent across the network to the server.
• Server Stub: The server stub unpacks the message and calls the actual server procedure.
• Execution & Return: The server runs the procedure and returns the result to the stub.
• Back to Client: The server stub sends the result back, and the client stub unpacks it.

 Notes:

• RPC Runtime: A library that manages the communication in RPC. It handles binding, sending/receiving data, selecting the protocol, and handling errors.

Inter-Process Communication, or IPC:

Inter-Process Communication, or IPC, is a mechanism that allows coordinating communication between different running processes (applications)

It helps processes synchronize their activities, share information, and avoid conflicts while accessing shared resources.

Agentic Design Pattern

Prompt chaining Agentic pattern:

Prompt chaining is a Agentic pattern where multiple LLM (Large Language Model) calls are linked together in a sequence. Each step performs a specific task, and the output from one step becomes the input for the next.

It can also include custom code between steps to guide or transform the data. This approach helps complex problems into manageable parts dividing into small pieces, allowing each prompt for better results. While it's called a "workflow," it can still include some autonomy, like letting the first LLM decide the topic that the rest of the chain works on.

○ Input Received→ Input to the workflow.

○ LLM → First model processes the input.

○ CODE → Optional custom logic or transformation.

○ LLM2 → Second model processes the transformed data.

        ○ LLM3 → Third model processes the transformed data.

○ Out → Final output.

 

%3CmxGraphModel%3E%3Croot%3E%3CmxCell%20id%3D%220%22%2F%3E%3CmxCell%20id%3D%221%22%20parent%3D%220%22%2F%3E%3CmxCell%20id%3D%222%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%2399CCFF%3B%22%20value%3D%22%F0%9F%93%9D%20Input%20Received%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2261%22%20width%3D%22110%22%20x%3D%22210%22%20y%3D%221488%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%223%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%224%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BentryX%3D0.5%3BentryY%3D0%3BentryDx%3D0%3BentryDy%3D0%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20target%3D%225%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%224%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%201%3A%20Initial%20Reasoning%20%2F%20Planning%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2283%22%20width%3D%22151%22%20x%3D%22419%22%20y%3D%221477%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%225%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%2399CCFF%3B%22%20value%3D%22%F0%9F%A7%A0%20CODE%3A%20Optional%20Logic%20%2F%20Transformation%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2280%22%20width%3D%22131%22%20x%3D%22429%22%20y%3D%221635%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%226%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%2212%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BentryX%3D0.5%3BentryY%3D0%3BentryDx%3D0%3BentryDy%3D0%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BexitX%3D0.636%3BexitY%3D1.043%3BexitDx%3D0%3BexitDy%3D0%3BexitPerimeter%3D0%3B%22%20target%3D%229%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%227%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%228%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20target%3D%2212%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%228%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%202%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2270%22%20width%3D%22108%22%20x%3D%22650%22%20y%3D%221640%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%229%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%23CCE5FF%3B%22%20value%3D%22%F0%9F%93%A4%20Final%20Output%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2260%22%20width%3D%22140%22%20x%3D%22850%22%20y%3D%221780%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2210%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%222%22%20style%3D%22startArrow%3Dnone%3BendArrow%3Dblock%3BexitX%3D1%3BexitY%3D0.5%3BentryX%3D0%3BentryY%3D0.5%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BedgeStyle%3DorthogonalEdgeStyle%3B%22%20target%3D%224%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2211%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%225%22%20style%3D%22startArrow%3Dnone%3BendArrow%3Dblock%3BexitX%3D1%3BexitY%3D0.5%3BentryX%3D0%3BentryY%3D0.5%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BedgeStyle%3DorthogonalEdgeStyle%3BentryDx%3D0%3BentryDy%3D0%3B%22%20target%3D%228%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2212%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%203%3A%20Final%20Reasoning%20%2F%20Synthesis%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2270%22%20width%3D%22140%22%20x%3D%22830%22%20y%3D%221640%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3C%2Froot%3E%3C%2FmxGraphModel%3E

Routing Agentic pattern:

 %3CmxGraphModel%3E%3Croot%3E%3CmxCell%20id%3D%220%22%2F%3E%3CmxCell%20id%3D%221%22%20parent%3D%220%22%2F%3E%3CmxCell%20id%3D%222%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%2399CCFF%3B%22%20value%3D%22%F0%9F%93%9D%20Input%20Received%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2261%22%20width%3D%22110%22%20x%3D%22210%22%20y%3D%221488%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%223%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%224%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BentryX%3D0.5%3BentryY%3D0%3BentryDx%3D0%3BentryDy%3D0%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20target%3D%225%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%224%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%201%3A%20Initial%20Reasoning%20%2F%20Planning%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2283%22%20width%3D%22151%22%20x%3D%22419%22%20y%3D%221477%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%225%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%2399CCFF%3B%22%20value%3D%22%F0%9F%A7%A0%20CODE%3A%20Optional%20Logic%20%2F%20Transformation%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2280%22%20width%3D%22131%22%20x%3D%22429%22%20y%3D%221635%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%226%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%2212%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BentryX%3D0.5%3BentryY%3D0%3BentryDx%3D0%3BentryDy%3D0%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BexitX%3D0.636%3BexitY%3D1.043%3BexitDx%3D0%3BexitDy%3D0%3BexitPerimeter%3D0%3B%22%20target%3D%229%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%227%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%228%22%20style%3D%22edgeStyle%3DorthogonalEdgeStyle%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BorthogonalLoop%3D1%3BjettySize%3Dauto%3Bhtml%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20target%3D%2212%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%228%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%202%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2270%22%20width%3D%22108%22%20x%3D%22650%22%20y%3D%221640%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%229%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BfillColor%3D%23CCE5FF%3B%22%20value%3D%22%F0%9F%93%A4%20Final%20Output%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2260%22%20width%3D%22140%22%20x%3D%22850%22%20y%3D%221780%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2210%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%222%22%20style%3D%22startArrow%3Dnone%3BendArrow%3Dblock%3BexitX%3D1%3BexitY%3D0.5%3BentryX%3D0%3BentryY%3D0.5%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BedgeStyle%3DorthogonalEdgeStyle%3B%22%20target%3D%224%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2211%22%20edge%3D%221%22%20parent%3D%221%22%20source%3D%225%22%20style%3D%22startArrow%3Dnone%3BendArrow%3Dblock%3BexitX%3D1%3BexitY%3D0.5%3BentryX%3D0%3BentryY%3D0.5%3Brounded%3D0%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3BedgeStyle%3DorthogonalEdgeStyle%3BentryDx%3D0%3BentryDy%3D0%3B%22%20target%3D%228%22%20value%3D%22%22%3E%3CmxGeometry%20relative%3D%221%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3CmxCell%20id%3D%2212%22%20parent%3D%221%22%20style%3D%22whiteSpace%3Dwrap%3BstrokeWidth%3D2%3Bsketch%3D1%3BhachureGap%3D4%3Bjiggle%3D2%3BcurveFitting%3D1%3BfontFamily%3DArchitects%20Daughter%3BfontSource%3Dhttps%253A%252F%252Ffonts.googleapis.com%252Fcss%253Ffamily%253DArchitects%252BDaughter%3B%22%20value%3D%22%F0%9F%A4%96%20LLM%203%3A%20Final%20Reasoning%20%2F%20Synthesis%22%20vertex%3D%221%22%3E%3CmxGeometry%20height%3D%2270%22%20width%3D%22140%22%20x%3D%22830%22%20y%3D%221640%22%20as%3D%22geometry%22%2F%3E%3C%2FmxCell%3E%3C%2Froot%3E%3C%2FmxGraphModel%3E

Routing is a design pattern where an input first goes to a router LLM, which decides which specialized model should handle the task. Each specialist LLM is optimized for different functions. The router classifies the input and directs it to the best-fit model, enabling separation of concerns. While it follows a workflow, the router introduces some autonomy by making decisions within guardrails.

Flow Steps:

○ Input Received - A user query or task enters the system.

○ Router LLM - Analyzes Task - The router examines the input to determine its nature

○ Classification & Decision - Based on task type and complexity, the router selects the best specialist model

○ Route to Specialist -The chosen model processes the task using its expertise.

○ Out - The result is sent back to the router or to the user, maintaining workflow consistency.

Notes: Routing – An LLM routes tasks to the most suitable expert LLM.
 

Parallelization Agentic pattern:

 

Parallelization is a design pattern where code (not an LLM) splits a task into multiple parts, sends them to several LLMs to run concurrently, and then uses code to aggregate the results—either combining different subtasks or averaging repeated runs of the same task.

 

Flow Steps:

parallelization pattern diagram:

○ Code: Breaks the main task into subtasks.

○ Task: Represents the split workload.

○ LLMs: Multiple models process subtasks in parallel.

○ Aggregator: Collects and combines results (e.g., stitching, averaging).

 

Notes: Parallelization – Code splits tasks and sends them to multiple LLMs in parallel; results are aggregated.

 

Orchestrator-Worker Agentic pattern:

An LLM acts as the orchestrator, breaking down a complex task into smaller steps, assigning them to other LLMs (workers), and then synthesizing their outputs into a final result.

 

Flow Steps:

Orchestrator-Worker pattern diagram:

○ Top LLM: Acts as the orchestrator.

○ Orchestrator: Breaks down the complex task into smaller subtasks.

○ Worker LLMs: Execute subtasks in parallel.

○ Output: Orchestrator synthesizes results into a final answer.

Notes: Orchestrator-Worker – An LLM orchestrates task breakdown and synthesis, assigning subtasks to other LLMs.

 

Evaluator-Optimizer Agentic attern:

One LLM generates a solution, and another LLM evaluates it. If accepted, the output is finalized; if rejected, feedback is sent back to the generator for improvement, creating a feedback loop to enhance accuracy and reliability.

 

Flow Steps:

Evaluator-Optimizer pattern diagram:

○ Generator LLM: Creates the initial solution.

○ Evaluator LLM: Validates the solution.

- If accepted, it goes to Output.

- If rejected, feedback loops back to the Generator for refinement.

○ This cycle continues until the solution meets standards.

 

Notes: Evaluator-Optimizer – One LLM generates, another evaluates; feedback loop improves accuracy.

Legends: 

• Red color shows LLM enagment no manaual code
• Blue color shows Manaul enagment and it does have manaual code

Use case scenarios for each design pattern:

 

1. Parallelization

Scenario:

    A company needs to summarize three different research papers quickly.

○ Code splits the task into three subtasks (one per paper).

○ Sends each subtask to a different LLM in parallel.

○ Aggregator combines the summaries into a single consolidated report.

 

2. Orchestrator

Scenario:

    A user asks for a comprehensive market analysis.

○ Orchestrator LLM breaks the task into subtasks:

-  Competitor analysis
-  Customer trends
-  Pricing strategy

○ Assigns each subtask to specialized LLMs.

○ Orchestrator then synthesizes all outputs into a final report.

 

3. Evaluator

Scenario:

    Generating legal contract clauses for compliance.

○ Generator LLM drafts the clause.

○ Evaluator LLM checks for compliance and clarity.

○ If rejected, evaluator provides feedback → generator revises.

○ Loop continues until evaluator approves.

 

4. Routing 

Scenario:

a. Customer Support Automation

○ Scenario: A chatbot receives a user query.

○ How it works:

- The router LLM analyzes the query and decides:

□ Billing → send to finance expert LLM.

□ Technical issue → send to troubleshooting LLM.

□ Account setup → send to onboarding LLM.

○ Benefit: Ensures queries are handled by the most relevant expert model, improving accuracy and response time.

 

b. Domain-Specific Question Answering

○ Scenario: A knowledge assistant handles questions across law, medicine, and finance.

○ How it works:

- Router LLM classifies the question:

□ Legal → Legal LLM.

□ Medical → Medical LLM.

□ Financial → Finance LLM.

○ Benefit: Reduces risk of incorrect answers by leveraging domain expertise.

Prompt Chaining

Scenario:

    produce a compliant, concise legal risk summary from a contract.

• Extraction prompt - Prompt the LLM to extract: clauses, obligations, parties, dates, deadlines, amounts, and referenced documents.
• Structuring prompt - Normalize and organize the extracted data, human-readable structure.
• Risk evaluation prompt - Assess legal and operational risks.
• Summary prompt - Turn detailed analysis into a short, structured summary.
• Validation prompt - Quality control and compliance alignment.
• Finalize and package outputs - deliver in the format needed for downstream workflows (PDF, doc, JSON, etc.). 

AI Agentic Tools:

A tool is an external capability or function that an AI agent can invoke to perform specific tasks. These tools allow the agent to interact with the world, gather information, manipulate data, or generate content in ways that go beyond simple conversation.

A tool is essentially a callable function or service that the agent uses to:

○ Access external data (e.g., web search)

○ Perform computations (e.g., Python execution)

○ Generate or modify images

○ Interact with APIs or databases

tools are modular and can be dynamically selected by the AI agent based on the request. 

Notes: 

• Agent is the decision-maker.
• Tool is a functional unit (API, connector, script) that performs specific tasks.