Agent
Also, known as assistant, copilot, chatbot
Framework: Langchain, OpenAI assistant.
Note:
- Think - Through Prompt Engineering/LLM
- Planning, Reflect, ReAct, CoT, Decomposition
- Communicate
- Talk to Human, Talk to another agent
- Add journal entry to shared memory
- Tools / Act
- Function or API calling
- Generate Code and Execute
- Environment
MAS (Multi-Agentic System)?
note: let’s use more than one agent
Why MultiAgentic System?
- Overcome the current limitation of LLM - A specialized agent does a better job than General purpose agents
- Complex Task Breakdown
- Scalability, Fault Tolerance
- Distributed/Decentralized System
- coordinate, compete, or cooperate,
note: Context window is limited, do we need to keep all the context to do a tiny job? Scalability - Instead of placing all responsibilities to a single agent, if we can assign an agent to only classify if the sentence belongs to happy or unhappy, we can use cheaper model.
Components
- Agent
- role-based
- skill-based
- Another MAS
- or, even traditional system
- Shared Memory
- Context (or short term memory)
- External Storage (long term memory)
- RAG, Metadata DB, …
- Communication Mechanism
- Sequential, Group Chat, Nested Chat, …
- Environment
- Human, Game, Physical Environment
Examples
Agent
(Emerging) Ecosystem
- Optimization
- Prompt Optimization
- Workflow Optimization
- Evolutionary Algorithms
- GraphRAG
- Environment Action
- Browser (ServiceNow), Desktop (Antropic)
Prompt Engineering
prompt evaluation prompt optimization
Agent
role based skill based
Communication Patterns
RAG Pattern
Long Context
Multimodal
- Image/Vision Understanding, Segmentation
- Object Detection
- Speech Understanding
- Robotics Sensors
Autogen Graph Rag
- GraphRAG
- Graph+RAG
- MicrosoftGraphRag - Expensive Agent LLM BigHug
- Preprocessing
- GraphRAG