Master CrewAI

Overview

CrewAI is a Python framework for orchestrating teams of AI agents that collaborate to accomplish complex tasks. Instead of a single LLM handling everything, CrewAI lets you define multiple agents -- each with a specific role, goal, and backstory -- and coordinate them through structured task workflows. Think of it as assembling a virtual team where a researcher gathers information, an analyst interprets it, and a writer produces the final output, all working together with clear handoffs.

What makes CrewAI distinctive is its emphasis on role-based agent design. Each agent has a persona that shapes how it approaches its tasks, and the framework handles delegation, memory sharing, and output passing between agents. CrewAI supports both sequential execution (agents work one after another) and hierarchical execution (a manager agent delegates to workers), plus a newer Flows system for complex multi-step workflows with conditional logic.

CrewAI has gained significant adoption for use cases like automated research, content pipelines, code review workflows, and data analysis teams. It strikes a balance between the simplicity of single-agent systems and the power of fully autonomous agent frameworks.

When to Use

• Building automated content pipelines where research, writing, editing, and publishing are separate concerns
• Creating code review workflows where different agents check for security, performance, style, and correctness
• Designing research automation that gathers data from multiple sources, analyzes it, and produces reports
• Implementing customer support triage where specialized agents handle different categories of issues
• Building data analysis pipelines where collection, cleaning, analysis, and visualization are distinct steps
• Automating hiring workflows with resume screening, skill assessment, and reporting agents
• Creating multi-perspective analysis systems that evaluate decisions from different viewpoints

Quick Start

Copy
# Install CrewAI with tools
pip install crewai crewai-tools

# Create a new project (recommended)
crewai create crew my_research_crew

# Or add to existing project
pip install crewai

Copy
# Minimal crew: researcher + writer
from crewai import Agent, Task, Crew, Process

researcher = Agent(
    role="Senior Research Analyst",
    goal="Find comprehensive, accurate information on the given topic",
    backstory="You have 15 years of experience in investigative research. "
              "You are known for uncovering insights others miss.",
    verbose=True,
)

writer = Agent(
    role="Technical Content Writer",
    goal="Transform research into clear, engaging content",
    backstory="You are a former journalist who now writes technical content. "
              "You make complex topics accessible without dumbing them down.",
    verbose=True,
)

research_task = Task(
    description="Research the topic: {topic}. "
                "Find key facts, recent developments, and expert opinions.",
    agent=researcher,
    expected_output="A structured research report with key findings and sources.",
)

writing_task = Task(
    description="Using the research provided, write a 1000-word article on {topic}.",
    agent=writer,
    expected_output="A polished, publication-ready article.",
    context=[research_task],  # Gets output from research_task
)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    process=Process.sequential,
    verbose=True,
)

result = crew.kickoff(inputs={"topic": "AI agents in production"})
print(result)

Core Concepts

Agent Design Philosophy

The most critical factor in CrewAI success is agent design. Each agent needs three elements: a specific role (job title), a clear goal (what they are trying to achieve), and a compelling backstory (their expertise and perspective). The backstory is not fluff -- it significantly shapes how the LLM approaches the task.

Copy
# Bad agent design - too vague
bad_agent = Agent(
    role="Developer",
    goal="Help with code",
    backstory="You write code.",
)

# Good agent design - specific and detailed
good_agent = Agent(
    role="Senior Backend Engineer specializing in Python microservices",
    goal="Design and review Python backend code for scalability, "
         "maintainability, and adherence to clean architecture principles",
    backstory="You have 12 years of experience building Python services at "
              "companies processing 10M+ requests per day. You have deep expertise "
              "in FastAPI, SQLAlchemy, and event-driven architectures. You are "
              "opinionated about code quality and always push for proper error "
              "handling and comprehensive logging.",
    verbose=True,
    allow_delegation=False,  # This agent does its own work
)

YAML-Based Configuration (Recommended)

CrewAI strongly recommends defining agents and tasks in YAML files for better maintainability and separation of concerns.

Copy
# config/agents.yaml
researcher:
  role: "Senior Research Analyst"
  goal: "Find comprehensive, accurate information on {topic}"
  backstory: "Expert researcher with 15 years of experience in gathering and analyzing information."
  tools: [SerperDevTool, WebsiteSearchTool]
  verbose: true

analyst:
  role: "Data Analyst"
  goal: "Analyze research data and extract actionable insights"
  backstory: "Quantitative analyst who supports conclusions with data."
  verbose: true

writer:
  role: "Content Strategist"
  goal: "Create engaging, well-structured content"
  backstory: "Content strategist combining storytelling with data-driven insights."
  verbose: true

Copy
# config/tasks.yaml
research_task:
  description: "Research {topic}. Focus on key facts, recent developments, expert opinions."
  agent: researcher
  expected_output: "Research report with executive summary, key findings, and sources."

analysis_task:
  description: "Analyze the research and extract top 3 insights, trends, and risk factors."
  agent: analyst
  expected_output: "Analysis brief with prioritized insights."
  context: [research_task]

writing_task:
  description: "Write an 800-word article about {topic} with data-backed claims and clear headers."
  agent: writer
  expected_output: "A publication-ready article in markdown format."
  context: [research_task, analysis_task]

Copy
# crew.py - Load from YAML
from crewai import Agent, Task, Crew, Process
from crewai.project import CrewBase, agent, task, crew

@CrewBase
class ResearchCrew:
    agents_config = "config/agents.yaml"
    tasks_config = "config/tasks.yaml"

    @agent
    def researcher(self) -> Agent:
        return Agent(config=self.agents_config["researcher"])

    @agent
    def analyst(self) -> Agent:
        return Agent(config=self.agents_config["analyst"])

    @agent
    def writer(self) -> Agent:
        return Agent(config=self.agents_config["writer"])

    @task
    def research_task(self) -> Task:
        return Task(config=self.tasks_config["research_task"])

    @task
    def analysis_task(self) -> Task:
        return Task(config=self.tasks_config["analysis_task"])

    @task
    def writing_task(self) -> Task:
        return Task(config=self.tasks_config["writing_task"])

    @crew
    def crew(self) -> Crew:
        return Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential,
            verbose=True,
        )

# Run
research_crew = ResearchCrew()
result = research_crew.crew().kickoff(inputs={"topic": "AI in healthcare 2026"})

Process Types

Copy
# Sequential: Tasks run in order, each getting output from the previous
sequential_crew = Crew(
    agents=[researcher, analyst, writer],
    tasks=[research_task, analysis_task, writing_task],
    process=Process.sequential,
)

# Hierarchical: A manager agent decides who handles what
hierarchical_crew = Crew(
    agents=[researcher, analyst, writer],
    tasks=[research_task, analysis_task, writing_task],
    process=Process.hierarchical,
    manager_llm="gpt-4o",  # Manager uses this model for delegation
)

# With Planning: CrewAI generates an execution plan before starting
planned_crew = Crew(
    agents=[researcher, analyst, writer],
    tasks=[research_task, analysis_task, writing_task],
    process=Process.sequential,
    planning=True,
    planning_llm="gpt-4o",  # Uses this model for the planning step
)

Tool Integration

Copy
from crewai_tools import (
    SerperDevTool,
    WebsiteSearchTool,
    FileReadTool,
    DirectoryReadTool,
    CodeInterpreterTool,
)

# Web search tool (requires SERPER_API_KEY)
search_tool = SerperDevTool()

# Website content extraction
web_tool = WebsiteSearchTool()

# File operations
file_tool = FileReadTool()
dir_tool = DirectoryReadTool(directory="./docs")

# Code execution
code_tool = CodeInterpreterTool()

# Assign tools to specific agents
researcher = Agent(
    role="Researcher",
    goal="Find accurate information",
    backstory="Expert researcher...",
    tools=[search_tool, web_tool],
)

developer = Agent(
    role="Developer",
    goal="Write and test code",
    backstory="Expert developer...",
    tools=[file_tool, code_tool],
)

Flows for Complex Workflows

CrewAI Flows orchestrate multiple crews with conditional logic and state management.

Copy
from crewai.flow.flow import Flow, listen, start, router

class ContentPipelineFlow(Flow):

    @start()
    def gather_requirements(self):
        return {"topic": self.state.get("topic"), "format": self.state.get("format")}

    @router(gather_requirements)
    def route_by_format(self):
        fmt = self.state.get("format", "blog_post")
        return "blog_pipeline" if fmt == "blog_post" else "doc_pipeline"

    @listen("blog_pipeline")
    def run_blog_crew(self):
        result = BlogCrew().crew().kickoff(inputs=self.state)
        return {"draft": result.raw}

    @listen(run_blog_crew)
    def review_and_publish(self):
        result = ReviewCrew().crew().kickoff(inputs={"draft": self.state["draft"]})
        return {"final": result.raw}

# Run the flow
flow = ContentPipelineFlow()
result = flow.kickoff(inputs={"topic": "AI agents", "format": "blog_post"})

Configuration Reference

Parameter	Default	Description
process	sequential	Execution type: sequential or hierarchical
verbose	False	Enable detailed execution logging
memory	False	Enable crew memory across executions
planning	False	Generate execution plan before running
planning_llm	None	LLM model for the planning step
manager_llm	None	LLM for hierarchical process manager
max_rpm	None	Rate limit for LLM API calls per minute
max_iter	15	Maximum iterations per task before stopping
allow_delegation	True	Allow agents to delegate to other agents
cache	True	Cache tool results for efficiency
output_log_file	None	Path to write execution logs
embedder	default	Embedding model configuration for memory

Best Practices

1. Keep crews small: 3-5 agents maximum. Each additional agent adds coordination overhead and increases the chance of miscommunication. If you need more than 5 agents, split into multiple crews connected by Flows.

2. Always specify expected_output on every task. Without it, agents do not know what "done" looks like. Be explicit about format, length, and structure. This is the single most impactful parameter for output quality.

3. Use YAML configuration files for agents and tasks. Hardcoding agent definitions in Python makes them harder to iterate on. YAML files let you tweak roles, goals, and task descriptions without touching code.

4. Start with sequential process and only use hierarchical when you have clear delegation needs. Sequential is simpler, more predictable, and easier to debug. Hierarchical adds a manager agent layer that can introduce its own errors.

5. Design agent backstories that create productive tension. A researcher who is "thorough and skeptical" paired with a writer who is "engaging and persuasive" creates better outputs than two agents with generic backstories. The tension between perspectives improves the final result.

6. Use context on tasks to explicitly wire data flow. Do not rely on agents to figure out which previous outputs they need. Explicitly listing context: [research_task] on the writing task ensures the writer receives the researcher's output.

7. Set allow_delegation=False on agents that should do their own work. By default, agents can delegate tasks to other agents, which sometimes leads to circular delegation loops. Disable delegation unless you specifically want it.

8. Use the planning feature for complex multi-step workflows. Enabling planning=True generates a structured plan before execution, which helps agents understand their role in the larger workflow and produces more consistent results.

9. Monitor token usage carefully. Multi-agent systems multiply token consumption because each agent makes its own LLM calls. Use max_rpm to rate-limit API calls and consider using cheaper models for less critical agents.

10. Test individual agents before assembling the crew. Verify each agent produces good output in isolation before combining them. This makes debugging much easier when crew-level issues arise.

Troubleshooting

Problem: Agents produce vague or generic outputs. The backstory is too generic. Add specific expertise, domain knowledge, and strong opinions. Example: "Technical writer with 10 years at Google who insists on code examples."

Problem: Agents get stuck in delegation loops. Set allow_delegation=False on agents that should do their own work. Ensure roles are distinct and non-overlapping.

Problem: Tasks produce outputs in wrong format. Make expected_output more specific: "Markdown report with H2 headers, bullet points, and a metrics table."

Problem: CrewAI execution is too slow. Use cache=True, faster models for non-critical agents, and lower max_iter values.

Problem: Hierarchical manager makes poor delegation decisions. Use a capable LLM (manager_llm="gpt-4o") and ensure agent roles are clearly distinct.