AI Deep Research Agent

Let's build a finance agent that:

• Deep Web Research: Automatically searches the web, extracts content, and synthesizes findings
• Iterative Research Process: Supports multiple layers of research depth for comprehensive exploration
• Event-Driven Architecture: Built using Motia Framework's event system for robust workflow management
• Parallel Processing: Efficiently processes search results and content extraction
• API Endpoints: REST API access for initiating research and retrieving reports
• Stateful Processing: Maintains research state throughout the entire process

The Steps

🚀 Features

• Deep Web Research: Automatically searches the web, extracts content, and synthesizes findings
• Iterative Research Process: Supports multiple layers of research depth for comprehensive exploration
• Event-Driven Architecture: Built using Motia Framework's event system for robust workflow management
• Parallel Processing: Efficiently processes search results and content extraction
• API Endpoints: REST API access for initiating research and retrieving reports
• Stateful Processing: Maintains research state throughout the entire process

📋 Prerequisites

• Node.js v18 or later
• npm or pnpm
• API keys for:
  • OpenAI (AI analysis)
  • Firecrawl (Web Crawler)

🛠️ Installation

1. Clone the repository:

   git clone https://github.com/MotiaDev/motia-examples
   cd examples/ai-deep-research-agent

2. Install dependencies:

   pnpm install
   # or
   npm install

3. Configure environment variables:

   cp .env.example .env

   Update .env with your API keys:

   # Required
   OPENAI_API_KEY=your-openai-api-key-here
   FIRECRAWL_API_KEY=your-firecrawl-api-key-here
    
   # Optional
   # OPENAI_MODEL=gpt-4o
   # FIRECRAWL_BASE_URL=http://your-firecrawl-instance-url

🏗️ Architecture

🚦 API Endpoints

Start Research

POST /research
Content-Type: application/json

{
  "query": "The research topic or question",
  "breadth": 4,  // Number of search queries to generate (1-10)
  "depth": 2     // Depth of research iterations (1-5)
}

Response:

{
  "message": "Research process started",
  "requestId": "unique-trace-id"
}

Check Research Status

GET /research/status?requestId=unique-trace-id

Response:

{
  "message": "Research status retrieved successfully",
  "requestId": "unique-trace-id",
  "originalQuery": "The research topic or question",
  "status": "in-progress",
  "progress": {
    "currentDepth": 1,
    "totalDepth": 2,
    "percentComplete": 50
  },
  "reportAvailable": false
}

Get Research Report

GET /research/report?requestId=unique-trace-id

Response:

{
  "message": "Research report retrieved successfully",
  "report": {
    "title": "Research Report Title",
    "overview": "Executive summary...",
    "sections": [
      {
        "title": "Section Title",
        "content": "Section content..."
      }
    ],
    "keyTakeaways": [
      "Key takeaway 1",
      "Key takeaway 2"
    ],
    "sources": [
      {
        "title": "Source Title",
        "url": "Source URL"
      }
    ],
    "originalQuery": "The research topic or question",
    "metadata": {
      "depthUsed": 2,
      "completedAt": "2025-03-18T16:45:30Z"
    }
  },
  "requestId": "unique-trace-id"
}

🏃‍♂️ Running the Application

1. Start the development server:

   pnpm dev

2. Access the Motia Workbench:

   http://localhost:3000

3. Make a test request:

   curl --request POST \
   --url http://localhost:3000/research \
   --header 'Content-Type: application/json' \
   --data '{
      "query": "Advancements in renewable energy storage",
      "depth": 1,
      "breadth": 1
   }'

🙏 Acknowledgments

• Motia Framework for the event-driven workflow engine
• OpenAI for AI analysis
• Firecrawl for Web search and content extraction API