Motia Icon

Multi-Language Data Processing: Building Event-Driven Pipelines with TypeScript, Python & JavaScript

Modern backend development often requires combining the strengths of different programming languages. TypeScript for APIs, Python for data processing and AI, JavaScript for rapid prototyping. Traditional approaches involve complex microservices architectures with intricate communication patterns.

This comprehensive guide explores how to build a unified multi-language data processing pipeline using Motia's step primitive. We'll cover:

  1. Steps as Core Primitive: How steps unify different languages under a single abstraction.
  2. Building the Pipeline: A step-by-step guide to creating a cohesive multi-language data processing workflow.
  3. Unified Execution Model: How steps enable seamless communication between different runtime environments.
  4. Hands-On Development: How to build, run, and observe your unified multi-language pipeline.

Let's build a production-ready data processing system where steps unify TypeScript, Python, and JavaScript into a single cohesive workflow.


The Power of Steps: A Unified Multi-Language Primitive

Multi-Language Data Processing in Motia Workbench

At its core, our data processing pipeline demonstrates how steps solve the fundamental challenge of multi-language systems: unifying different programming languages under a single, coherent abstraction. Traditional polyglot architectures require complex inter-process communication and deployment coordination. Motia's step primitive unifies everything.

Steps enable true language unification:

  • TypeScript steps: Strong typing and excellent tooling for APIs and orchestration
  • Python steps: Rich ecosystem for data processing, ML, and scientific computing
  • JavaScript steps: Dynamic processing and rapid development
  • Motia's Step Primitive: The unifying abstraction that makes all languages work as a single system

Instead of managing multiple services, steps provide a single programming model. Whether written in TypeScript, Python, or JavaScript, every step follows the same pattern: receive data, process it, emit events. This unification is what makes multi-language development straightforward.


The Anatomy of Our Multi-Language Pipeline

Our application consists of five specialized steps, each leveraging the optimal language for its specific task. Let's explore the complete architecture.

01-starter.step.ts
02-bridge.step.ts
simple-python.step.py
notify.step.ts
04-final.step.ts
05-summary.step.js
index.ts

The entry point for our multi-language workflow. This TypeScript API endpoint receives data, validates it with Zod schemas, and kicks off the processing pipeline.

import { Handlers } from 'motia'
import { z } from 'zod'
import { StartAppRequest, StartAppResponse, AppData } from '../types'
 
const bodySchema = z.object({
  data: z.record(z.unknown()).optional(),
  message: z.string().optional()
})
 
// Basic app starter - TypeScript API endpoint
export const config = {
  type: 'api',
  name: 'appStarter',
  description: 'Start the basic multi-language app',
 
  method: 'POST',
  path: '/start-app',
 
  emits: ['app.started'],
  flows: ['data-processing'],
  
  bodySchema,
  responseSchema: {
    200: z.object({
      message: z.string(),
      appId: z.number(),
      traceId: z.string()
    })
  }
} as const
 
export const handler: Handlers['appStarter'] = async (req, { logger, emit, traceId }) => {
  logger.info('🚀 Starting basic app', { body: req.body, traceId })
  
  const validationResult = bodySchema.safeParse(req.body)
  
  if (!validationResult.success) {
    logger.error('Invalid request body', { errors: validationResult.error.errors })
    return { 
      status: 400, 
      body: { 
        message: 'Invalid request body', 
        errors: validationResult.error.errors 
      } 
    }
  }
  
  const appData: AppData = {
    id: Date.now(),
    input: validationResult.data.data || {},
    started_at: new Date().toISOString(),
    traceId: traceId
  }
  
  // Emit to start the app
  await emit({
    topic: 'app.started',
    data: appData
  })
  
  logger.info('app initiated successfully', { appId: appData.id })
  
  const response: StartAppResponse = {
    message: 'Basic app started successfully',
    appId: appData.id,
    traceId: traceId
  }
  
  return {
    status: 200,
    body: response
  }
} 

Explore the Workbench

The Motia Workbench provides a visual representation of your multi-language pipeline, making it easy to trace data flow between TypeScript, Python, and JavaScript steps.

Multi-Language Workflow in Motia Workbench

You can monitor real-time execution, view logs from all languages in a unified interface, and trace the complete data flow from the TypeScript API through Python processing to JavaScript summary generation.


Key Features & Benefits

🧩 Step as Universal Primitive

Every piece of logic—whether TypeScript, Python, or JavaScript—follows the same step pattern, creating true unification.

🌐 Seamless Language Integration

Steps eliminate the complexity of multi-language systems by providing a unified programming model.

📊 Unified Development Experience

Write, debug, and monitor all languages through a single interface and shared execution model.

Hot Reload Across Languages

Edit any step in any language and see changes instantly across the entire pipeline.

🔄 Event-Driven Communication

Steps communicate through events, enabling loose coupling and independent scaling.

🎯 Single Deployment Model

Deploy all languages together as a cohesive system, not as separate microservices.


Trying It Out

Ready to build your first multi-language Motia application? Let's get it running.

Create Your Motia App

Start by creating a new Motia project with the interactive setup.

npx motia@latest create -i

Move into your project directory and start the development server.

cd my-app  # Replace with your project name
npx motia dev

Open the Workbench

Navigate to http://localhost:3000 to access the Workbench and run your workflow.

Test the Multi-Language Pipeline

Send a request to your API endpoint to see the multi-language workflow in action:

curl -X POST http://localhost:3000/start-app \
  -H "Content-Type: application/json" \
  -d '{"data":{"message":"Hello multi-language world!","value":42}}'

Watch in the Workbench as your data flows through:

  1. TypeScript validation and event emission
  2. TypeScript bridge processing and forwarding
  3. Python data processing with rich logging
  4. TypeScript notification handling
  5. TypeScript finalization and aggregation
  6. JavaScript summary generation and metrics

💻 Dive into the Code

Want to explore multi-language workflows further? Check out additional examples and the complete source code:

Multi-Language Examples

Access complete multi-language implementations, configuration examples, and learn how to integrate TypeScript, Python, and JavaScript in production applications.


Conclusion: The Power of Unification Through Steps

This multi-language data processing pipeline demonstrates how steps fundamentally change multi-language development. By providing a single primitive that works across TypeScript, Python, and JavaScript, we've eliminated the traditional complexity of polyglot architectures.

The step primitive enables true unification:

  • Universal Pattern - Every step, regardless of language, follows the same receive-process-emit pattern
  • Seamless Integration - Add Ruby, Go, Rust, or any language using the same step abstraction
  • Unified Deployment - All languages deploy together as a single, coherent system
  • Shared Development Model - Write, debug, and monitor everything through the same interface

Key benefits of step-based unification:

  • Single Mental Model - Learn the step pattern once, apply it to any language
  • Cohesive System - All components work together as parts of one application, not separate services
  • Consistent Experience - Development, debugging, and monitoring work the same way across all languages
  • Natural Scaling - Each step can scale independently while maintaining system coherence

Extend your pipeline with more steps:

  • Add specialized processing steps for different data types and business logic
  • Integrate machine learning workflows with Python steps for AI processing
  • Build real-time analytics with streaming steps for live data processing
  • Connect to enterprise systems through database and API integration steps
  • Implement scheduled processing with cron steps for batch operations

The step primitive makes all extensions natural and straightforward—every new capability follows the same unified pattern.

Ready to unify your multi-language systems? Start building with steps today!

Need help? See our Community Resources for questions, examples, and discussions.