Datadog入門 第4部 - アプリケーション監視の実践完全ガイド

インフラストラクチャ監視の基盤を構築したら、次はアプリケーション層での監視実装です。本記事では、Application Performance Monitoring (APM)言語別トレーシング設定、**Real User Monitoring (RUM)**まで、アプリケーション監視の全領域を実践的に解説します。分散システムの複雑な依存関係を可視化し、ユーザー体験を最適化するための包括的ガイドです。

4.1 Application Performance Monitoring (APM)

APMの基本概念とメリット

APMとは何か

Application Performance Monitoring (APM)は、アプリケーションのパフォーマンス、可用性、ユーザー体験を包括的に監視する手法です。Datadogでは、分散トレーシングプロファイリングエラー追跡を統合し、アプリケーションの内部動作を完全に可視化します。

APMが解決する課題

yaml
従来の課題:
  - 複雑な依存関係での問題切り分けの困難さ
  - マイクロサービス間通信の可視性不足
  - パフォーマンスボトルネックの特定の困難
  - 本番環境でのデバッグの制約

DatadogのAPMによる解決:
  - 分散トレーシングによる全体俯瞰
  - サービスマップでの依存関係可視化
  - リアルタイムパフォーマンス分析
  - プロダクション環境でのディープな洞察

分散トレーシングの実装

トレーシングの基本概念

分散トレーシングは、マイクロサービス架構において、一つのリクエストが複数のサービス間を移動する際の経路とパフォーマンスを追跡する技術です。

python
# Python Flask アプリケーションでのDatadog APM設定例
from ddtrace import tracer
from ddtrace.contrib.flask import TraceMiddleware
from flask import Flask

app = Flask(__name__)

# Datadogトレーシングの有効化
TraceMiddleware(app, tracer, service="web-api")

@app.route('/api/users/<int:user_id>')
def get_user(user_id):
    # 自動的にスパンが作成される
    with tracer.trace("database.query", service="postgres") as span:
        span.set_tag("user.id", user_id)
        span.set_tag("query.table", "users")
        
        # データベースクエリの実行
        user = db.get_user(user_id)
        
        if user:
            span.set_tag("query.result", "found")
            return {"user": user}
        else:
            span.set_tag("query.result", "not_found")
            span.set_tag("error", True)
            return {"error": "User not found"}, 404

サービスマップによる依存関係可視化

サービスマップは、アプリケーションのサービス間依存関係をリアルタイムで可視化し、パフォーマンスのボトルネックや障害の影響範囲を即座に特定できます。

javascript
// Node.js Express でのサービス間通信トレーシング
const tracer = require('dd-trace').init({
  service: 'order-service',
  env: 'production',
  version: '1.2.0'
});

const express = require('express');
const axios = require('axios');

app.post('/orders', async (req, res) => {
  const span = tracer.startSpan('order.create');
  
  try {
    // ユーザーサービスへの呼び出し
    const userSpan = tracer.startSpan('user.validate', {
      childOf: span
    });
    
    const userResponse = await axios.get(`/users/${req.body.userId}`, {
      headers: {
        'x-datadog-trace-id': userSpan.context().toTraceId(),
        'x-datadog-parent-id': userSpan.context().toSpanId()
      }
    });
    userSpan.finish();
    
    // 在庫サービスへの呼び出し
    const inventorySpan = tracer.startSpan('inventory.check', {
      childOf: span
    });
    
    const inventoryResponse = await axios.post('/inventory/reserve', 
      req.body.items,
      {
        headers: {
          'x-datadog-trace-id': inventorySpan.context().toTraceId(),
          'x-datadog-parent-id': inventorySpan.context().toSpanId()
        }
      }
    );
    inventorySpan.finish();
    
    // オーダー作成
    const order = await createOrder(req.body);
    
    span.setTag('order.id', order.id);
    span.setTag('order.amount', order.total);
    span.finish();
    
    res.json(order);
    
  } catch (error) {
    span.setTag('error', true);
    span.setTag('error.message', error.message);
    span.finish();
    
    res.status(500).json({ error: 'Order creation failed' });
  }
});

パフォーマンス分析と最適化

レイテンシ分析

P50、P95、P99パーセンタイルによるレイテンシ分析により、アプリケーションのパフォーマンス特性を詳細に把握できます。

yaml
# パフォーマンス監視のベストプラクティス
レイテンシしきい値設定:
  P50: 100ms  # 通常のユーザー体験
  P95: 500ms  # 95%のリクエストが満たすべき基準
  P99: 1000ms # ワーストケースの許容範囲

監視すべきメトリクス:
  - リクエスト/秒 (RPS)
  - エラー率
  - スループット
  - リソース使用率
  - 依存サービスの健全性

スロークエリ特定と最適化

go
// Go言語でのデータベースクエリトレーシング
package main

import (
    "context"
    "database/sql"
    "time"
    
    "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
    sqltrace "gopkg.in/DataDog/dd-trace-go.v1/contrib/database/sql"
    
    _ "github.com/lib/pq"
)

func main() {
    // Datadogトレーサー初期化
    tracer.Start(
        tracer.WithService("product-service"),
        tracer.WithEnv("production"),
    )
    defer tracer.Stop()
    
    // データベース接続にトレーシングを追加
    sqltrace.Register("postgres", &pq.Driver{})
    db, err := sqltrace.Open("postgres", "postgres://localhost/mydb")
    if err != nil {
        panic(err)
    }
    defer db.Close()
}

func getProducts(ctx context.Context, db *sql.DB, categoryID int) ([]Product, error) {
    span, ctx := tracer.StartSpanFromContext(ctx, "database.query")
    defer span.Finish()
    
    span.SetTag("sql.query", "SELECT * FROM products WHERE category_id = $1")
    span.SetTag("db.instance", "products")
    
    start := time.Now()
    
    rows, err := db.QueryContext(ctx, `
        SELECT id, name, price, description 
        FROM products 
        WHERE category_id = $1 AND active = true
        ORDER BY popularity DESC
        LIMIT 100
    `, categoryID)
    
    duration := time.Since(start)
    span.SetTag("query.duration", duration.Milliseconds())
    
    if err != nil {
        span.SetTag("error", true)
        span.SetTag("error.message", err.Error())
        return nil, err
    }
    defer rows.Close()
    
    // 結果処理...
    var products []Product
    for rows.Next() {
        // スキャン処理
    }
    
    span.SetTag("query.rows_returned", len(products))
    return products, nil
}

4.2 プログラミング言語別設定

Java/.NET アプリケーション監視

Java Spring Boot アプリケーション

java
// Java Spring Boot でのDatadog APM設定
// application.yml
management:
  metrics:
    export:
      datadog:
        api-key: ${DD_API_KEY}
        enabled: true
        step: 30s
        uri: https://api.datadoghq.com

// Spring Boot Application クラス
@SpringBootApplication
@EnableScheduling
public class EcommerceApplication {
    
    public static void main(String[] args) {
        // Datadogトレーサーの初期化
        System.setProperty("dd.service", "ecommerce-api");
        System.setProperty("dd.env", "production");
        System.setProperty("dd.version", "2.1.0");
        System.setProperty("dd.profiling.enabled", "true");
        
        SpringApplication.run(EcommerceApplication.class, args);
    }
}

// REST Controller でのトレーシング
@RestController
@RequestMapping("/api/v1")
public class ProductController {
    
    @Autowired
    private ProductService productService;
    
    @GetMapping("/products/{id}")
    @Timed(value = "product.get", description = "Time taken to get product")
    public ResponseEntity<Product> getProduct(@PathVariable Long id) {
        
        Span span = GlobalTracer.get().activeSpan();
        if (span != null) {
            span.setTag("product.id", id);
            span.setTag("controller", "ProductController");
        }
        
        try {
            Product product = productService.findById(id);
            
            if (product != null) {
                span.setTag("product.found", true);
                span.setTag("product.category", product.getCategory());
                return ResponseEntity.ok(product);
            } else {
                span.setTag("product.found", false);
                return ResponseEntity.notFound().build();
            }
            
        } catch (Exception e) {
            span.setTag("error", true);
            span.setTag("error.message", e.getMessage());
            throw e;
        }
    }
}

// サービス層でのカスタムトレーシング
@Service
@Transactional
public class ProductService {
    
    @Autowired
    private ProductRepository productRepository;
    
    @Autowired
    private CacheService cacheService;
    
    @NewSpan("product.service.findById")
    public Product findById(Long id) {
        
        // キャッシュから取得を試行
        Span cacheSpan = GlobalTracer.get()
            .buildSpan("cache.get")
            .start();
        
        try {
            Product cachedProduct = cacheService.get("product:" + id);
            if (cachedProduct != null) {
                cacheSpan.setTag("cache.hit", true);
                return cachedProduct;
            }
            cacheSpan.setTag("cache.hit", false);
        } finally {
            cacheSpan.finish();
        }
        
        // データベースから取得
        Span dbSpan = GlobalTracer.get()
            .buildSpan("database.findById")
            .start();
        
        try {
            Product product = productRepository.findById(id)
                .orElse(null);
            
            if (product != null) {
                // キャッシュに保存
                cacheService.put("product:" + id, product, Duration.ofMinutes(30));
                dbSpan.setTag("db.operation", "SELECT");
                dbSpan.setTag("db.statement", "SELECT * FROM products WHERE id = ?");
            }
            
            return product;
            
        } finally {
            dbSpan.finish();
        }
    }
}

.NET Core アプリケーション

csharp
// .NET Core でのDatadog APM設定
// Startup.cs
using Datadog.Trace;
using Datadog.Trace.Configuration;

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        // Datadogトレーシング設定
        var settings = TracerSettings.FromDefaultSources();
        settings.ServiceName = "inventory-api";
        settings.Environment = "production";
        settings.ServiceVersion = "1.0.0";
        
        Tracer.Configure(settings);
        
        services.AddControllers();
        services.AddScoped<IInventoryService, InventoryService>();
    }
    
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        app.UseRouting();
        app.UseEndpoints(endpoints =>
        {
            endpoints.MapControllers();
        });
    }
}

// Controller でのカスタムトレーシング
[ApiController]
[Route("api/[controller]")]
public class InventoryController : ControllerBase
{
    private readonly IInventoryService _inventoryService;
    
    public InventoryController(IInventoryService inventoryService)
    {
        _inventoryService = inventoryService;
    }
    
    [HttpPost("reserve")]
    public async Task<IActionResult> ReserveItems([FromBody] ReservationRequest request)
    {
        using var scope = Tracer.Instance.StartActive("inventory.reserve");
        var span = scope.Span;
        
        span.SetTag("request.items_count", request.Items.Count);
        span.SetTag("request.customer_id", request.CustomerId);
        
        try
        {
            var result = await _inventoryService.ReserveItemsAsync(request);
            
            span.SetTag("reservation.success", result.Success);
            span.SetTag("reservation.id", result.ReservationId);
            
            if (result.Success)
            {
                return Ok(result);
            }
            else
            {
                span.SetTag("reservation.failure_reason", result.FailureReason);
                return BadRequest(result);
            }
        }
        catch (Exception ex)
        {
            span.SetTag("error", true);
            span.SetTag("error.message", ex.Message);
            span.SetTag("error.type", ex.GetType().Name);
            throw;
        }
    }
}

// サービス層でのデータベース監視
public class InventoryService : IInventoryService
{
    private readonly IDbConnection _connection;
    
    public async Task<ReservationResult> ReserveItemsAsync(ReservationRequest request)
    {
        using var scope = Tracer.Instance.StartActive("inventory.service.reserve");
        var span = scope.Span;
        
        using var transaction = _connection.BeginTransaction();
        
        try
        {
            foreach (var item in request.Items)
            {
                using var itemScope = Tracer.Instance.StartActive("inventory.check_availability");
                var itemSpan = itemScope.Span;
                
                itemSpan.SetTag("item.sku", item.Sku);
                itemSpan.SetTag("item.quantity_requested", item.Quantity);
                
                var availability = await CheckAvailabilityAsync(item.Sku);
                itemSpan.SetTag("item.quantity_available", availability.Available);
                
                if (availability.Available < item.Quantity)
                {
                    itemSpan.SetTag("availability.sufficient", false);
                    transaction.Rollback();
                    
                    return new ReservationResult
                    {
                        Success = false,
                        FailureReason = $"Insufficient inventory for {item.Sku}"
                    };
                }
                
                itemSpan.SetTag("availability.sufficient", true);
                await ReserveItemAsync(item.Sku, item.Quantity, transaction);
            }
            
            transaction.Commit();
            span.SetTag("transaction.status", "committed");
            
            return new ReservationResult
            {
                Success = true,
                ReservationId = Guid.NewGuid().ToString()
            };
        }
        catch
        {
            transaction.Rollback();
            span.SetTag("transaction.status", "rolled_back");
            throw;
        }
    }
}

Python/Ruby アプリケーション監視

Python Django/Flask アプリケーション

python
# Django でのDatadog APM設定
# settings.py
import os
from ddtrace import config

# Datadogトレーシング設定
config.django['service_name'] = 'blog-api'
config.django['cache_service_name'] = 'blog-cache'
config.django['database_service_name'] = 'blog-db'

INSTALLED_APPS = [
    'ddtrace.contrib.django',
    # 他のアプリ...
]

MIDDLEWARE = [
    'ddtrace.contrib.django.TraceMiddleware',
    # 他のミドルウェア...
]

# カスタムビューでのトレーシング
from ddtrace import tracer
from django.http import JsonResponse
from django.views import View

class ArticleDetailView(View):
    def get(self, request, article_id):
        # カスタムスパンの作成
        with tracer.trace("article.fetch", service="blog-api") as span:
            span.set_tag("article.id", article_id)
            span.set_tag("user.authenticated", request.user.is_authenticated)
            
            try:
                # キャッシュから記事を取得
                with tracer.trace("cache.get", service="redis") as cache_span:
                    article = cache.get(f"article:{article_id}")
                    cache_span.set_tag("cache.key", f"article:{article_id}")
                    
                    if article:
                        cache_span.set_tag("cache.hit", True)
                        span.set_tag("source", "cache")
                    else:
                        cache_span.set_tag("cache.hit", False)
                
                # データベースから取得
                if not article:
                    with tracer.trace("database.query", service="postgresql") as db_span:
                        article = Article.objects.select_related('author', 'category')\
                                                .get(id=article_id)
                        
                        db_span.set_tag("db.table", "articles")
                        db_span.set_tag("db.operation", "SELECT")
                        span.set_tag("source", "database")
                        
                        # キャッシュに保存
                        cache.set(f"article:{article_id}", article, timeout=3600)
                
                # アクセス数増加
                with tracer.trace("analytics.increment", service="analytics") as analytics_span:
                    analytics_span.set_tag("article.id", article_id)
                    analytics_span.set_tag("user.id", request.user.id if request.user.is_authenticated else None)
                    
                    # 非同期でアクセス数を更新
                    increment_view_count.delay(article_id, request.user.id)
                
                span.set_tag("article.title", article.title)
                span.set_tag("article.category", article.category.name)
                span.set_tag("response.status", "success")
                
                return JsonResponse({
                    'article': {
                        'id': article.id,
                        'title': article.title,
                        'content': article.content,
                        'author': article.author.username,
                        'category': article.category.name,
                        'published_at': article.published_at.isoformat()
                    }
                })
                
            except Article.DoesNotExist:
                span.set_tag("error", True)
                span.set_tag("error.type", "NotFound")
                span.set_tag("response.status", "not_found")
                
                return JsonResponse({'error': 'Article not found'}, status=404)
            
            except Exception as e:
                span.set_tag("error", True)
                span.set_tag("error.message", str(e))
                span.set_tag("error.type", type(e).__name__)
                span.set_tag("response.status", "error")
                
                return JsonResponse({'error': 'Internal server error'}, status=500)

# Celeryタスクでのトレーシング
from celery import Celery
from ddtrace import patch

# Celeryにパッチを適用
patch(celery=True)

app = Celery('blog')

@app.task
def increment_view_count(article_id, user_id=None):
    with tracer.trace("task.increment_view_count", service="worker") as span:
        span.set_tag("article.id", article_id)
        span.set_tag("user.id", user_id)
        
        try:
            # データベース更新
            with tracer.trace("database.update", service="postgresql") as db_span:
                article = Article.objects.get(id=article_id)
                article.view_count += 1
                article.save(update_fields=['view_count'])
                
                db_span.set_tag("db.table", "articles")
                db_span.set_tag("db.operation", "UPDATE")
            
            # 統計情報更新
            with tracer.trace("analytics.update", service="analytics") as analytics_span:
                # Redis での集計情報更新
                redis_client.hincrby(f"daily_views:{date.today()}", article_id, 1)
                
                if user_id:
                    redis_client.sadd(f"article_viewers:{article_id}", user_id)
                
                analytics_span.set_tag("analytics.type", "view_increment")
            
            span.set_tag("task.status", "success")
            
        except Exception as e:
            span.set_tag("error", True)
            span.set_tag("error.message", str(e))
            span.set_tag("task.status", "failed")
            raise

Ruby on Rails アプリケーション

ruby
# Ruby on Rails でのDatadog APM設定
# config/initializers/datadog.rb
require 'ddtrace'

Datadog.configure do |c|
  # トレーシング設定
  c.service = 'rails-ecommerce'
  c.env = Rails.env
  c.version = '1.0.0'
  
  # 自動インストルメンテーション
  c.use :rails, service_name: 'web', controller_service: 'controller', cache_service: 'cache'
  c.use :active_record, service_name: 'db'
  c.use :redis, service_name: 'cache'
  c.use :sidekiq, service_name: 'worker'
  
  # サンプリング設定
  c.sampling.default_rate = 1.0
end

# コントローラーでのカスタムトレーシング
class ProductsController < ApplicationController
  def show
    # カスタムスパンの開始
    Datadog::Tracing.trace('product.fetch') do |span|
      span.set_tag('product.id', params[:id])
      span.set_tag('user.authenticated', user_signed_in?)
      
      begin
        # キャッシュから製品情報を取得
        Datadog::Tracing.trace('cache.get') do |cache_span|
          @product = Rails.cache.fetch("product:#{params[:id]}", expires_in: 1.hour) do
            cache_span.set_tag('cache.miss', true)
            
            # データベースから取得
            Datadog::Tracing.trace('database.query') do |db_span|
              product = Product.includes(:category, :reviews)
                              .find(params[:id])
              
              db_span.set_tag('db.table', 'products')
              db_span.set_tag('db.operation', 'SELECT')
              
              product
            end
          end
          
          if @product.persisted?
            cache_span.set_tag('cache.hit', true)
          end
        end
        
        # レコメンデーション取得
        Datadog::Tracing.trace('recommendation.fetch') do |rec_span|
          @recommended_products = RecommendationService.new(@product, current_user)
                                                      .get_recommendations(limit: 5)
          
          rec_span.set_tag('recommendation.count', @recommended_products.size)
          rec_span.set_tag('recommendation.algorithm', 'collaborative_filtering')
        end
        
        # 閲覧履歴の記録
        if user_signed_in?
          Datadog::Tracing.trace('analytics.track_view') do |track_span|
            ViewTrackingJob.perform_later(current_user.id, @product.id)
            track_span.set_tag('user.id', current_user.id)
          end
        end
        
        span.set_tag('product.category', @product.category.name)
        span.set_tag('product.price', @product.price)
        span.set_tag('response.status', 'success')
        
      rescue ActiveRecord::RecordNotFound
        span.set_tag('error', true)
        span.set_tag('error.type', 'NotFound')
        
        redirect_to products_path, alert: '製品が見つかりません'
        
      rescue => e
        span.set_tag('error', true)
        span.set_tag('error.message', e.message)
        span.set_tag('error.type', e.class.name)
        
        redirect_to products_path, alert: 'エラーが発生しました'
      end
    end
  end
end

# サービスクラスでのトレーシング
class RecommendationService
  def initialize(product, user = nil)
    @product = product
    @user = user
  end
  
  def get_recommendations(limit: 10)
    Datadog::Tracing.trace('recommendation.calculate') do |span|
      span.set_tag('product.id', @product.id)
      span.set_tag('user.id', @user&.id)
      span.set_tag('limit', limit)
      
      recommendations = []
      
      # 協調フィルタリング
      if @user
        Datadog::Tracing.trace('recommendation.collaborative_filtering') do |cf_span|
          similar_users = find_similar_users(@user)
          cf_span.set_tag('similar_users.count', similar_users.size)
          
          recommendations += collaborative_filtering_recommendations(similar_users, limit / 2)
          cf_span.set_tag('recommendations.count', recommendations.size)
        end
      end
      
      # コンテンツベースフィルタリング
      Datadog::Tracing.trace('recommendation.content_based') do |cb_span|
        content_based = content_based_recommendations(@product, limit - recommendations.size)
        recommendations += content_based
        
        cb_span.set_tag('content_based.count', content_based.size)
      end
      
      # ポピュラリティベース(フォールバック)
      if recommendations.size < limit
        Datadog::Tracing.trace('recommendation.popularity_based') do |pb_span|
          popular = popularity_based_recommendations(limit - recommendations.size)
          recommendations += popular
          
          pb_span.set_tag('popular.count', popular.size)
        end
      end
      
      span.set_tag('final_recommendations.count', recommendations.size)
      span.set_tag('algorithm.used', determine_primary_algorithm(recommendations))
      
      recommendations.first(limit)
    end
  end
  
  private
  
  def find_similar_users(user)
    Datadog::Tracing.trace('recommendation.find_similar_users') do |span|
      # 機械学習モデルやコサイン類似度を使用
      similar = User.joins(:product_views)
                   .where.not(id: user.id)
                   .group('users.id')
                   .having('COUNT(product_views.id) > ?', 5)
                   .limit(50)
      
      span.set_tag('query.similar_users', similar.size)
      similar
    end
  end
end

# Sidekiqワーカーでのトレーシング
class ViewTrackingJob < ApplicationJob
  def perform(user_id, product_id)
    Datadog::Tracing.trace('job.view_tracking') do |span|
      span.set_tag('user.id', user_id)
      span.set_tag('product.id', product_id)
      
      begin
        # ビュー記録の作成
        Datadog::Tracing.trace('database.insert') do |db_span|
          ProductView.create!(
            user_id: user_id,
            product_id: product_id,
            viewed_at: Time.current
          )
          
          db_span.set_tag('db.table', 'product_views')
          db_span.set_tag('db.operation', 'INSERT')
        end
        
        # Redis でのリアルタイム統計更新
        Datadog::Tracing.trace('cache.analytics_update') do |cache_span|
          redis = Redis.current
          
          # 日次ビュー数
          redis.hincrby("daily_views:#{Date.current}", product_id, 1)
          
          # ユーザーの閲覧履歴
          redis.lpush("user_views:#{user_id}", product_id)
          redis.ltrim("user_views:#{user_id}", 0, 99)  # 最新100件を保持
          
          cache_span.set_tag('analytics.type', 'view_tracking')
        end
        
        span.set_tag('job.status', 'success')
        
      rescue => e
        span.set_tag('error', true)
        span.set_tag('error.message', e.message)
        span.set_tag('job.status', 'failed')
        
        raise e
      end
    end
  end
end

Node.js/Go アプリケーション監視

Node.js Express アプリケーション

javascript
// Node.js Express でのDatadog APM設定
const tracer = require('dd-trace').init({
  service: 'api-gateway',
  env: process.env.NODE_ENV,
  version: '1.0.0',
  profiling: true,
  runtimeMetrics: true
});

const express = require('express');
const Redis = require('redis');
const { Pool } = require('pg');

const app = express();
const redis = Redis.createClient();
const pgPool = new Pool({
  connectionString: process.env.DATABASE_URL
});

// ミドルウェアでのリクエストトレーシング
app.use((req, res, next) => {
  const span = tracer.startSpan('http.request');
  
  span.setTag('http.method', req.method);
  span.setTag('http.url', req.url);
  span.setTag('user.agent', req.headers['user-agent']);
  
  // レスポンス完了時の処理
  res.on('finish', () => {
    span.setTag('http.status_code', res.statusCode);
    span.setTag('response.size', res.get('content-length') || 0);
    
    if (res.statusCode >= 400) {
      span.setTag('error', true);
    }
    
    span.finish();
  });
  
  req.span = span;
  next();
});

// APIエンドポイントの実装
app.get('/api/users/:userId/orders', async (req, res) => {
  const span = tracer.startSpan('orders.get_user_orders', {
    childOf: req.span
  });
  
  try {
    const { userId } = req.params;
    const { page = 1, limit = 20 } = req.query;
    
    span.setTag('user.id', userId);
    span.setTag('pagination.page', page);
    span.setTag('pagination.limit', limit);
    
    // キャッシュキーの生成
    const cacheKey = `user_orders:${userId}:${page}:${limit}`;
    
    // Redis キャッシュから取得を試行
    const cacheSpan = tracer.startSpan('cache.get', {
      childOf: span
    });
    
    let orders;
    try {
      const cachedOrders = await redis.get(cacheKey);
      
      if (cachedOrders) {
        cacheSpan.setTag('cache.hit', true);
        orders = JSON.parse(cachedOrders);
        span.setTag('data.source', 'cache');
      } else {
        cacheSpan.setTag('cache.hit', false);
      }
    } finally {
      cacheSpan.finish();
    }
    
    // データベースから取得
    if (!orders) {
      const dbSpan = tracer.startSpan('database.query', {
        childOf: span
      });
      
      try {
        const offset = (page - 1) * limit;
        
        const query = `
          SELECT o.id, o.total, o.status, o.created_at,
                 json_agg(json_build_object(
                   'id', oi.id,
                   'product_name', p.name,
                   'quantity', oi.quantity,
                   'price', oi.price
                 )) as items
          FROM orders o
          JOIN order_items oi ON o.id = oi.order_id
          JOIN products p ON oi.product_id = p.id
          WHERE o.user_id = $1
          GROUP BY o.id, o.total, o.status, o.created_at
          ORDER BY o.created_at DESC
          LIMIT $2 OFFSET $3
        `;
        
        dbSpan.setTag('db.statement', query);
        dbSpan.setTag('db.user', 'api_user');
        
        const result = await pgPool.query(query, [userId, limit, offset]);
        orders = result.rows;
        
        dbSpan.setTag('db.rows_affected', orders.length);
        span.setTag('data.source', 'database');
        
        // キャッシュに保存
        await redis.setex(cacheKey, 300, JSON.stringify(orders)); // 5分間キャッシュ
        
      } finally {
        dbSpan.finish();
      }
    }
    
    // 各注文に対して追加情報を取得
    for (let order of orders) {
      const enrichSpan = tracer.startSpan('order.enrich', {
        childOf: span
      });
      
      try {
        enrichSpan.setTag('order.id', order.id);
        
        // 配送情報の取得(マイクロサービス呼び出し)
        const shippingSpan = tracer.startSpan('shipping.get_status', {
          childOf: enrichSpan
        });
        
        try {
          const shippingResponse = await fetch(`${process.env.SHIPPING_SERVICE_URL}/orders/${order.id}/shipping`, {
            headers: {
              'x-datadog-trace-id': shippingSpan.context().toTraceId(),
              'x-datadog-parent-id': shippingSpan.context().toSpanId()
            }
          });
          
          if (shippingResponse.ok) {
            order.shipping = await shippingResponse.json();
            shippingSpan.setTag('shipping.status', order.shipping.status);
          }
          
        } finally {
          shippingSpan.finish();
        }
        
      } finally {
        enrichSpan.finish();
      }
    }
    
    span.setTag('orders.count', orders.length);
    span.setTag('response.status', 'success');
    
    res.json({
      orders,
      pagination: {
        page: parseInt(page),
        limit: parseInt(limit),
        total: orders.length
      }
    });
    
  } catch (error) {
    span.setTag('error', true);
    span.setTag('error.message', error.message);
    span.setTag('error.stack', error.stack);
    
    console.error('Error fetching user orders:', error);
    res.status(500).json({ error: 'Internal server error' });
    
  } finally {
    span.finish();
  }
});

// カスタムメトリクスの送信
const StatsD = require('node-statsd');
const dogstatsd = new StatsD({
  host: 'localhost',
  port: 8125,
  prefix: 'api_gateway.'
});

// ビジネスメトリクスの追跡
app.post('/api/orders', async (req, res) => {
  const span = tracer.startSpan('orders.create');
  
  try {
    // 注文作成処理...
    const order = await createOrder(req.body);
    
    // カスタムメトリクスの送信
    dogstatsd.increment('orders.created', 1, {
      payment_method: order.payment_method,
      user_type: order.user.type,
      order_value_tier: getOrderValueTier(order.total)
    });
    
    dogstatsd.histogram('orders.value', order.total, {
      currency: order.currency
    });
    
    span.setTag('order.id', order.id);
    span.setTag('order.total', order.total);
    span.setTag('order.payment_method', order.payment_method);
    
    res.status(201).json(order);
    
  } catch (error) {
    dogstatsd.increment('orders.creation_failed', 1, {
      error_type: error.constructor.name
    });
    
    span.setTag('error', true);
    span.setTag('error.message', error.message);
    
    res.status(500).json({ error: 'Order creation failed' });
    
  } finally {
    span.finish();
  }
});

function getOrderValueTier(total) {
  if (total < 50) return 'low';
  if (total < 200) return 'medium';
  if (total < 500) return 'high';
  return 'premium';
}

Go アプリケーション

go
// Go でのDatadog APM設定
package main

import (
    "context"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "strconv"
    "time"
    
    "github.com/gorilla/mux"
    "github.com/go-redis/redis/v8"
    "github.com/jmoiron/sqlx"
    _ "github.com/lib/pq"
    
    "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
    "gopkg.in/DataDog/dd-trace-go.v1/contrib/gorilla/mux"
    "gopkg.in/DataDog/dd-trace-go.v1/contrib/go-redis/redis"
    "gopkg.in/DataDog/dd-trace-go.v1/contrib/jmoiron/sqlx"
)

type Server struct {
    db    *sqlx.DB
    redis *redis.Client
}

type Product struct {
    ID          int       `json:"id" db:"id"`
    Name        string    `json:"name" db:"name"`
    Price       float64   `json:"price" db:"price"`
    Category    string    `json:"category" db:"category"`
    Description string    `json:"description" db:"description"`
    CreatedAt   time.Time `json:"created_at" db:"created_at"`
}

func main() {
    // Datadogトレーサーの初期化
    tracer.Start(
        tracer.WithService("product-service"),
        tracer.WithEnv("production"),
        tracer.WithServiceVersion("1.0.0"),
        tracer.WithRuntimeMetrics(),
        tracer.WithProfilerCodeHotspots(true),
        tracer.WithProfilerEndpoints(true),
    )
    defer tracer.Stop()
    
    // データベース接続
    db, err := sqlx.Connect("postgres", "postgresql://localhost/products?sslmode=disable")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()
    
    // Redis接続
    rdb := redis.NewClient(&redis.Options{
        Addr: "localhost:6379",
    })
    
    server := &Server{
        db:    db,
        redis: rdb,
    }
    
    // ルーターの設定
    r := mux.NewRouter()
    r.Use(muxtrace.Middleware(
        muxtrace.WithServiceName("product-service"),
    ))
    
    r.HandleFunc("/products", server.getProducts).Methods("GET")
    r.HandleFunc("/products/{id}", server.getProduct).Methods("GET")
    r.HandleFunc("/products", server.createProduct).Methods("POST")
    
    log.Println("Server starting on :8080")
    log.Fatal(http.ListenAndServe(":8080", r))
}

func (s *Server) getProducts(w http.ResponseWriter, r *http.Request) {
    span, ctx := tracer.StartSpanFromContext(r.Context(), "products.list")
    defer span.Finish()
    
    // クエリパラメータの取得
    category := r.URL.Query().Get("category")
    page, _ := strconv.Atoi(r.URL.Query().Get("page"))
    if page <= 0 {
        page = 1
    }
    limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
    if limit <= 0 || limit > 100 {
        limit = 20
    }
    
    span.SetTag("query.category", category)
    span.SetTag("query.page", page)
    span.SetTag("query.limit", limit)
    
    // キャッシュキーの生成
    cacheKey := fmt.Sprintf("products:list:%s:%d:%d", category, page, limit)
    
    // Redis キャッシュから取得を試行
    cacheSpan, ctx := tracer.StartSpanFromContext(ctx, "cache.get")
    var products []Product
    
    cachedData, err := s.redis.Get(ctx, cacheKey).Result()
    if err == nil {
        cacheSpan.SetTag("cache.hit", true)
        json.Unmarshal([]byte(cachedData), &products)
        span.SetTag("data.source", "cache")
    } else {
        cacheSpan.SetTag("cache.hit", false)
        span.SetTag("data.source", "database")
    }
    cacheSpan.Finish()
    
    // データベースから取得
    if len(products) == 0 {
        dbSpan, ctx := tracer.StartSpanFromContext(ctx, "database.query")
        
        query := `
            SELECT id, name, price, category, description, created_at
            FROM products
            WHERE ($1 = '' OR category = $1)
            ORDER BY created_at DESC
            LIMIT $2 OFFSET $3
        `
        
        offset := (page - 1) * limit
        
        dbSpan.SetTag("db.statement", query)
        dbSpan.SetTag("db.operation", "SELECT")
        dbSpan.SetTag("db.table", "products")
        
        err := s.db.SelectContext(ctx, &products, query, category, limit, offset)
        if err != nil {
            dbSpan.SetTag("error", true)
            dbSpan.SetTag("error.message", err.Error())
            dbSpan.Finish()
            
            span.SetTag("error", true)
            span.SetTag("error.message", err.Error())
            
            http.Error(w, "Internal server error", http.StatusInternalServerError)
            return
        }
        
        dbSpan.SetTag("db.rows_returned", len(products))
        dbSpan.Finish()
        
        // キャッシュに保存
        if len(products) > 0 {
            cacheData, _ := json.Marshal(products)
            s.redis.Set(ctx, cacheKey, cacheData, 5*time.Minute)
        }
    }
    
    // 各製品の詳細情報を非同期で取得
    for i := range products {
        enrichSpan, enrichCtx := tracer.StartSpanFromContext(ctx, "product.enrich")
        
        // 在庫情報の取得
        stockSpan, _ := tracer.StartSpanFromContext(enrichCtx, "inventory.get_stock")
        stockLevel, err := s.getStockLevel(enrichCtx, products[i].ID)
        if err == nil {
            products[i].StockLevel = stockLevel
            stockSpan.SetTag("stock.level", stockLevel)
        } else {
            stockSpan.SetTag("error", true)
            stockSpan.SetTag("error.message", err.Error())
        }
        stockSpan.Finish()
        
        enrichSpan.Finish()
    }
    
    span.SetTag("products.count", len(products))
    span.SetTag("response.status", "success")
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(map[string]interface{}{
        "products": products,
        "pagination": map[string]int{
            "page":  page,
            "limit": limit,
            "total": len(products),
        },
    })
}

func (s *Server) getProduct(w http.ResponseWriter, r *http.Request) {
    span, ctx := tracer.StartSpanFromContext(r.Context(), "product.get")
    defer span.Finish()
    
    vars := mux.Vars(r)
    productID, err := strconv.Atoi(vars["id"])
    if err != nil {
        span.SetTag("error", true)
        span.SetTag("error.message", "Invalid product ID")
        
        http.Error(w, "Invalid product ID", http.StatusBadRequest)
        return
    }
    
    span.SetTag("product.id", productID)
    
    // キャッシュから取得を試行
    cacheSpan, ctx := tracer.StartSpanFromContext(ctx, "cache.get")
    cacheKey := fmt.Sprintf("product:%d", productID)
    
    var product Product
    cachedData, err := s.redis.Get(ctx, cacheKey).Result()
    if err == nil {
        cacheSpan.SetTag("cache.hit", true)
        json.Unmarshal([]byte(cachedData), &product)
        span.SetTag("data.source", "cache")
    } else {
        cacheSpan.SetTag("cache.hit", false)
    }
    cacheSpan.Finish()
    
    // データベースから取得
    if product.ID == 0 {
        dbSpan, ctx := tracer.StartSpanFromContext(ctx, "database.query")
        
        query := `
            SELECT id, name, price, category, description, created_at
            FROM products
            WHERE id = $1
        `
        
        dbSpan.SetTag("db.statement", query)
        dbSpan.SetTag("db.operation", "SELECT")
        dbSpan.SetTag("db.table", "products")
        
        err := s.db.GetContext(ctx, &product, query, productID)
        if err != nil {
            if err == sql.ErrNoRows {
                dbSpan.SetTag("db.rows_returned", 0)
                span.SetTag("error.type", "NotFound")
                
                http.Error(w, "Product not found", http.StatusNotFound)
            } else {
                dbSpan.SetTag("error", true)
                dbSpan.SetTag("error.message", err.Error())
                span.SetTag("error", true)
                span.SetTag("error.message", err.Error())
                
                http.Error(w, "Internal server error", http.StatusInternalServerError)
            }
            
            dbSpan.Finish()
            return
        }
        
        dbSpan.SetTag("db.rows_returned", 1)
        dbSpan.Finish()
        
        span.SetTag("data.source", "database")
        
        // キャッシュに保存
        productData, _ := json.Marshal(product)
        s.redis.Set(ctx, cacheKey, productData, 10*time.Minute)
    }
    
    // 関連情報の取得
    relationSpan, relCtx := tracer.StartSpanFromContext(ctx, "product.get_relations")
    
    // 在庫レベル
    stockSpan, _ := tracer.StartSpanFromContext(relCtx, "inventory.get_stock")
    stockLevel, _ := s.getStockLevel(relCtx, product.ID)
    product.StockLevel = stockLevel
    stockSpan.Finish()
    
    // レビュー数
    reviewSpan, _ := tracer.StartSpanFromContext(relCtx, "reviews.get_count")
    reviewCount, averageRating := s.getReviewStats(relCtx, product.ID)
    product.ReviewCount = reviewCount
    product.AverageRating = averageRating
    reviewSpan.SetTag("reviews.count", reviewCount)
    reviewSpan.SetTag("reviews.average_rating", averageRating)
    reviewSpan.Finish()
    
    relationSpan.Finish()
    
    span.SetTag("product.name", product.Name)
    span.SetTag("product.category", product.Category)
    span.SetTag("product.price", product.Price)
    span.SetTag("response.status", "success")
    
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(product)
}

func (s *Server) getStockLevel(ctx context.Context, productID int) (int, error) {
    span, ctx := tracer.StartSpanFromContext(ctx, "inventory.stock_level")
    defer span.Finish()
    
    span.SetTag("product.id", productID)
    
    var stockLevel int
    query := "SELECT stock_level FROM inventory WHERE product_id = $1"
    
    err := s.db.GetContext(ctx, &stockLevel, query, productID)
    if err != nil {
        span.SetTag("error", true)
        span.SetTag("error.message", err.Error())
        return 0, err
    }
    
    span.SetTag("stock.level", stockLevel)
    return stockLevel, nil
}

func (s *Server) getReviewStats(ctx context.Context, productID int) (int, float64) {
    span, ctx := tracer.StartSpanFromContext(ctx, "reviews.get_stats")
    defer span.Finish()
    
    span.SetTag("product.id", productID)
    
    var count int
    var avgRating float64
    
    query := `
        SELECT COUNT(*) as count, COALESCE(AVG(rating), 0) as avg_rating
        FROM reviews
        WHERE product_id = $1 AND approved = true
    `
    
    row := s.db.QueryRowContext(ctx, query, productID)
    row.Scan(&count, &avgRating)
    
    span.SetTag("reviews.count", count)
    span.SetTag("reviews.avg_rating", avgRating)
    
    return count, avgRating
}

4.3 Real User Monitoring (RUM)

RUMの概要とブラウザ監視

Real User Monitoringとは

**Real User Monitoring (RUM)**は、実際のユーザーのブラウザやモバイルアプリから収集されるパフォーマンスデータを基に、ユーザー体験を監視・分析する手法です。合成監視とは異なり、実際のユーザー環境での体験を直接測定します。

RUMが収集するメトリクス

yaml
Core Web Vitals:
  - LCP (Largest Contentful Paint): 最大コンテンツの描画時間
  - FID (First Input Delay): 初回入力遅延
  - CLS (Cumulative Layout Shift): 累積レイアウトシフト

パフォーマンスメトリクス:
  - FCP (First Contentful Paint): 初回コンテンツ描画
  - TTI (Time to Interactive): インタラクション可能時間
  - TBT (Total Blocking Time): 総ブロック時間
  - FMP (First Meaningful Paint): 初回意味のある描画

ユーザー行動メトリクス:
  - セッション時間
  - ページビュー数
  - バウンス率
  - コンバージョン率
  - エラー発生率

ブラウザRUMの実装

JavaScript SDK の設定

html
<!-- HTML での Datadog RUM SDK の設定 -->
<!DOCTYPE html>
<html lang="ja">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Eコマースサイト</title>
    
    <!-- Datadog RUM SDK の読み込み -->
    <script src="https://www.datadoghq-browser-agent.com/us1/v4/datadog-rum.js" type="text/javascript"></script>
    <script>
        window.DD_RUM && window.DD_RUM.init({
            applicationId: 'your-application-id',
            clientToken: 'your-client-token',
            site: 'datadoghq.com',
            service: 'ecommerce-frontend',
            env: 'production',
            version: '1.0.0',
            sessionSampleRate: 100,
            sessionReplaySampleRate: 20,
            trackUserInteractions: true,
            trackResources: true,
            trackLongTasks: true,
            defaultPrivacyLevel: 'mask-user-input'
        });
        
        // ユーザー情報の設定
        window.DD_RUM && window.DD_RUM.setUser({
            id: getUserId(), // ユーザーID(認証済みの場合)
            name: getUserName(),
            email: getUserEmail(),
            plan: getUserPlan(), // サブスクリプションプラン
            country: getUserCountry()
        });
        
        // グローバルコンテキストの設定
        window.DD_RUM && window.DD_RUM.setGlobalContextProperty('feature_flags', {
            new_checkout: true,
            recommendation_engine: 'v2',
            ab_test_variant: 'B'
        });
    </script>
</head>
<body>
    <!-- ページコンテンツ -->
</body>
</html>

React アプリケーションでのRUM設定

javascript
// React アプリケーションでの Datadog RUM 設定
import { datadogRum } from '@datadog/browser-rum';

// RUM の初期化
datadogRum.init({
    applicationId: process.env.REACT_APP_DATADOG_APPLICATION_ID,
    clientToken: process.env.REACT_APP_DATADOG_CLIENT_TOKEN,
    site: 'datadoghq.com',
    service: 'ecommerce-react',
    env: process.env.NODE_ENV,
    version: process.env.REACT_APP_VERSION,
    sessionSampleRate: 100,
    sessionReplaySampleRate: 20,
    trackUserInteractions: true,
    trackResources: true,
    trackLongTasks: true,
    allowedTracingUrls: [
        { match: /^https:\/\/api\.example\.com/, propagatorTypes: ['datadog'] },
        { match: /^https:\/\/checkout\.example\.com/, propagatorTypes: ['datadog'] }
    ]
});

// カスタムReact Hookでのページビュー追跡
import { useEffect } from 'react';
import { useLocation } from 'react-router-dom';

export function usePageTracking() {
    const location = useLocation();
    
    useEffect(() => {
        // ページビューの手動追跡
        datadogRum.startView({
            name: location.pathname,
            service: 'ecommerce-react',
            version: '1.0.0',
            context: {
                page_type: getPageType(location.pathname),
                user_segment: getUserSegment(),
                ab_test_variant: getABTestVariant()
            }
        });
    }, [location]);
}

// Eコマース固有のイベント追跡
export function trackEcommerceEvents() {
    // 商品閲覧の追跡
    const trackProductView = (product) => {
        datadogRum.addAction('product_view', {
            product_id: product.id,
            product_name: product.name,
            product_category: product.category,
            product_price: product.price,
            product_availability: product.in_stock
        });
        
        // カスタムメトリクス
        datadogRum.addTiming('product_load_time', performance.now());
    };
    
    // カートへの追加
    const trackAddToCart = (product, quantity) => {
        datadogRum.addAction('add_to_cart', {
            product_id: product.id,
            product_name: product.name,
            quantity: quantity,
            cart_value: getCartTotal(),
            user_type: getUserType()
        });
    };
    
    // 購入完了
    const trackPurchase = (order) => {
        datadogRum.addAction('purchase', {
            order_id: order.id,
            order_value: order.total,
            payment_method: order.payment_method,
            shipping_method: order.shipping_method,
            item_count: order.items.length,
            currency: order.currency
        });
        
        // ビジネス KPI
        datadogRum.setGlobalContextProperty('conversion', {
            funnel_step: 'completed',
            session_value: order.total,
            time_to_purchase: getSessionDuration()
        });
    };
    
    return {
        trackProductView,
        trackAddToCart,
        trackPurchase
    };
}

// エラー追跡の強化
export function enhanceErrorTracking() {
    // JavaScript エラーの追跡
    window.addEventListener('error', (event) => {
        datadogRum.addError(event.error, {
            error_type: 'javascript_error',
            file_name: event.filename,
            line_number: event.lineno,
            column_number: event.colno,
            user_agent: navigator.userAgent,
            page_url: window.location.href
        });
    });
    
    // Promise rejection の追跡
    window.addEventListener('unhandledrejection', (event) => {
        datadogRum.addError(event.reason, {
            error_type: 'unhandled_promise_rejection',
            promise_rejection_reason: event.reason.toString(),
            page_url: window.location.href
        });
    });
    
    // React Error Boundary での追跡
    class ErrorBoundary extends React.Component {
        componentDidCatch(error, errorInfo) {
            datadogRum.addError(error, {
                error_type: 'react_error_boundary',
                component_stack: errorInfo.componentStack,
                error_boundary: this.constructor.name,
                react_version: React.version
            });
        }
        
        render() {
            if (this.state.hasError) {
                return <ErrorFallback />;
            }
            
            return this.props.children;
        }
    }
}

// パフォーマンス監視の強化
export function enhancePerformanceMonitoring() {
    // カスタムタイミングの追跡
    const trackCustomTiming = (name, duration, context = {}) => {
        datadogRum.addTiming(name, duration, context);
    };
    
    // API 呼び出しパフォーマンス
    const originalFetch = window.fetch;
    window.fetch = async (...args) => {
        const start = performance.now();
        const url = args[0];
        
        try {
            const response = await originalFetch(...args);
            const duration = performance.now() - start;
            
            trackCustomTiming('api_call', duration, {
                url: url,
                method: args[1]?.method || 'GET',
                status: response.status,
                response_size: response.headers.get('content-length')
            });
            
            return response;
        } catch (error) {
            const duration = performance.now() - start;
            
            datadogRum.addError(error, {
                error_type: 'fetch_error',
                url: url,
                duration: duration
            });
            
            throw error;
        }
    };
    
    // Core Web Vitals の詳細追跡
    import { getCLS, getFID, getFCP, getLCP, getTTFB } from 'web-vitals';
    
    getCLS((metric) => {
        datadogRum.addAction('core_web_vital', {
            name: metric.name,
            value: metric.value,
            rating: metric.rating, // 'good', 'needs-improvement', 'poor'
            delta: metric.delta,
            entries: metric.entries.length
        });
    });
    
    getFID((metric) => {
        datadogRum.addAction('core_web_vital', {
            name: metric.name,
            value: metric.value,
            rating: metric.rating,
            delta: metric.delta
        });
    });
    
    getLCP((metric) => {
        datadogRum.addAction('core_web_vital', {
            name: metric.name,
            value: metric.value,
            rating: metric.rating,
            element: metric.entries[metric.entries.length - 1]?.element?.tagName
        });
    });
}

モバイルアプリ監視

iOS Swift アプリケーション

swift
// iOS Swift での Datadog RUM 設定
import DatadogRUM
import DatadogCore

class AppDelegate: UIResponder, UIApplicationDelegate {
    
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        
        // Datadog SDK の初期化
        Datadog.initialize(
            with: Datadog.Configuration(
                clientToken: "your-client-token",
                env: "production",
                service: "ecommerce-ios"
            ),
            trackingConsent: .granted
        )
        
        // RUM の設定
        RUM.enable(
            with: RUM.Configuration(
                applicationID: "your-application-id",
                sessionSampleRate: 100,
                uiKitViewsPredicate: DefaultUIKitRUMViewsPredicate(),
                uiKitActionsPredicate: DefaultUIKitRUMActionsPredicate(),
                urlSessionTracking: .init(
                    firstPartyHostsTracing: .trace(hosts: ["api.example.com"])
                )
            )
        )
        
        // ユーザー情報の設定
        RUM.setUserInfo(
            id: UserManager.shared.userId,
            name: UserManager.shared.userName,
            email: UserManager.shared.userEmail,
            extraInfo: [
                "subscription_plan": UserManager.shared.subscriptionPlan,
                "user_segment": UserManager.shared.userSegment,
                "device_type": UIDevice.current.model
            ]
        )
        
        // グローバルコンテキスト
        RUM.addUserExtraInfo([
            "app_version": Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "",
            "build_number": Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "",
            "ios_version": UIDevice.current.systemVersion
        ])
        
        return true
    }
}

// カスタムビューコントローラーでの監視
class ProductDetailViewController: UIViewController {
    var product: Product?
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        // ビュー固有のコンテキスト設定
        RUM.addUserExtraInfo([
            "screen_name": "product_detail",
            "product_id": product?.id ?? "",
            "product_category": product?.category ?? ""
        ])
    }
    
    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        
        // カスタムイベントの追跡
        RUM.addUserAction(
            type: .custom,
            name: "product_view",
            attributes: [
                "product_id": product?.id ?? "",
                "product_name": product?.name ?? "",
                "product_price": product?.price ?? 0,
                "view_source": "product_list"
            ]
        )
    }
    
    @IBAction func addToCartTapped(_ sender: UIButton) {
        let startTime = Date()
        
        // カート追加処理
        CartManager.shared.addProduct(product!) { [weak self] result in
            DispatchQueue.main.async {
                let duration = Date().timeIntervalSince(startTime)
                
                switch result {
                case .success:
                    // 成功時のイベント追跡
                    RUM.addUserAction(
                        type: .custom,
                        name: "add_to_cart_success",
                        attributes: [
                            "product_id": self?.product?.id ?? "",
                            "cart_total": CartManager.shared.cartTotal,
                            "response_time": duration * 1000 // ミリ秒
                        ]
                    )
                    
                    // カスタムタイミング
                    RUM.addTiming(name: "add_to_cart_duration", time: duration)
                    
                case .failure(let error):
                    // エラーの追跡
                    RUM.addError(
                        message: "Add to cart failed",
                        type: error.localizedDescription,
                        source: .source,
                        attributes: [
                            "product_id": self?.product?.id ?? "",
                            "error_code": (error as NSError).code,
                            "error_domain": (error as NSError).domain
                        ]
                    )
                }
            }
        }
    }
}

// ネットワーク通信の監視
class APIClient {
    static let shared = APIClient()
    
    func fetchProducts(category: String, completion: @escaping (Result<[Product], Error>) -> Void) {
        let startTime = Date()
        
        // リクエスト開始の追跡
        RUM.startResource(
            resourceKey: "fetch_products",
            request: URLRequest(url: URL(string: "https://api.example.com/products")!),
            attributes: [
                "api_endpoint": "fetch_products",
                "category": category,
                "request_id": UUID().uuidString
            ]
        )
        
        // 実際のAPI呼び出し
        NetworkManager.shared.get("/products", parameters: ["category": category]) { result in
            let duration = Date().timeIntervalSince(startTime)
            
            switch result {
            case .success(let data):
                let products = try? JSONDecoder().decode([Product].self, from: data)
                
                // 成功時のリソース追跡
                RUM.stopResource(
                    resourceKey: "fetch_products",
                    response: HTTPURLResponse(
                        url: URL(string: "https://api.example.com/products")!,
                        statusCode: 200,
                        httpVersion: nil,
                        headerFields: nil
                    )!,
                    size: data.count,
                    attributes: [
                        "products_count": products?.count ?? 0,
                        "response_time": duration * 1000,
                        "cache_hit": false
                    ]
                )
                
                completion(.success(products ?? []))
                
            case .failure(let error):
                // エラー時のリソース追跡
                RUM.stopResourceWithError(
                    resourceKey: "fetch_products",
                    error: error,
                    response: nil,
                    attributes: [
                        "error_type": type(of: error).description(),
                        "response_time": duration * 1000
                    ]
                )
                
                completion(.failure(error))
            }
        }
    }
}

Android Kotlin アプリケーション

kotlin
// Android Kotlin での Datadog RUM 設定
class EcommerceApplication : Application() {
    
    override fun onCreate() {
        super.onCreate()
        
        // Datadog SDK の初期化
        val configuration = Configuration.Builder(
            clientToken = "your-client-token",
            env = "production",
            variant = "release"
        )
            .setService("ecommerce-android")
            .useSite(DatadogSite.US1)
            .trackInteractions()
            .trackLongTasks(250)
            .useViewTrackingStrategy(ActivityViewTrackingStrategy(true))
            .build()
        
        Datadog.initialize(this, configuration, TrackingConsent.GRANTED)
        
        // RUM の設定
        val rumConfiguration = RumConfiguration.Builder("your-application-id")
            .sampleRumSessions(100f)
            .trackUserInteractions()
            .trackLongTasks(250)
            .useFirstPartyHosts(listOf("api.example.com"))
            .build()
        
        RUM.enable(rumConfiguration)
        
        // ユーザー情報の設定
        RUM.setUserInfo(
            id = UserManager.getUserId(),
            name = UserManager.getUserName(),
            email = UserManager.getUserEmail(),
            extraInfo = mapOf(
                "subscription_plan" to UserManager.getSubscriptionPlan(),
                "user_segment" to UserManager.getUserSegment(),
                "device_model" to Build.MODEL,
                "android_version" to Build.VERSION.RELEASE
            )
        )
        
        // グローバルコンテキスト
        GlobalRum.addAttribute("app_version", BuildConfig.VERSION_NAME)
        GlobalRum.addAttribute("build_number", BuildConfig.VERSION_CODE.toString())
    }
}

// カスタムActivityでの監視
class ProductDetailActivity : AppCompatActivity() {
    private lateinit var product: Product
    
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_product_detail)
        
        product = intent.getParcelableExtra("product") ?: return
        
        // アクティビティ固有のコンテキスト
        GlobalRum.addAttribute("screen_name", "product_detail")
        GlobalRum.addAttribute("product_id", product.id)
        GlobalRum.addAttribute("product_category", product.category)
        
        setupUI()
        loadProductDetails()
    }
    
    override fun onResume() {
        super.onResume()
        
        // カスタムイベントの追跡
        GlobalRum.addUserAction(
            RumActionType.CUSTOM,
            "product_view",
            mapOf(
                "product_id" to product.id,
                "product_name" to product.name,
                "product_price" to product.price,
                "view_source" to intent.getStringExtra("source")
            )
        )
    }
    
    private fun setupUI() {
        findViewById<Button>(R.id.btnAddToCart).setOnClickListener {
            addToCart()
        }
        
        findViewById<Button>(R.id.btnBuyNow).setOnClickListener {
            buyNow()
        }
    }
    
    private fun addToCart() {
        val startTime = System.currentTimeMillis()
        
        // カート追加処理
        CartManager.addProduct(product) { result ->
            val duration = System.currentTimeMillis() - startTime
            
            runOnUiThread {
                when (result) {
                    is Result.Success -> {
                        // 成功時のイベント追跡
                        GlobalRum.addUserAction(
                            RumActionType.CUSTOM,
                            "add_to_cart_success",
                            mapOf(
                                "product_id" to product.id,
                                "cart_total" to CartManager.getCartTotal(),
                                "response_time" to duration
                            )
                        )
                        
                        // カスタムタイミング
                        GlobalRum.addTiming("add_to_cart_duration", duration)
                        
                        showSuccess("商品をカートに追加しました")
                        
                    is Result.Error -> {
                        // エラーの追跡
                        GlobalRum.addError(
                            "Add to cart failed",
                            RumErrorSource.SOURCE,
                            result.exception,
                            mapOf(
                                "product_id" to product.id,
                                "error_message" to result.exception.message,
                                "response_time" to duration
                            )
                        )
                        
                        showError("カートへの追加に失敗しました")
                    }
                }
            }
        }
    }
    
    private fun loadProductDetails() {
        // プロダクト詳細の読み込み開始
        GlobalRum.startResource(
            "load_product_details",
            RumResourceMethod.GET,
            "https://api.example.com/products/${product.id}",
            mapOf(
                "api_endpoint" to "product_details",
                "product_id" to product.id
            )
        )
        
        val startTime = System.currentTimeMillis()
        
        ApiClient.getProductDetails(product.id) { result ->
            val duration = System.currentTimeMillis() - startTime
            
            runOnUiThread {
                when (result) {
                    is Result.Success -> {
                        // 成功時のリソース追跡
                        GlobalRum.stopResource(
                            "load_product_details",
                            200,
                            result.data.toString().length.toLong(),
                            RumResourceKind.NATIVE,
                            mapOf(
                                "response_time" to duration,
                                "cache_hit" to false,
                                "data_size" to result.data.toString().length
                            )
                        )
                        
                        updateUI(result.data)
                        
                    is Result.Error -> {
                        // エラー時のリソース追跡
                        GlobalRum.stopResourceWithError(
                            "load_product_details",
                            500,
                            "Failed to load product details",
                            RumErrorSource.NETWORK,
                            result.exception,
                            mapOf(
                                "response_time" to duration,
                                "error_type" to result.exception.javaClass.simpleName
                            )
                        )
                        
                        showError("商品詳細の読み込みに失敗しました")
                    }
                }
            }
        }
    }
}

// カスタムOkHttpインターセプターでのネットワーク監視
class DatadogInterceptor : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Response {
        val request = chain.request()
        val resourceKey = "${request.method()}_${request.url().encodedPath()}"
        
        // リクエスト開始の追跡
        GlobalRum.startResource(
            resourceKey,
            RumResourceMethod.valueOf(request.method()),
            request.url().toString(),
            mapOf(
                "request_size" to (request.body()?.contentLength() ?: 0),
                "content_type" to (request.header("Content-Type") ?: "unknown")
            )
        )
        
        val startTime = System.currentTimeMillis()
        
        try {
            val response = chain.proceed(request)
            val duration = System.currentTimeMillis() - startTime
            
            // 成功時のリソース追跡
            GlobalRum.stopResource(
                resourceKey,
                response.code(),
                response.body()?.contentLength() ?: 0,
                RumResourceKind.NATIVE,
                mapOf(
                    "response_time" to duration,
                    "response_size" to (response.body()?.contentLength() ?: 0),
                    "content_type" to (response.header("Content-Type") ?: "unknown")
                )
            )
            
            return response
            
        } catch (e: Exception) {
            val duration = System.currentTimeMillis() - startTime
            
            // エラー時のリソース追跡
            GlobalRum.stopResourceWithError(
                resourceKey,
                0,
                "Network request failed",
                RumErrorSource.NETWORK,
                e,
                mapOf(
                    "response_time" to duration,
                    "error_type" to e.javaClass.simpleName
                )
            )
            
            throw e
        }
    }
}

ユーザージャーニー分析

セッション分析とコンバージョン最適化

javascript
// ユーザージャーニーとコンバージョンファネル分析
class EcommerceAnalytics {
    constructor() {
        this.sessionStartTime = Date.now();
        this.funnelSteps = [];
        this.currentFunnelStep = null;
    }
    
    // ファネルステップの追跡
    trackFunnelStep(stepName, stepData = {}) {
        const stepInfo = {
            step_name: stepName,
            timestamp: Date.now(),
            session_duration: Date.now() - this.sessionStartTime,
            previous_step: this.currentFunnelStep,
            ...stepData
        };
        
        this.funnelSteps.push(stepInfo);
        this.currentFunnelStep = stepName;
        
        // Datadog RUM でのファネル追跡
        datadogRum.addAction('funnel_step', stepInfo);
        
        // ステップ別のユーザー行動分析
        this.analyzeFunnelBehavior(stepName, stepInfo);
    }
    
    // ページ滞在時間と離脱率の分析
    trackPageEngagement() {
        let pageStartTime = Date.now();
        let scrollDepth = 0;
        let maxScrollDepth = 0;
        let clickCount = 0;
        let timeOnPage = 0;
        
        // スクロール深度の追跡
        const trackScroll = () => {
            const scrollPercentage = Math.round(
                (window.scrollY / (document.body.scrollHeight - window.innerHeight)) * 100
            );
            
            scrollDepth = scrollPercentage;
            maxScrollDepth = Math.max(maxScrollDepth, scrollPercentage);
            
            // スクロールマイルストーンの追跡
            if (scrollPercentage >= 25 && !this.scrollMilestones?.reached25) {
                this.scrollMilestones = { ...this.scrollMilestones, reached25: true };
                datadogRum.addAction('scroll_milestone', { depth: 25 });
            }
            if (scrollPercentage >= 50 && !this.scrollMilestones?.reached50) {
                this.scrollMilestones = { ...this.scrollMilestones, reached50: true };
                datadogRum.addAction('scroll_milestone', { depth: 50 });
            }
            if (scrollPercentage >= 75 && !this.scrollMilestones?.reached75) {
                this.scrollMilestones = { ...this.scrollMilestones, reached75: true };
                datadogRum.addAction('scroll_milestone', { depth: 75 });
            }
            if (scrollPercentage >= 90 && !this.scrollMilestones?.reached90) {
                this.scrollMilestones = { ...this.scrollMilestones, reached90: true };
                datadogRum.addAction('scroll_milestone', { depth: 90 });
            }
        };
        
        // クリック数の追跡
        const trackClicks = (event) => {
            clickCount++;
            
            datadogRum.addAction('page_interaction', {
                interaction_type: 'click',
                element_type: event.target.tagName,
                element_class: event.target.className,
                element_id: event.target.id,
                click_count: clickCount,
                time_since_page_load: Date.now() - pageStartTime
            });
        };
        
        // イベントリスナーの設定
        window.addEventListener('scroll', trackScroll);
        document.addEventListener('click', trackClicks);
        
        // ページ離脱時の分析
        window.addEventListener('beforeunload', () => {
            timeOnPage = Date.now() - pageStartTime;
            
            datadogRum.addAction('page_exit', {
                time_on_page: timeOnPage,
                max_scroll_depth: maxScrollDepth,
                click_count: clickCount,
                engagement_score: this.calculateEngagementScore(timeOnPage, maxScrollDepth, clickCount),
                exit_type: 'navigation'
            });
        });
        
        // 非アクティブ状態での離脱検知
        let inactiveTimer;
        const resetInactiveTimer = () => {
            clearTimeout(inactiveTimer);
            inactiveTimer = setTimeout(() => {
                datadogRum.addAction('user_inactive', {
                    time_on_page: Date.now() - pageStartTime,
                    max_scroll_depth: maxScrollDepth,
                    click_count: clickCount,
                    inactive_duration: 30000
                });
            }, 30000);
        };
        
        ['mousedown', 'mousemove', 'keypress', 'scroll', 'touchstart'].forEach(event => {
            document.addEventListener(event, resetInactiveTimer, true);
        });
    }
    
    // エンゲージメントスコアの計算
    calculateEngagementScore(timeOnPage, scrollDepth, clickCount) {
        // 時間スコア (0-40点)
        const timeScore = Math.min(40, (timeOnPage / 1000) * 2);
        
        // スクロールスコア (0-30点)
        const scrollScore = Math.min(30, scrollDepth * 0.3);
        
        // インタラクションスコア (0-30点)
        const interactionScore = Math.min(30, clickCount * 5);
        
        return Math.round(timeScore + scrollScore + interactionScore);
    }
    
    // コンバージョンファネル分析
    analyzeFunnelBehavior(stepName, stepData) {
        const funnelDefinition = {
            'homepage': 1,
            'product_list': 2,
            'product_detail': 3,
            'add_to_cart': 4,
            'cart_view': 5,
            'checkout_start': 6,
            'checkout_payment': 7,
            'checkout_complete': 8
        };
        
        const currentStep = funnelDefinition[stepName];
        const previousSteps = this.funnelSteps.filter(step => 
            funnelDefinition[step.step_name] < currentStep
        );
        
        // ファネル進行の分析
        if (previousSteps.length > 0) {
            const lastStep = previousSteps[previousSteps.length - 1];
            const stepDuration = stepData.timestamp - lastStep.timestamp;
            
            datadogRum.addAction('funnel_progression', {
                from_step: lastStep.step_name,
                to_step: stepName,
                step_duration: stepDuration,
                funnel_position: currentStep,
                total_steps: Object.keys(funnelDefinition).length,
                progression_rate: this.calculateProgressionRate(lastStep.step_name, stepName)
            });
        }
        
        // 離脱ポイントの特定
        this.identifyDropOffPoints();
    }
    
    // 離脱率の計算
    calculateProgressionRate(fromStep, toStep) {
        // 過去のセッションデータとの比較(実際の実装では外部データソースから取得)
        const historicalRates = {
            'homepage_to_product_list': 45,
            'product_list_to_product_detail': 25,
            'product_detail_to_add_to_cart': 15,
            'add_to_cart_to_cart_view': 85,
            'cart_view_to_checkout_start': 60,
            'checkout_start_to_checkout_payment': 40,
            'checkout_payment_to_checkout_complete': 75
        };
        
        const rateKey = `${fromStep}_to_${toStep}`;
        return historicalRates[rateKey] || 0;
    }
    
    // 離脱ポイントの特定
    identifyDropOffPoints() {
        const criticalDropOffThreshold = 0.3; // 30%以下の進行率
        
        for (let i = 0; i < this.funnelSteps.length - 1; i++) {
            const currentStep = this.funnelSteps[i];
            const nextStep = this.funnelSteps[i + 1];
            
            if (nextStep) {
                const progressionRate = this.calculateProgressionRate(
                    currentStep.step_name, 
                    nextStep.step_name
                ) / 100;
                
                if (progressionRate < criticalDropOffThreshold) {
                    datadogRum.addAction('critical_drop_off_point', {
                        drop_off_step: currentStep.step_name,
                        next_step: nextStep.step_name,
                        progression_rate: progressionRate,
                        session_context: this.getSessionContext()
                    });
                }
            }
        }
    }
    
    // セッションコンテキストの取得
    getSessionContext() {
        return {
            session_duration: Date.now() - this.sessionStartTime,
            pages_visited: this.funnelSteps.length,
            device_type: this.getDeviceType(),
            traffic_source: this.getTrafficSource(),
            user_segment: this.getUserSegment(),
            ab_test_variants: this.getActiveABTests()
        };
    }
    
    // A/Bテスト結果の追跡
    trackABTestResults(testName, variant, conversionEvent = null) {
        const abTestData = {
            test_name: testName,
            variant: variant,
            session_id: this.getSessionId(),
            user_id: this.getUserId(),
            assignment_time: Date.now()
        };
        
        if (conversionEvent) {
            abTestData.conversion_event = conversionEvent;
            abTestData.time_to_conversion = Date.now() - this.sessionStartTime;
        }
        
        datadogRum.addAction('ab_test_tracking', abTestData);
        
        // A/Bテストのグローバルコンテキストに追加
        datadogRum.setGlobalContextProperty(`ab_test_${testName}`, variant);
    }
}

// 実際の使用例
const analytics = new EcommerceAnalytics();

// ページロード時
analytics.trackFunnelStep('homepage');
analytics.trackPageEngagement();

// 商品一覧ページ
analytics.trackFunnelStep('product_list', {
    category: 'electronics',
    filter_applied: true,
    sort_order: 'price_asc'
});

// A/Bテストの設定
analytics.trackABTestResults('checkout_button_color', 'blue');

// コンバージョン時
analytics.trackABTestResults('checkout_button_color', 'blue', 'purchase_complete');

この第4部では、Application Performance Monitoring (APM)からReal User Monitoring (RUM)まで、アプリケーション監視の全領域を包括的に解説しました。分散トレーシング、言語別設定、モバイル監視、ユーザージャーニー分析を通じて、ユーザー体験の最適化アプリケーションパフォーマンスの向上を実現できます。

次の第5部では、ログ管理編として、Datadogのログ収集・分析・活用手法について詳しく学習していきます。