New Relic入門 第3.3部 - データ統合とクエリ活用

📖 ナビゲーション

第3章: New Relicの機能 ← メイン
前セクション: 3.2 各製品の詳細機能と活用方法
次章: 第4章 New Relic Infrastructure


🎯 このセクションで学べること

  • [ ] NRQL高度活用:複雑な分析・統計・時系列クエリの実践
  • [ ] カスタムダッシュボード:ビジネス価値を可視化する効果的なダッシュボード設計
  • [ ] Applied Intelligence実践:AI・機械学習機能によるインテリジェントな監視・自動化
  • [ ] ビジネス統合:技術メトリクスとビジネス価値の連携・ROI測定
  • [ ] 外部システム連携:API活用・データエクスポート・DevOpsツール統合
  • [ ] 自動化・最適化:アラート・ワークフロー・運用プロセスの効率化

🔍 NRQL高度クエリテクニック

複雑な統計分析・時系列処理

高度な統計関数活用:

sql
-- パーセンタイル・ヒストグラム分析
SELECT histogram(duration, width: 50, buckets: 20) as 'Response Time Distribution',
       percentile(duration, 50, 75, 95, 99) as 'Performance Percentiles',
       stddev(duration) as 'Std Deviation',
       rate(count(*), 1 minute) as 'Requests per Minute'
FROM Transaction
WHERE appName = 'ecommerce-app'
  AND name LIKE 'WebTransaction/Controller/%'
SINCE 4 hours ago
FACET name
TIMESERIES 5 minutes

-- 異常検知・統計的分析
SELECT count(*) as requests,
       anomaly_detection(count(*), 2) as anomaly_score,
       derivative(count(*), 1 minute) as request_velocity,
       integral(count(*)) as cumulative_requests
FROM Transaction
WHERE appName = 'critical-service'
SINCE 24 hours ago
TIMESERIES 5 minutes

-- 複雑な相関分析
SELECT correlation(apm_response_time, infra_cpu_percent) as cpu_correlation,
       correlation(apm_throughput, infra_memory_percent) as memory_correlation,
       correlation(error_rate, infra_disk_io) as disk_correlation
FROM (
  SELECT average(duration) as apm_response_time,
         rate(count(*), 1 minute) as apm_throughput,
         percentage(count(*), WHERE error = true) as error_rate
  FROM Transaction
  WHERE appName = 'web-app'
  TIMESERIES 1 minute
) JOIN (
  SELECT average(cpuPercent) as infra_cpu_percent,
         average(memoryUsedPercent) as infra_memory_percent,
         average(diskIORate) as infra_disk_io
  FROM SystemSample
  WHERE entityName = 'web-server-01'
  TIMESERIES 1 minute
) ON timestamp
SINCE 6 hours ago

ビジネスメトリクス・カスタム分析

sql
-- E-commerce ビジネス分析例
-- 売上・コンバージョン・顧客分析
SELECT sum(custom.orderValue) as 'Total Revenue',
       count(*) as 'Total Orders',
       uniqueCount(custom.customerId) as 'Unique Customers',
       average(custom.orderValue) as 'Average Order Value',
       
       -- コンバージョン率計算
       filter(count(*), WHERE custom.orderValue > 0) / 
       filter(count(*), WHERE name = 'WebTransaction/Controller/product-view') * 100 as 'Conversion Rate',
       
       -- 顧客セグメント別分析
       filter(sum(custom.orderValue), WHERE custom.customerTier = 'premium') as 'Premium Revenue',
       filter(count(*), WHERE custom.isNewCustomer = true) as 'New Customer Orders'
       
FROM Transaction
WHERE appName = 'ecommerce-app'
  AND (name = 'WebTransaction/Controller/checkout' OR 
       name = 'WebTransaction/Controller/product-view')
SINCE 1 day ago
FACET custom.region, custom.paymentMethod
ORDER BY sum(custom.orderValue) DESC

-- SaaS プラットフォーム分析
-- ユーザーエンゲージメント・ARR・チャーン分析
SELECT uniqueCount(custom.userId) as 'Monthly Active Users',
       sum(custom.sessionDuration) / uniqueCount(custom.userId) as 'Avg Session Duration',
       
       -- 機能利用率分析
       percentage(uniqueCount(custom.userId), WHERE custom.featureUsed = 'advanced-analytics') as 'Advanced Analytics Usage',
       percentage(uniqueCount(custom.userId), WHERE custom.featureUsed = 'api-access') as 'API Usage Rate',
       
       -- 収益・契約分析
       sum(custom.mrr) as 'Monthly Recurring Revenue',
       filter(count(*), WHERE custom.eventType = 'subscription-upgrade') as 'Upgrades',
       filter(count(*), WHERE custom.eventType = 'subscription-cancel') as 'Cancellations'
       
FROM PageView, Transaction
WHERE appName = 'saas-platform'
SINCE 30 days ago
FACET custom.subscriptionTier, custom.region
TIMESERIES 1 day

高度な時系列・フィルタリング

yaml
Advanced_NRQL_Patterns:
  
  Time_Series_Analysis:
    Sliding_Windows:
      # 移動平均・トレンド分析
      Query: |
        SELECT sliding(average(duration), 10) as 'Moving Average',
               derivative(average(duration), 1 hour) as 'Hourly Change Rate',
               forecast(average(duration), 4 hours) as 'Predicted Performance'
        FROM Transaction
        TIMESERIES 10 minutes
        SINCE 2 days ago
    
    Seasonal_Pattern:
      # 曜日・時間帯別パターン分析
      Query: |
        SELECT average(duration) as avg_response_time
        FROM Transaction
        WHERE appName = 'web-app'
        FACET weekdayOf(timestamp), hourOf(timestamp)
        SINCE 4 weeks ago
        ORDER BY avg_response_time DESC

  Complex_Filtering:
    Multi_Condition_Logic:
      Query: |
        SELECT count(*) as events
        FROM Transaction
        WHERE (
          (custom.userTier = 'premium' AND duration > 2.0) OR
          (custom.userTier = 'standard' AND duration > 5.0) OR
          (error = true AND httpResponseCode IN (500, 502, 503))
        )
        AND timestamp >= 1 hour ago
        FACET custom.userTier, error
    
    Regex_Pattern_Matching:
      Query: |
        SELECT count(*) as matching_requests
        FROM Transaction
        WHERE name RLIKE 'WebTransaction/Controller/(checkout|payment|billing).*'
          AND custom.userId RLIKE '^premium_user_[0-9]{6}$'
        FACET capture(name, r'Controller/(\w+)')
        SINCE 1 day ago

  Subqueries_Joins:
    Complex_Data_Correlation:
      Query: |
        SELECT *
        FROM (
          SELECT average(duration) as avg_app_time,
                 rate(count(*), 1 minute) as app_throughput
          FROM Transaction
          WHERE appName = 'web-app'
          TIMESERIES 5 minutes
        ) app_data
        JOIN (
          SELECT average(cpuPercent) as avg_cpu,
                 average(memoryUsedPercent) as avg_memory
          FROM SystemSample
          WHERE entityName LIKE 'web-server-%'
          TIMESERIES 5 minutes
        ) infra_data
        ON app_data.timestamp = infra_data.timestamp
        WHERE app_data.avg_app_time > 1.0
        SINCE 2 hours ago

📊 カスタムダッシュボード設計・構築

エグゼクティブ・ビジネスダッシュボード

yaml
Executive_Dashboard_Design:
  
  Business_Impact_Metrics:
    Revenue_Performance:
      Widget_Type: "Billboards + Line Charts"
      Metrics:
        - Total Revenue (daily/monthly)
        - Revenue Growth Rate (YoY/MoM)
        - Average Revenue Per User (ARPU)
        - Revenue by Channel/Region
      
      NRQL_Examples: |
        # 日次売上
        SELECT sum(custom.orderValue) as 'Daily Revenue'
        FROM Transaction
        WHERE name = 'WebTransaction/Controller/checkout'
        SINCE 1 day ago COMPARE WITH 1 day ago
        
        # 売上成長率
        SELECT sum(custom.orderValue) as revenue
        FROM Transaction
        WHERE name = 'WebTransaction/Controller/checkout'
        TIMESERIES 1 day
        SINCE 30 days ago
    
    Customer_Experience:
      Widget_Type: "Gauges + Heat Maps"
      Metrics:
        - Customer Satisfaction (Apdex/NPS相当)
        - Service Availability (SLA履行率)
        - Performance Score (Core Web Vitals)
        - Error Impact on Revenue
      
      NRQL_Examples: |
        # カスタマーサティスファクション
        SELECT apdex(duration, 0.5) as 'Customer Satisfaction'
        FROM Transaction
        WHERE appName = 'ecommerce-app'
        FACET custom.customerTier
        SINCE 1 day ago
        
        # エラーの売上影響
        SELECT sum(custom.lostRevenue) as 'Revenue Lost to Errors'
        FROM Transaction
        WHERE error = true
        SINCE 1 day ago
        FACET httpResponseCode

  Operations_Efficiency:
    System_Health:
      Widget_Type: "Status Indicators + Trend Lines"
      Key_Metrics:
        - System Uptime/Availability
        - Mean Time to Recovery (MTTR)
        - Incident Count/Severity
        - Deployment Success Rate
      
    Cost_Optimization:
      Widget_Type: "Cost Breakdown + Efficiency Ratios"
      Key_Metrics:
        - Infrastructure Cost per Transaction
        - Resource Utilization Efficiency
        - Cost Savings from Optimization
        - ROI from Monitoring Investment

Dashboard_Layout_Example:
  Row_1_Executive_Summary:
    - Revenue (Today vs Yesterday)
    - Active Users (Real-time)
    - System Health Status
    - Critical Alerts Count
  
  Row_2_Business_Performance:
    - Revenue Trend (30 days)
    - Conversion Funnel
    - Customer Satisfaction
    - Top Revenue Sources
  
  Row_3_Operational_Metrics:
    - Application Performance
    - Infrastructure Health
    - Error Rates & Impact
    - Deployment Activity

開発・運用チーム向けダッシュボード

yaml
Engineering_Dashboard_Design:
  
  Application_Performance:
    Real_Time_Monitoring:
      Widget_Configuration: |
        # レスポンス時間分布
        {
          "visualization": "histogram",
          "title": "Response Time Distribution",
          "nrql": "SELECT histogram(duration, 100, 20) FROM Transaction WHERE appName = 'web-app' SINCE 1 hour ago"
        }
        
        # スループット・エラー率
        {
          "visualization": "line_chart", 
          "title": "Throughput & Error Rate",
          "nrql": "SELECT rate(count(*), 1 minute) as 'Throughput', percentage(count(*), WHERE error = true) as 'Error Rate' FROM Transaction TIMESERIES SINCE 4 hours ago"
        }
    
    Code_Performance:
      Database_Analysis: |
        # データベース性能分析
        SELECT average(databaseDuration) as 'Avg DB Time',
               count(*) as 'Query Count', 
               percentage(count(*), WHERE databaseDuration > 0.1) as 'Slow Queries %'
        FROM Transaction
        FACET databaseName, operation
        SINCE 1 hour ago
        ORDER BY average(databaseDuration) DESC
      
      External_Services: |
        # 外部サービス依存性
        SELECT average(externalDuration) as 'Avg Response Time',
               percentage(count(*), WHERE externalCallCount > 0) as 'Requests with External Calls',
               sum(externalCallCount) as 'Total External Calls'
        FROM Transaction
        FACET externalService
        SINCE 1 hour ago

  Infrastructure_Monitoring:
    Resource_Utilization:
      Multi_Host_View: |
        # 複数ホストリソース監視
        SELECT latest(cpuPercent) as 'CPU %',
               latest(memoryUsedPercent) as 'Memory %',
               latest(diskUsedPercent) as 'Disk %',
               latest(networkTrafficTx + networkTrafficRx) as 'Network I/O'
        FROM SystemSample
        WHERE entityName LIKE 'web-server-%'
        FACET entityName
        SINCE 10 minutes ago
      
      Capacity_Planning: |
        # 容量計画・予測
        SELECT average(cpuPercent) as cpu,
               forecast(average(cpuPercent), 7 days) as cpu_forecast,
               derivative(average(memoryUsedPercent), 1 day) as memory_trend
        FROM SystemSample
        WHERE entityName LIKE 'web-server-%'
        TIMESERIES 1 hour
        SINCE 14 days ago

  DevOps_Integration:
    Deployment_Tracking:
      Release_Impact: |
        # デプロイメント影響分析
        SELECT count(*) as requests,
               average(duration) as avg_response_time,
               percentage(count(*), WHERE error = true) as error_rate
        FROM Transaction
        WHERE appName = 'web-app'
        FACET deploymentId
        SINCE 1 day ago
        TIMESERIES 30 minutes
      
      Feature_Flag_Analysis: |
        # 機能フラグ・A/Bテスト分析
        SELECT count(*) as user_interactions,
               average(custom.conversionTime) as avg_conversion_time,
               percentage(count(*), WHERE custom.converted = true) as conversion_rate
        FROM PageAction
        WHERE actionName = 'feature_interaction'
        FACET custom.featureFlag, custom.variant
        SINCE 7 days ago

実装例(ダッシュボード作成)

javascript
// New Relic Dashboard API を使用した自動ダッシュボード作成
const newrelicDashboard = require('@newrelic/dashboard-api');

class CustomDashboardBuilder {
  constructor(apiKey, accountId) {
    this.api = new newrelicDashboard(apiKey);
    this.accountId = accountId;
  }
  
  async createBusinessDashboard(appName) {
    const dashboardConfig = {
      name: `Business Dashboard - ${appName}`,
      description: `Executive business metrics for ${appName}`,
      permissions: "public_read_only",
      pages: [
        {
          name: "Business Overview",
          description: "Key business metrics and KPIs",
          widgets: [
            {
              title: "Daily Revenue",
              visualization: { id: "viz.billboard" },
              layout: { column: 1, row: 1, width: 4, height: 3 },
              rawConfiguration: {
                nrqlQueries: [
                  {
                    accountId: this.accountId,
                    query: `
                      SELECT sum(custom.orderValue) as 'Today',
                             sum(custom.orderValue) as 'Yesterday'
                      FROM Transaction
                      WHERE appName = '${appName}'
                        AND name = 'WebTransaction/Controller/checkout'
                      SINCE 1 day ago COMPARE WITH 1 day ago
                    `
                  }
                ],
                thresholds: [
                  { alertSeverity: "NOT_ALERTING", value: 10000 },
                  { alertSeverity: "WARNING", value: 5000 },
                  { alertSeverity: "CRITICAL", value: 1000 }
                ]
              }
            },
            {
              title: "Conversion Funnel",
              visualization: { id: "viz.funnel" },
              layout: { column: 5, row: 1, width: 4, height: 3 },
              rawConfiguration: {
                nrqlQueries: [
                  {
                    accountId: this.accountId,
                    query: `
                      SELECT funnel(session,
                        WHERE name = 'WebTransaction/Controller/home' as 'Home Page',
                        WHERE name = 'WebTransaction/Controller/product' as 'Product View',
                        WHERE name = 'WebTransaction/Controller/cart' as 'Add to Cart', 
                        WHERE name = 'WebTransaction/Controller/checkout' as 'Checkout'
                      )
                      FROM Transaction
                      WHERE appName = '${appName}'
                      SINCE 1 day ago
                    `
                  }
                ]
              }
            },
            {
              title: "Revenue Trend",
              visualization: { id: "viz.line" },
              layout: { column: 1, row: 4, width: 8, height: 3 },
              rawConfiguration: {
                nrqlQueries: [
                  {
                    accountId: this.accountId,
                    query: `
                      SELECT sum(custom.orderValue) as 'Revenue'
                      FROM Transaction
                      WHERE appName = '${appName}'
                        AND name = 'WebTransaction/Controller/checkout'
                      TIMESERIES 1 day
                      SINCE 30 days ago
                    `
                  }
                ]
              }
            }
          ]
        }
      ]
    };
    
    try {
      const dashboard = await this.api.createDashboard(dashboardConfig);
      console.log(`Dashboard created successfully: ${dashboard.permalink}`);
      return dashboard;
    } catch (error) {
      console.error('Dashboard creation failed:', error);
      throw error;
    }
  }
  
  async createSREDashboard(services) {
    // SRE・運用向けダッシュボード
    const sreConfig = {
      name: "SRE Operations Dashboard",
      description: "Site Reliability Engineering metrics and SLI/SLO tracking",
      pages: [
        {
          name: "Service Level Indicators",
          widgets: services.map((service, index) => ({
            title: `${service.name} - SLI`,
            visualization: { id: "viz.table" },
            layout: { 
              column: (index % 2) * 6 + 1, 
              row: Math.floor(index / 2) * 4 + 1, 
              width: 6, 
              height: 3 
            },
            rawConfiguration: {
              nrqlQueries: [
                {
                  accountId: this.accountId,
                  query: `
                    SELECT percentage(count(*), WHERE duration < ${service.sli.responseTime}) as 'Response Time SLI',
                           percentage(count(*), WHERE error = false) as 'Availability SLI',
                           rate(count(*), 1 minute) as 'Throughput'
                    FROM Transaction
                    WHERE appName = '${service.name}'
                    SINCE 1 hour ago
                    TIMESERIES 5 minutes
                  `
                }
              ]
            }
          }))
        }
      ]
    };
    
    return await this.api.createDashboard(sreConfig);
  }
}

// 使用例
const dashboardBuilder = new CustomDashboardBuilder(
  process.env.NEW_RELIC_USER_API_KEY,
  process.env.NEW_RELIC_ACCOUNT_ID
);

// ビジネスダッシュボード作成
await dashboardBuilder.createBusinessDashboard('ecommerce-web-app');

// SREダッシュボード作成  
await dashboardBuilder.createSREDashboard([
  { name: 'web-app', sli: { responseTime: 0.5, availability: 99.9 }},
  { name: 'api-service', sli: { responseTime: 0.2, availability: 99.95 }},
  { name: 'payment-service', sli: { responseTime: 1.0, availability: 99.99 }}
]);

🤖 Applied Intelligence実践活用

インテリジェント・アラート設定

yaml
AI_Powered_Alerting:
  
  Dynamic_Baseline_Alerts:
    Anomaly_Detection_Setup: |
      # 売上異常検知設定
      {
        "name": "Revenue Anomaly Detection",
        "description": "Detect unusual revenue patterns using AI baseline",
        "nrql": "SELECT sum(custom.orderValue) FROM Transaction WHERE name = 'WebTransaction/Controller/checkout'",
        "anomaly_detection": {
          "sensitivity": "medium",
          "direction": "both", 
          "baseline_window": "14 days",
          "ignore_seasonality": false
        }
      }
    
    Predictive_Capacity_Alerts: |
      # 容量不足予測アラート
      {
        "name": "CPU Capacity Prediction",
        "description": "Predict CPU exhaustion before it happens", 
        "nrql": "SELECT forecast(average(cpuPercent), 2 hours) FROM SystemSample",
        "condition": {
          "critical_threshold": 85,
          "warning_threshold": 70,
          "evaluation_offset": 1
        }
      }

  Intelligent_Incident_Management:
    Correlation_Rules:
      Multi_Signal_Analysis: |
        # 複数シグナル相関分析
        {
          "correlation_logic": {
            "conditions": [
              {
                "signal": "apm_error_rate_spike",
                "threshold": "error_rate > 5%"
              },
              {
                "signal": "infrastructure_resource_exhaustion", 
                "threshold": "cpuPercent > 80% OR memoryUsedPercent > 85%"
              },
              {
                "signal": "external_service_degradation",
                "threshold": "externalDuration > baseline + 2*stddev"
              }
            ],
            "correlation_window": "5 minutes",
            "minimum_signals": 2
          },
          "incident_creation": {
            "priority": "high",
            "title": "Multi-component Performance Degradation",
            "description": "Correlated performance issues detected across application and infrastructure"
          }
        }
    
    Automated_Root_Cause_Analysis:
      Analysis_Workflow: |
        # AI根本原因分析ワークフロー
        {
          "trigger": "incident_created",
          "analysis_steps": [
            {
              "step": "collect_contextual_data",
              "data_sources": ["apm", "infrastructure", "logs", "deployments"],
              "time_window": "incident_start - 30 minutes to incident_start + 10 minutes"
            },
            {
              "step": "pattern_matching", 
              "compare_against": "historical_incidents",
              "similarity_threshold": 0.7
            },
            {
              "step": "impact_analysis",
              "calculate": ["affected_users", "revenue_impact", "sla_impact"]
            },
            {
              "step": "remediation_suggestions",
              "source": "runbook_automation",
              "confidence_threshold": 0.8
            }
          ]
        }

  Workflow_Automation:
    Intelligent_Routing: |
      # インテリジェント通知ルーティング
      {
        "routing_rules": [
          {
            "condition": "incident.severity = 'critical' AND incident.service IN ['payment', 'checkout']",
            "action": "immediate_pager_duty_alert",
            "escalation_time": "5 minutes"
          },
          {
            "condition": "incident.type = 'performance_degradation' AND business_hours = true",
            "action": "slack_channel_alert",
            "channel": "#performance-team"
          },
          {
            "condition": "incident.predicted_impact.revenue > $10000",
            "action": "executive_notification",
            "recipients": ["[email protected]", "[email protected]"]
          }
        ]
      }

自動修復・オーケストレーション

javascript
// Applied Intelligence自動修復実装例
class IntelligentIncidentResponse {
  constructor(newrelicClient, orchestrationTools) {
    this.nr = newrelicClient;
    this.tools = orchestrationTools; // Ansible, Terraform, Kubernetes API等
  }
  
  async handleIncident(incident) {
    console.log(`Processing incident: ${incident.id}`);
    
    // 1. AI分析による根本原因推定
    const rootCauseAnalysis = await this.performRootCauseAnalysis(incident);
    
    // 2. 影響範囲・重要度評価
    const impactAssessment = await this.assessBusinessImpact(incident);
    
    // 3. 自動修復可能性判定
    const remediationPlan = await this.generateRemediationPlan(
      rootCauseAnalysis, 
      impactAssessment
    );
    
    // 4. 承認された場合の自動実行
    if (remediationPlan.confidence > 0.85 && impactAssessment.autoApprove) {
      return await this.executeRemediationPlan(remediationPlan);
    } else {
      return await this.requestHumanApproval(incident, remediationPlan);
    }
  }
  
  async performRootCauseAnalysis(incident) {
    // Applied Intelligence APIを使用した原因分析
    const contextualData = await this.nr.query(`
      SELECT * FROM Transaction, SystemSample, Log
      WHERE timestamp BETWEEN '${incident.startTime}' AND '${incident.endTime}'
        AND (
          entityName IN (${incident.affectedEntities.map(e => `'${e}'`).join(',')}) OR
          appName IN (${incident.affectedApplications.map(a => `'${a}'`).join(',')})
        )
      LIMIT 10000
    `);
    
    // 機械学習による相関分析・パターンマッチング
    const analysis = await this.nr.appliedIntelligence.analyzeIncident({
      incident: incident,
      contextualData: contextualData,
      comparisonPeriod: '7 days',
      includeDeployments: true,
      includeInfrastructureEvents: true
    });
    
    return {
      rootCause: analysis.rootCause,
      contributingFactors: analysis.contributingFactors,
      confidence: analysis.confidence,
      similarHistoricalIncidents: analysis.historicalPatterns
    };
  }
  
  async assessBusinessImpact(incident) {
    // ビジネス影響度の定量評価
    const revenueImpact = await this.nr.query(`
      SELECT sum(custom.lostRevenue) as totalLoss,
             count(*) as affectedTransactions,
             uniqueCount(custom.customerId) as affectedCustomers
      FROM Transaction
      WHERE timestamp BETWEEN '${incident.startTime}' AND NOW
        AND (error = true OR duration > 10)
        AND appName IN (${incident.affectedApplications.map(a => `'${a}'`).join(',')})
    `);
    
    const slaImpact = await this.calculateSLAImpact(incident);
    
    return {
      revenueImpact: revenueImpact[0].totalLoss,
      customerImpact: revenueImpact[0].affectedCustomers,
      slaViolation: slaImpact.violation,
      severity: this.calculateSeverity(revenueImpact, slaImpact),
      autoApprove: revenueImpact[0].totalLoss < 1000 && !slaImpact.violation // 自動承認条件
    };
  }
  
  async generateRemediationPlan(rootCause, impact) {
    // 根本原因に基づく修復プラン生成
    const remediationActions = [];
    
    switch (rootCause.rootCause.category) {
      case 'infrastructure_resource_exhaustion':
        if (rootCause.rootCause.resource === 'cpu') {
          remediationActions.push({
            action: 'scale_out_application',
            parameters: {
              replicas: Math.ceil(rootCause.rootCause.severity * 2),
              resource: 'cpu'
            },
            confidence: 0.9,
            estimatedRecoveryTime: '5 minutes'
          });
        }
        break;
        
      case 'database_performance_degradation':
        remediationActions.push({
          action: 'clear_database_cache',
          parameters: {
            databases: rootCause.contributingFactors.affectedDatabases
          },
          confidence: 0.7,
          estimatedRecoveryTime: '2 minutes'
        });
        break;
        
      case 'external_service_failure':
        remediationActions.push({
          action: 'enable_circuit_breaker',
          parameters: {
            services: rootCause.contributingFactors.failingServices,
            timeout: '30 seconds'
          },
          confidence: 0.95,
          estimatedRecoveryTime: '1 minute'
        });
        break;
    }
    
    return {
      actions: remediationActions,
      confidence: Math.min(...remediationActions.map(a => a.confidence)),
      estimatedImpact: impact,
      approvalRequired: remediationActions.some(a => a.confidence < 0.8)
    };
  }
  
  async executeRemediationPlan(plan) {
    const results = [];
    
    for (const action of plan.actions) {
      try {
        console.log(`Executing remediation action: ${action.action}`);
        
        let result;
        switch (action.action) {
          case 'scale_out_application':
            result = await this.tools.kubernetes.scaleDeployment(
              action.parameters.deployment,
              action.parameters.replicas
            );
            break;
            
          case 'clear_database_cache':
            result = await this.tools.database.clearCache(
              action.parameters.databases
            );
            break;
            
          case 'enable_circuit_breaker':
            result = await this.tools.circuitBreaker.enable(
              action.parameters.services,
              action.parameters.timeout
            );
            break;
        }
        
        results.push({
          action: action.action,
          status: 'success',
          result: result,
          timestamp: new Date().toISOString()
        });
        
      } catch (error) {
        results.push({
          action: action.action,
          status: 'failed',
          error: error.message,
          timestamp: new Date().toISOString()
        });
      }
    }
    
    // 修復結果の記録・追跡
    await this.recordRemediationResults(plan, results);
    
    return {
      planExecuted: plan,
      results: results,
      success: results.every(r => r.status === 'success')
    };
  }
}

// 使用例
const incidentHandler = new IntelligentIncidentResponse(
  newrelicClient,
  {
    kubernetes: k8sClient,
    database: dbClient,
    circuitBreaker: circuitBreakerService
  }
);

// ウェブフック・イベント受信での自動処理
app.post('/webhook/incident', async (req, res) => {
  const incident = req.body;
  
  try {
    const result = await incidentHandler.handleIncident(incident);
    res.json({ 
      success: true, 
      remediation: result 
    });
  } catch (error) {
    res.status(500).json({ 
      success: false, 
      error: error.message 
    });
  }
});

🔗 外部システム統合・API活用

DevOps・CI/CDパイプライン統合

yaml
DevOps_Integration_Patterns:
  
  Deployment_Tracking:
    GitHub_Actions_Integration: |
      # GitHub Actions ワークフロー例
      name: Deploy with New Relic Monitoring
      on:
        push:
          branches: [main]
      
      jobs:
        deploy:
          runs-on: ubuntu-latest
          steps:
            - name: Checkout
              uses: actions/checkout@v2
              
            - name: Deploy Application
              run: |
                # アプリケーションデプロイメント
                kubectl apply -f k8s-manifests/
                
            - name: Create New Relic Deployment Marker
              uses: newrelic/[email protected]
              with:
                apiKey: ${{ secrets.NEW_RELIC_API_KEY }}
                guid: ${{ secrets.NEW_RELIC_DEPLOYMENT_ENTITY_GUID }}
                version: ${{ github.sha }}
                changelog: ${{ github.event.head_commit.message }}
                description: "Production deployment from GitHub Actions"
                user: ${{ github.actor }}
                
            - name: Wait for Deployment Stability
              run: |
                # デプロイメント後の安定性確認
                sleep 300 # 5分待機
                
            - name: Performance Regression Check
              run: |
                # New Relic APIで性能回帰チェック
                python scripts/performance_check.py \
                  --api-key ${{ secrets.NEW_RELIC_API_KEY }} \
                  --app-id ${{ secrets.NEW_RELIC_APP_ID }} \
                  --deployment-id ${{ github.sha }} \
                  --baseline-days 7 \
                  --threshold-degradation 20
                
            - name: Rollback on Performance Issues
              if: failure()
              run: |
                # 性能劣化時の自動ロールバック
                kubectl rollout undo deployment/web-app
                
                # New Relic通知
                curl -X POST "https://api.newrelic.com/v2/applications/$APP_ID/deployments.json" \
                  -H "X-Api-Key: ${{ secrets.NEW_RELIC_API_KEY }}" \
                  -H "Content-Type: application/json" \
                  -d '{"deployment": {"revision": "rollback", "description": "Automated rollback due to performance regression"}}'

  Quality_Gates:
    Performance_SLO_Validation: |
      # Python製性能バリデーションスクリプト
      import requests
      import sys
      from datetime import datetime, timedelta
      
      def validate_deployment_performance(api_key, app_id, deployment_id):
          # デプロイメント後のメトリクス取得
          post_deploy_metrics = get_metrics_after_deployment(
              api_key, app_id, deployment_id
          )
          
          # ベースライン比較
          baseline_metrics = get_baseline_metrics(api_key, app_id, days=7)
          
          # SLOバリデーション
          slo_violations = []
          
          # レスポンス時間SLO (95th percentile < 500ms)
          if post_deploy_metrics['response_time_p95'] > 500:
              slo_violations.append({
                  'metric': 'response_time_p95',
                  'value': post_deploy_metrics['response_time_p95'],
                  'threshold': 500,
                  'severity': 'critical'
              })
          
          # エラー率SLO (< 1%)
          if post_deploy_metrics['error_rate'] > 1.0:
              slo_violations.append({
                  'metric': 'error_rate', 
                  'value': post_deploy_metrics['error_rate'],
                  'threshold': 1.0,
                  'severity': 'critical'
              })
          
          # 可用性SLO (> 99.9%)
          if post_deploy_metrics['availability'] < 99.9:
              slo_violations.append({
                  'metric': 'availability',
                  'value': post_deploy_metrics['availability'],
                  'threshold': 99.9,
                  'severity': 'critical'
              })
          
          return {
              'passed': len(slo_violations) == 0,
              'violations': slo_violations,
              'metrics': post_deploy_metrics
          }

  Infrastructure_as_Code:
    Terraform_Integration: |
      # Terraform New Relic Provider設定
      terraform {
        required_providers {
          newrelic = {
            source = "newrelic/newrelic"
            version = "~> 3.0"
          }
        }
      }
      
      provider "newrelic" {
        account_id = var.newrelic_account_id
        api_key    = var.newrelic_api_key
        region     = "US"
      }
      
      # アプリケーション監視設定
      resource "newrelic_application_settings" "web_app" {
        name               = "production-web-app"
        app_apdex_threshold = 0.5
        end_user_apdex_threshold = 7.0
        enable_real_user_monitoring = true
      }
      
      # インフラ監視設定
      resource "newrelic_infra_alert_condition" "high_cpu" {
        policy_id = newrelic_alert_policy.example.id
        name      = "High CPU Usage"
        type      = "infra_metric"
        
        event       = "SystemSample"
        select      = "cpuPercent"
        comparison  = "above"
        where       = "entityName LIKE 'web-server-%'"
        
        critical {
          duration      = 5
          value         = 90
          time_function = "all"
        }
        
        warning {
          duration      = 3
          value         = 80
          time_function = "all"
        }
      }
      
      # カスタムダッシュボード
      resource "newrelic_one_dashboard" "production_overview" {
        name = "Production Overview"
        
        page {
          name = "Application Performance"
          
          widget_line {
            title  = "Response Time"
            row    = 1
            column = 1
            width  = 6
            height = 3
            
            nrql_query {
              query = "SELECT average(duration) FROM Transaction WHERE appName = 'production-web-app' TIMESERIES"
            }
          }
          
          widget_billboard {
            title  = "Error Rate"
            row    = 1
            column = 7
            width  = 6
            height = 3
            
            nrql_query {
              query = "SELECT percentage(count(*), WHERE error = true) FROM Transaction WHERE appName = 'production-web-app'"
            }
            
            critical = 5.0
            warning  = 2.0
          }
        }
      }

データエクスポート・分析連携

yaml
Data_Export_Analytics:
  
  Business_Intelligence_Integration:
    Data_Pipeline_Setup: |
      # Python データエクスポート・ETLスクリプト
      import requests
      import pandas as pd
      from sqlalchemy import create_engine
      import schedule
      import time
      
      class NewRelicDataExporter:
          def __init__(self, api_key, account_id):
              self.api_key = api_key
              self.account_id = account_id
              self.base_url = "https://api.newrelic.com/graphql"
              
          def export_business_metrics(self, timeframe='SINCE 1 day ago'):
              # ビジネスメトリクスクエリ
              nrql_queries = {
                  'revenue': f"""
                      SELECT sum(custom.orderValue) as revenue,
                             count(*) as orders,
                             uniqueCount(custom.customerId) as customers
                      FROM Transaction
                      WHERE name = 'WebTransaction/Controller/checkout'
                      {timeframe}
                      FACET custom.region, custom.paymentMethod
                  """,
                  'performance': f"""
                      SELECT average(duration) as avg_response_time,
                             percentile(duration, 95) as p95_response_time,
                             percentage(count(*), WHERE error = true) as error_rate
                      FROM Transaction
                      {timeframe}
                      FACET appName
                  """,
                  'user_experience': f"""
                      SELECT average(largestContentfulPaint) as avg_lcp,
                             average(firstInputDelay) as avg_fid,
                             average(cumulativeLayoutShift) as avg_cls
                      FROM PageViewTiming
                      {timeframe}
                      FACET countryCode
                  """
              }
              
              exported_data = {}
              for metric_name, query in nrql_queries.items():
                  result = self.execute_nrql(query)
                  exported_data[metric_name] = self.parse_nrql_result(result)
              
              return exported_data
          
          def export_to_data_warehouse(self, data, connection_string):
              # データウェアハウス(BigQuery、Redshift等)への出力
              engine = create_engine(connection_string)
              
              for table_name, df in data.items():
                  df['export_timestamp'] = pd.Timestamp.now()
                  df.to_sql(
                      f'newrelic_{table_name}',
                      engine,
                      if_exists='append',
                      index=False
                  )
                  
              print(f"Exported {len(data)} tables to data warehouse")
      
      # 定期実行スケジュール
      def scheduled_export():
          exporter = NewRelicDataExporter(
              api_key=os.environ['NEW_RELIC_API_KEY'],
              account_id=os.environ['NEW_RELIC_ACCOUNT_ID']
          )
          
          daily_data = exporter.export_business_metrics('SINCE 1 day ago')
          exporter.export_to_data_warehouse(
              daily_data,
              os.environ['DATA_WAREHOUSE_CONNECTION']
          )
      
      # 毎日午前2時に実行
      schedule.every().day.at("02:00").do(scheduled_export)

  Real_Time_Analytics:
    Stream_Processing_Setup: |
      # Apache Kafka + New Relic Streaming設定例
      # New Relic → Kafka → 分析プラットフォーム
      
      version: '3.8'
      services:
        zookeeper:
          image: confluentinc/cp-zookeeper:latest
          environment:
            ZOOKEEPER_CLIENT_PORT: 2181
        
        kafka:
          image: confluentinc/cp-kafka:latest
          depends_on:
            - zookeeper
          environment:
            KAFKA_BROKER_ID: 1
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
        
        newrelic-kafka-connector:
          image: newrelic/kafka-connector:latest
          depends_on:
            - kafka
          environment:
            NEW_RELIC_API_KEY: ${NEW_RELIC_API_KEY}
            NEW_RELIC_ACCOUNT_ID: ${NEW_RELIC_ACCOUNT_ID}
            KAFKA_BOOTSTRAP_SERVERS: kafka:9092
            KAFKA_TOPIC_PREFIX: newrelic
            
            # ストリーミング設定
            EXPORT_INTERVAL: 60  # 60秒間隔
            EXPORT_METRICS: "Transaction,SystemSample,PageView"
            EXPORT_ATTRIBUTES: "duration,error,custom.*"
        
        stream-processor:
          image: apache/flink:latest
          depends_on:
            - kafka
          volumes:
            - ./flink-jobs:/opt/flink/jobs
          environment:
            KAFKA_SERVERS: kafka:9092
            
      # Flink Stream Processing Job
      import org.apache.flink.streaming.api.scala._
      import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
      import org.apache.flink.api.common.serialization.SimpleStringSchema
      
      object NewRelicStreamProcessor {
        def main(args: Array[String]): Unit = {
          val env = StreamExecutionEnvironment.getExecutionEnvironment
          
          // New Relicデータストリーム受信
          val nrStream = env
            .addSource(new FlinkKafkaConsumer[String](
              "newrelic-transactions",
              new SimpleStringSchema(),
              kafkaProps
            ))
            .map(parseNewRelicEvent)
          
          // リアルタイム集計・異常検知
          val alerts = nrStream
            .keyBy(_.appName)
            .timeWindow(Time.minutes(5))
            .aggregate(new PerformanceAggregator())
            .filter(_.anomalyScore > 0.8)
          
          // アラート送信
          alerts.addSink(new AlertingSink())
          
          env.execute("New Relic Real-time Analytics")
        }
      }

  Machine_Learning_Pipeline:
    Custom_ML_Models: |
      # 機械学習パイプライン構築例
      import pandas as pd
      import numpy as np
      from sklearn.ensemble import IsolationForest
      from sklearn.preprocessing import StandardScaler
      import joblib
      
      class NewRelicMLPipeline:
          def __init__(self):
              self.anomaly_detector = IsolationForest(contamination=0.1)
              self.scaler = StandardScaler()
              
          def prepare_training_data(self, nrql_data):
              # New Relicデータからの特徴量エンジニアリング
              features = []
              
              # 基本メトリクス
              features.extend([
                  'response_time_avg', 'response_time_p95',
                  'throughput', 'error_rate',
                  'cpu_percent', 'memory_percent'
              ])
              
              # 時系列特徴量
              df = pd.DataFrame(nrql_data)
              df['hour_of_day'] = pd.to_datetime(df['timestamp']).dt.hour
              df['day_of_week'] = pd.to_datetime(df['timestamp']).dt.dayofweek
              
              # ラグ特徴量
              for col in ['response_time_avg', 'throughput']:
                  df[f'{col}_lag1'] = df[col].shift(1)
                  df[f'{col}_lag2'] = df[col].shift(2)
              
              # 移動平均
              for col in ['response_time_avg', 'error_rate']:
                  df[f'{col}_ma_5'] = df[col].rolling(window=5).mean()
                  df[f'{col}_ma_15'] = df[col].rolling(window=15).mean()
              
              return df.fillna(0)
          
          def train_anomaly_detection(self, training_data):
              # 異常検知モデル学習
              X = self.scaler.fit_transform(training_data)
              self.anomaly_detector.fit(X)
              
              # モデル保存
              joblib.dump(self.anomaly_detector, 'anomaly_detector.pkl')
              joblib.dump(self.scaler, 'scaler.pkl')
              
          def predict_anomalies(self, new_data):
              # リアルタイム異常検知
              X_scaled = self.scaler.transform(new_data)
              anomaly_scores = self.anomaly_detector.decision_function(X_scaled)
              predictions = self.anomaly_detector.predict(X_scaled)
              
              return {
                  'anomaly_scores': anomaly_scores,
                  'is_anomaly': predictions == -1,
                  'confidence': np.abs(anomaly_scores)
              }

📚 セクション・第3章まとめ

🎯 データ統合・クエリ活用の価値

yaml
Chapter3_Value_Summary:
  
  Technical_Capabilities:
    ✅ NRQL_Mastery: 統一クエリ言語による横断的分析
    ✅ Dashboard_Excellence: ビジネス価値を可視化する効果的ダッシュボード
    ✅ AI_Integration: Applied Intelligence による自動化・最適化
    ✅ External_Integration: DevOps・分析ツールとのシームレス連携
  
  Business_Outcomes:
    ✅ Data_Driven_Decisions: データに基づく迅速な意思決定
    ✅ Operational_Efficiency: 運用プロセス自動化・効率化
    ✅ Proactive_Management: 予測的問題解決・容量計画
    ✅ ROI_Optimization: 投資対効果の継続的向上

Strategic_Advantages:
  1. Unified_Platform_Power:
     - 技術メトリクス・ビジネスメトリクス統合分析
     - リアルタイム意思決定支援
     - 組織横断データ活用文化構築
  
  2. AI_Automation_Benefits:  
     - 人的作業削減・ヒューマンエラー排除
     - 予測的問題対応・MTTR短縮
     - インテリジェントアラート・ノイズ削減
  
  3. Ecosystem_Integration:
     - DevOpsパイプライン統合・品質保証
     - ビジネスインテリジェンス・データウェアハウス連携
     - 機械学習・予測分析活用

🏆 New Relic統合プラットフォームの真価

第3章を通じて、New Relicが単なる監視ツールではなく、統合されたオブザーバビリティ・プラットフォームとして、以下の革新的価値を提供することを理解できました:

  1. 統一アーキテクチャ: NRDB・NRQL・Applied Intelligence による一貫したデータ体験
  2. 製品統合: APM・Infrastructure・Browser・Logs等の自然な連携
  3. データ活用: 高度な分析・AI活用・外部システム統合による価値最大化

次の学習: 第4章 New Relic Infrastructure で、インフラストラクチャ監視の詳細な実装・運用方法を学びましょう!


📖 関連記事:第3章メイン: New Relicの機能
第1章: New Relicとは、New Relicの優位性
NRQL実践ガイド
Applied Intelligence活用法