시리즈 소개

Series Introduction

1. Part 1: OpenTelemetry Instrumentation
2. Part 2: 마이크로서비스 분산 추적
3. Part 3: 구조화된 로깅과 Correlation ID (현재 글)
4. Part 4: Prometheus/Grafana로 메트릭과 알림
5. Part 5: Observability 데이터로 프로덕션 이슈 디버깅

1. Part 1: OpenTelemetry Instrumentation
2. Part 2: Microservices Distributed Tracing
3. Part 3: Structured Logging and Correlation ID (Current Post)
4. Part 4: Metrics and Alerting with Prometheus/Grafana
5. Part 5: Debugging Production Issues with Observability Data

로그의 문제점

기존 로그 방식의 한계:

The Problem with Traditional Logs

Limitations of conventional logging approaches:

2026-01-27 10:30:15 INFO OrderService - Processing order
2026-01-27 10:30:15 ERROR PaymentService - Payment failed
2026-01-27 10:30:15 INFO OrderService - Order completed

• 어떤 요청의 로그인지 알 수 없음
• 서비스 간 로그 연관성 부재
• 검색과 필터링이 어려움

• Cannot identify which request a log belongs to
• No correlation between logs across services
• Difficult to search and filter

구조화된 로깅 (Structured Logging)

JSON 로그 포맷

Structured Logging

JSON Log Format

{
  "timestamp": "2026-01-27T10:30:15.123Z",
  "level": "INFO",
  "service": "order-service",
  "traceId": "abc123",
  "spanId": "def456",
  "correlationId": "req-789",
  "message": "Processing order",
  "order.id": "order-123",
  "customer.id": "cust-456",
  "thread": "http-nio-8080-exec-1"
}

Logback JSON 설정

Logback JSON Configuration

<configuration>
    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <includeMdcKeyName>traceId</includeMdcKeyName>
            <includeMdcKeyName>spanId</includeMdcKeyName>
            <includeMdcKeyName>correlationId</includeMdcKeyName>
            <customFields>{"service":"order-service"}</customFields>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="JSON"/>
    </root>
</configuration>

MDC (Mapped Diagnostic Context)

MDC 필터 설정

MDC (Mapped Diagnostic Context)

MDC Filter Configuration

@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
class CorrelationIdFilter : OncePerRequestFilter() {

    override fun doFilterInternal(
        request: HttpServletRequest,
        response: HttpServletResponse,
        filterChain: FilterChain
    ) {
        try {
            val correlationId = request.getHeader("X-Correlation-ID")
                ?: UUID.randomUUID().toString()

            MDC.put("correlationId", correlationId)
            MDC.put("requestPath", request.requestURI)
            MDC.put("requestMethod", request.method)

            response.setHeader("X-Correlation-ID", correlationId)

            filterChain.doFilter(request, response)
        } finally {
            MDC.clear()
        }
    }
}

OpenTelemetry Trace ID 연동

OpenTelemetry Trace ID Integration

@Component
class TraceIdMdcFilter : OncePerRequestFilter() {

    override fun doFilterInternal(
        request: HttpServletRequest,
        response: HttpServletResponse,
        filterChain: FilterChain
    ) {
        try {
            val span = Span.current()
            if (span.spanContext.isValid) {
                MDC.put("traceId", span.spanContext.traceId)
                MDC.put("spanId", span.spanContext.spanId)
            }
            filterChain.doFilter(request, response)
        } finally {
            MDC.remove("traceId")
            MDC.remove("spanId")
        }
    }
}

로깅 모범 사례

의미 있는 로그 메시지

Logging Best Practices

Meaningful Log Messages

// 나쁜 예 / Bad example
logger.info("Processing")
logger.error("Error occurred")

// 좋은 예 / Good example
logger.info("Starting order processing", kv("orderId", orderId), kv("customerId", customerId))
logger.error("Payment processing failed", kv("orderId", orderId), kv("errorCode", e.errorCode), e)

구조화된 로깅 유틸리티

Structured Logging Utility

import net.logstash.logback.argument.StructuredArguments.kv

@Service
class OrderService(private val logger: Logger = LoggerFactory.getLogger(OrderService::class.java)) {

    fun processOrder(order: Order) {
        logger.info("Order processing started",
            kv("orderId", order.id),
            kv("customerId", order.customerId),
            kv("itemCount", order.items.size),
            kv("totalAmount", order.totalAmount)
        )

        try {
            // 처리 로직 / Processing logic
            logger.info("Order processing completed",
                kv("orderId", order.id),
                kv("processingTimeMs", processingTime)
            )
        } catch (e: Exception) {
            logger.error("Order processing failed",
                kv("orderId", order.id),
                kv("errorType", e.javaClass.simpleName),
                kv("errorMessage", e.message),
                e
            )
            throw e
        }
    }
}

Kafka 메시지에서 Correlation ID 전파

Propagating Correlation ID in Kafka Messages

@Component
class KafkaCorrelationInterceptor : ProducerInterceptor<String, String> {

    override fun onSend(record: ProducerRecord<String, String>): ProducerRecord<String, String> {
        MDC.get("correlationId")?.let { correlationId ->
            record.headers().add("X-Correlation-ID", correlationId.toByteArray())
        }
        MDC.get("traceId")?.let { traceId ->
            record.headers().add("X-Trace-ID", traceId.toByteArray())
        }
        return record
    }
}

@Component
class OrderEventConsumer {

    @KafkaListener(topics = ["order-events"])
    fun handleOrderEvent(
        @Payload payload: String,
        @Header("X-Correlation-ID") correlationId: String?,
        @Header("X-Trace-ID") traceId: String?
    ) {
        try {
            correlationId?.let { MDC.put("correlationId", it) }
            traceId?.let { MDC.put("traceId", it) }

            logger.info("Processing order event", kv("eventType", "OrderCreated"))
            // 이벤트 처리 / Event processing
        } finally {
            MDC.clear()
        }
    }
}

로그 레벨 가이드라인

레벨	사용 시점	예시
ERROR	즉각적인 조치 필요	결제 실패, DB 연결 실패
WARN	잠재적 문제	재시도 발생, 성능 저하
INFO	비즈니스 이벤트	주문 생성, 사용자 로그인
DEBUG	개발/디버깅용	메서드 진입/종료, 변수 값
TRACE	상세 디버깅	루프 내부 값

Log Level Guidelines

Level	When to Use	Examples
ERROR	Immediate action required	Payment failure, DB connection failure
WARN	Potential issues	Retry occurred, performance degradation
INFO	Business events	Order created, user login
DEBUG	Development/debugging	Method entry/exit, variable values
TRACE	Detailed debugging	Values inside loops

로그 집계 (Log Aggregation)

Loki 설정 (Grafana Stack)

Log Aggregation

Loki Configuration (Grafana Stack)

# docker-compose.yml
services:
  loki:
    image: grafana/loki:2.9.0
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yaml:/etc/loki/local-config.yaml

  promtail:
    image: grafana/promtail:2.9.0
    volumes:
      - /var/log:/var/log
      - ./promtail-config.yaml:/etc/promtail/config.yaml

LogQL 쿼리 예시

LogQL Query Examples

# 특정 traceId로 모든 서비스 로그 조회 / Query all service logs by specific traceId
{service=~".+"} |= "traceId=abc123"

# 에러 로그만 필터링 / Filter only error logs
{service="order-service"} | json | level="ERROR"

# 특정 orderId 관련 로그 / Logs related to specific orderId
{service=~".+"} | json | orderId="order-123"

정리

구조화된 로깅의 핵심:

항목	설명
JSON 포맷	파싱과 검색이 용이
Correlation ID	서비스 간 요청 추적
MDC	스레드별 컨텍스트 관리
Trace ID 연동	분산 추적과 로그 통합

다음 글에서는 Prometheus/Grafana를 활용한 메트릭과 알림을 다루겠습니다.

Summary

Key aspects of structured logging:

Item	Description
JSON Format	Easy to parse and search
Correlation ID	Track requests across services
MDC	Per-thread context management
Trace ID Integration	Unified distributed tracing and logs

In the next post, we will cover metrics and alerting with Prometheus/Grafana.