×ðÁú¿­Ê±¹ÙÍøµÇ¼

ÔõÑùÔÚLinuxÇéÐÎÖÐʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÆÊÎö£¿

ÔõÑùÔÚlinuxÇéÐÎÖÐʹÓÃlogstash¾ÙÐÐÈÕÖ¾ÆÊÎö£¿

LogstashÊÇÒ»¸öÇ¿Ê¢µÄ¿ªÔ´¹¤¾ß£¬±»ÆÕ±éÓÃÓÚ´¦ÀíºÍÆÊÎöÖÖÖÖÀàÐ͵ÄÈÕÖ¾Êý¾Ý¡£Ëü¿ÉÒԺܼòÆӵؽ«ÈÕÖ¾Êý¾Ý´Ó²î±ðµÄȪԴÍøÂç¡¢¹ýÂË¡¢×ª»»ºÍ·¢Ë͵½ÖÖÖÖÄ¿µÄµØ¡£±¾ÎĽ«ÏÈÈÝÔõÑùÔÚLinuxÇéÐÎÖÐʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÆÊÎö£¬²¢ÌṩһЩ³£¼ûµÄ´úÂëʾÀý¡£

1. ×°ÖúÍÉèÖÃLogstash

ÔÚ×îÏÈ֮ǰ£¬ÇëÈ·±£ÒѾ­ÔÚLinuxÇéÐÎÖÐ×°ÖÃÁËJavaÔËÐÐʱÇéÐΡ£È»ºó£¬Æ¾Ö¤ÒÔÏ°취װÖúÍÉèÖÃLogstash¡£

ÏÂÔØLogstashµÄѹËõ°ü²¢½âѹµ½Ä¿µÄÎļþ¼Ð£º

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.10.2.tar.gz
tar -xzf logstash-7.10.2.tar.gz

µÇ¼ºó¸´ÖÆ

½øÈë½âѹºóµÄÎļþ¼Ð£º

cd logstash-7.10.2

µÇ¼ºó¸´ÖÆ

½¨ÉèÒ»¸öеÄÉèÖÃÎļþlogstash.conf²¢Ð´ÈëÒÔÏÂÄÚÈÝ£º

input {
  # ÉèÖÃÊäÈëÔ´£¬ÈçÎļþ¡¢ÍøÂçµÈ
  file {
    path => "/path/to/your/logfile.log"
    start_position => "beginning"
  }
}

filter {
  # ÉèÖùýÂËÆ÷£¬Æ¾Ö¤ÐèÇó¾ÙÐйýÂ˺Íת»»
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
}

output {
  # ÉèÖÃÊä³öÄ¿µÄµØ£¬ÈçElasticsearch¡¢ÎļþµÈ
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "mylogs-%{+YYYY.MM.dd}"
  }
}

µÇ¼ºó¸´ÖÆ

ÖµµÃ×¢ÖصÄÊÇ£¬ÉÏÊöÉèÖÃÎļþÖ»ÊÇÒ»¸ö¼òÆÓµÄʾÀý£¬Äã¿ÉÒÔƾ֤×Ô¼ºµÄÐèÇó¾ÙÐÐÏìÓ¦µÄÐ޸ĺÍÀ©Õ¹¡£

Æô¶¯Logstash£º

bin/logstash -f logstash.conf

µÇ¼ºó¸´ÖÆ

È·±£LogstashÀÖ³ÉÆô¶¯£¬²¢¼ì²éÊÇ·ñ½«ÈÕÖ¾Êý¾Ý·¢Ë͵½ÁËÖ¸¶¨µÄÄ¿µÄµØ¡£

2. Logstash³£ÓÃÉèÖÃʾÀý

ÏÂÃæÊÇһЩ³£ÓõÄLogstashÉèÖÃʾÀý£¬ÓÃÓÚʵÏÖ²î±ðµÄ¹¦Ð§ºÍ´¦ÀíÐèÇó¡£

a. ʹÓÃÕýÔò±í´ïʽÌáÈ¡Òªº¦ÐÅÏ¢

filter {
  grok {
    match => { "message" => "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes}" }
  }
}

µÇ¼ºó¸´ÖÆ

ÉÏÊöÉèÖÃʹÓÃÁËÕýÔò±í´ïʽÀ´ÌáÈ¡ÈÕÖ¾ÖеÄIPµØµã¡¢ÇëÇóÒªÁì¡¢ÇëÇ󷾶ºÍÊý¾Ý¾ÞϸµÈÒªº¦ÐÅÏ¢¡£

b. Ìí¼ÓÌØÁíÍâ×Ö¶Î

filter {
  mutate {
    add_field => { "environment" => "dev" }
  }
}

µÇ¼ºó¸´ÖÆ

ÉÏÊöÉèÖý«Ò»¸öÃûΪenvironmentµÄÌØÊâ×Ö¶ÎÌí¼Óµ½Ã¿ÌõÈÕÖ¾¼Í¼ÖУ¬²¢½«ÆäÖµÉèÖÃΪdev¡£

c. ɾ³ýÖ¸¶¨×Ö¶Î

filter {
  mutate {
    remove_field => [ "fieldname1", "fieldname2" ]
  }
}

µÇ¼ºó¸´ÖÆ

ÉÏÊöÉèÖý«ÃûΪfieldname1ºÍfieldname2µÄ×ֶδÓÿÌõÈÕÖ¾¼Í¼ÖÐɾ³ý¡£

d. ת»»Ê±¼äÃûÌÃ

filter {
  date {
    match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
  }
}

µÇ¼ºó¸´ÖÆ

ÉÏÊöÉèÖý«ÃûΪtimestampµÄ×Ö¶ÎÖеÄʱ¼ä×Ö·û´®×ª»»ÎªÖ¸¶¨µÄÈÕÆÚÃûÌá£

3. ½áÓï

LogstashÊÇÒ»¸ö¹¦Ð§Ç¿Ê¢µÄ¹¤¾ß£¬Äܹ»×ÊÖúÎÒÃÇÔÚLinuxÇéÐÎÖжÔÈÕÖ¾Êý¾Ý¾ÙÐÐÍøÂç¡¢¹ýÂË¡¢×ª»»ºÍ·¢Ë͵ȲÙ×÷¡£±¾ÎÄÏÈÈÝÁËLogstashµÄ×°ÖúÍÉèÖð취£¬²¢ÌṩÁËһЩ³£¼ûµÄÉèÖÃʾÀý¡£Ï£Íûͨ¹ý±¾ÎĵÄÏÈÈÝ£¬ÄãÄܶÔÔõÑùÔÚLinuxÇéÐÎÖÐʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÆÊÎöÓÐËùÏàʶºÍÕÆÎÕ¡£

ÒÔÉϾÍÊÇÔõÑùÔÚLinuxÇéÐÎÖÐʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÆÊÎö£¿µÄÏêϸÄÚÈÝ£¬¸ü¶àÇë¹Ø×¢±¾ÍøÄÚÆäËüÏà¹ØÎÄÕ£¡

ÃâÔð˵Ã÷£ºÒÔÉÏչʾÄÚÈÝȪԴÓÚÏàÖúýÌå¡¢ÆóÒµ»ú¹¹¡¢ÍøÓÑÌṩ»òÍøÂçÍøÂçÕûÀí£¬°æȨÕùÒéÓë±¾Õ¾Î޹أ¬ÎÄÕÂÉæ¼°¿´·¨Óë¿´·¨²»´ú±í×ðÁú¿­Ê±¹ÙÍøµÇ¼ÂËÓÍ»úÍø¹Ù·½Ì¬¶È£¬Çë¶ÁÕß½ö×ö²Î¿¼¡£±¾ÎĽӴýתÔØ£¬×ªÔØÇë˵Ã÷À´ÓÉ¡£ÈôÄúÒÔΪ±¾ÎÄÇÖÕ¼ÁËÄúµÄ°æȨÐÅÏ¢£¬»òÄú·¢Ã÷¸ÃÄÚÈÝÓÐÈκÎÉæ¼°ÓÐÎ¥¹«µÂ¡¢Ã°·¸Ö´·¨µÈÎ¥·¨ÐÅÏ¢£¬ÇëÄúÁ¬Ã¦ÁªÏµ×ðÁú¿­Ê±¹ÙÍøµÇ¼ʵʱÐÞÕý»òɾ³ý¡£

Ïà¹ØÐÂÎÅ

ÁªÏµ×ðÁú¿­Ê±¹ÙÍøµÇ¼

18523999891

¿É΢ÐÅÔÚÏß×Éѯ

ÊÂÇéʱ¼ä£ºÖÜÒ»ÖÁÖÜÎ壬9:30-18:30£¬½ÚãåÈÕÐÝÏ¢

QR code
ÍøÕ¾µØͼ