ÔõÑùʹÓÃDocker¾ÙÐÐÈÝÆ÷µÄÈÕÖ¾ÆÊÎöºÍÒì³£¼à²â
DockerÊÇÒ»ÖÖÊ¢ÐеÄÈÝÆ÷»¯ÊÖÒÕ£¬¿ÉÒÔ½«Ó¦ÓóÌÐò¼°ÆäÒÀÀµÏ´ò°üΪһ¸öÈÝÆ÷£¬×÷Ϊµ¥¸ö¿ÉÒÆÖ²µÄÓ¦ÓóÌÐòµ¥Î»ÔËÐС£ÕâÖÖÊÖÒÕʹµÃ¿ª·¢Ö°Ô±¿ÉÒÔÔÚ²î±ðµÄÇéÐÎÖÐÇáËɵذ²ÅźÍÖÎÀíÓ¦ÓóÌÐò¡£ÔÚÏÖʵӦÓÃÖУ¬¹ØÓÚDockerÈÝÆ÷µÄÈÕÖ¾ÆÊÎöºÍÒì³£¼à²â£¬ÊǺÜÊÇÐëÒªµÄ¡£±¾ÎĽ«ÏÈÈÝÔõÑùʹÓÃDocker¾ÙÐÐÈÝÆ÷µÄÈÕÖ¾ÆÊÎöºÍÒì³£¼à²â£¬°üÀ¨ÒÔϼ¸¸ö·½Ã棺
DockerÈÝÆ÷µÄÈÕÖ¾
ʹÓÃDocker logÏÂÁîÉó²éÈÕÖ¾
ʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÍøÂçºÍÆÊÎö
ʹÓÃElasticsearch¾ÙÐÐÊý¾ÝË÷ÒýºÍ´æ´¢
ʹÓÃKibana¾ÙÐÐÊý¾Ý¿ÉÊÓ»¯Õ¹Ê¾
Ê×ÏÈÎÒÃÇÐèÒªÏàʶ¹ØÓÚDockerÈÝÆ÷µÄÈÕÖ¾¡£
Ò»¡¢DockerÈÝÆ÷µÄÈÕÖ¾
DockerÈÝÆ÷µÄÈÕÖ¾£¬¼Í¼ÁËÈÝÆ÷ÖеIJÙ×÷ÐÅÏ¢£¬°üÀ¨£ºÓ¦ÓóÌÐòµÄÊä³öÐÅÏ¢¡¢¹ýʧÐÅÏ¢¡¢»á¼ûÈÕÖ¾¡¢ÏµÍ³ÈÕÖ¾µÈµÈ¡£ÕâЩÐÅÏ¢¹ØÓÚÓ¦ÓóÌÐòµÄÔËά¡¢×·×Ù¡¢Òì³£´¦ÀíµÈ¶¼ºÜÊÇÖ÷Òª£¬Òò´ËÎÒÃÇÐèÒª¶ÔDockerÈÝÆ÷µÄÈÕÖ¾¾ÙÐÐÍøÂçºÍÆÊÎö¡£
¶þ¡¢Ê¹ÓÃDocker logÏÂÁîÉó²éÈÕÖ¾
DockerÌṩÁËlogÏÂÁ¿ÉÓÃÓÚÉó²éÈÝÆ÷Êä³öµÄÈÕÖ¾ÐÅÏ¢¡£Ê¹ÓÃlogÏÂÁÎÒÃÇ¿ÉÒÔÇáËɵØÉó²éÕýÔÚÔËÐеÄÈÝÆ÷µÄʵʱÊä³öÐÅÏ¢£¬²¢½«ÕâЩÐÅÏ¢Êä³öµ½¿ØÖÆ̨»òÉúÑĵ½Ò»¸öÎļþÖС£ÒÔÏÂÊÇʹÓÃlogÏÂÁîÉó²éÈÝÆ÷ÈÕÖ¾µÄʾÀý£º
// Éó²éÈÝÆ÷IDΪxxxµÄÈÕÖ¾ docker logs xxx // Éó²éÈÝÆ÷IDΪxxxµÄÈÕÖ¾£¬Êä³öµ½¿ØÖÆ̨²¢ÊµÊ±¸üРdocker logs -f xxx // Éó²éÈÝÆ÷IDΪxxxµÄ×î½ü10ÌõÈÕÖ¾ docker logs --tail 10 xxx
µÇ¼ºó¸´ÖÆ
ͨ¹ýʹÓÃlogÏÂÁ¿ª·¢Ö°Ô±¿ÉÒÔÀû±ãµØÉó²éÈÝÆ÷µÄʵʱÊä³öÐÅÏ¢£¬²¢Äܹ»¿ìËÙÅжÏÎÊÌâËùÔÚ£¬µ«ÕâÖÖ·½·¨ÊÊÓÃÓÚµ¥Ì¨Ö÷»úÉϵÄÈÝÆ÷£¬µ±ÈÝÆ÷¹æÄ£Ôö´óʱ£¬ÊÖ¶¯Éó²éÈÕÖ¾±äµÃÄÑÌ⣬Òò´ËÐèҪʹÓÃlogÍøÂ繤¾ß¶ÔÈÕÖ¾¾ÙÐÐ×Ô¶¯ÍøÂçºÍÆÊÎö¡£
Èý¡¢Ê¹ÓÃLogstash¾ÙÐÐÈÕÖ¾ÍøÂçºÍÆÊÎö
LogstashÊÇÒ»¸öÓÃÓÚÍøÂç¡¢¹ýÂË¡¢×ª»»ºÍ·¢ËÍÈÕÖ¾µÄ¿ªÔ´¹¤¾ß£¬Í¨¹ýÊäÈë²å¼þÍøÂçÊý¾Ý£¬¾ÓɹýÂËÆ÷´¦ÀíºÍת»»Êý¾Ý£¬È»ºóÊä³ö²å¼þ½«´¦ÀíºóµÄÊý¾Ý·¢Ë͵½Ä¿µÄµØ£¬ÈçElasticsearch¡¢Kafka¡¢Amazon S3µÈ¡£ÔÚDockerÈÝÆ÷µÄÈÕÖ¾ÍøÂçÖУ¬ÎÒÃÇ¿ÉÒÔʹÓÃLogstash×÷ΪÍøÂçºÍÆÊÎöÈÕÖ¾µÄ¹¤¾ß¡£ÒÔÏÂÊÇʹÓÃLogstash¾ÙÐÐÈÕÖ¾ÍøÂçºÍÆÊÎöµÄʾÀý£º
1¡¢×°ÖÃLogstash
ÔÚ¹ÙÍøÉÏÏÂÔØLogstash£¬½âѹÎļþºó¼´¿ÉʹÓá£Æô¶¯LogstashµÄÏÂÁîÈçÏ£º
cd logstash-7.15.1/bin ./logstash -f logstash.conf
µÇ¼ºó¸´ÖÆ
2¡¢ÉèÖÃLogstash
ʹÓÃLogstash×÷ΪÈÝÆ÷µÄÈÕÖ¾ÍøÂ繤¾ß£¬ÎÒÃÇÐèÒªÔÚLogstashÖÐÉèÖÃÊäÈë²å¼þºÍÊä³ö²å¼þ¡£ÒÔÏÂÊÇÉèÖÃÎļþlogstash.confµÄʾÀý£º
input { docker { endpoint => "unix:///var/run/docker.sock" container_id => "ALL" } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } } output { elasticsearch { hosts => "localhost:9200" } stdout { codec => "json_lines" } }
µÇ¼ºó¸´ÖÆ
ÒÔÉÏÉèÖÃÎļþÒâζ×ÅÎÒÃÇÒª´ÓËùÓеÄdockerÈÝÆ÷ÖÐÍøÂçÈÕÖ¾ÐÅÏ¢£¬¾ÓÉgrok¹ýÂËÆ÷¾ÙÐÐÊý¾Ý¹ýÂËÆÊÎö£¬×îºó½«´¦ÀíºóµÄÊý¾ÝÊä³öµ½ElasticsearchÖС£
ËÄ¡¢Ê¹ÓÃElasticsearch¾ÙÐÐÊý¾ÝË÷ÒýºÍ´æ´¢
ElasticsearchÊÇÒ»¸öÂþÑÜʽµÄ¿ªÔ´ËÑË÷ÒýÇ棬¿ÉÓÃÓÚËÑË÷ÖÖÖÖÀàÐ͵ÄÎĵµ¡£ÔÚDockerÈÝÆ÷µÄÈÕÖ¾ÍøÂçÖУ¬ÎÒÃǽ«Ê¹ÓÃElasticsearch×÷ΪÊý¾ÝµÄË÷ÒýºÍ´æ´¢¡£ÒÔÏÂÊÇʹÓÃElasticsearch¾ÙÐÐÊý¾ÝË÷ÒýºÍ´æ´¢µÄʾÀý£º
1¡¢×°ÖÃElasticsearch
ÔÚ¹ÙÍøÉÏÏÂÔØElasticsearch£¬½âѹÎļþºó¼´¿ÉʹÓá£Æô¶¯ElasticsearchµÄÏÂÁîÈçÏ£º
cd elasticsearch-7.15.1/bin ./elasticsearch
µÇ¼ºó¸´ÖÆ
2¡¢ÉèÖÃElasticsearch
ͨ¹ýÐÞ¸Äelasticsearch.ymlÎļþÉèÖÃES¼¯ÈºµÄÃû³ÆºÍ½ÚµãÃû³Æ¡£ÒÔÏÂÊÇÒ»¸ö¼òÆÓµÄelasticsearch.ymlÉèÖÃÎļþʾÀý£º
cluster.name: docker-cluster node.name: es-node1 network.host: 0.0.0.0
µÇ¼ºó¸´ÖÆ
ÒÔÉÏÉèÖÃÒâζ×ÅÎÒÃǽ¨ÉèÁËÒ»¸öÃû³ÆΪdocker-clusterµÄ¼¯Èº£¬ÆäÖнڵãÃû³ÆΪes-node1£¬ESЧÀÍ°ó¶¨ÔÚËùÓпÉÓõÄÍøÂç½Ó¿ÚÉÏ¡£
3¡¢½¨ÉèË÷Òý
ÔÚElasticsearchÖУ¬ÎÒÃÇÐèÒªÏÈΪÊý¾Ý½¨ÉèÒ»¸öË÷Òý£¬²¢Ö¸¶¨Ãü¾ÝÖеÄ×ֶΡ£Ê¾Àý´úÂëÈçÏ£º
PUT /logstash-test { "mappings": { "properties": { "host": { "type": "keyword" }, "message": { "type": "text" }, "path": { "type": "text" }, "verb": { "type": "keyword" } } } }
µÇ¼ºó¸´ÖÆ
ÒÔÉÏ´úÂëÊÇÔÚElasticsearchÖн¨ÉèÒ»¸öÃûΪ”logstash-test”µÄË÷Òý£¬²¢½ç˵¸ÃË÷ÒýÖаüÀ¨µÄ×Ö¶ÎÒÔ¼°×ֶεÄÀàÐÍ¡£
Î塢ʹÓÃKibana¾ÙÐÐÊý¾Ý¿ÉÊÓ»¯Õ¹Ê¾
KibanaÊÇÒ»¸ö¿ªÔ´µÄÊý¾Ý¿ÉÊÓ»¯¹¤¾ß£¬¿ÉÒÔÓÃÀ´Õ¹Ê¾´ÓElasticsearchÖлñÈ¡µÄÊý¾Ý¡£ÔÚDockerÈÝÆ÷µÄÈÕÖ¾ÍøÂçÀú³ÌÖУ¬ÎÒÃǽ«Ê¹ÓÃKibana¾ÙÐÐÊý¾Ý¿ÉÊÓ»¯Õ¹Ê¾¡£ÒÔÏÂÊÇʹÓÃKibana¾ÙÐÐÊý¾Ý¿ÉÊÓ»¯Õ¹Ê¾µÄʾÀý£º
1¡¢×°ÖÃKibana
ÔÚ¹ÙÍøÉÏÏÂÔØKibana£¬½âѹÎļþºó¼´¿ÉʹÓá£Æô¶¯KibanaµÄÏÂÁîÈçÏ£º
cd kibana-7.15.1/bin ./kibana
µÇ¼ºó¸´ÖÆ
2¡¢Ë÷ÒýÄ£°åµÄÉèÖÃ
ÔÚKibanaÖУ¬ÎÒÃÇÐèҪΪË÷ÒýÄ£°å¾ÙÐÐÉèÖá£Ë÷ÒýÄ£°å°üÀ¨ÁËÊý¾ÝµÄ×ֶνç˵ºÍÅÌÎÊÆÊÎöµÄÐÅÏ¢¡£Ê¾Àý´úÂëÈçÏ£º
PUT _index_template/logstash-template { "index_patterns": ["logstash-*"], "template": { "mappings": { "properties": { "@timestamp": { "type": "date" }, "@version": { "type": "keyword" }, "message": { "type": "text" }, "path": { "type": "text" } } } } }
µÇ¼ºó¸´ÖÆ
ÒÔÉÏ´úÂëÒâζ׎¨ÉèÁËÒ»¸öÃûΪ”logstash-template”µÄË÷ÒýÄ£°å£¬²¢ÇÒ½«ÆäÓ¦ÓÃÓÚÃû×ÖÒÔ”logstash-*”×îÏȵÄË÷Òý¡£
3¡¢Êý¾Ý¿ÉÊÓ»¯
ÔÚKibanaµÄ²å¼þÃæ°åÖУ¬¿ÉÒÔÑ¡ÔñÉèÖúÍÖÎÀí¿ÉÊÓ»¯Ä£°å¡£ÎÒÃÇ¿ÉÒÔºÜÈÝÒ×µØͨ¹ýÃæ°å½¨ÉèÖÖÖÖÀàÐ͵ĿÉÊÓ»¯Í¼±í£¬ÈçLineͼ¡¢BarͼºÍPieͼµÈµÈ¡£
×ÛÉÏËùÊö£¬±¾ÎÄÏÈÈÝÁËÔõÑùʹÓÃDocker¾ÙÐÐÈÝÆ÷µÄÈÕÖ¾ÆÊÎöºÍÒì³£¼à²â£¬²¢¸ø³öÁËÏêϸµÄ´úÂëʾÀý¡£Docker×Ô¼ºÌṩÁËlogÏÂÁîÀ´Éó²éÈÝÆ÷µÄÈÕÖ¾£¬µ«ÊÖ¶¯Éó²éÈÕÖ¾ÔÚÈÝÆ÷¹æÄ£À©´óºó±äµÃÔ½·¢ÄÑÌ⡣ͨ¹ýʹÓÃLogstash¡¢ElasticsearchºÍKibanaÕâЩ¹¤¾ß£¬ÎÒÃÇ¿ÉÒÔ¶ÔÈÝÆ÷µÄÈÕÖ¾¾ÙÐÐ×Ô¶¯»¯µÄÍøÂçºÍÆÊÎö£¬²¢Õ¹Ê¾³öÈÝÆ÷µÄÔËÐÐ״̬£¬Õâ¹ØÓÚÓ¦ÓóÌÐòµÄÔËάºÍ¹ÊÕÏ´¦Àí¶¼ºÜÊÇÓÐ×ÊÖú¡£
ÒÔÉϾÍÊÇÔõÑùʹÓÃDocker¾ÙÐÐÈÝÆ÷µÄÈÕÖ¾ÆÊÎöºÍÒì³£¼à²âµÄÏêϸÄÚÈÝ£¬¸ü¶àÇë¹Ø×¢±¾ÍøÄÚÆäËüÏà¹ØÎÄÕ£¡