{"id":167,"date":"2018-07-17T13:29:34","date_gmt":"2018-07-17T04:29:34","guid":{"rendered":"https:\/\/dong1lkim.oboki.net\/?p=167"},"modified":"2019-09-01T22:21:42","modified_gmt":"2019-09-01T13:21:42","slug":"fluentd-%ec%86%8c%ea%b0%9c","status":"publish","type":"post","link":"https:\/\/oboki.net\/workspace\/data-engineering\/fluentd\/fluentd-%ec%86%8c%ea%b0%9c\/","title":{"rendered":"[Fluentd] Fluentd \uc18c\uac1c"},"content":{"rendered":"

Fluentd \uc18c\uac1c<\/h2>\n

with Fluentd<\/h3>\n

<\/p>\n

\ub0b4\ubd80 \uc5b8\uc5b4<\/h3>\n

Cruby \ub85c \uc791\uc131. \ub8e8\ube44\ub294 \uac1d\uccb4 \uc9c0\ud5a5 \uc778\ud130\ud504\ub9ac\ud130 \uc5b8\uc5b4
\nhttps:\/\/www.ruby-lang.org\/ko\/about\/<\/a><\/p>\n

Cruby vs. Jruby<\/h4>\n

The default Ruby, the one people think of as "just Ruby," is CRuby.<\/p>\n

JRuby is a Ruby interpreter written in Java. It’s written and maintained by a different team. It focuses hard on performance – especially for long-running servers like web servers. It’s far better for concurrency, especially multithreading. The garbage collection is more advanced, but JRuby uses far more memory and has much longer startup time. Don’t write your tiny command-line apps in it! It also takes more warmup to get to full speed. JRuby has great compatibility with Java libraries, but has more trouble with the C libraries CRuby is good with. It’s basically a whole different language project that happens to interpret exactly the same source code.
\nhttp:\/\/engineering.appfolio.com\/appfolio-engineering\/2017\/12\/28\/cruby-mri-jruby-rubyspec-rubinius-yarv-a-little-bit-of-ruby-naming<\/a><\/p>\n

Fluentd vs. Logstash<\/h4>\n

https:\/\/www.loomsystems.com\/blog\/single-post\/2017\/01\/30\/a-comparison-of-fluentd-vs-logstash-log-collector<\/a><\/p>\n

\ub370\uc774\ud130 \ucc98\ub9ac \ud615\ud0dc<\/h3>\n

Fluentd\ub294 \uac00\ub2a5\ud55c\ud55c \ubaa8\ub4e0 \ub370\uc774\ud130\ub97c JSON<\/b> \ud615\ud0dc\ub85c \uad6c\uc870\ud654 \ud558\uc5ec \uc218\uc9d1\/\ud544\ud130\/\ubc84\ud37c\/\uc801\uc7ac \uc791\uc5c5\uc744 \uc9c4\ud589\ud55c\ub2e4. \ub530\ub77c\uc11c \ub2e4\uc591\ud55c \ub370\uc774\ud130 \uc18c\uc2a4, \ud0c0\uac9f\uc5d0 \uc801\uc6a9 \uac00\ub2a5\ud558\ub2e4. But, \ub2e8\uc21c\ud788 \ud14d\uc2a4\ud2b8 \ud30c\uc77c\uc744 \uc77d\uc5b4 \uc804\ub2ec\ub9cc \ud558\uba74 \ub418\ub294 \uc0c1\ud669 (CSV \ud30c\uc77c\uc744 \ub2e8\uc21c\ud788 \uc77d\uc5b4, CSV \ud615\ud0dc\ub85c \ud558\ub461\uc5d0 \uc800\uc7a5) \uc5d0\uc11c\ub294 \uad73\uc774 JSON \ud3ec\ub9e4\ud305 \ubc0f CSV \uc7ac\uc804\ud658 \uc791\uc5c5\uc774 \ubd88\ud544\uc694\ud558\uac8c \uc218\ud589\ub418\uc5b4 \uc131\ub2a5\uc774 \ub5a8\uc5b4\uc9c0\ub294 \uac83\uc73c\ub85c \ubcf4\uc778\ub2e4.<\/p>\n

\uc7a5\uc810<\/h3>\n
    \n
  • Open community \uc5d0\uc11c \ub2e4\uc591\ud55c Plugin\uc744 \uc81c\uacf5\ud558\uae30 \ub54c\ubb38\uc5d0 \ub2e4\uc591\ud55c \ub370\uc774\ud130 \uc18c\uc2a4, \uc800\uc7a5\uc18c\ub97c \ucc44\ud0dd\ud560 \uc218 \uc788\ub2e4. But, \ucd5c\uc2e0 \ubc84\uc804\uc5d0\uc11c \uc81c\ub300\ub85c \ub3d9\uc791\ud558\uc9c0 \uc54a\ub294 plugin \ub4e4\uc774 \uc788\ub294 \ub4ef. \uc790\uccb4 \ubc84\uc804\uad00\ub9ac\uac00 \ud544\uc694\ud560 \uc218 \uc788\ub2e4. <\/li>\n
  • File-based\/Memory-based buffering \uc744 \uc81c\uacf5\ud568\uc73c\ub85c\uc368 inter-node data loss \ub97c \uc0ac\uc804\uc5d0 \ubc29\uc9c0\ud560 \uc218 \uc788\ub2e4. \ub610\ud55c in\/out forwarding plugin \uc744 \ud1b5\ud574\uc11c \ub2e4\uc591\ud55c \uad6c\uc870\/\ub2e8\uacc4\ub85c \uc218\uc9d1\uae30 \uad6c\uc131\uc744 \ud560 \uc218 \uc788\ub2e4. <\/li>\n
  • Fluentd agent \ubcc4\ub85c 1\uac1c\uc529\uc758 config \ud30c\uc77c \uc124\uc815\uc774 \ud544\uc694\ud55c\ub370, configuration \ubb38\ubc95\uc774 \uc9c1\uad00\uc801\uc774\uace0 \ud50c\ub7ec\uadf8\uc778\ubcc4\ub85c \ud30c\ub77c\ubbf8\ud130\ub4e4\uc744 \uc54c\uae30\ub9cc \ud558\uba74 \uc27d\uac8c \uc124\uc815\ud560 \uc218 \uc788\ub2e4. fluentd-ui \ub97c \ud1b5\ud574\uc11c configuration \uc218\uc815, fluentd \ub85c\uadf8 \ubaa8\ub2c8\ud130\ub9c1, fluentd \ub370\ubaac \uc2e4\ud589\/\uc885\ub8cc \uac00 \uac00\ub2a5\ud558\ub2e4. <\/li>\n
  • java \ub85c \uad6c\ub3d9\ub418\ub294 \uc218\uc9d1\uae30\ub4e4\uc5d0 \ube44\ud574 \uac00\ubcbc\uc6b4 \uac83 \uac19\ub2e4. \uc2e4\uc81c \uc11c\ube44\uc2a4\/\uc6b4\uc601 \uc2dc\uc2a4\ud15c\uc5d0\uc11c ruby, openssl \uc774 \uae30\ubcf8\uc801\uc73c\ub85c \uc124\uce58\ub3fc \uc788\ub294\uc9c0\ub294 \ubaa8\ub974\uaca0\uc9c0\ub9cc ruby, openssl \uc758\uc874\uc131\uc774 \ud544\uc694\ud558\ub2e4. <\/li>\n
  • https:\/\/www.fluentd.org\/ \uc0ac\uc774\ud2b8\uc5d0\uc11c core plugins \uc5d0 \ub300\ud55c guides & recipes \ub4e4\uc774 \uc798 \uc815\ub9ac\ub3fc \uc788\ub2e4. <\/li>\n<\/ul>\n

    \uc544\ud0a4\ud14d\ucc98, \ud575\uc2ec \uad6c\uc131 \uc694\uc18c<\/h3>\n

    Input<\/h4>\n

    HTTP, tail, TCP\/UDP \ub4f1<\/p>\n

    Parser<\/h4>\n

    Input plugin\uc5d0\uc11c \uc218\uc9d1\ub41c \ub370\uc774\ud130\ub97c \uace7\ubc14\ub85c Fluentd \ub0b4\ubd80\uc5d0\uc11c \uc0ac\uc6a9\ud560 \uc218 \uc5c6\ub294 \ud615\ud0dc\uc778 \uacbd\uc6b0 \uc120\ud0dd\uc801\uc73c\ub85c \uc0ac\uc6a9 \uac00\ub2a5.
    \n\uc815\uaddc\ud45c\ud604\uc2dd, apache, nginx, syslog \ub4f1 plugin \uc744 \uc774\uc6a9\ud558\uc5ec json \ub370\uc774\ud130\ub85c \ud30c\uc2f1.<\/p>\n

    Filter<\/h4>\n
      \n
    • row filter
      \ngrep plugin \uc744 \uc774\uc6a9\ud558\uc5ec \ud2b9\uc815 \ud544\ub4dc\uc5d0 \ud2b9\uc815 \ud328\ud134\uc774 \uc874\uc7ac\ud558\ub294 \uacbd\uc6b0\uc5d0\ub9cc \uc218\uc9d1 \uac00\ub2a5.<\/li>\n
    • column add
      \n\ub85c\uadf8 \ub370\uc774\ud130\uc5d0 \ud574\ub2f9 \ub85c\uadf8 \uc18c\uc2a4 \ub610\ub294 \ub85c\uadf8\uac00 \ubc1c\uc0dd\ud55c \ud638\uc2a4\ud2b8 \uc815\ubcf4 \ub4f1\uc744 \ucd94\uac00 \uac00\ub2a5.<\/li>\n
    • column del
      \n\ubd88\ud544\uc694\ud55c \ud544\ub4dc \uc0ad\uc81c \uac00\ub2a5<\/li>\n<\/ul>\n

      Formatter<\/h4>\n

      elasticsearch \ub97c \uc800\uc7a5\uc18c\ub85c \uc9c0\uc815\ud558\ub294 \uacbd\uc6b0 \ubb34\uc870\uac74 json \uc73c\ub85c \uc800\uc7a5\uc774 \ub420 \uac83\uc774\uae30 \ub54c\ubb38\uc5d0 \ud544\uc694\uc5c6\uc9c0\ub9cc, file, hdhfs, stdout \uacfc \uac19\uc740 \ud30c\uc77c \ud3ec\ub9f7 \ubcc0\uacbd\uc774 \uac00\ub2a5\ud55c \uc800\uc7a5\uc18c\ub97c \uc9c0\uc815\ud558\ub294 \uacbd\uc6b0 \ud14d\uc2a4\ud2b8 \ud30c\uc77c \ub0b4, \ub370\uc774\ud130 \ud615\ud0dc\ub97c \ubcc0\uacbd\ud560 \uc218 \uc788\ub2e4.<\/p>\n

        \n
      • csv<\/li>\n
      • json<\/li>\n<\/ul>\n

        Buffer<\/h4>\n
          \n
        • flush \uc8fc\uae30 \uc124\uc815<\/li>\n
        • file buffer \uc0ac\uc6a9 \uc2dc temp file \uc0dd\uc131 \uacbd\ub85c \uc124\uc815<\/li>\n<\/ul>\n

          Output<\/h4>\n
            \n
          • stdout\/file\/elasticsearch\/webhdfs\/s3\/mongo\/mysql… <\/li>\n
          • \ub2e4\uc591\ud55c \uc800\uc7a5\uc18c\uc5d0 \ub300\ud55c plugin \uc744 \uc9c0\uc6d0\ud55c\ub2e4. \uac01 \ud50c\ub7ec\uadf8\uc778 \ubcc4\ub85c \uc0ac\uc6a9\ub418\ub294 \ud30c\ub77c\ubbf8\ud130\ub4e4\uc774 \ub2e4\ub974\ub2e4.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

            Fluentd \uc18c\uac1c with Fluentd \ub0b4\ubd80 \uc5b8\uc5b4 Cruby \ub85c \uc791\uc131. \ub8e8\ube44\ub294 \uac1d\uccb4 \uc9c0\ud5a5 \uc778\ud130\ud504\ub9ac\ud130 \uc5b8\uc5b4 https:\/\/www.ruby-lang.org\/ko\/about\/ Cruby vs. Jruby The default Ruby, the one people think of as "just Ruby," is CRuby. JRuby is a Ruby interpreter written in Java. It’s written and maintained by a different team. It focuses hard on performance – especially for […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[142],"class_list":["post-167","post","type-post","status-publish","format-standard","hentry","category-fluentd","tag-fluented"],"_links":{"self":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts\/167","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/comments?post=167"}],"version-history":[{"count":4,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts\/167\/revisions"}],"predecessor-version":[{"id":1270,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts\/167\/revisions\/1270"}],"wp:attachment":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/media?parent=167"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/categories?post=167"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/tags?post=167"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}