はじめに

最近、Fluent Bitというものを知りました。

fluentbit.io

これを使ってRailsのログをFluent Bitに送信することをやってみます。

参考

参考情報として、Fluentdが公開している以下のドキュメントをみます。

www.fluentd.org

Fluent BitはFluentd傘下で開発された軽量なログルータという位置付けで、アーキテクチャを踏まえて実装されているので参考になりそうです。ただ、設定ファイルの書き方は異なるので、適宜読み替えて実装します。

実装

gemとしてact-fluent-loggerを使います。

github.com

接続先情報をconfig/fluent-logger.ymlに記述します。

development:
  fluent_host:   'fluent'
  fluent_port:  24224
  tag:          'development'
  message_type: 'string'

test:
  fluent_host:   '127.0.0.1'
  fluent_port:  24224
  tag:          'test'
  message_type: 'string'

production:
  fluent_host:   '127.0.0.1'
  fluent_port:  24224
  tag:          'production'
  message_type: 'string'

また、config/application.rbにact-fluent-loggerの設定を行います。

module HogeHoge
  class Application < Rails::Application
    # 省略
    config.logger = ActFluentLoggerRails::Logger.new(
      flush_immediately: true
    )
  end
end

今回、Dockerを使って環境を用意するので、docker-compose.ymlは以下のように記述しました。

version: '3'

services:
  web:
    build: .
    image: rails_and_sidecar:0.0.2
    command: ["bin/dev"]
    expose: ["3000"]
    ports: ["3000:3000"]
    depends_on:
      - fluent
    working_dir: /app

  fluent:
    image: fluent/fluent-bit
    hostname: fluent
    volumes:
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
    ports:
      - "24224:24224"

Fluent Bitの設定ファイルであるfluent-bit.confは以下のように書きました。

[SERVICE]
  Log_Level  info

[INPUT]
  Name forward
  Listen 0.0.0.0
  Port   24224

[FILTER]
  Name stdout
  Match *

[OUTPUT]
  Name null
  Match *

実行

実装ができたら実行してみます。docker-compose upで起動していくらかWebアプリにアクセスしてみたらFluent Bitから標準出力の結果が出力されました。

fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Started GET "/posts/new" for 192.168.192.1 at 2023-03-12 06:50:12 +0000"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Cannot render console from 192.168.192.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  ActiveRecord::SchemaMigration Pluck (0.4ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Processing by PostsController#new as HTML"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendering layout layouts/application.html.erb"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendering posts/new.html.erb within layouts/application"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendered posts/_form.html.erb (Duration: 11.7ms | Allocations: 3235)"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendered posts/new.html.erb within layouts/application (Duration: 13.6ms | Allocations: 3772)"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendered layout layouts/application.html.erb (Duration: 122.3ms | Allocations: 25300)"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Completed 200 OK in 140ms (Views: 125.6ms | ActiveRecord: 2.1ms | Allocations: 32821)
fluent-1  | "], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Started GET "/posts" for 192.168.192.1 at 2023-03-12 06:50:12 +0000"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Cannot render console from 192.168.192.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Processing by PostsController#index as HTML"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendering layout layouts/application.html.erb"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendering posts/index.html.erb within layouts/application"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Post Load (1.9ms)  SELECT "posts".* FROM "posts""], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  ↳ app/views/posts/index.html.erb:8"], "severity"=>"DEBUG"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendered posts/index.html.erb within layouts/application (Duration: 6.5ms | Allocations: 1180)"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["  Rendered layout layouts/application.html.erb (Duration: 16.7ms | Allocations: 3583)"], "severity"=>"INFO"}]
fluent-1  | [0] development: [1678603812.000000000, {"messages"=>["Completed 200 OK in 19ms (Views: 15.6ms | ActiveRecord: 1.9ms | Allocations: 3997)
fluent-1  | 
fluent-1  | "], "severity"=>"INFO"}]

今回は、Railsの標準的なログ出力形式をFluent Bitに送ってみましたが、個人的にはJSON形式を送りたいです。Rails 7でログをJSON化するには、rails_semantic_loggerというgemがあります。

github.com

ちょっと試してみましたが、どうもうまくFluent Bitにデータを送ることができませんでした。