Сбор лямбда-логов из cloudwatch с помощью fluentd
Может кто-нибудь, пожалуйста, помогите мне в сборе журналов из cloudwatch и отправьте их в asticsearch. Моя конфигурация
<source>
@type cloudwatch_logs
region us-east-1 # You must supply a region
aws_use_sts true
aws_sts_role_arn arn:aws:iam::xxx:role/fluentd
log_group_name /aws/lambda/event_processor
log_stream_name 2019/04/04
use_log_stream_name_prefix true
state_file /var/log/td-agent/group_stream.in.state
tag cloudwatch.logs
</source>
<filter cloudwatch.log>
type record_transformer
</filter>
<match cloudwatch.log>
@type stdout
</match>
Я получаю сообщение об ошибке в журналах
2019-04-10 10:06:54 +0000 [warn]: #0 thread exited by unexpected error plugin=Fluent::Plugin::CloudwatchLogsInput title=:in_cloudwatch_logs_runner error_class=Yajl::ParseError **error="lexical error: invalid char in json text.\n START RequestId: 4cf17868-77f0-\n (right here) ------^\n"**
2019-04-10 10:06:54 +0000 [error]: #0 unexpected error error_class=Yajl::ParseError error="lexical error: invalid char in json text.\n START RequestId: 4cf17868-77f0-\n (right here) ------^\n"
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/yajl-ruby-1.4.0/lib/yajl.rb:44:in `parse'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/yajl-ruby-1.4.0/lib/yajl.rb:44:in `parse'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/yajl-ruby-1.4.0/lib/yajl.rb:18:in `load'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:144:in `emit'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:119:in `block (2 levels) in run'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:118:in `each'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:118:in `block in run'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:115:in `each'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-cloudwatch-logs-0.7.3/lib/fluent/plugin/in_cloudwatch_logs.rb:115:in `run'
2019-04-10 10:06:54 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.3.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2019-04-10 10:06:54 +0000 [error]: #0 unexpected error error_class=Yajl::ParseError error="lexical error: invalid char in json text.\n START RequestId: 4cf17868-77f0-\n (right here) ------^\n"
2019-04-10 10:06:54 +0000 [error]: #0 suppressed same stacktrace
2019-04-10 10:06:54 +0000 [info]: Worker 0 finished unexpectedly with status 1
2019-04-10 10:06:55 +0000 [warn]: #0 'include' is deprecated. Use '@include' instead
2019-04-10 10:06:55 +0000 [info]: gem 'fluent-plugin-cloudwatch' version '2.1.1'
и вы можете помочь определить динамическое имя потока, если мое имя Stram, как
2019/04/04/[$LATEST]7ed82a142a934cea9209d6315d42b95f
2019/03/29/[$LATEST]1e5b72624b784ec3b1022147e3c736f6