Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to get plugin running #10

Open
dannylesnik opened this issue Mar 21, 2019 · 0 comments
Open

Not able to get plugin running #10

dannylesnik opened this issue Mar 21, 2019 · 0 comments

Comments

@dannylesnik
Copy link

Hello.
I followed your documentation, but I was not able to get this plugin running.
I believe I have configuration issue, bu I don't know what exectly.

I have a set of containers in docker swarm which send all data to fluentd container and I would like to grab all logs which tag starts from "ccms_".

I installed pugin in container using gem and I can see in log that actually plugin presents in fluentd.

This is my fluentd.conf file:

<source>
  @type forward
  @log_level debug
  port 24224
  bind 0.0.0.0
</source>
<match ccms_**>
  @type flatten_hash
  add_tag_prefix flattened.
  separator _
  flatten_array false
</match>
<match flattened.ccms_**>
  type "aws-elasticsearch-service"
  @log_level debug
  type_name "access_log"
  logstash_format true
  include_tag_key true
  tag_key "@log_name"
  flush_interval 10
  buffer_chunk_limit 4M
  buffer_queue_limit 256
  <endpoint>
    url https://xxx-ccms-kibana-xxxx.us-east-1.es.amazonaws.com
    region us-east-1
    assume_role_arn arn:aws:iam::xxxxxxxx:role/TMS_Standard_EC2
  </endpoint>
</match>

This is my json log example.

{
  "_index": "logstash-2019.03.21",
  "_type": "access_log",
  "_id": "E6uxn2kBys5QzvwH_Ii7",
  "_version": 1,
  "_score": null,
  "_source": {
    "container_id": "5d97e6299ef4ee8a7f589ca47d49b306db17229e82c4c58124d2beb7604d8c94",
    "container_name": "/ccms_write-service.3.xscsw6kiui3yaasa9h7hdei5l",
    "source": "stdout",
    "log": "{\"@timestamp\":\"2019-03-21T10:00:35.001+00:00\",\"message\":\"event [27760845] was successfully inserted to msg queue\",\"method\":\"com.tr.tms.ccms.write.WriteService$$anon$2\",\"threadname\":\"WriteService-blocking-code-dispatcher-2456\",\"level\":\"INFO\",\"level_value\":20000,\"prefix\":\"APP\"}",
    "@timestamp": "2019-03-21T10:00:35.000000000+00:00",
    "@log_name": "ccms_qa_write_service"
  },
  "fields": {
    "@timestamp": [
      "2019-03-21T10:00:35.000Z"
    ]
  },
  "sort": [
    1553162435000
  ]
}

When I use this configuration this code has not been flatten. Can you please help?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant