Blog-Post

A logstash plugin to store events via webhdfs

Jan 17, 2014

When we had the need to pump data from logstash into Hadoop-HDFS to be read directly by hive the problem was that there was no usable hdfs output for logstash meeting the criterias:

  • append support
  • snappy compression
  • no heavy dependencies

So we did our own implementation which you might like, find it on github