How to install Flume on Linux
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.
For more information see Flume Installation
First you need to update your Yum repository:
cd /etc/yum.repos.d/ sudo wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo
In order to install Flume node:
sudo yum install flume-node -y
In order to install Flume master:
sudo yum install flume-master -y
Congratulations! You have installed Flume.
Before start using it, the first step you must do, is to tell each node about its master
sudo su -l flume --shell=/bin/bash cd /etc/flume/conf.empty/ cp flume-site.xml.template flume-site.xml
Edit flume-site.xml and set the value of flume.master.servers to be the name of your master server name (use the following command on the master server uname -n).
Start your Flume node:
sudo /etc/init.d/flume-node start
Start your Flume master:
sudo /etc/init.d/flume-master start
To validate that that all nodes configured to the properly master, go to the master URL address at port 35871 and validate you see all your nodes listed there (e.g. http://webserver-name:35871/flumemaster.jsp).
You can check it by entering a simple configuration setup in the master web GUI:
some_node : tail("/tmp/flumeInput.txt") | text("/tmp/flumeOutput.txt") ;
This will configure the node to tail an input file and write the lines into an output file.