Spark – consumer from Kafka

This tutorial illustrate how to consume messages from kafka using spark shell. Lets start installing kafka:

At this point kafka is up and runnig. Create a new sbt project, edit *.sbt and add the following lines

Then

Now copy the following jars

Start spark shell

On another shell, create kafka.scala […]

Spark – Configure and use log4j

Hello, a little tutorial to configure and use log4j with Scala and Spark

Then create a file called “log4j.properties” and put this code inside

This configuration will log to /var/log/spark.log and on console. I partially review this post: http://blog.quantifind.com/posts/logging-post/ Bye 🙂 Related posts: Spark – consumer from Kafka Python – daemons Pacemaker – […]