Spark – consumer from Kafka

This tutorial illustrate how to consume messages from kafka using spark shell. Lets start installing kafka:

At this point kafka is up and runnig. Create a new sbt project, edit *.sbt and add the following lines


Now copy the following jars

Start spark shell

On another shell, create kafka.scala […]

Spark – Configure and use log4j

Hello, a little tutorial to configure and use log4j with Scala and Spark

Then create a file called “” and put this code inside

This configuration will log to /var/log/spark.log and on console. I partially review this post: Bye 🙂 Related posts: Spark – consumer from Kafka Python – daemons Pacemaker – […]