If you are using Spark Streaming Kafka Direct API and you are storing offsets in Zookeeper, you might want to reuse offsets from Zookeeper and not always rely on checkpoint. KafkaUtils.createDirectStream() has a fromOffsets parameter which you can provide
Read More

If you are using Spark Streaming's new Kafka Direct API you might be missing some things, like monitoring tools that only work with offsets in ZK. For example, Kafka's kafka-consumer-offset-checker.sh which is very helpful to check consumer lag, meaning
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their Nesting exercise. object Solution { def solution(S: String): Int = { val results = S.foldLeft(0) { (total, chr) => if (total < 0
Read More

I am currently taking Stanford's Algorithms: Design and Analysis, Part 1 class on Coursera. I wanted to share my solutions which I am writing in Scala. Week 2 is mostly about QuickSort. QuickSort - Recursive, In-place package algo import scala
Read More

I am currently taking Stanford's Algorithms: Design and Analysis, Part 1 class on Coursera. I wanted to share my solutions which I am writing in Scala. Count Inversions using Merge Sort package algo import scala.math import Ordering.Implicits._ import
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their MissingInteger exercise. object Solution { def solution(A: Array[Int]): Int = { val bitz = new java.util.BitSet(A.size) val n = A
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their TapeEquilibrium exercise. import scala.math.{min, abs} object Solution { def solution(A: Array[Int]): Int = { if (A.size < 2 || A
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their MinAvgTwoSlice exercise. object Solution { def solution(A: Array[Int]): Int = { (A.foldLeft[(Int, Option[Double], Int)]((0, None, -1)) { (t, item
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their Missing Integer exercise. When I did A.filter(_ > 0).sorted, which would be more clear, I got only 75% in
Read More

I've been having some fun doing Codility exercises now that they have Scala support. Here's my solution for their Distinct exercise. I could also have sorted the input and then go through each element to save some space. Or better
Read More