Understanding Apache Spark Failures and Bottlenecks

When everything goes according to plan, it's easy to write and understand applications in Apache Spark. However, sometimes a well-tuned application might fail due to a data change or a data layout change — or an application that had been running well so far, might start behaving badly due to resource starvation. It's important to understand underlying runtime components like disk usage, network usage, contention, and so on, so that we can make an informed decision when things go bad.



We use cookies to optimize your experience, enhance site navigation, analyze site usage, assist in our marketing efforts. Privacy Policy