I recently configured a Kafka enabled Event Hub in Azure. I used a Spark Scala cluster to stream these events. The following are my findings. Kafka Enabled Event Hub First…
Category: Development
Background On previous posts, we explored Event Sourcing and how to implement a Fast Lane Notebook in Databricks. Now the question is: How to check the performance of a Spark…
We need to capture requirements in a clear and consistent manner. We will be using Gherkin syntax for doing this. Gherkin syntax provides a structured way to writing requirements for…
We have started working with the Fexco API Developer Portal as the main entry point for most of the resources that development engineers and SRE are currently building. It counts…
BACKGROUND Sometimes I think it makes sense to keep very simple tasks outside core application code. For instance, suppose a particular individual stakeholder wants the results of an SQL query…
Hi all, We’ll try to reflect in this post a summary of the main steps to follow when we want to create an ETL process in our Computing Platform. What…
Hi there, Many things have been told about the adoption of programming and languages in software projects. Too many times the criteria to choose some language, framework or tool are…
The following is a guideline for code reviewing Spark Applications. The code review doesn’t just end at the code, there are several configurations to review too. A clean configuration is…
Introduction This is the second part of a series of posts about how to develop Spark Applications. The first post was about working with notebooks. Objectives As we move forward…
Background This post aims to give an in-depth description about our continuous deployment pipeline. We are going to use the Continuous Deployment term as Martin Flower defined in this post,…