In this brave new world of distributed systems, we are entrusted with keeping the infrastructure up and running.The source of the challenge is to monitor the services themselves and the space in between. We face non-determinism, sometimes we can’t tell if our system is up, down, or partially working, and every failure is a taskContinue reading “Building resilient Distributed Systems at scale”
Tag Archives: big data
Data consistency across Microservices
We were told a monolith is evil and microservices are the answer. What nobody told us is that microservices come with many pain points deriving from its distributed nature. In the past, we built an application connected to one database where normalized data was queried using “joins”. Then came: big data, big traffic and withContinue reading “Data consistency across Microservices”
Querying our Data Lake in S3 using Zeppelin and Spark SQL
Until recently, most companies used the traditional approach for storing all the company’s data in a Data Warehouse. The internet growth caused an increase in the number of data sources and the massive quantities of data to be stored, requiring scaling these Data Warehouses constantly. They were not designed to handle petabytes of data, so companies wereContinue reading “Querying our Data Lake in S3 using Zeppelin and Spark SQL”
Denise Schlesinger 