How I passed GCP Associate Cloud Engineer certification

I attended the Google cloud summit Bangalore’19 which piqued my interest in Google Cloud Platform. Given the interest and demand around multi-cloud experience in the industry, I decided to dive in. I’ve 3+ years of managing cloud infrastructure in AWS, so I’m familiar with the concept of Compute, Storage, Networking offerings in Cloud platforms. Coursera […]

Continue Reading

Password less SSH connection setup

SSH is the indispensable service for the Linux servers, which is a method of secure login from one server to a remote server. It is also used to transfer files over the network using secure copy (SCP) Protocol. In Hadoop clusters, SSH is commonly used to do connect to all nodes from the SCM server. […]

Continue Reading

Linux commands – Basics

Linux servers are used in all the organizations and irrespective of your roles, likes/dislikes, you’d use Linux in one way or another. So it’s good to learn the linux commands which would be handy for you whenever you get a chance to work on it. I have listed the basic and commonly used commands in […]

Continue Reading

Hadoop Cluster – Pre Maintenance procedure

In IT, it’s inevitable that all the servers will go for monthly security, vulnerability patching and hadoop servers are no exception. There’d be a separate Systems team to perform OS related patching, security updates, etc and your role is to bring down/up the cluster, ensure the application is good post patching. You’ve have to schedule […]

Continue Reading

CCA131 – Exam Answers Validation Steps

In this post, I’ll explain some of the ways to validate the problems you’ve solved in the exam. Using this steps, you can ensure that the you’ve solved the problems as expected and avoid shocks/surprises in the exam result. When you complete the exam before the alloted time and if you have free time of […]

Continue Reading

Determine reason for application failure

There are many possible causes for a job/application failure varying from code error, environments, files availability, permissions, mapreduce/yarn configuration, resources allocation and even due to server i/o, network issue etc., So the first thing you’ve to do when a job fails is, to look at the error message and correlate with your job. If an […]

Continue Reading

Benchmark the cluster (I/O, CPU, network)

Benchmarking is the process of stress testing the resources of the cluster. It’s very useful in understanding the performance of your cluster and to check whether it’s performing as expected before taking it live. Here we are going to test speed in which files are being read/write in HDFS, time taken for mappers/reducers to process […]

Continue Reading