Open in app

Sign In

Write

Sign In

James (Anh-Tu) Nguyen
James (Anh-Tu) Nguyen

56 Followers

Home

About

Published in Geek Culture

·May 18, 2021

Spark on K8s — Send Spark job’s Metrics to DataDog Using Autodiscovery

Today, I will share how we send Spark job metrics on Kubernetes to DataDog, which will be used to create a monitor or dashboard for Spark applications. Requirements I would recommend you spend some times reading my previous blogs to understand about Spark on K8s and DataDog: How to run a…

Kubernetes

8 min read

Spark on K8s — Send Spark Job’s Metrics to DataDog using Autodiscovery
Spark on K8s — Send Spark Job’s Metrics to DataDog using Autodiscovery
Kubernetes

8 min read


Published in Geek Culture

·May 13, 2021

Observability on K8s — DataDog Autodiscovery and DogStatsD

In the previous article, I have introduced about DataDog and how to deploy it to the Kubernetes cluster. Today I will share about the Autodiscovery feature in DataDog, and how we can send custom metrics to DogStastD socket. Requirements Basic knowledge about K8s, Docker, and DataDog. Administrator permission on a running…

Kubernetes

7 min read

Observability on K8s — DataDog Autodiscovery and DogStatsD
Observability on K8s — DataDog Autodiscovery and DogStatsD
Kubernetes

7 min read


Published in Geek Culture

·Oct 17, 2020

Observability on K8s — Monitor Kubernetes Clusters with DataDog

When you are managing more than 10 Kubernetes (EKS) clusters for your team with some critical applications running on them, you will need to set up an observability stack with monitoring, logging, and tracing. In this blog, I will share how we build a monitoring system, set up a dashboard…

Kubernetes

8 min read

Observability on K8s — Monitor Kubernetes Clusters With DataDog
Observability on K8s — Monitor Kubernetes Clusters With DataDog
Kubernetes

8 min read


Published in The Startup

·Aug 14, 2020

Spark on K8s — Perform a Spark-Submit to Amazon EKS Cluster With IRSA

In previous article, I have introduced how we submit a Spark job to an EKS cluster. As long as we’re using other AWS components for our pipelines to interact, like S3/DynamoDB/etc., we need assign IAM policy to the Spark driver and executor pods. In this tutorial, I will show you how-to…

Spark

7 min read

How-to Perform a Spark-Submit to Amazon EKS Cluster With IRSA
How-to Perform a Spark-Submit to Amazon EKS Cluster With IRSA
Spark

7 min read


Sep 5, 2019

Spark on K8s — Run a Spark job on Amazon EKS cluster

It took me 2 weeks to successfully submit a Spark job on Amazon EKS cluster, because lack of documentations, or most of them are about running on Kubernetes with kops or GKE, not Amazon EKS. …

Kubernetes

7 min read

How-to run Spark job on EKS cluster
How-to run Spark job on EKS cluster
Kubernetes

7 min read

James (Anh-Tu) Nguyen

James (Anh-Tu) Nguyen

56 Followers

A Fulltime SRE and 24/7 Oncall Dad

Following
  • Palantir

    Palantir

  • Tony

    Tony

  • Darius Foroux

    Darius Foroux

  • Michael Lin

    Michael Lin

  • Pavan Belagatti

    Pavan Belagatti

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech