Kubernetes

Kubernetes

DevOps / Build, Test, Deploy / Container Tools
CEO at Scrayos UG (haftungsbeschränkt)·

We primarily use Prometheus to gather metrics and statistics to display them in Grafana. Aside from that we poll Prometheus for our orchestration-solution "JCOverseer" to determine, which host is least occupied at the moment.

While there are existing orchestration softwares/suites like Kubernetes, that we also plan to adopt in the future, we're of the opinion that those solutions do not fit our special environment within minecraft and our own solution will outperform them in the limited scope that it needs to cover.

READ MORE
1 upvote·222.1K views
Senior Architect at Rainforest QA·

We recently moved our main applications from Heroku to Kubernetes . The 3 main driving factors behind the switch were scalability (database size limits), security (the inability to set up PostgreSQL instances in private networks), and costs (GCP is cheaper for raw computing resources).

We prefer using managed services, so we are using Google Kubernetes Engine with Google Cloud SQL for PostgreSQL for our PostgreSQL databases and Google Cloud Memorystore for Redis . For our CI/CD pipeline, we are using CircleCI and Google Cloud Build to deploy applications managed with Helm . The new infrastructure is managed with Terraform .

Read the blog post to go more in depth.

READ MORE
Why Rainforest QA Moved from Heroku to Google Kubernetes Engine (rainforestqa.com)
20 upvotes·1 comment·1.6M views
Dev Suryawanshi
Dev Suryawanshi
·
January 19th 2020 at 10:11AM

Great information

·
Reply
Lead Engineer at StackShare·

We began our hosting journey, as many do, on Heroku because they make it easy to deploy your application and automate some of the routine tasks associated with deployments, etc. However, as our team grew and our product matured, our needs have outgrown Heroku. I will dive into the history and reasons for this in a future blog post.

We decided to migrate our infrastructure to Kubernetes running on Amazon EKS. Although Google Kubernetes Engine has a slightly more mature Kubernetes offering and is more user-friendly; we decided to go with EKS because we already using other AWS services (including a previous migration from Heroku Postgres to AWS RDS). We are still in the process of moving our main website workloads to EKS, however we have successfully migrate all our staging and testing PR apps to run in a staging cluster. We developed a Slack chatops application (also running in the cluster) which automates all the common tasks of spinning up and managing a production-like cluster for a pull request. This allows our engineering team to iterate quickly and safely test code in a full production environment. Helm plays a central role when deploying our staging apps into the cluster. We use CircleCI to build docker containers for each PR push, which are then published to Amazon EC2 Container Service (ECR). An upgrade-operator process watches the ECR repository for new containers and then uses Helm to rollout updates to the staging environments. All this happens automatically and makes it really easy for developers to get code onto servers quickly. The immutable and isolated nature of our staging environments means that we can do anything we want in that environment and quickly re-create or restore the environment to start over.

The next step in our journey is to migrate our production workloads to an EKS cluster and build out the CD workflows to get our containers promoted to that cluster after our QA testing is complete in our staging environments.

READ MORE
8 upvotes·622.1K views
Needs advice
on
PrometheusPrometheus
and
SysdigSysdig

We have Prometheus as a monitoring engine as a part of our stack which contains Kubernetes cluster, container images and other open source tools. Also, I am aware that Sysdig can be integrated with Prometheus but I really wanted to know whether Sysdig or sysdig+prometheus will make better monitoring solution.

READ MORE
11 upvotes·765.3K views
Tech Lead, Big Data Platform at Pinterest·

To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

#BigData #AWS #DataScience #DataEngineering

READ MORE
Presto at Pinterest - Pinterest Engineering Blog - Medium (medium.com)
38 upvotes·1 comment·3.3M views
Kaibo Hao
Kaibo Hao
·
January 28th 2020 at 12:46AM

ECS on AWS will reduce your cost on EC2 and Kubernetes. Athena may be another tool for reducing your cost by replacing the Presto. It takes advantage of the S3 as the storage and provided the serverless management for your infrastructure.

·
Reply
Senior Fullstack Developer at QUANTUSflow Software GmbH·

Our whole DevOps stack consists of the following tools:

  • GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
  • Respectively Git as revision control system
  • SourceTree as Git GUI
  • Visual Studio Code as IDE
  • CircleCI for continuous integration (automatize development process)
  • Prettier / TSLint / ESLint as code linter
  • SonarQube as quality gate
  • Docker as container management (incl. Docker Compose for multi-container application management)
  • VirtualBox for operating system simulation tests
  • Kubernetes as cluster management for docker containers
  • Heroku for deploying in test environments
  • nginx as web server (preferably used as facade server in production environment)
  • SSLMate (using OpenSSL) for certificate management
  • Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
  • PostgreSQL as preferred database system
  • Redis as preferred in-memory database/store (great for caching)

The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

  • Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
  • Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
  • Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
  • Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
  • Scalability: All-in-one framework for distributed systems.
  • Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
READ MORE
30 upvotes·2 comments·11.6M views
Larry Gryziak
Larry Gryziak
·
April 30th 2020 at 6:34PM

So why is your deployment different for your (Heroku) test/dev and your stage/production?

·
Reply
Simon Reymann
Simon Reymann
·
May 1st 2020 at 10:32AM

When it comes to testing our web app we do not demand great computational resources and need a very simple, convenient and fast PaaS solution for deploying the app to our testers. In production though, the demand of great computational resources can rise very fast. With Amazon we are able to control that in better way.

·
Reply
Product Manager | SaaS | Traveller ·
Needs advice
on
DockerDockerMongoDBMongoDB
and
RabbitMQRabbitMQ

Hi, I am building an enhanced web-conferencing app that will have a voice/video call, live chats, live notifications, live discussions, screen sharing, etc features. Ref: Zoom.

I need advise finalizing the tech stack for this app. I am considering below tech stack:

  • Frontend: React
  • Backend: Node.js
  • Database: MongoDB
  • IAAS: #AWS
  • Containers & Orchestration: Docker / Kubernetes
  • DevOps: GitLab, Terraform
  • Brokers: Redis / RabbitMQ

I need advice at the platform level as to what could be considered to support concurrent video streaming seamlessly.

Also, please suggest what could be a better tech stack for my app?

#SAAS #VideoConferencing #WebAndVideoConferencing #zoom #stack

READ MORE
16 upvotes·435.7K views
Replies (8)
Recommends
on
WebRTC

You're going to want to look hard at WebRTC. It's what almost every realtime video service uses. The appeal is that it establishes a direct connection between peers so that the massive video bandwidth doesn't need to go through your backend. That aside, actor clusters will be the other technology that handle that sort of traffic well. It was popularized by erlang for telecom backbone, akka is another choice for actor systems.

Infrastructure wise, kubernetes would be a fine choice. Just make sure to look up some benchmarks for Container Network Interface (CNI) implementations that support high bandwidth traffic.

READ MORE
14 upvotes·1 comment·8.5K views
Nikhil Gurnani
Nikhil Gurnani
·
November 4th 2020 at 7:34AM

I absolutely second this recommendation! WebRTC is a must for apps that have high amount of audio / video streaming.

·
Reply
SDET at Ledningkart·
Recommends
on
Docker

Kubernetes provides Auto-scaling whereas Docker Swarm doesn't support autoscaling. Kubernetes supports up to 5000 nodes whereas Docker Swarm supports more than 2000 nodes. Kubernetes is less extensive and customizable whereas Docker Swarm is more comprehensive and highly customizable. So if your main usecase is autoscaling go for kubernetes else Docker is always a good choice.

READ MORE
10 upvotes·8.4K views
View all (8)
Head of Community at CTO.ai·
Needs advice
on
CTO.aiCTO.ai
and
GitHub ActionsGitHub Actions

Curious to get feedback from the community on automating developer workflows for DevOps and other admin tasks. Looking to use one of these for everything from delivery metrics to Kubernetes, and everything in between.

READ MORE
2 upvotes·1.3K views
Needs advice
on
DatadogDatadogNew RelicNew Relic
and
SysdigSysdig

We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.

We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.

We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.

You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?

READ MORE
7 upvotes·1.5M views
Replies (3)
Recommends
on
Datadog

Can't say anything to Sysdig. I clearly prefer Datadog as

  • they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
  • easy to code (python) agent plugins / api for own metrics
  • brillant dashboarding / alarms with many customization options
  • pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)

IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.

Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!

READ MORE
10 upvotes·2 comments·417.9K views
Medeti Vamsi Krishna
Medeti Vamsi Krishna
·
June 30th 2020 at 11:52AM

Thanks for the reply, I am working on DataDog trail version now. I am able to see my containers/pods/VMs metrics in the DataDog.

I am trying to do the jmx integration with autodiscovery now. But I am not able to see the jvm metrics in DataDog. Can you please help on this?

Here is my deployment yaml:

`

apiVersion: apps/v1

kind: Deployment

metadata:

name: myapp

namespace: datadog

annotations:

ad.datadoghq.com/myapp.check_names: >-

'["myapp"]'

ad.datadoghq.com/myapp.init_configs: >-

'[{"is_jmx": true, "collect_default_metrics": true}]'

ad.datadoghq.com/tomcat.instances: >-

'[{"host": "%%host%%","port":"5000"}]'

labels:

app: myapp

spec:

selector:

matchLabels:

app: myapp

template:

metadata:

labels:

app: myapp

spec:

containers:

- name: myapp

image: nexus.nslhub.com/sample-java-app:2.0

imagePullPolicy: Always

ports:

- containerPort: 8080

name: http

- containerPort: 5000

name: jmx

imagePullSecrets:

- name: myappsecret

nodeSelector:

kubernetes.io/hostname: ip-10-5-7-173.ap-south-1.compute.internal

`

·
Reply
Jens Günther
Jens Günther
·
June 30th 2020 at 11:57AM

Would like to help, but there could be hundreds of reasons why the incoming and outgoing jmx ports are not accessible from the agent.

·
Reply
Recommends
on
Instana

Hi Medeti,

you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.

As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/

We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/

Let me know if you need anything on top.

READ MORE
8 upvotes·417.8K views
View all (3)
Needs advice
on
NGINXNGINXTraefikTraefik
and
ZuulZuul

Which gateway/reverse proxy should I use? We are using a microservices architecture. we will be also using Kubernetes.

READ MORE
3 upvotes·585 views
Replies (1)
Recommends
on
Kong

We're using Kong Ingress with K8s and so far for our microservices and it's been super stable and also very easy to setup. Main reason we picked it over other products was built-in API key auth for their open source offering.

READ MORE
2 upvotes·499 views