Cloud Computing Lab (IT-704) - Unit 4 Short Notes

UNIT 4: CLOUD IMPLEMENTATION, ORCHESTRATION & OPERATIONS (LAB-FOCUSED)

4.1 Advanced Virtualization & Containerization Labs

4.1.1 Deep Dive: Hypervisor Types (Type-1 vs. Type-2) - Practical Setup & Comparison

[!TIP] Exam Focus: Know which hypervisor is used in major cloud platforms (AWS Nitro/Hypervisor, Azure Hyper-V, GCP KVM). Type-1 is mandatory for production cloud infrastructure.

4.1.2 Docker Core Commands & Image Management Lab

Core Container Lifecycle Commands:

docker run -d --name <container> <image>        # Create & start

docker exec -it <container> /bin/bash          # Execute command in running container

docker commit <container> <new_image_name>     # Create image from container changes

docker stop/start/restart <container>          # Control state

docker rm -f <container>                       # Remove (force if running)

docker ps -a                                   # List all containers

docker images                                  # List local images

Building Custom Images with Dockerfile:

FROM ubuntu:20.04                # Base image

RUN apt-get update && apt-get install -y nginx  # Install packages

COPY index.html /var/www/html/   # Copy files

EXPOSE 80                        # Document port

CMD ["nginx", "-g", "daemon off;"] # Default command

Build & Tag: docker build -t my-nginx:latest .

Volume & Network Management:

• Volumes: docker volume create <name>; mount with -v <volume>:/path for persistent data.

• Networks: docker network create <net>; connect containers with --network <net> for inter-container communication.

[!TIP] Common Pitfall: Forgetting to EXPOSE ports in Dockerfile doesn't publish them; must use -p flag in docker run. Volume data is lost if not using named volumes or bind mounts.

4.1.3 Docker Compose for Multi-Container Application Orchestration

docker-compose.yml Structure (Web + DB Example):

version: '3.8'

services:

  web:

    image: my-nginx:latest

    ports:

      - "8080:80"

    volumes:

      - app-data:/var/www/html

    depends_on:

      - db

  db:

    image: postgres:13

    environment:

      POSTGRES_PASSWORD: secret

    volumes:

      - db-data:/var/lib/postgresql/data

volumes:

  app-data:

  db-data:

Key Operations:

• docker-compose up -d — Start all services in detached mode.

• docker-compose scale web=3 — Scale a service (legacy; use deploy in Swarm mode).

• docker-compose down -v — Stop and remove containers & volumes.

[!TIP] Exam Ready: depends_on only controls start order, not readiness. Use health checks in production. For scaling in production, use Kubernetes or Docker Swarm.

4.2 Container Orchestration with Kubernetes (K8s) - Practical Labs

4.2.1 K8s Architecture & Core Components Lab

Core Components:

Local Cluster Setup: Use Minikube (minikube start) or Kind (kind create cluster).

[!TIP] Visual Aid: Search

DiagramSEARCH: kubernetes architecture diagram

for control plane/worker node relationships.

4.2.2 Declarative Application Deployment & Management

Sample Deployment YAML (deployment.yaml):

apiVersion: apps/v1

kind: Deployment

metadata:

  name: nginx-deploy

spec:

  replicas: 3

  selector:

    matchLabels:

      app: nginx

  template:

    metadata:

      labels:

        app: nginx

    spec:

      containers:

      - name: nginx

        image: nginx:1.21

        ports:

        - containerPort: 80

Workflow:

kubectl apply -f deployment.yaml   # Create/update

kubectl get pods                  # Verify

kubectl rollout status deployment/nginx-deploy

kubectl set image deployment/nginx-deploy nginx=nginx:1.22  # Rolling update

kubectl rollout undo deployment/nginx-deploy                # Rollback

kubectl scale deployment/nginx-deploy --replicas=5         # Scale

[!TIP] Critical: YAML indentation is syntax-sensitive. Use kubectl explain <resource> to learn field structure. Rolling updates are automatic; rollbacks require --to-revision or undo.

4.2.3 Service Exposure & Ingress Configuration Lab

Service Types:

Service YAML Example:

apiVersion: v1

kind: Service

metadata:

  name: nginx-service

spec:

  type: LoadBalancer

  selector:

    app: nginx

  ports:

    - protocol: TCP

      port: 80

      targetPort: 80

Basic Ingress Setup (requires Ingress Controller like Nginx):

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: example-ingress

spec:

  rules:

  - host: app.example.com

    http:

      paths:

      - path: /api

        pathType: Prefix

        backend:

          service:

            name: api-service

            port:

              number: 8080

      - path: /

        pathType: Prefix

        backend:

          service:

            name: web-service

            port:

              number: 80

[!TIP] Common Error: Ingress won't work without an Ingress Controller installed. Use minikube addons enable ingress for Minikube.

4.3 Infrastructure as Code (IaC) - Hands-On

4.3.1 Terraform Fundamentals & State Management Lab

Terraform Workflow:

1. terraform init — Initialize working directory (download providers).

2. terraform plan — Show execution plan.

3. terraform apply — Execute plan to create resources.

4. terraform destroy — Destroy managed infrastructure.

HCL Structure Example (main.tf):

terraform {

  required_providers {

    aws = {

      source  = "hashicorp/aws"

      version = "~> 4.0"

    }

  }

}

provider "aws" {

  region = "us-east-1"

}

resource "aws_instance" "web" {

  ami           = "ami-0c55b159cbfafe1f0"

  instance_type = "t2.micro"

  tags = {

    Name = "WebServer"

  }

}

output "public_ip" {

  value = aws_instance.web.public_ip

}

State Management:

• State file (terraform.tfstate) tracks resource mappings.

• Never edit manually; use terraform state commands.

• For teams, use remote state backend (S3, Azure Storage) with locking (DynamoDB).

[!TIP] Critical: State file contains sensitive data. Secure it. Use terraform state list and terraform state show <resource> to inspect.

4.3.2 Provisioning Cloud Resources with Terraform

Lab Example: AWS VPC + EC2 + Security Group

resource "aws_vpc" "main" {

  cidr_block = "10.0.0.0/16"

}

resource "aws_security_group" "allow_http" {

  vpc_id = aws_vpc.main.id

  ingress {

    from_port   = 80

    to_port     = 80

    protocol    = "tcp"

    cidr_blocks = ["0.0.0.0/0"]

  }

}

resource "aws_instance" "web" {

  ami           = "ami-0c55b159cbfafe1f0"

  instance_type = "t2.micro"

  vpc_security_group_ids = [aws_security_group.allow_http.id]

  subnet_id = aws_subnet.public.id

}

Using Modules: Create reusable modules (e.g., module "vpc" { source = "./modules/vpc" }).

[!TIP] Best Practice: Use variables (variable "region" {}) and outputs for modularity. Tag all resources for cost tracking.

4.4 Configuration Management & Automation

4.4.1 Ansible Basics: Playbooks, Inventory, and Ad-Hoc Commands Lab

Ad-Hoc Command: ansible all -i inventory.ini -m ping (test connectivity).

Inventory File (inventory.ini):

[webservers]

web1 ansible_host=192.168.1.10

web2 ansible_host=192.168.1.11

[db]

db1 ansible_host=192.168.1.20

Idempotent Playbook (site.yml):

- hosts: webservers

  become: yes

  tasks:

    - name: Install Nginx

      apt:

        name: nginx

        state: present

    - name: Start and enable Nginx

      service:

        name: nginx

        state: started

        enabled: yes

    - name: Copy index.html

      copy:

        src: files/index.html

        dest: /var/www/html/index.html

      notify: Restart Nginx

  handlers:

    - name: Restart Nginx

      service:

        name: nginx

        state: restarted

Variables & Templates: Use vars.yml or group_vars/. Templates with Jinja2: {{ variable }}.

[!TIP] Key Concept: Idempotency — running a playbook multiple times yields same result. Use state: present not command: apt install.

4.4.2 Integrating Ansible with Provisioned Infrastructure

Lab: Terraform Output → Ansible Inventory

1. Terraform output (outputs.tf):

   
   output "web_public_ips" {
   
     value = aws_instance.web[*].public_ip
   
   }
   
   

2. After terraform apply, capture IPs: terraform output -raw web_public_ips > ips.txt.

3. Dynamic Inventory Script (Python/JSON) or generate static inventory:

   
   # Generate inventory.ini from ips.txt
   
   echo "[webservers]" > inventory.ini
   
   cat ips.txt | awk '{print $$\displaystyle 1 " ansible_host=" $$2}' >> inventory.ini
   
   

4. Run Ansible: ansible-playbook -i inventory.ini site.yml.

[!TIP] Automation: Use local-exec provisioner in Terraform to run Ansible post-apply, or use CI/CD pipeline to orchestrate.

4.5 Cloud Security & Identity Lab Exercises

4.5.1 Identity and Access Management (IAM) Deep Dive

Principle of Least Privilege (PoLP): Grant only minimum permissions needed.

• AWS IAM Policy Example (S3 read-only):

  
  {
  
    "Version": "2012-10-17",
  
    "Statement": [
  
      {
  
        "Effect": "Allow",
  
        "Action": "s3:GetObject",
  
        "Resource": "arn:aws:s3:::mybucket/*"
  
      }
  
    ]
  
  }
  
  

• Azure RBAC: Assign built-in roles (e.g., Contributor) at narrowest scope (resource group, not subscription).

• GCP IAM: Use predefined roles (e.g., roles/storage.objectViewer) or custom roles.

[!TIP] Exam Trap: Avoid wildcards (*) in Action/Resource unless absolutely necessary. Use conditions ("Condition") for extra restriction (e.g., IP whitelisting).

4.5.2 Secrets Management in the Cloud

Integration Example (K8s + AWS Secrets Manager):

1. Store secret in AWS Secrets Manager.

2. Use External Secrets Operator or Secrets Store CSI Driver to sync to K8s Secret.

3. Mount as environment variable or volume in Pod spec.

[!TIP] Never commit secrets to Git. Use .gitignore for state files and inventory with secrets. Rotate keys regularly.

4.5.3 Network Security Groups & Firewall Rules Configuration Lab

Micro-segmentation Example (AWS Security Group for 3-Tier App):

• Web SG: Allow 80/443 from 0.0.0.0/0, 8080 from App SG only.

• App SG: Allow 8080 from Web SG, 5432 from DB SG only.

• DB SG: Allow 5432 from App SG only, no internet access.

Azure NSG Rule:

az network nsg rule create --nsg-name myNSG --name Allow-App-To-DB \

  --priority 100 --source-address-prefixes <App-SG-ID> \

  --destination-port-ranges 5432 --access Allow --protocol Tcp

[!TIP] Security Hygiene: Default deny all inbound. Use descriptive names. Log traffic with VPC Flow Logs / NSG flow logs.

4.6 Monitoring, Logging & Cost Management Labs

4.6.1 Cloud-Native Monitoring & Alerting

AWS CloudWatch for EC2/K8s:

• Metrics: CPUUtilization, MemoryUtilization (custom), NetworkIn.

• Create Alarm: aws cloudwatch put-metric-alarm --alarm-name HighCPU --metric-name CPUUtilization --namespace AWS/EC2 --statistic Average --period 300 --threshold 80 --comparison-operator GreaterThanThreshold --dimensions Name=InstanceId,Value=i-12345 --evaluation-periods 2 --alarm-actions arn:aws:sns:...

• Dashboard: Add widgets for graphs, numbers.

Azure Monitor / GCP Monitoring: Similar concepts — metrics, alerts, dashboards.

[!TIP] K8s Monitoring: Use Prometheus + Grafana (self-managed) or cloud-managed (Amazon Managed Service for Prometheus). Exporters: node-exporter, kube-state-metrics.

4.6.2 Centralized Logging with EFK/ELK Stack or Cloud Logging Services

EFK Stack (Elasticsearch, Fluentd, Kibana) on K8s:

1. Deploy Elasticsearch (statefulset).

2. Deploy Fluentd as DaemonSet to collect container logs.

3. Deploy Kibana for visualization.

4. Configure Fluentd to parse JSON logs, add Kubernetes metadata.

Cloud Logging (AWS CloudWatch Logs):

aws logs create-log-group --log-group-name /aws/containerinsights/mycluster/application

aws logs put-log-events --log-group-name ... --log-stream-name ...

Query Example (CloudWatch Logs Insights):

fields @timestamp, @message

filter @logStream like 'nginx'

sort @timestamp desc

limit 20

[!TIP] Structured Logging: Ensure applications output JSON logs for easier parsing. Use fluentd filters to enrich with pod name, namespace.

4.6.3 Cloud Cost Tracking & Optimization Lab

Cost Management Steps:

1. Enable Cost Allocation Tags: Tag all resources (Environment=Prod, Owner=TeamA).

2. Use Cost Explorer/Budgets:

   • AWS: aws ce get-cost-and-usage --time-period Start=$$\displaystyle (date -d '1 month ago' +%Y-%m-%d),End= $$(date +%Y-%m-%d) --granularity MONTHLY --metrics "BlendedCost"

   • Set budget alerts at 80% of monthly budget.

3. Identify Underutilized Resources:

   • Low Utilization EC2: CloudWatch CPUUtilization < 10% for 7 days.

   • Unattached EBS Volumes: aws ec2 describe-volumes --filters Name=status,Values=available.

   • Idle Load Balancers: No healthy hosts or low request count.

4. Right-Sizing: Use AWS Compute Optimizer recommendations.

[!TIP] Cost Formula: Total Monthly Cost = \sum (Resource_Hourly_Rate \times Hours_Used) + Data_Transfer_Costs. Always check data transfer costs between zones/regions.

4.7 CI/CD Pipeline Integration for Cloud Applications (Lab)

4.7.1 End-to-End Pipeline with a CI/CD Tool

GitHub Actions Pipeline Example (.github/workflows/deploy.yml):

name: Deploy to K8s

on:

  push:

    branches: [ main ]

jobs:

  build:

    runs-on: ubuntu-latest

    steps:

    - uses: actions/checkout@v3

    - name: Build Docker image

      run: |

        docker build -t myapp:${{ github.sha }} .

        docker tag myapp:${{ github.sha }} myrepo/myapp:latest

    - name: Push to Docker Hub

      run: |

        echo $$\displaystyle {{ secrets.DOCKER_PASSWORD }} | docker login -u $${{ secrets.DOCKER_USER }} --password-stdin

        docker push myrepo/myapp:latest

  deploy:

    needs: build

    runs-on: ubuntu-latest

    steps:

    - uses: actions/checkout@v3

    - name: Deploy to K8s

      run: |

        kubectl set image deployment/myapp myapp=myrepo/myapp:${{ github.sha }}

      env:

        KUBECONFIG: ${{ secrets.KUBECONFIG }}

Pipeline Stages: Code → Test → Build Image → Push Registry → Update K8s (via kubectl set image or Terraform).

[!TIP] Security: Store secrets (Docker Hub creds, kubeconfig) in CI/CD vault (GitHub Secrets, Jenkins Credentials). Never in code.

4.7.2 Pipeline Security: Integrating SAST/DAST and Secret Scanning

• SAST (Static Application Security Testing): Scan code for vulnerabilities (e.g., Trivy for Dockerfiles, Bandit for Python).

• DAST (Dynamic Application Security Testing): Scan running app (e.g., OWASP ZAP).

• Secret Scanning: Use GitGuardian, TruffleHog, or built-in GitHub secret scanning.

Integration Example (GitHub Actions):

- name: Run Trivy vulnerability scanner

  uses: aquasecurity/trivy-action@master

  with:

    image-ref: myrepo/myapp:latest

    format: 'sarif'

    output: 'trivy-results.sarif'

- name: Upload Trivy scan results to GitHub Security tab

  uses: github/codeql-action/upload-sarif@v2

  with:

    sarif_file: 'trivy-results.sarif'

[!TIP] Shift Left: Run security scans early (build stage). Fail pipeline on high-severity vulnerabilities.

4.8 Lab Integration & Mini-Project

4.8.1 Designing and Deploying a Secure, Scalable Multi-Tier Application

Architecture:

1. IaC (Terraform): Provision VPC, subnets (public/private), EC2/K8s nodes, RDS, Load Balancer.

2. Orchestration (K8s): Deploy frontend (Deployment + Service), backend (Deployment + Service), database (StatefulSet or cloud-managed RDS).

3. Configuration (Ansible): Bootstrap nodes (install Docker/K8s), configure app settings via ConfigMaps/Secrets.

4. CI/CD: Automate build/test/deploy on Git push.

Example Flow:

# Terraform creates infra & outputs K8s kubeconfig

terraform apply

# Ansible configures nodes (if bare metal)

ansible-playbook -i inventory.ini setup.yml

# CI/CD pipeline builds image & updates K8s deployment

# (Triggered by Git push)

4.8.2 Implementing Full Observability Stack

• Metrics: Prometheus + Grafana (scrape K8s /metrics endpoint) or CloudWatch.

• Logs: Fluentd → Elasticsearch → Kibana (or CloudWatch Logs).

• Traces: Jaeger or AWS X-Ray for distributed tracing (instrument app code).

• Dashboards: Single pane of glass showing request rate, error rate, latency (RED metrics), resource utilization.

4.8.3 Cost Report & Security Audit

Cost Report:

• Use cloud provider cost tools with tags.

• Generate CSV/PDF: aws ce get-cost-and-usage --query 'ResultsByTime[].{Date:TimePeriod.Start, Cost:Total.BlendedCost}'.

• Identify top 5 costly resources.

Security Audit:

• IAM: Review policies with IAM Access Analyzer / Azure AD Identity Protection.

• Network: Check security group rules (no 0.0.0.0/0 on non-web ports).

• K8s: Scan with kube-bench (CIS benchmarks), kubesec.

• Compliance: Use AWS Config / Azure Policy to evaluate against standards.

[!TIP] Mini-Project Deliverables: Architecture diagram, IaC code repo, CI/CD pipeline config, monitoring dashboards, cost report, security audit checklist. Document everything — examiners love clear documentation.