Shantanu's Blog

If we have a dataframe that needs to be iterated, we can use itertupes method as shown in this example.

import pandas as pd
df = pd.DataFrame([('a1','b1','c1','1Jan1990','15Aug1990'),
                   ('a3','b3','c3','11May1986','11May1986'),
                   ('a5','b5','c5','1Dec1984','31Dec2017')],
                  columns=['A','B','C','Date1','Date2'])

for tup in df.itertuples():
    print (tup)

pd.PeriodIndex(start='1Jan1990', end='15Aug1990', freq='D')

result = []
for tup in df.itertuples():
    index = pd.PeriodIndex(start=tup.Date1, end=tup.Date2, freq='M')
    new_df = pd.DataFrame([(tup.A, tup.B, tup.C)], index=index)
    new_df['Month'] = new_df.index.month
    new_df['Year'] = new_df.index.year
    result.append(new_df)
result = pd.concat(result, axis=0)
print(result)

https://stackoverflow.com/questions/53780270/pythonic-way-to-fill-rows-with-date-range

Basic Syntax of AWS-SSM

AWS::SSM::Parameter::Name
AWS::SSM::Parameter::Value
AWS::SSM::Parameter::Value>
AWS::SSM::Parameter::Value

_____

Supported AWS-Specific Parameter Types
AWS CloudFormation supports the following AWS-specific types:

AWS::EC2::AvailabilityZone::Name
An Availability Zone, such as us-west-2a.

AWS::EC2::Image::Id
An Amazon EC2 image ID, such as ami-0ff8a91507f77f867. Note that the AWS CloudFormation console doesn't show a drop-down list of values for this parameter type.

AWS::EC2::Instance::Id
An Amazon EC2 instance ID, such as i-1e731a32.

AWS::EC2::KeyPair::KeyName
An Amazon EC2 key pair name.

AWS::EC2::SecurityGroup::GroupName
An EC2-Classic or default VPC security group name, such as my-sg-abc.

AWS::EC2::SecurityGroup::Id
A security group ID, such as sg-a123fd85.

AWS::EC2::Subnet::Id
A subnet ID, such as subnet-123a351e.

AWS::EC2::Volume::Id
An Amazon EBS volume ID, such as vol-3cdd3f56.

AWS::EC2::VPC::Id
A VPC ID, such as vpc-a123baa3.

AWS::Route53::HostedZone::Id
An Amazon Route 53 hosted zone ID, such as Z23YXV4OVPL04A.

List
An array of Availability Zones for a region, such as us-west-2a, us-west-2b.

List
An array of Amazon EC2 image IDs, such as ami-0ff8a91507f77f867, ami-0a584ac55a7631c0c. Note that the AWS CloudFormation console doesn't show a drop-down list of values for this parameter type.

List
An array of Amazon EC2 instance IDs, such as i-1e731a32, i-1e731a34.

List
An array of EC2-Classic or default VPC security group names, such as my-sg-abc, my-sg-def.

List
An array of security group IDs, such as sg-a123fd85, sg-b456fd85.

List
An array of subnet IDs, such as subnet-123a351e, subnet-456b351e.

List
An array of Amazon EBS volume IDs, such as vol-3cdd3f56, vol-4cdd3f56.

List
An array of VPC IDs, such as vpc-a123baa3, vpc-b456baa3.

List
An array of Amazon Route 53 hosted zone IDs, such as Z23YXV4OVPL04A, Z23YXV4OVPL04B.

_____

Supported SSM Parameter Types
AWS CloudFormation supports the following SSM parameter types:

AWS::SSM::Parameter::Name
The name of a Systems Manager parameter key.

Use this parameter when you want to pass the parameter key. For example, you can use this type to validate that the parameter exists.

AWS::SSM::Parameter::Value
A Systems Manager parameter whose value is a string. This corresponds to the String parameter type in Parameter Store.

AWS::SSM::Parameter::Value> or AWS::SSM::Parameter::Value
A Systems Manager parameter whose value is a list of strings. This corresponds to the StringList parameter type in Parameter Store.

AWS::SSM::Parameter::Value
A Systems Manager parameter whose value is an AWS-specific parameter type. For example, the following specifies the AWS::EC2::KeyPair::KeyName type:

AWS::SSM::Parameter::Value

AWS::SSM::Parameter::Value>
A Systems Manager parameter whose value is a list of AWS-specific parameter types. For example, the following specifies a list of AWS::EC2::KeyPair::KeyName types:

AWS::SSM::Parameter::Value>

_____

# Use public Systems Manager Parameter
Parameters :
  LatestAmiId :
    Type : 'AWS::SSM::Parameter::Value'
    Default: ‘/aws/service/ami-windows-latest/Windows_Server-2016-English-Core-Containers’

Resources :
  Instance :
    Type : 'AWS::EC2::Instance'
    Properties :
      ImageId : !Ref LatestAmiId
# Create-stack CLI call
aws cloudformation create-stack --stack-name S1 --template-body

# Describe stack output’s ‘Parameters’ section for this stack
aws cloudformation describe-stacks --stack-name S1


_____

# Create a parameters for Dev and Prod environments in Systems Manager Parameter Store
aws ssm put-parameter --name myEC2TypeDev --type String --value “t2.small”
aws ssm put-parameter --name myEC2TypeProd --type String --value “m4.large”

# Reference/use existing Systems Manager Parameter in CloudFormation
Parameters:
  InstanceType :
    Type : 'AWS::SSM::Parameter::Value'
    Default: myEC2TypeDev
  KeyName :
    Type : 'AWS::SSM::Parameter::Value'
    Default: myEC2Key
  AmiId:
    Type: 'AWS::EC2::Image::Id'
    Default: 'ami-60b6c60a'
   
Resources :
  Instance :
    Type : 'AWS::EC2::Instance'
    Properties :
       Type : !Ref InstanceType
       KeyName : !Ref KeyName
       ImageId : !Ref AmiId

# Call create-stack for Dev environment by passing SSM parameter key as template parameter value
aws cloudformation create-stack --stack-name S1 --template-body

# Call create-stack for Prod environment by passing SSM parameter key as template parameter value
aws cloudformation create-stack --stack-name S1 --template-body --parameters ParameterKey=InstanceType,ParameterValue=myEC2TypeProd

_____

Parameters:
  KeyName:
    Type: 'AWS::EC2::KeyPair::KeyName'
    Default: brinks
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
  PSBInstance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: !Ref LatestAmiId
      KeyName: !Ref KeyName
      InstanceType: '{{resolve:ssm:myEC2TypeDev:1}}'

_____

aws ssm put-parameter --name ssbRDSiClass --type String --value "db.t2.medium"
aws ssm put-parameter --name ssbRDSmEcntl --type SecureString --value "ch4ng1ng-s3cr3t"

# This template creates a MariaDB RDS instance using the following:
Resources:
  MyRDSDB:
    Type: "AWS::RDS::DBInstance"
    Properties:
      # The following line uses a plain-text Parameter Store dynamic reference
      DBInstanceClass: "{{resolve:ssm:ssbRDSiClass:1}}"
      AllocatedStorage: '20'
      Engine: mariadb
      EngineVersion: '10.2'
      MasterUsername: appadmin
      # The following line uses a secure-string Parameter Store dynamic reference
      MasterUserPassword: "{{resolve:ssm-secure:ssbRDSmEcntl:1}}"
Outputs:
  DbInstanceId:
    Description: InstanceID of My RDS DB
    Value: !Ref MyRDSDB

I have assumed following 3 points.

1) Using us-east-1 region
2) "Default" security group is already configured correctly.
3) Sub-domain training.xyz.com is already assigned to IP address 18.208.241.12

The following shell script will run when the instance starts for the first time:

vi userdata.sh

#!/bin/bash -ex
yum install -y docker mysql git python-pip
sudo amazon-linux-extras install -y docker
pip install aws-ec2-assign-elastic-ip
aws-ec2-assign-elastic-ip --access-key XXX --secret-key XXX  --valid-ips 18.208.241.12
service docker start

docker run -d -p 8888:8888 -v /tmp:/tmp continuumio/miniconda3 /bin/bash -c "/opt/conda/bin/conda install jupyter -y && cd /tmp/ && /opt/conda/bin/jupyter notebook --NotebookApp.token='india' --notebook-dir=/tmp --ip='0.0.0.0' --port=8888 --no-browser --allow-root"

And here is the aws command to start the spot instance...

aws ec2 request-spot-instances --spot-price 0.01 --instance-count 1 \
  --launch-specification "{ \
    \"ImageId\":\"ami-009d6802948d06e52\", \
    \"InstanceType\":\"t2.small\", \
    \"KeyName\":\"dec15a\", \
    \"UserData\":\"`base64 -w 0 userdata.sh`\", \
    \"BlockDeviceMappings\": [ { \
       \"DeviceName\":\"/dev/xvda\", \
       \"Ebs\": { \
          \"VolumeSize\": 700, \
          \"VolumeType\": \"standard\" \
        } \
    } ] \
}"
a) t2 has a range of micro (1 GB), small (2 GB), medium (4 GB), large (8 GB), xlarge (16 GB), 2xlarge (32 GB) Or use c4 series (large: 4GB, xlarge: 8 GB, 2x: 16 GB, 4x:30, 8x:60)
b) Spot price needs to be adjusted according to instance type.
c) The default Amazon linux Image is being used in this example.