Posted in : Azure

2 years ago

In this blog post, I’ll describe how to deploy CoreOS using an ARM Template and auto start the Docker service as well as create four services for the VSTS Agent container.
Container Linux by CoreOS (now part of the Red Hat family) is a Linux distribution and comes with the minimal functionality required to deploy containers. One feature that is really handy when it comes to deploying CoreOS in Azure is Iginition, which is a provisioning utility built for the distribution. This utility makes it possible to (for example) configure services to auto start from an Azure Resource Manager (ARM) Template.
Before we begin, you will also be able to download what I describe in this post here.
First of, we need to describe the service:

[Unit]
Description=vsts-agentX
After=docker.service network.target
Requires=docker.service network.target
[Service]
Restart=always
RestartSec=10s
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill vsts-agentX
ExecStartPre=-/usr/bin/docker rm vsts-agentX
ExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard
ExecStart=/usr/bin/docker run --name vsts-agentX -e VSTS_ACCOUNT=<Account> -e VSTS_TOKEN=<Token> -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard
[Install]
WantedBy=multi-user.target

Note: VSTS_ACCOUNT and VSTS_TOKEN will be dynamic in the ARM Template and defined using parameters passed to the Ignition configuration dynamically at deployment. I’m using a static pool name ubuntu-16-04-docker-17-12-0-ce-standard.
When we know that the service works, we add it to the Ignition configuration:

{
    "ignition": {
        "version": "2.2.0"
    },
    "systemd": {
        "units": [
            {
                "name": "docker.service",
                "enabled": true
            },
            {
                "name": "vsts-agent1.service",
                "enabled": true,
                "contents": "[Unit]\nDescription=vsts-agent1\nAfter=docker.service network.target\nRequires=docker.service network.target\n\n[Service]\nRestart=always\nRestartSec=10s\nTimeoutStartSec=0\nExecStartPre=-/usr/bin/docker kill vsts-agent1\nExecStartPre=-/usr/bin/docker rm vsts-agent1\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\nExecStart=/usr/bin/docker run --name vsts-agent1 -e VSTS_ACCOUNT=<Account> -e VSTS_TOKEN=<Token> -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\n\n[Install]\nWantedBy=multi-user.target"
            },
            {
                "name": "vsts-agent2.service",
                "enabled": true,
                "contents": "[Unit]\nDescription=vsts-agent2\nAfter=docker.service network.target vsts-agent1.service\nRequires=docker.service network.target vsts-agent1.service\n\n[Service]\nRestart=always\nRestartSec=10s\nTimeoutStartSec=0\nExecStartPre=-/usr/bin/docker kill vsts-agent2\nExecStartPre=-/usr/bin/docker rm vsts-agent2\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\nExecStart=/usr/bin/docker run --name vsts-agent2 -e VSTS_ACCOUNT=<Account> -e VSTS_TOKEN=<Token> -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\n\n[Install]\nWantedBy=multi-user.target"
            },
            {
                "name": "vsts-agent3.service",
                "enabled": true,
                "contents": "[Unit]\nDescription=vsts-agent3\nAfter=docker.service network.target vsts-agent1.service\nRequires=docker.service network.target vsts-agent1.service\n\n[Service]\nRestart=always\nRestartSec=10s\nTimeoutStartSec=0\nExecStartPre=-/usr/bin/docker kill vsts-agent3\nExecStartPre=-/usr/bin/docker rm vsts-agent3\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\nExecStart=/usr/bin/docker run --name vsts-agent3 -e VSTS_ACCOUNT=<Account> -e VSTS_TOKEN=<Token> -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\n\n[Install]\nWantedBy=multi-user.target"
            },
            {
                "name": "vsts-agent4.service",
                "enabled": true,
                "contents": "[Unit]\nDescription=vsts-agent4\nAfter=docker.service network.target vsts-agent1.service\nRequires=docker.service network.target vsts-agent1.service\n\n[Service]\nRestart=always\nRestartSec=10s\nTimeoutStartSec=0\nExecStartPre=-/usr/bin/docker kill vsts-agent4\nExecStartPre=-/usr/bin/docker rm vsts-agent4\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\nExecStart=/usr/bin/docker run --name vsts-agent4 -e VSTS_ACCOUNT=<Account> -e VSTS_TOKEN=<Token> -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\n\n[Install]\nWantedBy=multi-user.target"
            }
        ]
    }
}

Note: In the same way as in the service description, we will be dynamically adding the VSTS_ACCOUNT and VSTS_TOKEN during the deployment.
Now when we have the Ignition configuration it’s just a matter of adding it to the ARM Template. One thing to note is that you will need to escape backslash making \n to \\n in the template.
The ARM Template can look like this: (note that variable coreosIgnitionConfig is a concatenated version of the json above)

{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "commonName": {
            "type": "string"
        },
        "locationShort": {
            "type": "string"
        },
        "environmentShort": {
            "type": "string"
        },
        "virtualMachineSize": {
            "type": "string"
        },
        "diskSizeGB": {
            "type": "int"
        },
        "adminUsername": {
            "type": "string"
        },
        "adminPublicKey": {
            "type": "string"
        },
        "virtualNetworkRG": {
            "type": "string"
        },
        "virtualNetworkName": {
            "type": "string"
        },
        "subnetName": {
            "type": "string"
        },
        "VstsAccount": {
            "type": "string"
        },
        "VstsToken": {
            "type": "string"
        }
    },
    "variables": {
        "vnetId": "[resourceId(parameters('virtualNetworkRG'),'Microsoft.Network/virtualNetworks', parameters('virtualNetworkName'))]",
        "subnetRef": "[concat(variables('vnetId'), '/subnets/', parameters('subnetName'))]",
        "envLocation": "[concat(parameters('environmentShort'), '-', parameters('locationShort'))]",
        "vmName": "[concat('vm-', variables('envLocation'), '-', parameters('commonName'))]",
        "nic1Name": "[concat('nic-', variables('vmName'))]",
        "osDisk1Name": "[concat('osdisk-', variables('vmName'))]",
        "coreosIgnitionConfig": "[concat('{\"ignition\":{\"version\":\"2.2.0\"},\"systemd\":{\"units\":[{\"name\":\"docker.service\",\"enabled\":true},{\"name\":\"vsts-agent1.service\",\"enabled\":true,\"contents\":\"[Unit]\\nDescription=vsts-agent1\\nAfter=docker.service network.target\\nRequires=docker.service network.target\\n\\n[Service]\\nRestart=always\\nRestartSec=10s\\nTimeoutStartSec=0\\nExecStartPre=-/usr/bin/docker kill vsts-agent1\\nExecStartPre=-/usr/bin/docker rm vsts-agent1\\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\nExecStart=/usr/bin/docker run --name vsts-agent1 -e VSTS_ACCOUNT=', parameters('VstsAccount'), ' -e VSTS_TOKEN=', parameters('VstsToken'), ' -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\n\\n[Install]\\nWantedBy=multi-user.target\"},{\"name\":\"vsts-agent2.service\",\"enabled\":true,\"contents\":\"[Unit]\\nDescription=vsts-agent2\\nAfter=docker.service network.target vsts-agent1.service\\nRequires=docker.service network.target vsts-agent1.service\\n\\n[Service]\\nRestart=always\\nRestartSec=10s\\nTimeoutStartSec=0\\nExecStartPre=-/usr/bin/docker kill vsts-agent2\\nExecStartPre=-/usr/bin/docker rm vsts-agent2\\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\nExecStart=/usr/bin/docker run --name vsts-agent2 -e VSTS_ACCOUNT=', parameters('VstsAccount'), ' -e VSTS_TOKEN=', parameters('VstsToken'), ' -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\n\\n[Install]\\nWantedBy=multi-user.target\"},{\"name\":\"vsts-agent3.service\",\"enabled\":true,\"contents\":\"[Unit]\\nDescription=vsts-agent3\\nAfter=docker.service network.target vsts-agent1.service\\nRequires=docker.service network.target vsts-agent1.service\\n\\n[Service]\\nRestart=always\\nRestartSec=10s\\nTimeoutStartSec=0\\nExecStartPre=-/usr/bin/docker kill vsts-agent3\\nExecStartPre=-/usr/bin/docker rm vsts-agent3\\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\nExecStart=/usr/bin/docker run --name vsts-agent3 -e VSTS_ACCOUNT=', parameters('VstsAccount'), ' -e VSTS_TOKEN=', parameters('VstsToken'), ' -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\n\\n[Install]\\nWantedBy=multi-user.target\"},{\"name\":\"vsts-agent4.service\",\"enabled\":true,\"contents\":\"[Unit]\\nDescription=vsts-agent4\\nAfter=docker.service network.target vsts-agent1.service\\nRequires=docker.service network.target vsts-agent1.service\\n\\n[Service]\\nRestart=always\\nRestartSec=10s\\nTimeoutStartSec=0\\nExecStartPre=-/usr/bin/docker kill vsts-agent4\\nExecStartPre=-/usr/bin/docker rm vsts-agent4\\nExecStartPre=/usr/bin/docker pull microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\nExecStart=/usr/bin/docker run --name vsts-agent4 -e VSTS_ACCOUNT=', parameters('VstsAccount'), ' -e VSTS_TOKEN=', parameters('VstsToken'), ' -e VSTS_POOL=ubuntu-16-04-docker-17-12-0-ce-standard -v /var/run/docker.sock:/var/run/docker.sock microsoft/vsts-agent:ubuntu-16.04-docker-17.12.0-ce-standard\\n\\n[Install]\\nWantedBy=multi-user.target\"}]}}')]"
    },
    "resources": [
        {
            "type": "Microsoft.Compute/virtualMachines",
            "name": "[variables('vmName')]",
            "apiVersion": "2018-04-01",
            "location": "[resourceGroup().location]",
            "properties": {
                "osProfile": {
                    "computerName": "[variables('vmName')]",
                    "adminUsername": "[parameters('adminUsername')]",
                    "customData": "[base64(variables('coreosIgnitionConfig'))]",
                    "linuxConfiguration": {
                        "disablePasswordAuthentication": "true",
                        "ssh": {
                            "publicKeys": [
                                {
                                    "path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
                                    "keyData": "[parameters('adminPublicKey')]"
                                }
                            ]
                        }
                    }
                },
                "hardwareProfile": {
                    "vmSize": "[parameters('virtualMachineSize')]"
                },
                "storageProfile": {
                    "imageReference": {
                        "publisher": "CoreOS",
                        "offer": "CoreOS",
                        "sku": "Stable",
                        "version": "latest"
                    },
                    "osDisk": {
                        "name": "[variables('osDisk1Name')]",
                        "createOption": "fromImage",
                        "diskSizeGB": "[parameters('diskSizeGB')]",
                        "managedDisk": {
                            "storageAccountType": "Premium_LRS"
                        }
                    },
                    "dataDisks": []
                },
                "networkProfile": {
                    "networkInterfaces": [
                        {
                            "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('nic1Name'))]"
                        }
                    ]
                }
            },
            "dependsOn": [
                "[concat('Microsoft.Network/networkInterfaces/', variables('nic1Name'))]"
            ]
        },
        {
            "type": "Microsoft.Network/networkInterfaces",
            "name": "[variables('nic1Name')]",
            "apiVersion": "2018-04-01",
            "location": "[resourceGroup().location]",
            "properties": {
                "ipConfigurations": [
                    {
                        "name": "ipconfig1",
                        "properties": {
                            "subnet": {
                                "id": "[variables('subnetRef')]"
                            },
                            "privateIPAllocationMethod": "Dynamic"
                        }
                    }
                ]
            },
            "dependsOn": []
        }
    ],
    "outputs": {
        "adminUsername": {
            "type": "string",
            "value": "[parameters('adminUsername')]"
        },
        "vmName": {
            "type": "string",
            "value": "[variables('vmName')]"
        },
        "privateIP": {
            "type": "string",
            "value": "[reference(concat('Microsoft.Network/networkInterfaces/', variables('nic1Name'))).ipConfigurations[0].properties.privateIPAddress]"
        }
    }
}

Note: I’ve also created a parameter file which can be modified for your environment. See more info here.
After deployment, you’ll have a simple VM with four containers running – and four agents in the agent pool:

Tags : ARM, Azure, Container, CoreOS, DevOps, Docker, Template, VSTS

Add comment

Your comment will be revised by the site if needed.