Skip to content

Blog

How to Build an NGINX Server and Deploy a Python Flask App Using Chef

In this post we show how you can use Chef to build an Ubuntu 14.04 LTS web server running Nginx, Python 2, virtualenv, uWSGI, and Flask - all done on Windows 10 host.

All commands are executed in PowerShell on a Windows workstation. The Chef version that is used is 12.5.1 - it is the version that comes with ChefDK 0.10.0. To avoid unexpected behavior, we recommend using those versions, when following this step-by-step guide.

Before You Begin

Test Flask Application

The test application that we use for this deployment is available in the my_flask_app repository. The application was created using Visual Studio 2015 with Python Tools for Visual Studio (PTVS).

Install Chocolatey

If you do not have Chocolatey, you can install it by following the instructions on chocolatey.org. You may also see this post for instructions on how to set the Chocolatey cache location.

Install Chef Development Kit (ChefDK)

In PowerShell as Administrator:

choco install chefdk -version 0.10.0.1

Find your Chef version:

chef --version

Chef Development Kit Version: 0.10.0
chef-client version: 12.5.1
berks version: 4.0.1
kitchen version: 1.4.2

Install VirtualBox and Vagrant

To install VirtualBox and Vagrant, follow the steps described in Ubuntu with Vagrant and VirtualBox on Windows.

Install Vagrant Plugins

vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-berkshelf
vagrant plugin install vagrant-hostmanager
vagrant plugin install vagrant-cachier

Give your user Modify access to WINDIR%\System32\drivers\etc\hosts

This is needed by the vagrant-hostmanager Vagrant plugin. In PowerShell as Administrator:

$acl = Get-Acl -Path $env:SystemRoot\System32\drivers\etc\hosts
$ar = New-Object System.Security.AccessControl.FileSystemAccessRule($env:username, 'Modify', 'None', 'None', 'Allow')
$acl.SetAccessRule($ar)
Set-Acl -Path $env:SystemRoot\System32\drivers\etc\hosts -AclObject $acl

Create Chef Cookbook

This directory will become the root of your source repository:

chef generate cookbook my_flask_server
cd my_flask_server

Ensure the apt cache is up to date

Reference the apt Cookbook

Add this line to metadata.rb:

depends 'apt', '~> 2.9.2'

To get the latest version string, run knife cookbook site show apt:

chef exec knife cookbook site show apt | grep latest_version
latest_version:     https://supermarket.chef.io/api/v1/cookbooks/apt/versions/2.9.2

Here is the complete file:

name 'my_flask_server'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures my_flask_server'
long_description 'Installs/Configures my_flask_server'
version '0.1.0'

depends 'apt', '~> 2.9.2'

Set the apt cookbook's default recipe to run

Add this line to recipes/default.rb:

include_recipe 'apt::default'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

Prepare for Test Driven Development

Add .rspec file

touch .rspec
Paste this in the .rspec file:

--color
--format documentation

Add .kitchen.vagrant_cachier.rb file

This file will be merged within Test Kitchen's generated Vagrantfile. In it, we put the configuration for the vagrant-cachier plugin

touch .kitchen.vagrant_cachier.rb
Paste this in the .kitchen.vagrant_hostmanager.rb file:

# This requires vagrant-cachier plugin.
# For more information see http://fgrehm.viewdocs.io/vagrant-cachier/
#
Vagrant.configure("2") do |config|
  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.auto_detect = true
    config.cache.scope = :box
  end

  if Vagrant.has_plugin?("vagrant-omnibus")
    config.omnibus.cache_packages = true
    config.omnibus.chef_version = "12.5.1"
  end

  config.vbguest.auto_update = false
end

Add .kitchen.vagrant_hostmanager.rb file

This file will be merged within Test Kitchen's generated Vagrantfile. In it, we put the configuration for the vagrant-hostmanager plugin, which updates C:/Windows/System32/drivers/etc/hosts, and allows you to access the virtual machine by name instead of IP address, e.g. default-ubuntu-1404.

touch .kitchen.vagrant_hostmanager.rb

Paste this in the .kitchen.vagrant_hostmanager.rb file:

# This requires vagrant-hostmanager plugin.
# For more information see https://github.com/smdahlen/vagrant-hostmanager
#
Vagrant.configure("2") do |config|
  if Vagrant.has_plugin?("vagrant-hostmanager")
    # update /ect/hosts on all running guests
    config.hostmanager.enabled = true

    # do not add offline guests to /etc/hosts
    config.hostmanager.include_offline = false

    # use the private ip address with ip_reslover, see below
    config.hostmanager.ignore_private_ip = false

    # custom IP resolver
    # get each guest's IP address by running `hostname -I` on the guest
    config.hostmanager.ip_resolver = proc do |vm, resolving_vm|
      if hostname = (vm.ssh_info && vm.ssh_info[:host])
        `vagrant ssh -c "hostname -I"`.split()[1]
      end
    end

    # also update host's /ect/hosts
    config.hostmanager.manage_host = true
  end
end

Update .kitchen.yml

Update .kitchen.yml to include only the ubuntu-14.04 platform for now. Also add the data bag path and Chef version. This is how it should look at the end:

---
driver:
  name: vagrant
  require_chef_omnibus: 12.5.1
  network:
    - ["private_network", {type: "dhcp"}]
  vagrantfiles:
    - .kitchen.vagrant_cachier.rb
    - .kitchen.vagrant_hostmanager.rb

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-14.04

suites:
  - name: default
    run_list:
      - recipe[my_flask_server::default]
    attributes:

Run All Tests Manually

Run all tests manually to verify everything works as expected.

First install cookbook dependencies:

chef exec berks install

then run the tests:

chef exec rubocop
chef exec foodcritic .
chef exec rspec
chef exec kitchen test --destroy=never

Setup a Build System

We will use Rake to automate the build and test tasks. Rake already comes preinstalled in the ChefDK.

Create a Rakefile

touch Rakefile

Paste this into the Rakefile:

# See:
# https://github.com/chef-cookbooks/chef-server/blob/master/Rakefile

require 'rspec/core/rake_task'
require 'rubocop/rake_task'
require 'foodcritic'
require 'kitchen'

# Style tests. Rubocop and Foodcritic
namespace :style do
  desc 'Run Ruby style checks'
  RuboCop::RakeTask.new(:ruby)

  desc 'Run Chef style checks'
  FoodCritic::Rake::LintTask.new(:chef) do |t|
    t.options = {
      fail_# categories: ['any']
    }
  end
end

desc 'Run all style checks'
task style: ['style:ruby', 'style:chef']

# Rspec and ChefSpec
desc 'Run ChefSpec examples'
RSpec::Core::RakeTask.new(:spec)

# Integration tests. Kitchen.ci
namespace :integration do
  desc 'Same as `chef exec kitchen test -d=never`'
  task :test do
    Kitchen.logger = Kitchen.default_file_logger
    Kitchen::Config.new.instances.each do |instance|
      instance.test(:never)
    end
  end
end

# Default
task default: ['style', 'spec', 'integration:test']

Test

List Tasks
chef exec rake -T
Run all tests
chef exec rake

Create Application Directory

Add an integration test

touch test/integration/default/serverspec/my_app_dir_spec.rb

Add this code to my_app_dir_spec.rb:

require 'spec_helper'

describe 'my_flask_server::my_app_dir' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  # verify virtual environment dir
  describe file('/var/www/my_flask_app/shared/.env2') do
    it { should be_directory }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end

  # verify uwsgi dir
  describe file('/var/www/my_flask_app/shared/.uwsgi') do
    it { should be_directory }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end

  # verify ssh dir
  describe file('/var/www/.ssh') do
    it { should be_directory }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end

  # verify pip cache dir
  describe file('/var/www/.cache') do
    it { should be_directory }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end
end

Write the my_app_dir recipe

The first step is to create the recipe file, my_app_dir.rb. Run the following command to generate it:

chef generate recipe my_app_dir
rm spec/unit/recipes/my_app_dir_spec.rb

Write out recipes/my_app_dir.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_app_dir
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

# app root directory
directory '/var/www/my_flask_app' do
  action :create
  recursive true
  user 'www-data'
  group 'www-data'
end

# app shared directory
directory '/var/www/my_flask_app/shared' do
  action :create
  user 'www-data'
  group 'www-data'
end

# for virtualenv
directory '/var/www/my_flask_app/shared/.env2' do
  action :create
  user 'www-data'
  group 'www-data'
end

# for uwsgi socket and uwsgi config
directory '/var/www/my_flask_app/shared/.uwsgi' do
  action :create
  user 'www-data'
  group 'www-data'
end

# required by ssh
directory '/var/www/.ssh' do
  action :create
  recursive true
  user 'www-data'
  group 'www-data'
end

# required by pip
directory '/var/www/.cache' do
  action :create
  recursive true
  user 'www-data'
  group 'www-data'
end

Set the my_app_dir.rb recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_dir'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

Run all tests

chef exec rake

Install Python

Add an integration test

touch test/integration/default/serverspec/my_app_python_spec.rb

Add this code to my_app_python_spec.rb:

require 'spec_helper'

describe 'my_flask_server::my_app_python' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  # verify system python
  describe package('python') do
    it { should be_installed }
  end

  # verify system pip
  describe command('which pip') do
    its(:stdout) { should contain '/usr/local/bin/pip' }
  end

  # verify system setuptools
  describe command('which easy_install') do
    its(:stdout) { should contain '/usr/local/bin/easy_install' }
  end

  # verify system virtualenv
  describe command('which virtualenv') do
    its(:stdout) { should contain '/usr/local/bin/virtualenv' }
  end

  # verify my_flask_app virtual environment
  # See `helpers/serverspec/type/virtualenv.rb`
  describe virtualenv('/var/www/my_flask_app/shared/.env2') do
    it { should be_virtualenv }
  end
end

Create virtualenv serverspec type

mkdir test/integration/helpers/serverspec/type
touch test/integration/helpers/serverspec/type/virtualenv.rb

Paste this code into virtualenv.rb:

##############################################################################
# You can find the original code at:
# <https://github.com/jantman/serverspec-extended-types>
# Licensed under the MIT License
##############################################################################

# Serverspec
module Serverspec
  # Type
  module Type
    # Virtualenv
    class Virtualenv < Base
      # Test whether this appears to be a working venv
      #
      # Tests performed:
      # - venv_path/bin/pip executable by owner?
      # - venv_path/bin/python executable by owner?
      # - venv_path/bin/activate readable by owner?
      # - 'export VIRTUAL_ENV' in venv_path/bin/activate?
      #
      # @example
      #   describe virtualenv('/path/to/venv') do
      #     it { should be_virtualenv }
      #   end
      #
      # @api public
      # @return [Boolean]
      def virtualenv?
        pip_path = ::File.join(@name, 'bin', 'pip')
        python_path = ::File.join(@name, 'bin', 'python')
        act_path = ::File.join(@name, 'bin', 'activate')
        cmd = "grep -q 'export VIRTUAL_ENV' #{act_path}"

        @runner.check_file_is_executable(pip_path, 'owner') &&
          @runner.check_file_is_executable(python_path, 'owner') &&
          @runner.check_file_is_readable(act_path, 'owner') &&
          @runner.run_command(cmd).exit_status.to_i == 0
      end
    end

    # Serverspec Type wrapper method for Serverspec::Type::Virtualenv
    #
    # @example
    #   describe virtualenv('/path/to/venv') do
    #     # tests here
    #   end
    #
    # @param name [String] the absolute path to the virtualenv root
    #
    # @api public
    # @return {Serverspec::Type::Virtualenv}
    def virtualenv(name)
      Virtualenv.new(name)
    end
  end
end

include Serverspec::Type

Add this line to test/integration/helpers/serverspec/spec_helper.rb:

require 'type/virtualenv'

Reference the poise-python cookbook

Add this line to metadata.rb:

depends 'poise-python', '~> 1.2.1'

To get the latest version string, run knife cookbook site show nginx:

chef exec knife cookbook site show poise-python | grep latest_version
latest_version:     https://supermarket.chef.io/api/v1/cookbooks/poise-python/versions/1.2.1

Here is the complete file:

name 'my_flask_server'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures my_flask_server'
long_description 'Installs/Configures my_flask_server'
version '0.1.0'

depends 'apt', '~> 2.9.2'
depends 'poise-python', '~> 1.2.1'

Write the my_app_python recipe

The first step is to create the recipe file, my_app_python.rb. Run the following command to generate it:

chef generate recipe my_app_python
rm spec/unit/recipes/my_app_python_spec.rb
Write out recipes/my_app_python.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_python
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

# See https://supermarket.chef.io/cookbooks/poise-python

# install python 2
python_runtime 'python_2' do
  version '2'
end

# create my_flask_app virtual environment
python_virtualenv 'my_flask_app_env' do
  path '/var/www/my_flask_app/shared/.env2'
  python 'python_2'
  user 'www-data'
  group 'www-data'
end

Set the my_app_python recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_python'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

include_recipe 'my_flask_server::my_app_python'

Run all tests

chef exec rake

Install SSH Key

Update .kitchen.yml

Add this code to .kitchen.yml under provisioner:

    data_bags_path: "./data_bags"
    encrypted_data_bag_secret_key_path: <%= ENV['CHEF_DATA_BAG_SECRET'] %>

Here is the complete file:

---
driver:
  name: vagrant
  require_chef_omnibus: 12.5.1
  network:
    - ["private_network", {type: "dhcp"}]
  vagrantfiles:
    - .kitchen.vagrant_cachier.rb
    - .kitchen.vagrant_hostmanager.rb

provisioner:
  name: chef_zero
  data_bags_path: "./data_bags"
  encrypted_data_bag_secret_key_path: <%= ENV['CHEF_DATA_BAG_SECRET'] %>

platforms:
  - name: ubuntu-14.04

suites:
  - name: default
    run_list:
      - recipe[my_flask_server::default]
    attributes:

Generate a Data Bag Secret

mkdir ~/.chef
$key = New-Object byte[](512)
$rng = [System.Security.Cryptography.RNGCryptoServiceProvider]::Create().GetBytes($key)
[Convert]::ToBase64String($key) | Out-File "~/.chef/chef_data_bag_secret" -encoding "UTF8"
[array]::Clear($key, 0, $key.Length)

This key will be used to encrypt sensitive cookbook information like SSH private keys. Make sure you store a copy of the ~/.chef/chef_data_bag_secret file in a secure location.

Create Environment Variable
$chef_secret = (rvpa '~/.chef/chef_data_bag_secret').Path
$env:CHEF_DATA_BAG_SECRET = $chef_secret
[Environment]::SetEnvironmentVariable("CHEF_DATA_BAG_SECRET", $chef_secret, "User")

Generate SSH Key

mkdir ~/.ssh

Make sure you use an empty passphrase.

pushd
cd ~/.ssh
ssh-keygen -f id_my_flask_app_deploy
popd 

If you have an existing key with a passphrase, remove the passphrase from a private key using code similar to:

ssh-keygen -p -P 'PASSPHRASE' -N '' -f id_deploy
Add the SSH key to Bitbucket

We will use Bitbucket to host the source code of our test application. Go ahead and fork the my_flask_app repository.

Then add the public part of the SSH key, ~/.ssh/id_my_flask_app_deploy.pub, to your Bitbucket fork. See Use deployment keys for more details.

Create Data Bag Item

$json = @"
{
    "id" : "my_flask_app",
    "deploy_key" : "<key>"
}
"@
$json = $json -replace "<key>", ([io.file]::ReadAllText(".ssh/id_my_flask_app_deploy").Replace("`n", "\n") + "\n")
[io.file]::WriteAllText(".chef/my_flask_app.json", $json)

Encrypt Data Bag Item

mkdir ./data_bags/secrets

chef exec knife data bag from file secrets ~/.chef/my_flask_app.json -z --secret-file (rvpa ~/.chef/chef_data_bag_secret).Path

Create SSH Wrapper Script

Create files/default/wrap-ssh4git.sh file:

chef generate file wrap-ssh-4-git.sh

Add this content to the files/default/wrap-ssh-4-git.sh:

#!/bin/bash
ssh -o "StrictHostKeyChecking=no" -i "/tmp/my_flask_app/ssh/id_my_flask_app_deploy" $1 $2

IMPORTANT: Make sure the file uses Unix Line Ending (\n, LF) and UTF-8 encoding. Otherwise you may get weird errors when Chef client runs the script during deployment.

Add an integration test

touch test/integration/default/serverspec/my_app_ssh_spec.rb

Add this code to my_app_ssh_spec.rb:

require 'spec_helper'

describe 'my_flask_server::my_app_ssh' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  describe file('/tmp/my_flask_app/ssh/wrap-ssh-4-git.sh') do
    it { should be_file }
    it { should be_mode '755' }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end

  describe file('/tmp/my_flask_app/ssh/id_my_flask_app_deploy') do
    it { should be_file }
    it { should be_mode '600' }
    it { should be_owned_by 'www-data' }
    it { should be_grouped_into 'www-data' }
  end
end

Write the my_app_ssh recipe

The first step is to create the recipe file, my_app_ssh.rb. Run the following command to generate it:

chef generate recipe my_app_ssh
rm spec/unit/recipes/my_app_ssh_spec.rb

Write out recipes/my_app_ssh.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_app_ssh
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

directory '/tmp/my_flask_app/ssh' do
  recursive true
  user 'www-data'
  group 'www-data'
end

# copy git ssh wrapper
cookbook_file '/tmp/my_flask_app/ssh/wrap-ssh-4-git.sh' do
  source 'wrap-ssh-4-git.sh'
  mode 0755
  user 'www-data'
  group 'www-data'
end

# decrypt the private ssh key
my_flask_app = Chef::EncryptedDataBagItem.load('secrets', 'my_flask_app')

if my_flask_app['deploy_key']
  ruby_block 'decrypt `id_my_flask_app_deploy` private ssh key' do
    block do
      f = ::File.open('/tmp/my_flask_app/ssh/id_my_flask_app_deploy', 'w')
      f.print(my_flask_app['deploy_key'])
      f.close
    end

    not_if do
      ::File.exist?('/tmp/my_flask_app/ssh/id_my_flask_app_deploy')
    end
  end

  # change permissions
  file '/tmp/my_flask_app/ssh/id_my_flask_app_deploy' do
    mode 0600
    user 'www-data'
    group 'www-data'
  end
end

Set the my_app_ssh recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_ssh'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

include_recipe 'my_flask_server::my_app_python'

include_recipe 'my_flask_server::my_app_ssh'

Run all tests

chef exec rake

Deploy Application From Git

The test application is available in the my_flask_app repository on Bitbucket. Make sure you have forked it, and configured your SSH key, as explained earlier in this tutorial.

Reference the git cookbook

Add this line to metadata.rb:

depends 'git', '~> 4.3.6'

To get the latest version string, run knife cookbook site show git:

chef exec knife cookbook site show git | grep latest_version
latest_version:     https://supermarket.chef.io/api/v1/cookbooks/git/versions/4.3.6

Here is the complete file:

name 'my_flask_server'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures my_flask_server'
long_description 'Installs/Configures my_flask_server'
version '0.1.0'

depends 'apt', '~> 2.9.2'
depends 'poise-python', '~> 1.2.1'
depends 'git', '~> 4.3.6'

Add an integration test

touch test/integration/default/serverspec/my_app_deploy_spec.rb

Add this code to my_app_deploy_spec.rb:

require 'spec_helper'

describe 'my_flask_server::my_app_deploy' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  # verify git deployment
  describe file('/var/www/my_flask_app/current') do
    it { should be_symlink }
  end

  describe file('/var/www/my_flask_app/current/app.py') do
    it { should exist }
  end

  # verify requirements.txt has been processed
  # i.e. local python packages like `Flask` have been installed
  describe command('/var/www/my_flask_app/shared/.env2/bin/pip list') do
    its(:stdout) { should contain 'Flask' }
    its(:exit_status) { should eq 0 }
  end
end

Write the my_app_deploy recipe

The first step is to create the recipe file, my_app_deploy.rb. Run the following command to generate it:

chef generate recipe my_app_deploy
rm spec/unit/recipes/my_app_deploy_spec.rb

Write out recipes/my_app_deploy.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_app_deploy
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'git::default'

# checkout app from git repo / master branch
# creates `releases`, and `current` directories
# links `current` to latest revision
deploy_revision '/var/www/my_flask_app' do
  action :deploy

  repo 'ssh://git@bitbucket.org/vkantchev/my_flask_app.git'
  branch 'master'

  user 'www-data'
  group 'www-data'

  ssh_wrapper '/tmp/my_flask_app/ssh/wrap-ssh-4-git.sh'

  purge_before_symlink []
  create_dirs_before_symlink []

  symlinks({})

  migrate false
  symlink_before_migrate({})
end

# install python packages in `my_flask_app_env` via pip
# `my_flask_app_env` is defined in the `my_app_python.rb`
pip_requirements '/var/www/my_flask_app/current/requirements.txt' do
  virtualenv 'my_flask_app_env'
  user 'www-data'
  group 'www-data'
  action :install
end

Set the my_app_deploy recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_deploy'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

include_recipe 'my_flask_server::my_app_python'

include_recipe 'my_flask_server::my_app_ssh'
include_recipe 'my_flask_server::my_app_deploy'

Run all tests

chef exec rake

Install uWSGI

Add an integration test

Create test/integration/default/serverspec/my_app_uwsgi_spec.rb

Add this code to my_app_uwsgi_spec.rb:

require 'spec_helper'

describe 'my_flask_server::my_app_uwsgi' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  # verify uWSGI package
  describe command('/var/www/my_flask_app/shared/.env2/bin/pip list') do
    its(:stdout) { should contain 'uWSGI' }
    its(:exit_status) { should eq 0 }
  end

  # verify my_flask_app uWSGI serice
  describe service('my_flask_app') do
    it { should be_enabled }
    it { should be_running }
  end
end

Create uWSGI Python Config File

Create files/default/uwsgi/my_flask_app.ini file:

mkdir files/default/uwsgi
chef generate file uwsgi/my_flask_app.ini

Add this content to the files/default/uwsgi/my_flask_app.ini:

# See:
# http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html
# http://uwsgi-docs.readthedocs.org/en/latest/Upstart.html
# http://uwsgi-docs.readthedocs.org/en/latest/Options.html#plugin-python
[uwsgi]

# socket configuration
socket = /var/www/my_flask_app/shared/.uwsgi/my_flask_app.sock
chmod-socket = 664
vacuum = true

# process configuration
master = true
processes = 2
threads = 4
die-on-term = true

# app configuration

# virtual environment
virtualenv = /var/www/my_flask_app/shared/.env2

# call the app instance from the app.py module
chdir = /var/www/my_flask_app/current
module = app
callable = app

IMPORTANT: Make sure the file uses Unix Line Ending (\n, LF) and UTF-8 encoding. Otherwise you may get weird errors when Chef client runs the script during deployment.

Create uWSGI Service Config File

Create files/default/uwsgi/my_flask_app.conf file:

chef generate file uwsgi/my_flask_app.conf

Add this content to the files/default/uwsgi/my_flask_app.conf:

description "uWSGI instance to serve my_flask_app"

start on runlevel [2345]
stop on runlevel [!2345]

setuid www-data
setgid www-data

script
  cd /var/www/my_flask_app/shared
  . .env2/bin/activate
  uwsgi --ini .uwsgi/my_flask_app.ini
end script

IMPORTANT: Make sure the file uses Unix Line Ending (\n, LF) and UTF-8 encoding. Otherwise you may get weird errors when Chef client runs the script during deployment.

Write the my_app_uwsgi recipe

The first step is to create the recipe file, my_app_uwsgi.rb. Run the following command to generate it:

chef generate recipe my_app_uwsgi
rm spec/unit/recipes/my_app_uwsgi_spec.rb

Write out recipes/my_app_uwsgi.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_app_uwsgi
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

# Install the uwsgi package in virtual environment using pip
# See https://supermarket.chef.io/cookbooks/poise-python

# install uwsgi python package
python_package 'uwsgi' do
  virtualenv 'my_flask_app_env'
  user 'www-data'
  group 'www-data'
end

# my_flask_app uWSGI configuration
cookbook_file '/var/www/my_flask_app/shared/.uwsgi/my_flask_app.ini' do
  source 'uwsgi/my_flask_app.ini'
  owner 'root'
  group 'root'
  mode 0644
end

# create my_flask_app uWSGI service
cookbook_file '/etc/init/my_flask_app.conf' do
  source 'uwsgi/my_flask_app.conf'
  owner 'root'
  group 'root'
  mode 0644
end

# start my_flask_app uWSGI service
service 'my_flask_app' do
  provider Chef::Provider::Service::Upstart
  supports status: true, restart: true, reload: true
  action [:enable, :start]
end

Set the my_app_uwsgi recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_uwsgi'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

include_recipe 'my_flask_server::my_app_python'

include_recipe 'my_flask_server::my_app_ssh'
include_recipe 'my_flask_server::my_app_deploy'

include_recipe 'my_flask_server::my_app_uwsgi'

Run all tests

chef exec rake

Install Nginx

Add an integration test

Rename test/integration/default/serverspec/default_spec.rb to test/integration/default/serverspec/my_app_nginx_spec.rb

Replace the contents of my_app_nginx_spec.rb with this code:

require 'spec_helper'

describe 'my_flask_server::my_app_nginx' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html

  # verify nginx package
  describe package('nginx') do
    it { should be_installed }
  end

  # verify nginx serice
  describe service('nginx') do
    it { should be_enabled }
    it { should be_running }
  end
end

Create Nginx Site File

Create files/default/nginx/my_flask_app file:

mkdir files/default/nginx
chef generate file nginx/my_flask_app

Add this content to the files/default/nginx/my_flask_app:

server {
  listen 80 default_server;
  listen [::]:80 default_server ipv6only=on;

  # Make site accessible from http://localhost/
  server_name localhost;

  location / {
    include uwsgi_params;
    uwsgi_pass unix:/var/www/my_flask_app/shared/.uwsgi/my_flask_app.sock;
  }
}

IMPORTANT: Make sure the file uses Unix Line Ending (\n, LF) and UTF-8 encoding. Otherwise you may get weird errors when Chef client runs the script during deployment.

Write the my_app_nginx recipe

The first step is to create the recipe file, my_app_nginx.rb. Run the following command to generate it:

chef generate recipe my_app_nginx
rm spec/unit/recipes/my_app_nginx_spec.rb

Write out recipes/my_app_nginx.rb like this:

#
# Cookbook Name:: my_flask_server
# Recipe:: my_nginx
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

# install nginx
package 'nginx' do
  :upgrade
end

# start nginx service
service 'nginx' do
  supports status: true, restart: true, reload: true
  action [:enable, :start]
end

# Uncomment to use a custom nginx.conf
# cookbook_file '/etc/nginx/nginx.conf' do
#   source 'nginx/nginx.conf'
#   mode 0640
#   owner 'root'
#   group 'root'
#   notifies :restart, 'service[nginx]'
# end

# disable default site
link '/etc/nginx/sites-enabled/default' do
  action :delete
  only_if 'test -L /etc/nginx/sites-enabled/default'
  notifies :restart, 'service[nginx]'
end

# add my_flask_app site
cookbook_file '/etc/nginx/sites-available/my_flask_app' do
  source 'nginx/my_flask_app'
  mode 0640
  owner 'root'
  group 'root'
end

# enable my_flask_app site
link '/etc/nginx/sites-enabled/my_flask_app' do
  to '/etc/nginx/sites-available/my_flask_app'
  notifies :restart, 'service[nginx]'
end

Set the my_app_nginx recipe to run

Add this line to recipes/default.rb:

include_recipe 'my_flask_server::my_app_nginx'

Here is the complete file:

#
# Cookbook Name:: my_flask_server
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

include_recipe 'my_flask_server::my_app_dir'

include_recipe 'my_flask_server::my_app_python'

include_recipe 'my_flask_server::my_app_ssh'
include_recipe 'my_flask_server::my_app_deploy'

include_recipe 'my_flask_server::my_app_uwsgi'

include_recipe 'my_flask_server::my_app_nginx'

Run all tests

chef exec rake

Open a browser and navigate to http://default-ubuntu-1404/. You should see the "Hello World!" page of the my_flask application.

Getting Started with FlashDevelop on Windows 10

FlashDevelop is a free IDE for Windows. You can use it to develop Adobe Flash and AIR applications using ActionScript.

Install Chocolatey

Chcocolatey is a package manager for Windows.

Open PowerShell as Administrator and run the following command:

iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))

Install Java 8 SDK 32-bit

FlashDevelop requires a 32-bit java on the system path.

In PowerShell as Administrator:

choco install jdk8 -params "x64=false"

Install FlashDevelop

Download the latest FlashDevelop setup package from www.flashdevelop.org and install it, FlashDevelop-5.0.2.exe at the time of writing.

Install Additional Software

  1. Start FlashDevelop
  2. Install SDKs and other tools using AppMan, from Tools -> Install Software ...
    • Flex SDK + AIR SDK
    • Flash Player (SA)
  3. Restart FlashDevelop, so that it can detect the installed SDKs and tools properly.

Create New Project

  1. Create a new project from: Project -> New Project
  2. For project type choose AS3 Project
  3. Set a breakpoint in Main method - Ctrl+Shift+B
  4. Compile - F11
  5. Debug - F5

How To Install Nginx On Ubuntu Using Chef

In this post we show how to install NGINX on Ubuntu using Chef on Windows 10 workstation.

All commands are executed in PowerShell on a Windows workstation.

Before You Begin

Install Chocolatey

If you do not have Chocolatey, you can install it by following the instructions on chocolatey.org. You may also see this post for instructions on how to set the Chocolatey cache location.

Install Chef Development Kit (ChefDK)

In PowerShell as Administrator:

choco install chefdk

Install VirtualBox and Vagrant

To install VirtualBox and Vagrant, follow the steps described in Ubuntu with Vagrant and VirtualBox on Windows.

Install Vagrant Plugins

vagrant plugin install vagrant-omnibus
vagrant plugin install vagrant-berkshelf
vagrant plugin install vagrant-hostmanager

Create Chef Cookbook

This directory will become the root of your source repository:

chef generate cookbook my_ubuntu_nginx
cd my_ubuntu_nginx

Ensure the apt cache is up to date

Reference the apt Cookbook

Add this line to metadata.rb:

depends 'apt', '~> 2.9.2'

To get the latest version string, run knife cookbook site show apt:

chef exec knife cookbook site show apt | grep latest_version
latest_version:     https://supermarket.chef.io/api/v1/cookbooks/apt/versions/2.9.2

Here is the complete file:

name 'my_ubuntu_nginx'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures my_ubuntu_nginx'
long_description 'Installs/Configures my_ubuntu_nginx'
version '0.1.0'

depends 'apt', '~> 2.9.2'

Set the apt cookbook's default recipe to run

Add this line to recipes/default.rb:

include_recipe 'apt::default'

Here is the complete file:

#
# Cookbook Name:: my_ubuntu_nginx
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'

Test

chef exec berks install

This generates Berksfile.lock file.

Configure NGINX

Reference the nginx cookbook

We need the nginx cookbook for it's nginx::repo recipe. Add this line to metadata.rb:

depends 'nginx', '~> 2.7.6'

To get the latest version string, run knife cookbook site show nginx:

chef exec knife cookbook site show nginx | grep latest_version
latest_version:     https://supermarket.chef.io/api/v1/cookbooks/nginx/versions/2.7.6

Here is the complete file:

name 'my_ubuntu_nginx'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures my_ubuntu_nginx'
long_description 'Installs/Configures my_ubuntu_nginx'
version '0.1.0'

depends 'apt', '~> 2.9.2'
depends 'nginx', '~> 2.7.6'

Write the install_nginx recipe

The first step is to create the recipe file, install_nginx.rb. Run the following command to generate it:

chef generate recipe install_nginx

Write out recipes/install_nginx.rb like this:

#
# Cookbook Name:: my_ubuntu_nginx
# Recipe:: install_nginx
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

# Only needed if you want to install latest stable package from nginx.org
include_recipe 'nginx::repo'

package 'nginx' do
  action :install
end

service 'nginx' do
  supports status: true, restart: true, reload: true
  action :enable
end

Set the install_nginx recipe to run

Add this line to cookbooks/my_web_server/recipes/default.rb:

include_recipe 'my_ubuntu_nginx::install_nginx'

Here is the complete file:

#
# Cookbook Name:: my_ubuntu_nginx
# Recipe:: default
#
# Copyright (c) 2016 The Authors, All Rights Reserved.

include_recipe 'apt::default'
include_recipe 'my_ubuntu_nginx::install_nginx'

Update the integration test

Replace the contents of test/integration/default/serverspec/default_spec.rb with this code:

require 'spec_helper'

describe 'my_ubuntu_nginx::default' do
  # Serverspec examples can be found at
  # http://serverspec.org/resource_types.html
  describe package('nginx') do
    it { should be_installed }
  end

  describe service('nginx') do
    it { should be_enabled }
    it { should be_running }
  end
end

Test

Ruby Lint

chef exec rubocop

Chef Lint

chef exec foodcritic .

Unit Tests

Install / Update cookbooks:

chef exec berks install

Create a .rspec file:

touch .rspec

Paste these lines in .rspec

--color
--format documentation

Run:

chef exec rspec

Integration Tests

Update .kitchen.yml to include the Ubuntu platform only. This is how it should look:

---
driver:
  name: vagrant
  network:
    - ["private_network", {type: "dhcp"}]

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-14.04

suites:
  - name: default
    run_list:
      - recipe[my_ubuntu_nginx::default]

Run the integration tests:

chef exec kitchen test --destroy=never

How to use Berkshelf, Chef Zero, Vagrant and VirtualBox

In this post we show how you can use Berkshelf, Chef Zero, Vagrant, and VirtualBox to provision Ubuntu 14.04 guest on Windows 10 host.

Before You Begin

You need VirtualBox and Vagrant installed. To do that, you can follow the steps described in Ubuntu with Vagrant and VirtualBox on Windows.

Install Vagrant-Omnibus Plugin

vagrant plugin install vagrant-omnibus

Install Vagrant-Berkshelf Plugin

vagrant plugin install vagrant-berkshelf

Install Chef Development Kit (ChefDK)

In PowerShell as Administrator:

choco install chefdk

Install Knife Solo

At this time we only need knife solo to generate an empty Chef repository that is compatible with Chef Zero. For more information see Knife Solo.

chef gem install knife-solo

Create Chef Repository

Create a chef-repo directory

mkdir ubuntu-chef-repo
cd ubuntu-chef-repo

Setup the the chef-repo directory

touch Berksfile
chef exec knife solo init . --no-git

Paste this into the Berksfile:

source 'https://supermarket.chef.io'

cookbook 'main', path: './site-cookbooks/main'

Create New Cookbook

cd site-cookbooks
chef exec berks cookbook main --skip-git --skip-test-kitchen --no-foodcritic --no-chef-minitest --no-bundler --skip-vagrant

Check the structure of the main cookbook:

tree /F main

It should have the following structure:

│   Berksfile
│   CHANGELOG.md
│   chefignore
│   LICENSE
│   metadata.rb
│   README.md
│   Thorfile
├───attributes
├───files
│   └───default
├───libraries
├───providers
├───recipes
│       default.rb
├───resources
└───templates
    └───default

Add the Apt Cookbook as a Dependency

The apt cookbook runs the apt-get update command on the guest system. See apt Cookbook - Chef Supermarket for details.

Open the site-cookbooks/main/metadata.rb file and add the following code at the end (the version may be different than 2.9.2):

depends 'apt', '~> 2.9.2'

This is how the final file should look like:

name             'main'
maintainer       'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures main'
long_description 'Installs/Configures main'

version          '0.1.0'

depends 'apt', '~> 2.9.2'

Include the Apt Cookbook in the Default Recipe

Open site-cookbooks/main/recipes/default.rb and add the following code at the end:

include_recipe 'apt::default'

The final file looks like this:

#
# Cookbook Name:: main
# Recipe:: default
#
# Copyright (C) 2016 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'apt::default'

Test

This will generate Berksfile.lock

berks install

Create Vagrantfile

Go to the Chef Repo root (ubuntu-chef-repo).

touch Vagrantfile

Open Vagrantfile and replace its content with the following one:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://atlas.hashicorp.com/search.
  #
  config.vm.box = "ubuntu/trusty64"

  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  #
  config.vm.provider "virtualbox" do |vb|
    # For a complete reference, please see the online documentation at
    # https://docs.vagrantup.com/v2/virtualbox/configuration.html

    # Name used in Oracle VM VirtualBox Manager GUI
    vb.name = "berkshelf-ubuntu-kitchen"

    # Customize the amount of memory on the VM (in MB):
    vb.memory = "2048"

    # Customize the amount of video memory on the VM (in MB):
    vb.customize ["modifyvm", :id, "--vram", "128"]
  end

  # Install the latest version of Chef.
  # For more information see https://github.com/chef/vagrant-omnibus
  #
  config.omnibus.chef_version = :latest

  # Enabling the Berkshelf plugin.
  config.berkshelf.enabled = true

  # Provision with Chef Zero
  #
  config.vm.provision :chef_zero do |chef|
    # Specify the local paths where Chef data is stored
    chef.cookbooks_path = [ 'cookbooks', 'site-cookbooks' ]
    chef.data_bags_path = "data_bags"
    chef.nodes_path = "nodes"
    chef.roles_path = "roles"

    # Add a recipe
    chef.add_recipe "main::default"
  end
end

Test

vagrant up
vagrant provision

Provision Chef with Vagrant-Omnibus Plugin

In this post we show how you can provision Chef from Vagrant using the vagrant-omnibus plugin. This post is for Ubuntu 14.04 guest running in VirtualBox on Windows 10 host.

Prerequisites

You need VirtualBox and Vagrant installed. To do that, you can follow the steps described in Ubuntu with Vagrant and VirtualBox on Windows.

Install Chef Development Kit (ChefDK)

In PowerShell as Administrator:

choco install chefdk

Provision Chef

Before you can use Chef Solo, Chef Zero, or Chef Client for provisioning, you need to install the latest Chef client on the guest. This can be done with a shell script or with a Vagrant plugin called vagrant-omnibus.

In PowerShell:

vagrant plugin install vagrant-omnibus

Open Vagrantfile and add the following code at the end of the configuration:

# Install the latest version of Chef.
# For more information see https://github.com/chef/vagrant-omnibus
#
config.omnibus.chef_version = :latest

The complete Vagrantfile file should look like this:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
#
Vagrant.configure(2) do |config|

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://atlas.hashicorp.com/search.
  #
  config.vm.box = "ubuntu/trusty64"

  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  #
  config.vm.provider "virtualbox" do |vb|
    # For a complete reference, please see the online documentation at
    # https://docs.vagrantup.com/v2/virtualbox/configuration.html

    # Name used in Oracle VM VirtualBox Manager GUI
    vb.name = "Ubuntu-x64-Vagrant"

    # Customize the amount of memory on the VM (in MB):
    vb.memory = "2048"

    # Customize the amount of video memory on the VM (in MB):
    vb.customize ["modifyvm", :id, "--vram", "128"]
  end

  # Install the latest version of Chef.
  # For more information see https://github.com/chef/vagrant-omnibus
  #
  config.omnibus.chef_version = :latest
end

Test

Finally launch and provision the Vagrant box. It should install the latest Chef Client:

vagrant reload
vagrant provision