Django with Azure Webjob.. it works!!

Azure Webjob is another great feature from Azure WebApp which provides cost effectiveness for running background job. With all the great features, and lots of buzz about open source supports in other languages like Java or python. I was let down by the available documentations which can properly guide a newbie like me to use the Webjob properly.

Architecture wise, Webjob and WebApp are running on the same logical machine. The Django Codes that I have deployed in production are available in "wwwroot", the root of IIS. On the other hand, Webjob is located in a different folder at the home directory, "data/jobs". This is clearly to show that the Webjob is not running on the IIS. Hence it will not cost any performance on the IIS.

Back to executing Webjob with python, I have been doing a lot of research and trying to ask a lot of resources from MS. Most of the answer that I received are similar, "Webjob and WebApp are sharing the same server". None of the answers were actually showing me the example codes in python. However, I solved the puzzle from different parts of the blogs.

Here are solutions that I have managed to work on and put in run.py for the Webjob.

1. Instead of creating/install a new set of packages via PIP (webjob does not support PIP), we will refer to the existing packages that were installed via PIP in virtual environment:

import sys

sitepackage = "D:\home\site\wwwroot\env\Lib\site-packages"
sys.path.append(sitepackage)

2. With the similar approach at step one, we may need to call functions/models that are defined in different django app as part of the scheduled task in Webjob. Hence, the following are the reference of the django project path:

myproject = "D:\home\site\wwwroot"
sys.path.append(myproject)

3. We still need to refer to django settings.py to make sure that we do not need to replicate the settings that are already defined. With the research of stackoverflow, I managed to find a code that help me to solve the problem:

from optparse import OptionParser

usage = "usage: %prog -s SETTINGS | --settings=SETTINGS"
parser = OptionParser(usage)
parser.add_option('-s', '--settings', dest='settings', metavar='SETTINGS', help="The Django settings module to use", default='DjangoProject.settings')
(options, args) = parser.parse_args()

if not options.settings:
parser.error("You must specgfy a settings module")
os.environ['DJANGO_SETTINGS_MODULE']=options.settings

4. Executing the tasks that are designed for the relevant jobs as usual python code.

Voala! Your webjob is ready to be executed. Remember, you need to zip up the run.py before you upload it. Now, you can run your webjob by reusing the codes in your original django project without replicating them,

Please leave comments for this blog.

Solving CORS error with Azure CDN

There're a lot of blogs and information about CORS error on Azure CDN. Lot's of solutions had been provided but none of them works for me. After a lot of trials and errors, I have finally found the solution. It's the sequence of defining CORS rules to Azure Blog storage and also loading the static files to the blog storage. I found this link, an easy to use Azure CORS rules manager to manage my blog account without writing any code.

The make life easier for other developers who may move from other platform, here are the simple steps to setup Azure CDN without getting CORS error.

1. Create Azure Blog Storage account and a container. Remember to update this container's property to allow Public Access.
2. *Important: Before upload any other static files to the Azure Blog storage, download the Visual Studio solution package for Azure CORS Rules Manager from the link above. Set the rules according to the picture.
3. Once, the rules are defined, you can start to upload the static files to the container that you have created.
4. Create CDN on Azure portal and map to the storage that you have created on the steps above.
5. Now, you can define the customheaders and other rules for the CDN in web.config. You can find the example from the blog. 

After a few minutes/an hour depends on your rules defined in the CORS, your static files will be available and display accordingly.

Please refer to this article: Why and how we migrated babylonjs.com to Microsoft Azure for more live example on how to make use Azure's CDN and storage.

Deploying psycopg2 to Azure Webapp

Another challenge with Azure-Django-Postgres deployment is to install psycopg2. Azure is running on windows and IIS, hence any existing psycopg2 installation with PIP will not work. As the solution of this, I removed the psycopg2 from the requirements.txt and used easy_install to install the driver from http://www.stickpeople.com/projects/python/win-psycopg/

I added the following to the deployment script: deploy.cmd

:: Addition psycopg2 for windows
echo Pip install git+https://github.com/nwcell/psycopg2-windows.git@win32-py27#egg=psycopg2
env\scripts\easy_install http://www.stickpeople.com/projects/python/win-psycopg/2.6.0/psycopg2-2.6.0.win32-py2.7-pg9.4.1-release.exe

IF !ERRORLEVEL! NEQ 0 goto error

Hope, this will help you to kick start the Azure for your existing this Django migration.

Django deployment headache with Azure Webapp

Cost is always the key factor in my decision making. Through the research, I found out that Microsoft Azure is offering some compiling package for startup compare to other big company. It comes with 3 years of $150 worth of Azure credit each month and also with the benefits of accessing other MS products. As for the startup, who just want to start to launch a business, I believe this can be quite cost effective compare of 12 months free-tier with Amazon AWS and also soon to be ended free server access with Heroku.

However, I will advise to start building on some easy to use server like Heroku, it's way too easy to deploy the code into that server within a day.

After a month of effort in activating the MSDN and Azure account due to my profile into the system. Hooray, but wait... how am I going to deploy the code? With Heroku, there's a very clear instruction in deploying the code for python. However, the article in Azure does not give much detailed on the steps to deploy a workable Django/python WebApp. I have been spending about a week to find out a way to deploy my existing django project to Azure environment. 

In this article, I will highlight the steps for Django deployment. The following is a simple architecture design that I am currently working on my Django project. 

1. Azure Webapp - for the website (front end)

2. Azure VM - for the postgresql and elasticsearch server (back end)

First challenge that I am facing.. how can VM and Webapp communicate and VM is not accessible by public. Without any gurus or experts with me as I am bootstrapping, I need to do the work myself. After some research, with different keywords search, I found a solution.

Azure Virtual Network with point-to-site-connectivity!!

Important: The virtual network should be created before start creating the VM. You can follow this link for the instructions to create virtual network.

Now, it's time to create the VM, I am using ubuntu server VM for postgresql and elasticsearch servers. I will not elaborate much in this blog, as the instructions is quite straightforward.

2nd challenge, no simple but detailed instructions in properly deploy a django project into Azure environment. Through some trial and error as well as getting clue from the instructions, I have finally managed to deploy the django project into Azure server successfully but it's taking too much time to do that. Hence, I decide to share the knowledge with others who may encounter the same issue.

1. Read this link to give you an overview on creating, and deploying Django to Azure Webapp.
2. Azure offers different management sites, like http://manage.windowsazure.com and http://portal.azure.com
2. I am using manage.windowsazure.com. Create a new webapp using custom gallery.
3. Follow the instructions and continue with the steps show on the screen, for example:

4. Once you have completed Azure will deploy the source code automatically.
5. However, this is not ready. You will be getting server error as due to missing files such as web.config, which is used by IIS of Azure.
6. Install Azure CLI to any environment you have. You can follow the instructions in Azure for the installation and understand the usage. Make sure you are connected to azure site with your command line.
7. Generate a custom deployment script with Azure CLI. With the following line on the root of your repository.
     azure site deploymentscript
8. A deployment script file will be generated with name, deploy.cmd. Edit the file to include the syncdb, and other relevant deployment steps that you normally performed with python manage.py command. *important: Please remember that you need to put --noinput at the end of syncdb as Azure deployment script will hang when an interaction is needed.
9. Once you have completed all the necessary changes for the settings.py and deploy.cmd. Check in you changes to the git repository, Azure will automatically pickup the changes and run the deployment.
10. Remember, you need to have web.config in the git too.

Voila. The deployment now should have include the steps to run syncdb, loaddata, createsuperuser or execute easy_install if you need to install packages specific for windows environment as Azure webapp is running on Windows.

Send me some comments if you have any questions or suggestions.