The main issue with writing a site like this is deal with scraping the score page. It changes minorly every year, and the scraping must be changed to meet it. Since this is a real time event, with players hungry to find out their scores, it is a bit hard to fully test and be ready for all the curveballs. Last year the site didn’t work, and I gave up after a couple hours. It is very hard to play the trivia contest, program, and drink all at the same time! Here are the things I think made this year a success and things that could have been done better:

Logging!

I would have had a much harder time diagnosing and fixing the site if I wasn’t using a lot of logging and Sentry. Sentry is a server that you can send all your logs to and easily sort and view the messages via a web page. I was able to notice issues much faster and deal with them much more easily because of Sentry. Also, seeing minor errors popping up is reassuring. It is much more likely I made a small mistake somewhere than I made no mistakes anywhere and there are no error logs being generated. Even just shooting off an info logging message is reassuring.

Test like the event is happening

Your tests should mirror the entire event, not just points in the event. For example, a lot of my testing focused on getting past years data, where all scores were available. The testing should have been in stages, like “only hour 1 posted”, “hours 1 and 2 posted”, “hours 1 and 3 posted, 2 is missing”, etc.

Things are going to go wrong

In recent years, 90FM created a new page with something like “resultsXX.html”, where XX was the hour number. This year, they only had a results.html with all the scores that got changed each hour. That means there will be no archival like past years, which allowed me to get past years if there was a problem. I made an assumption that I could use the XX in the page name to figure out which hour was posted. That was easier than parsing “RESULTS FOR HOUR XX” at the top of the page. Once the first scores got posted, I realized my mistake, and fixed it in a few minutes. I’m very happy it was an easy fix, or I could have lost an hour worth of scores!

You made a mistake somewhere

The assumption above lead me to an edge case that I had properly tested. Because I had to change how the numbering worked, it messed up which hours were getting emailed out as scores got posted. When hour 1 was up, it emailed out hour 1. When hour 2 was up, it still emailed out hour 1. Same for hour 3, which is when I realized and started fixing it. Once you start getting curveballs thrown at you, all those little bugs in your code are going to be much more apparent. Proper testing that really keys in on edge cases will help avoid this, but you never know when something is going to go horribly wrong.

Replay

If you hit an unexpected issue in your code, everything could break. For example, if there was a problem while notifying players that I didn’t catch, all subsequent notifications will be lost. I could go delete all the scores for that hour, re-scrape them, and then hope notifications don’t fail again. However, that isn’t elegant. Having checkpoints in your code where you can replay an event as if it just happened makes life much easier after you fix the issue. A smart way to do this would be saving a small log object to the database, representing a successful notification, or adding a field to each subscriber for “last hour notified”. Another option is adding a command to re-send notifications for a certain team/hour combo. I went for the even easier version: resend all notifications. My thought was that it was more likely that the whole thing would break rather than just sometimes breaking for certain users. I created a Django management command for notifications and a debugging flag to test the notifications before actually sending them. In the end, it worked fairly well. A few people missed the first few hours because they put in the wrong team name, so they got no notification (which I manually corrected).

Another possible replay would have been downloading the page and running the scraper against that. If there was an issue, I’d at least have the page still around. If something had gone totally wrong and I didn’t fix it before the next hour went up, I could have lost those scores forever.

Conclusion

Overall, TriviaStats.com has 477 unique visitors (with 376 total teams) and over 15k page view during the contest. A lot of people were very interested and appreciative. Needless to say, I’m excited to improve it for next year.

If you do unlock your phone, you may face a civil suit from your cell phone provider. It is also possible to face a criminal suit if you unlock phones for commercial gain, such as setting up a company to unlock phones. If you, as a company, do find yourself in a civil or criminal suit, you will find Ms. Allison B. Spinner to be helpful. A civil suit could cost you up to $2500. A criminal suit carries a potential of a $500,000 fine, and possibly prison time.

Some companies have been defending this practice, because they subsidize the cost of expensive cell phones with a slightly higher bill. This is a bad argument at best. The cell phone companies will charge you the same amount whether you are have a subsidized phone or not. This means people who bring their own phone (such as phones that were sold unlocked) are still paying extra for nonexistent subsidies. Also, if you leave the company early, you will be hit with an early termination fee. This fee is to pay back the company for the subsidies you didn’t pay over your contract. If you go past your contract and don’t get a new one, you’ll pay the same fee, meaning you continue paying the subsidy rate, even though you already paid off the phone.

The Electronic Frontier Foundation (EFF) does not agree with the ruling. It is possible they will be challenging the result in court. They have previously challenged many other rulings that do not promote software and hardware freedom, patent disputes, and a whole host of other things.

Even when this ruling takes effect, some manufacturers will sell unlocked phones. However, they will be very expensive and you will still have to pay the subsidy built into most contracts. An unlocked iPhone 5, for example, is $649.

The ruling will be revisited in 2015, hopefully with a more consumer-friendly decision.

To find out more, please contact Ms. Allison B. Spinner .

Profile, Profile, Profile

Your app is slow. But why? Which functions? Is a function getting called too many times? Is one function way too slow? You won’t know until you profile it. In your views.py, you can add a profile decorator that will print out all your queries and a lot of data about functions (how many times they are called, cumulative time taken, time per call, module/line number, etc). Look for the highest cumulative time, ones being called more than the length of your QuerySets, etc. Also check that you aren’t making too many queries to your database. I currently make about 7 after optimization. I could get this down to about 3 with some clever reworking, but none of the calls take very much time (less than 0.001 seconds), so I left them as they are. At one point, it was making over 1000. That is a problem.

To install and use Django Profiler and start profiling views:

pip install django-profiler

# In views.py
from profiling import profile, Profiler

# On top of the view
@profile(stats=True)

If you just want to see your queries, Django gives you an easier way to check them.

from django.db import connection
from django.conf import settings

if settings.DEBUG == True:
    print connection.queries

You can also print the .query attribute of any QuerySet to see what queries created it.

Use Querysets Effectively

You need to understand when Querysets are executed. For example, intersecting QuerySets with the ‘|’ operator does not execute a search, but accessing the values does. If you then add more filters later, you’ll execute another search. Organizing your data flow is hugely important. To get a better understanding, read When QuerySets Are Evaluated and Database Access Optimization in the Django documentation. A big one is using ManyToMany fields properly. If used improperly, you can get thousands of database calls, and horrifying performance. Reconfiguring our database to not require access to ManyToMany fields for normal queries shaved off about 5 seconds on its own. That was kind of shocking.

model_to_dict() or values()?

In my case, I needed all 100 of the Apartment objects to be a dictionary before passing them back to the frontend as JSON. At first, I tried using model_to_dict() on the complete object just before returning them. At first, I was converting all of the objects to dicts with model_to_dict(), but that was causing about a 0.3 second slow down. Even after moving it, I was still getting a slow down of about 1/10 of that, which isn’t unbearable. Instead, I switched to calling .values() at the end of my query.

You need to be careful with this. If you have model functions, they will be unavailable after you run .values(). For Swoop, Apartment objects have a .address() function, combining the various address parts (house number, street, etc) into a standard looking address for display. I moved this function into a utility function, and experienced no more slow downs. Good.

Defer When Possible

Using defer() is great if you have fields that are very large or ManyToMany fields, you can defer them from getting retreived from the database by listing them in defer() at the end of your query. A subtle note is that the Django ORM tries to fill up its QuerySet cache, and part of that is using select_related(). select_related() prevents you from needing to hit the database again if you follow a ForeignKey, but if you have too many objects, this can massively slow down your queries. This accounted for about half of the time.

As a user on reddit pointed out, you can also use only() if you only need a couple columns. Instead of excluding columns listed in defer, it only grabs the ones you specify. In my case, there were about 12 columns I need for every object, so defer made more sense. Your mileage may vary.

Decimals Are Slow

I originally stored the number of beds and baths as a DecimalField in the database. When pulling them out of the database, they have to be converted by the decimal module. According to this StackOverflow article, decimal is about 100 times slower than the float builtin. I changed the fields in models.py to FloatFields, and issued an alter table to the database, “alter table TABLE modify COLUMN double precision;”. If you need Decimals, you can instead try to defer out those fields when not needed. A minor change, but it saved about 0.2 seconds. Nice.

Dates Are Slow

Running a datetime conversion when bringing objects out of a database is slow when run a couple thousand times. Our database keeps track of when apartments are available for rent (sometimes needed for searches), when they were added to the database, and the last time they were updated (to prioritize updates). The added and updated fields never need to be shown to a user, so they are automatically added to the defer list. If the available date isn’t required, it is also deferred. This saved around 0.1 second again. Yay!

Trim The Queryset Before Doing Expensive Operations

Some of our semi-expensive operations, include some map preprocessing and other backend preprocessing, were being applied before we trimmed down the Queryset. Simply moving the trimming to 100 to much earlier saved about 0.1 second. Awesome.

Slower Production vs Dev

We are running on Amazon servers, with our MySQL database on a separate server than our app servers. In development, we use sqlite3 for our database, and it is local. Also, the Amazon servers use slower, using 2007 Opterons, where my laptop has a quad core Ivy Bridge. Obviously, everything is going to act much faster on my laptop, and this initially lulled me into believing the app would perform much better when deployed. This is very hard to account for, but that is why you use development and staging servers that mirror your production environment as close as possible.

Caching

One thing I have not implemented yet is a Memcache server. From what I read, it is great for improving performance, but since Swoop uses a custom weighting system for each search, I don’t know that many queries will be improved. Your app might be different, and Memcache might be the first place to look for performance increases.

I hope these tips help you improve your app. This is not an exhaustive list by any means. If you have more tips, add them in the comments and I’ll add them to the main article. Also, if you have any corrections, please note them in the comments!

First, we need to create SSH keys to save to BitBucket for authentication. You can skip the second line if you already created SSH keys.

sudo su
ssh-keygen -t rsa
# Copy this output to BitBucket as a new SSH key
cat ~/.ssh/id_rsa.pub

Now we need to install git, and create a folder to save all our configurations in. We’ll do this by symlinking the files we’re using to a folder as root.

sudo apt-get install git-core
mkdir ~/git
cd ~/git
git init
# Create an account and repository at BitBucket, sub in your own username/repo name
git remote add origin ssh://git@bitbucket.org/USERNAME/REPO.git

Now that we have the repository set up, let’s add our first file. I’m going to add my NFS exports file. You’ll need to do this for each file. You can commit separately, or symlink all the files then do the commits at the end.

# Repository is set up. Now let's add some stuff and commit it.
mkdir ~/git/etc
# Symlink
ln -s /etc/exports etc/exports
# Add everything
git add -A
# Commit with a commit message
git commit -am "Adding NFS exports"
# Push the master branch to the origin server (BitBucket)
git push origin master

Finally, to help myself remember to commit, I am adding a reminder to my bashrc. Every time I log in as root, I will receive a message if there are uncommited files my git directory.

nano ~/.bashrc
-----
# Add this to the end of the file
# Check git status
cd /root/git
DIRTY=$(git status --porcelain 2>/dev/null| grep "^??" | wc -l)
if [ $DIRTY != 0 ]; then
        echo -e '\E[47;31m'"\033[1mGit changes in /root/git need to be commited.\033[0m"
fi

# Python compiled files
*.py[co1]

# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg

# Installer logs
pip-log.txt

# Unit test / coverage reports
.coverage
.tox

#Translations
*.mo

#Mr Developer
.mr.developer.cfg

# Logging
*output.log

#Kate editor
*kate-swp*

#Ubuntu One (if also stored for syncing)
*u1conflict*

#Mercurial
*/.hg/

#Additional editor
*~
*.save

Post your .gitignore additions in the comments!

sudo iptables -t nat -I PREROUTING -p tcp -d $PUBLICIP --dport 25565 -j DNAT --to 10.0.3.116:25565
sudo iptables -A FORWARD -p tcp -d 10.0.3.116 --dport 25565 -j ACCEPT

I was having trouble getting access to the Internet from containers whenever I enabled UFW on the host machine. I tried a thousand different rules, but none worked. After much digging, I found the solution. UFW defaults to dropping forwarded packets. All my containers interact with the Internet via forwarding over a bridge (lxcbr0 in 12.04), so this is a big problem. Basically, with UFW enabled, the LXC containers could not talk to the internet. To solve the problem:

sudo nano /etc/default/ufw
----
# Change:
# DEFAULT_FORWARD_POLICY="DROP"
# to
DEFAULT_FORWARD_POLICY="ACCEPT"

Once I changed the rules, I just reload UFW, and all was well.

ufw reload

The first Rightscript installs the Amazon EC2 API tools and passes all your credentials into the AMI through RightScale’s platform. You can fill in the inputs on a server by simply selecting the drop down next to them, selecting “Cred” and then changing the next drop down to match the label for that input. For example, match “AWS_ACCESS_KEY_ID” with “AWS_ACCESS_KEY_ID”. This script was originally from the RightScale documentation. This script should be assigned as a boot script to your server template as the 2nd to last script (with the ELB Registration script as the last).

#!/bin/bash
# installs ec2inst script that allows running ec2commands from the shell

echo "$AWS_X509_CERT" > /root/cert.pem
echo "$AWS_X509_KEY" > /root/key.pem

echo 'export AWS_ACCESS_KEY_ID='$AWS_ACCESS_KEY_ID > /root/ec2inst
echo 'export AWS_SECRET_ACCESS_KEY='$AWS_SECRET_ACCESS_KEY >> /root/ec2inst
echo 'export AWS_ACCOUNT_NUMBER='$AWS_ACCOUNT_NUMBER >> /root/ec2inst
echo 'export EC2_CERT=/root/cert.pem' >> /root/ec2inst
echo 'export EC2_PRIVATE_KEY=/root/key.pem' >> /root/ec2inst
echo 'export EC2_HOME=/root/ec2' >> /root/ec2inst
echo 'export JAVA_HOME=/usr/java/default' >> /root/ec2inst

chmod 700 /root/ec2inst

cd /root
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip ec2-api-tools.zip
chmod +x ec2-api-tools-1.3-46266/bin/*
ln -s ec2-api-tools-1.3-46266 ec2 

The next script should be assigned as a boot script for your server template as the last script. It need to have the previous script run before it can run. It installs the AWS ELB API tools, saves the pertinent data to files for the decommission script, and then notifies the ELB there is a new instance available. Its sole input is the name of the Elastic Load Balancer you want it assigned to.

#!/bin/bash
echo "Beginning ELB Registration for load balancer $LOAD_BALANCER."
source /root/ec2inst
export EC2_INSTANCE_ID="$( wget -q -O - http://169.254.169.254/latest/meta-data/instance-id )"

wget -O /root/ElasticLoadBalancing.zip http://ec2-downloads.s3.amazonaws.com/ElasticLoadBalancing.zip
unzip -o /root/ElasticLoadBalancing.zip -d /root
export AWS_ELB_HOME=~/ElasticLoadBalancing-1.0.15.1
echo 'export EC2_INSTANCE_ID='$EC2_INSTANCE_ID > /root/ec2elb
echo 'export AWS_ELB_HOME='$AWS_ELB_HOME >> /root/ec2elb

$AWS_ELB_HOME/bin/elb-register-instances-with-lb $LOAD_BALANCER --instances $EC2_INSTANCE_ID
echo "ELB Registration for load balancer $LOAD_BALANCER finished with exit code $?"

The final script should be added as one of the first decommission scripts in your server template. It will notify the load balancer to stop sending requests to this instance. Its sole input is the name of the Elastic Load Balancer you want it assigned to.

#!/bin/bash
echo "Starting ELB Deregistering."
source /root/ec2inst
source /root/ec2elb

$AWS_ELB_HOME/bin/elb-deregister-instances-from-lb $LOAD_BALANCER --instance $EC2_INSTANCE_ID
echo "ELB Deregistration finished with exit code $?"

However, when you’re writing code outside of Django, or code that is portable outside of Django (like some of my utility scripts), you need these same abilities without the Django settings file. DictConfig allows you to use Django-style logging, outside Django, but it was only introduced in Python 2.7. Many servers are still on Python 2.6. Luckily, you can just drop one extra file into your directory, and you’re good to go!

First, download dictconfig.py. Put it into your project directory or somewhere on your path.

Next, we’ll configure a logger. We’re going to have it capture all message levels, and log them to both the console and a file in our directory. This is the best combination and makes migrating to a production server much easier.

import os
import logging
import dictconfig

# PROJECT_ROOT is the folder you'll be logging to.
PROJECT_ROOT='/home/josh/programming/project/'

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'handlers': {

        'file_log': {                 # define and name a second handler
            'level': 'DEBUG',
            'class': 'logging.FileHandler', # set the logging class to log to a file
            'formatter': 'verbose',         # define the formatter to associate
            'filename': os.path.join(PROJECT_ROOT, 'output.log')  # log file
        },
        'console':{
            'level':'DEBUG',
            'class':'logging.StreamHandler',
            'formatter': 'simple'
        },
    },
    'loggers': {
        'logger': {               # define another logger
            'handlers': ['file_log', 'console'],  # associate a different handler
            'level': 'DEBUG',                 # specify the logging level
            'propagate': True,
        },
    }
}

dictconfig.dictConfig(LOGGING)

Now when we want to log something, instead of “print ‘Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.’”, we just do:

logging.critical('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')
logging.error('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')
logging.warning('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')
logging.info('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')
logging.debug('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')
# Only within an exception handler:
logging.exception('Fire, exclamation mark. Fire, exclamation mark. Help me, exclamation mark.')

There you go! Logging is easy!
Note: The Fire quote is from The IT Crowd, a hilarious TV show that you should definitely check out. It is on Netflix!

Some of the BlackBerry apps that are available in 3D include Need for Speed Shift 3D. This mobile version of the racing game uses dynamic camera angles to make the viewer appear as though he is actually racing the car. Eight different vehicles are available for driving, and the user can participate in challenges in Chicago London, Italy and Dubai, for a varied experience. This app is available for $5 from BlackBerry App World.

Not to be outdone, developers have released 3D games for the Android platform. Larger tablet screens enable users to control even small on-screen objects with finesse. Dungeon Defenders: Second Wave has been rated over 18,000 times, making it one of the popular games on the Android Market. With its epic plot line, this action/RPG game allows the player to participate in campaigns solo or with up to three friends.

Apple’s iPad offers several 3D apps as well. For example, iGlass is a game that will distract the user for hours. It allows the user to virtually blow molten glass with the tablet’s touch screen. At the end of the simulation, the user can rotate the glass creation to view it from every angle and even take screenshots to display the creation. Another similar app is iDough, a sculpting virtualization to create 3D sculptures. Hobbyists can play with a new medium, without all the mess, while professionals can hone their skills at any time.

Finally, 3D wallpapers are among the newest trends on mobile devices. Deep space vistas such as the one offered by Starfield 3D Live Wallpaper display in 3D, and the user can touch the screen to move through the stars.