I’ve been sharing these documents with friends who ask me, “I want to start programming and build a web app, where do I start?”. These resources have also been useful to existing programmers who know C, C++ or Java, but who want to embrace dynamic and web-based programming.
Python is the core backend and web programming language used at Parse.ly. It also happens to be a quickly-growing language with wide adoption in the open source community, and it is a very popular choice for web startups. (Update from the future in 2021: Python rose to the #1 spot in a popular programming language index; it’s safe to say it is now one of the most widely-used languages across all use cases.)
I’ve written a blog post with some original materials for learning Python, import this — learning the Zen of Python with code and slides.
This is a good starting point, but you may also find these resources very helpful:
- For absolute beginners, “Learn Python the Hard Way”. This teaches Python using a series of programming examples, but it really assumes you have no programming background whatsoever. After going through the examples in LPTHW, it may be a good idea to supplement your understanding with Think Python.
- For existing programmers, “Dive into Python 3”. This teaches Python from the starting point that you have already programmed in a mainstream language like C or Java, and want to know what makes Python really cool. Similar audience to my “Zen of Python” slides. At this point, you may be curious about whether to learn Python 2.7 or Python 3; the quick answer is that Python 3 is preferred. See Python2orPython3 on Python wiki to see the differences.
- For advanced programmers, “Python Essential Reference, 4th Edition”. Unfortunately, this book costs money, but it’s basically the best book on Python on the market, and it’s very comprehensive for Python 2.7. But it’s very dense and weighs in at 717 pages, so this is only for those who want to go deep on Python. (Update from the future in 2021: The author of “Python Essential Reference” has released an updated rewrite of his book, that shrinks its size and focuses on Python 3, which is called Python Distilled. Give that one a try if you want to skip over Python 2.7!)
- For cheap advanced programmers, “Official Python Tutorial”. Though the Python tutorial doesn’t have the best narrative style nor the best real-world examples, for advanced programmers, it will teach the reality of the language in a comprehensible way. And it is kept up-to-date with new Python releases. And, it’s free.
Since HTML is basically useless without CSS, you can get by with a short tutorial on HTML and then more advanced tutorials on CSS styling. Here’s what I recommend.
Learn the basics of HTML from MDC’s Introduction to HTML and Wikipedia’s page on HTML. This is a rare case where using Wikipedia is actually a perfect way to get the right background because half the battle with understanding HTML is understanding its history.
An excellent new guide to HTML & CSS together has been published by Shay Howe in 2013.
These look like a great first stop.
You can also use these dedicated resources for CSS specifically:
- For absolute beginners: Use W3C’s official tutorial on Starting with HTML + CSS. This was written all the way back in 2004, but provides the basics with screenshots and real code examples, so is a great way to get started.
- For existing programmers: Mozilla has done a great job putting together a quick and readable tutorial that gives you the basics at a glance.
- For advanced programmers: You’ll want to buy the best book on the subject, CSS Mastery. It has the best explanation of the box model and browser rendering engine’s that I’ve seen, and covers all the edge cases nicely.
- For cheap advanced programmers: You’ll need to look over the MDC (Mozilla) CSS Reference. Pay particularly close to articles on the Box Model and the Visual Formatting Model.
jQuery adds common utilities for DOM manipulation, HTTP/JSON server requests, basic animations, and dynamic CSS.
This post was originally written in 2012 and updated over the years. I have made a strong effort over the years to ensure the suggestions in this post are timeless and work well regardless of when you are beginning your journey of learning web development.
These frameworks are complex, even for advanced programmers like me. They are primarily built to support a style of web development known as Single-Page Applications. This is an important frontend development practice and will be increasingly important as more and more web applications are built, but it won’t replace web sites. SPAs are overkill for building simple web sites and web applications. And, they are hard to learn for beginners.
OK: take a deep breath. You’re learning the building blocks of a modern web application: backend / frontend programming languages and their associated code libraries. Let’s aim to solidify this knowledge using modern web frameworks.
Putting it all together: Python web frameworks
Put simply, a view function lets a web developer respond to user queries and interactions with dynamically rendered response pages. Typically, a view function will query a database, which is where persistent data may live that the user is aiming to retrieve. There are a slew of database technologies and depending on the requirements of a web application, they may combine several database technologies to respond to requests. It will then take the retrieved information and render it into a page that the user can view. This rendering process is the job of the template engine, which is able to plug dynamic values into page templates. Google likely has a single result page template, but depending on your query (and potentially, user profile data), the template will be populated with different results and advertisements. Finally, the web server is a piece of software that receives the requests (e.g. responds to google.com and to the URL for searching), executes the view functions, and returns the responses to the browser. The web server is like the glue that binds everything together.
Now that you understand these basics, you have to face an unfortunate truth: lots of different web frameworks exist that provide this functionality.
Since you only want to develop web apps fast, I’m only going to briefly cover three of these frameworks, and their relative trade-offs. These are: Django, Tornado, and Flask.
Django is, by far, the most popular web framework for Python. It has excellent documentation and is very opinionated in how you should structure your web application. There are also a number of books written about it and a slew of open source modules and extensions.
Django has been used for a number of use cases: enterprise software-as-a-service web applications; consumer-facing, page-oriented software; rapid web application prototypes; content management systems; the list goes on and on.
Let’s evaluate it on the important functionality areas above:
- View Functions: They are defined either as plain Python functions or classes defined inside modules, typically a module called “views.py” living within a Django “application”, which is nothing more than a Python package that contains that file. They are mounted to certain URLs using a special URL dispatcher using regular expression patterns.
- Template Engine: Django has its own template engine designed to be user-friendly even to non-programmers. In this respect, the language is somewhat limited and quirky, and does not really re-use your knowledge of Python for templating. Many advanced programmers end up Jinja2 instead of the default template engine, which is officially supported by the Django project as of Django 1.8.
- Database: Django is very opinionated about your database engine. It was written with the idea that everyone would use a SQL database system of some sort, such as MySQL, Postgres, or SQLite. It provides an object-relational mapper system, or ORM, which makes it easy to define new data storage objects through what are called Models. It also provides an excellent and customizable automatic admin interface that allows instance data to be created and managed using web-based interface, complete with support for search, filtering, bulk operations, and the like. Despite these advantages, the Django ORM is derided by SQL purists as having a worse architecture than the more widely respected SQLAlchemy project.
- Web Server: There is no web server bundled in Django, save a development server not meant to be used in production. This leaves it up to you to integrate Django with a number of WSGI-compliant web servers that are out there, including Apache, nginx, gunicorn, and others.
At Parse.ly, we use Django for our main web application, but swap the default template engine for Jinja2. Though we have a Postgres database that benefits a bit from Django’s ORM and admin interface, the bulk of our data is stored in Amazon S3 and Elasticsearch, with Redis used for caching, and thus does we do not leverage the ORM much in our web application. (See my related article, “On multi-form data”, for an explanation of why we combine databases in this way.) Further, for other parts of our system that require access to our Postgres DB, we sometimes reach for SQLAlchemy. For production deployments, we run Django under nginx and uWSGI.
Tornado is a web framework that was released by Facebook after its acquisition of Friendfeed. It has a significant architectural difference from Django in that it is built to solve the C10k problem: the challenge of building web servers to handle thousands of simultaneous web connections at one time. As a result, it bundles its own web server and expects you to use it.
Traditional web frameworks like Django expect that every web request will be handled by a separate web server thread. With thousands of simultaneous connections, this can overwhelm your web server with excessive memory usage, causing the server to slow down or even crash. Tornado is written the same way as other asynchronous web servers like nginx and NodeJS. As a result, it has the same scaling benefits: it can handle thousands of concurrent requests while keeping memory of your server stable.
This architectural difference has ramifications throughout your codebase, however. Tornado view functions tend to look different, and the usage of databases tends to be entirely different, too. So this isn’t the best choice for beginners, unless you know for a fact that your application is going to involve lots of concurrent connections from the get-go. Examples of this include: web chat systems, telephony applications, API servers, mobile backends, or some classes of “real-time” web applications.
Tornado has a nice user guide intended for beginners. The O’Reilly book, Introduction to Tornado, is also an excellent (and quick) read that goes through most the facilities available in the framework.
- View Functions. Tornado view functions are implemented via classes known as Handlers, which are subclasses of
tornado.web.RequestHandler. Similarly to Django, there is a URL dispatcher called the Application that maps URL regex patterns to Handlers. Unlike Django view functions, Tornado view functions are not meant to do much work. The reason for this is that all view functions run in a single thread, and thus any long-running code will slow down your entire web server. Instead, the responsibility of the function is to delegate work to other asynchronous services handled by Tornado’s server. The primary candidate here is to have Tornado make an asynchronous HTTP request to some other service. There are also some databases and database drivers that are written in an “asynchronous” style which you can use reliably with Tornado, but in general, the idea is to avoid database queries in your view functions.
- Database: As mentioned earlier, Tornado doesn’t expect your view functions to hit a database often since this could slow your entire web server down. As a framework, it expects data querying to be “your problem”. Instead, I have seen most people put Tornado in front of other HTTP services that might be written using blocking frameworks like Django or Flask. I have also seen usage of async-friendly data stores such as MongoDB and Elasticsearch. Elasticsearch uses HTTP as the client interface, so it is easy to hit these directly using Tornado’s built-in HTTP client. MongoDB has an official asynchronous driver called Motor, meant for use specifically with Tornado. Async drivers continue to become more common, especially thanks to the work in Python 3 on
asyncioand Tornado’s asyncio bridge; for example, Tornado’s blog engine example uses
psycopg2directly, which, as of modern versions, supports async. For databases whose drivers do not support async operation with an event loop, Tornado provides a mechanism to automatically leverage thread pools via the
tornado.concurrentmodule and the
- Web Server: Tornado bundles its own web server, which is perhaps the most powerful and convenient aspect of the framework. The beautiful thing about this is that the exact same web server you run locally for development is the one you can run in production.
Flask is the simplest web framework of these, and it is just as popular, both for rapid prototypes and for production use cases. The author of the framework has a PyCon presentation explaining its motivation.
Funny enough, it was built out of an April Fool’s joke where the author “zipped up” two of his existing projects — Jinja2 (template engine) and Werkzeug (HTTP library) — and glued them together with a small Python file, thus declaring it a new web “microframework”.
The joke became a real open source project which is notable for its simplicity, respect of Python’s facilities, strong documentation, and ease of use. Due to its reliance on existing, high-quality Python modules, the actual web framework is only approximately 1,000 lines of code. The quickstart application requires only a single Python file which, when run, gives you a working development web server that renders a dynamic response. For all these factors and more, it is my preferred web framework for new web applications, especially those that wouldn’t benefit from Django’s admin interface or Tornado’s concurrent request scaling.
- View Functions: View Functions are as simple as it gets in Flask. They are simply plain Python functions. They are mounted to URL patterns using a Python Decorator called
route. This includes support for Variable Rules which tend to be much more comprehensible compared to regex-based routes as in Django and Tornado. Similarly to Django, view functions in Flask are where the bulk of your application’s logic will go, including things like database queries.
- Template Engine: Flask is meant to be used with Jinja2, an excellent and well-documented template engine that is also widely used by Django developers as a drop-in replacement for that framework’s template engine. It strikes a balance between Django’s template language, meant to be understood by non-programmers (see Template Designer Documentation) while also having good interoperability with Python code and support for a wide range of control structures.
- Database: This is the least opinionated part of the Flask framework; it makes no recommendation as to what database to use, considering this to be beyond the scope of a core web framework. That said, the Flask Extension Registry contains some modules that help integrate Flask with this or that database technology, such as Flask-SQLAlchemy (provides support for all SQL data stores) and Flask-PyMongo (provides support for MongoDB connections). However, you can just easily query databases by simply importing appropriate Python client libraries which often exist for that particular DB — and that is, indeed, “The Flask Way” of doing this.
- Web Server: Though no web server is bundled in the framework, it can be deployed even more easily than Django to any number of WSGI-compliant web servers, as described in the Deployment Options section of their documentation. The built-in debug-mode development server is extremely handy for local development, supporting full stack traces and even an embedded Python interpreter for inspecting the state of variables at the time of the web server crash.
This gives you the benefit of local development with a Flask web server, where code changes to your templates and view functions will be reflected instantaneously. But, then, when you are ready to deploy your website, you can “freeze” all of your pages (render all your templates as plain HTML files), and deploy those static pages to a simple web server (e.g. nginx) or content delivery network (e.g. GitHub Pages, Cloudflare Pages, or Netlify).
Conclusion: pick a stack
This article has lots of resources that can help you pick a stack, but I have some opinions about how you can get started easily.
- Use Python 3. The Python community has mostly moved on from Python 2.7.x to Python 3.x versions. Flask is fully Python 3 compatible, Django has made Python 3 the default language of its documentation, and Tornado 4.0+ include Python 3 specific features that make it easier to use the framework with Python 3’s new
awaitkeywords. What’s more, mostly all of Python’s most popular projects are ported. (For more background, here’s a link from the future in 2021: Python 3 is here, and the sky is not falling.) Python 3… just do it!
- Start with Flask, switch as necessary. If you are just getting started with web development, you’ll be able to assemble an application with the above components easily in Flask. You may not know yet whether your application requires thousands of concurrent requests (as provided by Tornado), or, whether you would benefit from an extensive ecosystem of open source plugins; a full-featured data model for a SQL database; or, an auto-generated administrative interface (all benefits in Django). So defer those decisions to when you are better able to make them in an informed and careful way.
- Pick the simplest database possible, upgrade later. Since Flask doesn’t impose any database on you, you can choose to pick the simplest database that could possibly work. This even includes the simplest option: “no database at all”, e.g. a static website with Frozen-Flask. But if your web application needs a database, some good candidates for your early days are SQLite, MongoDB, or Redis. As you start to understand your requirements more, you may need to upgrade to a full-fledged SQL RDBMS such as Postgres or a full-text index such as Elasticsearch. But you won’t know for sure until you fully understand the form of your data, so why lock into heavyweight solutions up-front?
- Pick a host that matches your system administration skill level. You will need a Linux server available to you to deploy your app, and for that I suggest simple hosting providers such as DigitalOcean (works on the “VPS” or “VM” model, which means you need to know quite a bit about Linux system administration) or Opalstack (works on the “shared hosting model”, which means you don’t have to hand-configure your Linux server). You will hear a lot of people mention Amazon Web Services or Google Cloud Platform, but I recommend you only switch to AWS or GCP later, once you understand their quirks and tradeoffs vs traditional providers (as well as benefits). Setting your server up may require some knowledge of UNIX and system administration which is beyond you. If that is the case, you should use a shared hosting provider like Opalstack, since it has good Python support and pre-fabricated deployment setups for Django, uWSGI, and nginx available. (Note for old-timers: Opalstack was created by the same team that originally brought you Webfaction, one of the most popular original Django shared hosts.) If you don’t mind paying a premium to have someone else host all your infrastructure for you, you may also want to consider a PaaS provider such as Heroku or Google Cloud Run. Personally I don’t recommend these services for economic and complexity reasons, but they have many proponents.
- Develop locally, deploy simply. Each of these web frameworks have workflows for developing locally. You should use those until you get comfortable with the frameworks. That said, the day will come where you want to see your web application live on a real web server. For that moment, if you are developing with Django or Flask, I recommend you deploy to uWSGI behind nginx. Flask has docs for this, so does Django. To actually push your code to your servers, I recommend you start with extremely simple UNIX tools: rsync and ssh. With rsync, you can copy your Python project quickly to your server, and thanks to rsync’s incremental copy, it will only copy changed files. For example:
rsync -Pav myproject/ remoteserver:myproject/. With ssh, you can execute remote commands on your server such as
ssh remoteserver sudo restart nginxto restart your nginx web server. Once you start to need fancier deployment options, you can upgrade to Fabric, a Python-based deployment tool, replacing your rsync command with project tools and replacing your ssh commands with calls to run().
- Don’t develop on Windows. This is unfortunate but true. If you really must use Windows, I recommend you read up on the Windows Subsystem for Linux and the support for running Ubuntu under Windows. I don’t have personal experience with this, but, traditionally, the lack of support for UNIX tools on Windows puts it at a significant disadvantage for building modern web applications that deploy to Linux servers running Python. Another option you can investigate is virtualization, which will let you use a third-party application to run a Linux guest virtual machine. These include virtualbox, vagrant, and VMWare.
I look forward to seeing the web apps you build and deploy.
Note that in 2013, I converted this blog post into a full-blown tutorial. You can find the code, slides, and video in this post, which covers all my “suggested” technologies, like Twitter Bootstrap, jQuery, Flask, Jinja2, MongoDB, Fabric, nginx, supervisor, uWSGI, etc. Despite the age of this tutorial, all of the technologies used are still well-maintained (just at more modern versions) and are still solid production choices.
Note also that in 2021, I played around with the piku open source project, and the extensive README.md I published on GitHub via the webappfast-piku repo explains how you can use piku to setup a deployment stack similar to the one described in this article, but on your own DigitalOcean VPS or local Linux homelab server.
Do you want to do modern Python web development on a daily basis, working on some of the most interesting problems at the intersection of large-scale data analysis and information visualization? Check out Parse.ly — we’re hiring for our fully distributed team.