Friday, February 16, 2007

How to draw 3D objects using python and VPython

I found another cool python package-- VPython, "3D Programming for Ordinary Mortals". It is very easy to use. Here is a simple example that creates a green sphere and a red box.

1. Download and install VPython. For Windows users who already have Python 2.4.3 installed, go to, and install "VPython-Win-Py2.4-3.2.9.exe" only. Other users can select the proper version from here
2. Open IDLE ("Start" -> "All Programs" -> "Python 2.4" -> "IDLE (Python GUI)")
3. "File" -> "New Window"
4. Type this inside:

from visual import *
redbox=box(pos=vector(4,2,3), size=(8.,4.,6.),
greenball=sphere(pos=vector(4,7,3), radius=2,
5. Hit "F5"
6. To change the camera angle, right-click and drag.

Scientific equations can also be used to create more complex diagrams and animations. Here are a couple more links to get started:
VPython applications for Teaching Physics
Simple 3D Programming Using VPython

How to use python and mechanize to grab a web page

In a previous post, I looked at how to grab a web page using urllib2. Mechanize looks to be the best python package for automatic web browsing or spidering the web. The author also wrote ClientForm and ClientCookie which is included in Mechanize. I also looked at Mechanoid, which is a fork of Mechanize, but it looks like Mechanize is the way to go. It is still under active development. The author has given examples of how to access a password protected page using Mechanize. This first example will just show how to grab a page without password protection.

How to:
1. Install Easy Install
2. open "cmd.exe"
3. "cd c:\python24\Scripts
4. run "easy_install mechanize"
5. run the following python code:
from mechanize import Browser

br = Browser()"")
print br.response().read() 
That's it! vs. bookmarks (and Firefox add-on deficiencies)

I had heard of before but just started using it and it looks pretty good so far. It seems like tags are a better way of organizing than folders. I like how clicking on a tag in the web interface on brings up the related tags. These work like sub-folders for me. If I click on my "python" tag, for instance, will show a list of related tags such as "documentation", "howto", "matplotlib", "package", etc. Then if I click on one of these related tags, will show the webpages that have both of these tags. This is a great feature because it gives you the organization of folders with the speed and flexibility of tags. Also, the use of tags doesn't constrain you to a particular location in your folder hierarchy. If you want to find a page tagged with "python" and "howto", you can either click on "python", and then "howto" or vice versa.

Even though I just said that I use IE7 over Firefox, I saw there was a extension for Firefox so I decided to try this out. This extension makes things faster and more like your normal bookmarking experience. The search feature is good for searching tags which is not available on the web interface. However, the Firefox extension doesn't have the thing that makes so delicious to me in the first place-- related tags. I cannot type in two tags and have only the pages with both those tags show up. Instead of an intersection of the tags it shows a union of the tags. I don't want that. Just when I thought I would switch from IE to Firefox.

Why I choose IE7 over Firefox 2

The answer is simple: ClearType Text in IE7. I have an LCD screen and the ClearType text just looks a lot better. Maybe Firefox has more cool extensions, implements the standards better, etc., but for now IE7 wins for me.
Update: After reading a comment from reader, I found out you can use ClearType for almost everything in Windows. So I'm using Firefox now.

How to grab a page using urllib2

I got this from It's easy. The next step is to learn basic authentication so I can get to my bank site and download my bank statements automatically.
import urllib2

response = urllib2.urlopen('')
html =

print html

How to use python and Beautiful Soup to screen scrape the links from a Google Blog search

I wanted to organize some results of a Google Blog search and was planning to use pyGoogle and the Google SOAP search API but found out that they have discontinued giving out keys. (See this article.) So an alternative method is screen scraping the html. I searched for python screen scraping and found Beautiful Soup. It works well-- much better than my previous attempts at writing my own screen scraping code. Here is how to do it. (Note: This works for using the Google Blog search, but not the Google web search. I am getting a HTTP Error 403: Forbidden. Probably because this is a script? Maybe I need to investigate the Google AJAX search API.)

UPDATE (1/23/07): I was right about the 403 Error. Google's terms of service do not allow automated queries. There is a way around this, but I don't want to promote bad behavior. Alternatively, running the automated search on Yahoo and Dogpile seems to work just fine. Here is an interesting comment about why Google got rid of the SOAP search API.

1. Goto
2. At the bottom of the page under the "Download Beautiful Soup" heading, click tarball.
3. Save the file.
4. Uncompress the contents to c:\temp
5. Open a "cmd.exe" shell
6. "cd c:\temp"
7. "cd BeautifulSoup-3.0.3"
8. type "python install"

Create this file and run it:
from BeautifulSoup import BeautifulSoup
import re
import urllib2

url = ''
response = urllib2.urlopen(url)
html =

soup = BeautifulSoup(html)
links = soup.findAll('a', id=re.compile("^p-"))
for link in links:
    print link['href']
Your results should be something like this:;amp;amp;amp;amp;AndOr=0&Sort=2&amp;amp;amp;amp;amp;JobType1=20&Rate=180&RateType=4<xt=nn14+2je%2C+Kettering&Radius=40&LIds1=fi,E,Q,d,CFV,CFu,CF7,CGy,CHX,CHY,CIB,CII,CIS,CJL,CJM,CM3,CNm,COB,COh,CPx,CQQ,CSf,CTB,CTd,CVS,CWR,CWn,CW6,CW_,CXi,CXs&LIds2=c9,G,s,2,9,BL&LIds6=Drc,B,I,g,o,p,q,r,Ih,Ij,N9,Pb,b6,il&From=%2FJobSearch%2FAdvancedJobSearch.aspx&DCMP=R_RS_XML_XML_090107