Tennis on Campus Counter – XML Style
When I first started Tennis on Campus Counter, I put the scraped information into a comma separated for processing by the PHP scripts. This changed a few days ago however.
I recently dug deeper into the world of XML and started to think about how it could be of use to me. In the past I had always used XML for something someone else started. After I thought about it, I realized that TOC Counter was the perfect place for this. As such, I set about converting it to XML.
This ended up being a very simple process. The PHP parsing code is also now cleaner. Overall I was very impressed with how handy XML was.
Check out the end product: http://www.petederemer.com/toc/schools.xml.
Tennis on Campus Counter 2.1
Following the 2.0 rewrite of the data gathering tools, Â I am ready to release TOC Counter 2.1. Â This release does one major feature and several minor ones. Â Check out the list below for more information.
- Schools by state. Â The scrapper now pulls in what state each school is in, allowing us to put together some comparisons between all of the states. Â Data will start coming in 12/01/2009 with state information, and at that time you will see the new state information and statistics.
- Chart size adjustments. Â The charts have been widened and shortened allowing us to view them a little better
. - Individual Charts. Â Profiles now contain a chart of team number history if the team has any history of their roster number changing.
- Standardized Tables. Â All tables now look the same, and have the same sorting feature.
- Several bug fixes.
Tennis on Campus Counter Updates
I have created version 2.0 of the Tennis on Campus Counter.  For this release I have completely rewritten the scraping tool in Python, effectively disbanding the old BASH scraper.  Also, There a lot of bug fixes on the teams list, updated teams, and new teams pages.  Finally a few tweaks were included to make the data easier to read on the charts.
Tennis on Campus Tracker Updates
Version 0.5 of Tennis on Campus Tracker (aka Tennis on Campus Counter), is close to being completed. Â You can check out the progress so far at http://www.petederemer.com/toc/. Â The themes are still being worked on, however most of everything else is completed as of now.
- Everything is now stored in MYSQL.
- Keeps unique  information changes on each team.
- Display new teams added since last month
- Display updated teams added since last month
- Team profile pages
- Team listing page
Tennis on Campus Data Project
Over the holidays I was looking for a cool little project to work on. What I realized is that it would be really cool to see how many people and teams are currently tallied up in the Tennis on Campus program.  So I started to think about doing it.
Limitations: Now, this little experiment is limited majory because the Tennis on Campus site is still new and does not list all of the teams. In fact it only lists about 1/5 of the total data right now. However these are the only numbers we have available to us right now
Results: What I ended up doing was writing a screen scapping program that does the following:
- Finds the URLs of all the teams listed on the Tennis on Campus website
- Tallies the total number of teams
- Finds the number of players from each team (listed on the profile page we discovered earlier)
- Adds all of the players up for a total number
I originally was going to stop here because I had the data: 131 teams, and 5,153 people. However, I realized that this little project could go one step further. I could actually create a data page where people could see just how many teams there where (listed on the website), per month. I went about a made everything more dynamic and made the script output the data into files that would be easier for parsing.
Next thing I tought myself was how to parse data in PHP. I hadn't done it before, but it proved to be a very easy process. The end result is that I was able to take all the data and put them into arrays. These arrays can be used in any number of ways now.
What I did with the data was to use it and dynamically create Google Charts using it. So now we can not only see how many people and teams are in the program, but we can see (visually with charts) just how much of an increase the numbers have each month.
To run it each month, the scrapping tool I had built previously is just setup on a cron job to run at the first of every month. The new data is added to the bottom of the data file and is dynamically added into the tables!
Check it out: http://www.petederemer.com/toc
License: GPLv3
Source: Download
Archives
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- July 2009
- May 2009
- March 2009
- February 2009
- January 2009
- December 2008
- October 2008
- June 2008
Tag Cloud
Meta
