HTML parsing

While working on my new personal project today, I came across an unexpected problem - how to parse an html file using C#. My business partner Shawn had previously done something similar using VB.Net, but this was my first stab at it. Luckily, there are a number of people who have already dove into this area, so it wasn't hard to find some answers. My first find was the best - a guy named Jeff Heaton had already created a number of classes in C# to address this problem and put them out under Limited GNU Public License(LGPL). Although these were easy to implement and worked without any difficulties, I quickly ran into a limitation - he had designed his classes to be used as a spider and was only interested at the attributes of the html tags (like 'href' attribute in the 'a' tag). Unfortunately, I need the descriptive contents of the tag, i.e. the words between '' and ''. My first attempts to modify his code have left me getting the urls and descriptions of half of the links - it skips every second one... So tomorrow will be trying to re-engineer the whole thing to get what I need.

Well, on my continuing search for employment as a developer, I am noticing some sights using Macromedia's Flex technology. First, there is my past client Launch Vision and a site they have developed, Stem Cell Therapeutics. Nice job guys - both of them look great! Also, I'm pretty sure 5By5 Software is using it at least partially for their product. Looks like it is definetly a growing technology, at least in Calgary...

By the way, I have found one of the most annoying things I've ever seen in Windows tonight - I accidentally held down the shift key for 8 seconds (my thoughts were drifting as I was listening to some tunes) and this dialog popped up! I pressed cancel, but it was too late - the "Filter Keys" option had been engaged through the Windows Accessibility option. The most immediate effect was not being able to point my mouse to a new point on a page and position the Ibar there - instead it would highlight from the first point to the new point! I was trying to code and this was infuriating! Well, after checking Google out, I found a page that allowed me to get by it (I unplugged my keyboard and plugged it back in), but now this stupid little stop watch is in my display icons in the taskbar and I can't get rid of it! I've hidden it and hopefully when I reboot the computer it will go away... But who thought of this?!

Lean Software

Tonight I attended Mary Poppendieck's talk about Lean Software development that the Calgary Agile Methods User Group (CMAUG) was putting on. It was interesting - Lean Software Development essentially boils down to streamlining the whole process so that software is made when requested by a client and is developed with the client in mind. It takes the larger view of the development cycle where Agile development is a part of the process, rather than being focused purely on the development.

I was wanting to talk in depth about it but my ISP was down for a while and so my thought's have gotten mixed up unfortunately. Regardless, one of the things she talked about was Just In Time development. In a manufacturing environment, a company wants to have as little inventory of finished product as possible on hand - better to create as needed (as Dell has much to its success). From a development perspective, Mary argues that inventory in software is any piece of software that is exists in a semi-finished state (i.e. code that addresses a goal and is tested) that is not making money. I realized that this was essentially what the project that I recently finished was - and it was a failed project.

Another concept was that of releasing software as soon as it addressed a basic goal. Once this has been done, features can be added in any order. From this perspective, the timeline between iterations would be very small. Also, Mary believed that the organization of human resources in software projects was in itself wasteful, wanting teams to stay together through different projects, allowing for greater cohesion and specialization.

My thought on this goes a step further - a team as a whole should follow the project through all cycles, from design to implementation, to testing, and finally deployment. While I recognize that there is the need for some specialization and people have different strengths and weaknesses, my thoughts are that as a worker in IT, I already have to know the basics of everything. I'm a developer, and not only that, a contractor who handles custom projects. So, I have to be able to take a project through all the cycles. Also, as everybody in the industry knows, it is a constant learning process - new technologies and new methodologies. So my suggestion is this - have the team work together throughout the various stages, but the person(s) who specialize in the area lead for that section with the project manager to keep the project focused on the needs of the client. So the architect would lead for the design of application, the developers would code the design with the architect looking on, and all taking part in testing with the QA people designing the tests for the application, with the others implementing the tests. Keep the team in a smallish area (but not cramped - just not spread out through a building) and allow them to work on very short cycles.

Just some thoughts...