August 2006 - Posts
We are human beings. What makes us the intelligent creatures of the earth is our ability to not just communicate, but to communicate complex thoughts and ideas. It comes naturally to us to want to communicate and we learn to communicate at a very young age.
So why is it that some of the biggest problems we as people have in this world is driven by poor communication? We're in the information age of e-mail, blackberries, and text messaging, but we're still crappy communicators.
When you're in an environment where the size of your development team is in constant flux, and is driving towards growth; where the code you are working on is the same code being worked by others on a different tangent; where the decisions you make are taken by others to be policy... why when you're in this environment would you NOT focus on effective communication?
Disparate teams of developers, whether working in a different location, or one row of cubes over, who are working on common code sets, or common data stores, need to be pushing their designs out to the other teams early on in the process to ensure all the other teams are in sync. This just makes good sense. So why is this communication so lacking? Is it the people involved? Is there some underlying fear of someone really scrutinizing and criticizing your work? A fear that maybe if someone finds something that is a show stopper that you'd have to stop and start all over again? I would see these as reasons to why you WOULD communicate.
So I ask... why is this such a big problem? Someone please explain this to me.
So Microsoft professes that all these fabulous counters in .NET that are available to the Performance Monitor tool provide a good level of insight... well bah! bah! I say.
Consider a multi-tier environment with multiple approaches to remoting (.NET Remoting and Web Services). Now - consider multiple front-ends, multiple - nigh - dozens of services - and one or two databases.
Try to get monitors on that beastie. Takes a while just to set them all up... Then I try using the performance monitor ActiveX control to try and go through the data... but therein lies the problem. Different counters, when recorded and analyzed later, will have zeros for some values, which causes any line graph to look crappy and be somewhat useless. The thorns in my side are the .NET Framework monitors mostly.
Then I go through the effort to capture all this - and then the .NET SqlClient counters don't even pick up any activity. What the deuce!?
So - trying to see if I can find someway to make this all more meaningful (honestly, I just want a nice pretty graph) I dump the binary data to a comma-delimited text file - pull it into excel - and this is where I discovered the zeros - and multiple records for the same second.
Feeling jipped - I try writing the data directly to SQL Server - wow - this was a mistake. Logging a LOT of counters across 4 identical machines - and two other machines with 1/3 as many counters (9000 individual counters in all since I'm monitoring several web services in separate app pools across multiple servers -- and in some cases multiple threads) -- within 15 minutes I had created a database that was over 300MB in size - although I did get to see how effective SQL Server 2005 garbage collects (or ineffective in some cases).
The sheer amount of data is not queryable. Really - it isn't. So that was a major waste of my time today. I went back to the binary files and pulling them into the activex control within MMC.
Finding a way to trend the data is really where the "art" comes in. Being able to select which data you want to look at - then see a trend, drop another counter across all the servers - now its unreadable again.
PLEASE MICROSOFT! THERE MUST BE A BETTER WAY!
Then there's the art of figuring out what's going on - if these servers are getting requests on the front end - and these web servers on the web service layer are getting this many requests - which servers are really hitting which? Its pure speculation.
So now I'm looking at how to make this all easier to interpret. How to easily pull all this data together. Right now I want to ditch the MS counters and build in our own so we are getting the data we care about attached to instances which make sense.
The .NET Framework makes all of this possible - it's a matter of the strategy by which you go about it all. At a minimum it means touching every web service method in the service layer... and on the front-end... I'm thinking key UI methods and page event handlers.
The biggest key is going to be merging the logging strategy with the counter strategy in such a way as you can see the counters within the logs. Example: we log whenever we add an object to the cache api - well - at the same time we should also be querying the Cache API Objects counter (or our own counter so we can actually count how many of what type of object) to and write out how many are there.
But it has to be a real strategy - something from the ground up - not something cooked up as we go along... otherwise it won't be as effective.
In the end - it's a "what do we want to see". Me - I'd like to see a visual representation of the environment and watch where the traffic is going. Isolate a particular request and watch it travel from the front to the back... which web services are being called a which points - and how long each one takes - otherwise it's still speculation.
Visiting the WashingtonPost.com today, I came across an article titled "The FBI's Upgrade That Wasn't". Working in Washington a couple years ago, and working in government at the time (not the FBI), I had certainly heard about the "Virtual Case File" project. It was an intriguing project to hear about even just in general terms. It would truly be what we all imagine or in some cases see played out in movies and on tv - an FBI computer system that could be use to pull up files on an agent's PC. What in reality is still mostly green-screen mainframe work with no ability to attach a photo or have advanced searching.
Now, this kind of a system is not small. Not even Joe Schmoe off the street would think this to be an easy task. Nor one that could be done quickly. Stuck in the 80's and in the wake of 9/11, the FBI, in an effort to try and get the system they needed to fight terrorism soonest - as all of us Americans, the President, and Congress wanted -- made their first mistake, they upped the deadlines and added new requirements.
If you're a developer - or a development manager - aside from the newsworthy-ness of the article - it's a great example of poor software maangement - you bail on a project like this. Article is well worth the read.
http://www.washingtonpost.com/wp-dyn/content/artic...
I'm spending some time learning how CommunityServer implements master pages. Since they want to be backwards compatible with ASP.NET 1.1, they haven't adopted the ASP.NET 2.0 Master Pages.
Instead, they've taken on a piece of community work from "MetaBuilders" that they call "MasterPages". MetaBuilder's MasterPages is a small single assembly that simulates an approach similar to that of ASP.NET 2.0.
The design is rather elegant... simple, yet with a keen mastery of ASP.NET and object-oriented techniques.
The deliverable is in the form of a WebForm control library containing four custom controls: ContentContainer, Content, Region and NoBugForm.
Now the concept here starts to go a bit off kilter as you walk the approach... but once you understand what is what - it becomes clear how everything works.
Every ASP.NET page request is going after an .aspx file (forgive my over-generalization, but stay with me here...). Your ASPX file is an implementation of a page (well, duh, right?). Your MasterPage is a template for a part of the page; it defines a common appearance, and marks places where content should be loaded. With the MetaBuilders MasterPages library - your MasterPages are actually user controls (less the code-behind file). Hence the term MasterPages gets a bit confusing since your master isn't really a "page" but a "control". Picky-picky? Maybe. But the fact remains it can be confusing for some.
This ContentContainer control is the keystone to the entire works. Your levels of nested masterpages is rather unlimited. When adding a container control to your user control, you specify the MasterPageFile that should be used as the template for your content, then specify the content to load into each region on the MasterPage. Each ContentContainer must have at least one "Content" control inside and the ID of the Content control must match that of the Region on the MasterPage, otherwise it will not get loaded.
The Region control is your placeholder. This is where you want various blocks of content to go. In DotNetNuke, these are defined in a skin as a "ContentPane" of which you can have as many as you'd like. In the case of the Region control, it does not rely on any other control which may render output like the ContentPanes in DNN do, so there is a lot more flexibility in design.
In the example included with the download from MetaBuilders, a ComplexPage.aspx file, the file which we would want to construct from the various MasterPage templates, starts everything off by having a ContentContainer control with a Content control inside. The ContentContainer control has specified as its MasterPageFile the user control "ComplexMaster.ascx". ComplexMaster.ascx has its own ContentContainer control, a Content control with an id that is different from the ComplexPage.aspx, includes a little formatting and embeds a Region control that matches the id of the Container within the ComplexPage.aspx. The ContentContainer control within ComplexMaster.ascx has a MasterPageFile specified of "BasicMaster.ascx". BasicMaster has some basic HTML and then includes a "Region" control with the same id as the Content control within ComplexMaster.
Now, I just took you from the start of execution down through the layers... What takes some time to get over is - from the standpoint of constructing HTML, it's all backwards. BasicMaster has all the outer HTML, ComplexMaster does a bit more HTML, but is still a template, and the actual ASPX page only contains the content that we need to include inside the template. Nothing more.
The solution works for both ASP.NET 1.1 and 2.0 (although is compiled 1.1). It is available free to download from MetaBuilders, and is worth taking a look at. 
One of the reasons why I find it difficult to spend the time to write a blog entry is the eventual "timeout" while I'm typing the thing up. I've had it happen before... entering the blog on the web form and spend 45 minutes writing something cool and "blammo!" it's gone. I hadn't been too keen on any of these blog post tools - mostly because they tended to not work with CommunityServer (the software I use to run my blog).
Microsoft released in the past few days this cool tool which I'm using right now to write this post called "Windows Live Writer". Now, the product is free and in BETA right now. We can only hope that they plan on keeping it free. From what I've been reading, if they do maintain the "for free" policy, some of the "paid for" products could be in for a bumpy ride.
The great thing about this - no session state. And I can save drafts! How sweet! (Yes - all you out there who are saying, "but my blogjet could do that" - yes well - I'm a Microsoft fanatic - and it works with CommunityServer so... 