My blog has gotten... stale. Since my last post, I did a lot of good stuff at JPMorgan Chase programming wise... then moved on to Stanley Associates up in Dumfries for a year which showed promise at first, and then... not-so-much... and now I have my four mile commute back at a young defense contractor called ACGS, headquartered here in Fredericksburg. There are a lot of refreshing new challenges here... challenges I look forward to blogging about from a technical perspective. Most of the challenges center around working with GIS, creating/modifying GIS-driven apps, and working with video in a new and innovative way. My hope is that this isn't the last post you'll see from me on my geek blog for the next two years, and that this particular blog will start to really take off.
More soon.
Want to really know when everything went horribly wrong and you ended up tightly coupling something that was supposed to be modular or service oriented? Take a look at your documentation.
More and more as I have been digging through our team's archives, I've started to discover some of the reasons why we time-and-again are exerting additional effort to de-couple something, or end up throwing something out and starting over, or not being able to leverage something that, while being built, was billed as "reusable". What I have found is that, while in a lot of cases we generate a lot of good doco, we're missing something. And it's not more doco. See the documentation is good, however its typically developed based on the project or business effort at hand. Business efforts that may be, and typically are, driven by a very specific product or product line. Now, the documentation itself spells out things like "...this should also be able to be used by..." or "... thereby creating a standard (enter common component name here)...". Such statements, however, are buried within a document very specific to the project. Or, these statements were stated (and restated to the point of folklore) by the development team when the system was built, but something fell just a bit short.
The reason why this happens typically boils down to time pressure and concessions made on the design in favor of time. This is a very common situation is software development. With every project, this WILL certainly occur without a doubt. (And if you're working on a project where it's not occurring, you're probably very tired of dealing with release dates that never come... or - somehow - are very very lucky and whoever you're working for has money to spend without limitation.)
The documentation I was going through led me to realize something: with a few notable exceptions, I was unable to truly separate individual system components (services, ui modules, ui framework, etc) from the project they were originally constructed for. Each of these components had become so wired to the original project and product it was supporting, that it was natural for the developers to cross the lines... what would be the harm? right?! But it is exactly the harm. And by crossing the lines, whether it be for speed or a general misunderstanding of the system design and intent of underlying components, the coupling began.
So what can be done differently? One way to build on top of your team's creativity and desire to reuse as much as possible, is to encourage, nay, require them to document each reusable component in separate documents, laying out the requirements for their isolated system, the system design, etc. This forces the developers to THINK about which document, which forces them to THINK about the system impact, and it no longer becomes convenient to couple these systems together because there spec lives in the same document.
I know what you're thinking... "Isn't that more documentation?" No. More files, yes. More doco, no - not unless you weren't creating enough documentation to begin with.
The separation of component documentation starts to drive better decisions on alternatives when the time pressure kicks in too. Consider: the "quick" solution is to make your underlying component knowledgeable of the components that ride on top. (Say your service now needs to have some knowledge, however big, of your UI.) When the developer goes to document this - if it wasn't obvious before (and when working in a pressure cooker, it's an easy miss) - the realization of the coupling is apparent. This is now the perfect time to start considering alternatives to prevent the coupling from occurring.
So - document your reusable componets. Reference the designs for the components from your system documentation for those systems that will use them.
Remember Always:
If it is supposed to be reusable - it doesn't "belong" to a system, but is a system of its own leveraged by multiple larger systems.
We are human beings. What makes us the intelligent creatures of the earth is our ability to not just communicate, but to communicate complex thoughts and ideas. It comes naturally to us to want to communicate and we learn to communicate at a very young age.
So why is it that some of the biggest problems we as people have in this world is driven by poor communication? We're in the information age of e-mail, blackberries, and text messaging, but we're still crappy communicators.
When you're in an environment where the size of your development team is in constant flux, and is driving towards growth; where the code you are working on is the same code being worked by others on a different tangent; where the decisions you make are taken by others to be policy... why when you're in this environment would you NOT focus on effective communication?
Disparate teams of developers, whether working in a different location, or one row of cubes over, who are working on common code sets, or common data stores, need to be pushing their designs out to the other teams early on in the process to ensure all the other teams are in sync. This just makes good sense. So why is this communication so lacking? Is it the people involved? Is there some underlying fear of someone really scrutinizing and criticizing your work? A fear that maybe if someone finds something that is a show stopper that you'd have to stop and start all over again? I would see these as reasons to why you WOULD communicate.
So I ask... why is this such a big problem? Someone please explain this to me.
So Microsoft professes that all these fabulous counters in .NET that are available to the Performance Monitor tool provide a good level of insight... well bah! bah! I say.
Consider a multi-tier environment with multiple approaches to remoting (.NET Remoting and Web Services). Now - consider multiple front-ends, multiple - nigh - dozens of services - and one or two databases.
Try to get monitors on that beastie. Takes a while just to set them all up... Then I try using the performance monitor ActiveX control to try and go through the data... but therein lies the problem. Different counters, when recorded and analyzed later, will have zeros for some values, which causes any line graph to look crappy and be somewhat useless. The thorns in my side are the .NET Framework monitors mostly.
Then I go through the effort to capture all this - and then the .NET SqlClient counters don't even pick up any activity. What the deuce!?
So - trying to see if I can find someway to make this all more meaningful (honestly, I just want a nice pretty graph) I dump the binary data to a comma-delimited text file - pull it into excel - and this is where I discovered the zeros - and multiple records for the same second.
Feeling jipped - I try writing the data directly to SQL Server - wow - this was a mistake. Logging a LOT of counters across 4 identical machines - and two other machines with 1/3 as many counters (9000 individual counters in all since I'm monitoring several web services in separate app pools across multiple servers -- and in some cases multiple threads) -- within 15 minutes I had created a database that was over 300MB in size - although I did get to see how effective SQL Server 2005 garbage collects (or ineffective in some cases).
The sheer amount of data is not queryable. Really - it isn't. So that was a major waste of my time today. I went back to the binary files and pulling them into the activex control within MMC.
Finding a way to trend the data is really where the "art" comes in. Being able to select which data you want to look at - then see a trend, drop another counter across all the servers - now its unreadable again.
PLEASE MICROSOFT! THERE MUST BE A BETTER WAY!
Then there's the art of figuring out what's going on - if these servers are getting requests on the front end - and these web servers on the web service layer are getting this many requests - which servers are really hitting which? Its pure speculation.
So now I'm looking at how to make this all easier to interpret. How to easily pull all this data together. Right now I want to ditch the MS counters and build in our own so we are getting the data we care about attached to instances which make sense.
The .NET Framework makes all of this possible - it's a matter of the strategy by which you go about it all. At a minimum it means touching every web service method in the service layer... and on the front-end... I'm thinking key UI methods and page event handlers.
The biggest key is going to be merging the logging strategy with the counter strategy in such a way as you can see the counters within the logs. Example: we log whenever we add an object to the cache api - well - at the same time we should also be querying the Cache API Objects counter (or our own counter so we can actually count how many of what type of object) to and write out how many are there.
But it has to be a real strategy - something from the ground up - not something cooked up as we go along... otherwise it won't be as effective.
In the end - it's a "what do we want to see". Me - I'd like to see a visual representation of the environment and watch where the traffic is going. Isolate a particular request and watch it travel from the front to the back... which web services are being called a which points - and how long each one takes - otherwise it's still speculation.
Visiting the WashingtonPost.com today, I came across an article titled "The FBI's Upgrade That Wasn't". Working in Washington a couple years ago, and working in government at the time (not the FBI), I had certainly heard about the "Virtual Case File" project. It was an intriguing project to hear about even just in general terms. It would truly be what we all imagine or in some cases see played out in movies and on tv - an FBI computer system that could be use to pull up files on an agent's PC. What in reality is still mostly green-screen mainframe work with no ability to attach a photo or have advanced searching.
Now, this kind of a system is not small. Not even Joe Schmoe off the street would think this to be an easy task. Nor one that could be done quickly. Stuck in the 80's and in the wake of 9/11, the FBI, in an effort to try and get the system they needed to fight terrorism soonest - as all of us Americans, the President, and Congress wanted -- made their first mistake, they upped the deadlines and added new requirements.
If you're a developer - or a development manager - aside from the newsworthy-ness of the article - it's a great example of poor software maangement - you bail on a project like this. Article is well worth the read.
http://www.washingtonpost.com/wp-dyn/content/artic...
I'm spending some time learning how CommunityServer implements master pages. Since they want to be backwards compatible with ASP.NET 1.1, they haven't adopted the ASP.NET 2.0 Master Pages.
Instead, they've taken on a piece of community work from "MetaBuilders" that they call "MasterPages". MetaBuilder's MasterPages is a small single assembly that simulates an approach similar to that of ASP.NET 2.0.
The design is rather elegant... simple, yet with a keen mastery of ASP.NET and object-oriented techniques.
The deliverable is in the form of a WebForm control library containing four custom controls: ContentContainer, Content, Region and NoBugForm.
Now the concept here starts to go a bit off kilter as you walk the approach... but once you understand what is what - it becomes clear how everything works.
Every ASP.NET page request is going after an .aspx file (forgive my over-generalization, but stay with me here...). Your ASPX file is an implementation of a page (well, duh, right?). Your MasterPage is a template for a part of the page; it defines a common appearance, and marks places where content should be loaded. With the MetaBuilders MasterPages library - your MasterPages are actually user controls (less the code-behind file). Hence the term MasterPages gets a bit confusing since your master isn't really a "page" but a "control". Picky-picky? Maybe. But the fact remains it can be confusing for some.
This ContentContainer control is the keystone to the entire works. Your levels of nested masterpages is rather unlimited. When adding a container control to your user control, you specify the MasterPageFile that should be used as the template for your content, then specify the content to load into each region on the MasterPage. Each ContentContainer must have at least one "Content" control inside and the ID of the Content control must match that of the Region on the MasterPage, otherwise it will not get loaded.
The Region control is your placeholder. This is where you want various blocks of content to go. In DotNetNuke, these are defined in a skin as a "ContentPane" of which you can have as many as you'd like. In the case of the Region control, it does not rely on any other control which may render output like the ContentPanes in DNN do, so there is a lot more flexibility in design.
In the example included with the download from MetaBuilders, a ComplexPage.aspx file, the file which we would want to construct from the various MasterPage templates, starts everything off by having a ContentContainer control with a Content control inside. The ContentContainer control has specified as its MasterPageFile the user control "ComplexMaster.ascx". ComplexMaster.ascx has its own ContentContainer control, a Content control with an id that is different from the ComplexPage.aspx, includes a little formatting and embeds a Region control that matches the id of the Container within the ComplexPage.aspx. The ContentContainer control within ComplexMaster.ascx has a MasterPageFile specified of "BasicMaster.ascx". BasicMaster has some basic HTML and then includes a "Region" control with the same id as the Content control within ComplexMaster.
Now, I just took you from the start of execution down through the layers... What takes some time to get over is - from the standpoint of constructing HTML, it's all backwards. BasicMaster has all the outer HTML, ComplexMaster does a bit more HTML, but is still a template, and the actual ASPX page only contains the content that we need to include inside the template. Nothing more.
The solution works for both ASP.NET 1.1 and 2.0 (although is compiled 1.1). It is available free to download from MetaBuilders, and is worth taking a look at. 
One of the reasons why I find it difficult to spend the time to write a blog entry is the eventual "timeout" while I'm typing the thing up. I've had it happen before... entering the blog on the web form and spend 45 minutes writing something cool and "blammo!" it's gone. I hadn't been too keen on any of these blog post tools - mostly because they tended to not work with CommunityServer (the software I use to run my blog).
Microsoft released in the past few days this cool tool which I'm using right now to write this post called "Windows Live Writer". Now, the product is free and in BETA right now. We can only hope that they plan on keeping it free. From what I've been reading, if they do maintain the "for free" policy, some of the "paid for" products could be in for a bumpy ride.
The great thing about this - no session state. And I can save drafts! How sweet! (Yes - all you out there who are saying, "but my blogjet could do that" - yes well - I'm a Microsoft fanatic - and it works with CommunityServer so... 
Ok... So I finally have the Agent Desktop portion installed. What this means is I have installed all of the basic web services, I have installed the database on Sql Server 2000, I have configured "AzMan" - aka Authorization Manager - and I go to run the "Agent Desktop" - and i get an error...
An hour later I'm able to get the thing to tell me what the error is... but it's not helping much. A simple "Access Denied" a 401. Fabulous.
So I went through IIS and configured all the web services to allow Anonymous access. This gave a wonderful appearance that it might work... I could pull the asmx files up... but then I tried to pull up the application and it failed miserably. Why? Well, this was obvious... because it uses Integrated Auth all the way through to SQL server.
So I'm stuck. What I have now is the web services and iis are not really performing integrated windows authentication properly... I'm sure this is some IIS6 lock down thing I just need to sort out. A gent at the office, Doug McCoy, brought up an interesting point - the original IISLockdown tool would not just disable, but would configure whatever it was you wanted. So that's my next clue. If I figure this out - I'm blogging it for sure...
Ah--- the mysteries of authentication and authorization and Windows and NTLM and NTFS... *sigh* what a pain in the kiester! A whole day to unravel the mystery.
Oh - and of course, none of this b.s. is in the Customer Care Framework deployment guide. None of it! Now - I'd assume the author(s) either forgot this particular detail - or weren't involved in configuring it.
There is something to be said for having knowledgeable IIS geeks as developers... and I know IIS pretty well - but this is one of those things where there are too many layers of authorization.
This is the third day I've spent working on setting up Microsoft's Customer Care Framework 2005 (CCF 2005). Now, why has it taken 3 days? (or more?) - I've had to spend some time getting the virtual servers set up and configured before even performing the steps in the 100+ page "Deployment Guide".
Now, this is a very spread out deployment, although, sometimes it just "feels" big when you have to crank up 4, then 5, then 6 virtual machines to emulate the environment they call for.
CCF is truly an end-to-end-lets-see-how-many-Microsoft-techs-we-can-use product. To set up the "Deployment" environment for development (what I'm attempting to do) I've installed Windows Server 2003 (a couple of times - although one install copied a couple times for the most part), SQL Server 2000, SQL Server 2000 Reporting Services, and the Enterprise Instrumentation Framework. I haven't gotten to the BizTalk 2004 Server set up beyond configuration of the Enterprise Single-sign-on. The clients I am setting up right now on two Windows XP SP2 virtual machines. The authentication makes extensive use Active Directory (AD) and a tool I was once taken with - AzMan (or Authorization Manager).
So my voyage of discovery for CCF is just three days old, and I haven't looked at a single line of code, and I'm on page 37 of 116 in the deployment guide. Hopefully at this point I have all the virtual hardware I need because I'm running out of physical RAM.
As my voyage begins to get exciting, I'm going to blog about what I discover and the questions I'm pondering as I go through the Customer Care Framework. Now, CCF currently runs on .NET Framework 1.1, and Sql 2000. Two techs we're hoping to upgrade very soon. And with Microsoft announcing .NET Framework 3.0 -- yeah - seems like if MS does put together a CCF 2006 running on .NET 2.0 and Sql 2005, that it'll be slated for upgrade in a year when the next version of the framework hits the streets.
Stay tuned, more to come on the Customer Care Framework 2005.
I saw a WebCast today from TechEd in Boston given by Martin Granell. At the end of the web cast we received the URL to his blog to eventually get code from the session. Visiting Martin's blog uncovered more than just some cool info on Grandmothers and Missile Defense Systems... it uncovered some really cool tips for setting up virtual server and optimizing it for best performance.
I'm looking forward to putting these to good use here in the next few days, and will let you know how successful I am.
Honestly, I'm finding that Virtual Server 2005 R2 runs a lot faster than Virtual PC 2004. And I'm already pleased with it's performance. But like most folks, I'll take any opportunity to eek more power out of anything! Especially if I'm having to run Visual Studio 2005 (the processor and memory hog that it is!).
Martin Granell's Blog.
So it has happened again - Microsoft has developed a reasonable control for newbies that's not quite enterprise-class... that control is the ASP.NET 2.0 ObjectDataSource control. Feature rich - works great - but when it comes to complex objects... yeah... and what if you just want to bind to a singleton class that's passed in to a sub-control? Forgettaboutit.
I'm not the first to realize the pain... Brian Johnson, blogging at dotnet.org.za, actually wrote down the scenario I have shared with others in a blog post titled "ASP.NET 2 Data Binding Hell"... the scenario I'm talking about is that of Microsoft building a tool to a point... and then letting all the heavy hitters (senior-level devs / designers and above) down...
But did they?
Another blog entry I found, this one from Nikhil Kothari, a member of the ASP.NET team, talks about how to create a "custom" data source. Nikhil actually grazes the issue and demonstrates how to quickly create your own custom data source. Now - like Microsoft demos, this again tends to fall short - but it was enough to get the juices flowing on a couple of roadblocks I'm facing at work with our new enterprise framework. The one thing that would have been more helpful would have been if Nikhil at talked to some concepts on how to do some universal reuse here - in otherwords - how to create enhancements to the ObjectDataSource. But I'll get by.
When I've done some experimenting with this, and some additional hands-on, I'll blog more about my success/failure/frustrations.
Yesterday I needed to copy a database from my local machine to one of our development servers... both the server and my local pc are running SQL Server 2005 Dev.
My first thought was to simply detach my database from my local, copy to the server, and then reattach on the server. As I right-clicked on the database, the last menu item caught my eye... "Copy Database..." I decided to see what this would do for me and "click" I began what started out to be a pretty cool journey...
The Copy Database wizard allows you to do the copy by the "detatch and attach" method (which I was willing to do by hand) or using the SQL Management Object method, which I wanted to avoid... (I wasn't in the mood).
So I go through the wizard just fine - some of the things I had to do to actually make it work....
- These were both relatively "virgin" installs of SQL Server 05 Dev. So the first obstacle I ran into was Microsoft's new security initiative: Install with minimal risk exposure. In other words - I had to turn on Named Pipes and TCP/IP - on both servers.
- The account that the SQL Agent service was running under on my destination server had to be running under an account that had access to a file share to the database file on the source server. Now- this isn't just the proxy account you can configure in SQL Server... this is the actual account the service runs as.
Now - at this point - everything transferred over beautifully - it even created the server login for my database... at least I had thought.
So the next thing I encountered were reports that no one was able to log in to the database at its new location. During the copy process, it might create the login, it might maintain the integrity of your database... but it doesn't link the server login to the database user. In fact - it made it quite impossible because now my database user is configured "WITHOUT LOGIN" and I can't change it. I unwire some of the dependencies that would allow me to drop the user - drop it - and recreate it for my login.
But my problems don't end there - people still are not able to use the account - final resort - I reset the password. voila.
Now - for this last part - is that all I had to do was change the password? Did I have to go through dropping the account? I don't know. But doing all of this worked. If you have a different experience - I'd be interested in hearing about it.
Jon Henning, the lead for the DNN Core Team ClientAPI team and creator of the SOLPARTMENU featured in DNN, provided some thoughts on the future of the DotNetNuke ClientAPI and Microsoft's Atlas in his blog. Having seen a recent video from MSDN on Atlas, due out sometime this year, there's a lot of potential for creating a richer user experience on top of DNN as this year unfolds.
Here are his thoughts.
Here is the video/demo on Atlas on MSDN.
And finally - the Atlas web site.
I was going through the ASP.NET Forums for DotNetNuke and came across this post from "Sponge_Bob".
Is there anyway that I could run my own custom code on installation of my own PA's. I have a Module the need to be able to check and see if a certain Dll file is installed on the server before installing. Also I need the install to be able to check the version of the Dll file on the server an make certain the it is of the required version. Finally if the Dll not of the required version I want to point the person installing the PA to the place where to get the latest version of the install.
I have the option here to use Code (Preferred) or Sql to accomplish this task as I can make it so the required Dll I need to check for has its version entered into the database as it will be originally installed as a skin object.
It would be really sweet if Dnn could check the version of a Dll file before overwriting it with a dll file from a PA to make certain the it is not installing a Dll File of a lesser version. I am more that willing to write the code to accomplish this if the core team is willing to implement it.
So - I took a few minutes to help "Sponge_Bob" out. Here's my response...
I have to answer a question from Sponge_Bob. Dig yer username.
DotNetNuke provides an interface called IUpgradeable. The interface is implemented on your controller class for you module, similar to the way IPortable and ISearchable are implemented. There's very little written about this interface, however. The interface contract contains one method definition -
Function
UpgradeModule(ByVal Version As String) As String
So digging a little deeper...
The the lastest version of the Private Assembly Installer (V3) will check to see if your BusinessController class supports the IUpgradeable interface. If it does support it, it will iterate through an ArrayList of versions and call your UpgradeModule implementation for each version.
So - similar to the way DNN does version control - you can handle each incremental upgrade... a nice litte switch / case statement on the version string sent in to your method.
Now if a needed DLL is not there - you could always throw an error... OR - use your string return to add an error to the module install... (that would be the kindler, gentler approach). Use HTML within the string to color the text red so it stands out. Well - you should always return a string at least letting the user know that each upgrade was successful. Any string you return will be added to the module install / parsing results.
Seeing that this is done during install of your module or module upgrade - I wouldn't recommend putting any code in here to check for latest version or anything like that. However, what I would recommend is creating a separate edit control for "Check for New Version" which can do that. Unless it is mission critical to push latest versions (i.e. OS Security Patches, Virus Software) I personally prefer choosing an option like that.
I've just finished an article on how to customize the DotNetNuke installation package. I spent a few hours today digging around the DNN framework looking at how DNN does installs, how portal and host templates work, and how to specify which modules need to be installed... as well as how to not install some of the basic dnn modules. The more I dig around inside the framework, the more impressed I become with DNN (which says a lot since I've been impressed with DNN for quite some time). A lot of times when looking through other people's code, specifically some of the open source stuff out there, you get to a point and say - "ah - they're going to regret that they did that". I've been saying a lot of the opposite - "yeah - that makes sense" or "jeez, thats how I would have done it."
In this case, I was setting up a shared DNN development environment and realized that I hadn't "sanitized" my DNN 4 install yet... I ended up on a journey that has me dreaming about host templates, a common module set for all of our install points, and... and... our own portal that shows up after install with our own skin. Nice.
Take a read. It looks pretty lengthy - but i try to make it an easy read. If anything - especially for those developers visiting from work - you can get an idea of the capabilities this framework provides... and the power beneath your fingertips.
>> Article <<
More Posts
Next page »