Topic: school

2010.09.27

idempotency
gigabit ethernet => 128 megabytes ethernet => subtract tcp/http overhead => 50 megabytes data
multiple networks => prioritize commands over queries. (CustomerBecamePreferred vs GetAllCustomers
- network admins can prioritize data on separate networks.
- move time critical data to separate networks.
properties suck! accessing one property from another is not detministically determined. (time to access)
everytime you create an assocation their is cost down the road involved.
8 fallacies of distributed computing..
1 - the network is reliable
2 - latency isn't a problem
3 - bandwith isn't a problem
4 - the network is secure - security
5 - the topology wont change
6 - the admin will know what to do.
7 - transport cost isnt' a problem.
8 - the network is homogeneous
9 - the system is atomic
- centralized dba committee to sign off on schema changes.
- solution: internal loose coupling, modularize, design for scale out in advance, design for interaction with other software.
10 - the system is finished.
- maintenance costs more than development - design for maintenance
- the system is never finished. - design for upgrades.
- how will you upgrade the system.
11 - business logic can and should be centrailized
- re-use can be bad. context matters.
- single generic abstraction can make things more difficult, and can cause performance problems.
- more classes but more maintainable. the number of lines and coupling is reduced.
- generic abstractions can cause a lot of problems down the line. performance, maintainability. (small code base but much more complex to jump in)
- rules that change often can be segrated from rules that don't change often.
- we are taught that re-use is one of the greatest values in software development, in reality, it doesn't really help as much as we think it should.
solution:
- accept that business logic and validation will be distributed. plan for it.
big idea! (logical centralization and physical centralization is one to one.) at design time package rules enforcement together.
```
  - development time artifact
  - 12.sln that only has files that relate to rule "12"
      - ie. js file, cs file, and sql file all related to business rule 12...
  - business says we want to change rule # 12, then open up 12.sln and make those changes.???
  -- COULD THIS WORK??? 
  "-" any new sql files would have to have the next ordinal number to run migration scripts in order.
  "+" order migrations by timestamp would solve this problem.
```
best practices have yet to catch up to "best thinking"
tech cannot solve all problems
adding hardware doesn't necessarily help lunch
coupling
- afferent : depends on you
- efferent : you depend on
attempt to minimize afferent and efferent coupling.
zero coupling is not possible.
types of coupling for systems -platform -temporal -spatial

shared database is one of the worst forms of coupling.
- e.g. one system writes to database, and another reads from it.
make coupling visible. no more shared database, it's to hard to figure out the coupling when you do that.

1 danger is not being aware of the coupling by not seeing it.

Platform coupling

aka "interoperability"
using protocols only available on one platform. e.g. .NET [A] --------> JAVA [B]

solutions: - use standards based transfer protocol like http, tcp, udp, smtp, snmp

Temporal coupling

coupling to time.
e.g. service [A] sends message to service [B] and is waiting for a response. ---> TIMEOUT
stop trying to solve this type of coupling with multithreaded code.
most people should not be writing multi threaded code.
fowler says that only 4 people in the world know how to write proper multi threaded code.
publish/subscribe vs request/response

Spatial coupling

where in space are we depending on.
how closely tied am i in space to the physical machine where this is running.

summary

loose coupling is more than just a slogan.
coupling is a function of 5 diff dimensions.
- platform
- spatial
- temporal
- afferent
- efferent

mitigate temporal and spatial coupling by hosting in the same process. [A[B]] or [B[A]]

to do: - add zip file to google docs. - look up 8 fallacies of distributed computing.

Messaging:

reduces coupling
RPC crashes when load increases, messaging does not.
messaging: data can sit and wait to be processed in a queue
rpc: data cannot just sit, threads are blocked. as load increases, more threads are needed.
throttling: tell clients to go away. all threads are tied up, and dont intend to activate more.
- get to max throughput then stay there.
messaging: fire and forget
- we can set the max # of threads to process items off of a queue.
- store items in persistent storage (queue) which is cheap.
- this works well for commands.
- how do you deal with queries? the result needs to be returned in almost real-time.

"represent methods as messages" - Authorization : IHandles --> runs against every message for authorization.

DAY 2 --- 2010.09.28

role playing... kinky? - previous assumptions are incorrect - cross functional teams

benefits - same infrastructure - slightly different architecture - increase performance

cost of messaging:

- learning curve

10:15 am - need "correlation id" to tell us which response belongs to which request. - eg this is response for request 123 - now you can have multiple responses for a single request.

browser -> send request to server

        -> poll server for response with correlation id
        <- return responses for corellation id

Publish/Subscribe

events are...

todo:: - topic hierarchies

1:08 pm

Broker

a little bit of orchestration
a little bit of transformation
Bus
all about connecting event sources to event sinks events are core abstraction of bus arch style.
supports federated mode?
doesn't break service autonomey
disadvantages:
- harder to design

Service Orientation

4 tenets of service orientation

services are autonomous
services have explicit boundaries
services share contract & schema, not class or type
service interaction is controlled by policy

what is a service?

Service: technical authority for a specific business capability. Service: all data and business rules reside within the service.

what a service is not:

has only functionality
only has data is a database
- like [create, read, update, delete] entity

service examples

service deployments

many services can be deployed to the same box
many services can be deployed in the same app
many services can cooperate in a workflow
many services can be mashed up in the same page.

Availability

if subscriber goes down, messages are buffered to disk
- this can require a significant amount of disk space for an outage.
- this can take the publisher down.
schema: defines the logical message types
contract: provides additional tech info

The IT/Operations Service

responsible for keeping info flowing in the enterprise
owns all hosting technologies.
hosts all the business services
- configures it's own handlers to run first
  - authentication, authorization - LDAP/AD access
HR - employee hired, employee fired.
- provisions/de-provisions machines, accounts, etc.

todo:: * watch big bang theory * nerdtree vim plugin * tcommment vim plugin

DAY 3 = 2009.09.29

Review: (Hotel management system exercise) define the services + page composition really simplifies things. + avoid request/response between services. + avoid data duplication between service boundaries. + defining the services exercise helps identify the key business capabilities and fosters communication about the business. +

Services

Availability/Booking:
- booking[bookingid, customerid, daterange, roomclassification]
- tells how many of what room classifications are available at what times.
Facilities/Housekeeping
- tells what rooms are physically available
Customer Care
- tracks customers information like first name and last name.
Billing
- has price and bills customers for their bookings.

break.

Service Decomposition

large scale business capability that a service provides can be futher broken down.

Business Components - BC

[yes] multiple databases schemas, no fk relationships between schemas
- this mitigates deadlocks, and perf issues.
- referential integrity argument: delete => deletions usually mean the data will not be shown.
- duplicate data on different islands of data is a huge problem, this is why we do not duplicate data across service boundaries.
- data guy says do not delete data ever.
- we effectively scale our our databases by having multiple databases.

be aware of solutions masquerading as requirements - udi dahan

Services and transactions

"autonomous components" : AC

#

- responsible for one of more message types
- thing that performs a unit of work.
- is independently deployable, has it's own endpoint.
- common code that is needed to handle a specific message type not a single piece of code.
- running part of a service.

Layout

- multiple AC's
- single DB

usually when users report the system is slow, they are usually talking about a specific use case.
- using the typical monolithic architecture it is difficult to scale.
- when the business components are separated it is much easier to scale specific business capabilities.
from a runtime perspective there are more moving parts, and it can be difficult to monitor.

SOA building blocks summary

you see the AC's running. - autonomous components are the unit of deployment in SOA. - ac's takes responsibility for a specific set of messages types in the service. - ac uses the bus to communicate with other ac's - ac's can communicate using a message bus instead of a service bus. same technological bus, but used differently. - not all ac's require the same infrastructure, within a bc or across all ac's.

- eg. one ac my use an orm, another might write straight sql.

Service Structure

single user domain / multi-user collaborative domain.
model multi user data explicitly.

Queries in a collaborative multi user domain.

is this the right screen to be built for the purpose of collaboration?

tell users how stale the data is. "data correct as of 10 minutes ago."
eg bank statement without date.
decision support. all query screens are for decision support.
which decision(s) are you looking to support for each screen.
create separate screens for different levels of decision support.
persistent view model. what if we had a table for each screen that stores everything for that screen.
how do you keep the data in sync between the persistent view model tables and the source data.
no coupling between screens in the ui.
no fk relationships in the persisten view model tables.
- avoid calculations when doing queries.

Deployment and security

Role based security

different screens for different roles go to different tables. select permissions per role.
use the persistent view model to run some validation before issuing the command.

Commands

validation: is the input potentially good? structured correctly? ranges, lengths, etc?
- this is not a service, this is a function.
rules: should we do this?
- based on current system date.
- what the user saw is irrelevant.

model user intent - udi dahan we want to implement a fair system. if no one can see that a system is unfair, then it is fair enough.

HFT - high frequency trading.

good commands
- "thank you. your confirmation email will arrive shortly."
- inherently asynchronous
it's easier to validate a command, because
- we have context,
- less data
- more specific
in most cases it's difficult to justify many to many relationships within a bc.
document databases are great for persistent view models.

CQRS

Validation

schema

internal schema
- message types are commands
  - ChangeCustomerAddress
- faults
  - customer.... missed it.
external schema
- primarily based on events
  - CustomerBecamePreferred
  - OrderCancelled
- past tense
- something that has already occurred
- stay away from db thinking = no CRUD
  - think about business status changes
faults
- an enum value
  - CustomerNotFound
  - OrderLimitExceededForCustomer
- not an exception
  - we expect these things to occur
  - exceptions don't really work in async programming.

ultimately, we are refactoring a bus into a plane while it's driving. - udi dahan

Day 4 -- 2009.09.30

Review: Order cancelled - don't process the refund until the products have been returned.

- the cancel order command has no reason to fail now.
    - you can now cancel an order at anytime, for the customer to receive the refund they must return the product.
- the ship order command has no reason to fail now.
    - we do not need to ship the order if it is cancelled, but even if they cancel the order after it has been shipped that's ok.

if the new requirements don't fit the rules then
- your service boundaries might be incorrect.
- there is something missing in the requirements or process.
svn => single user domain
git => multi user domain

break

Long Running Processes

mortgage lending process
time is important. you don't see time, but it's important to model.

SAGA - handles long lived transaction

triggers ares messages
similar to message handlers
- can handle a number of different message types.
different from message handlers
- have state, message handlers don't
sql server isolation level repeatable read is like "select for update"
- i am going to execute this query again in this transaction and i would like to get back the same result set.
within a single business component

Testing

test it as a black box.
make sure you test the saga's because they are core, and they will change, and it's important to maintain the original business behaviour as it grows and changes.

workflow foundation WWF - transaction management is manual do it yourself - hard to test - no timeout mechanism

biztalk - can't actually represent this kind of functionality in a drag and drop orchestration. - can be useful when doing something procedural and synchronous. - latency tends to be slower. - haven't modeled time. - biztalk rules engine (bre)

the hard part

the easy part is using the building blocks.
hard part is getting them to tell you what the process needs to be.
interacting with legacy systems, each response becomes a message which triggers an activity.
- legacy systems are usually internal to a service.

summary

whenever you hear about workflows, orchestration etc then saga's are likely a candidate. if we see a saga handling 50 messages, that's usually a smell.

SCALING

SERVICE LAYER & DOMAIN MODEL

domain model - if you have complicated and ever changing business rules - if you have simple not-null checks and a couple of sums to calculate, a transaction script is a better bet. - independent of all concerns - poco - plain old c# objects - testing a connection between objects does not test any sort of behaviour - a unit is something that has a boundary. - you have been testing the innards of a unit - can be deployed multi tier - it's not about persistence, it's about behaviour.

service layer

manages persistence
- e.g. uses orm to persist domain model.

concurrency models

at least with eventual consistency we will effectively get true consistency.
with the current way we develop we do not have consistency.
the current domain models you've built are great for single user model, but not multi user model.
realistic concurrency
happy face
- you change the customers address
- i update the customers credit history
sad face
- you cancel an order
- i try to ship the order
only get one domain object.
- ask it to update itself
- domain object runs business rules
- eg. -- customer care using() {
```
  var customer = session.Get<Cusomter>(id);
  customer.MakePreferred();
```
  }
violation
- crosses 3 service boundaries
  - shipping, billing, customer care public void MakePreferred(){ foreach( var order in this.UnshippedOrders) foreach(var orderLine in order.OrderLines)
```
  orderLine.Discount(10.Percent());
```
    }

DAY 5 - 2009.10.01

review

out of order events

Building a saga for shipping

public class ShippingSagaData : IContainSagaData
{
    public virtual Guid Id{get;set;}
    public virtual string Originator {get;set;}
    public virtual string OriginalMessageId {get;set;}

    public virtual bool Billed {get;set;}
    public virtual bool Accepted {get;set;}
    public virtual Guid OrderId {get;set;}
}

public class ShippingSaga : Saga<ShippingSagaData>,
    IAmStartedByMessage<OrderAccepted>
    IAmStartedByMessage<OrderBilled>
{
    public override void ConfigureHowToFindSaga()
    {
        ConfigureMapping<OrderAccepted>(s =>s.OrderId, m =>m.OrderId);
        ConfigureMapping<OrderBilled>(s =>s.OrderId, m =>m.OrderId);
    }

    public void Handle(OrderAccepted message)
    { 
        Data.OrderId = message.OrderId;
        Data.Accepted = true;

        if(Data.Billed)
            MarkAsComplete();
        else
            RequestTimeout(TimeSpan.FromDays(7), "bill");
    }

    public void Handle(OrderBilled message)
    { 
        Data.OrderId = message.OrderId;
        Data.Billed = true;
        if(Data.Accepted)
            MarkAsComplete();
    }
}

public class OrderAccepted : IMessage
{
    public Guid OrderId{get;set;}

}

the tests...

[TestFixture]
public class ShippingTests
{
    [Test]
    public void WhenBillingArrivesAfterAcceptedSagaShouldComplete()
    {
        Test.Initialize();
        Test.Saga<ShippingSaga>()
        .WhenReceievesMessageFrom("client")
        .When(s =>s.Handle(new OrderAccepted())
        .AssertSagaCompletionIs(false)
        .When(s =>s.Handle(new OrderBilled())
        .AssertSagaCompletionIs(true);
    }

    [Test]
    public void WhenBillingArrivesBeforeAcceptedSagaShouldComplete()
    {
        Test.Initialize();
        Test.Saga<ShippingSaga>()
        .WhenReceievesMessageFrom("client")
        .When(s =>s.Handle(new OrderBilled())
        .AssertSagaCompletionIs(false)
        .When(s =>s.Handle(new OrderAccepted())
        .AssertSagaCompletionIs(true);
    }

    [Test]
    public void SagaRequestsBillingTimeout()
    {
        Test.Initialize();
        Test.Saga<ShippingSaga>()
            .WhenReceievesMessageFrom("client")
            .ExpectSend<TimeoutMessage>(m => true)
            .When(s =>s.Handle(new OrderAccepted())
            .When(s =>s.Timeout(null))
            .AssertSagaCompletionIs(true)
        ;
    }
}

Web

synchronous user login
caching
- keeping cache up to date across farms is challenging.
- cache invalidation
- track hit rate (number of times that item was in cache.)
  - hard to do
  - google, facebook, try to get a hit rate above 95%.
start using a cdn (content delivery network.)
- akamai
in the db, reads interfere with writes - hurts perf.

the number one reason why people are having scaling their web applications, is because they are ignoring the web. - udi dahan
90 % of page does not need to be rendered server side for every request.
different interface for search engine than for users.
- meta tags for search engine.
- ui for users.
persistent view model browser side using cookies.
ho

smart clients

use synchronization domains for thread synchronization.
provide information radiators for your knowledge workers.
client side domain model.
use property grid to display the status of objects without needing to attach a debugger.
cloning-proxies for views
- create a proxy for your views so that any data handed to the view can be cloned and bound to the view.

map display

see your wells on a map
constraints
- may receive 100's of updates per second at peak

What a week!! Well as I expected it was awesome, intense, and career altering. My wife, daughter and I traveled to Austin, TX to take part in the Nothin But .NET boot camp. We love Austin! It's an amazing city, and the people are fantastic. We often hear about how friendly Canadians are, but honestly it was unbelievable how kind people are in Austin. Every where you go people seem to be having fun and loving life. We spoke with bus drivers, cab drivers, people sitting at the bus stop, people at restaurants, people downtown.

My wife and daughter traveled the city using transit while I was in class all week. Everyday my wife would tell me stories about how nice people were to her. She dislikes taking transit here in Calgary but really found it fun and a pleasant experience in Austin. The buses are pretty cool. For $1 USD you get a ticket that lasts for 24 hours. When you get on the bus you swipe it at the front and get on. Bus drivers were so friendly and told stories and jokes during the ride. They really made an extra effort to help people in wheel chairs get on the bus. I'd never seen it before, but seems to be common. Awesome!

A lot of people we met in Austin, aren't originally from Austin. It seems that there are a lot of people currently migrating there. We were fortunate to meet a pretty cool cab driver named Ed. He moved to the states from Brazil and told us a lot about the city of Austin.

The course itself was awesome. There were students there from Austin, Winnipeg, Houston, Denver, Calgary, Louisiana, and even as far as Brazil. It's amazing how close you seem to get over such a short time. The same thing seemed to happen in the Calgary course. The collaborative environment really gets people to drop their defenses, open up and be comfortable with their current skill set knowing that it's only where they are today, but not where they'll be tomorrow.

There were so many great conversations about so many different topics. Everyone seemed to have great opinions and ideas to further and push the .NET community. Everyone in the room was definitely passionate about developing better software.

It was pretty cool to meet Scott Bellware, and here some of his ideas about .NET and software development in general. At first I didn't really get it, but by the last night it clicked in. I remember saying in my head... "He's right!" Scott is a super passionate person, and questions everything. I remember making it a motto to "question everything" and he truly does that well. The .NET community definitely needs a voice like Scott, to keep us all on our toes to make sure we pay attention to what we're doing, to expose more effective ways of doing things.

I realize that as a young dev that I'm part of the next generation, and it's important to me to learn from the trail blazers and continue pave new paths when they've finished.

One of the key things that Scott talked about was the concept of Solubility. "It's so easy to read that it melts into your brain." The new style of writing unit tests that target specific contexts make it so much easier to jump in to a specific context and continue to write in new chapters of the novel. I really enjoy reading code and tests that read like chapters from a novel. It's a higher level abstraction that allows me to focus on the problem domain rather then the technical details. Let the compiler do the interpreting...

It was so much fun being a teaching assistant. I love answering questions and helping out, at first I was pretty nervous but after being able to fix a few small issues I felt better. One of the things I learned this week was that when I didn't know the answer or wasn't sure it was important to make sure I made that known. The last thing I want to do is pass along incorrect information or pretend to know more than I do. I found that by just communicating that I didn't know the answer to a problem, that chances were that someone else in the room did. This is one of the reasons why the open, collaborative work space is so important. You can save so much time by just asking for help.

When we left Austin to go home, we were pretty sad to leave. The further we got away from Austin, the more we talked about how we could totally move to Austin... I can't wait to go back and help out at the next boot camp!

I'm so spoiled, spoiled, spoiled! Since our Nothin' But .NET experience last month, 3 lucky contestants including myself were issued our... "Cool Kids Club Card" from Mr. JP himself!

What I'm trying to say is that JP's taken an interest in our work and has been meeting with us as often as he can to discuss what we're currently working on. Sean calls it our "Agile Therapy".

If you're not already doing it, start your own "Agile Group Therapy" with your team where you can just discuss the up's and downs of the project (some people call it the sprint retrospective, but I think "Agile Group Therapy" is a cooler name.) I've found it super helpful to just get in touch and talk. (Why am I so touchy, feely today?)

Back to what I was saying. With our weekly discussions we get a preview of some of the events coming up in the life of JP. Without saying to much... Next year is going to be awesome! (Some might even say awesome, blossom!)

Time for a scolding...

Shame on you kids out in College Station, Texas. It's only a few weeks away and I can't believe the January boot camp hasn't sold out yet. Sure, sure it's a "fundamentals" course, but how many of us actually know the core "fundamentals" of .NET. Do you really know events, delegates, generics, app domains, threading... Do you really know it?

I surely don't... I've only started to skim the surface. This is definitely a course that I would love to attend. I'm telling you, that it's probably going to be a lot more "fundamental" then what you might think a typical .NET "fundamentals" course is all about. Shame on you, go Register!

Nothin But .NET Fundamentals Course Abstract

Nothin But .NET is expanding, and next year sounds like it's going to be an awesome year for new courses... and... *drumroll* ...new instructors!

No... no... not me... I've got a lot of growing up to do still! They're people you probably already know...

One last lesson from today's session is all about VIM.

Yesterday, I had my first trip out to the back country to go quading. This was my first experience out in the forest and on an ATV. It was a lot of fun, and a I'm feeling a little sore this morning from it. While we were out there I saw lots of wildlife. I don't know why but I felt like a little kid, I just couldn't stop staring at all the animals, the hills, the trees, the mountains and the forest. It was so very calming.

I thought I was going to spend the day contemplating how to deal with different events currently taking place in my life, but in the end the exact opposite happened. When you're out traversing the rough terrain you've got to put all your focus on over coming the land. I found the day to be just what I needed. I didn't need a day to think, I needed a day to not think and I feel like I have a clearer mind for it.

Within the first 10 minutes of traversing the terrain we came across a hunter gutting a dear on the back of his pickup. I had not seen a sight like this before, and it struck me deeply. To stare at the lifeless dear in the eyes as the hunter held a thick blade with blood dripping from his hands. It seemed barbaric, but in hindsight it's probably not as barbaric as how our meat gets processed in large factories. Either way I felt weird and fuzzy inside.

I'm still buzzing from the Nothin But .NET boot camp last week. My dreams last night can prove that I am. I'm not sure how many times I woke up in the night trying to recite the many different patterns we covered last week. I'm so looking forward to reviewing the countless hours of video from the week.

I think one of the aspects that made the week so enjoyable was the people. I've been to a couple of week long courses before where you show up at 9am and leave at 5pm. You don't feel like you really got to know anyone and you don't feel like like you've gained a lot from the experience. Besides touring a new city of course.

Because of the atmosphere that was put in the place it felt like we were all on a level playing field. It felt like we forgot about our fears of looking good or bad to our peers. It felt like a culture for learning. We all quickly shed our thick skins to really open ourselves up and be vulnerable, I think this made the week long event so much more fulfilling. I feel like I really got to know some really kind people, and hopefully some relationship from the event will blossom and continue to grow. I think I went into the event wanting to be the weakest link, so that I could grow to become the strongest one. In the end I don't really remember caring about which link in the chain I actually was.

So today is the first day back in the real world. Today is the day that I decide to either start practicing the gospel from last week, or I go back to the way things were. Today is a pivotal day!

When I reflect on this year, I think back to January when I remember feeling like I had no options. I felt like I had no idea what direction to take my career and where I would end up. I felt like I had few options. After less than a year of dedicated hard work, and extreme focus I feel like I'm starting to develop a few new options that seem to be gateways to long lasting career paths. I'm looking forward to reflecting 5 years from now after 5 solid years of dedicated hard work and extreme focus. I can only imagine good things.

Is it just me or am I really "touchy", "feely" today?

I know that I'm a junior developer. (Really I do!) I'm always going to be a junior developer, because of one thing. I don't care how good others are or how I rank up next to them. I'm always going to judge myself as junior compared to how good I can be. Dude, I couldn't care less how "good" I'm supposed to be at my age. There's always going to be someone faster, smarter, and better looking then me, and I'm cool with that.

Last week's experience at the "Nothin But .NET Boot Camp" was humbling. Why? Because I realized that my life is pretty darn good.

There are people who have faced so much more adversity then I will probably ever face, and wake up every morning enjoying life. This year I've been allowing myself to play the role of "please feel sorry for me, my life is so hard and I'm not sure I can make it." Last week, I realized that I need to smarten the heck up. My life is pretty good! The day to day battle with money, are trivial in the long run. I have a beautiful wife and daughter who love me dearly. How hard could my life possibly be?

"Success it not to be measured by one's status in society, but rather by the obstacles one over comes." - Booker T. Washington

Software development has taken over my life. It really has, I live, breathe, eat, and sleep thinking about patterns and curly braces. I hold my daughter in my arms as I'm rocking her to sleep thinking about code, and I feel guilty about doing it. Why is it that when my wife is talking to me about things that happened to her during the day, I'm drifting off thinking about how I need to be on the computer hammering out an idea for a block of code? Dude, I suck!

When I picture success...

- I picture myself sitting on pillows next to a warm fire with my wife staring me lovingly in the eyes. With our two healthy kids upstairs sleeping. Knowing that at that very moment, we had the freedom to do anything we wanted to do. That doesn't mean that we would, it's just the feeling of knowing that you could.

- I picture myself sitting on a rocking chair on our porch out on a hot summers day reading biographies of great people. Knowing that if at any moment my wife, children or grand children needed me, that I could be there for them in less than a second.

So if this is my picture of success, why am I not fully present when I'm with my family. Why am I consumed by code?

Software development was a means to an end. But it's becoming an end to what was meaningful. Coming home on Saturday to my wife and daughter brought me a lot of joy. After not having seen them for most of the week, it helped me realized how much I love and care for them and helped me remember why it is that I'm buried deep in books, and why I care so much about my craft of software development.

Here it is my final assignment for CMPP298 - Database Programming ADO.NET. Assignment is as follows:

"Create a Windows Forms application to support data maintenance (CRUD functionality) of Invoices per Vendor in the Payables database.

Make use of the Connection, Transaction, Command, and DataReader objects of the ADO.NET SQLClient Provider, not DataSets, DataTables, DataAdapters or the Configuration Wizard.

The user has to select a Vendor and then view, add, edit or delete Invoice records for the selected vendor. Changes have to be saved before the user can select another vendor.

Feel free to make use of a combination of UI controls such as Grids, TextBoxes, ComboBoxes, etc. "

The focus of the assignment was on ADO.NET 2.0. So my implementation of validation, UI, and domain is rather weak. I wanted to focus my attention on learning the core of ADO.NET 2.0 by learning more about...

IDbConnection: The interface the the abstract DbConnection type implements that all connection sub classes inherit from for each ado.net provider. (OleDb, Odbc, Oracle, SQL Server, MySQL)
IDbCommand: The interface that the abstract DbCommand type implements that all command objects inherit from.
DataTable: An in memory type used to store data in a tabular format.
IDataReader: A forward only, read-only type that allows you to iterate through a result set.
IDataParameter: This type represents a parameter for a IDbCommand object.
DbProviderFactories: Provides a factory method to return a DbProviderFactory abstract factory with methods such as CreateConnection.

IDbConnection

The IDbConnection interface demands a set of very simple methods and properties. I found "CreateCommand()" and "Open()" to be the most useful. Working against the interface as opposed to the concrete implementation simplified learning for myself. I was able to focus on the most important traits and behaviors that all Connection types share.

The "CreateCommand()" method returns a "IDbCommand" type that has it's connection property set to the the IDbConnection type that created it. Further reducing the amount of code needed to be written.

In my assignment I created and ConnectionFactory that has a single factory method that returns IDbConnection types. This separates where the connection string settings are retrieved from, from where the connection object is actually used.

using ( IDbConnection connection = _connectionFactory.Create( ) ) 
{
  IDbCommand command = connection.CreateCommand( );
  command.CommandText = sqlQuery;
  connection.Open( );
  IDataReader reader = command.ExecuteReader( );
  DataTable table = new DataTable( );
  table.Load( reader );
  return table;
}

You might have also noticed that IDbConnection also implements the IDisposable interface which allows me to use the type in a "using" block, which implicitly calls the Dispose method on the type. The Dispose method will then close the connection and clean up any other unmanaged resources.

IDbCommand

Types that implement the IDbCommand interface are used to execute "commands" against a database. This uses the Command pattern with 3 types of execute methods.

ExecuteReader: will execute a SQL command against the data source and return and IDataReader type that allows you to read through a result set.
ExecuteNonQuery: will execute a SQL command against the data source and returns the number of rows affected by the command.
ExecuteScalar: will execute a SQL command against the data source and returns the value in the first column of the first row.

The CommandText property is either the name of the stored procedure or the raw SQL to execute against the data source.

1   IDbCommand command = connection.CreateCommand( );
2   connection.Open( );
3   command.CommandText = query;
4   command.ExecuteNonQuery( );

DataTable

A DataTable is an in memory container for tabular data. It has a method named "Load()" that takes in an IDataReader type and will load the data table with the entire result set from the IDataReader. The overloaded Load method looks like this in Lutz Reflector:

A client usage of the Load method looks like this:

1   IDataReader reader = command.ExecuteReader( );
2   DataTable table = new DataTable( );
3   table.Load( reader );

IDataReader

Types that implement the IDataReader interface allow for forward-only reading through 1 or more result sets.

The IDataReader type also inherits from IDataRecord which exposes a set of get methods like "GetDecimal()", "GetBoolean()", "GetOrdinal()", "GetString()" to read out the value from each column as you iterate through the result set. This can definitely lead to some sloppy code such as...

1   private void SloppyReader( IDataReader reader ) {
2     string customerFirstName = !reader.IsDBNull( 1 ) ? reader.GetString( 1 ) : "Unknown";
3     string customerLastName = !reader.IsDBNull( 2 ) ? reader.GetString( 2 ) : "Unknown";            
4   }

IDataParameter

The IDataParameter is a parameter that is used by command objects. For example in the following SQL syntax @FirstName and @LastName are parameters. INSERT INTO Customer (FirstName, LastName) VALUES (@FirstName, @LastName); The following C# will insert the values "Mo" and "Khan" as the @FirstName, and @LastName parameters.

 1   IDbCommand command = connection.CreateCommand( );
 2   command.CommandText = "INSERT INTO Customer (FirstName, LastName) VALUES (@FirstName, @LastName);";
 3 
 4   IDataParameter commandParameter = command.CreateParameter( );
 5   commandParameter.ParameterName = "@FirstName";
 6   commandParameter.Value = "Mo";
 7   command.Parameters.Add( commandParameter );
 8 
 9   commandParameter = command.CreateParameter( );
10   commandParameter.ParameterName = "@LastName";
11   commandParameter.Value = "Khan";
12   command.Parameters.Add( commandParameter );

DbProviderFactories

This type has a factory method called "GetFactory()" which returns an abstract factory for creating connections, commands, adapters and parameters. To construct a DbProviderFactory you need to specify the ADO.NET provider to use. For example for the SQL Server Provider, you would use the invariant provider name of "System.Data.SqlClient". This works great with the ConnectionStringSettings section of the .config file. Using the ConnectionStringSettings type you can extract the connection string as well as the provider name and construct a DbProviderFactory without a re-compile. With a quick change to the .config I can switch from a SQL Server provider to an Oracle provider or MySQL provider, or an OleDb provider or and Odbc provider.

1   <connectionStrings>
2         <add name="PayablesConnection" 
3              connectionString="data source=(local);Integrated Security=SSPI;Initial Catalog=Payables;" 
4              providerName="System.Data.SqlClient" />
5    </connectionStrings>

The DbProviderFactory allows you to create database agnostic ADO.NET types through factory methods.

1   public DatabaseConnectionFactory( ConnectionStringSettings connectionStringSettings ) {
2       _settings = connectionStringSettings;
3   }
4 
5   public IDbConnection Create( ) {
6       IDbConnection connection = DbProviderFactories.GetFactory( _settings.ProviderName ).CreateConnection( );
7       connection.ConnectionString = _settings.ConnectionString;
8       return connection;
9   }

ADO.NET seems intimidating at first, but when you break it down piece by piece it's not such a scary beast. Now none of the examples provided use stored procedures, but using similar techniques discussed you could transition to stored procedures with ease and confidence.

Now as for DataSets, DataAdapters, CodeGen and Wizards... that's a whole other story!

Sait.Cmpp298.Assignment3.zip (2.35 MB)

Here is my latest assignment for my CMPP297 class at SAIT.

"Write a program to demonstrate that, as a high-priority thread executes, it will delay the execution of all lower-priority threads."

It took me a while to figure out a way to do this. At first I was thinking I would try to create a circular buffer with a consumer and producer object that was reading/writing to a shared data buffer. I was going to give one a higher priority then the other using the Priority property of the thread class.

This seemed like a lot of work, and I was wondering if I would be able to capture the actual requirements by doing this. I later thought back to the example we got in class about me and a friend running. I thought that if I could count the number of laps we each ran this would tell me who ran "faster".

My final solution I decided to just increment a counter in 2 different threads, giving each thread a different priority hoping that the higher priority thread would run "faster". Turns out it did, even on my P4 with hyper threading.

My Solution: (It's not elegant, but hopefully it conveys the message!)

 1 public class ThreadCounter 
 2 {
 3   public ThreadCounter( ) 
 4   {
 5       _continue = true;
 6   }
 7 
 8   public Boolean Continue 
 9   {
10       set { _continue = value; }
11   }
12 
13   public void Count( ) 
14   {
15       Int32 i__ = 0;
16 
17       while ( _continue ) 
18       {
19           i__++;
20       }
21       Console.WriteLine( "{0} has a count = {1}", Thread.CurrentThread.Name, i__ );
22   }
23 
24   private bool _continue;
25 }
26 
27 internal class Program 
28 {
29     private static void Main( ) 
30     {
31         const Int32 TimeoutMilliSeconds = 5000;
32         ThreadCounter priorityTest = new ThreadCounter( );
33         ThreadStart startDelegate = new ThreadStart( priorityTest.Count );
34 
35         Thread threadOne = new Thread( startDelegate );
36         threadOne.Name = "Highest Priority Thread";
37         Thread threadTwo = new Thread( startDelegate );
38         threadTwo.Name = "Lowest Priority Thread";
39 
40         threadOne.Priority = System.Threading.ThreadPriority.Highest;
41         threadTwo.Priority = System.Threading.ThreadPriority.Lowest;
42         threadOne.Start( );
43         threadTwo.Start( );
44 
45         Console.WriteLine( "Putting thread to sleep for 5 seconds... Please wait!" );
46         Thread.Sleep( TimeoutMilliSeconds );
47         priorityTest.Continue = false;
48         Console.ReadLine( );
49     }
50 }

The results look something like this... the counts will be different based on the speed of you CPU, whether you have a single core, duo core or hyper threading!

A deadlock occurs when two or more threads are trying to access the same data but are blocking each other from getting at the resources necessary to continue.

 1     internal class Program 
 2     {
 3         private static void Main( ) {
 4             Deadlocker deadlock = new Deadlocker( );
 5 
 6             Thread first = new Thread( new ThreadStart( deadlock.First ) );
 7             Thread second = new Thread( new ThreadStart( deadlock.Second ) );
 8             first.Start( );
 9             second.Start( );
10 
11             first.Join( );
12             second.Join( );
13         }
14     }
15 
16     /// <summary>
17     /// 1. First thread starts and locks resourceA
18     /// 2. Second thread starts and locks resourceB
19     /// 3. First thread blocks waiting for resourceB to be freed.
20     /// 4. Second thread blocks waiting for resourceA to be freed.
21     /// 5. The application stops in it's tracks.
22     /// </summary>
23     internal class Deadlocker {
24 
25         public void First( ) {
26             lock ( _resourceA ) {
27                 lock ( ( _resourceB ) ) {
28                     Console.WriteLine( "First" );
29                 }
30             }
31         }
32 
33         public void Second( ) {
34             lock ( _resourceB ) {
35                 lock ( ( _resourceA ) ) {
36                     Console.WriteLine( "Second" );
37                 }
38             }
39         }
40 
41         private object _resourceA = new object( );
42         private object _resourceB = new object( );
43     }

Remember that lock implicitly translates too...

 1     lock ( _resourceB ) {
 2         
 3     }
 4     
 5     // translates too
 6     Monitor.Enter( _resourceB);
 7     try {
 8         
 9     }
10     finally {
11         Monitor.Exit( _resourceB );
12     }

Instead of using Monitor.Enter(), you can use Monitor.TryEnter();

1     if ( !Monitor.TryEnter( _resourceB, TimeSpan.FromSeconds( 5 ) ) ) {
2         throw new TimeoutException( );
3     }
4     try {
5         
6     }
7     finally {
8         Monitor.Exit( _resourceB );
9     }

For more info check out...

MCTS Self-Paced Training Kit (Exam 70-536): Microsoft .NET Framework 2.0 Application Development Foundation by Tony Northrup, Shawn Wildermuth, Bill Ryan

Archive

topics

September 2011

July 2011

June 2011

May 2011

February 2011

January 2011

December 2010

November 2010

October 2010

September 2010

April 2010

July 2009

April 2009

March 2009

February 2009

December 2008

November 2008

October 2008

September 2008

August 2008

July 2008

June 2008

May 2008

April 2008

March 2008

February 2008

January 2008

December 2007

November 2007

October 2007

September 2007

August 2007

July 2007

June 2007

May 2007

March 2007

Topic: school

2010.09.27

8 fallacies of distributed computing..

1 danger is not being aware of the coupling by not seeing it.

Platform coupling

Temporal coupling

Spatial coupling

summary

Messaging:

DAY 2 --- 2010.09.28

Publish/Subscribe

Broker

Bus

Service Orientation

4 tenets of service orientation

what is a service?

what a service is not:

service examples

service deployments

Availability

The IT/Operations Service

DAY 3 = 2009.09.29

Services

Service Decomposition

Business Components - BC

Services and transactions

#

Layout

SOA building blocks summary

Service Structure

Queries in a collaborative multi user domain.

Deployment and security

Role based security

Commands

CQRS

Validation

schema

Day 4 -- 2009.09.30

Long Running Processes

SAGA - handles long lived transaction

Testing

the hard part