White Wizard @ Leavewizard.com | Continuous improvement catalyst | Agile, LeanStartUp, Kanban, SystemsThinking & Stewardship
Tuesday, 25 November 2008
Scrum and The Enterprise (Panel - Day 1 afternoon)
In the afternoon of day 1 I chose to attend a discussion about Scrum and the enterprise with Bas Vodde, Steve Greene, Nigel Baker and Robin Dymond. I found it very useful as I am interested in applying Scrum for large scale projects and organisations. Unfortunately my notes from the session could have been a lot better but I focused mainly on listening and therefore it took me some time to connect all these notes in my notebook.
A book by John Kotter has been mentioned with title “Leading Change” which was recommended as useful read for discovering and being aware of 7 key errors in the process of change (I intend to buy it). Bas said that Nokia scrum implementation was a bottom up one which I found interesting as I always thought that such a large scale change must be driven from the top. A couple of the speakers agreed on the advice that coaching should be done on the basis of one person at a time and this is one of the reasons that changing the organisation is slow. Somebody said that culture and cultural change is overestimated. A key thing is to make sure testing drives development rather than waiting for story completion.
An interesting topic was how to do planning when teams have many related functionality and dependencies. The suggestion was that all dependencies are displayed on a white board and teams then link with other teams and list commitments. When you have such a large implementation you end up with many teams, the scrum masters of these teams form teams as well and they have scrum masters as well which is a reason for having the 3S meeting(Scrum of Scrum of Scrums) which was recommended to be done once per week.
In terms of keeping team members engaged it was suggested doing regular innovative workshops which should include the business owners and also having OpenSpace days to raise Scrum awareness and to support the coaching community. In terms of working with HR the advice given was to have Scrum roles – Scrum team member, Scrum Master and Product Owner and define competencies for these roles. I guess the challenge is to convince HR and Senior Management that this is needed.
There was the inevitable question about architecture. The common opinion was that architecture is best to emerge – Google walking skeleton.
A huge importance is being placed on having a skilled Agile Coach not just a volunteer. I would say it is the same with Scrum Masters and Product Owners. It is difficult to ask people who are not passionate about Scrum to show the same energy and engagement and deliver the same leadership as an experienced and successful practitioner.
Somebody also said that while agile coaches are skilled change agents, traditional PMs are only skilled to tell lies ;)
The change to scrum is a change from activity based hierarchy to value based streams. For the enterprise it was suggested that Scrum/Lean combination makes sense. Scrum provides the tools; Lean makes sure waste is eliminated.(or so my notes say).
Labels:
agile,
scrum,
Scrum Gathering,
ScrumAlliance,
Stockholm
Friday, 7 November 2008
Scrum –dominant in the Agile Community
(Day 1 of Scrum Gathering Autumn 2008)
Jeff Sutherland started the day with a resourceful session about the long way Scrum has gone to become the dominant agile approach. Following are my notes so most of the information comes directly from Jeff except where notes weren’t enough I tried filling in the gaps. If you find some of this confusing it is because of me ;)
Interesting facts in the presentation included results of a research showing that 49% of Agile implementations are Scrum based and another 22% are Scrum/XP. Jeff also touched on productivity improvement levels and how Scrum adoption can affect them. For example an excellent Scrum implementation should result in 400% revenue increase; Good Scrum would give about 300%; Pretty good about 150-200% and something he called ScrumButt just mere 0-35%. Just for clarification ScrumButt also referred to by other authors as MURSC, ScrummerFall, and others is an implementation that doesn’t fully follow the rules of the framework and as result this affects the level of productivity and profitability increase.
I liked the suggestion about what kind of goals work better. I am sure many company directors or owners will say their goal is to meet revenue targets. In fact goals that are likely to work better should focus on customer satisfaction and employee happiness like for example “customers to be ecstatic” or “make life of employees better”.
One of the biggest problems during adoption phase is the fact that traditional managers and often the whole institution have problem with self-organizing teams. They’ll try to do control planning which kills self organization. They find it difficult to abandon command and control which is likely to crush the team. They carry on doing isolated activities and promote lack of transparency which undermines self organization. They object impediments removal which delays progress. So no wonder why 70% of change initiatives fail! This highlights a bigger issue – lack of sense of urgency among leadership.
Jeff suggested using the enhanced Nokia test for finding out if a team is doing Scrum or not. There has been a vast discussion on scrumdevelopment about how and if such tests are useful and while generally I do not like the idea of measuring using the enhanced test for initial assessment seems a really good approach.
A look into the most productive ever projects (where productivity was documented) shows that the most productive one – Borland Quattro failed. As for the second most productive - a project at Motorolla, the team “died” after the company went for a 3rd party product. This comes to show that productivity itself does not guarantee success.
Key factor to success is actually failure. Fast and furious failure plus good retrospectives drives the super performance. Einstein says that one who has never made a mistake has never tried anything new.
At the end of the presentation Jeff talked about well run teams. According to him having a well maintained product backlog, minimized work in progress and the ability to stop when something is wrong are the top factors leading to good Scrum.
Labels:
agile,
scrum,
Scrum Gathering,
ScrumAlliance,
Stockholm
Friday, 24 October 2008
2008 Stockholm Scrum Gathering
Just back from Stockholm and WOW what can I say.. being my first gathering I was impressed with many things. People in the first place. People willing to work together, people ready to form functional teams right from the start. People understanding communication and collaboration. People willing to improve regardless of where they are at the moment. I know gathering "veterans" will have suggestions for improvement but I thought organization and quality of the session was just right and I managed to make tons of notes with ideas, recommendations and contacts as well!
A few good ideas, perhaps initiatives and events also emerged during the evenings even though the price of beer was a bit too high but overall Stockholm is a great place and the stay was enjoyable.
I am certainly going to review my notes, put them in order and publish all the (hopefully) useful ideas for the benefit of the community.
All that when I find enough time between my contributions to transform the way we work!
Labels:
scrum,
Scrum Gathering,
ScrumAlliance,
Stockholm
Thursday, 16 October 2008
Team Dysfunctions And Scrum (2)
ScrumAlliance.org have decided to publish my article on team dysfunctions and scrum here. I really like the edited version as it actually seems more "alive" ;)
Monday, 6 October 2008
NxtGenUG October Event - Southampton
This month NxtGen in Southampton turns 1 year and to mark the event Rich Allen invited me to do a Scrum talk and game together.
Details about the event are here.
I will report back with more information and maybe pictures ;)
Details about the event are here.
I will report back with more information and maybe pictures ;)
Friday, 15 August 2008
SQL Server Best Practices 1. Big sets, small sets
SQL Server Best Practices 1. Big sets, small sets
There are two things to remember about sets: the bigger the number of rows we’re working with the slower our query will be and the more columns we return the less efficient transfer of data will be.
I am going to use the Northgale database, for those unfamiliar with it is a script developed by Erland Sommarskog that creates a huge database based on Northwind DB (provided by Microsoft). Just to illustrate the size Northgale has about 6,000 products, 344,000 orders and 2.3 million rows in Order details. I intend to use relatively big sets in order to illustrate better the differences in performance.
Consider the following query:
The returned set from the query above contains about 1.5 million rows and takes about 1 minute and 36 seconds to complete on my machine.
Clearly not fast enough.
The first thing to always remember is that the WHERE clauses are applied at the end. This means that before we apply the where clauses in this example we will always have to use all rows in the tables to perform the joins. For example the first join will join all products with all order details assuming every order detail has product id. Then we’ll have the entire order details table (2.3 million rows) to perform a join with orders and then another one with customers. No wonder the query takes 96 seconds!
So, what can we do about this?
The smaller the number of rows you work with the better. In the example above we should be looking at restricting the set as early as possible.
What we have done with this one is to move all conditions from the where section to become conditions in the joins. And we’re trying to apply them as early as possible – ie the first time we use the table they apply to.
This reduces the execution time by 30 seconds to 1 minute and 6 seconds.
In Northgale Products table has 10 columns with total size 87 bytes per row. Order adds 200 bytes per row, Order detail 22 and Customer 268. This makes a total of 577 bytes per row. If the query returns 1.5 million rows this gives a massive 825 megabytes return set size. Let’s see what happens if we return only a small number of columns.
This one only takes 20 seconds which represents about 400% improvement compared to our initial query.
As mentioned earlier I have chosen a huge set to illustrate the differences in speed of execution. In reality it is very unlikely that you will need to return 1.5 million rows. It isn’t exactly practical even for one of those large “end of year” bank reports which we used to generate years ago to allow bank officers to waste 20,000 pages to print them. And I am sure that even if you have to select more than a million rows you will most likely need to use paging.
Of course with smaller sets the improvements in performance will be smaller but the changes we made to this query when applied would be valid for any other query even though speed improvements may not be that obvious. If your set is really small the optimizer will notice that and will not use indexes as scanning your small set will be faster.
Hopefully you will find this practice useful and will remember the two advices: restrict sets as early as possible and return only the columns you need.
There are two things to remember about sets: the bigger the number of rows we’re working with the slower our query will be and the more columns we return the less efficient transfer of data will be.
I am going to use the Northgale database, for those unfamiliar with it is a script developed by Erland Sommarskog that creates a huge database based on Northwind DB (provided by Microsoft). Just to illustrate the size Northgale has about 6,000 products, 344,000 orders and 2.3 million rows in Order details. I intend to use relatively big sets in order to illustrate better the differences in performance.
Consider the following query:
SELECT *
FROM Products p
INNER JOIN [Order Details] od ON p.ProductId = od.ProductId
INNER JOIN Orders o ON o.OrderID = od.OrderID
INNER JOIN Customers c ON c.CustomerId = o.CustomerId
WHERE Discontinued = 0
AND o.RequiredDate > '05/04/1997'
AND od.Quantity * od.UnitPrice > 100
The returned set from the query above contains about 1.5 million rows and takes about 1 minute and 36 seconds to complete on my machine.
Clearly not fast enough.
The first thing to always remember is that the WHERE clauses are applied at the end. This means that before we apply the where clauses in this example we will always have to use all rows in the tables to perform the joins. For example the first join will join all products with all order details assuming every order detail has product id. Then we’ll have the entire order details table (2.3 million rows) to perform a join with orders and then another one with customers. No wonder the query takes 96 seconds!
So, what can we do about this?
The smaller the number of rows you work with the better. In the example above we should be looking at restricting the set as early as possible.
SELECT *
FROM Products p
INNER JOIN [Order Details] od ON p.ProductId = od.ProductId
AND Discontinued = 0
AND od.Quantity * od.UnitPrice > 100
INNER JOIN Orders o ON o.OrderID = od.OrderID
AND o.RequiredDate > '05/04/1997'
INNER JOIN Customers c ON c.CustomerId = o.CustomerId
What we have done with this one is to move all conditions from the where section to become conditions in the joins. And we’re trying to apply them as early as possible – ie the first time we use the table they apply to.
This reduces the execution time by 30 seconds to 1 minute and 6 seconds.
In Northgale Products table has 10 columns with total size 87 bytes per row. Order adds 200 bytes per row, Order detail 22 and Customer 268. This makes a total of 577 bytes per row. If the query returns 1.5 million rows this gives a massive 825 megabytes return set size. Let’s see what happens if we return only a small number of columns.
SELECT ProductName, CompanyName, AmountPaid = od.Quantity * od.UnitPrice
FROM Products p
INNER JOIN [Order Details] od ON p.ProductId = od.ProductId
AND Discontinued = 0
AND od.Quantity * od.UnitPrice > 100
INNER JOIN Orders o ON o.OrderID = od.OrderID
AND o.RequiredDate > '05/04/1997'
INNER JOIN Customers c ON c.CustomerId = o.CustomerId
This one only takes 20 seconds which represents about 400% improvement compared to our initial query.
As mentioned earlier I have chosen a huge set to illustrate the differences in speed of execution. In reality it is very unlikely that you will need to return 1.5 million rows. It isn’t exactly practical even for one of those large “end of year” bank reports which we used to generate years ago to allow bank officers to waste 20,000 pages to print them. And I am sure that even if you have to select more than a million rows you will most likely need to use paging.
Of course with smaller sets the improvements in performance will be smaller but the changes we made to this query when applied would be valid for any other query even though speed improvements may not be that obvious. If your set is really small the optimizer will notice that and will not use indexes as scanning your small set will be faster.
Hopefully you will find this practice useful and will remember the two advices: restrict sets as early as possible and return only the columns you need.
Friday, 8 August 2008
Optimize me!
Finally, yesterday I decided to actually read one of the newsletters I get flooded with by SqlServerCentral and then I discovered a really good tsql script to help with identifying slow queries.
The original post is here but as with everything I like I have made a few small changes and would like to post it so I can find it quickly when I need it. Oh, and I might read these newsletters a bit more often from now on ;)
Please note: The code below will only work on Sql Server 2005 or above .
The original post is here but as with everything I like I have made a few small changes and would like to post it so I can find it quickly when I need it. Oh, and I might read these newsletters a bit more often from now on ;)
Please note: The code below will only work on Sql Server 2005 or above .
USE master
GO
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[dba_QueryTimeDelta]') AND type in (N'P', N'PC'))
EXEC sys.sp_executesql N'CREATE PROCEDURE [dbo].[dba_QueryTimeDelta] AS SELECT ''this is just temp. please replace with actual code.'''
GO
ALTER PROC [dbo].[dba_QueryTimeDelta]
@DatabaseName nvarchar(255) = ''
AS
/*----------------------------------------------------------------------
Purpose: Identify queries that are running slower than normal , when taking into account IO volumes.
------------------------------------------------------------------------
Parameters: @DatabaseName, optional
Revision History:
13/01/2008 Ian_Stirk@yahoo.com Initial version
08/08/2008 plamen.balkanski.net Added filtering by Database name
Example Usage:
1. exec YourServerName.master.dbo.dba_QueryTimeDelta
2. exec master.dbo.dba_QueryTimeDelta 'AllItems20080808'
----------------------------------------------------------------------*/
BEGIN
-- Do not lock anything, and do not get held up by any locks.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
-- Identify queries running slower than normal.
SELECT TOP 100 [Runs] = qs.execution_count
, [Total time] = qs.total_worker_time - qs.last_worker_time
, [Avg time] = (qs.total_worker_time - qs.last_worker_time) /(qs.execution_count - 1)
, [Last time] = qs.last_worker_time
, [Time Deviation] = (qs.last_worker_time - ((qs.total_worker_time - qs.last_worker_time) /(qs.execution_count - 1)))
, [% Time Deviation] = CASE WHEN qs.last_worker_time = 0 THEN 100 ELSE (qs.last_worker_time - ((qs.total_worker_time - qs.last_worker_time) /(qs.execution_count - 1)))* 100 END /(((qs.total_worker_time - qs.last_worker_time) /(qs.execution_count - 1)))
, [Last IO] = last_logical_reads + last_logical_writes + last_physical_reads , [Avg IO] = ((total_logical_reads + total_logical_writes + total_physical_reads) - (last_logical_reads + last_logical_writes + last_physical_reads)) / (qs.execution_count - 1)
, [Individual Query] = SUBSTRING (qt.text,qs.statement_start_offset/2, (CASE WHEN qs.statement_end_offset = -1 THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2 ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)
, [Parent Query] = qt.text
, [DatabaseName] = DB_NAME(qt.dbid)
INTO #SlowQueries
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.plan_handle) qt
WHERE qs.execution_count > 1
AND DB_NAME(qt.dbid) = CASE WHEN @DatabaseName='' THEN DB_NAME(qt.dbid) ELSE @DatabaseName END
ORDER BY [% Time Deviation] DESC
-- Calculate the [IO Deviation] and [% IO Deviation].
-- Negative values means we did less I/O than average.
SELECT TOP 100 [Runs] ,
[Avg time] ,
[Last time] ,
[Time Deviation] ,
[% Time Deviation] ,
[Last IO] ,
[Avg IO] ,
[IO Deviation] = [Last IO] - [Avg IO] ,
[% IO Deviation] = CASE WHEN [Avg IO] = 0 THEN 0 ELSE ([Last IO]- [Avg IO]) * 100 / [Avg IO] END ,
[Individual Query] ,
[Parent Query] ,
[DatabaseName]
INTO #SlowQueriesByIO
FROM #SlowQueries
ORDER BY [% Time Deviation] DESC
-- Extract items where [% Time deviation] less [% IO deviation] is 'large'
-- These queries are slow running, even when we take into account IO deviation.
SELECT TOP 100 [Runs] ,
[Avg time] ,
[Last time] ,
[Time Deviation] ,
[% Time Deviation] ,
[Last IO] ,
[Avg IO] ,
[IO Deviation] ,
[% IO Deviation] ,
[Impedance] = [% Time Deviation] - [% IO Deviation] ,
[Individual Query] ,
[Parent Query] ,
[DatabaseName]
FROM #SlowQueriesByIO
WHERE [% Time Deviation] - [% IO Deviation] > 20
ORDER BY [Impedance] DESC
-- Tidy up.
DROP TABLE #SlowQueries
DROP TABLE #SlowQueriesByIO
END
-- Test script
-- EXEC master.dbo.dba_QueryTimeDelta 'AllItems20080808'
Labels:
Database,
DBA,
Optimization,
SQL Server,
Stored procedure,
T-SQL,
TSQL
Wednesday, 6 August 2008
Agile Presentations - free or paid?
I was amazed (if not shocked) to find this website today: http://www.agile-software-development.com
It is selling presentations to help with Agile.
I refuse to accept this.
The Agile community, as is obvious when you look at the search results returned by your favourite search engine, has been trying for years to give these things for free. Here's some evidence:
http://www.mountaingoatsoftware.com/presentations
www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf
http://richardsbraindump.blogspot.com/2007/07/agile-presentations.html
and these are just a few of thousands of examples.
And after all when trying to explain Scrum/XP or agile pracitces like user stories you need to understand it otherwise how do you expect to change others view about the way it is done? And if you do understand it then you will be able to do the presentation yourself!? Even if you struggle for ideas about your presentation simply look for other people's presentation or articles or even books(!) that are available out there. For free. Not for the "price of a light meal" (10 GBP!?)
PS. I promise to find a more positive topic for my next post ;)
It is selling presentations to help with Agile.
I refuse to accept this.
The Agile community, as is obvious when you look at the search results returned by your favourite search engine, has been trying for years to give these things for free. Here's some evidence:
http://www.mountaingoatsoftware.com/presentations
www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf
http://richardsbraindump.blogspot.com/2007/07/agile-presentations.html
and these are just a few of thousands of examples.
And after all when trying to explain Scrum/XP or agile pracitces like user stories you need to understand it otherwise how do you expect to change others view about the way it is done? And if you do understand it then you will be able to do the presentation yourself!? Even if you struggle for ideas about your presentation simply look for other people's presentation or articles or even books(!) that are available out there. For free. Not for the "price of a light meal" (10 GBP!?)
PS. I promise to find a more positive topic for my next post ;)
Labels:
agile,
presentations,
scrum,
user stories,
XP
Wednesday, 16 July 2008
Scrum Gurus Speak
I am sure not everyone can spend much time reading on scrum development so I thought it might be useful to post this list on here. Will make my life easier also whenever I need to use it ;)
To tolerate a problem is to insist on it. ~ Ron Jeffries
A tool is nothing without a skilled artisan to handle it. ~ Tobias
The practices are not the knowing: they are a path to the knowing. ~Ron Jeffries~
No matter the circumstances you can always improve. You can always start improving with yourself. You can always start improving today. ~Kent Beck~
Anyone who uses the term "resource" when referencing people has to put $1 in the "inappropriate comment" jar! ~ Ken Schwaber
Reality is what we make, not just what we live with. ~Ron Jeffries~
Make it real or else forget about it -- Carlos Santana
Without prioritization, nothing is a priority. ~??
Agility might be said to be about encountering all the problems so early and so often that the effort to fix them is less than the pain of enduring them. ~Ron Jeffries~
"Everything is simple, until you have to do it yourself". ~??
"Once you add people to even the simplest problem, it can get pretty messy in a hurry". ~??
The practices are not the knowing: they are a path to the knowing. ~Ron Jeffries~
Ability to estimate correctly is not an 'ability' ... it is a fluke and lucky guess ~Roy Morien~
A dead scrum master is no good to anyone. ~??
If we're not shipping our software when it's ready, it's poor business practice. If we're not sure whether our software is ready, it's poor software practice. ~Ron Jeffries
Once you start measuring something, you can easily end up in a situation where the measurement itself starts influencing the things you want to measure. – Wolfgang
"A team consists of people under pressure to do their best. Conflict is natural and the team needs to know how to deal with the conflict and have resources to draw on when needed." --Ken Schwaber
Don’t worry about the enterprise rollout today when you have not started even one project. ~??
To tolerate a problem is to insist on it. ~ Ron Jeffries
A tool is nothing without a skilled artisan to handle it. ~ Tobias
The practices are not the knowing: they are a path to the knowing. ~Ron Jeffries~
No matter the circumstances you can always improve. You can always start improving with yourself. You can always start improving today. ~Kent Beck~
Anyone who uses the term "resource" when referencing people has to put $1 in the "inappropriate comment" jar! ~ Ken Schwaber
Reality is what we make, not just what we live with. ~Ron Jeffries~
Make it real or else forget about it -- Carlos Santana
Without prioritization, nothing is a priority. ~??
Agility might be said to be about encountering all the problems so early and so often that the effort to fix them is less than the pain of enduring them. ~Ron Jeffries~
"Everything is simple, until you have to do it yourself". ~??
"Once you add people to even the simplest problem, it can get pretty messy in a hurry". ~??
The practices are not the knowing: they are a path to the knowing. ~Ron Jeffries~
Ability to estimate correctly is not an 'ability' ... it is a fluke and lucky guess ~Roy Morien~
A dead scrum master is no good to anyone. ~??
If we're not shipping our software when it's ready, it's poor business practice. If we're not sure whether our software is ready, it's poor software practice. ~Ron Jeffries
Once you start measuring something, you can easily end up in a situation where the measurement itself starts influencing the things you want to measure. – Wolfgang
"A team consists of people under pressure to do their best. Conflict is natural and the team needs to know how to deal with the conflict and have resources to draw on when needed." --Ken Schwaber
Don’t worry about the enterprise rollout today when you have not started even one project. ~??
Friday, 11 July 2008
Team Dysfunctions and Scrum
I firmly believe and support the idea that Scrum is a management framework and it is not supposed to sort all problems on Earth. I also think I understand well that Scrum was created to solve “a problem” and it duly delivers the goal. This article however is not about how great Scrum is. This article is about a common problem which I believe most if not all Scrum implementations experience. What I am talking about is the problem of overcoming team dysfunctions.
It is known that a Scrum implementation, especially in its early days is expected to and almost always reveals hidden problems and issues that usually cause one or more major team dysfunctions. Finding ways to overcome these dysfunctions is the main goal of this article.
A beautiful beginning
So, you have introduced Scrum and you are working with the first ever Scrum team in your organisation. Luckily all team members seem to be interested and attend all the meetings. You have managed to get a room with whiteboards where the Daily stand-up takes place and you are also able to book an appropriately equipped room for reviews and plannings. Everyone in the organisation seems to be quite excited by the use of index cards and blu-tac which creates an energetic culture and you really feel good about the way things are going.
Retrospectives though feel a bit weird although you can’t quite figure out why. And then suddenly it all kicks-off with an against the rules reaction of a team member during a retrospective meeting after a disastrous review. The team achieved mere 5 points out of 12 planned. Tom, a senior developer says that Chris, a test engineer wouldn’t let the rest of the team do any testing. As a result the team ended up with half of the stories in the final stages of testing just because test engineers would not trust the rest of the team members to provide adequate testing. Surprised by the fact that he is being blamed Chris replies that no one else is qualified to do “proper testing” and anyway testing is not being paid enough attention in Scrum so he believes quality is going down. You finally realise that you need to stop this and you speak about how blaming is not allowed and accent on the positives of identifying a problem which the team now needs to focus on resolving.
The event throws you a little bit out of focus. You see this as a technical issue where testing needs to be automated to reduce the workload of testers. You start reading about test automation and distribute various useful web casts and studies about it. You even change your Scrum training and do it specifically for your test engineers. They seem to understand more and more about cross functional teams and the need of automation. You forget about that retrospective for now.
Although the problem may disappear unfortunately the issue is a lot bigger. Your test engineers do not trust the rest of the team and perhaps vice versa. Lack of trust is usually caused by the unwillingness to be vulnerable. Team members are not open with one another and are afraid to talk about their mistakes and weaknesses. At the same time the fact that your previous retrospectives seemed to be running well suggests that the team might be avoiding conflict. Because of the lack of trust the Team cannot engage in open debates, instead they resort to indirect discussions and shielded comments. Fear of conflict usually only postpones the conflict and once the disappointment reaches certain level the reaction will cause far bigger problem than if the conflict had arisen earlier, or even better resolved in a team debate.
Definitely getting better
Your testers now seemed to have finally got it. At least they no longer question automated testing and seem to be spending time on finding an appropriate tool to use. You still have slight concerns about the way the team reaches quick agreement during planning although you can easily explain this with the team getting more and more used to Scrum. Finally you’re edging close to completing the list of stories in the backlog so you start talking about release. On what seem to certainly be the last planning meeting you start discussing final steps to produce deliverable product which appears to cause a little more discomfort compared to previous meetings. Most unexpectedly Jane, your other test engineer, states out of the blue that she doesn’t care if the team decides that the product is ready for release as she anyway isn’t convinced that enough testing has been done. This sparks Chris’s comment that he wouldn’t accept responsibility for this product as he has never done less testing nor seen a most weird way to develop something. Peter who is a web developer in the team responds that test engineers still do not trust the rest of the team so how can more testing be done when only two team members are testing.
You are disappointed and very concerned that what seemed to be a maturing Scrum team suddenly doesn’t even act as a team. You spend the rest of the day reading about teamwork and various ways to improve it. You are beginning to realise that without sorting out underlying problems it would be difficult to improve practices and productivity.
When a team lacks conflict or what is usually called healthy conflict this could lead to lack of commitment. Because team members cannot air their opinions in an open discussion they rarely if at all commit to decision although they may demonstrate agreement in meetings. Teams members feel as if their opinions do not matter therefore find it difficult to support team decisions. Unfortunately when this is the case it triggers an even bigger problem – avoidance of accountability. Without committing to a well understood plan of action, it doesn’t matter how focused your people are they often fail to call their peers on actions and behaviors that seem counterproductive to the good of the team.
Did you see it coming?
The next morning things seem to become even worse. On the daily scrum Pete, senior developer shows impatience while Chris is talking, interrupts him to make a point about team members not following practices which he believes are a must. Tom also joins in by ignoring the usual order and stating that he feels it became more difficult for him to do “proper” development and increase his expertise. Jane follows to confirm what is obvious to the rest – she doesn’t enjoy working “that way” because she doesn’t see any career progression opportunities with Scrum.
You feel betrayed. It seems like the hundreds of hours spent on convincing people and setting up the basics have disappeared. The team hasn’t been maturing, it hasn’t even formed. Team members were creating artificial harmony by avoiding conflict and now they feel disengaged and demonstrate no commitment.
If the team fails to hold one another accountable this leads to an environment where the most damaging dysfunction is demonstrated. When team members put their individual needs e.g. career development, recognition, etc. or even the needs of a group of people inside the team above the collective goals of the team this leads to Inattention to results.
You are in an awkward situation because you firmly believe Scrum is not the reason for these problems and yet Scrum exposed the dysfunctions. Everyone will be convinced that pre-Scrum no dysfunctions existed and now your team looks like falling apart. How can you change this? How can you resurrect the team and prove to senior management that Scrum is worth the effort?
Can somebody please explain!
I am convinced there is no easy answer to these questions; however I also believe that by following a few simple rules a lot can be done to save a dysfunctional team like the one we just looked at. While a lot of sources suggest that Scrum Masters should not deal with cases like this I tend to disagree. What if middle management doesn’t want to assist? What if they blame Scrum/You for the problem? What if you don’t want to lose the battle?
Before we get onto it I need to warn you that this is not something that can change overnight. You will need a lot of patience, good coaching and facilitation skills and then even more patience. It will probably be many days if not weeks until you see some change and might take up to 6-9 months until you start feeling optimistic about your team.
Are you vulnerable?
It all starts with trust so it is a good idea to look at establishing trust first. Ask yourself these questions: Are you vulnerable? Do you act like you are? Are you communicating enough to make the team aware that you are ready to make yourself vulnerable? Talk to the team or individually. Do a group exercise where you ask everyone to make themselves vulnerable by sharing details they otherwise would not want to. You will find examples easily but something in the lines of: strong and weak side, biggest challenge in school, etc. will do the trick. You also need to find a way to get the team together outside of normal work environment. This may range from organising a night out for the whole team to off-site event. Do as appropriate depending on your budget. Try to do the same activity at least once a month.
How would you now if you succeeded?
- When you start hearing team members answering honestly “I don’t know”
- Team members happily share information and offer help
Plain talking
Politics is when people choose their words and actions based on how they want others to react rather than on what they really think. Politics is when attention is paid to the speaker’s rank rather than dialogue content. Politics kill progress; therefore you need to stop political behaviour immediately.
Is there a lot of whispering in the team? Can your team members freely express what they think without fearing negative reaction? Start by telling your team to think as if they are two levels higher in the hierarchy. Encourage honest comments regardless of how ruthless they may sound. Print out posters explaining why politics is bad and put them on the walls around your team or if allowed in the whole building. Protect and support your people to speak openly with anyone in the company. Encourage healthy debate by asking open questions like: What other options do we have? What would happen if we go with this solution?
How would you know if you succeeded?
- When whispering is gone
- When you can join a team discussion and the topic doesn’t get changed
Something smells bad?
Agreement is good but be careful how you reach it. If you often feel that the team agrees too easily, if decisions are taken too quickly and meetings go too quietly then perhaps not everything is as good as it looks. Such “easy” decisions are bad for the team because team members will not feel fully involved because they could not voice their opinions due to dysfunctions already discussed above.
The only thing you can do in this case is to be there as facilitator and ask the questions required to spark a healthy debate. Ask about any weakness you see or suggest solutions but be careful – try to be only an advisor. When a debate is going in the wrong direction you should try to bring back the business goal and accent on the responsibility and effort required to achieve a great solution. Avoid digging into too much detail – this is where you will usually lose most of your time.
How would you know if you succeeded?
- When team members can say "I may not agree with your ideas but I understand them and can support them."
A group or a team
According to a popular definition a team is a small group of people with complementary skills and abilities who are committed to a common goal and approach for which they hold each other accountable. Presumably if a group of people doesn’t meet these criteria it remains just a group of people.
Do you often notice team members not particularly interested in team decisions? Have you seen team members only interested in their propositions and not willing to discuss or support others’ solutions? Do you often here “This is not my area. You need to speak to X”? Then perhaps you see what I mean. You need to set team goals and make sure everyone in the team is motivated to meet these goals. Sprint targets are good goals to meet but not always enough. Think about some more. These could be one off events like team games or reoccurring like review process. If you do have or want to start a performance review process make sure you go with the review process for agile teams because other process will make things worse. The review process for agile teams however used with well specified team goals might be just what you need to make sure the team is committed to a common goal and meeting these targets is really important for everyone and for the team.
How would you know if you succeeded?
- When sprint targets met rate increases significantly and anyway you will KNOW it on “that” review meeting
I scored two goals!
A friend was telling me a story about his son. The boy used to play football and once when his father (who actually hates football) picked him up after a game he asked: who won the game? The boy answered: I scored two goals. His father took a deep breath and asked again: Who won the game? Then the boy said: we lost 2:7 but I scored two goals isn’t that the important bit?
Clearly team members interested mainly in their individual goals and not paying attention to team’s results are causing the biggest of all damages. Perhaps this is caused by your organisation’s implementation of traditional performance review process, or just because team members fail to see benefits in achieving the team goal, something that is obvious with achieving their personal goal.
Make sure to explicitly set performance objectives and individual goals which are aligned with the team goals. This may seem a difficult task but there is plenty of information how to do it – you may begin by reading about the review process for agile teams. Note that you may need a lot of support from the senior management and it may seem as a difficult battle however only by winning it you can avoid ambiguous goals and the inattention to results behaviour. Team members will still have goals to work towards but now these goals will not get in the way of team’s targets.
How would you now if you succeeded?
- When team members start using more “we” rather than “I”
The show must go on...
The goal I set at the start of this article was to look at a few common scenarios, identify team dysfunctions and suggest ways to resolve them in Scrum environment. I’d like to believe the goal is mostly achieved. Of course this is only a five page article and you will need to read a lot more than this in order to succeed but my feeling is that since the thoughts I share are from experience you may find them useful. And if you do find this article helpful feel free to get in touch but do also remember that the battle to keep your team functional is one that you will have to keep winning over and over again.
Good thing you’re not alone on the way!
References:
The Five Dysfunctions of a Team -http://www.amazon.co.uk/Five-Dysfunctions-Team-Leadership-Fable/dp/0787960756/ref=sr_1_1?ie=UTF8&s=books&qid=1215814459&sr=8-1
Various ideas generated by reading posts on http://groups.yahoo.com/group/scrumdevelopment
It is known that a Scrum implementation, especially in its early days is expected to and almost always reveals hidden problems and issues that usually cause one or more major team dysfunctions. Finding ways to overcome these dysfunctions is the main goal of this article.
A beautiful beginning
So, you have introduced Scrum and you are working with the first ever Scrum team in your organisation. Luckily all team members seem to be interested and attend all the meetings. You have managed to get a room with whiteboards where the Daily stand-up takes place and you are also able to book an appropriately equipped room for reviews and plannings. Everyone in the organisation seems to be quite excited by the use of index cards and blu-tac which creates an energetic culture and you really feel good about the way things are going.
Retrospectives though feel a bit weird although you can’t quite figure out why. And then suddenly it all kicks-off with an against the rules reaction of a team member during a retrospective meeting after a disastrous review. The team achieved mere 5 points out of 12 planned. Tom, a senior developer says that Chris, a test engineer wouldn’t let the rest of the team do any testing. As a result the team ended up with half of the stories in the final stages of testing just because test engineers would not trust the rest of the team members to provide adequate testing. Surprised by the fact that he is being blamed Chris replies that no one else is qualified to do “proper testing” and anyway testing is not being paid enough attention in Scrum so he believes quality is going down. You finally realise that you need to stop this and you speak about how blaming is not allowed and accent on the positives of identifying a problem which the team now needs to focus on resolving.
The event throws you a little bit out of focus. You see this as a technical issue where testing needs to be automated to reduce the workload of testers. You start reading about test automation and distribute various useful web casts and studies about it. You even change your Scrum training and do it specifically for your test engineers. They seem to understand more and more about cross functional teams and the need of automation. You forget about that retrospective for now.
Although the problem may disappear unfortunately the issue is a lot bigger. Your test engineers do not trust the rest of the team and perhaps vice versa. Lack of trust is usually caused by the unwillingness to be vulnerable. Team members are not open with one another and are afraid to talk about their mistakes and weaknesses. At the same time the fact that your previous retrospectives seemed to be running well suggests that the team might be avoiding conflict. Because of the lack of trust the Team cannot engage in open debates, instead they resort to indirect discussions and shielded comments. Fear of conflict usually only postpones the conflict and once the disappointment reaches certain level the reaction will cause far bigger problem than if the conflict had arisen earlier, or even better resolved in a team debate.
Definitely getting better
Your testers now seemed to have finally got it. At least they no longer question automated testing and seem to be spending time on finding an appropriate tool to use. You still have slight concerns about the way the team reaches quick agreement during planning although you can easily explain this with the team getting more and more used to Scrum. Finally you’re edging close to completing the list of stories in the backlog so you start talking about release. On what seem to certainly be the last planning meeting you start discussing final steps to produce deliverable product which appears to cause a little more discomfort compared to previous meetings. Most unexpectedly Jane, your other test engineer, states out of the blue that she doesn’t care if the team decides that the product is ready for release as she anyway isn’t convinced that enough testing has been done. This sparks Chris’s comment that he wouldn’t accept responsibility for this product as he has never done less testing nor seen a most weird way to develop something. Peter who is a web developer in the team responds that test engineers still do not trust the rest of the team so how can more testing be done when only two team members are testing.
You are disappointed and very concerned that what seemed to be a maturing Scrum team suddenly doesn’t even act as a team. You spend the rest of the day reading about teamwork and various ways to improve it. You are beginning to realise that without sorting out underlying problems it would be difficult to improve practices and productivity.
When a team lacks conflict or what is usually called healthy conflict this could lead to lack of commitment. Because team members cannot air their opinions in an open discussion they rarely if at all commit to decision although they may demonstrate agreement in meetings. Teams members feel as if their opinions do not matter therefore find it difficult to support team decisions. Unfortunately when this is the case it triggers an even bigger problem – avoidance of accountability. Without committing to a well understood plan of action, it doesn’t matter how focused your people are they often fail to call their peers on actions and behaviors that seem counterproductive to the good of the team.
Did you see it coming?
The next morning things seem to become even worse. On the daily scrum Pete, senior developer shows impatience while Chris is talking, interrupts him to make a point about team members not following practices which he believes are a must. Tom also joins in by ignoring the usual order and stating that he feels it became more difficult for him to do “proper” development and increase his expertise. Jane follows to confirm what is obvious to the rest – she doesn’t enjoy working “that way” because she doesn’t see any career progression opportunities with Scrum.
You feel betrayed. It seems like the hundreds of hours spent on convincing people and setting up the basics have disappeared. The team hasn’t been maturing, it hasn’t even formed. Team members were creating artificial harmony by avoiding conflict and now they feel disengaged and demonstrate no commitment.
If the team fails to hold one another accountable this leads to an environment where the most damaging dysfunction is demonstrated. When team members put their individual needs e.g. career development, recognition, etc. or even the needs of a group of people inside the team above the collective goals of the team this leads to Inattention to results.
You are in an awkward situation because you firmly believe Scrum is not the reason for these problems and yet Scrum exposed the dysfunctions. Everyone will be convinced that pre-Scrum no dysfunctions existed and now your team looks like falling apart. How can you change this? How can you resurrect the team and prove to senior management that Scrum is worth the effort?
Can somebody please explain!
I am convinced there is no easy answer to these questions; however I also believe that by following a few simple rules a lot can be done to save a dysfunctional team like the one we just looked at. While a lot of sources suggest that Scrum Masters should not deal with cases like this I tend to disagree. What if middle management doesn’t want to assist? What if they blame Scrum/You for the problem? What if you don’t want to lose the battle?
Before we get onto it I need to warn you that this is not something that can change overnight. You will need a lot of patience, good coaching and facilitation skills and then even more patience. It will probably be many days if not weeks until you see some change and might take up to 6-9 months until you start feeling optimistic about your team.
Are you vulnerable?
It all starts with trust so it is a good idea to look at establishing trust first. Ask yourself these questions: Are you vulnerable? Do you act like you are? Are you communicating enough to make the team aware that you are ready to make yourself vulnerable? Talk to the team or individually. Do a group exercise where you ask everyone to make themselves vulnerable by sharing details they otherwise would not want to. You will find examples easily but something in the lines of: strong and weak side, biggest challenge in school, etc. will do the trick. You also need to find a way to get the team together outside of normal work environment. This may range from organising a night out for the whole team to off-site event. Do as appropriate depending on your budget. Try to do the same activity at least once a month.
How would you now if you succeeded?
- When you start hearing team members answering honestly “I don’t know”
- Team members happily share information and offer help
Plain talking
Politics is when people choose their words and actions based on how they want others to react rather than on what they really think. Politics is when attention is paid to the speaker’s rank rather than dialogue content. Politics kill progress; therefore you need to stop political behaviour immediately.
Is there a lot of whispering in the team? Can your team members freely express what they think without fearing negative reaction? Start by telling your team to think as if they are two levels higher in the hierarchy. Encourage honest comments regardless of how ruthless they may sound. Print out posters explaining why politics is bad and put them on the walls around your team or if allowed in the whole building. Protect and support your people to speak openly with anyone in the company. Encourage healthy debate by asking open questions like: What other options do we have? What would happen if we go with this solution?
How would you know if you succeeded?
- When whispering is gone
- When you can join a team discussion and the topic doesn’t get changed
Something smells bad?
Agreement is good but be careful how you reach it. If you often feel that the team agrees too easily, if decisions are taken too quickly and meetings go too quietly then perhaps not everything is as good as it looks. Such “easy” decisions are bad for the team because team members will not feel fully involved because they could not voice their opinions due to dysfunctions already discussed above.
The only thing you can do in this case is to be there as facilitator and ask the questions required to spark a healthy debate. Ask about any weakness you see or suggest solutions but be careful – try to be only an advisor. When a debate is going in the wrong direction you should try to bring back the business goal and accent on the responsibility and effort required to achieve a great solution. Avoid digging into too much detail – this is where you will usually lose most of your time.
How would you know if you succeeded?
- When team members can say "I may not agree with your ideas but I understand them and can support them."
A group or a team
According to a popular definition a team is a small group of people with complementary skills and abilities who are committed to a common goal and approach for which they hold each other accountable. Presumably if a group of people doesn’t meet these criteria it remains just a group of people.
Do you often notice team members not particularly interested in team decisions? Have you seen team members only interested in their propositions and not willing to discuss or support others’ solutions? Do you often here “This is not my area. You need to speak to X”? Then perhaps you see what I mean. You need to set team goals and make sure everyone in the team is motivated to meet these goals. Sprint targets are good goals to meet but not always enough. Think about some more. These could be one off events like team games or reoccurring like review process. If you do have or want to start a performance review process make sure you go with the review process for agile teams because other process will make things worse. The review process for agile teams however used with well specified team goals might be just what you need to make sure the team is committed to a common goal and meeting these targets is really important for everyone and for the team.
How would you know if you succeeded?
- When sprint targets met rate increases significantly and anyway you will KNOW it on “that” review meeting
I scored two goals!
A friend was telling me a story about his son. The boy used to play football and once when his father (who actually hates football) picked him up after a game he asked: who won the game? The boy answered: I scored two goals. His father took a deep breath and asked again: Who won the game? Then the boy said: we lost 2:7 but I scored two goals isn’t that the important bit?
Clearly team members interested mainly in their individual goals and not paying attention to team’s results are causing the biggest of all damages. Perhaps this is caused by your organisation’s implementation of traditional performance review process, or just because team members fail to see benefits in achieving the team goal, something that is obvious with achieving their personal goal.
Make sure to explicitly set performance objectives and individual goals which are aligned with the team goals. This may seem a difficult task but there is plenty of information how to do it – you may begin by reading about the review process for agile teams. Note that you may need a lot of support from the senior management and it may seem as a difficult battle however only by winning it you can avoid ambiguous goals and the inattention to results behaviour. Team members will still have goals to work towards but now these goals will not get in the way of team’s targets.
How would you now if you succeeded?
- When team members start using more “we” rather than “I”
The show must go on...
The goal I set at the start of this article was to look at a few common scenarios, identify team dysfunctions and suggest ways to resolve them in Scrum environment. I’d like to believe the goal is mostly achieved. Of course this is only a five page article and you will need to read a lot more than this in order to succeed but my feeling is that since the thoughts I share are from experience you may find them useful. And if you do find this article helpful feel free to get in touch but do also remember that the battle to keep your team functional is one that you will have to keep winning over and over again.
Good thing you’re not alone on the way!
References:
The Five Dysfunctions of a Team -http://www.amazon.co.uk/Five-Dysfunctions-Team-Leadership-Fable/dp/0787960756/ref=sr_1_1?ie=UTF8&s=books&qid=1215814459&sr=8-1
Various ideas generated by reading posts on http://groups.yahoo.com/group/scrumdevelopment
Sunday, 15 June 2008
The Nokia Test
About a month ago I have decided to ask everyone in the department to rate our scrum implementation based on questions from the nokia test. I think this was a brilliant idea because it exposed a number of problems with both the teams and the product owners. Also some of the questions were misunderstood by several team members. The summary of results is below:
1. Iteration length less than 6 weeks – 91.67%
2. Fixed iterations – 100%
3. Product Backlog items always estimated – 0%
4. Velocity known– 91.67%
5. Burndown charts generated – 100%
6. Product Owner known – 66.67%
7. No Project Managers interfering – 75%
Interesting that for example on question number one somebody said he isn't quite sure if iterations should be less than 6 weeks and fixed at all! Also somebody has replied with "I don't know what it is" to question number 4(?!). The results of 6 and 7 were just a good reminder to some of our managers to stop messing with scrum.
Overall we had a very positive response following these results so I am thinking about doing it every 6 months ;)
1. Iteration length less than 6 weeks – 91.67%
2. Fixed iterations – 100%
3. Product Backlog items always estimated – 0%
4. Velocity known– 91.67%
5. Burndown charts generated – 100%
6. Product Owner known – 66.67%
7. No Project Managers interfering – 75%
Interesting that for example on question number one somebody said he isn't quite sure if iterations should be less than 6 weeks and fixed at all! Also somebody has replied with "I don't know what it is" to question number 4(?!). The results of 6 and 7 were just a good reminder to some of our managers to stop messing with scrum.
Overall we had a very positive response following these results so I am thinking about doing it every 6 months ;)
Friday, 30 May 2008
Implementing Scrum
right, this is not a long and boring article with advices which you don't need. it is just a link to a site which I find quite funny :)
Wednesday, 23 April 2008
Drop a SQL Server database from your .net code
Have you ever been in a situation where you need to provide uninstall funcitonality for database installation? Imagine you have just created your nice new database in Sql server and all the scripts have been created but at the last line something happens and you need to rollback. This also includes dropping the database and it isn't straight forward. Closing your connection(s) doesn't help as they don't close straight away. While looking for a solution I came accross one and decided to share it here.
This is the method I created:
private string GetKillProcessStatement(string database)
{
string result = string.Empty;
StringBuilder sql = new StringBuilder();
sql.AppendLine("DECLARE @SQL varchar(max)");
sql.AppendLine("SELECT @SQL = COALESCE(@SQL,'') + 'Kill ' + Convert(varchar, SPId) + ';'");
sql.AppendLine("FROM MASTER..SysProcesses");
sql.AppendLine("WHERE DBId = DB_ID('" + database + "') AND SPId <> @@SPId");
sql.AppendLine("SELECT @SQL");
SqlConnection dropDbSqlConn = new SqlConnection(_masterConnStr);
dropDbSqlConn.Open();
using (SqlCommand sqcomm = new SqlCommand())
{
sqcomm.Connection = dropDbSqlConn;
sqcomm.CommandType = CommandType.Text;
sqcomm.CommandText = sql.ToString();
sqcomm.Prepare();
object obj = sqcomm.ExecuteScalar();
if (obj!=null)
result = obj.ToString();
}
dropDbSqlConn.Close();
return result;
}
A few clarifications: _masterConnStr is the connection string to the master db on this server. If you need to create a db you will probably already have this one available. In this case it is a private member in the installer class. I explicitly need it to be inline sql as i don't want to install stored procs in other databases on the server. Database access rights to do that will be available because the user we require also needs to create dbs.
The inline sql when executed returns a list of kill statements like this one: 'Kill 11;Kill 23;Kill 45;' . The numbers will be the process ids in Sql Server. Once you get this you are just a simple ExecuteNonQuery of this string before you will be able to drop your database regardless of the number of connections on it. Ok I know there might be more arguments for connection initiated between your kill execute and your drop statement execute but there is cure for this as well. If this is a worry then execute this sql script before you issue the drop statement:
"ALTER DATABASE [" + database + "] SET SINGLE_USER WITH ROLLBACK IMMEDIATE";
I wouldn't be bothered to do so as in my case this is a newly installed db and there is no chance someone will know about it.
Back to the point I believe this is a good solution to ensure that you can drop a sql server db and the statement won’t fail because of open connections.
This is the method I created:
private string GetKillProcessStatement(string database)
{
string result = string.Empty;
StringBuilder sql = new StringBuilder();
sql.AppendLine("DECLARE @SQL varchar(max)");
sql.AppendLine("SELECT @SQL = COALESCE(@SQL,'') + 'Kill ' + Convert(varchar, SPId) + ';'");
sql.AppendLine("FROM MASTER..SysProcesses");
sql.AppendLine("WHERE DBId = DB_ID('" + database + "') AND SPId <> @@SPId");
sql.AppendLine("SELECT @SQL");
SqlConnection dropDbSqlConn = new SqlConnection(_masterConnStr);
dropDbSqlConn.Open();
using (SqlCommand sqcomm = new SqlCommand())
{
sqcomm.Connection = dropDbSqlConn;
sqcomm.CommandType = CommandType.Text;
sqcomm.CommandText = sql.ToString();
sqcomm.Prepare();
object obj = sqcomm.ExecuteScalar();
if (obj!=null)
result = obj.ToString();
}
dropDbSqlConn.Close();
return result;
}
A few clarifications: _masterConnStr is the connection string to the master db on this server. If you need to create a db you will probably already have this one available. In this case it is a private member in the installer class. I explicitly need it to be inline sql as i don't want to install stored procs in other databases on the server. Database access rights to do that will be available because the user we require also needs to create dbs.
The inline sql when executed returns a list of kill statements like this one: 'Kill 11;Kill 23;Kill 45;' . The numbers will be the process ids in Sql Server. Once you get this you are just a simple ExecuteNonQuery of this string before you will be able to drop your database regardless of the number of connections on it. Ok I know there might be more arguments for connection initiated between your kill execute and your drop statement execute but there is cure for this as well. If this is a worry then execute this sql script before you issue the drop statement:
"ALTER DATABASE [" + database + "] SET SINGLE_USER WITH ROLLBACK IMMEDIATE";
I wouldn't be bothered to do so as in my case this is a newly installed db and there is no chance someone will know about it.
Back to the point I believe this is a good solution to ensure that you can drop a sql server db and the statement won’t fail because of open connections.
Thursday, 10 April 2008
C# Regex to split multiple SQL statements
You know if you want to use SQLCommand to execute a big list of SQL Statements you cannot have the reserved SQL word GO in the script. So ideally you would want to split these statements to exclude the "GO"s and (perhaps in a transaction) execute each line separately. I wanted one of those for long time and was getting decent results with "^GO" + the multiline option, like this:
Regex regex = new Regex("^GO", RegexOptions.IgnoreCase | RegexOptions.Multiline);
But with xml columns in tables this quickly becomes useless - Yes I script data to insert statements also. Then finally today I found one that seems to be a lot better or at least works with all my data. Here it is:
string[] lines = Regex.Split(sql, "\\s+GO\\s+|^GO\\s+", RegexOptions.Multiline);
And NO I do not know what it means - I only know that it works ;)
Regex regex = new Regex("^GO", RegexOptions.IgnoreCase | RegexOptions.Multiline);
But with xml columns in tables this quickly becomes useless - Yes I script data to insert statements also. Then finally today I found one that seems to be a lot better or at least works with all my data. Here it is:
string[] lines = Regex.Split(sql, "\\s+GO\\s+|^GO\\s+", RegexOptions.Multiline);
And NO I do not know what it means - I only know that it works ;)
Wednesday, 9 April 2008
“Scaling agile is the last thing you want to do” — Martin Fowler
Here's some alchoholic perspective about Scrum. I quite enjoyed reading it.
Now, seriously it is an excellent REAL world example and I recommend reading this article before beginning any scrum scaling.
Now, seriously it is an excellent REAL world example and I recommend reading this article before beginning any scrum scaling.
Tuesday, 8 April 2008
SCRUM: Lessons learned
I wrote this article several months ago but only managed to publish it now which actually helped me improve it. Enjoy ;)
First project with Scrum (Lessons learned)
I know the first thought that will come to almost everyone’s mind when you read the title will be – yeah we all know that, but is it that simple in practice? This article tells a story about how all those “waterfall” years’ affect people’s thinking about releases and what we’ve learned from experience.
When on a sunny day in June we have had our first ever planning meeting I didn’t really realise what kind of roller coaster I am getting on. This planning meeting happened to be in the middle of a huge project aiming to release five products in total, consisting of three majorly reworked and two brand new products. To make things more clear about the size of the work: our decent spec build server takes 2+ hours to build the full set.
As the person responsible for the core products in the company at that time I welcomed the idea about scrum and joined forces with Rich and Sri to persuade senior management that we should go with implementing Scrum in an attempt to save this release which by that time was already 2 months late. While we had the full senior management support once we started doing sprints a few things seemed to have changed immediately in the entire organization. First everyone without really understanding Scrum expected that this “new way of working” will somehow “magically” enable us to release the software in a few weeks time and second they reminded us that they need the whole lot of functionality by a certain date and this is business critical. This of course added some more pressure to the team which was the last thing we needed.
Lesson number 1 - “End of the world” mentality
There is one phrase that will always remind me about this specific release – “good that we found it now”. This of course refers to discovering a fault and the fact that we’ve discovered it before doing the release. To me it pictures a big problem – we are so much focused on getting things right by a specific deadline that we forget a few simple facts – one - whatever we’ve found does not come from the end user hence might really not be that important to fix; and two - this is not the last release we will do and as long as we release often customers should get what they ask for. So the lesson learned was plan regular releases that add small amount of functionality to reduce the pressure.
Lesson number 2 – All the legacy stuff
Yes we’ve introduced Scrum in the middle of a big chaotically executed waterfall. We’ve inherited tons of legacy code (not documented, not unit tested and some of it not even integrated) and this affected our productivity. We had to add additional effort to fully automate builds and maintain continuous integration but yet once again we ended up with a whole lot if integration issues at the end. We had many problems with transforming from traditional to agile testing and perhaps we in fact made very little progress during the course of the project.
So is there a way to predict these problems? I would say no. But this doesn’t help a lot when senior management is asking for dates. So what would you do? First of all always answer with a range and don’t afraid if there are many unknowns to use a wide range. If management doesn’t like it explain this is due to too many unknowns and then ask for more time to research. If they don’t understand at least you know that there isn’t a better way to solve the problem and if they don’t understand then ask them do they really understand and support Scrum.
Lesson number 3 – Integration, integration…
Back to the time when we were in the middle of the mess and tried to sort it out we failed to start integrating straight away. Of course there was so much to do with sprint organisation and backlog management and all those unfinished features and task but still it was a mistake. We found out about it 2 months later when we realised that all the integration bits between four projects are now broken. So like on a typical waterfall project we started integration in the last 3 sprints and of course we paid the price for it. Now the word “integrate” is pretty much part of our DONE definition and really that says it all about integration.
Lesson number 4 - How long does regression testing take?
When I asked this question for just one of our products the answer I received was – two months if done by one tester. We should have really packed our stuff and gone home if we were to do that. Waterfall testers will refuse to think about automation. They will try to persuade you that it is extremely difficult and very slow to do and in fact not really worth the effort as it cannot cover “a lot” of the testing plan. Lesson learned was- throw the plan in the bin immediately and ask the whole team to find a better way to test. And the team effort will do it.
Lesson number 5 – Do we really understand it?
I was really pleased about the number of trainings we have done about Scrum and the overall understanding seemed to be good however it turns out to be a battle you have to win over and over again. Is it because of the waterfall mentality which is difficult to change or because some people get it and some not? I am not ready to give final answer however we all need to understand that coaching is needed pretty much all the time. And making assumptions that “yes after this training they really seem to get it” is not helping. We learned that we need to continue coaching with the same patience and enthusiasm as on day one.
Actually thanks to a lot of those “good that we found it now” problems we spent 2 months stabilizing the whole set of products. Really the root cause to me was the size of this work, the fact that we’ve been asked to release so much in one go and the deadline wasn’t any flexible.
The good point is that we learned a lot about Scrum, ourselves, software development, and why waterfall is bad. One year later we know learning never ends but I believe we appreciate what we’ve been through and we value what we’ve got.
First project with Scrum (Lessons learned)
I know the first thought that will come to almost everyone’s mind when you read the title will be – yeah we all know that, but is it that simple in practice? This article tells a story about how all those “waterfall” years’ affect people’s thinking about releases and what we’ve learned from experience.
When on a sunny day in June we have had our first ever planning meeting I didn’t really realise what kind of roller coaster I am getting on. This planning meeting happened to be in the middle of a huge project aiming to release five products in total, consisting of three majorly reworked and two brand new products. To make things more clear about the size of the work: our decent spec build server takes 2+ hours to build the full set.
As the person responsible for the core products in the company at that time I welcomed the idea about scrum and joined forces with Rich and Sri to persuade senior management that we should go with implementing Scrum in an attempt to save this release which by that time was already 2 months late. While we had the full senior management support once we started doing sprints a few things seemed to have changed immediately in the entire organization. First everyone without really understanding Scrum expected that this “new way of working” will somehow “magically” enable us to release the software in a few weeks time and second they reminded us that they need the whole lot of functionality by a certain date and this is business critical. This of course added some more pressure to the team which was the last thing we needed.
Lesson number 1 - “End of the world” mentality
There is one phrase that will always remind me about this specific release – “good that we found it now”. This of course refers to discovering a fault and the fact that we’ve discovered it before doing the release. To me it pictures a big problem – we are so much focused on getting things right by a specific deadline that we forget a few simple facts – one - whatever we’ve found does not come from the end user hence might really not be that important to fix; and two - this is not the last release we will do and as long as we release often customers should get what they ask for. So the lesson learned was plan regular releases that add small amount of functionality to reduce the pressure.
Lesson number 2 – All the legacy stuff
Yes we’ve introduced Scrum in the middle of a big chaotically executed waterfall. We’ve inherited tons of legacy code (not documented, not unit tested and some of it not even integrated) and this affected our productivity. We had to add additional effort to fully automate builds and maintain continuous integration but yet once again we ended up with a whole lot if integration issues at the end. We had many problems with transforming from traditional to agile testing and perhaps we in fact made very little progress during the course of the project.
So is there a way to predict these problems? I would say no. But this doesn’t help a lot when senior management is asking for dates. So what would you do? First of all always answer with a range and don’t afraid if there are many unknowns to use a wide range. If management doesn’t like it explain this is due to too many unknowns and then ask for more time to research. If they don’t understand at least you know that there isn’t a better way to solve the problem and if they don’t understand then ask them do they really understand and support Scrum.
Lesson number 3 – Integration, integration…
Back to the time when we were in the middle of the mess and tried to sort it out we failed to start integrating straight away. Of course there was so much to do with sprint organisation and backlog management and all those unfinished features and task but still it was a mistake. We found out about it 2 months later when we realised that all the integration bits between four projects are now broken. So like on a typical waterfall project we started integration in the last 3 sprints and of course we paid the price for it. Now the word “integrate” is pretty much part of our DONE definition and really that says it all about integration.
Lesson number 4 - How long does regression testing take?
When I asked this question for just one of our products the answer I received was – two months if done by one tester. We should have really packed our stuff and gone home if we were to do that. Waterfall testers will refuse to think about automation. They will try to persuade you that it is extremely difficult and very slow to do and in fact not really worth the effort as it cannot cover “a lot” of the testing plan. Lesson learned was- throw the plan in the bin immediately and ask the whole team to find a better way to test. And the team effort will do it.
Lesson number 5 – Do we really understand it?
I was really pleased about the number of trainings we have done about Scrum and the overall understanding seemed to be good however it turns out to be a battle you have to win over and over again. Is it because of the waterfall mentality which is difficult to change or because some people get it and some not? I am not ready to give final answer however we all need to understand that coaching is needed pretty much all the time. And making assumptions that “yes after this training they really seem to get it” is not helping. We learned that we need to continue coaching with the same patience and enthusiasm as on day one.
Actually thanks to a lot of those “good that we found it now” problems we spent 2 months stabilizing the whole set of products. Really the root cause to me was the size of this work, the fact that we’ve been asked to release so much in one go and the deadline wasn’t any flexible.
The good point is that we learned a lot about Scrum, ourselves, software development, and why waterfall is bad. One year later we know learning never ends but I believe we appreciate what we’ve been through and we value what we’ve got.
Saturday, 15 March 2008
A game to help with estimation
A lot more useful than the BS by MS.
I've been looking for or even wanting to create a similar one (damn, I'm late again!) but anyway there it is a game developed to help with doing the XP planning game which I think is perfectly suitable for introducing team members to the SCRUM estimation and planning techniques as well.
Read about it here
I've been looking for or even wanting to create a similar one (damn, I'm late again!) but anyway there it is a game developed to help with doing the XP planning game which I think is perfectly suitable for introducing team members to the SCRUM estimation and planning techniques as well.
Read about it here
Friday, 14 March 2008
A World of change [taken from the yahoo scrum group]
Below is a quote which I find quite interesting and so much reflecting the reality that it made me laugh ;)
Frank Herbert (from his book Heretics of Dune):
"Technology, in common with many other activities, tends toward
avoidance of risks by investors. Uncertainty is ruled out if possible.
Capital investment follows this rule, since people generally prefer
the predictable. Few recognize how destructive this can be, how it
imposes severe limits on variability and thus makes whole populations
fatally vulnerable to the shocking ways our universe can throw the
dice."
(Assessment of Ix, Bene Gesserit Archives)
Frank Herbert (from his book Heretics of Dune):
"Technology, in common with many other activities, tends toward
avoidance of risks by investors. Uncertainty is ruled out if possible.
Capital investment follows this rule, since people generally prefer
the predictable. Few recognize how destructive this can be, how it
imposes severe limits on variability and thus makes whole populations
fatally vulnerable to the shocking ways our universe can throw the
dice."
(Assessment of Ix, Bene Gesserit Archives)
Wednesday, 12 March 2008
UK Scrum Practitioners
I started a group on linked-in which I named Uk Scrum Practitioners.
If you are even remotely involved in Scrum you are more than welcome to join by clicking here.
If you are even remotely involved in Scrum you are more than welcome to join by clicking here.
A collection of Agile Videos
I've been keeping this in my links collection for a while and I tought it might be a good idea to share it.
I think there are a number of "must watch" videos for agile practitioners.
Enjoy ;)
I think there are a number of "must watch" videos for agile practitioners.
Enjoy ;)
Wednesday, 27 February 2008
Northgale or one dream less
I found this link today - http://www.sommarskog.se/dynsearch/Northgale.sql
ok I know I am a bit late most of you will probably know about it ;)
But it is simply great. It uses an existing Northwind database to generate the same tables inside Northgale but with a lot more data. For exmpla the order details table ends up with more than 2.3mln rows!
When I tried to select from order details all rows with quantity > 30 it took 13 seconds!
What can be better than that to allow you to experiment with your queries when you're looking to improve performance?!
I think the only requirement is a northwind database and it takes just about 5 minutes to setup.
I hope someone some day will discover this post and will find it useful ;)
ok I know I am a bit late most of you will probably know about it ;)
But it is simply great. It uses an existing Northwind database to generate the same tables inside Northgale but with a lot more data. For exmpla the order details table ends up with more than 2.3mln rows!
When I tried to select from order details all rows with quantity > 30 it took 13 seconds!
What can be better than that to allow you to experiment with your queries when you're looking to improve performance?!
I think the only requirement is a northwind database and it takes just about 5 minutes to setup.
I hope someone some day will discover this post and will find it useful ;)
Labels:
Northgale,
Optimization,
Performance,
SQL Server
Subscribe to:
Posts (Atom)