CruiseControl.NET / header.xsl / and DiskSpace

I had a nice situation today.
After a code check-in to SVN, CruiseControl.NET reported an error.

The culprit??

No more disk space.

The error is seen below.  "Out of space" isn’t mentioned, thus why I am posting this.
But it makes sense, the xsl transformation didn’t have enough disk space to handle the transform output.

There
was an exception trying to carry out your request.

Exception Message

Unable to execute transform: C:\Program Files
(x86)\CruiseControl.NET\webdashboard\xsl\header.xsl
(or)
Unable to execute transform: C:\Program Files\CruiseControl.NET\webdashboard\xsl\header.xsl

Exception Full Details

ThoughtWorks.CruiseControl.Core.CruiseControlException:
Unable to execute transform: C:\Program Files
(x86)\CruiseControl.NET\webdashboard\xsl\header.xsl —>
System.Xml.XmlException: Unexpected end of file while parsing CDATA has
occurred. Line 384, position 106.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type,
Int32& outStartPos, Int32& outEndPos)
at System.Xml.XmlTextReaderImpl.ParseCDataOrComment(XmlNodeType type)
at System.Xml.XmlTextReaderImpl.ParseElementContent()
at System.Xml.XPath.XPathDocument.LoadFromReader(XmlReader reader,
XmlSpace space)
at System.Xml.XPath.XPathDocument..ctor(TextReader textReader)
at
ThoughtWorks.CruiseControl.Core.Util.XslTransformer.Transform(String
input, String xslFilename, Hashtable xsltArgs)
— End of inner exception stack trace —
at
ThoughtWorks.CruiseControl.Core.Util.XslTransformer.Transform(String
input, String xslFilename, Hashtable xsltArgs)
at
ThoughtWorks.CruiseControl.Core.Util.HtmlAwareMultiTransformer.Transform(String
input, String[] transformerFileNames, Hashtable xsltArgs)
at
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.PathMappingMultiTransformer.Transform(String
input, String[] transformerFileNames, Hashtable xsltArgs)
at
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.BuildRequestTransformer.Transform(IBuildSpecifier
buildSpecifier, String[] transformerFileNames, Hashtable xsltArgs)
at
ThoughtWorks.CruiseControl.WebDashboard.Dashboard.Actions.MultipleXslReportBuildAction.Execute(ICruiseRequest
cruiseRequest)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ServerCheckingProxyAction.Execute(ICruiseRequest
cruiseRequest)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.BuildCheckingProxyAction.Execute(ICruiseRequest
cruiseRequest)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ProjectCheckingProxyAction.Execute(ICruiseRequest
cruiseRequest)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.CruiseActionProxyAction.Execute(IRequest
request)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.CachingActionProxy.Execute(IRequest
request)
at
ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ExceptionCatchingActionProxy.Execute(IRequest
request)
Posted in Software Development | 2 Comments

Rule of 5 : Better machines for Developers

I have just invented the "Rule of 5" when it comes to justifying a computer upgrade for a developer.

5 compiles an hour (when you’re actually developing code (like you love to do) and not in some meeting).
Each compile saves 55 seconds**.
5 hours of development on average a day.  (Hopefully more, but bear with me, I’m trying to use a "5" here.)
5 days a week.

That is ~115 minutes/week of savings.  ~2 hours a week of savings.

Throw into the equation a developer hourly cost of  some kind of middle ground amount of $50 (and another "5" of course!).

~$95.50 per week of savings per developer.  Each new $1000 machine pay for itself in ~10.5 weeks.

Don’t forget about the morale factor, which is priceless.

=====================================

** 55 seconds compile time savings is an estimate of course. 
But I actually conducted a real-world test on a solution I am working on right now.
I’d suggest taking a code base you currently have and perform a simple test.
When you build with Msbuild, you only need the code and not the Visual Studio IDE.

Test #1
No Previous Build. (no sln.cache file)
    E6300 time : 00:01:10.23  ( 1 minute 10 seconds )
    Q6400 time : 00:00:14.21  ( 14 seconds )

Test #2
Subsequent Build. (sln.cache file exists)
    E6300 time : 00:00:51.43  ( 51 seconds )
    Q6400 time : 00:00:07.55  ( 7 seconds )

Time savings:
No Previous Build. (no sln.cache file)
        56 seconds savings.
Subsequent Build. (sln.cache file exists)
        44 seconds.

Test machines were an older 1.86 Dual Core E6300 vs a 2.4 Quad Core Q6600.  Keep in mind this is an older Quad Core CPU.
My personal recommendation (based in price point at the time of writing this in 4th quarter 2010) is the i7-9
50 quad-core.

CPU Benchmarks:
http://www.cpubenchmark.net/common_cpus.html

Intel Core i7 950 @ 3.07GHz == 6,275
Intel Core2 Quad Q6600 @ 2.40GHz == 2,975
Intel Core2 Duo E6300 @ 1.86GHz == 1,113

Why didn’t I test with a i7-950?  That’s the point, I don’t have one!!

Give your developers more Cores !
A.  Because the code can compile faster.
B.  Because most production machines have (at the very least) 4 CPU’s/Cores.

__CPU’s/Cores
Because of advances in Parallel Computing, the minimal core number requirement is 2.
However, the number of CPU’s should be closer to the production servers.
Recommendation.  4 Cores is recommended because most modern production servers have at least 4 CPU’s.
(The connection here is that with Parallel Computing, developers will be able to actually take advantage of extra processors/cores.

In fact, Microsoft has invented a new namespace in DotNet Framework 4.0 to allow easier coding against this model.
System.Threading.Tasks Namespace
http://msdn.microsoft.com/en-us/library/system.threading.tasks.aspx

__Visual Studio IDE as it relates to the build process (faster builds = more efficient developer)
http://msdn.microsoft.com/en-us/library/bb383805%28v=VS.100%29.aspx
Visual Studio 2008 and 2010 can take advantage of systems that have multiple processors, or multiple-core processors. A separate build process is created for each available processor. For example, if the system has four processors, then four build processes are created. MSBuild can process these builds simultaneously, and therefore overall build time is reduced. However, parallel building introduces some changes in how build processes occur. This topic discusses those changes.

Fourth Quarter 2010 : Machine Recommendation (Leveraging Frugality vs Performance)
    • Genuine Windows 7 Professional 64-bit
    • Intel(R) Core(TM) i7-9
50 quad-core (or i7-930 quad-core but the newegg difference is only $10 (at the time of writing)
    • 6GB DDR3-1333MHz SDRAM
    • 1TB RAID 0 (2 x 500GB SATA HDDs)
    • 1GB ATI Radeon HD.  A version that supports dual monitors.

Why the i7-950?  There is usually a pretty clear cut "jump" that takes you out of "very, very good and reasonable" to "super great and a lot more expensive".
http://www.cpubenchmark.net/high_end_cpus.html
Just start walking "down" until you get to around $300 or under.  the i7-950 (at the time of writing) was the one sitting at the top of the best of the reasonable.  There are processors above the i7-950 that are at the $570 to $1600 range.  Youch!  Of course, by the time I click "publish" that list will become out of date, so you just gotta walk the list until you see the clear cut "very, very good and reasonable" winner.

Supporting URLS:

http://www.joelonsoftware.com/articles/fog0000000043.html
9. Do you use the best tools money can buy?
Writing code in a compiled language is one of the last things that still can’t be done instantly on a garden variety home computer. If your compilation process takes more than a few seconds, getting the latest and greatest computer is going to save you time.
Debugging GUI code with a single monitor system is painful if not impossible. If you’re writing GUI code, two monitors will make things much easier.

(A personal note about the above comments, people who use Word/Excel don’t need the superpower, they’ll do fine with something more mainstream.
Code Compile == More Horsepower)

(The other personal note, Joel On Software makes for some good all around How-To-Develop software well.)

http://www.codinghorror.com/blog/2006/08/the-programmers-bill-of-rights.html
Every programmer shall have two monitors
Every programmer shall have a fast PC
Every programmer shall have their choice of mouse and keyboard
Every programmer shall have a comfortable chair
Every programmer shall have a fast internet connection
Every programmer shall have quiet working conditions

//Quote from codinghorror article//
It’s unbelievable to me that a company would pay a developer $60-$100k in salary, yet cripple him or her with terrible working conditions and crusty hand-me-down hardware. This makes no business sense whatsoever. And yet I see it all the time. It’s shocking how many companies still don’t provide software developers with the essential things they need to succeed.
//End Quote//

And don’t forget about the dual monitors!

http://www.nytimes.com/2009/01/15/technology/personaltech/15basics.html
http://www.nytimes.com/2006/04/20/technology/20basics.html

"Survey after survey shows that whether you measure your productivity in facts researched, alien spaceships vaporized, or articles written, adding an extra monitor will give your output a considerable boost 20 percent to 30 percent, according to a survey by Jon Peddie Research."

http://research.microsoft.com/en-us/news/features/vibe.aspx
Microsoft researchers haven’t perfected the genie, but they’ve found a tool that can increase your productivity by 9 to 50 percent and make your work day easier.
The researchers conducted user studies that proved the effectiveness of adding a second or even third monitor to your workstation, creating a wide-screen effect.

Minimum Suggestion: Dual 20" (or 20.5") Monitors.
Suggestion: Dual 22" (23" or 24") monitors.

Posted in Software Development | 1 Comment

SVN (Subversion) case sensitivity issue with Linux (RedHat) SVN Server and Windows Client

I had a weird subversion (svn) issue today.

Here are some clues (for future googlers).

"svn: Can’t open file" ".svn\tmp\text-base" "svn-base" "The system cannot find the file specified".
svn: Can’t open file .svn\tmp\text-base svn-base The system cannot find the file specified

After backtracking, here is what happened.

I had a file (a compiled dll) in source control.
I found out that I had a non-signed version of the dll (DotNet assembly).
I found the "signed version" of the dll.  I wanted to replace the unsigned version with the signed version.

So I replaced the existing file (in my local directory) with the signed version.

There was a slight issue with the file name.

The unsigned version was
MyAssembly.dll

The signed version was ~~~slightly off (case wise):
MyAssembLY.dll

(The file names are obviously for demonstration only).

So when I overwrote MyAssembly.dll with MyAssembLY.dll, it was fine (local windows directory).
The files were "Committed to Subversion".

(FYI, You can ignore the "signed" vs "unsigned" issue I cite above if you’re having this issue…..
the issue was not
related to signed/unsigned, but rather cAsE SenSiTiviTy of the replacement file).

Ok…so I thought everything was cool.  WRONG.

The next time my continuous integration ran, I got the error message seen above.  ("svn: Can’t open file).

I mimicked the command line call ( to take the CI environment out of the equation).

"C:\Program Files\CollabNet\Subversion Client\svn.exe" checkout https://www.something.com/SomeFolder SomeLocalFolder –username myUserName –non-interactive –no-auth-cache

and got the same error message.

The fix?

I deleted all the files in the folder via the repository browser.  (Aka, cleared them OUT of svn).
I uploaded the signed assembly.  (I did this via "Add" and "Commit" with the local tortoise shell).

Please note this is NOT ideal, because of revision history.  I was lucky in that for this branch, history retention was not a priority.

Now the case of the files on the server (svn) matched the case of my (signed) assemblies(dll’s).

Here is the thread that gave me the clue:
http://issues.apache.org/jira/browse/STDCXX-14

Server:  CollabNet for RedHat SVN Server.  (Aka, a case sensitive environment)
Client:  CollabNet svn client for Windows.

Posted in Software Development | Leave a comment

Sql Server Data to a Jet Database

Ok.  This is another "not rocket science" blog.
But if/when you need it, it is good to know the syntax.

Basically, this will run a tsql query and dump the data into a jet database.  ("access database" is another way to say it, but I prefer to call the program "Microsoft Access" and then call the database file (.mdb) a jet database to make the distinction clear).

The syntax sugar is below.
However, one key thing to know is that you need to have a .mdb file created ~~before running the code.
Aka, you’ll need to manually make an .mdb and add tables/columns which matches your tsql-select query(ies).

The attached zip (zip file coming soon!) file give you the tsql code and a sample jet database (as a complete example).

Here are the key lines.

–Note, the … before "PersonTable" is there on purpose.
DELETE
FROM OPENDATASOURCE(‘Microsoft.Jet.OLEDB.4.0’,
    ‘Data Source=C:\WUWUTemp\JetDatabaseExportExample\person.mdb;’)
    …PersonTable

INSERT INTO OPENROWSET (‘Microsoft.Jet.OLEDB.4.0’, ‘C:\WUWUTemp\JetDatabaseExportExample\person.mdb’;’Admin’;”,
‘Select LastName , FirstName From PersonTable’)
SELECT LastName , FirstName
From dbo.SomeRealTable;

Posted in Software Development | Leave a comment

BCP Export

I found this one in some old code, and decided to post it.

Not rocket science, but very handy sometimes.

The below is a "fixed width" export, using bcp.

The syntax sugar for "fixed-width" is the cast to a char(SomeNumber) where SomeNumber is your width of course.
The other gotcha is the -t argument (with nothing specified) which overrides the default column delimiter of a tab.  Aka, below I’m specifying nothing for the -t argument so the default ‘tab’ does not get inserted.  If you want to see how it works, specify the -t argument like this:
-t:***:
Then it will be obvious what the -t argument does.  Remember, there is a default value, so if you don’t want the default, you must override it.

The other syntax sugar is that there is no space after the -S argument.  As seen below (-SMyServerName\MyInstanceName).

bcp.exe  "SELECT cast(LastName as char(50)) , cast(FirstName as char(50)) , cast(MiddleName as char(50)) , cast(Suffix as char(50)) FROM MyAdventureWorksDB.Person.Person ORDER BY NEWID()" queryout PeopleRock.txt -c -t -T -SMyServerName\MyInstanceName

MyAdventureWorksDB is the name of the database.
Person (the first one) is the schema. 
Person (the second one) is the table.

I am using the AdventureWorks2008 database I got from codeplex.

The Order by NEWID() is completely useless and can be removed.  However, if I’m testing the export (most likely importing the data through another system) I like to leave that in so I don’t get the same data or same order every time when I am testing the output.

The other command line argument of interest is queryout.  I’ll let you see what that one does in the microsoft url below.

Here is the full bcp command line list:
http://msdn.microsoft.com/en-us/library/ms162802.aspx

Posted in Software Development | Leave a comment

CTE Running Total Example

I happened across a CURSOR based “Running Total” example on the web today.
Everyone with whom I work knows I detest Cursors in TSQL. 

So I coded up (one) alternate solution.

Below is (one variation) of a ‘Running Total’ solution using a CTE.
http://technet.microsoft.com/en-us/library/ms175972.aspx  (CTE Link)

———–START TSQL

Use NorthwindGO

declare @CustomerID varchar(6)

declare @BeginDate datetime

declare @EndDate datetime

select @CustomerID = (select top 1 CustomerID from dbo.Orders )

select @BeginDate = ’01/01/1900′

select @EndDate = ’12/31/2010′

;
WITH
MyCTE /* http://technet.microsoft.com/en-us/library/ms175972.aspx */
( ShipName,ShipAddress,ShipCity,ShipRegion,ShipPostalCode,ShipCountry,CustomerID,CustomerName,[Address],
City,Region,PostalCode,Country,Salesperson,OrderID,OrderDate,RequiredDate,ShippedDate,ShipperName,
ProductID,ProductName,UnitPrice,Quantity,Discount,ExtendedPrice,Freight,ROWID) AS
(
SELECT
ShipName ,ShipAddress,ShipCity,ShipRegion,ShipPostalCode,ShipCountry,CustomerID,CustomerName,[Address]
,City ,Region,PostalCode,Country,Salesperson,OrderID,OrderDate,RequiredDate,ShippedDate,ShipperName
,ProductID ,ProductName,UnitPrice,Quantity,Discount,ExtendedPrice,Freight
, ROW_NUMBER() OVER ( ORDER BY OrderDate , ProductName ASC ) as ROWID
FROM
dbo.Invoices inv /* “Invoices” is a VIEW, FYI */
where
inv.CustomerID = @CustomerID and (inv.OrderDate between @BeginDate and @EndDate)
)
SELECT
/*
ShipName,ShipAddress,ShipCity,ShipRegion,ShipPostalCode,ShipCountry,CustomerID,CustomerName,[Address],
City,Region,PostalCode,Country,Salesperson,OrderID,OrderDate,RequiredDate,ShippedDate,ShipperName,
ProductID,ProductName,UnitPrice,Quantity,Discount,ExtendedPrice,Freight,
*/

/*trim the list down a little for the final output */

CustomerID ,Salesperson,OrderID,OrderDate,ProductName,UnitPrice,Quantity,Discount,ExtendedPrice,Freight,(ExtendedPrice + Freight) as ComputedTotal

/*The below line is the “trick”. I reference the above CTE, but only get data that is less than or equal to the row that I am on (outerAlias.ROWID)*/
, (Select SUM (ExtendedPrice + Freight) from MyCTE innerAlias where innerAlias.ROWID <= outerAlias.ROWID ) as RunningTotal
, ROWID as ROWID_SHOWN_FOR_KICKS , OrderDate as OrderDateASecondTimeForConvenience
FROM
MyCTE outerAlias

/*Two Order By Options*/
ORDER BY outerAlias.OrderDate , ProductName

/* << Whatever the ORDER BY is here, should match the “ROW_NUMBER() OVER ( ORDER BY ________ ASC )” statement inside the CTE */
/*ORDER BY outerAlias.ROWID */ /* << Or, to keep is more “trim”, ORDER BY the ROWID, which will of course be the same as the “ROW_NUMBER() OVER ( ORDER BY” inside the CTE */

Posted in Software Development | Leave a comment

SQLCMD (Simple Example)

 
 
Download example HERE. (<< Right-click. "Save (Target) As" usually works best).

If I haven’t mentioned it before, thank you Microsoft for the sqlcmd.exe utility.
If you are new to sqlcmd or new to the concept of repeatability, below is a simple example.
It creates 3 databases that are exactly the same except for the database-name.
Can you say "repeatable" (as in every-single-time-repeatable-and-dependable)?
If you’re still using the GUI in Sql Server Management Studio and doing a "Right-Click"/"Add Database" for than anything for casual development, then please stop.
It is not repeatable.  It does not promote repeatability for "D-DAY" (that’s what I call deployment-day). 
 
My motto:
If you cannot deploy your code, your code and hard-work is essentially worthless.

 
The demo is simple.  A database, one table, and 2 stored procedures.  But the skeleton is there.
 
The example zip file is a demo example of creating a basic “OrganizationDB” (OrganizationDB01, OrganizationDB02, and OrganizationDB03).

Again, this is a sqlcmd.exe example of using $(Variable)’s to deploy databases….which emphasizes repeatability but with configurability as well.

Instructions:

Unzip the files……….read the README.txt file (the top portion contains setup information).

If you have ".\SqlExpress" setup as your development environment, you should be able to just click:
RebuildDatabaseMaster_AllThree.bat
and it should just work.


Look at the gif screenshot (in the zip), and compare to your database.

Then you can look at the README.txt file for explanation about what is happening.

There are also output log files you can look at (mentioned in the README.txt file) which will provide clues if anything does not work.

 

After you get it down, you’re essentially (always) one copy/paste/edit (the variable file) and a second copy/paste/edit (the .bat file) away from deploying a new database.

 

 

 ISQLW OSQL << These are the early versions of this tool, FYI.

 

Posted in Software Development | 2 Comments