Saturday, July 7, 2012

F# in SQL Server

SQL Server allows for routines written in .NET. There are plenty of examples everywhere. However, virtually all are written in C# or Visual Basic. I decided to give F# a try. I created a simple aggregate in F#, that provides a concatenation of input strings.

namespace _St

    [<SqlUserDefinedAggregateAttribute(Format.UserDefined, MaxByteSize = -1)>]
    type ConcatenationAggregate() =
       
       [<DefaultValue>] val mutable private agg : StringBuilder

       member private this.Append( s : string) =
           ignore <| this.agg.Append( if( this.agg.Length > 0) then ";" else "" ).Append( s )

       member public this.Init() = 
           this.agg <- StringBuilder()
       member public this.Accumulate(s : SqlString) =
           this.Append(s.Value)
       member public this.Merge(t2:ConcatenationAggregate) =
           this.Append(t2.agg.ToString())
       member public this.Terminate() =
           SqlString(this.agg.ToString())

       interface IBinarySerialize with
           member this.Read(r : BinaryReader) =
              this.agg <- StringBuilder( r.ReadString() )
           member this.Write(w : BinaryWriter) =
              w.Write( this.agg.ToString())

However, CREATE ASSEMBLY failed, because the referenced assembly fsharp.core was not present in the current database. So I tried importing the assembly to the database:

CREATE ASSEMBLY fsharp_core FROM 'c:\Program Files\FSharp-2.0.0.0\bin\FSharp.Core.dll'

but that failed too:

CREATE ASSEMBLY failed because type 'System.LazyFailure' in safe assembly 'FSharp.Core' has a static field 'undefined'. Attributes of static fields in safe assemblies must be marked readonly in Visual C#, ReadOnly in Visual Basic, or initonly in Visual C++ and intermediate language.

The solution was to import fsharp.core as UNSAFE assembly. Indeed, there was some more security stuff required for that. Luckily, there was a guidance on MSDN blogs.

CREATE ASYMMETRIC KEY CLR_SP_Key
FROM EXECUTABLE FILE = 'c:\Program Files\FSharp-2.0.0.0\bin\FSharp.Core.dll'
GO
CREATE LOGIN CLR_SP_Login
FROM ASYMMETRIC KEY CLR_SP_Key
GO
GRANT UNSAFE ASSEMBLY TO CLR_SP_Login
GO
USE MyTestDatabase;
GO
CREATE USER CLR_SP_Login FOR LOGIN CLR_SP_Login
GO
CREATE ASSEMBLY fsharp_core 
FROM 'c:\Program Files\FSharp-2.0.0.0\bin\FSharp.Core.dll'
WITH PERMISSION_SET = UNSAFE

Then it was rather straightforward.

CREATE ASSEMBLY ConcatenationAggregate
FROM 'C:\Test\ConcatenationAggregate.dll'
GO
CREATE AGGREGATE Test.CatAgg (@s nvarchar(max))
RETURNS nvarchar(max)
EXTERNAL NAME ConcatenationAggregate.[_St.ConcatenationAggregate]
GO
SELECT Test.CatAgg(EmailAddress) FROM Test.Person;

Tuesday, July 3, 2012

Hierarchical Data with FOR XML

I was playing with selection of hierarchical XML using FOR XML clause of Microsoft SQL Server. I was using AdventureWorks2008 database and wanted to select something like

<Root>
  <Group Name="Europe">
    <Country Code="DE">
      <Area>Germany</Area>
    </Country>
    <Country Code="FR">
      <Area>France</Area>
    </Country>
    <Country Code="GB">
      <Area>United Kingdom</Area>
    </Country>
  </Group>
  <Group Name="North America">
    <Country Code="CA">
      <Area>Canada</Area>
    </Country>
    <Country Code="US">
      <Area>Northwest</Area>
      <Area>Northeast</Area>
      <Area>Central</Area>
      <Area>Southwest</Area>
      <Area>Southeast</Area>
    </Country>
  </Group>
  <Group Name="Pacific">
    <Country Code="AU">
      <Area>Australia</Area>
    </Country>
  </Group>
</Root>
from Sales.SalesTerritory table.

FOR XML PATH

First, I tried to do that using FOR XML PATH. It was nothing particularly difficult. The only thing that is worth mentioning is that subqueries must be used to allow for grouping. (With grouping, I mean having multiple children inside a single parent node.)

SELECT [Group] [@Name],
        (SELECT [CountryRegionCode] [@Code],
                (SELECT t3.[Name] [text()]
                FROM Sales.SalesTerritory t3
                WHERE t3.[Group] = t2.[Group] 
                AND t3.[CountryRegionCode] = t2.[CountryRegionCode]
                ORDER BY [Group], [CountryRegionCode], [Name]
                FOR XML PATH('Area'), TYPE)
        FROM Sales.SalesTerritory t2
        WHERE t1.[Group] = t2.[Group]
        GROUP BY t2.[Group], t2.[CountryRegionCode]
        ORDER BY t2.[Group], t2.[CountryRegionCode]
        FOR XML PATH('Country'), TYPE)
FROM Sales.SalesTerritory t1
GROUP BY [Group]
ORDER BY [Group]
FOR XML PATH ('Group'), ROOT('Root')

FOR XML AUTO

It is not possible to mix attributes and elements when using FOR XML AUTO. However, if we do not mind having <Area Name="Germany" /> instead of <Area>Germany</Area>, it is still an option.

An interesting thing is that grouping is done automatically when selecting from joined tables. (Let me remind that FOR XML PATH requires subqueries for that to work.)

SELECT [Group].[Group] [Name]
 , [Country].[CountryRegionCode] [Code]
 , [Area].[Name]
FROM Sales.SalesTerritory [Group]
JOIN Sales.SalesTerritory [Country] 
 ON [Group].[Group] = [Country].[Group]
JOIN Sales.SalesTerritory [Area] 
 ON [Country].[Group] = [Area].[Group] 
 AND [Country].[CountryRegionCode] = [Area].[CountryRegionCode]
GROUP BY [Group].[Group], [Country].[CountryRegionCode], [Area].[Name]
ORDER BY [Group].[Group], [Code], [Area].[Name]
FOR XML AUTO, ROOT('Root')

FOR XML EXPLICIT

This is rather specific mode. The proper ordering is essential. (Note the Tag in ORDER BY clause.) That said, it works all right.

SELECT 1 Tag
 , NULL Parent
 , NULL [Root!1!!element]
 , NULL [Group!2!Name]
 , NULL [Country!3!Code]
 , NULL [Area!4!!element]
UNION ALL
SELECT 2, 1, NULL, [Group], NULL, NULL
FROM Sales.SalesTerritory
GROUP BY [Group]
UNION ALL
SELECT 3, 2, NULL, [Group], [CountryRegionCode], NULL
FROM Sales.SalesTerritory
GROUP BY [Group],[CountryRegionCode]
UNION ALL
SELECT 4, 3, NULL, [Group], [CountryRegionCode], [Name]
FROM Sales.SalesTerritory
ORDER BY [Group!2!Name], [Country!3!Code], Tag
FOR XML EXPLICIT

Conclusion

Both FOR XML PATH and FOR XML EXPLICIT can provide the required XML. The output of FOR XML AUTO would require an additional transformation, as it provides all data as attributes.

While FOR XML PATH is in general the most recommended mode, I did not like the fact I needed to use correlated subqueries. This very case is perhaps too tiny (10 rows, that is), yet the IO statistics are remarkable.

modescans reads
EXPLICIT36
AUTO3104
PATH10(!)140

That is, if you need to select XML and performance matters, it is always a good idea to try multiple FOR XML modes.

Friday, May 25, 2012

HtmlAgilityPack in F# Interactive

While using HtmlAgilityPack (version 1.4.0 Stable) in F# Interactive (of Visual Studio 2010), I was unable to access attributes of nodes. In order to debug, I converted my script into a compiled code and soon was notified that HtmlAgilityPack requires a reference to the System.Xml, Version=2.0.0.0 assembly. After switching the target framework of my project to .NET 3.5, I was able to reference the required assembly and all worked like a charm.

Tuesday, November 8, 2011

Dynamic Positioning of Overlay

jQuery Tools Overlay can be positioned statically when used out of the box, by setting top and left properties.

The good news is that the configuration can be overwritten in onBeforeLoad event handler. This is how an overlay can be positioned next to its trigger:


jQuery('.overlayTrigger').overlay({
   ...

   onBeforeLoad: function() {
      var overlay = this.getOverlay();
      var trigger = this.getTrigger();
      var overlayConfig = this.getConf();

      //position overlay
      overlayConfig.top = trigger.offset().top 
            - overlay.outerHeight() 
            - jQuery(document).scrollTop();
      overlayConfig.left = trigger.offset().left 
            - overlay.outerWidth() 
            - jQuery(document).scrollLeft();

      //... flip popup over when appropriate
      if (overlayConfig.top < 0)
          overlayConfig.top = trigger.offset().top 
                + trigger.outerHeight() 
                - jQuery(document).scrollTop();
      if (overlayConfig.left < 0)
          overlayConfig.left = trigger.offset().left 
                + trigger.outerWidth() 
                - jQuery(document).scrollLeft();
   }
});

Rendering of HyperLink

I have been using HyperLink control recently and have found its MSDN documentation incomplete.

The HyperLink control can be displayed as text or an image. Use the Text property to specify the text to display for the HyperLink control.

If both the Text and ImageUrl properties are set, the ImageUrl property takes precedence. If the image is unavailable, the text in the Text property is displayed. In browsers that support ToolTip functionality, the Text property also becomes the ToolTip.

I needed to insert some other controls into the child Controls of HyperLink and that is what is not covered in MSDN.

There are actually three mutually exclusive branches in rendering of HyperLink.

  1. If ImageUrl is set, an Image is rendered (Text property possibly used for an alternate text),
  2. children Controls are rendered when available, (there is presumably more to it, but I have not analyzed it further,)
  3. and finally plain Text is output otherwise.

By the way, ILSpy has proved useful in revealing the HyperLink internals.

Tuesday, June 14, 2011

Pex and unit testing

Everyone says that Pex generates unit tests. I am not sure if it holds. Pex knows what does your code do, but it definitely does not know what it should do.


Still I see three situations Pex can be really useful with.


  • It can report you exceptions caused by inputs you have not thought of.

  • When your code coverage is lower than you would expect, you may use Pex to see what paths you have not considered.

  • When you want to refactor a component that is not covered with unit tests, you can use unit tests generated by Pex to ensure you do not break the original functionality.


Wednesday, April 6, 2011

Learning F# with "Pex for fun"

Microsoft Research has published Pex for fun. It is a puzzle game. The only thing you are initially given is a signature of a function and you have to work out the implementation. There is an "Ask Pex!" button that will provide you some hints about how close you are. Wonderful!

There are couple of drawbacks though, such as that sometimes you are not given negative answers (inputs for which your implementation does not provide expected outputs), which is not very helpful. The most ugly thing I have found is how ChallengeFactorial puzzle behaves. Even the most stupid implementation (returning 0) is said to exceed the time-limit, thus effectively preventing you from succeeding.

Nevertheless, I like Pex for fun very much! And I find it especially useful when learning F# as it makes you explore the possibilities of the language.

Saturday, January 15, 2011

Sudoku solver in Prolog

Noticing sudoku almost everywhere, I thought it might be a good excercise to develop a sudoku solver. I spend a bit of time thinking and remembered that it could go easily in Prolog. So I started playing.


Sudoku has simple rules, every row, every column and every marked block contains numbers 1 to 9, each exactly once. Hence my first idea:

sudoku(Plan) :-
get_rows( Plan, Rows), % Plan = Rows,
get_columns( Plan, Columns), % transpose( Plan, Columns),
get_blocks( Plan, Blocks),
% every row, every column, every block is actually permutation of 1..9
maplist( permutation([1,2,3,4,5,6,7,8,9]), Rows),
maplist( permutation([1,2,3,4,5,6,7,8,9]), Columns),
maplist( permutation([1,2,3,4,5,6,7,8,9]), Blocks).


I am afraid the first idea, did not give any answer in reasonable time. I remembered that in large backtracking you need to fail as soon as possible. The second attempt

sudoku(Plan) :-
rows( Plan, []),
% columns are OK thanks to rows predicate
get_blocks( Plan, Blocks),
maplist( permutation([1,2,3,4,5,6,7,8,9]), Blocks).

rows( [], _).
rows( [ActualRow|RowsBelow], RowsAbove) :-
% instantiate free variables in the actual row
permutation( [1,2,3,4,5,6,7,8,9], ActualRow),
% only allow meaningful rows
maplist( constrain_difs( ActualRow), RowsAbove),
rows( RowsBelow, [ActualRow|RowsAbove]).

constrain_difs( [], []).
constrain_difs( [X|Xs], [Y|Ys]) :-
X \= Y,
constrain_difs( Xs, Ys).

had a great improvement. Immediately after the free variables were instantiated in the ActualRow, there was a constraint that the ActualRow had to be different from any RowsAbove on every column. As an effect, any row that would obviously break the sudoku was rejected soon and resulting columns were following the sudoku rules.


The second version responded in few seconds already.


Notes:

  • I have been using SWI-Prolog.

  • While browsing the documentation I discovered another sudoku solver in clpfd library, that limited variables to 1..9 domain and then used clpfd:all_distinct predicate. Elegant and fast.


Updated on 4 Mar 2011: Format of examples updated.














A sudoku example
521379468
46  15 7 
3  68425 
893526147
  6 4 9 3
 4 9 1  6
2   63895
 841 7  2
  5 9 7 4












... and solved
521379468
468215379
379684251
893526147
156748923
742931586
217463895
984157632
635892714

Thursday, November 11, 2010

Disabling Ctrl + Enter in Outlook

I appreciate the Ctrl + Enter keystroke in Visual Studio, it inserts a line above the one you are currently at and puts a cursor there. I got used to is so much, that I tend to use it out of Visual Studio even. I am afraid I have managed to send a good deal of unfinished emails like that from Outlook.

Finally, I have found a cure. There is an option to disable Ctrl + Enter keystroke in Outlook: Tools → Options... → E-mail Options... → Advanced E-mail Options... → clear a tick at "Press CTRL+ENTER to send messages".

Tuesday, July 27, 2010

Duplicated Page Parameters

I have just wondered what will happen if you set a page parameter twice. It does not seem to be explicitly documented in MSDN (I have not been particularly thorough in my searching), so I have just tried.

Having http://server/page.aspx?myparam=1&myparam=2, it is
"1,2"
that you receive from HttpContext.Current.Request.QueryString["myparam"].

The order seems to be kept, that is you obtain
"2,1"
having request http://server/page.aspx?myparam=2&myparam=1.

Tuesday, May 4, 2010

On "newly added" comments

I have just seen an enumeration modified. Some new items have been added there. The developer, probably taught to comment his code, added //newly added comments next to each new item. I guess he is very satisfied about the cleanness of his work.

I can imagine three possible scenarios.

First, no one will ever touch those comments. As a result, the items will remain //newly added for anyone reading the file, no matter the date.

Second, after some time, these comments will be deleted.

Third, and perhaps most in line with "comment your code" imperative, the developer will once change his comments to something like //not recently added and finally to //added ages ago.

What is the purpose of these comments? They are valid for some limited time. Then they just cause some confusion and/or maintenance burden.

If your code repository provides versioning, and our does indeed, anyone can reliably see what was changed in files. There is no need to manually list the changes anymore.

Monday, April 19, 2010

Why comments are evil

Comments usually are not maintained and sooner or later become confusing rather than helpful.

Recently, I have experienced another proof that comments are evil. I needed to examine usages of an identifier (web control name), that could be used both in C# code and in ASPX markup. Thus I needed to use "Find in files", i.e. text searching. There were found 51 occurrences, of which only 4 were not commented out. 47 occurrences were just slowing me down.

I wish there was an option "skip comments" in "Find in files" dialog box.

I wish people preferred deleting stuff to commenting it out.

Friday, April 2, 2010

Troubled Agile Adoption

Adoption of agile software development fails often. It has been written much on why does it happen. Recently, Cyndi Mitchell from ThoughtWorks Studios published an article named The half-agile path leads nowhere. I also liked the reaction of Steve Moyer, saying that Scrum is easy and XP is hard.

Scrum is rather easy. That is why companies tend to apply Scrum when they want to be agile. And they all want to be agile nowadays, don't they?

So what is the problem? James Shore wrote some year ago about The Decline and Fall of Agile, and he pointed out that it is how Scrum is applied that is bringing the disappointment. He even managed to put a reason -- people tend to say agile when in fact they mean sprints and daily meetings, that is to say processes and tools that Scrum offers.

And that is unsatisfactory, of course. They do not value Individuals and interactions over processes and tools!

I believe it is Manifesto for Agile Software Development that should guide you in becoming agile. Any processes and tools, although valuable, are secondary.

Sunday, March 21, 2010

Verbatim

Recently, we have discussed verbatim programs with my colleagues. Verbatim program prints itself on the output. I know that we did such a thing at the university in Pascal, but I could remember little of it. Thus I have decided to create one from scratch, although in C#. It took me some time to find a proper quotes escaping, that was the most difficult part of the program I would say. Still it is nice to see that even tough looking problems (initially) may have relatively easy solutions to be reached in a few minutes.

Here comes the source, perhaps not the best of its kind, but working fine.


using System;

namespace Verbatim
{
class Program
{
static void Main(string[] args)
{
const string ap = "\"";
const string s =
@"using System;

namespace Verbatim
{{
class Program
{{
static void Main(string[] args)
{{
const string ap = {1}\{1}{1};
const string s =
@{1}{0}{1};
Console.WriteLine(s, s, ap);
}}
}}
}}";
Console.WriteLine(s, s, ap);
}
}
}

Thursday, January 28, 2010

Long file names and paths

It was somewhat difficult for me to realize the file name length restriction in Windows. Finally I got it: The file name can only have the length such that the resulting full path will not exceed 260 characters.

E.g.: In C:\SomeDir\ you can create a file with 248 characters in name (including the extension), while in C:\SomeDir\SubDir\ you can use only 241 characters for a file name.

However, file systems used in Windows can handle paths up to 32,760 Unicode characters with each path component no more than 255 characters[1] It is possible to work with such long paths (i.e. exceeding 260 characters) in Windows, although there are some drawbacks. The best I have found regarding this topic is on BCL Team Blog.

Note: If you assign a drive letter to some folder, then working on that "drive", the full path will not contain the original folder anymore. Thus you can create a file, that you cannot work with properly in the original folder (the resulting path too lengthy), but still can be fully accessed on the "drive".

E.g.: Having mapped \\server\some_long_directory_structure\ to a virtual Z: drive, you can create and fully use a file Z:\some_file_with_long_name. However, that file may be difficult to work with when accessed as \\server\some_long_directory_structure\some_file_with_long_name.

Friday, January 15, 2010

TDD Kata

I did Roy Osherove's TDD Kata. It was refreshing and fun. When it came to step 8, my learning was stimulated, too. Well done, Roy!

I even plan to try this when doing pair programming again. I think such an exercise may help the pair to acclimatize.

I wish there was some "TDD Kata of the Day" page...

Monday, November 23, 2009

SelectedIndex if nothing selected

SelectedIndex property of Infragistics' WebCombo class has the following syntax


public int SelectedIndex {get; set;}

The documentation reads 'If no row is selected, this property returns Nothing or Null.' That sounds suspiciously since int cannot be null.

The property actually returns -1 if there are no selected rows.

By the way, the index is zero-based, i.e. the first row has the index 0.

Thursday, November 12, 2009

'+' not supported in datakey

Users reported us that sometimes the selection is lost in UltraWebGrid. We managed to discover that the problem occurs only when the key column contains '+'. The selection is recognized OK on client side, but on server side it behaves as if the selected row was the one without '+', i.e. none if there was no such row.

Having the data source


[DataObject]
public class DataProvider
{
[DataObjectMethod(DataObjectMethodType.Select)]
public static IEnumerable SelectData()
{
yield return new { Column1 = "a+b" };
yield return new { Column1 = "a b" };
}
}

and the grid

<igtbl:UltraWebGrid ID="UltraWebGrid1" runat="server" DataSourceID="ObjectDataSource1">
<Bands>
<igtbl:UltraGridBand DataKeyField="Column1">
<Columns>
<igtbl:UltraGridColumn Key="Column1" BaseColumnName="Column1" IsBound="True"></igtbl:UltraGridColumn>
</Columns>
</igtbl:UltraGridBand>
</Bands>
<DisplayLayout …>…</DisplayLayout>
</igtbl:UltraWebGrid>

you can select the row with "a+b", but once you enter server code (e.g. for server event handling), the selection is changed to "a b".

We needed to introduce another column to store the key value with '+' escaped somehow (e.g. replaced for another character).

Updated 3 February 2010:
I have been told by a colleague that even server events are not raised for a row having '+' in its key.

Thursday, September 3, 2009

Comma-separated Lists

One needs to convert a list of objects to comma-separated list of values sometimes. I often see StringBuilder used with some sort of loop. The most intellectual challenge is not to write comma after the last item; either there is some conditional expression in the loop step or some post processing is included to remove the last comma. Usually the result is consisting of several lines of code, perhaps too many for the purpose.

Users enthusiastic about Linq tend to involve Enumerable.Aggregate. Still String.Join may be the most straightforward method many times.

Thursday, July 30, 2009

The First Contact with FO.NET

I had a training recently on XSLT and XSL-FO. We were using java Apache FOP there. I was quite interested in the technology and was keen to give FO.NET a try.

Yesterday, I got to it finally. I found two drawbacks so far. Attribute width on inline element had no effect and I only found a workaround by using table. Then I discovered that block-container is not supported by FO.NET. But since you can have nested blocks, it is not a big issue.

I only worked with relatively simple document, so I guess there may be much more issues when using FO.NET. Still I was pleased with the output. I think FO.NET (and XSL-FO in general) might be a good option for creating PDF documents, especially when the source data are in XML.