Symmetri Developer Blog

Does adding the same event listener multiple times to a Flex object cause multiple, duplicate postbacks? I didn’t know for sure, but I just tried it and the answer is (thankfully) no. Let the good times carry on, then.

We just joined UBank and I noticed that their secure online banking site uses Flex within the HTML page as the most important and functional part of the interface. You can’t see what I mean unless you are a customer and have a valid login … and I realize this is hardly exciting to anyone else but me. But having created a series of real-world Flex applications over the last year I was happy to see that the most”serious” type of online applications are now starting to move in the direction of becoming RIAs.

This is just a little thing, but I have noticed that it isn’t widely understood. When using CSS in Flex to style components, the font-family property can be case sensitive on some operating systems. For example, the following CSS will not work on my Safari browser with Flash Player 10:

.content{font-family: arial;}

but this will work:

.content{font-family: Arial;}

I’ve been using SQL Server 2005 SSIS Import Wizard (the replacement for SQL Server 2000 Data Transformation Services (DTS)) lately to import data from Excel files, and I’ve come across a serious bug that should be documented. It appears that when importing data from a spreadsheet that is sparsely populated - having a lot of empty cells - the import process can fail to import some of your data. Specifically, if you have a column in your Excel spreadsheet that does not contain any data for the first 1000 rows (or even a few hundred - I have not been able to work out the exact threshold), then you may find that data in subsequent rows for that column is imported as NULL.

What is worse is that no error messages or warnings are shown, so you have no idea that anything has gone wrong. The only way to detect the problem is to manually compare the imported data row by row with the import data source spreadsheet.

This problem persists no matter what data type you might choose for the destination column.

Searching around in the forums for this issue, I have come across some talk of changing the connection string used to the OLEDB connection to include “IMEX=1″. That works fine if you are building a SSIS Integration Services package in the full version of SQL Server 2005. But there is no way to do anything like that in the SSIS Import Wizard - no access to connection strings and no way to choosing different providers for importing Excel files.

So I have found my own workaround which is quite simple. What I do now before importing any Excel sheet is to check the first few rows. If there are columns that have empty values in the first few rows, I check further down the sheet. If the same column has data further down, I know that I am in danger of losing it during the import. So I create a dummy first row of data that has all columns filled out with representative data. For example, if a column contains integers, I might put a 1 in the dummy row; if it is string data, just a few characters. This seems to make the import process less cavalier about ignoring data, and I find that all columns now import correctly for all rows.

After the import, I run a query to delete the dummy row of data.

The workaround is fairly simple, but I must say this is a horrendous bug. Not only is it a high impact bug but the user has no feedback that anything has gone wrong. This is on top of the problems I have had with arcane error messages when an import fails because of bad data; and problems with row delimiters when importing CSV files. I have lost confidence in the SQL Server SSIS import now. The old SQL Server 2000 DTS seemed much more stable and reliable. SQL Server 2005 has been generally good to use, but not having a reliable way to import data is a severe handicap and makes me less likely to continue using it.

By the way, you can find an excellent tutorial on how to use the new SSIS Import/Export Wizard in SQL Server 2005 here.

The Microsoft SQL Server 2005 Express Edition install does not automatically include the Data Transfer Services (DTS) package that allows you - for example - to easily import data from spreadsheets. That’s a surprise to those of us who are used to using DTS in the SQL Server 2000 Enterprise Manager. It turns out you can get a newer version of the DTS by downloading the Microsoft SQL Server 2005 Express Edition Toolkit from http://go.microsoft.com/fwlink/?LinkId=65111. - it’s a fairly big file of over 200MB.

After installation, the DTS Wizard is found at C:\Program Files\Microsoft SQL Server\90\DTS\Binn\DTSWizard.EXE. Unfortunately you can’t get full integration into Management Studio Express as we are used to in Enterprise Manager (right-click database->Import Data etc.) but you can create a shortcut to the wizard within MSE by going to Tools->External Tools->Add.

There is a fair bit of chatter over at Adobe’s Flex blog about the Data Visualization Components and the AdvancedDataGrid in particular. Some developers have complained to Adobe about the number of bugs in AdvancedDataGrid and the slow pace of bug fixing; many have said that the ADG is basically unsuitable for production.

Here is another popular post that makes the point that the ADG (and the Flex framework codebase in general) is ill-designed and bloated.

I’ve just completed a commercial project which used the AdvancedDataGrid and Charting components from the Data Visualization libraries, and would like to share my view. Generally speaking, I felt that the experience was a positive one. There were bugs that I encountered and had to work around (such as this one and this one). I did have my share of WTF moments. But in the end I was able to get a complex visualization product out the door on schedule without hitting any real roadblocks. Some workarounds required creativity, but I never found myself stuck for a solution; I was always able to overload a method, extend a class or find an alternative way to get things working. Given the specifications for the project - delivering rich, interactive reports through the browser for a variety of client OSes and platforms - and the very tight timelines, I have to say that I was happy in the end that I chose to use Flex.

Here’s another point I want to make. Adobe has open-sourced the code for the Flex framework, and that is a positive development. Any negative comment about Flex framework code quality is offset by the benefit of transparency - for a .NET developer, the fact that we can read the framework code at all is a novelty. The big picture here is that we have a commercially developed codebase that is being poked and prodded to death by the developer community. Many of the concerns about code quality are valid, but I think Adobe has done the smart thing by throwing its baby to the wolves, as it were. In the long term, the Flex framework will be improved, and the engagement with the developer community - albeit negative in parts - will be a genuine differentiation between their RIA platform and the Microsoft competition*.

Also, I think the developer community as whole should has a collective personality that is hard to please. We software developers love to take other people’s code and tear it apart. What we see are the logical and design flaws, the shortcuts, the omissions and the lack of documentation. What we don’t see is the context within which the code was written in the first place - the deadlines, the commercial pressures, the collaborative environment, office politics and the whole zero-sum game of allocating finite resources between new development and support. That stuff can be mitigated but it can’t be eliminated. In the real world, code is never written in a vacuum, and no one has the luxury of getting their code exactly right.

Of course the Flex framework has flaws, and Adobe has a responsibility to the development community to listen to feedback and address issues in good faith. If they fail to do that, they deserve the bad reputation that would follow. But the real test of software is whether it gets the job done, not whether it violates the latest popular design principles or whether it conforms to one’s personal taste. By that yardstick, I give Flex a (qualified) thumbs-up based on the experience of taking two-and-a-half real-world Flex projects through to production.

One final point. I have done RIA projects in both Flex and Silverlight/WPF over the last year. Each experience has had its share of ups and downs, but neither has come out clearly better for me. As developers, we can’t write our own frameworks and SDKs - we have to work with the available alternatives. The Flex SDK may be flawed in absolute terms, but what is more important is its relative quality as compared to the competition. And that we can’t compare right now, because the .NET framework is a black box. I wonder, after Adobe’s move, would Microsoft consider making it transparent?

*UPDATE: Microsoft is indeed planning to release the codebase for the .NET Framework libraries. That’s great news, and an important step forward for the whole concept of open-source. It will generate a lot of goodwill for Microsoft.

Shapefiles are a commonly used format for storing GIS data. But often you have a shapefile that is too large for a particular application - for example, loading into a webpage. In these cases, you may want to simplify the geometry of the data and deal with a reduced filesize at the cost of geometrical complexity.

Mapshaper.org is an online Flash tool for exactly this scenario. You can upload a shapefile, select the mode and extent of simplification, preview the result and then export the modified shapefile. Only caveat - I could not get it to work in the Flash player plugin for Safari.

I have a project that requires building a web application to do data visualization using maps; showing data by postcode region within Australia and making those visualizations available through the web browser. Postcode region data is available in GIS data files in MapInfo (MIF) format; other data will be available in some sort of database driven format such as XML or JSON. We want output that looks something like this (with or without the 3D effect):

I’ve spent some time researching possible solutions and thought I might publish a summary, to help someone else who might be researching a similar problem.

Eight options for a solution are considered, the last of which was to build map interfaces completely from scratch; the other seven involve using software packages or using online APIs for building maps. I evaluated each solution on several different criteria and made some summary notes about each one. Hope this might help someone else to get started in building online mapping solutions.

UPDATE: I have found a commendable little project on the web at DreamingWell that parses shapefiles directly from Flex and renders maps. That could the basis for a custom solution.

The Random class in the .NET framework is used to generate “random” numbers. I use quotations around the word “random” because the output of the function Random.Next() is actually what is known as “pseudo-random”; it is the output of a deterministic mathematical algorithm that “mimics” randomness. Specifically, it is an implementation of Knuth’s subtractive random number generator, which is an algorithm that performs incremental lookups on a large table of values and outputs the difference between selected numbers. The output is close enough to random for most practical purposes encountered in software development, with the exception of cryptography (for which purpose the .NET framework also includes the System.Security. Cryptography.RandomNumberGenerator class).

Recently I was confronted with an interesting question in the line of work: just how “random” is the Random class? Is there a way to quantify the “randomness” of a so-called random number generator (RNG)? One can imagine a linear scale of RNGs ranked by “randomness” - at one end would be entirely predictable, non-random generators (such as a routine that returns a constant value); and at the other end what we might call “true” random processes that occur in nature such as atomic decay¹. In between, all other RNGs that one can think of would lie somewhere along that scale. But how exactly can they be ranked and “randomness” quantified?

It turns out that there are several suggested ways of measuring randomness. They all start by taking a sample output from the RNG in question, and one that is of sufficiently large size (bit streams of about 12MB have been mentioned in the literature); and then subjecting that output file to tests. There are many different testing processes that have been proposed for the task. Below are a selected few:

1) compression - a simple test is to compress the sample output using commercial compression formats such as ZIP and then to compute the ratio of compressed to uncompressed size. This test is really only good at finding large biases in your RNG; i.e. if the compression is able to siginificantly reduce filesize, you should assume that the RNG is producing output with definitive patterns

2) a visual test of Monte-Carlo plot - by plotting the values of RNG output onto a two-dimensional array, any large scale patterns become apparent to the human eye. A great example can be found at RANDOM.ORG²

3) DIEHARD test - the DIEHARD series of tests was developed by Prof. George Marsaglia at Florida State University. It applies a number of statistical tests to the sample RNG output; the composite of their results can be used to make an overall pass/fail judgement. It is a well-regarded process that is used, for example, to certify the fairness of the operation of online gambling sites

4) test for FIPS 140-2 - FIPS 140-2 is a standard maintained by the National Institute of Science and Technology in the United States that concerns itself with the security of cryptographic providers. Section 4.7.1 outlines requirements for approved RNGs; they are based mainly around the idea of entropy, which very loosely speaking is a measure of the information content of a sequence

Exactly how far one wants to go in testing a RNG for randomness depends on the context. For commercial non-cryptographic applications, it is probably good enough to use the built in RNGs that come with software development platforms - such as the .NET Random class (a caveat is that the seeding process for the RNGs should use a source that incorporates some natural uncertainty - such as a combination of time, local machine state and user input). For stronger “randomness”, there are a number of implementations of RNGs floating around in the public domain that can be used (e.g. a popular one is the Mersenne Twister). RNGs with a very high “randomness” have to rely on external, physical sources of entropy such as atmospheric data or atomic decay, and are probably only warranted in very specialized scientific applications.

¹There remains the philosophical question of what is “truly” random; for example, do the quantum mechanical laws that govern atomic decay generate “output” that is the result of some as-yet undiscovered deterministic process? The answer is probably no, but it’s a question that exercises the minds of some of the world’s best physicists and philosophers.

²The demonstration shows the alarming large scale patterns inherent in the RNG that is the PHP rand() function - alarming because most developers think of functions like that as “true” random number generators

So I’ve just deployed my first WPF project via the web using ClickOnce and thought I would share some of the issues that came up. Firstly, the idea behind ClickOnce is great - creating a single web page link that can do the installation of an application and all of its prerequisite software in one go. However, there were a few issues that came up with the publishing using VS2008:

1) the FTP option in the publishing refused to work. I checked the server name and username/password credentials a hundred times, but the publish failed again and again. The error messages aren’t particularly helpful; you really have no way of telling what exactly went wrong. In the end, I decided to publish to a local folder and do the FTP myself. No biggie, but annoying nonetheless.

2) after deploying to a webpage, I decided to check the installation process; I clicked the “Install” button on the generated webpage from a fresh Vista install. After the usual security warnings, the first thing the install did was to install .NET 3.5 SP1. However, it took a really long time - over ten minutes; and the progress bar hardly moved all the time. I am not sure if that is normal, but it seems that I’ll have to make a note of that when deploying to client’s machines; the average user will probably lose hope and lose patience with that kind of wait unless forewarned.

3) next the installation script tried to install the application itself, but I found that it would not install; the error message I got was “The assembly Microsoft.Windows.Design.Extensibility needs to be added to the GAC.” So it seems that there is a required assembly that is not being installed by the ClickOnce process. Checking back in Visual Studio, I noticed that the assembly “Microsoft.WPFToolkit.VisualStudio.Design” was part of the project references; it seemed to me that any library which is specific to Visual Studio should have nothing to do with the deployed application; so I removed and re-published.

4) That got rid of the previous error all right, but now I found that the install complained about missing other assemblies: Microsoft.Data.SqlServerCe.*. My application did indeed use SQL Server Compact Edition 3.5, but it seemed that the install was not checking for it or downloading it.

The issue here is that the project has references to certain assemblies (Microsoft.Data.SqlServerCe.*), but the publishing process does not automatically pick up the prerequisite software associated with those assemblies. The solution is that we must explicitly define those prerequisites. This can be done under Project Properties in Visual Studio under the Publish tab; press the “Prerequisites” button. In my case, I needed to add “SQL Server Compact 3.5″ as a prerequisite to be downloaded by the installation.