theThought's thoughts

Kevin A Gray - Creative Strategy Guy

A Week in Malaysia

Three months into my new role as Creative Director at Synovate, I found myself travelling to Malaysia to identify whether it was possible to build a new design team in Kuala Lumpar.  This has been my first trip to Malaysia, although not my first to Asia as I previously travelled to Xi-An in China to work with the IBM SPSS Data Collection Development team.

The purpose of the trip:  to interview and hopefully recruit a new design team that will help me meet the main objectives of my role (improving respondent engagement, improving client engagement and delivering end to end solutions that increase company profitability). 

Kl_skyline

The process started two weeks before with a conference call to a number of local recruitment agencies.  During the call I defined the roles I wanted to fulfil and the skills I was looking to find.  Previous attempts to engage in the recruitment process remotely had resulted in a slurry of applicants with the wrong combination of skills.  Consequently when I started this recruitment drive I was not convinced that I would be able to find the talent that I required.

During the ensuing two weeks the agencies aimed to fulfil their target of providing 20 candidates that I would accept for interview.  As my requirements are quite stringent they only managed 20 in total of which five were rejected as unsuitable.

I will not go into detail regarding the recruitment process we used, its enough to say that the technique employed was based on my recruitment experience in IT, Design and Education (I have sat on a number of recruitment panels for teachers, deputy head teachers, school business administrators and even head teachers).  I find that the process used in education is more formal, more structured than those used normally used in business.  The process used in this instance included focus on four areas: personality, skill, technical awareness and team work.

I arrived expecting to encounter a certain degree of “culture shock”.  Having never been on the island and certainly never recruited there I was pretty sure that there would be a range of social and cultural differences. In China, for example, there seems to be a real reluctance to ask questions, it took a significant amount of effort to teach new staff that asking questions is essential, that their perception of requests and reality could well be different and that only by asking questions and seeking clarification could they ensure that communication is efficient.  As a consequence I engaged three members of the local team to help with the interviews.

There is no doubt that there were differences.  The designs in their portfolios had a distinctly Asian influence to the exclusion of any real non-Asian style.  Their technical skills were based on technologies that are not particularly new and most had little or no knowledge of the subtleties of events in their relevant fields of expertise.  On the positive side, the majority were keen, enthusiastic and willing to learn, the skills that they did have were practically sound even if their theoretical knowledge was not as complete as I would have liked.

By the end of the process it was obvious that in previous jobs the candidates had been encouraged to listen to the exact tasks they have to do and to execute without really thinking about the relative merits, or lack thereof of those tasks.  They were not encouraged to monitor new technologies and identify ways in which they could introduce new ideas.  Despite all this I believe that the recruitment process was successful I managed to find five candidates who showed good skill (albeit dated) and significant potential and hopefully will develop into a strong versatile team.

Synovate_kl

My trip was not limited to the short term issue of building a team,  I also wanted to look to the long term and identify ways in which it might be possible to improve the quality of future candidates.  Consequently I spent a day visiting  a number of universities looking at the courses that related to the skills I look for and identifying if there is a way in which we can share expertise to our mutual benefit.  It is too early to say whether this will really be possible but there is no doubt that there were a number of potentially exciting and beneficial opportunities that could deliver significant reward in the years to come.

So, as I travel home ,(currently I am flying between Malaysia and Hong Kong) I await confirmation that each of the selected candidates is willing to join the team and I am hopeful that we will be able to deliver a range of solutions over the forthcoming months that will show the true possibilities when you apply creativity to the IBM SPSS Data Collection software.

Filed under  //   IBM SPSS Data Collection   Malaysia   Recruitment   Synovate  

"one for all" is definitely possible - the Test Centre

In my previous post (read it here) I classified browsers into three types:

  • Basic
  • Simple
  • Rich

This was done so that I could organise the way that I build templates for IBM SPSS Data Collection.  This is part of my project to build a single template that works for all browsers both desktop and mobile.

I defined these three classifications by building some simple web pages and testing a number of different browsers on them.  The web pages has a range of capabilities from basic HTML with CSS styling to complex HTML 5/CSS 3 and JavaScript.

The Environment
A first I tried building a test, test environment on my new Dell 1645, a 64bit laptop running Windows 7 64bit.   Installation of the main desktop browsers was not an issue, however almost all the emulators failed to work properly.  Consequently I stopped and reverted to a Windows XP operating system running in a vmWare image.  I built this using vmWare Workstation and started by taking a blank Windows XP SP 2 install and then performing a series of upgrades to ensure that I had all the latest patches applied and .net 3.5.

The Emulators
Once that was done I started installing desktop browsers:

Image001
 

Then I moved on to emulators for mobile phones.  There were a number of emulators that I wanted to try including the OpenWave emulator that I simply could not download.  It seems that the OpenWave browser and many of the older emulators are no longer available.

I installed a Sony Ericsson emulator which was written in Flash.  This was, consequently a simple installation from Sony Ericsson’s Developer Site (PhoneGap Simulator).  This was by far the easiest of the installers to get working.

Image002

The other two emulators that I wanted to install were the Blackberry, because I believe that this is a fairly limited browser that being built by RIM cannot be described as standard and the Android emulator.  I would have liked an iPhone emulator but alas my machine is a PC not a mac so I have to make do with testing on my iPod Touch.

Both of these emulators require the Java Runtime emulator (JRE) so that was the first element to install (JRE is available here).  Additionally the RIM emulator requires the Java Development Kit (download JDK here).

Next it was off to the Blackberry site to download an emulator.  I chose one for the Blackberry Storm (9500), however it is my understanding that you can download a number of them (get a blackberry emulator here).  Downloading the emulator, however is not enough if you want to browse the Internet.  It is also necessary to install MDS which is a Mobile Delivery Service created by BlackBerry.  Not only does this need to be installed but it also needs to be run at the same time as the emulator.  Testing with the RIM was concerning.  There are two modes the first is Page View this is a smartphone mode that I had expected to be as good as Sony Ericsson, Android and iPhone.  What I found was that although it supported HTML 5 and a fair amount of CSS it did not really support JavaScript (even though JavaScript support is switched on).  The other mode is called Column view and is a more basic view that is similar to that provided by very basic mobile browsers.  As can be seen by the following example neither gives a very good experience.  It is really troubling that these phones are the phones of choice for Businesses.

Image003
 

The last emulator was the Android emulator (get the android emulator here).  Like RIM there is an opportunity to download a number of different emulators, unlike RIM downloading them all is much easier and therefore I did just that.  Also like RIM the Android needs more than just JRE and the emulator.  There is also a need for eclipse (get eclipse here).  Careful with this download.  It is a zip which needs to be extracted in the location where you want to run it.  The setup is not an installation routine just a way of quickly setting up the environment and getting to the emulators once they have been installed.  Once these items are installed you are presented with an array of different device types.  Each takes quite a while to load but there is no doubt that the wait is worth it.

Image004

So that just left aDesigner.  This too needs Eclipse but as that was already installed this final installation was easy.  I may extend my list of browsers and emulators over time (a proper Nokia S60 is very likely).  I would really like a proper iPhone emulator rather than just Safari as Safari does not give an indication of how screen real-estate is working nor does it allow me to tilt the device.

If you have any emulators that you think I have missed feel free to drop me a line.

Filed under  //   Android   Browsers   CSS 3   Chrome   Emulators   Firefox   HTML 5   IBM SPSS Data Collection   IE   JavaScript   Opera   Safari   Simulators   Windows XP   iPhone   vmWare  

"one for all" is definitely possible - the Browsers

A couple of weeks ago I wrote an article questioning whether it would be possible to create a single template that handles the vast majority of browsers and allows Data Collection to deliver the best experience each browser can deliver (read Is “one for all” even possible).

I am pleased to announce, after a number of weeks of hard work that the answer is “YES”.

So what has happened over the last few weeks that allowed me to come to this answer?

The steps so far
I have not yet reached the end of my journey, however, I thought it might be a good idea to allow you to catch up.  I have started/completed a number of tasks including:

  • A review of different browsers on the market (or previously on the market) identifying the capabilities they provide
  • Locate and install a number of browsers and emulators into a Test environment
  • Build a small Data collection survey to allow testing
  • Start building a solution

Browsers

I started with the currently installable browsers (Internet Explorer, Firefox, Opera, Safari, Chrome)  Then I tried older versions of IE.  Lastly I obtained some emulators (Blackberry, Sony Ericsson, Nokia), finally I installed a copy of aDesigner for testing JAWS screen reader compatibility.

aDesigner an aSide
If you have not encountered aDesigner before and you are seriously interested in web accessibility you need to look at it now (aDesigner is available here).  aDesigner was built by IBM but it has been donated to the Accessibility Tools Framework (ACTF).  Not only does aDesigner provide clear information about the general level of accessibility of a website (based on its markup) it will also render the website from the perspective of “low vision“ showing how those with poor eyesight will see the site.  Furthermore, if that was not enough, in addition to testing websites it can also test ODF (Open Document Format) files, Flash content and general GUI accessibility of any application.

Image001

3 Classifications of Browsers
Having reviewed a multitude of browser capabilities I ended up with three simple classficiations.  These are based on the capabilities of the browser and will affect the way in which the template will be designed.  The basis of the categorisation is the fact that there are, currently, primarily, two platforms that have to be handled:

  • Desktop Browsers (PC, Mac and others)
  • Mobile Browsers (smartphone, basic phone, portable devices)

Against these two basic platforms there are a number of capabilities that may or may not be used:

  • HTML
  • CSS
  • HTML 5
  • CSS 3
  • JavaScript

All of the browsers support HTML although some only have limited support of CSS.  Only a few have support for HTML 5 and CSS 3 (as I wrote this Microsoft announced that this list will expand to include IE 9 – review by the Register.com).  Lastly there is JavaScript.  This is more complex as browsers that naturally support JavaScript can be “crippled” by corporations that disable its use.  The following graphic shows how these different elements interplay with each other.

Image002
 

From this I defined three classifications of Browser as follows:

Basic Browsers           OpenWare browser, Basic (old) Mobile browsers, very early versions of IE and Netscape (Full HTML coverage, limited CSS coverage, little or no JavaScript coverage)

Simple Browsers         IE, Netscape, RIM, Nokia browser (Full HTML/CSS coverage, no HTML 5/CSS 3 coverage, some JavaScript Coverage)

Rich Browsers            Chrome, Firefox, Safari, Sony Ericsson (Full HTML/CSS and HTML 5/CSS 3 coverage, Full JavaScript Coverage)

Filed under  //   ACTF   Blackberry   Browsers   CSS   CSS 3   Chrome   Emulators   Firefox   Flash   Form Field   HTML   HTML 5   IBM   IBM SPSS Data Collection   JAWS   JavaScript   Netscape   Safari   Simulators   aDesigner  

Is "one for all" even possible

I have seen a number of different approaches to the creation of templates for IBM SPSS Data Collection that aim to handle multiple languages, multiple browsers and multiple question types.  The one approach I have not seen is a single master template with supporting sub-templates and transparently handle all desktop (Mac/PC) browsers and also provide an experience for basic mobile browsers.

If you look into the Internet you can find many companies who have achieved this for some or all of their websites.

So the question is, is this possible? Does it provide any benefits? Is it an approach that all customers can take to extend the reach of their surveys?

At the heart of the issue is the plethora of browsers available today and the range of capabilities of each of those browsers.  This can be seen in wikipedia’s firstly their comparison of web browsers and secondly their comparison of mobile browsers.  If there were a clear dominiant force in either or both of these markets they the majority of website designers would build for that format.  This was the case through the 90’s with the dominance of Internet Explorer (IE).  However that dominance is effectively at an end as shown by the following infographic:

The above graphic shows the long term evolution of desktop browsers but it ends in August 2009 and quite a lot has changed since then.  Furthermore it does not cover mobile browsers.  The next three charts are from the same web service and provide a more up to date picture (for more detail and other charts go to StatCounter).

Figure 1: Desktop Browsers by Version

Image004

This chart really highlights the rapid decline in Microsoft’s dominance.  Okay there are three versions out there and the still amount to a very high percentage of market share, but it has fallen by 20% over the last few years and with issues such as the provision of choice for Europe (where Europeans are presented with a choice of browser rather than automatic installation of IE) market share can only continue to decline.

Figure 2: Desktop vs Mobile Browsers

Image002

 

This graphic surprised me because there is a lot of noise about mobile internet but this indicates that that noise is largely irrelevant at this time.  This probably reflects the fact that a large portion of users are concerned about data costs and that combined with the percentage of users who do not have good internet access (no smartphone) means that mobiles have a way to go before they will challenge desktops.  Further exploration at a country level shows that emerging territories such as India are showing more movement towards mobile browsing than Europe or the US.

Figure 3: Mobile Browsers

Image003

This graphic shows that there is no clear dominant factor in terms of Browser when looking at mobile devices.  Many in Europe and the US may be surprised by the fact that Opera is top but in territories such as Japan this browser has real dominance.  Opera Mini is seen as a very good browser and other non mobile products (such as the WII) use this browser.  It is not clear whether the WII would be seen in the desktop or the mobile category.

With the advent, but not wholesale adoption of, HTML 5/CSS 3 and the fact that this standard is not due to be ratified until 2022.  There are new ways to engage browsers users with more visual richness.  One of the most talked about features of HTML 5 being its ability to embed video without the need for a plug-in such as Flash or Silverlight and its consequential high profile adoption by YouTube (amongst others).  The approach by IBM/SPSS Data Collection has always been to be browser agnostic.  This means delivering solutions without JavaScript and therefore without any richness.  As much as this may have been a sensible policy 5 or 10 years ago, it now runs against the mainstream perception of the Internet and so Data Collection consumers need to add their own solution. 

A small number of customers have explored User Agents to discover details regarding the device being used and to then send the person to the relevant template.  This requires constant support as User Agents are not delivered in a standard way by providers and they only provide a relatively small amount of information.  Also the building of multiple templates significantly increases the cost of support should customers come along requiring their own branded versions.

Its seems to me, therefore that a new approach is required.  This would be a three stage approach utilising a combination of Graceful degradation and Progressive Enhancement (a short explanation).  It starts with a basic HTML solution that is a fits all solution.  It is then built upon using CSS capabilities to improve presentation and overall layout.  Finally JavaScript is used to add richness where richness can be added.  But is this solution possible?

Because I like a challenge I have set myself an objective of delivering such a solution over the next few weeks.  I am interested in your opinion.  Will I manage it? what should I be looking out for?

As usual I will keep you informed of my progress and show the end results so watch this space.

"No Product is just One Man"

I have been delighted to receive no small number of e-mails (mostly via LinkedIn) in relation to my last two posts and my departure from SPSS.  A fair percentage of them have expressed concern for Data Collection now that I am not there.

These concerns are very kind, however it needs to be made clear that as influential as I was in the shaping of Data Collection (into its current form - and that of its next release later in 2010), I was not the only person on the team.

Peter Snow gets enthusiastic with the Swingometer
http://en.wikipedia.org/wiki/Swingometer

In the British election of 2005 Peter Snow famously used the Swingometer to help predict the election result based on which seats were won a lost. As a keen watcher of election nights (geek that I am) I have always used a similar principle to look at the potential damage inflicted on a team when it looses a staff.

In my case I use a dart board.  Positioned on the dart board are people who influence a product, be they developers, marketeers, sales people, company directors or strategists.  The closer they are to the centre the more important they are and the bigger the impact if they leave.

The board is not static, not only are people added and removed as they join/leave but people move in and out as their influence waxes and wanes.

In 2004 when I joined SPSS I was very much on the outer rim and when I left, in 2010,  I was indeed much closer to the centre.

During those 5 years, Data Collection definitely took a number of serious hits.  The most notable of those in the early days was Paul Petersen.  His drive, influence, imagination, the strength of the team he ran and the respect he had in the industry resulted in his departure being a "significant loss".  However people not quite so  near the centre, like Lance Nichols, stepped up to help fill the gap.  There is no doubt that I (still travelling inwards took up a certain amount of Paul's responsibility).

An equally big loss happened in 2009 with the departure of Patrick Quigley.  His loss was probably more noticeable to me than Paul's because I had worked with him for longer.  As most strategic customers will know Patrick was a shining light in the SPSS organisation and helped Data Collection grow as a product by delivering new customers and ensuring existing ones stayed.

But again his leaving gave others the opportunity to step up.  Dave Suedkamp (still at SPSS) is now probably closest to the centre his knowledge, willingness to help customers understand how to get the most of Data Collection is immensely important as is his influence in the higher echelons of SPSS.   And lastly, but certainly not leastly Jane Moore (Sales Person of the Year 2009 at SPSS - well done Jane) is an ever present who I most enjoyed working with and who is helping IBM understand that Data Collection is worth investing in.

So yes, there will be an impact to Data Collection now that I am no longer there, but it will recover.  As a result it may be slightly different in the future but it should still be the best survey platform in the world and worth including in your software portfolio.

Filed under  //   IBM SPSS Data Collection   LinkedIn   Synovate   swingometer  

1903 days, but no longer counting

Today - 29th January 2010 is my last day at SPSS, and IBM Company.  

The last five years have been dramatic and very interesting.  I have been given an opportunity to take IBM SPSS Data Collection (formerly Dimensions) from a script orientated web-survey platform for high-end market researchers to a truly versatile interviewing solution where the respondent can choose the mode of interviewing that fits their lifestyle.  

But enough of the marketing spin.  I hope that I have helped those companies out there who are using Data Collection by delivering a number of high quality releases and transitioning the Development team from a two small groups based in the UK and Denmark to a large team based in Xi-An supported by a smaller team based in Rochester.

It has not been all smooth running, the team has had its issues and so has the product, but I leave with the expectation that Data Collection is very much on the up.  The 5.6 release is solid and the next release (currently in the latter stages of development) will deliver a big step up in various areas.

I have been well supported, especially by my boss (Jason Verlen) and my team (Nick Read and Hetal Thaker).  They remain to ensure that the effort I have put in to delivering consistency and transparency to the Strategic direction of the product set is not wasted.

There are many unsung heroes in the team including Rob Bristow ( a stalwart in support), Lance Nichols (a naturally gifted engineer), Allison Carleton (who terrified me in my interview but was great to work with), Jane Moore (without whom customer engagements would not have been so exciting)

Finally to Colin Linsky who was one of two people who saw something during my initial interview that made them want to offer me a job and then has been working with me ever since, challenging my abilities and supporting my efforts.  Thank you very much to all of you and the many others who I have not mentioned individually but were equally an important part of the team.

In leaving it is often customary to ponder the good and bad points, I have always been a keen evangelist and I hope that those who have attended my presentations at various events have enjoyed the experience.  I would certainly like to say thanks to all my groupies (I wont mention names) who asked intelligent questions at the right time and clapped loudly at the end.  You always made it worth while.

I joined SPSS from the world of New Media and the one major area of disappointment during my time was the lack of understanding of the need for creativity and community. It is my personal opinion that this is a flaw with SPSS that comes from its history and is very difficult to change (even if they wanted to) It is probably this that is the main cause for my departure.  I missed these things too much and want to get them back.

So where am I going? well to find that out you will have to wait till Monday ...

Filed under  //   IBM   IBM SPSS Data Collection   SPSS an IBM Company  

First Post from Google Wave

First Post from Google Wave

As Promised this is to be my first post from Google Wave. I have to say that my experience has not been perfect already. Firstly I could not get the Posterous Robot to work properly in Chrome. When I tried to login it would not put the password in the password box, instead it wrote it in full view in just above. So I switched to Firefox and that had no problem.

Secondly when I start a new wave there is a comment from the Posterous Robot. It says Edit the root wavelets Title and body. Well the body was obvious because it was immediately above. But where is the title box. It took a moment to be brave and enter the title into the first line of the body, but, hey presto it worked.

Nice points include a really easy way to add tags (no longer do I have to remember how to put the tag on the subject line.

Anyway my flight to Vegas is boarding so I have to go. More to come.

Filed under  //   Chrome   Firefox   Google Wave   IBM SPSS Data Collection   Posterous Robot  

Data Collection to OpenXML a simple class for populating custom parts (the code)

The previous post provided a technical specification for the DLL required to link IBM SPSS Data Collection to the Custom Part in a Word document so that the responses in the survey can be pushed into the Word document.  This post will show the code used to implement that specification.  The code has been broken down into a simplified form to help me learn and to make it easier to explain.  There will be shortcuts that you may want to adopt, you may want less functions preferring to write the code in a single method, however I prefer the small is beautiful approach.

Defining the class:
As OpenXML constructs are going to be manipulated it is necessary to define the right references within the project and to import them into the class itself.  As streams are going to be used to read and write xml files into the Word document System.IO has to be imported.  There is no constructor required as the key actions will be on the main method (UpdateCustomPart). Consequently the class definition is as follows:

Imports DocumentFormat.OpenXml.Packaging

Imports System.IO

Public Class Complaint

End Class

UpdateCustomPart (Public Method)
The first and main method is the UpdateCustomPart, this will receive three parameters and will, for the purpose of this example, output a Boolean indicating success or fail.   The process this method follows is:

Ø  Parse XML string into a new XML Document

Ø  Open the Word Document

Ø  Locate the Custom Part

Ø  Update the Custom Part

Within this process the XML String, the name and location of the Word document and the name of the Custom Part are passed as parameters so that the same class could be used to update any custom part within any Word document.  The code for this method is as follows:

Public Function UpdateCustomPart(ByVal theFileName As String, ByVal theCustomPart As String, ByVal theXML As String) As Boolean

Dim WordDoc As WordprocessingDocument
Dim CustomPart As DocumentFormat.OpenXml.Packaging.OpenXmlPart
Dim xmlUpdate As Xml.XmlDocument

    xmlUpdate = New Xml.XmlDocument

    Try
        xmlUpdate.LoadXml(theXML)
    Catch ex As Exception
        Return False
    End Try

    WordDoc = OpenWordDoc(theFileName)

    If Worddoc Is Not Nothing Then CustomPart = FindCustomPart(WordDoc, theCustomPart)

    If CustomPart Is Not Nothing Then

Return changeCustomPart(CustomPart, xmlUpdate)

   End If

Return True

End Function

In the above function the try … catch is used to capture any errors generated from loading the XML (caused by badly formed xml).  Each of the subsequent functions are detailed below, if any fail the subsequent actions are not performed.

OpenWordDoc (Private Function)
The purpose of this function is to open the Word document identified by the path/filename string and to return the opened document to the calling routine.  Rather than just attempt to open the file (only to have it fail because the file does not exist) the function first uses FindFile to locate the document.  Only if this returns details of the file will the system attempt to open it. 

Private Function OpenWordDoc(ByVal theFileName As String) As WordprocessingDocument

Dim WordDoc As WordprocessingDocument
Dim FindFile As System.IO.FileInfo

    FindFile = New System.IO.FileInfo(theFileName)

    If Not FindFile Is Nothing Then
        WordDoc = WordprocessingDocument.Open(theFileName, True)
        Return WordDoc
    Else
        Return Nothing
    End If

End Function

FindCustomPart (Private Function)
This function was actually quite difficult to work out, it maybe that it is not actually the best way of doing it.  The word document is a package that contains a significant number of XML sub-documents.  It is necessary to find the right XML document to update.  The word document was created from within Word and the Word 2007 Content Control Toolkit.  As a consequence I did not name the Custom Part it was named for me.  Its name is Item1.xml.  It is contained in the customXml folder.  This document can be located using the relationships definition for the Main document part.  If this relationship definition is examined it can be seen that there is a relationship described for the custom part.

<?xml version="1.0" encoding="utf-8"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/settings" Target="settings.xml" Id="rId3" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/theme" Target="theme/theme1.xml" Id="rId7" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml" Id="rId2" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/glossaryDocument" Target="glossary/document.xml" Id="rId6" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/fontTable" Target="fontTable.xml" Id="rId5" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/webSettings" Target="webSettings.xml" Id="rId4" />
                <Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/customXml" Target="../customXml/item1.xml" Id="R456ea5aec8a94caf" />
</Relationships>

The target attribute is the URI of the part.  Based on the fact that the name of the part is provided by the consumer of the DLL this URI can be constructed through some simple text manipulation (“customXml/” + thename + “.xml”).  There may be several custom parts within a document so the Find function iterates through all the custom parts matching the URI of the custom part with a calculated URI.  When it finds a match it returns the matching part.  If it does not find a matching URI the function returns nothing.

Private Function FindCustomPart(ByVal theDoc As WordprocessingDocument, ByVal theName As String) As DocumentFormat.OpenXml.Packaging.CustomXmlPart

Dim CustomPart As DocumentFormat.OpenXml.Packaging.CustomXmlPart
Dim MainParts As IEnumerable(Of CustomXmlPart)
Dim ID As String
Dim matchURI As Uri
Dim CalculatedLocation As String = "/customXml/" & theName & ".xml"

    MainParts = theDoc.MainDocumentPart.CustomXmlParts

    matchURI = New Uri(CalculatedLocation, UriKind.Relative)

    For Each CustomPart In MainParts
        ID = theDoc.MainDocumentPart.GetIdOfPart(CustomPart)

        If CustomPart.Uri = matchURI Then
            Return CustomPart
        End If

    Next

    Return Nothing

End Function

ChangeCustomPart (Private Function)
This function uses a stream to Create new content for the custom part using the XML document created from the XML string passed to the DLL.  It effectively ties all the pieces together.  The writing of the stream updates the file (effectively saves it) this means that there is no need to save the document itself.

Private Function changeCustomPart(ByVal thePart As DocumentFormat.OpenXml.Packaging.OpenXmlPart, ByVal theXML As Xml.XmlDocument) As Boolean

Dim XMLStream As Stream

    XMLStream = thePart.GetStream(FileMode.Create, FileAccess.ReadWrite)

    Using (XMLStream)
        theXML.Save(XMLStream)
    End Using

End Function

Filed under  //   IBM   IBM SPSS Data Collection   OpenXML   SPSS   VB.net  

Data Collection to OpenXML a simple class for populating custom parts (Technical Design)

So far in this project a survey (the survey) has been created that accepts details of a train complaint.  A Word document (the word doc) has been created that provides a printed copy of the complaint details and a custom part (the custom part) has been defined and added to the Word document to facilitate the transfer of information from IBM SPSS Data Collection to Microsoft Word.  In the last post (creating XML in Data Collection) logic was added to the end of the survey that creates an XML document containing all the responses to the survey in the same structure as the custom part in the Word document.  In this post VB will be used to create a simple class that will accept the XML document and re-build the custom part.  It will also include methods to convert the resulting document into a PDF ready for emailing to the respondent.

Technical Design

The purpose of this DLL is to handle the interaction  between Data Collection and an OpenXML document.  The class will receive the XML to be inserted into the document and the details of the custom part to be updated.  IT will then replace the content of the existing custom part with the new XML content.  It will not delete the custom part itself as this would result in the loss of the existing unique ID for the document. This would in turn cause all the document relationships and form field bindings to break.

The second part of the process is to create a PDF version of the document.  Word can now do this without the use of third party components.  The returned PDF can then be attached to an e-Mail for distribution to the survey respondent (complainant)

Update Part method
Inputs:

·         XML (containing the survey responses)

·         Document name (the name of the Word document to change)

·         Custom Part name (name of the custom part to be changed)

Outputs

·         Success/Fail Signal (indicating whether update succeeded)

Create PDF Method

Inputs: None

Outputs:

·         PDF version of document

Internal Functions

As well as methods exposed to other applications (in this case IBM SPSS Data Collection) this dll will require some internal functions.

OpenWordDoc (Opens the document defined by the Update Part Method)

Locate Custom Part (takes the open document and finds the custom part)

Adjust Custom Part (having found the custom part the XML needs to be fed in)

There is no need to save the document as the OpenXML SDK will automatically save when the custom part is changed.

Filed under  //   Custom Part   Data Collection   IBM   IBM SPSS Data Collection   OpenXML   SPSS   VB.net   XML  

Taking a step back: Data Collection creates XML to populate OpenXML Custom Part

 

Over the last week I have been attempting to complete the project so that I can ensure I document the last few steps correctly.  In the last post I created the Data Collection routing for generating the XML that needs to be pushed into the Custom Part.  Since then I managed to create a automaton for pushing the XML into the custom part, however, when testing it in a test chassis I found that Word was struggling to understand attributes.  The XML I created manually was fine but the XML generated through the script did not work.  Having determined this but not determined why I have decided to take a step back and change the way that Data Collection creates the XML so that Word will work.

What do I mean: “Word does not work”?  Well when I manually created the CustomPart and bound it to the Form Fields you could see the results on the page (binding form fields) as shown in the image below.  Once I had automated the creation of the XML and the pushing of that XML into Word the content of the custom part was not showing in the document (the form was blank). 

Image001

If I then went into Design mode to view the Form Field details and then switched back out of Design Mode I would get an error.  It took me quite some time to realise that this error was simply saying that the Form Fields are empty.  This seems a bad error (one that Microsoft needs to change).

Image002

Furthermore as the PASW name for Data Collection has been dropped by IBM, I decided to change my namespace from PASW to CaO2.  CaO2 is an old term from a previous life, it has nothing to do with SPSS or IBM but I like using it (for example I use it as a domain name for my e-Mail: kgray@cao2.net)

To explain the change, let’s examine the name node of the xml.  In the previous post the following XML would have been generated by Data Collection:

<cao2:name cao2:title=”Mr” cao2:fname=”Kevin” cao2:lname=”Gray” />

Word did not seem to like the namespace qualified attributes.  Dropping the namespace for each attribute did work. So I could change the XML to:

<cao2:name title=”Mr” fname=”Kevin” lname=”Gray” />

However I am not happy doing this I like the namespace to be qualified properly.  Consequently I have decided to only use textnodes and not use attributes.  This will get me through this exercise but is not a perfect solution.  As a consequence I will have to return to this in a later blog to ensure I can use attributes in the long run.  The new XML will, consequently look like this:

<cao2:name>
                <cao2:title>Mr</cao2:title>
                <cao2:fname>Kevin</cao2:fname>
                <cao2:lname>Gray</cao2:lname>

</cao2:name>

This format is much more verbose but it does work when used in Word.  In order to have this format of output the routing in Data Collection needs to change.  The creation of an attribute was, previously,  executed through the following code (which creates the name node and then adds a title attribute):

set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:name", "http://www.kevinagray.co.uk/railcomplaint")
xmlPersonal.appendChild(xmlNode)     

if Title.Response.Value <> null then
                xmlNode.setAttribute("cao2:title", format(Title.Response,"b"))
else
                xmlNode.setAttribute("cao2:title", "")
end if

The new code will still create the name node but then will create another node for title.  The title node is always created whether or not there is a title, however the textnode within the new node is only created if there is response on the form, otherwise the title node will be null.

set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:name", "http://www.kevinagray.co.uk/railcomplaint")
xmlPersonal.appendChild(xmlNode)

set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:title", "http://www.kevinagray.co.uk/railcomplaint")
xmlNode.appendChild(xmlParent)

if Title.Response.Value <> null then
                set xmlText =  xmlPart.createTextNode(format(Title.Response,"b"))
                xmlParent.appendChild(xmlText)
end if

The full code in the routing is now:

Routing(Web)

Dim xmlPart, xmlNode, xmlRoot, xmlPersonal, xmlIncident, xmlParent, xmlText
Const Node_Element = 1

                IOM.LayoutTemplate = "railcomplaint"
                IOM.MustAnswer = false            

                Personal.Ask()

                Incident.Ask()

                set xmlPart = CreateObject("Microsoft.XMLDOM")
                set xmlRoot = xmlPart.createNode(Node_Element, "cao2:root", "http://www.kevinagray.co.uk/railcomplaint")
                xmlPart.appendChild(xmlRoot)

                set xmlPersonal = xmlPart.CreateNode(Node_Element, "cao2:personal", "http://www.kevinagray.co.uk/railcomplaint")
                xmlRoot.appendChild(xmlPersonal)

                set xmlIncident = xmlPart.CreateNode(Node_Element, "cao2:incident", "http://www.kevinagray.co.uk/railcomplaint")
                xmlRoot.appendChild(xmlIncident)

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:when", "http://www.kevinagray.co.uk/railcomplaint")
                xmlIncident.appendChild(xmlParent)

                if IncidentDate.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentDate.Response.Value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:destination", "http://www.kevinagray.co.uk/railcomplaint")
                xmlIncident.appendChild(xmlParent)

                if IncidentDestination.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentDestination.Response.Value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:name", "http://www.kevinagray.co.uk/railcomplaint")
                xmlPersonal.appendChild(xmlNode)     

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:title", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if Title.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(format(Title.Response,"b"))
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:fname", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if fname.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(fname.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:lname", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if lname.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(lname.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:address", "http://www.kevinagray.co.uk/railcomplaint")
                xmlPersonal.appendChild(xmlNode)

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:line1", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if address1.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(address1.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:line2", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if address2.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(address2.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:town", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if Town.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(town.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:postcode", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if PostCode.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(postcode.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:contact", "http://www.kevinagray.co.uk/railcomplaint")
                xmlPersonal.appendChild(xmlNode)

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:work", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if Phone.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(phone.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:home", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if Phone2.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(Phone2.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:mobile", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if mobile.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(mobile.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:email", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if email.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(email.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:departure", "http://www.kevinagray.co.uk/railcomplaint")
                xmlIncident.appendChild(xmlNode)

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:from", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if IncidentDepart.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentDepart.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:expected", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if IncidentExpected.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentExpected.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:actual", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if IncidentActual.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentActual.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:impact", "http://www.kevinagray.co.uk/railcomplaint")
                xmlIncident.appendChild(xmlNode)

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:severity", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if IncidentSeverity.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentSeverity.Response. value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:outcome", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if ExpectedOutcome.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(ExpectedOutcome.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlParent = xmlPart.CreateNode(Node_Element, "cao2:cost", "http://www.kevinagray.co.uk/railcomplaint")
                xmlNode.appendChild(xmlParent)

                if CostIncurred.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(CostIncurred.Response.value)
                                xmlParent.appendChild(xmlText)
                end if

                set xmlNode = xmlPart.CreateNode(Node_Element, "cao2:description", "http://www.kevinagray.co.uk/railcomplaint")
                xmlIncident.appendChild(xmlNode)

                if IncidentDescription.Response.Value <> null then
                                set xmlText =  xmlPart.createTextNode(IncidentDescription.Response.Value)
                                xmlParent.appendChild(xmlText)
                end if

                debug.Log(xmlPart.xml)

End Routing

Filed under  //   CustomPart   IBM   IBM SPSS Data Collection   OpenXML   Routing   SPSS