Analyzing the Results from the WritersUA 2014 User Assistance Tool Survey
This past Saturday WritersUA came out with its 2014 User Assistance Tool Survey, based on a wide-ranging poll of those working in the technical writing field. If you look at the WritersUA page it simply displays the raw poll results without any analysis. It would be more useful if the data that came out of this was analyzed in some way, but it is not. DITAWriter to the rescue! 😉
There’s more than one way to examine the data from this survey, but what the data shows is that the technical writing tool market has become highly fragmented, and while there are still some tools that are used widely, they are not as popular as they are popularly perceived.
“Yes, I Have This Tool”
The methodology for the WritersUA survey is muddled: users are asked to rate a tool they are using on a five-point scale, where “1” = least used, while “5” = most used. This is known as a Likert Scale and has been used by statisticians since the 1930s.
What a lot of people do when looking at the results is simply to sum up the numbers, which muddles the results. This is a common approach to the problem (look for an example of this in the comments to the original article), but it is not the best way to analyze the numbers. The method does not tell you which tool is the most popular, it just says “Yes, I have this tool”, and inflates the implied number of users of that tool, even when someone is just saying that they have it but don’t use it much.
So when the results are summed up in this manner, here’s what you get (Top 10 only):
Wow, it looks like Adobe has a lock on the most popular tools, doesn’t it? 6 out of the top 10 certainly isn’t bad. But what is Visio doing at #3, and is PhotoShop really being used more than FrameMaker? And given its popularity, aren’t you surprised that MadCap Flare rates lower than its predecessor RoboHelp? I mean, you can’t even produce a publication using three out of the top four tools, so clearly the results are skewed when you process the data this way.
Summing things up in this manner doesn’t tell the real story. Let’s look at the results in the way they are meant to be processed, in terms of tool popularity.
“How Much I Use a Tool”
There are several ways you can process the raw data from the WritersUA survey, but the one that makes the most intuitive sense is to subtract the negative scores from the positive ones as use the result to rank the popularity of a tool.
The methodology is simple: sum the positive scores (in this case, I am summing all of the values ranked “5”, “4” and “3”*), sum the negative scores (“2” and “1”) and then subtract the total of the negative score from the positive one. What you get presents a more believable picture of the state of technical writing tools in terms of popularity/most used:
(If you are confused by the negative numbers that appear, don’t think of those tools as being “disliked” as that’s definitely not the case; just think of them as a way to rank the items. Basically it says that only a few people have these programs and that they don’t use them a lot).
When processed this way the top two tools are still the same (Adobe Acrobat and TechSmith’s SnagIt), Madcap Flare jumps up to #3, and FrameMaker comes in at #4 just above PhotoShop at #5. I think this method of processing the raw numbers from the WritersUA survey presents a more accurate view of tool popularity in the technical writing world.
Looking at the Ranking from a DITA Perspective
There are a few interesting things that emerge from this ranking of tools: the first is that the DITA Open Toolkit comes in at #13, fairly high up in the list. Next is the presence of two popular XML editors: oXygen ranks at #19, and XMetaL slightly lower in the #22 slot. Both have a long way to go to achieve dominance in the technical writing market—compare to the more traditional technical writing tools like FrameMaker at #4 and Author-It at #14—but it does appear that some form of XML (DITA, S1000D, DocBook) is increasingly becoming the lingua franca of technical documentation.
Another interesting result is the presence of Content Management Systems appearing in the list: SDL’s LiveContent Architect (formerly SDL Trisoft) at #38, and Vasont at #48. It’s likely that some portion of the respondents who use Arbortext (ranked at #29) are not referring to its editor but to its CMS. SDL’s product is a DITA-based CCMS, and while Vasont and Arbortext can work with other XML schemas, it is highly likely that a significant portion of the users who responded to this survey are using DITA as well. (It is also worth noting that while Ixiasoft’s DITA CMS didn’t make the list, it did get a few “write in” votes by email, as noted at the bottom of the survey).
Having said all of that I think the main takeaway from this survey is how fractured/diverse the technical writing tool market is now. This same point has been expertly conveyed in an article by Tim Johnson, so all I want to add to that is the large number of new and up-and-coming tools that appear in this list also indicates that there is a robust technical writing tools market, which can only be good for technical writers. Even the top 10 has been penetrated by the very popular and robust text editor Notepad++, and at #25 the open source graphics editor the GIMP makes an appearance. Below that there are a large number of niche tools created by smaller firms. The technical writing tool marketplace is definitely in flux, and I think that all of the “churn” happening at the lower end of the rankings bodes well: the market is big enough to support fresh development, and let’s face it, this is where the new tools we’ll be using tomorrow are likely to come from.
* You could argue that the mid-range value is not positive but neutral. But not wanting to toss out the results for this option completely and also to keep things simple, I added those numbers to the positive group.
P.S. I learned when putting this article together that Omni Group’s founder Jeremy H. Griffith, who put together such popular products as MIF2Go and more recently DITA2Go, passed away this past May. My condolences to his family and friends, and while I never met the man himself, I always found his software to be top-notch. He will be missed. According to the Omni Systems website, following his wishes the source code for these products will be made freely available on SourceForge for further development by others.