Search should be clairvoyant: like a magical librarian who somehow correctly guesses what it was you were looking for; offering it up within a fraction of a second, along with a wealth of additional filtering options and navigational possibilities.

Without wishing to destroy the magic, here are my seven steps to improving findability:

1. Define what relevancy means in this context

Relevancy is a difficult thing to pin down. A set of search results is more or less relevant on the basis of how well the information retrieved meets the need of the user. Arriving at a definition of relevancy therefore means doing some fairly detailed analysis of your users and content. Some BIG questions need answering.
Is recall or precision more important? How do you go about catering to the competing needs of different user groups?

Clearly, discussions need to be had and decisions made. During this process it will be worth considering a range of scenarios where you might like to boost certain results over others:

• Field weighting
  e.g. results within titles are more relevant
• Recency of data
  e.g. results from recent data are more relevant
• Search phrase density
  e.g. results which contain the most number of uses of the search phrase are the most relevant
• Search phrase term proximity
  e.g. results where multiple terms are nearer to each other are more relevant
• Records which have been bookmarked, cited or linked to
  e.g. results which have already proved themselves to be useful to other users are more relevant

2. Provide lots of options, not just lots of results

Findability is not just about returning relevant results. A good search implementation will also provide lots of additional further options to the user, which they can use to hone in on exactly what they were searching for. Examples of this include providing:

• Meaningful facets through which the results can be filtered
• ‘Did you mean…?’ option – for alternative spellings
• ‘Users who searched for x also searched y’ option – for related searches
• Clustering of search results, e.g. by topic
• Sort options
• Hit-highlighting – to highlight the phrase terms in context

3. Enriching the data

Data not only has to be marked up consistently and correctly, it can often benefit from some enhancement before it goes online. In practice, this can mean additional classification processing or entity extraction through text mining. The goal is to ensure the content itself is rich enough to support the sort of advanced searching and filtering that we want to build within the site.

4. Measure relevancy

It’s worth setting up some relevancy metrics to monitor how search is performing over time. A good method is Mean Reciprocal Rank. To implement this you track click-throughs to search results, giving each click-through to a first result a score of 1, each click-through to a second result a score of 1/2, each click-through to a third result a score of 1/3, and so on. Adding all these together will give you the ability to track an overall relevancy score, with a higher score meaning that top links are performing better.

It is also a good idea to monitor searches that return zero results. A monthly list should be reviewed in case there are some sensible search queries in there which will have resulted in user frustration.

Regular reviews of search analytics are a vital part of ensuring that search is still performing well as the site and its content change over time.

5. Improving the user’s query

Normalising the user’s search phrase (and indeed the search index data) can help to improve findability. The following are all ways in which you do this:

• Converting all letters to lower or upper case
• Removing punctuation, accent marks or diacritics
• Expanding abbreviations
• Removing stopwords or “too common” words

Recall might also be improved upon in certain scenarios by converting the users query into a fuzzy query (to return results for close matches to the search terms in order of how well they match). It may also be worth expanding the user’s search to include synonyms using a thesaurus (to return results where matches have been found for the same or similar concept).

In these ways it is possible to enhance the input query before it has even been sent to the search engine.

6. Tuning the site for third party findability

Lots of users (yes, perhaps even MOST) start their search using a third party search engine. It is therefore essential that the site is Search Engine Optimised, meaning lots of quality in-bound links, the use of semantic markup, micro formats and much, much more.

It may also be worth creating an Open Search API – so that third party use of the site’s search facility is possible.

7. Finding also means re-finding

There is a very good chance that users will want to re-use the entries that satisfy their information need. Consequently, improving fundability should also mean making it as easy as possible for users to re-find what they found before. Helping users in this way can be done with features such as:

• Bookmarks
• Saved searches
• Direct exporting to citation software

So there you have it: seven steps to findability. It is a BIG topic and I’m certain to have missed out important considerations. Please do feel free to publicly rub my nose in some of them by responding below!

Originally held as a means for informing customers about the technical development of Semantico’s products, the event has since grown to become a forum for knowledge sharing and debate about a wide range of issues around online publishing.

S3UG 2010 focused primarily on project management with a secondary theme on the growing importance of mobile devices and the future of publishing.

If you were unable to attend but would like to view the slides from S3UG or were present but would like to do it all again, we have provided a link for you to view the slides for each of the talks below.

From all the Semantico team, we look forward to seeing you at next year’s S3UG Event.

Richard Padley – Keynote and company update

Gareth Bish: Business Models

Liam Sheerin – Agile Development

Andrew Grimes: Waterfall vs Agile – Project Methodologies

Richard Padley – Choices in mobile content delivery

As quality assurance assistant and junior developer for Semantico I spend a lot of time developing and implementing QA tests. Testing an e-publishing website can be time consuming.

Even a simple test of search functionality has several steps; go to the designated URL, log in, search, verify the search results, check hit highlighting, start an advanced search, verify those results, check hit highlighting again, start another search with a different term… well, you get the idea. It’s repetitive. And there are many aspects to test, not just the search facilities.

After a while you might find yourself thinking that there should be a more efficient way to test your website. You can hire someone to do all your QA tests for you of course. But will they test your website in an efficient way? Will they follow your test cases to the letter; the tests you have spent hours designing, editing and documenting? If only there were an automated tool to do all this – and one which did not require a degree in Computer Science to operate.

Enter Selenium

At Semantico we do extensive testing. To get the most out of budgets and man hours we now use Selenium IDE for the repetitive tasks.

The official website describes Selenium as “a suite of tools to automate web app testing across many platforms”. You can think of Selenium as a little piece of software that does not mind doing your testing for you. It will test your website exactly the way you want it to be tested, as many times as you wish. You just need to tell it what to test and how. Selenium is a life saving tool and can cut out a lot of time, effort and money. It’s got no price tag (it’s free!) and is fairly easy to learn. It comes in different flavours but we’ll only talk about the Firefox add-on version here because that’s the version we know best.

Installation and tutorials

Download Selenium IDE and let’s get started. Install it on Firefox, restart your browser and you are ready. Selenium now lives in Firefox’s Tools menu and has a minimal interface. If you have ever recorded macros in an office suite of applications (like OpenOffice or Microsoft Office for example) then using Selenium will be a breeze. If you haven’t, it will only take you a couple of hours to learn how to use it efficiently.

I have found two very useful tutorials so far, one from the official Selenium Wiki and one from the Software Testing Geek website . Both tutorials require Flash to run. These are only a couple of tutorials from the many available on the web. A quick search on your favourite search engine should help you find even more.

After that, you are ready to start recording your test cases and then share them with your colleagues if you want, so that they can do some QA work when you are not available.

Using Selenium

Using Selenium is fairly straightforward. You start Firefox, fire up Selenium IDE and it is already in record mode. As soon as you start clicking on links, typing in text boxes and pressing buttons, Selenium will record these actions. The simplest thing to do when you first run Selenium is use it to run a search on a website. Once you have initiated the search and the search results are displayed on the page, press the ‘Record’ button to deactivate it. Afterwards, you can run the newly-created test case and Selenium will play it back on demand, repeating your actions in exactly the same order as you have taken them.

Translating your test cases

Selenium stores your commands in its own programming language. This language, called Selenese, uses English words and can be easily read by non-programmers. Once you’ve created your test cases you can translate them into Selenese and they are ready to be used whenever you need them. This is where you can realise the true power of this tool. You simply run Selenium and it will handle the rest. You can either run a test suite or individual test cases. If there are any errors, Selenium will report them, along with an explanation, in a log file which is directly accessible from the programme’s interface.

Selenium’s interface will not win any beauty contests. But it’s incredibly easy to use and, most importantly of all, is constantly giving you helpful feedback on what is happening. Brains over beauty. Simple.

Most automated tools we have tried in the past have had one common problem: they will not let you change the base URL. The base URL is essentially the URL of your website. But as all experienced QA testers know, the URL can change. One day you are testing something on a test environment, later that day you are testing something on the live site. Implementing your test cases multiple times for each URL can be time consuming and inefficient. With Selenium IDE you can choose not to store a base URL, which makes it very versatile and easy to use on different environments.

Sharing your test cases

Once you are happy with the implementation of your test cases and you have separated them in test suites, you are able to share them with your colleagues and across different departments. The files are small enough to email, or you can save them to a shared drive.

It is not a panacea

No matter how powerful a testing tool is, it will never entirely substitute the work done by a QA team. Selenium IDE is great for testing specific functionalities of your website and can deal with most of your test cases – but it is not ideal for checking presentation-related issues. You cannot use it, for instance, to test whether your website looks the same on a wide-screen monitor, or to check the text on a particular menu for correct alignment.

Also, it is a Firefox add-on. Unless you install the server-based version of Selenium, Selenium RC on a server, you can only check your website on Firefox. There are CSS issues between browsers that you also need to examine before you give the developers the green light to go live.

Lastly, if there is one area that needs improving it’s that Selenium is a bit weak on dynamic content. If, for example, one of your pages involves different sections that are presented in tabs with JavaScript, Selenium will not always work.

Recap

Using Selenium to improve your tests on your website has many benefits: you save time, money and resources. Your tests are done in a fraction of the time it used to take you to fully test a website. Your test cases become more flexible, more dynamic and of course, you never forget to test anything.

I’ve found Selenium to be an exceptionally useful tool when it comes to regression tests and rapid functional tests. It’s not here to replace testing done by humans, but it does a great job of assisting and complementing human testing.

More on this excellent seminar can be found at The International Association of Scientific Technical and Medical Publishers website.

Video by River Valley TV.

Poor communication can kill online publishing projects. Here are solutions to five common communication pitfalls which, if left unchecked, can lead to big trouble.

1. Needless information

There are just too many ways in which needless information can distract us. Email, by its very nature, is a tool used to push information to one or multiple recipients. It therefore relies heavily on the sender to get the right information to the right people. Thus, it creates the frustration, familiar to us all, of being copied in on messages of no direct relevance.

The solution

A centrally stored, single place for all project communications, document sharing and collaboration – enabling team members to pull information from it and when it is needed. Basecamp, currently being trialled by Semantico, is a good example of a tool which has arisen out of the need to improve project communications. It includes the ability to subscribe or unsubscribe to any particular discussion.

2. Information silos

The opposite danger is that of not receiving or being able to access the information you need. Ever needed to access important emails in another user’s account?

The solution

Google wave looks towards a brave new world of online collaboration, arising out of the desire to model a communications tool on the capabilities of our current technologies rather than on historical non-electronic forms of communication. On a ‘wave’, documents don’t go back and forth – they evolve in real time! Read more about Google wave on the official Google blog.

3. Interruptions

Most office workers – if they are honest – would admit to scanning their emails the second they arrive in their inbox. We are surprisingly accepting of the fact that this means constant interruptions! Yet it is well known that interruptions impede productivity. Instant messaging, phone calls, impromptu desk visits all pose similar threats.

The solution

How about setting your email to check the server every hour, instead of every minute?

With regards to the impromptu desk visits, growling is not the only solution. Why not treat yourself a set of traffic lights to let your colleagues know when you’re uninterruptible?

4. Mistimed details

Plans and specifications tend to change as you progress through a project. It is therefore important wherever possible to define the details of each project deliverable just before it is to be delivered. The alternative is to risk wasting significant time on thinking about details which will change by the time they are relevant.

The solution

Only produce detailed plans or specifications for work that is going to happen in the coming few weeks.

5. Unfocused meetings

Meetings can be highly toxic!
How often have you sat in a meeting without being clear on what the meeting was trying to achieve? Even when agendas are circulated there is still a great likelihood that time will be wasted due to lack of focus or the wrong people being invited. Meetings are usually very expensive too; a 30 minute meeting with 7 attendees costs a half day!

The solution

1. Prepare well for meetings.
2. Keep them as short as possible.
3. Only invite the people who absolutely have to be there.

Conclusion

It is always sensible to agree appropriate checkpoints and communication channels at the outset of a project. Indeed, this is perhaps the all important first step to avoiding the dangers of poor project communication.

The following is a small collection of tips for making sure your XML data is in tip-top condition.

1. Use Semantic Markup

Specify “what” not “how” in your data.  If an item is bold in print, then consider using a tag which describes “why” the element is bold.  Perhaps it is a title in a reference.  Using <title> rather than <i> describes the intent rather than the output.  This way, data can be re-used in many different contexts.  What happens later when titles need to be shown in bold?  The names of all tags and attributes should reflect the information they contain rather than how they’ll be presented to an end user.

2. Mark up each unit of information

Insufficient markup is often one of the biggest problems with online content.  If there is no markup, there is no way to detect and process a piece of data.  You should tag each unit of information, and avoid implicit structure.

3. Validate your XML

Use a DTD or XML Schema to validate your markup.  This is vital for good QA.

4. Use opaque IDs

An ID should uniquely identify a piece of content, and be persistent.  Any other information gleaned from the ID might change in future, causing the ID to change.

5. Use elements for data, attributes for metadata

This isn’t a strict rule, but whenever in doubt, use an element.  Attributes should not contain structured data – structured information is easier to process as elements.  Some good candidates for attributes are: IDs, URLs, revision dates, types.

6. Separate print and online content

Things like page numbers are irrevelant for online use.  Consider using attributes to indicate whether content is meant for print or online use.

7. Don’t reinvent the wheel

Use TEI, NLM, Docbook, Dublin Core, etc. rather than making up your own standard.

Good, structured data is the key to a functionally rich user experience online.  If you want to find out more, I can heartily recommend these titles:

“Effective XML: 50 Specific Ways to Improve Your XML”, by Elliotte Rusty Harold
“XML Data Management: Native XML and XML-Enabled Database Systems”, by Akmal B. Chaudhri; Awais Rashid; Roberto Zicari

Enter stage-right: The Business Case.

The business case in PRINCE2 terms is a document that defines the justification for undertaking a project. It describes the benefits a project is intended to deliver against the costs and risks which will need to be taken along the way.

A strong business case is one where the envisioned benefits outweigh the costs and risks.

The business case should be evaluated at intervals throughout the project in order to test that it is still on a worthwhile course. As such, it is vital that it should employ measurable criteria against which the ongoing viability of the project can be assessed.

It is equally important that the business case should only focus on a core list of benefits that the project must deliver. It is only through resisting the urge to include all suggestions from stakeholders – and instead narrowing the list down to only those with genuine business benefits – that the project will achieve a focused target.

At Semantico, we use a range of workshop exercises to assist in preparing a business case in a focused and manageable way. One of my favorite exercises is described below. Do give it a try and let us know how you get on!

Business case prioritisation exercise

1. Gather together all project stakeholders for a one hour meeting. Where possible include representatives from user groups, suppliers and from the business. Tell them to bring their thinking caps and their wish lists for the project. Offer biscuits.

2. Brainstorm a list of potential benefits that the project could deliver, writing each benefit on a separate post-it-note.

3. Ask a group of the stakeholders (perhaps those representing the business and/or the users) to sort the full list by importance, sticking each post-it-note to a whiteboard with the most important at the top of the board and the least important at the bottom.

4. Give each post-it-note an ‘importance score’ in the bottom left hand corner of each (e.g. if there are ten post-its, give the top most a score of 10, the next one 9 and so on). The key is that no post-it should receive the same score.

5. Now ask a group of the stakeholders (perhaps those representing the suppliers and/or the business this time) to sort the same post-it-notes in order of feasibility – putting the most achieveable of the benefits to the top of the white board and the least to the bottom.

6. Write their ‘feasibility score’ in the bottom right hand corner of each (as in step 4 above).

7. Now chart the post-it-notes on a graph with ‘Importance’ and ‘Feasibility’ as the x and y axis as shown in the example below.

8. Draw attention to the 3 or 4 post-its that occupy the top-right corner of your graph – and confidently proclaim: “here’s what our project will focus on!”

Click for larger image

Of course this exercise will only give an indication of what the project should concern itself with. It will need to be validated with costings and risk analysis. It will also need to be checked against the corporate strategy/vision…

…multiple stakeholders, competing priorities and limited budgets will still come into play. However, with this exercise, you should be a step closer to understanding the key opportunities for your project and therefore be well on the road to developing a focused business case.

But what does a Project Manager really do?

Here are my top seven attributes of effective project management: the things we PMs ought to be doing to keep everything on track …

1. Building confidence within the whole project team.

Without confidence, a project can very quickly lose its way. The Project Manager should therefore regularly check the health of ‘project confidence’ by reviewing the following questions:

• Is there confidence that the project is on track?
• Are roles and responsibilities clearly defined and understood?
• Is everyone happy?

In my experience, regular demonstrations of progress are the best way to build trust. With regular review meetings, there is the opportunity to review the project against its business case and to provide assurance that the development is on track. It is the Project Manager’s job to ensure that these reviews take place at the right intervals, that the right people are there, and that the points raised during the review are followed through.

2. Managing change

As the saying goes: the only constant is change. Requirements that were specified, costed and signed off at the outset of a project will often need to adjust to fit the changing business context. Project teams usually uncover unexpected additional complexities, or indeed opportunities, along the way. Either way, it is sensible to plan for change.

Part of the Project Manager’s role is to coordinate change with as little pain as possible. This can involve de-scoping lower priority deliverables in exchange for incorporating new ideas within the existing budget. More often it means arranging for an additional budget with a corresponding adjustment to the project plan. Clear communication and a documented agreement are essential for ensuring that change is managed effectively.

3. Planning

One could say that a Project Manager is like the conductor of an orchestra. Without necessarily being proficient on any particular instrument, the conductor coordinates the intricate timings and dependencies within a piece of music. He is responsible for each detail and also for the overall quality of the final piece.

In reality, a Project Manager is perhaps more like a bus conductor – handing out tickets and coordinating at what points people get off and on. However, it is crucially important to plan the route the bus is going to take. As such, a Project Manager needs to agree ’smart targets’ with the production team and to track progress carefully. Without this the passengers would have no idea as to where they were going or indeed when they were going to arrive.

4. Communication

As everyone knows, enterprises of any sort in our knowledge economy rely heavily on the flow of knowledge. Project Managers must ask themselves:

• Does each team member know all that is necessary to do their job?
• Are project requirements well documented?
• Is the business case for the project well defined and understood?
• Are the key stakeholders being given the appropriate opportunities to review progress?
• Is there an audit trail of project decisions, issues, risks, changes?

It is easy to see that without an effective communication strategy in place a project is likely to suffer. Here at Semantico we have set up project processes and template documents to help to make our communication consistent and improvable. We use web-based tools such as our issue-tracking system Jira which ensures that all issues are managed through a scheduled workflow and can be audited.

Of the various types of project meetings I coordinate, my favorite is the morning Scrum with each project team. These meetings are ultra-efficient – no more than 15 mins – and the agenda is always the same:

• What progress was made yesterday?
• What is the plan for today?
• Are there any impediments stopping you from proceeding?

These meetings are a great way to monitor progress and identify any ‘blockers’ which need addressing.

5. Risk analysis

The Project Manager is ultimately responsible for ensuring the health of the project. Regular risk analysis is an essential part of this. The process starts with identifying possible risks so that they can be logged, evaluated (scored according to impact and probability) and assigned to ‘risk owners’. Naturally, logging them is not enough, it is also important to make a decision about how best to manage each risk. This will sometimes take the form of deciding on a contingency plan. However, it is preferable, where possible, to mitigate the risks in the short term by taking actions to reduce the probability or impact.

6. Problem solving

Even with the best planning, communication and risk analysis, you can still encounter tricky times in a project. Here the Project Manager must do their best to identify win-win outcomes. I believe that the best way to do this is to remember to focus on ‘interests’ as opposed to particular ‘positions’.

There is a famous story about two sisters arguing over an orange. They eventually settle on dividing it in two. They later discover that one sister had wanted the peel and the other, the fruit. Had they communicated more effectively what their interests were, both could have had 100% of what they wanted. However, because they focused on winning rather than win-win, they each only got 50%. The moral is to remember to delve into the real interests of each party, and only then to consider solutions that deliver to all. Likewise, a Project Manager must consult with all key stakeholders before recommending a solution that works well for everybody.

7. Quality control

Quality is defined in ISO 9000:2000 as ‘The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs’.

It is difficult to deliver ‘implied needs’ with precision – but it is important to try! The unspoken, undocumented requirements can be very important. It is likewise essential to manage even the smallest details that crop up. Project Managers know from bitter experience exactly where the devil is – and by paying close attention to the detail they help to ensure that small problems don’t become diabolical ones!

All Semantico projects, big or small, include a traditional ‘Quality Assurance Cycle’ where our software gets tested thoroughly at the end of each iteration. Where possible, we also like to arrange mid-project web demos to provide our clients with the opportunity to review our progress. As already mentioned, this is the single most effective way to build project confidence. Crucially, it also provides an opportunity to review the software’s features and functions before it becomes too expensive to change direction if the need arises.

—-

So there you go, seven important attributes of what a Project Manager’s job should be – if they’re doing the job properly!

If we take the example further, and look at how such an estimate would change over the duration of the journey, we can see that the only time the estimate can be guaranteed as 100% accurate is once the journey is complete – when the estimate is an actual.

The situation is no different in software development and the accuracy of the estimation of software tasks also increases over time. This change in accuracy over time often referred to as the ‘Cone of Uncertainty’.

Cone of Uncertainty

The Cone of Uncertainty shows how an estimate will start with a high degree of uncertainty – either over-estimated or under-estimated – which then improves over time as more is known about the task(s). The diagram shows several common stages within a project and shows how accuracy will be increased as the project progresses.

If, therefore, it is not possible to guarantee the accuracy of an estimate, then another approach is required.

It is reasonable to assume that an estimate for completion of a single task could be bounded by a range of best and worst possible cases. If such a range is further defined as having a most likely completion point, an estimation profile for the task can be demonstrated.

Program Evaluation and Review Technique

The Program Evaluation and Review Technique (PERT) for software estimations requires developers to provide a range of estimates for a task. PERT requests a best (B), likely (L) and worst (W) case for each task with which one can plot a probability curve for the task. The probability curve plots the probability of completion against the completion time.

Task Profile

From the ranged estimate an expected case (E) can be calculated that has a probability of 50%. As can be seen in the graph there are a number of ways of calculating the expected case, based on a weighted average of the best, likely and worst estimates for the task. The weights selected for the expected case should ideally be based on historical data from previous actuals and estimates. In the absence of such historic data, the E=(B+4L+W)/6 equation is the prudent choice.

Another benefit of the ranged estimation approach is that it highlights tasks that have a high risk. These can be seen as tasks with the largest range. Where such tasks are identified action can be taken to further understand their nature and therefore provide a more accurate estimation. This can be done through:

• research and analysis
• prototyping
• additional requirements workshops

Combining multiple tasks

Multiple Task Profiles

As multiple tasks are estimated a number of task probability profiles will be created. The tasks can be combined to produce an overall project profile that describes the full project.

Combining Multiple Tasks

Understanding the project profile is useful but the practicalities of project scheduling and costing dictate that a single figure is produced. As has been demonstrated, completely accurate single estimates are not possible. However, with the application of some basic statistical methods, it is possible to produce estimates at known probability or confidence levels.

The importance of selecting the right confidence level can be seen in the chart below which shows how the confidence level and break-even point for a project define whether the project is likely to be profitable or loss-making.

Project Break-even

Understand that estimates have two dimensions

Effective delivery of projects on time and on budget requires the application of a clear risk management approach throughout the whole project life cycle. The key learning from this approach is that any project estimate is meaningless without an accompanying confidence level. Applying this principle allows the management of both dimensions in project estimation, which in turn improves the identification and mitigation of risks throughout the project life cycle.