This will be good news for those librarians who are concerned that the figures publishers provide after the first year of compliance might be inaccurate. An annual audit should ensure that no software errors suddenly appear.

However this will be bad news for those wanting to see costs kept down in a sector which is already hugely challenged by budget cuts. Publishers must bear the extra costs of the annual audit, and it’s hard to imagine that these will not be passed on to libraries in some way.

The audit must be carried out by a professional organisation recognised by the COUNTER project. This includes ABC Electronic in the UK, who offer a fixed price for this service. I contacted ABC Electronic to find out their prices but I did not recieve a reply.

It’s hard to imagine a worse time for this change. Although the financial impact will be fairly small, costs and budgets are being squeezed from all sides at the moment. Lets hope there are no more changes like this in the pipeline.

Update: After I published this ABC Electronic saw this blog post and called me to apologise for not replying to my email enquiry, and I subsequently received pricing details for the COUNTER compliance auditing service they provide.

Some publishers choose to adapt an existing off-the-shelf journals platform to meet their needs for consolidation. However there can be technical problems inherent in this approach.

In a recent post to the CrossTech blog, publishing guru Geoffrey Bilder analysed the issues facing CrossRef members wishing to use the DOI system for non-journal content. Geoff’s analysis holds good for more than just the CrossRef DOI system, so I’ve taken it as a starting point below.

Reference publications and databases introduce fundamental challenges to any existing system designed around the journal article content model. These challenges fall into two areas:

• Structure. Reference works and databases can have complex nested substructures and there is a need for granular identification of these content substructures along with a mechanism for recording the relationship between them (e.g. “part-of” relationships between sub-section, section and chapter divisions, as well as “previous-next” navigational relationships between entries).
• Versioning. Unlike most journals, many reference books and databases change over time. To properly support archiving and perpetual access business models, there is a need to identify and maintain previous versions of reference content.

Both of these areas require support from the core architecture of any publishing system; without such support publishers have to resort to ugly work-arounds such as coercing all content into journal-article-shaped chunks. Such work-arounds fundamentally compromise the end-user experience and ultimately risk devaluing the publisher brand.

Furthermore the technology stack commonly used to build a typical journals system does not necessarily provide a good base to build support for the structural and versioning requirements outlined above. This is because the mixture of technologies used tends to match closely the typical structure of a journal article.

In an article the metadata and content are two separable parts which map nicely to a relational metadata store (typically SQL or RDF) plus a full text retrieval engine (Lucene/Solr, Autonomy, etc). Although the full text may contain semantic annotations such as chemical or gene markup, the homogenous nature of the content means that a single metadata schema can be devised which will fit the whole collection well.

However, when publishers want to add non journal content to the system this simple separation of content and metadata will no longer suffice. Reference and book content is much more demanding in terms of hierarchical structure and navigation. Many hierarchical structures in reference books cannot easily (if at all) be modelled using conventional relational metadata; these structures fit much more naturally and easily into native XML. Trying to create a single relational metadata schema capable of modelling all hierarchies across the full variety of reference and book content that publishers produce is an impossible task.

The structural problems of adding books and reference content to systems designed to separate full text and metadata can be broken down into two distinct areas:

1. Fragile schema problem. The problem of updating conventional database schemas as requirements (and the real world) change is often called the fragile schema problem. Because the design of the database must be fixed at the outset, any changes to the system, to accommodate extra metadata, new content types or changes to business process become costly and risk-laden. This is because assumptions about the structure or schema of a database tend to be hard-wired into the structure of all the software written to talk to the database.
2. Divorce of metadata from full text. Systems which use RDF or SQL databases for metadata suffer from a fundamental structural weakness as it is normally impossible to issue queries which examine both the full text and metadata within a single request. RDF suffers particularly in this respect as it has no support for XML mixed content; full text search plays no part in the RDF world view. Furthermore, such systems also often require content to be ‘loaded’ into several different internal content stores (e.g. once for full text index, once to ‘strip’ metadata to the RDF/SQL store). This builds a structural inefficiency into the system as content must be found, queried and updated in multiple locations.

When we design the architecture for our publishing platforms we like to think hard about these problems. Often we conclude that XML database technology (such as MarkLogic Server) provides the best solution because it allows content to be stored and queried in a single place. Using an XML database removes the artificial split between content and metadata inherent in conventional journals systems and allows us to build search queries across an entire collection of content. This in turn helps to drive discovery and usage by building deeper links between related content and combining and assembling content from an entire collection in new ways.

However we also realise that different clients have different needs and that it is critical to prioritise meeting business goals ahead of making specific technology choices. The motivation for our overall technical architecture choice in a given project is simple; to learn the lessons of past systems and ensure we choose the best possible technology basis for any new system. Understanding the problems publishers have faced in trying to adapt legacy journals systems to the more challenging world of books and reference helps us to make the most appropriate technology choices for future publishing platforms.

For those who were unable to attend, or would like to listen in full again, you can listen to a recording of eBook readers and the Future of Other New Technologies and view the accompanying slides below.

If you would like to view the slides or listen later at your own leisure I have made the accompanying slides and an mp3 recording of the presentation available here.

Developing mobile optimised websites

By developing a mobile optimised website publishers can provide the best user experience  for users with limited screen sizes and limited connection speeds. For example, this blog is fully optimised for mobile devices, so do try reading it from your smartphone to check out how the experience differs.

Advantages

• Lower cost of development (compared to app development)
• Fast time to market
• Support for dynamic content, content updates can happen in real time
• Completely cross platform and cross device
• The ability to leverage existing publishing commercial infrastructure; access controls and subscription models for existing websites apply equally well to mobile optimised websites
• Leverage existing development and production processes and tools
• Leverage publishers existing hosting and support services

Disadvantages

• Internet connection required at all times
• Speed & latency of the site must be fully optimised
• Mobile delivery creates new ways of using content and business models will have to adapt
• Discovery of the content will be via existing channels; SEO work may be needed to ensure users can find your content effectively

Developing apps

Developing an app allows a publisher to take full control of all of the capabilities of the underlying device. It gives you the opportunity to use every advantage that mobile delivery has to offer. If fast interactive graphics and immersive functionality are important to your content then developing an app will be the most appropriate choice.

Advantages

• You can control the user experience, ensuring all users get the best you can offer
• Ability to use native UI for fast graphics
• You can leverage existing distribution and monetization services via app stores (e.g. Apple and Google)
• B2B publishers can extend their reach into direct consumer markets by using app store services
• The app upgrade mechanism can be used to push new content and functionality onto devices automatically

Disadvantages

• Custom app development may be expensive
• Publishers may have to re-invent the wheel by re-implementing solutions to problems they have already solved in the web environment
• There is little or no potential for sharing development effort between different mobile platforms
• Lack of ways to hyperlink apps together. Academic citation depends on linking but apps cannot be linked in the way web resources can.
• Limited memory of mobile devices means large publisher datasets cannot be accomodated.

These are just some of the pros and cons of each approach. Inevitably the best choice in any given situation will depend on the content, the market, the level of investment needed and the risk publishers are willing to take on individual titles.

At Semantico we anticipate that some new titles and projects will need development for both web and apps whilst existing online titles may have apps designed for them. By offering both web and app development we’re giving publishers the chance to treat the two ways of presenting content as one project.

Each one of the bullet points could be a blog post in itself and no doubt we’ll be revisiting some of these issues future posts. Because whilst the issue of whether or not apps are the future of ebooks is still undecided, those publishers who’ve not yet dipped their toe in the water need to start rolling up their trousers. Do contact us now to start a conversation about your mobile delivery options.

Many publishers have started using the iPhone App Store as a channel to sell book content by packaging e-books as applications. There are currently 18,000 books in the App Store, and books are the fastest growing category of application in the store.

With the launch of the iPad and the iBookstore, Apple has given publishers another option for delivering content. In this post we analyse the pros and cons of both approaches.

Conclusion

For straightforward chapter based book content it seems clear there is no longer a compelling case for publishers to deliver e-books as apps. The extra cost of software development, combined with the slowness and lack of scalability in the approval process no longer make sense now that Apple have introduced the iBookstore.

For other types of content the case is not so clear cut. Publishers with the ability to invest can develop reference based apps which add value by delivering content in context. Mobile workflow applications will still be a signifiant growth opportunity for publishers.

As the iBookstore is currently geared toward the consumer market, publishers who deliver large databases of journal and book content to institutional markets should look to the iBook model as a way of tapping into a traditionally harder to reach individual market. And publishers wishing to monetise currently offline backlist content should look carefully at the opportunities afforded by the iBook platform.

Although the iBook reader app is not currently available on the iPhone there seems no compelling reason why it will not be released in response to market demand. Bearing in mind the popularity of e-book applications on the iPhone this would appear to be a very simple decision for Apple.

Are we there yet?

Along the way I’ve surveyed the current landscape and looked at the multi-dimensional influence of Web 2.0. The journey has highlighted tensions caused by Web 2.0, and the new software models of the API-driven web, and the pressure it puts on existing models of identity management which, as I have attempted to show, struggle to cope with the complexity of this new universe.

In this post I want to delve a little deeper into the privacy and security implications of that environment and to look forward to the semantic web, before making some recommendations for how I think identity management needs to develop in order to get us to our goal.

Lock-in, ownership & control

The providers of social network services have been quick to understand the potential of being online identity hubs for their users. This is a natural function of their prime aim of driving up usage; being an identity provider is just one more service that Google or Yahoo can deliver and one which keeps them firmly at the centre of our online worlds.

As we move more of our identity information online it becomes potentially much easier to move that information around. With that ease of moving data around, concern increases that our personal data could be passed on to third parties. At best it might then be used to spam us, or be sold to our competitors. At worst, we might loose control over ownership of our online identity altogether, and it could be used for fraudulent purposes. These data protection issues are furthermore set in a global context where national legal frameworks may no longer make sense.

Who to trust

In this federated environment of identity providers there are a number of important questions that must be addressed, including:

1. How much do we trust these identity providers with our personal details?
2. Who audits services such as Facebook or LinkedIn to ensure security issues are addressed?
3. How much to we trust the downstream sites using these identity services with our personal details?

With these questions in mind, it is possible to imagine an ‘identity supply chain’ where different entities within the chain only know the smallest parts of a given identity that are needed to perform their function. For example, I could log in to a website without the website itself knowing my password. Similarly I could order goods without disclosing my full identity to the shipping agent, and I could leave commentary on a blog without the blog system needing to know my postal address.

The semantic web

In the web of linked data, identity is centrally important for determining trust, provenance and authenticity. Understanding who made a particular assertion is essential within scientific communication, for example, which is necessarily a continuous debate. In such a discourse, degrees of trust and certainty are necessarily important in evaluating and combining facts from different sources. Identity is needed for:

• Annotation – making commentary and building discussion around facts and data
• Augmentation – adding new data and assertions to existing data sets based on new evidence and experiments
• Refutation – allowing statements to be contradicted according to new evidence or new interpretation of existing evidence, based on different degrees of trust within a system

All of the above types of communication are quite possible using the linked data semantic web model we have today. For example, I am free to publish any kind of statements I like which refer to other statements. However in doing this it is critical that the notion of my identity is preserved in relation to the statements I make, just as its is critical that the identity of the author of the original statements is clear.

Wikipedia or Schizopedia?

The huge success of Wikipedia has shown that collaboration and openness combined with low cost of usage can combine to produce true value. The centralised model of Wikipedia allows the tracking of individual edits but not necessarily on a named or identified user basis. Essentially, the decreased end user cost of building the information resource has been traded against a consequent lack of traceability and provenance.

Compare this with the semantic web linked data model. Here no centralisation is expected (or even possible). I can publish some facts, you can comment, agree, refute or augment these facts, and by publishing your assertions in the linked data cloud you can also join the conversation. So can many other people, with many other agreements, contradictions and additional observations. Here the conversation could start to resemble schizophrenia, with many voices talking at once. Without a solid notion of identity and provenance it is impossible to build a consistent and coherent model of the facts.

New licence models

Many publishers have traditionally licensed their intellectual property to third parties in the form of data sets. This could be, for example, to provide language translation devices to language students, or alternatively to provide an abstract service to complement the primary source materials in a particular discipline. These licence deals inevitably involve the simple transfer of the published content as a set of static files from licensor to licensee.

However, as the service offered by publishers matures to include APIs to their data and services instead of simple file transfer, the need to address identity issues arises. Typical identity issues include:

• The need to ensure the API is used only by the licensee
• The need to record usage metrics by each licensee

Additional issues may include the need to track the licensee’s individual users and their usage patterns. Furthermore if the API delivers services to the licensee’s end users beyond simple search and content retrieval, then it may be necessary to exchange authentication and identity information about the end user of the site.

Conclusions

Drawing all the threads together that I have explored in this and my previous posts on Identity Management I have come up with the following conclusions.

The answer the question ‘are we there yet?’ is of course no, not yet. The more interesting question is whether we are even on the right road to get there. In order for us to reach our destination, a rational and usable system for managing identities on the web, the following needs to happen in my view.

Publishers and information professionals need to collaborate to design an identity framework what meets the needs of all stakeholders including contributors, researchers and institutions. This framework should be built on existing open standards such as OpenID and DOI but must not sacrifice usability. The solution must be built on an organisational infrastructure which is credible and can be trusted across the industry, and should ideally be based on open source software which can be independently audited for security concerns by any interested party.

The starting point for such a collaboration may already be in place with the recently announced ORCID (Open Researcher and Contributor ID) initiative. I for one will be watching this project closely over the coming months and I look forward to their developments and progress.

And in the meantime … keep quiet you kids there in the back!

The iPad

Apple have approached the challenge of creating a tablet computer from the opposite direction to most established devices on the market. Instead of stripping down a laptop or netbook they have chosen to create a device which is essentially a scaled up iPhone. The iPad runs the iPhone OS, and consequently runs all of the 140,000+ currently available iPhone applications. The user experience is very similar to the iPhone, albeit with all of the advantages of a much larger screen, faster processor and more memory.

The iBookstore

OK, lets cut straight to the chase. For publishers the iBookstore is the iPad’s main event. Apple have signed deals with Penguin, HarperCollins, Simon & Schuster, Macmillan and Hachette to seed the iBookstore with content in the EPUB format. Apple are planning to offer 70% of the sales revenue to publishers, following closely the model already in place for application developers in their app store. This compares very favourably to the cut of up to 50% which Amazon take from titles sold on the Kindle.

The business model is a one-time purchase of the content which will be downloaded to the user’s iPad device. The iBookstore uses the EPUB format, which allows the iBook reader application to reflow the text according to screen orientation and to change fonts and font sizes.

The iBookstore will be the only way of purchasing eBook content to use with the built in iBook application, although other applications, such as Stanza, will be free to compete with Apple’s business model. If they can.

The DRM issue

Digital rights management will not be needed with the iPad because content, once downloaded from the iBookstore, cannot be copied from one device to another. This is because Apple have followed the iPhone model in making the iPad an essentially closed, locked device. There is no user access to the underlying file system, and moreover only books purchased from the iBookstore can be read with the built in iBook reader application.

A device I’d give my grandmother

Apple know a thing or two about usability. This device neatly fills the gap between the iPhone and a MacBook. It is essentially a laptop for those who don’t want a laptop or a smartphone. The iPhone OS is so simple and intuitive to use that it would really be suitable for even the most computer phobic. And this weighs heavily in my thinking that it will win out as a consumer eBook device.

Although at $499 it is twice the cost of the Kindle or Sony readers (for an entry level model) the extra cost is extremely easy to understand. Colour, smooth graphics and the availability of 140,000 applications make it clear this is much more than a single function device. In this respect it simply can’t be compared to the Kindle or Sony reader.

Conclusion

In many ways Apple are attempting to replicate the situation which led to the huge success of the iPod/iTunes combination in disrupting the offline music sales and distribution industry nine years ago. By providing sexy, easy to use commoditised hardware at the right price point, combined with a single simple way to purchase content, they stand an extremely good chance of disrupting the established eBook readers. Furthermore the sheer appeal of the device can only fuel the further growth of the eBook marketplace.

More on this excellent seminar can be found at The International Association of Scientific Technical and Medical Publishers website.

Video by River Valley TV.

Personalisation

The ability to personalise and customise web sites is taken for granted in the Web 2.0 world. We are now used to logging on to services such as Twitter, Facebook and Google and being delivered into environments that we can customise and personalise to our own needs.

The expectations of users within this new world provide problems for identity management, which, as we saw in my last post, has to use an infrastructure that was never designed to cope with such user requirements. The danger is that too inflexible a system might act as a brake, or even a deterrent.

Flexible identity management solutions are critical to the uptake of personalisation services because they lower the barriers to access to new services. Commenting on a blog post must be made as simple as possible, yet in order to preserve my online identity in any comments I make it is essential that I can authenticate myself. The barrier here in many cases is that I don’t want to register yet another username and password with each blog site I visit in order to identify myself. One solution to this is the OpenID protocol which allows me to log into participating third party sites using the identity that I’ve already established using an identity provider such as Google, Yahoo or Flickr. Few publishers have so far adopted the OpenID standard; notably Reed-Elsevier have set up an OpenID identity provider, IDkee; let’s hope OpenID authentication is soon to be implemented for their flagship site Science Direct.

Many current publisher sites provide personalisation features but these functions tend to exist exclusively within the publisher’s own silo. If I store bookmarks or reading lists within one publisher silo, there is no way to share these with my social contacts across the web. If I contribute to a discussion within one publisher platform, that discussion tends to stay behind the paywall of the silo. Why can’t I call up a view that shows my comments across all the publisher sites I use on the web?

Usage metrics

If the multiple identities we all have to maintain within the Web 2.0 world provide problems for users trying to access services, they also cause snarl-ups at the back-end of the process – with the tracking of usage by providers of those services.

Usage metrics are important for publishers. Mainstream web analytics software and techniques tend to focus on general trends and usage patterns because it is not straightforward to reliably identify individual users, but for publishers, identifying institutional users and correctly accounting for their usage is critical in delivering industry reports compliant with the COUNTER standard.

However, the COUNTER model does not even begin to address the complexities that can arise when multiple identities are in play. For example, I might need to express three different levels of identity at the same time:

1. My own personal account, for content I have purchased for my own private use.
2. My department’s subscription to an information resource (not shared with the rest of the institution)
3. My institution’s identity and its subscription rights to an online resource.

In this scenario, when I access a piece of content online in a publisher platform, the access could be accounted for at any one of the three levels above. The COUNTER standards do not currently cover this type of compound identity problem.

The semantic web also presents a serious challenge for usage metrics. Once facts are published in the linked data cloud, measurement of usage becomes impossible: those facts could reside in any system or platform and are no longer under the control of one publisher.

Usage metrics are important for publishers not only as an accounting tool – they also help publishers to improve the usability of their services. And usability is another critical issue bearing on access management.

The Shibboleth protocol was set up as a way of making things easier for users accessing services, and to streamline this question of multiple identities somewhat. However, Shibboleth itself has turned out to have some usability problems.

Usability issues

If Wikipedia required users to login with a personal username and password how much would it affect usage?

This question aside, the federated nature of the Shibboleth single-sign-on system means that in order to log in users first have to confirm where they come from, in order that they can be directed to the correct identity provider. This problem is further compounded by the large number of identity providers in the Shibboleth system (740 in the UK Federation alone). Asking a user to select from a list of 740 places is clearly a significant barrier to usability.

OpenID based systems don’t have the problem of large federations, since anyone, potentially, can set up an identity provider service. The usability challenge here is to pick the simplest list of providers and give the user an alternative way of entering a full OpenID URL if their provider is not on the list. Again, another usability challenge most users would rather not face.

Finally, and not to be forgotten, is the thorny problem of logging users out of a service. In a single-sign-on environment the problems with single log-out are surprisingly complex. It requires coordination between all the sites a user has visited and, combined with the difficulty in educating users that logging out is even necessary, presents another usability challenge.

Conclusion

Identity management plays a key role in the Web 2.0 world and this being so, Web 2.0 tools need to step up to the privacy and security challenges raised by the new software models of the API-driven web. In particular, thought needs to be given to these key issues of personalisation, usage metrics and usability.

I have one further post to deliver on the subject of online identity. This will consider the question of how close we are in reality to delivering on what has been called Online Identity 2.0. And I’ll also be examining a further trio of key issues key for the semantic web: provenance, trust and authenticity.

The web as it exists today suffers from the lack of a consistent way of managing identity. There are challenges in both identifying myself to the sites I visit and in identifying those sites myself. Without any standard mechanism to deal with this, web developers have devised an array of different and incompatible schemes to manage identity. This presents serious challenges, since authenticity and trust are critically important concerns for publishers and information providers.

In the beginning, the internet was a place of implicit trust. Networks were small, and users were trusted with not abusing services. During the rapid growth of the network, many security and identity problems with the underlying network protocols and subsystems were discovered and solved. However the web itself has followed a different path of evolution.

Hypertext and identity

Early hypertext systems were firmly based on the idea that all users would need to be able to read and write documents. The models built into the first systems, such as Xanadu, allowed users to compose their own documents and to annotate documents authored by others. This functionality naturally required users to authenticate and identify themselves.

However, when Tim Berners-Lee invented the web in the late 1980s the distributed authentication services to support the read/write web did not yet exist. He simplified the original hypertext vision in order to make it easier to implement by removing the composition and annotation elements. This in turn removed the need for identity management systems which significantly reduced the complexity of the software needed. This simplification allowed the web to grow quickly at the cost of initially making it into an essentially read only environment for most users.

Visitors and residents

Recent JISC research into how people use online services has started to focus on the distinction between ‘visitors’ and ‘residents’ (proposed as a more useful replacement for the previous talk of ‘natives’ and ‘immigrants’). Those who use the web as a tool for specific tasks can be seen as visitors whereas those who spend a large amount of time online can be categorised as residents. Residents clearly have a strong need to maintain their online identity, since their presence on the web is an essential part of their overall social interactions. Visitors still need to manage identity, though their needs are more in the context of being able to authenticate themselves and the sites they visit.

This distinction between visitors and residents is in reality more of a spectrum of different behaviours, but it’s critical that publishers understand the common needs for identity and authentication services. As the participative read/write web returns with the move towards Web 2.0 services the traditional roles within publishing of author and reader are disrupted, and again this is exactly the point where online identity, and trust in that identity, is centrally important.

Publisher platforms

Current publisher platforms provide a good example of the patchwork of different services and approaches to authentication and identity management developed over the past decade. Most feature a combination of the following approaches:

1. IP authentication to identify a user’s home institution. This works well for on-campus users as it allows invisible authentication and thus provides the lowest possible barrier to usage. However this scheme is useless for remote users.
2. Federated authentication to identify an individual user within an institution. This covers both the historical Athens protocol used within the UK and the Shibboleth protocol now gaining significant traction in the UK and America. This works equally well for on and off campus users at the cost of a sometimes difficult user experience.
3. Other username/password schemes. Individual identity outside of the institutional context is invariably handled by requiring users to create their own account within a publisher’s system.
4. Other remote access schemes. Schemes such as EZproxy or referrer authentication allow remote users to identify their institution to a publisher platform by first logging into their institutional home page. Again, usability is compromised by having to go through the institutional portal to gain access to the resource.
5. Personalisation log-in. In order to uniquely identify individual users within an organisation it has often been necessary to have a second level of log-in within a publisher platform. This requires users to create and maintain yet another identity within the publisher platform.

These approaches cannot usually be combined or aggregated, which acts as a further barrier to usability and usage. For instance, to access individual purchases it may be necessary to log-out of the institutional account and log-in again as an individual user.

Publishers are increasingly looking for business models which allow them to combine different levels of institutional access rights (e.g. school, department and college) with individual access rights. The existing infrastructure is in many cases not able to support these new models and so new solutions are needed.

Contributor identity

As the distinction between reader and author becomes increasingly fluid both parties need to be able to manage their online identity to interact with the read/write web. The CrossRef ContributorID project is focused on providing an identity framework designed to address the issues around knowledge discovery and authentication for the scholarly publishing community.

Conclusion

I’ve attempted to provide a survey of the existing landscape here, but where Online Identity is going next is a much bigger question – not to mention our ideas for where it should end up! Next month I want to discuss the influence of trends associated with Web 2.0 and how they are affecting the field, as we move towards Online Identity 2.0.