Overview over all blog items

Oct 06, 2014

Plone Developer Survey

Help me to collect some metrics about the Plone developer community.

Here is small Plone developer survey. The aim is to get some metrics about the Plone developer community in preparation to my "Why Plone is going to die" to be given at the Plone Conference in Bristol at the end of this month. Numbers and feedback will help to make this talk too personal and not too crushing :-)


Oct 03, 2014

Upcoming talks

October is filled with exhibitions and conferences. So I am going to present three different talks this month.

Onkopedia - Ein Leitlinienportal auf dem Weg zum XML-basierten Publikationsworkflow

This talk will discuss our ongoing Onkopedia publishing project where we switch the customer to an XML-based publishing workflow. The basis of all publications are Word/DOCX documents that we convert to XML, store them into an XML database and generate HTML and PDF versions from XML content.

Where: DMS Expo 2014 Stuttgart, 08.10.2014

Structured content rocks - Integration of eXist-db (XML database) with Plone

Structured content (XML) is widely used in the publishing industry and documentation applications. This talk will demonstrate our approach of integrating XML content with Plone in a sane way. Instead of storing XML content natively in Plone we take a different approach and integrate Plone with the open-source XML database eXist-db. eXist-db is in some ways similar to Plone and provides an hierarchy storage model, support for latest XML technology like XPath 3, XQuery and XUpdate. Plone applications can communicate with eXist-DB both through WebDAV or through the REST API of eXist-db. The integration layer is provided through a small Plone add-on zopyx.existdb that makes provides out-of-the-box traversal support from Plone to eXist-db content, a generic API approach, ACEditor integration for basic content-management from within Plone. The overall talk is driven by an ongoing publishing project where we switching from an HTML-to-PDF workflow to XML-to-PDF with Plone as integration platform (https://www.onkopedia-guidelines.info).

Where: Plone Conference 2014, Bristol, 27.10.2014 - 02.11.2014

Why Plone is going to die!

Plone is now more than 12 years old and has grown a lot. A burden of 300 and more modules and dependencies make the Plone programmers life a pain. My claim for this talk: Plone became with every version more unstable, more unapproachable and more unpredicatable. The whole complexity of Plone can only be managed and overseen by a small number of core developers. Even integrators with a decade of Plone experience have a hard life with Plone with every new release. My talk is the result of our 2014 Plone projects (legacy code and new projects).

I will present many examples where working with Plone become a major project risk from the technical prospective, budget-wise and developer-wise. Many of the Plone APIs are inconsistent, have a major lack of error handling or expose a lot of intern functionality (like the ZCA) that show have never surfaced. This is not a rant talk but it takes a clear and provocative position against the current Plone developer team. Approachability of a programming framework (or however you would call Plone) for developers is a key feature for success. In our experience Plone is a declining CMS (like many others) - it is also declining because of its complexity and many inadequateness. This is why Plone is going to die - because it can not attract new developers in order to survive in the market.

Where: Plone Conference 2014, Bristol, 27.10.2014 - 02.11.2014


In our current Plone project we have the usecase where we need to copy over all schema-defined values of a Dexterity-based content-type to a new "empty" instance of the same type. In Archetypes the functionality would be a three-liner however the Dexterity world with schemas, behaviors etc. is much more complicated and by a magnitude less approachable. The naive approach would be to read the values directly as attributes from the original object and assign them back to the target instance like

value = source.some_field
target.some_field = value

However the Plone world is bad and evil to developers and behaviors require some special attention since because every behavior may implement a different storage strategy under the hood that would be bypassed by direct attribute access. Instead you need to adapt the source and target instance for each field to the specific schema adapter and read/write the values from the adapter. 

Here is the complete code how to copy over all schema-based values from a 'source' object to the 'target' object of the same class.

The insanity is directly visible and it is obvious that this programming approach is hard to sell as a pro-argument to non-Plone developers. 

def clone_plone_metadata(source, target):
    """ Copy all metadata key-value pairs of the 'source'
        Dexterity content-object to 'target'.
    """

    from plone.dexterity.interfaces import IDexterityFTI
    from plone.behavior.interfaces import IBehaviorAssignable

    if not isinstance(source, target.__class__):
        raise TypeError('source and target must be the same class ({} vs {})'.format(source.__class__, target.__class__))

    schema = getUtility(IDexterityFTI, name=source.portal_type).lookupSchema()
    fields = [(schema, schema[name]) for name in schema]

    assignable = IBehaviorAssignable(source)
    for behavior in assignable.enumerateBehaviors():
        behavior_schema = behavior.interface
        for name in behavior_schema:
            fields.append((behavior_schema, behavior_schema[name]))

    for schema_adapter, field in fields:
        source_adapted = schema_adapter(source)
        target_adapted = schema_adapter(target)
        value = getattr(source_adapted, field.getName())
        if isinstance(value, str):
            value = unicode(value, 'utf-8')
        setattr(target_adapted, field.getName(), value)

Sep 12, 2014

Berlin Python Sprint result: new PDF version of the "Plone Nutzerhandbuch"

PDF generation of Plone documentation using CSS Paged Media

As part of the Berlin Python sprint this week at Veit Schiele Communications GmbH I worked on a new PDF version of the "Plone Nutzerhandbuch" generated using Sphinx and PDFreactor.

Here is the preliminary draft version of the PDF - a more detailed blog post will follow soon.


Jul 31, 2014

EuroPython 2014 finally over - some notes on the EuroPython Society

Some notes on EuroPython 2014 that ended last Sunday with over 1200 attendees.

EuroPython 2014 is finally over and it was a great success. We counted 1226 attendees which is an increase of more than 40 percent of the EuroPython 2013 in Florence. We received lots of positive and overwhelming feedback for organizing a full week with Python related stuff. I personally worked as head of communications since fall 2013 on EuroPython 2014. I wrote dozens of blog entries on blog.europython.eu, I was the public @europython voice on Twitter, most of the incoming helpdesk issues (more than 1200 JIRA tickets) went through my fingers and finally I worked together with Stefania Trabucchi on the automatic PDF generation for parts of the EuroPython 2014 brochure (generation of talks, speakers and schedule) and handled the setup of the EuroPython Guidebook application. I am personally very happy with the outcome of our work over the last nine months. This was really hard work with a small team, lots of ups and downs, lots of new experiences working many new people, many new challenges like dealing with upcoming shitstorm discussions (e.g. the diversity discussion around April on social media)...I learned a lot, I had fun, I had frustration but in the end we all had the same common goal: a EuroPython 2014 that would rock - and EuroPython 2014 rocked!!!

Organizing  such a big conference is a huge undertaking. Several people worked partly full-time on this project (beside their usual job and business). A not so perfect role during the organization process played the EuroPython Society (EPS).  The EPS is a non-profit society under Swedish law and is the  holder of the „EuroPython“ trademark. The EPS holds this trademark since some weeks.

The goals of the EPS are according to their bylaws:

2. The society is a non-profit society (ideel förening) under Swedish law. The purpose of the EP is to organise conferences centered around the programming language Python

This is in general a very honorable goal but goals and reality are two different things. The first public discussion about problems between the Berlin organizers and the EPS popped up early 2014 with an email (https://mail.python.org/pipermail/europython/2014-January/008250.html, https://mail.python.org/pipermail/europython/2014-January/008252.html) of a former EPS member which basically brought the complete situation with the EPS to the point. Although I was not directly involved with the communication between EPS and the Berlin organizers I can only summarize the behavior of the EPS  as only-demanding, an arrogant top-down attitude and a conduct that is far away from being acceptable for an open-source community.  Speaking of „the EPS“ as an organization is perhaps not valid because there are always persons connected with an association - approachable persons, non-approachable persons and silent persons.

Some weeks ago I became an official member of EPS and had the pleasure to participate in the general assembly of the EPS held during the EuroPython 2014 conference in Berlin. The first weird thing was that there was a total of 30 minutes planned for the general assembly which includes reports of the chairman, the treasurer, discharge of the board, voting of the new board and especially discussions. How strange I thought from the beginning. A typical general assembly of comparable organizations hardly take under  90 minutes, usually even longer.

EPS counted 68 members at the time of the general assembly where about 30 members joined the EPS shortly before general assembly. Out of these 68 members only 14 members appeared to the general assembly including two of the five EPS board members (Fabio Pliger, Marc-André Lemburg).

This one hour general assembly appears in the retrospective like a bad theater piece of a provincial theater. The board was not able to give a reasonable report over the last year and always stressed out about mistakes made in the past. For example  there are no records and protocols available of board meetings and general assemblies available for the last years - either lack of transparency or just a complete unorganized society.

Marc-André Lemburg was not able to give a reasonable and acceptable financial report (the slides really contained „XX Euro“). He could just give a vague estimate on the money that EPS has in the bank - not very much money as it turned out. Dear EPS, every rabbit breeders association is better organized.

They have five persons on the board and their only day business is to meta-organize a EuroPython conference and dealing with EuroPython trademark issues.  The real organization work is always carried out by the local organizers.

What are these people doing? Especially when the trademark issue was the only major operation carried out over the last 12 months?

Then the voting for the board…too make it short: old board = new board. All members of the old board became elected as new board members. Both Fabio and Marc-André had been involved (in a good and bad way) with the Berlin organizers over the last year. All other board members (Jacob, Anton, Vicky) are not very well known in the European Python community and had been elected in absence - legal and covered by the bylaws but a slightly absurd election. I felt a bit like in the 80s with the election processes of the politbureau in communist or socialist states.

One might argue why I did not run myself for the board? Answer: lack of information about the EPS and their current inner state in advance, lack of knowledge about all candidates and especially where was no significant possibility for changing the power within the EPS at this point.

Let us talk about the „EuroPython“ trademark. The EPS claimed from the beginning of the EP14 organization that they would hold the „EuroPython“ trademark. However EPS could never provide any evidence. In fact the trademark was granted to EPS some weeks before the conference. But EPS urged the Berlin organizers  multiple times to include the trademark claim in press releases and on EuroPython 2014 related information The conference was organized by officially under the hood of the Python Software Verband e.V. - a society under German law. And according to German law we could not support this trademark claim. However EPS always insisted on their point of view. The organizers had to comply with German law and not with Swedish law in this case.

Speaking of legal issues: EPS tried for the first time to regulate the relation between EPS and the local organizers with a contract. To our knowledge there had been never contracts for former EuroPython conferences in the past. To make it short: the draft of this contract had to be rejected by the organizers because it was completely one-sided. These kind of contracts can be made between businesses but not between entities of an open-source community. The cumulation of insolence was the requirement that a conference profit should be split 50:50 between EPS and local organizers but a conference loss should be carried by the organizers themselves with a maximum involvement of the EPS limited to 7.500 Euro. You could read the draft as: we do nothing and get half of the money but it is your risk if there is a loss. Fortunately EPS showed some insight during the general assembly that this contract was a mistake.

The new EPS approach: EPS want to take more control over the EuroPython conference organization, EPS wants to be the main contracting partner and it want the organization to be taken over by permanent teams. This is approach will never ever work out. Like with all projects: a small amount of people will be doing the major work and these people will be local organizers. It is not imaginable that other EPS member will and can spend a significant amount their personal time in leading positions dealing with local organization issues. The programm committee of EuroPython 2014 as an example was already internationally distributed organized. It did not solve the problem that the programme chair had the burden with organizing the programme. It is unlikely that the chair of EP14 would take over the responsibility for a theoratical EP15 that would not take place in Berlin. The key position of conference organization teams are time-consuming and exhausing and you do it for one or two years - especially when you take into account that many of the organizers run their own businesses. No, the workgroup approach is a nice idea but it is going to fail. There are too many local aspects in a conference organization that can not be handled by EPS or EPS workgroups. Yes, EPS wants, wants a lot...but theses are only visions and dreams that will not become reality. Life is different, EPS never organized a conference, EPS just contributed the name.

Talking of EPS persons: Fabio Pliger worked with the Berlin organizers in a goal-oriented and constructive way while others glazed by non-visibility or through their unacceptable arrogant attitude. My respect to Fabio for his support.

Conclusion

The EPS is a poorly organized society with little backing by the community and not much assets. The trademark on „EuroPython“ is the only thing that the EPS has in their hands - apart from little money in the bank. EuroPython was an overwhelming success - by numbers and by feedback received by attendees. The experience with EPS tells us that nobody really needs the EPS for organizing a big Python conference in Europe. The only thing that the EPS can provide: the name „EuroPython“. But wait: did you notice our branding „EP14“. Would it be a problem for you visiting a „EP15“ conference instead of „EuroPython 2015“? Or could a similar conference be named „PyCON Europe 2015“ or „Python Europe 2015“ or something like that? You see the problem…the EPS is constructed solely around the name and their trademark „EuroPython“.  

So their are obviously the following options for the future

  • Forget EPS and let the local communities do their own thing. EPS can only give the name „EuroPython“ but they are not able to put more effort into controlling and directing the organization of such a big conference nor they have the financial or personal resources for backing a conference themselves. The proposed workgroup model for organizing further conferences is not well thought and will never work in reality for various reasons.
  • Let us send the current EPS board (except Fabio) into the desert and install persons as board members that are interested in one thing: organizing further great conferences with local organizers in a constructive way instead of defending their only poor assets. There is no reason why the EPS needs a big board of 5 persons. Reduce it to three board members where one member could represent the „old“ EPS and their interests, a treasurer and person representing the local organizers of the upcoming conference…something like that…I admit that this is only a rough thought…

For the rescue of the EPS: it seems that the EPS got the notice and took their lessons from that what happened over the last year. The European Python community is about people and not about exercising power build on top of a trademarks and a poor standing within the community. The board mentioned that they want to held a new general assembly soon in order to fix the problems. The EPS wants to be more open and transparent. I appreciate these goals but time will tell how this will work out in reality.


 


Jul 30, 2014

EuroPython 2014 finally over - some notes ... Part 1

Some notes on EuroPython 2014 that ended last Sunday with over 1200 attendees.

EuroPython 2014 is finally over and it was a great success. We counted 1226 attendees which is an increase of more than 40 percent of the EuroPython 2013 in Florence. We received lots of positive and overwhelming feedback for organizing a full week with Python related stuff. I personally worked as head of communications since fall 2013 on EuroPython 2014. I wrote dozens of blog entries on blog.europython.eu, I was the public @europython voice on Twitter, most of the incoming helpdesk issues (more than 1200 JIRA tickets) went through my fingers and finally I worked together with Stefania Trabucchi on the automatic PDF generation for parts of the EuroPython 2014 brochure (generation of talks, speakers and schedule) and handled the setup of the EuroPython Guidebook application. I am personally very happy with the outcome of our work over the last nine months. This was really hard work with a small team, lots of ups and downs, lots of new experiences working many new people, many new challenges like dealing with upcoming shitstorm discussions (e.g. the diversity discussion around April on social media)...I learned a lot, I had fun, I had frustration but in the end we all had the same common goal: a EuroPython 2014 that would rock - and EuroPython 2014 rocked!!!

Part 2 - to be published soon - will cover some remarks on the experience with the EPS.

 


Jul 09, 2014

Plone - the broken parts - a non-pythonic programming model

This is a loose series of blog posts about parts of Plone that I consider as broken from the prospective of a programmer. The blog entries are based on personal experiences with Plone over the last few months collected in new Plone 4.3 projects and some legacy projects but they also reflect experienced learned from other non-core Plone developers involved in these projects (developers on the customer side).

Archetypes served for than ten years as framework for building content-types in Plone. Dexterity had been designed as successor of Archetypes with the goal making the content-types development within Plone more easy with less boilerplate. Content-type development should become be more pythonic. Creating a new of a content-type in Plone in a "pythonic way" would look like this:

obj = plone.api.content.create(some_type)
obj.id = 'foo'
obj.title = u'hello world'
obj.start_time = datetime(...)

This approach is in general working as long as all data are stored directly as attributes on the object itself. However there is a problem with the introduction of Dexterity behaviors. Behaviors can implement different storage strategies other than the standard attribute storage. So if a content-type is composed of several behaviors it is not legitimate to read and write object attributes directly that are introduced and managed by a behavior. With many behaviors you need to know which field comes from which behavior exactly. So your code may look like this:

obj = plone.api.content.create(some_type)
adapter1 = IBehavior1(obj)
adapter1.field1 = ...
adapter1.field2 = ...
adapter2 = IBehavior2(obj)
adapter2.field3 = ... 

You see the problem that this code is no longer pythonic. It exposes the Zope Component Architecture for simple object modifications to a degree that appears wild to non-core Plone developers. It is getting more scary when you need to call a content-type functionality for performing some internal processing like it is needed in plone.app.event:

from plone.app.event.dx.behaviors import data_postprocessing
from plone.app.event.dx.behaviors import IEventBasic
event = plone.api.content.create('Event')
adapter = IEventBasic(event)
adapter.start = datetime(..)
adapter.end = datetime(..)
data_postprocessing(obj, None)

This is no the point where we have a communication problem teaching Plone and Dexterity development to average developers. Such a programming model can not be taught to non-Plone people. Most people scratched their head and called what-the-fuck!?

Related topic: hooks and event subscribers. Archetypes offered a bunch of hooks (good or bad) in order to provide additional functionality inside the content-types implementation without needing to hack for example the auto-generated forms for view + edit. E.g. at_post_edit_script() could be used to perform arbitrary actions after submitting the edit form. The ZCA-ish approach are event subscribers where you can register arbitrary methods and handler for various life cycle events. This concept is in general a nice solution if you want to add external behavior to existing (3rd party code) without modifying the actual implementation. But with Dexterity this is also the only solution for hooking your own additional functionality into your own content-type. 

Example: in zopyx.existdb we need to push some information from the Dexterity content-type after saving the data in Plone to a different system. This is a core functionality of the content-type we are implementing - it is not business logic, it is core functionality and therefore it should be able to implement this directly inside the content-type. Not possible with Dexterity right now. Instead the related code must be moved into an event subscriber outside the scope of the content-type implementation although it belongs to the core implementation. The result is scattered code that actually belongs together into one class. From the prospective of reading code and understanding the programming logic this is not desirable. So in the current programming model you end of with code that belongs to a particular content-type spread over the content-type class, event subscribers and browser views. So there is really a need for having a way to implement hooks in a reasonable way in the scope of the content-type class without explicit event subscribers.


Jul 09, 2014

Plone - the broken parts - Member schema extenders and plone.api

This is a loose series of blog posts about parts of Plone that I consider as broken from the prospective of a programmer. The blog entries are based on personal experiences with Plone over the last few months collected in new Plone 4.3 projects and some legacy projects but they also reflect experienced learned from other non-core Plone developers involved in these projects (developers on the customer side).

It is a common project requirement to extend the Plone user memberdata schema. The common approach is perhaps documented here (https://pypi.python.org/pypi/collective.examples.userdata). The additional fields of the memberdata must be defined as a zope.schema. Nothing special - same as definining forms using z3c.form or writing content-types with Dexterity. In a recent project we had a working memberdata  extender and had to extend it with two schema.List fields. Both fields were added to the schema together with the following (working) adapter implementation: 

from plone.app.users.browser.personalpreferences import UserDataPanelAdapter

class EnhancedUserDataPanelAdapter(UserDataPanelAdapter):
    """ Adapter for extended user schema """

    def __init__(self, context):
        super(EnhancedUserDataPanelAdapter, self).__init__(context)
        self.add_property('academic')
        self.add_property('gender')
        self.add_property('phone')
        self.add_property('academic')
        self.add_property('expertise')
        self.add_property('title')
        self.add_property('firstname')
        self.add_property('lastname')
        self.add_property('position')
        self.add_property('phone')
        self.add_property('db_projects')
        self.add_property('specialties')
        self.add_property('institution')
        self.add_property('institution_location')
        self.add_property('memberships')
        self.add_property('projects')
        self.add_property('gender')
        self.add_property('birthday')
        self.add_property('cooperation_interests')
        self.add_property('locations')        

    def add_property(self, name, value=None):
        fget = lambda self: self._get_property(name)
        fset = lambda self, value: self._set_property(name, value)
        setattr(self.__class__, name, property(fget, fset))
        setattr(self, '_' + name, value)

    def _set_property(self, name, value):
        return self.context.setMemberProperties({name: value})

    def _get_property(self, name):
        return self.context.getProperty(name, None)

 After restarting Plone, the @@user-information view  crashed directly with the following non-speaking traceback: 

Module ZPublisher.Publish, line 138, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 48, in call_object
Module zope.formlib.form, line 795, in __call__
Module five.formlib.formbase, line 50, in update
Module zope.formlib.form, line 758, in update
Module plone.fieldsets.form, line 30, in setUpWidgets
Module zope.formlib.form, line 402, in setUpEditWidgets
Module zope.formlib.form, line 332, in _createWidget
Module zope.component._api, line 107, in getMultiAdapter
Module zope.component._api, line 120, in queryMultiAdapter
Module zope.component.registry, line 238, in queryMultiAdapter
Module zope.interface.adapter, line 532, in queryMultiAdapter
Module zope.component.security, line 77, in factory
Module zope.formlib.itemswidgets, line 52, in CollectionInputWidget
Module zope.component._api, line 109, in getMultiAdapter
ComponentLookupError: ((, None, ), , u'') 

What is the problem here from the programmer's prospective: no information about the real problem, no information about the schema field causing the problem. Further investigations using the Python debugger then showed that the problem is related to the two new schema.List fields. "Related" means that we still don't know the reason for the real problem after two or three hours debugging with two persons. Bad programmer experience because we do not get reasonable information from the underlaying view or call it user-information subsystem.

What makes the memberdata extension mechanism even more complicated:

  • exposure of many ZCA related magic or explicit configuration to the programmer
  • the programmer still needs to care about additional boilerplate like the adapter implementation
  • the programmer still needs to care about the memberdata_schema.xml file
From the high-level prospective: the schema definition of the extender should be complete enough for defining the memberdata extender. A programmer must be not be confronted with two or three levels of complexity for common tasks. In a perfect world this complexity would be hidden behind some reasonable API providing some reasonable consistency checks and error handling. Right now half-talented programmers are exposed to the full evil of the Zope Component Architecture. Key point of a programmer friendly environment are reasonable APIs. The Zope Component Architecture is not an API, it is an framework. In a perfect world a programmer must not know about the ZCA.
plone.api is a step in the right direction however partly half-baked and not well-defined. 
  • plone.api.user.get_permissions() returns a list of all permissions granted to a user (on a particular object). The common usecase is that you want to check one particular permission. The "old" user.has_permission(permission, context) API is more closely to real usecases than the implemented functionality of plone.api.
  • plone.api.user.get() claims to return the current user object. This is true as long as the user is defined within the context of the Plone site. The method raises an exception for users defined on the Zope root level. You might argue that this an intentional behavior - possibly it is intential but in reality I find this method completely unusable because some code parts are possibly called by site administrators and Zope manager accounts. In both case I want the same code to work with one way to retrieve the current user instead having to provide fallback code for dealing with non-Plone users accounts.



Jul 02, 2014

Plone and eXist-db

Connecting Plone with the XML database eXist-db

Since some years we maintain the Onkopedia site, a medical portal with guidelines in the field of hematology and medical oncology.

The simplified workflow is like this:

  • authors (doctors) write their guidelines in Word
  • an internal editorial process brings the Word document in shape and checks the documents for consistency and for some style-guide compliance
  • we can convert the Word documents to XHTML/CSS using OpenOffice/LibreOffice
  • we generate PDF from XHTML/CSS based on a CSS3 Paged Media workflow and publish the PDF documents together with a HTML version on the web through Plone
For a variety of reasons the complete editorial process will be switched to an XML driven workflow where Plone remains the main web CMS but with a complete new workflow under the hood.  There will be a new Word to XML conversion workflow that will produce XML documents according to some XML schemas that are currently in the making. All documents including their assets (associated images (PNG+SVG), conversion reports etc.) will be stored within the eXist-db XML database.
Why eXist-db?
  • hierarchial storage model like Plone/ZODB with collections and subcollections
  • indexes HTML/XML out-of-the-box
  • stores arbitrary binary data
  • support for the most recent XML technologies like latest XSLT and XQuery versions
  • various web-service APIs: WebDAV, REST, HTTP, RESTXQ...
  • easily approachable and easy to use
  • open-source and a smart and helpful community

For the integration of Plone with eXist-db we wrote a small Dexterity-based connector to eXist-db (package zopyx.existdb).

The functionality is similar to the old Reflecto product for mounting a local filesystem into Plone.

The connector provides the following functionality:

  • mounts an arbitary eXist-db collection into Plone
  • traversal support for traversing by path into subcollections
  • indexing support (limited to one content document per Connector instance)
  • pluggable API for custom views
  • ACE editor integration
  • ZIP export from eXist-db
  • ZIP import into eXist-db
  • preliminary API for calling arbitrary XQuery scripts from Plone 
In our scenario we have several hundreds of individual documents stored in eXist-db in their own collection (aka folder).  Such a collection contains the XML document, the converted HTML version, the PDF version, associated images and SVG graphics, the original Word document etc. Such a collection can be "mounted" into Plone by creating an instance of the Connector associated it with the path to the subcollection. Accessing content by URL traversal is very natural:
http://host:port/plone/documents/connector/@@view/path/inside/existdb/index.html - as part of the application it is possible to register dedicated views by type. In our case we have a special view that renders the main content but also some additional information, links to other resources based on the metadata stored within the XML document. 
eXist-db is fully transactional internally - unfortunately transaction support across multiple API operations is currently not available. This is not a major problems since Plone will basically read only from eXist-db - there are only some special functions that allows us to update some metadata stored within some XML documents from the Plone side.

Right now we access content stored within the XML database through the WebDAV layer of eXist-db. A major relief is the Python pyfilesystem module that abstracts the filesystem layer (local, WebDAV, SFTP, HTTP, ZIP etc.) through a uniform API. You can read and write to ZIP files and WebDAV directories using the same method. This is a huge advantage because you can easily reconfigure your filesystems or underlaying storage layer easily by changing the URLs of the related systems - this real transparency .