Learning XQuery and eXist-db
(2Q19)
This article provides tips and resources for newcomers to XQuery and eXist-db.
Key Points to Learning XQuery
This is a guide to help you learn XQuery. It contains some brief background information on XQuery and then lists a number of resources you can use to learn it.
XQuery is unique in the development stack: It replaces both SQL and the traditional software layers that convert SQL into presentation formats such as HTML, PDF and ePub. XQuery can both retrieve information from your database and format it for presentation.
Learning how to select basic data from an XML document can be accomplished in just a few hours, especially if you are already familiar with SQL and other functional programming languages. Learning how to create custom XQuery functions, how to design XQuery modules, and how to execute unit tests on XQuery takes considerably longer.
Learning by Example
Many people find that they learn a new language best by reading and trying small examples of code. One of the ideal locations for this is the XQuery Wikibook Beginning Examples. These examples are all designed and tested to work with eXist. Please let us know if there are specific examples you would like to see.
Learning Functional Programming
XQuery is a functional programming language, so many things that you do in procedural programs are not recommended or not possible. In XQuery all variables should be immutable, meaning they should be set once and never changed. This aspect of XQuery allows it to be stateless and side-effect free.
Learning FLWOR statements
Iteration in XQuery uses parallel programming statements called FLWOR (For, Let, Where, Order by, Return). Each loop of a FLWOR statement is performed in a separate thread of execution. As a result you cannot use the output of any computation in a FLWOR loop as input for the next iteration. This concept can be difficult to learn if you have never used parallel programming systems.
Learning XPath
XQuery includes the use of XPath to select various nodes from an XML document. In native XML databases (and eXist is no exception) the shortest XPath expression is often the fastest since short expressions use element indexes. You can use a tool such as an XPath "builder" tool within an IDE such as oXygen to learn how to build XPath expressions.
Using eXide
eXist comes with a web-based tool for XQuery development called eXide. Although this tool is not as advanced as a full IDE such as oXygen, it is ideal for small queries if an IDE is not accessible.
Learning how to update XML documents
eXist comes with a set of operations for updating on-disk XML documents. eXist XQuery Update Operations
Learning how to debug XQuery
Many people choose to debug complex recursive functions directly within XML IDEs such as oXygen that support step-by-step debugging, using the internal Saxon XQuery library. The oXygen IDE allows you to set breakpoints and watch the output document get created one element at a time. This process is strongly recommended if you are learning topics like recursion.
Learning recursion in XQuery
XML is an inherently recursive data structure: trees contain sub-trees. Therefore many XQuery functions for transforming documents are best designed using recursion. A good place to start learning recursion is the identity node filter functions in the XQuery Wikibook.
Effective use of your IDE
Most developers who do XQuery more than a few hours a day eventually end up using a full commercial XQuery IDE, with oXygen being the best integrated with eXist. See Using oXygen.
An important feature is the XQuery auto-complete. As you type within XQuery, all
eXist functions and their parameters are listed. For example if you type
xmldb:, all the functions of the XMLDB module will automatically appear
in a drop-down list. As you continue to type or select a function the parameters and
types are also shown. This becomes an important time-saver!
Other Resources
See Priscilla Walmsley's book, XQuery on the XQuery language, Adam Retter and Erik Siegel's book, eXist: a NoSQL Document Database and Application Platform, on XQuery application development, and the Learning XQuery list.