-<xlog server="chat.freenode.net" channel="#existdb" date="2008-03-18">

<message nick="alphatester" time="05:16:02.434Z">Anything about exist here ?</message>

<message nick="ljo" time="10:38:19.448Z">hi from mpi nijmegen's wired net.</message>

<message nick="_delirium" time="10:38:20.529Z">hey i see someone just down the road from me wants to take part in GSoC</message>

<message nick="_delirium" time="10:41:37.247Z">ljo, no i mean, whats mpi nijmegens wired net?</message>

<message nick="_delirium" time="10:42:29.708Z">ah its where you work</message>

<message nick="_delirium" time="10:42:42.525Z">sorry no its not</message>

<message nick="newbie2" time="10:56:18.924Z">hello</message>

<message nick="newbie2" time="10:56:55.003Z">yesterday i was asking about possible reason of dead database...</message>

<message nick="newbie2" time="10:57:25.184Z">now it working</message>

<message nick="newbie2" time="10:57:37.771Z">the way full deletion of the *.dbx</message>

<message nick="newbie2" time="10:57:43.078Z">and restoring xml backup</message>

<message nick="newbie2" time="10:57:56.907Z">i'm just interested in </message>

<message nick="newbie2" time="10:58:31.895Z">how is that possible that xml rpc (Admin Client) connected successfully and saved backup...</message>

<message nick="newbie2" time="10:59:03.077Z">and SOAP access way return empty nodes for everything</message>

<message nick="newbie2" time="10:59:14.693Z">xquery execution</message>

<message nick="newbie2" time="10:59:30.765Z">is that possible?</message>

<message nick="newbie2" time="11:00:21.443Z">i've run xq over the database it have found nothing</message>

<message nick="newbie2" time="11:00:49.161Z">but user was connected successfully</message>

<message nick="_delirium" time="11:29:48.323Z">So admin client works but SOAP doesnt?</message>

<message nick="newbie2" time="12:43:32.211Z">_delirium: yeah</message>

<message nick="_delirium" time="12:43:50.236Z">hmm i dont know to be honest</message>

<message nick="newbie2" time="12:44:20.257Z">it that possible at all? or SOAP might using the same procedures...</message>

<message nick="newbie2" time="12:45:34.027Z">do you what could happen if i set collectionCacheSize and cacheSize to 256mb but don't set -Xmx to 512...?</message>

<message nick="newbie2" time="12:46:06.361Z">could full database crush be the result of such actions?</message>

<message nick="wolf77" time="12:46:40.958Z">yes. it could lead to an out-of-memory error in the jvm, which is nearly always fatal.</message>

<message nick="newbie2" time="12:48:06.864Z">database size is very low</message>

<message nick="newbie2" time="12:48:27.552Z">anyway?</message>

<message nick="wolf77" time="12:48:43.033Z">mmmh, probably not enough to fill 256mb cache</message>

<message nick="newbie2" time="12:51:25.274Z">how that possible... i restored backup... and it works now..</message>

<message nick="newbie2" time="12:52:46.464Z">are you saying that cache reduce with time...?</message>

<message nick="wolf77" time="12:53:06.301Z">no, but you need enough data to fill it once</message>

<message nick="newbie2" time="12:56:15.578Z">you mean it could happen anyway...</message>

<message nick="newbie2" time="12:59:12.679Z">what files must be executed with -XmX512... i remembered from documentation that is startup.bat, and server... or i can set all of them to 512... </message>

<message nick="newbie2" time="12:59:32.085Z">i have 1-2gb memory available</message>

<message nick="newbie2" time="13:03:36.056Z">by the way, i didn't mention that database has not much users, it just had been running for a two month... </message>

<message nick="dimitar_" time="13:35:56.402Z">hello all</message>

<message nick="dimitar_" time="13:36:22.627Z">I'm new in using eXist</message>

<message nick="dimitar_" time="13:37:44.298Z">I have to write some java code which will connect to an eXist database and do query/insert/update</message>

<message nick="dimitar_" time="13:38:02.974Z">any input on what could I read - documentation/tutorials for this?</message>

<message nick="dimitar_" time="13:38:22.43Z">or which part of the eXist source?</message>

<message nick="perig_" time="13:39:29.04Z">we have numerous examples in the source code</message>

<message nick="Leela" time="13:42:12.432Z">[object Object]</message>

<message nick="Leela" time="13:43:06.996Z">Any one here to clarify me some doubts on GSoC Program?</message>

<message nick="_delirium" time="13:44:01.951Z">dimitar_, the documentation on the website will give you the basics of connecting from Java</message>

<message nick="_delirium" time="13:44:06.748Z">Leela, sure shoot</message>

<message nick="Leela" time="13:44:33.824Z">delirium: I am a M.S Student (Part Time) and would like to participate in the GSoC Programm for the year 2008. I have gone through the ideas put for eXist and believe my skills would suit in this well</message>

<message nick="_delirium" time="13:45:03.975Z">Leela, okay sounds good</message>

<message nick="Leela" time="13:45:16.893Z">in this context I would like to know what are the minimum skill sets you are looking for, from a student who expects to join this program?</message>

<message nick="perig_" time="13:46:01.433Z">some knowledge about eXist maybe ?</message>

<message nick="Leela" time="13:46:18.306Z">I am new to eXist</message>

<message nick="Leela" time="13:46:45.183Z">I am planning to opt for either XSLT 2.0 Processor or Distributed Search Ideas put</message>

<message nick="_delirium" time="13:46:48.307Z">Leela, Well Java and XML some knowledge of things like DOM, SAX etc would be an advantage as would XQuery</message>

<message nick="_delirium" time="13:47:19.58Z">Leela, if you have a good knowledge or working experience of XSLT then I would recommend the XSLT 2.0 processor</message>

<message nick="dimitar_" time="13:47:55.138Z">I've just found trunk/exist/eXist/samples/src/org/exist/examples, I guess that's what I need?</message>

<message nick="perig_" time="13:48:38.427Z">dimitar_ : not sure that's the best place. see /test instead</message>

<message nick="Leela" time="13:48:49.061Z">_delirium: I introduced myselves as a M.S Part Time Student, I am also employed with a company and have work experience in Java, XML with DOM and SAX</message>

<message nick="_delirium" time="13:48:50.112Z">dimitar_, I would start with the Documentation on the website if i was you</message>

<message nick="_delirium" time="13:49:00.146Z">Leela, nice</message>

<message nick="_delirium" time="13:49:05.023Z">Leela, have you done any XSLT>?</message>

<message nick="Leela" time="13:49:11.351Z">I have a basic, knowledge on XSLT</message>

<message nick="_delirium" time="13:49:13.024Z">Leela, also which uni are you from?</message>

<message nick="_delirium" time="13:49:17.9Z">Leela, okay sounds good</message>

<message nick="dimitar_" time="13:49:25.36Z">ok, thanks a lot!</message>

<message nick="Leela" time="13:50:57.746Z">I am doing M.S from Bits-Pilani, India</message>

<message nick="Leela" time="13:51:16.201Z">M.S Software Systems</message>

<message nick="perig_" time="13:51:18.264Z">well, I'd possibly be the mentor for an XSLT 2.0 application ;-)</message>

<message nick="Leela" time="13:52:13.789Z">Hi perig_</message>

<message nick="Leela" time="13:54:30.595Z">Can you tell what is the selection process for selecting on a student</message>

<message nick="perig_" time="13:54:32.778Z">A knowledge of XPath 2.0/XSLT2.0 is a must IMHO. also a good knowledge of org.exist.xquery.* package since the developement will have to consider optimizations</message>

<message nick="Leela" time="13:55:06.734Z">I am knew to exist and haven't seen the code base of exist still</message>

<message nick="perig_" time="13:55:18.25Z">AFAIC, the selection process will heavily rely on motivation and ability to learn quickly :-)</message>

<message nick="Leela" time="13:55:36.635Z">will there be a interview or something like that?</message>

<message nick="perig_" time="13:56:02.169Z">No : this IRC channel and a few mails are enough.</message>

<message nick="Leela" time="13:56:25.351Z">what is expected from me while submitting a proposal?</message>

<message nick="perig_" time="13:56:36.546Z">From now, you just have a few days to show us how motivated your are...</message>

<message nick="Leela" time="13:57:26.013Z">Also perig, I would like to know what technologies are required for the distributed search option</message>

<message nick="perig_" time="13:57:46.421Z">A few code snippets showing how you would do are definitely a good way to be selected.</message>

<message nick="perig_" time="13:58:12.897Z">as _delirium for this one ;-)</message>

<message nick="Leela" time="13:59:10.074Z">_delirium, can you tell what are the technologies that are required for the distributed search option?</message>

<message nick="_delirium" time="14:00:34.549Z">Leela, 2 secs...</message>

<message nick="_delirium" time="14:02:15.987Z">Leela, basic network programming - probably HTTP although that would be for you to decide - also strong understanding of XML and infosets and merging XML datasets</message>

<message nick="_delirium" time="14:02:58.665Z">Leela, im not entirely sure how wolfgang envisages it working - but I would like to see results streamed over http as they are processed and the resultset constructed as results arrive</message>

<message nick="_delirium" time="14:03:08.208Z">Leela, perhaps Wolf could give you some info?</message>

<message nick="_delirium" time="14:03:21.516Z">wolf77, any comments?</message>

<message nick="Leela" time="14:05:44.841Z">If I understand that means student is expected to be good at algorithm design and stuff. Am I correct?</message>

<message nick="perig_" time="14:07:13.481Z">IMHO : the main quality IMHO is the ability to enter someone else's code :-)</message>

<message nick="perig_" time="14:07:21.772Z">Ours namely ;-)</message>

<message nick="_delirium" time="14:08:11.41Z">i wouldnt say algorithm design is a nessecity</message>

<message nick="_delirium" time="14:08:18.439Z">just good programming skills generally</message>

<message nick="_delirium" time="14:08:27.982Z">IMHO = In My Humble Opinion</message>

<message nick="Leela" time="14:08:38.386Z">Thanks _delirium</message>

<message nick="Leela" time="14:09:49.503Z">Thanks perig_</message>

<message nick="Leela" time="14:10:26.904Z">I will take code base of exist and do some work and learning on XSLT and also distributed search</message>

<message nick="Leela" time="14:10:37.408Z">I will get back if I get any doubts</message>

<message nick="perig_" time="14:11:06.457Z">please do so be fore applying ;-)</message>

<message nick="Leela" time="14:11:15.45Z">Sorry Wolf77, I running short of time, I will try to contact you some time latter</message>

<message nick="Leela" time="14:11:44.549Z">surely perig_ </message>

<message nick="Leela" time="14:12:13.308Z">thanks _delirium and perig_</message>

<message nick="Leela" time="14:12:24.664Z">see you later</message>

<message nick="cefn" time="14:25:16.061Z">guys I've lost track of a function which I thought was defined somewhere in the eXist function space - from memory it filters a tree based on an Xpath - returning a partial tree with all the same relationships as the original, but only preserving the nodes which matched the filter. I can't find it in the function reference on exist.sourceforge.net looking for 'filter'. Does this ring any bells</message>

<message nick="perig_" time="14:27:20.661Z">errr... as such, it just sounds like a path expression on an arbitrary node set :-)</message>

<message nick="cefn" time="14:27:41.579Z">except I want a tree back, not a sequence</message>

<message nick="perig_" time="14:28:12.861Z">indeed</message>

<message nick="cefn" time="14:28:32.238Z">I'll give you the scenario</message>

<message nick="_delirium" time="14:28:45.336Z">sounds like you need a recursive function</message>

<message nick="_delirium" time="14:29:02.849Z">well pretty good so far only the first day and already 3 GSoC enquiries</message>

-<message nick="cefn" time="14:29:40.861Z">

I think I've found the reference I first encountered

<a href="http://www.xmlportfolio.com/xquery.html#d39e629">http://www.xmlportfolio.com/xquery.html#d39e629</a>

(Q21)

</message>

<message nick="cefn" time="14:30:35.876Z">the purpose is that I'm processing data, then marking it up with validation information (e.g. extra nodes in a different namespace which say 'this isn't valid for X reason'. But after modifying the data structure, I want to strip out all the nodes from the tree that's in the validation namespace, but leave all the rest in place.</message>

<message nick="cefn" time="14:31:07.809Z">then the validation can be repeated from scratch without all the left-over validation information from the last time</message>

<message nick="cefn" time="14:32:46.433Z">Do I take it that the function described at that URL doesn't actually exist in the final XQuery standard or something? I can write it recursively, but this 'filter' function would be damn useful</message>

<message nick="cefn" time="14:33:53.324Z">I'll write it myself if necessary</message>

<message nick="perig_" time="14:34:21.843Z">I see. This feature would be convenient indeed and i confirm that it doesn't exist in the specs... as you should already know ;-)</message>

<message nick="cefn" time="14:35:46.538Z">Thx - don't know where this info came from in the original discussion (which claims to give reference code examples for XSLT vs XQuery) - maybe a version of XQuery before ratification. I'll write a version myself.</message>

<message nick="perig_" time="14:36:14.516Z">as fas as I remember, we never offered such a feature</message>

<message nick="perig_" time="14:36:32.32Z">... although we offer a deep-copy() function</message>

-<message nick="cefn" time="16:11:54.742Z">

OK, I've implemented something close to working, but some kind of bug. There's source in local: form at

<a href="http://cefn.com/eXist/filter.txt">http://cefn.com/eXist/filter.txt</a>

Some xpaths noted as 'Works', and others annotated as 'Busted' when they're introduced into the local:filter() call. I think there's maybe something wrong with the membership test I'm using in the recursive call to check if the current node is in the $preserve list. I though I could use the = operat

</message>

<message nick="cefn" time="16:14:57.57Z">If I was more sure of the syntax for 'is the actual node X a member of the sequence Y' in a way which didn't trigger implicit string casts and other kinds of equality tests, maybe I could get it to work </message>

<message nick="perig_" time="16:21:56.108Z">cefn : what does Saxon say ? It might be that eXist is buggy here...</message>

<message nick="perig_" time="16:23:36.655Z">(we still have some problems with in-memory nodes in some situations)</message>

<message nick="cefn" time="16:50:55.661Z">saxon seems less conformant than eXist in this respect - I've updated the function definition to use 'some' for iteration and 'is' for equality which I think is the least ambiguous and eliminates unnecessary casting. eXist handles at least some cases. Saxon seems to universally ignore all of the specifications for 'nodes to keep'</message>

-<message nick="cefn" time="16:51:25.291Z">

You can see the modified version at

<a href="http://cefn.com/eXist/filter.txt">http://cefn.com/eXist/filter.txt</a>

</message>

<message nick="cefn" time="16:51:49.234Z">Saxon gives <?xml version="1.0" encoding="UTF-8"?></message>

<message nick="cefn" time="16:52:24.632Z">eXist returns the full XML structure (as it should)</message>

<message nick="cefn" time="16:53:53.723Z">but even eXist only works for some of the node specifications in the list</message>

<message nick="perig_" time="16:56:23.827Z">I doubt Saxon is "less conformant".</message>

<message nick="cefn" time="16:57:21.815Z">then I'm quite puzzled - does XQuery define the 'is' operator against in-memory fragments as I would expect?</message>

<message nick="perig_" time="16:57:57.814Z">XQuery doesn't know what an in-memory fragment is :-)</message>

<message nick="cefn" time="16:58:21.106Z">but you can assign literals to variables - what else is that?</message>

<message nick="cefn" time="16:59:07.108Z">let $fragment := element parent { element child {} }</message>

<message nick="perig_" time="16:59:33.324Z">this is a constructed node</message>

<message nick="cefn" time="16:59:48.204Z">$fragment//child[1] is $fragment//child[1]</message>

<message nick="cefn" time="16:59:56.456Z">should return true</message>

<message nick="perig_" time="17:01:03.036Z">I think it should</message>

<message nick="cefn" time="17:01:26.498Z">that's what my test case disproves I think</message>

<message nick="perig_" time="17:02:52.465Z">As I've told you, we really might have some problems with the "is" operator on in-memory nodes</message>

<message nick="perig_" time="17:03:15.115Z">The reason is that those nodes might be saved to temporary documents.</message>

<message nick="perig_" time="17:03:49.302Z">on-demand, that is they might be saved several times in several documents</message>

<message nick="perig_" time="17:04:02.77Z">hence the failure for "is"</message>

<message nick="cefn" time="17:04:06.725Z">but Saxon has a worse problem - 'is' never works - hence my suggestion that Saxon may be no better reference</message>

<message nick="cefn" time="17:04:17.72Z">eXist sometimes works at least</message>

<message nick="cefn" time="17:04:37.317Z">Maybe it's time for me to look at memproc - would this perhaps resolve the issue?</message>

<message nick="perig_" time="17:04:40.241Z">I'm not sure your assumption is correct</message>

<message nick="perig_" time="17:04:52.488Z">memproc should help, for sure.</message>

<message nick="perig_" time="17:05:08.589Z">At worse, your test case might help to improve it ;-)</message>

<message nick="cefn" time="17:05:50.566Z">I'm interested that maybe my assumptions on conformance are wrong, though. You think Saxon IS conformant, despite failing this test. </message>

<message nick="perig_" time="17:06:17.673Z">Saxon returns true here</message>

-<message nick="cefn" time="17:06:46.062Z">

Not in the full reference case

<a href="http://cefn.com/eXist/filter.txt">http://cefn.com/eXist/filter.txt</a>

it never returns true

</message>

<message nick="cefn" time="17:07:43.139Z">no node 'is' itself</message>

<message nick="cefn" time="17:08:00.944Z">from those constructed on the fly, and passed as sequences into the recursive function</message>

<message nick="perig_" time="17:08:38.004Z">could you try to simplify your test case in order to understand when eXist and Saxon take different routes ?</message>

<message nick="perig_" time="17:14:26.017Z">still not sure "is" is the best way to achivewhat you want though...</message>

<message nick="cefn" time="17:15:13.972Z">could be right - happy with anything if it delivers the XSLT-style recursive process, driven by an Xpath parameter)</message>

-<message nick="cefn" time="17:19:59.592Z">

There's a much simpler test function at

<a href="http://cefn.com/eXist/simple_is_test.txt">http://cefn.com/eXist/simple_is_test.txt</a>

with two different outputs from eXist and Saxon at

<a href="http://cefn.com/eXist/simple_is_exist.xml">http://cefn.com/eXist/simple_is_exist.xml</a>

and

<a href="http://cefn.com/eXist/simple_is_saxon.xml">http://cefn.com/eXist/simple_is_saxon.xml</a>

</message>

<message nick="cefn" time="17:21:53.599Z">You can see the divergence in the two outputs. Simple enough? The test cases are deliberately a bit complicated (e.g. a constructed tree, and xpaths for 'all elements' 'all elements and attributes' 'all elements and attributes and text nodes' and so on, from that constructed tree.) The function itself is truly dumb though.</message>

<message nick="cefn" time="17:24:05.72Z">or maybe I'm being an idiot about sequences and axes with this particular example - hang on</message>

<message nick="cefn" time="17:30:33.406Z">no I'm gonna stick my neck out and say all these should return 'true' in both eXist and Saxon. eXist returns true from 3 out of 5 and Saxon scores zero - but I'm happy with a workaround which means I can get code like this to work</message>

<message nick="cefn" time="17:36:27.397Z">better documented now - I'll punt it over to Michael Kay too, and see if he's any views on conformance or whether I'm just doing something completely wrong in my tests</message>

<message nick="perig_" time="17:39:05.812Z">if($needle//*=$haystack) will give you true everywhere :-)</message>

<message nick="cefn" time="17:42:19.966Z">even that doesn't return true for the 'namespace filter', but you're kidding right? It's not the same test at all.</message>

<message nick="cefn" time="17:42:31.812Z">or did I miss something</message>

<message nick="perig_" time="17:44:09.264Z">I'm not kidding : Saxon returns true everywhere</message>

<message nick="cefn" time="17:46:10.399Z">aha - Saxon returns true everywhere. I was looking at the eXist test case. But I am looking to see if the needle is in the haystack, not whether there are any of it's children which evaluate to strings which are the same as the string cast of some item in haystack (which is I think what this test shows)</message>

<message nick="cefn" time="17:46:43.864Z">or at least that's what I fear happens with = and its implicit casts</message>

<message nick="cefn" time="17:46:56.742Z">hence the use of 'is'</message>

<message nick="wolf78" time="17:47:19.292Z">I think you should use "instance of" or similar</message>

<message nick="wolf78" time="17:47:49.744Z">or a typeswitch</message>

<message nick="cefn" time="17:48:10.422Z">to verify identity?</message>

<message nick="wolf78" time="17:48:45.099Z">ah, you are looking for identity. ok, forget what I said.</message>

<message nick="wolf78" time="17:51:14.472Z">if (index-of($needle, $haystack)) then</message>

-<message nick="cefn" time="17:51:18.157Z">

The original cas

<a href="http://cefn.com/eXist/filter.txt">http://cefn.com/eXist/filter.txt</a>

had an 'instance of' check to prevent mysterious casting before, but it was still not really testing for identity, it was just preventing a text node and an element containing the text node evaluating to the same string. I think I need to get away from string casts altogether, as I want to know 'is this node one of the nodes in the approved list' when filtering the tree

</message>

<message nick="cefn" time="17:51:45.724Z">needs to have a count(index-of($needle,$haystack)) but still didn't work (at least before - I could try again)</message>

<message nick="wolf78" time="17:52:13.322Z">index-of should definitely work</message>

<message nick="wolf78" time="17:52:32.928Z">but the count() doesn't make sense</message>

<message nick="cefn" time="17:52:33.479Z">I hope I didn't get seqParam and srchParam back to front in my index-of</message>

<message nick="wolf78" time="17:53:05.583Z">index-of will return the empty sequence if nothing is found</message>

<message nick="wolf78" time="17:53:27.012Z">ah, no, sorry, forget it</message>

<message nick="cefn" time="17:53:28.664Z">I think it's index-of($haystack,$needle)</message>

<message nick="wolf78" time="17:53:31.148Z">index-of will not work</message>

<message nick="wolf78" time="17:53:39.509Z">it is based on the eq operator. sh*t</message>

<message nick="wolf78" time="17:54:23.008Z"&g