-<xlog server="chat.freenode.net" channel="#existdb" date="2008-04-23">

<message nick="[giulio]" time="08:18:06.163+01:00">buongiorno</message>

<message nick="ljo" time="08:19:50.795+01:00">Giorno Giulio</message>

<message nick="am_" time="08:21:12.937+01:00">i'm currently looking for a way to import xml files to exist</message>

<message nick="am_" time="08:21:21.298+01:00">via xquery / the command line</message>

<message nick="am_" time="08:21:42.647+01:00">not via exist's admin client</message>

<message nick="am_" time="08:22:29.941+01:00">could you by any chance give me a hint where i can find some more information about this?</message>

<message nick="wolf77" time="08:25:33.911+01:00">Giorno Giulio! Good to hear my attribute axis fix did finally improve your old query ;-)</message>

<message nick="wolf77" time="08:25:58.274+01:00">am_: you could have a look at eXist's web admin client</message>

<message nick="wolf77" time="08:26:20.024+01:00">e.g. webapp/admin/setup.xqm</message>

<message nick="am_" time="08:28:02.563+01:00">thanks, i'm on the way...</message>

<message nick="ljo" time="08:29:19.688+01:00">am_, you mean not even the admin client on the command line?</message>

<message nick="ljo" time="08:30:35.921+01:00">Otherwise look at the ant tasks.</message>

<message nick="am_" time="08:30:57.25+01:00">i mean, i want to automate the import, not to have to use the web or fat client manually</message>

<message nick="am_" time="08:32:01.388+01:00">in sedna for example i can use 'load "/path/to/file" "filename" "collection"'</message>

<message nick="ljo" time="08:32:03.53+01:00">so client.sh or ant task then.</message>

<message nick="wolf77" time="08:32:15.737+01:00">or xquery</message>

<message nick="am_" time="08:33:05.274+01:00">or for monetdb it's 'pf:add-doc("/path/to/file", "filename", "collection")'</message>

<message nick="ljo" time="08:33:14.397+01:00">Please read the docs</message>

<message nick="wolf77" time="08:35:50.168+01:00">especially the xmldb function module ;-)</message>

<message nick="am__" time="09:36:46.434+01:00">ok, i have it: xdb:store("/db/collection", "resource", xs:anyURI("file:///path/to/file"));</message>

<message nick="am__" time="09:36:55.677+01:00">thanks for this hint!</message>

<message nick="chrisww" time="10:11:05.322+01:00">Any one there to check a wee problem I have ?</message>

<message nick="ljo" time="10:14:12.366+01:00">meeting in two mins, but please fire away.</message>

<message nick="chrisww" time="10:15:14.591+01:00">hi ljo - i have a small fragment which throws and odd error</message>

<message nick="chrisww" time="10:15:49.188+01:00">return <p>{$a}<p>help</p></p></message>

<message nick="chrisww" time="10:16:46.526+01:00">could you check the latst to see if its still a fault</message>

<message nick="wolf77" time="10:31:35.553+01:00">Chris: it still throws an error</message>

<message nick="chriswww" time="10:37:46.567+01:00">Hi wolf - Its only because its all one one line!</message>

<message nick="chriswww" time="10:38:05.843+01:00">so an easy work-around once found</message>

<message nick="MoBy" time="10:42:04.808+01:00">mhh when i change the admin password, the admin.xql interface tells me; Message:Invalid password for user 'guest'</message>

<message nick="MoBy" time="10:42:19.458+01:00">what's that about?</message>

<message nick="MoBy" time="10:46:21.426+01:00">mhh a server restart fixes the error, strange?</message>

<message nick="wolf77" time="14:39:28.373+01:00">we temporarily have some new failures in the XQTS. this is expected, so no complains please ;-)</message>

<message nick="am_" time="15:21:09.259+01:00">hi, now i have another question...</message>

<message nick="am_" time="15:21:57.535+01:00">i want to import xml files via command line</message>

-<message nick="am_" time="15:22:53.761+01:00">

currently i'm working on: declare namespace xdb="

<a href="http://exist-db.org/xquery/xmldb">http://exist-db.org/xquery/xmldb</a>

"; xdb:store("/db/samples", "0.xml", xs:anyURI("file://C:/Data/...")), [...]

</message>

<message nick="am_" time="15:24:11.587+01:00">for 10000 xml files i get java.lang.StackOverflowError</message>

<message nick="am_" time="15:24:31.934+01:00">for less documents, like 1000, this seems to work</message>

<message nick="MoBy" time="15:24:46.935+01:00">no expert, but maybe increase your memory?</message>

<message nick="MoBy" time="15:25:18.588+01:00">how much did you give exist?</message>

<message nick="am_" time="15:25:42.711+01:00">it's default installation on windows, don't know</message>

<message nick="MoBy" time="15:25:55.087+01:00">ok then you should increase the memory</message>

<message nick="MoBy" time="15:26:50.232+01:00">look in the startup.bat (if you use that one) and add some parameters to the java startline like; -Xmx500M etc.</message>

<message nick="MoBy" time="15:27:05.563+01:00">dunno if there is a cleaner method, but this works</message>

<message nick="MoBy" time="15:27:32.81+01:00">see "!google jvm tuning" for more parameters</message>

<message nick="am_" time="15:31:10.986+01:00">until now it's running without the exception, took 256M</message>

<message nick="am_" time="15:34:59.917+01:00">ok, 10000 finished in 5:30 minutes, that's fine</message>

<message nick="am_" time="15:35:04.373+01:00">thank you for your help!</message>

<message nick="paul_adrian" time="15:58:12.749+01:00">got a minute?</message>

<message nick="dimitar" time="16:08:35.064+01:00">is there any way to safely delete everything from an eXist database?</message>

<message nick="dimitar" time="16:08:54.801+01:00">like completely reset it</message>

<message nick="dimitar" time="16:09:32.202+01:00">there's some corrupted collection in the database and I need to remove it somehow</message>

<message nick="dimitar" time="16:10:03.705+01:00">otherwise the database is empty</message>

<message nick="ljo" time="16:17:06.028+01:00">Stop it and remove the dbx-files and the journal.</message>

<message nick="dimitar" time="16:19:59.744+01:00">ljo: are these in the eXist installation directory?</message>

<message nick="dimitar" time="16:30:26.275+01:00">ljo: ok found them :)</message>

<message nick="dimitar" time="16:30:29.369+01:00">thanks a lot!</message>

<message nick="dimitar" time="16:39:09.736+01:00">ljo: are you sure that webapp/journal.xml should be deleted?</message>

<message nick="dimitar" time="16:39:41.028+01:00">ljo: because I deleted it but then it's not automatically generated when I start eXist</message>

<message nick="ljo" time="16:40:34.371+01:00">No, the journal is named 0xxxx.log.</message>

<message nick="ljo" time="16:41:25.54+01:00">By default in the data directory where you found the dbx-files.</message>

<message nick="dimitar" time="16:42:19.934+01:00">ljo: aah ok, thanks!</message>

<message nick="paul_adrian" time="16:46:20.491+01:00">hello...a little problem...i used xmldb to connect to exist through a java class...i can display on a web page the content of the db but when i try a refresh i gives me a stack error...does anybody know why? </message>

<message nick="paul_adrian" time="16:46:41.439+01:00">oh...and the exist service stops</message>

<message nick="MoBy" time="16:47:09.498+01:00">how much mem have you reserved for exist?</message>

<message nick="paul_adrian" time="16:48:05.844+01:00">i didn't reserve any...doesn't it have enough by default?</message>

<message nick="MoBy" time="16:48:33.882+01:00">and not in most cases i guess</message>

<message nick="paul_adrian" time="16:48:46.169+01:00">i should modify the conf.xml file?</message>

<message nick="MoBy" time="16:49:25.312+01:00">i did it in the startup file as java argument -Xmx500m for example</message>

<message nick="ljo" time="16:49:52.439+01:00">That is true, it is a very conservative setting to be able to run in 256MB.</message>

<message nick="paul_adrian" time="16:50:17.713+01:00">but what's the default memory?</message>

<message nick="MoBy" time="16:50:20.046+01:00">defaut is 64m right</message>

<message nick="MoBy" time="16:50:47.373+01:00">yeah check in the admin or status</message>

<message nick="paul_adrian" time="16:50:57.778+01:00">because i try to acces a 4kb file...</message>

<message nick="paul_adrian" time="16:51:04.387+01:00">and it's the only file in my collection</message>

<message nick="ljo" time="16:51:20.128+01:00">Pass it in JAVA_OPTIONS if you can.</message>

<message nick="paul_adrian" time="16:51:39.675+01:00">the file is x3d...should this be a problem?</message>

<message nick="paul_adrian" time="16:51:49.638+01:00">because it has the xml structure</message>

<message nick="ljo" time="16:51:53.533+01:00">No chance to say if this is your only info.</message>

<message nick="paul_adrian" time="16:52:41.539+01:00">u want more details?</message>

<message nick="paul_adrian" time="16:53:49.871+01:00">so i wrote the xmldb code in a jsp file...all jars in place and classes imported...i use the minimal example on the exist page...</message>

<message nick="paul_adrian" time="16:57:48.836+01:00">so when i try to read the collection first time it displays collection=org.exist.xmldb.RemoteCollection@7b5cb8 Resources: HelloWorld1.x3d and afterwards when i try to refresh the collection is null</message>

<message nick="paul_adrian" time="16:59:30.814+01:00">sorry i was disconnected</message>

<message nick="paul_adrian" time="16:59:43.311+01:00">so when i try to read the collection first time it displays collection=org.exist.xmldb.RemoteCollection@7b5cb8 Resources: HelloWorld1.x3d and afterwards when i try to refresh the collection is null</message>

<message nick="paul_adrian" time="17:00:03.859+01:00">and i need to start again exist service</message>

<message nick="paul_adrian" time="17:02:09.45+01:00">and the "try to read collection" and "database shutdown" lines require try-catch expression for stack error...</message>

<message nick="paul_adrian" time="17:02:20.555+01:00">any ideas?</message>

<message nick="wolf77" time="18:45:43.62+01:00">Moin Dannes</message>

<message nick="rebelThor" time="21:04:58.012+01:00">hey, i've got aproblem, i'm using the RPC API to call some functions</message>

<message nick="rebelThor" time="21:05:17.168+01:00">and in describeCollection there's a date returned as a long value (?)</message>

<message nick="rebelThor" time="21:05:29.955+01:00">how can i convert it to an unix timestamp or something?</message>

<message nick="Cassi3" time="21:16:46.304+01:00">Is there a difference with querying with " " when searching for strings instead of ' ' in eXist? </message>

<message nick="MindHack" time="21:17:30.033+01:00">so, Ive got this wad of about 20 million xml documents, the whole collection being around a terabyte in size. Anyone here deal with these numbers and can offer any scalability projections?</message>

<message nick="wolf77" time="21:17:38.314+01:00">no difference</message>

<message nick="wolf77" time="21:17:55.137+01:00">err, 20 million docs is a bit heavy</message>

<message nick="wolf77" time="21:18:30.175+01:00">a terabyte is clearly out of scope for us</message>

<message nick="wolf77" time="21:19:13.213+01:00">even if you buy one of the big commercial dbs, you'll need to throw a lot of hardware on it</message>

<message nick="cakeface_" time="21:19:21.755+01:00">Is increasing capacity to that size something that you are working for?</message>

<message nick="MindHack" time="21:19:57.944+01:00">would if I could. Regrettebly my computer science skills are quite substandard</message>

<message nick="MindHack" time="21:20:28.586+01:00">Im a novice with regards to any xml datastores, so Im just poking around. You guys seem to be the most alive :)</message>

<message nick="wolf77" time="21:20:43.656+01:00">right now my goals are a bit below the terabyte. let's say I would like to see eXist handle queries on 100 gig in reasonable time.</message>

<message nick="MindHack" time="21:21:16.321+01:00">what are typical query times in that range?</message>

<message nick="Cassi3" time="21:21:54.853+01:00">How complex are the queries and the data?</message>

<message nick="wolf77" time="21:22:02.173+01:00">it depends on the exact query and the structure of your data. XML can be flat or deeply structured.</message>

<message nick="MindHack" time="21:22:48.175+01:00">Im not so interested in child/parent relationships, just searching for documents containing tag x with a specific value</message>

<message nick="wolf77" time="21:23:17.425+01:00">mmmh, but why do you want to store it as XML then?</message>

<message nick="wolf77" time="21:24:00.724+01:00">key/value lookups are very fast with standard btrees</message>

<message nick="MindHack" time="21:24:48.849+01:00">application spits out heiarchichal data in the form of xml files, so yeah, I guess I could just strip out key/values</message>

<message nick="MindHack" time="21:25:11.7+01:00">maybe I dont need the full capabilities of xpath afterall</message>

<message nick="wolf77" time="21:26:05.123+01:00">using an xmldb always comes at a price. you gain more freedom with respect to query possibilities, adaptibility etc., but you have to pay for that.</message>

<message nick="MindHack" time="21:27:52.609+01:00">the one remaining question I have is on how the data is stored. If I have a 1mb xml file on disk, is its storage in exist neccessarily 1mb?</message>

<message nick="wolf77" time="21:29:23.102+01:00">no. eXist stores the node tree of the document and this can be smaller or larger than the original doc, depending on its structure. however, you also need some index structures for efficiency, so you'll usually end up with more disk consumption</message>

<message nick="MindHack" time="21:30:24.245+01:00">thats what I figured, so I guess experimenting is my next step :)</message>

<message nick="MindHack" time="21:30:28.05+01:00">thanks</message>

<message nick="wolf77" time="21:30:39.365+01:00">ok, welcome ;-)</message>

<message nick="Cassi3" time="21:38:10.728+01:00">In what languages have you got eXists GUI translated and do you think it would be useful translating it into other languages ?</message>

<message nick="wolf77" time="21:40:26.923+01:00">The client (and only the client) is translated into several languages, including Spanish, French, Italian, Norwegian, Dutch, Russian and Chinese.</message>

<message nick="wolf77" time="21:41:27.225+01:00">More translations will certainly be welcome.</message>

<message nick="Cassi3" time="21:44:26.989+01:00">In the summer when I'll probably have more spare time in my hands, I'd like to do a Greek translation. What would I need to do that (i.e. does the download pack include the client source code, did you use a software like eclipse or netbeans, ant scripts...)?</message>

<message nick="wolf77" time="21:45:16.567+01:00">The source is included. src/org/exist/client contains all the translations as property files.</message>

<message nick="wolf77" time="21:45:27.862+01:00">You can just copy the english one and fill it out.</message>

<message nick="ljo" time="21:46:31.799+01:00">or the swedish one if you are fluent ;)</message>

<message nick="Cassi3" time="21:48:29.078+01:00">:) I think I'll pass for the Sweedish at the moment.</message>

<message nick="avanti" time="22:03:50.099+01:00">Hi. How come this doesn't work? collection("../notes") - I thought relative paths were okay?</message>

<message nick="wolf77" time="22:04:30.233+01:00">Not for collection paths</message>

<message nick="avanti" time="22:04:49.75+01:00">So I should do collection("/db/notes")?</message>

<message nick="avanti" time="22:04:59.463+01:00">Is there a logical explanation for this?</message>

<message nick="wolf77" time="22:05:12.371+01:00">or maybe it works, but I would not recommend it. The current collection depends on how the query was called.</message>

<message nick="avanti" time="22:05:25.519+01:00">It doesn't work, I should say.</message>

<message nick="wolf77" time="22:05:30.005+01:00">Yes, collection("/db/notes") would be better.</message>

<message nick="avanti" time="22:05:38.897+01:00">Okay. Thanks.</message>

<message nick="avanti" time="22:07:59.928+01:00">I have a more profound question. With PHP and an Access database, I would give the db read permissions for All but place it out of web scope. Then I would be able to read anything from it in my PHP file, but not give my data away. How would this be done with the REST style eXist?</message>

<message nick="avanti" time="22:08:49.796+01:00">What permissions in the db does an XQuery have, when called by an anonymous surfer?</message>

<message nick="wolf77" time="22:09:11.716+01:00">It has guest permissions.</message>

<message nick="wolf77" time="22:09:31.112+01:00">So everybody can execute it.</message>

<message nick="avanti" time="22:10:11.347+01:00">It is important that my documents can not be downloaded by any user, but it is also important that my XQuery-"scripts" have full access to any part of the database - how is this achieved?</message>

<message nick="wolf77" time="22:10:52.803+01:00">The XQuery script could switch to a more powerful user which can read the documents.</message>

<message nick="wolf77" time="22:11:16.385+01:00">And you block read for guest on all your non-query documents.</message>

<message nick="avanti" time="22:11:39.506+01:00">So I would have to block guest for all documents and then change user in every one of my queries?</message>

<message nick="wolf77" time="22:11:58.252+01:00">Yes. Something like that.</message>

-<message nick="avanti" time="22:13:22.797+01:00">

Is there an easy way to make fx.

<a href="http://localroot:8080/exist/rest/db/coll1/subc">http://localroot:8080/exist/rest/db/coll1/subc</a>

(or something like that) simply

<a href="http://localroot:8080/">http://localroot:8080/</a>

</message>

-<message nick="wolf77" time="22:14:04.984+01:00">

you can at least redirect

<a href="http://localhost:8080/exist/">http://localhost:8080/exist/</a>

</message>

<message nick="ljo" time="22:14:41.934+01:00">or use proxy</message>

-<message nick="wolf77" time="22:15:00.029+01:00">

by using e.g.

<a href="http://tuckey.org/urlrewrite/">http://tuckey.org/urlrewrite/</a>

or an Apache proxy. The SVN trunk version of eXist also has its own redirector.

</message>

<message nick="avanti" time="22:15:21.979+01:00">But it's not a built in thing with the whole java-rest-module that the whole path should be part of the http request?</message>

<message nick="avanti" time="22:15:50.307+01:00">I'm aware that my level is not exactly high.......</message>

<message nick="avanti" time="22:16:48.546+01:00">Oh, I guess that's what a proxy helps with...</message>

<message nick="wolf77" time="22:16:58.6+01:00">a proxy or url rewriter can just pass the correct path to the servlet without the user recognizing it</message>

<message nick="avanti" time="22:17:23.053+01:00">Thanks so much! This is great! :)</message>

<message nick="Cassi3" time="22:26:53.578+01:00">Could someone please tell me what I'm doing wrong here? I am trying to query for a float, $c is predefined and this is a part of my query where numericValue was defined in the schema as a float</message>

<message nick="Cassi3" time="22:26:53.608+01:00">for $c in $b//*:readingComponent/@*:numericValue</message>

<message nick="Cassi3" time="22:26:53.628+01:00">where $c < '1.0'</message>

<message nick="Cassi3" time="22:26:53.648+01:00">return</message>

<message nick="Cassi3" time="22:26:53.658+01:00"><ProteinName>{string($c/..//@*:value)}<ProteinName></message>

<message nick="<