-<xlog server="chat.freenode.net" channel="#existdb" date="2008-02-25">

<message nick="wolf77" time="09:25:50.821Z">Morning</message>

<message nick="perig" time="09:26:47.037Z">Piotr has broken the test suite ; it is the best way to see one's bug fixed ? :-)</message>

<message nick="wolf77" time="09:28:22.887Z">I guess nobody told him that failing tests need to be hidden</message>

<message nick="perig" time="09:29:18.663Z">that's obvious...</message>

<message nick="perig" time="09:29:55.493Z">however, his test results in yet another "proxy differ" error</message>

<message nick="wolf77" time="09:31:57.359Z">yeah? that would be interesting</message>

<message nick="perig" time="09:36:02.922Z">yeah, confirmed : run resourceTest on a fresh DB, then run it again</message>

<message nick="perig" time="09:37:32.394Z">I've left a comment at line 2449 in NativeBroker : I suspect it is somehow related to our issue</message>

<message nick="wolf77" time="10:20:29.243Z">but I have another serious error here, though not as easy to reproduce</message>

<message nick="wolf77" time="10:20:44.875Z">ah, XDrone reconnected automatically</message>

<message nick="perig" time="10:21:00.005Z">another serious error ? sh*</message>

<message nick="wolf77" time="10:21:53.598Z">a btree issue</message>

<message nick="perig" time="10:24:35.968Z">BTW, Google SoC will start next weeek</message>

<message nick="wolf77" time="10:24:47.484Z">again?</message>

<message nick="wolf77" time="10:24:53.773Z">oh man and we missed it</message>

<message nick="perig" time="10:25:32.645Z">no : we're OK, but we have to apply debore the end of march...</message>

-<message nick="perig" time="10:27:56.591Z">

<a href="http://code.google.com/soc/2008/">http://code.google.com/soc/2008/</a>

</message>

<message nick="wolf77" time="10:28:21.655Z">ok, that sounds good</message>

<message nick="perig" time="10:29:56.113Z">On what should we focus ? transactions ?</message>

<message nick="wolf77" time="10:30:34.686Z">no, transactions are much too difficult</message>

<message nick="wolf77" time="10:31:09.443Z">I would suggest: 1) indexes for order by and aggregate functions</message>

<message nick="wolf77" time="10:33:46.827Z">we could include collation support</message>

<message nick="perig" time="10:34:48.22Z">oh yes, good point. </message>

<message nick="perig" time="10:35:25.18Z">maybe someone would be interested by a XSLT 2.0 work ?</message>

<message nick="wolf77" time="10:36:06.336Z">2) remote debugging api - unless ljo wants to work on that</message>

<message nick="perig" time="10:37:40.554Z">remote debugging should be OK indeed. </message>

<message nick="perig" time="10:38:24.283Z">I wonder if Brussels university would be interested for its high-level students. Maybe I should ask them ?</message>

<message nick="wolf77" time="10:39:48.558Z">sure, this time we should try to better promote the GSoC projects</message>

<message nick="wolf77" time="10:40:11.419Z">in order to get more people to apply</message>

<message nick="helicopter2012" time="12:00:11.328Z">hi, i've done two changes in XmlReaderObjectFactory to ignore parameters that the xml reader doesn't support it</message>

<message nick="helicopter2012" time="12:00:25.838Z">now exist works if another xml reader is defined (for example piccolo)</message>

<message nick="helicopter2012" time="12:02:13.554Z">in the previous release i coul define an uri similar to this: xmldb:mydb:///db</message>

<message nick="helicopter2012" time="12:02:29.085Z">database.setProperty("database-id", "exist");</message>

<message nick="helicopter2012" time="12:02:35.113Z">database.setProperty("database-id", "mydb");</message>

<message nick="helicopter2012" time="12:02:45.127Z">this doesn't work in the embedded mode</message>

<message nick="helicopter2012" time="12:02:58.064Z">i have to define xmldb://exist///db</message>

<message nick="alphatester" time="13:49:58.204Z">Hi, I am running the junit tests for exist 1.0.1 and am only getting a success rate of 29.87. Tests:780, Failures:244 and Errors:225. What should I look for to improve success?</message>

<message nick="alphatester" time="13:50:57.755Z">Sorry, success rate is 39.87</message>

<message nick="wolf77" time="13:51:14.117Z">1.0.1 is a very old release, isn't it?</message>

<message nick="wolf77" time="13:51:34.435Z">but usually we won't release anything before the test suite is ok</message>

<message nick="wolf77" time="13:51:51.318Z">did you clean your data directory before running the tests?</message>

<message nick="alphatester" time="13:52:17.323Z">I've run the test 3 times and each time cleared out the test/data directory</message>

<message nick="wolf77" time="13:52:37.971Z">you should clear webapp/WEB-INF/data</message>

<message nick="alphatester" time="13:53:31.143Z">do you mean the test/webapp/WEB-INF/data dir as I wnat to keep the old db intact?</message>

<message nick="wolf77" time="13:54:17.416Z">no, I mean the webapp/WEB-INF/data directory</message>

<message nick="wolf77" time="13:54:25.267Z">you have to copy your old data somewhere first</message>

<message nick="alphatester" time="13:55:29.584Z">Am I right in assuming that failures (not errors) are actually tests that have passed because a failure was anticipated?</message>

<message nick="wolf77" time="13:56:19.482Z">No. Failures are test failures while errors are unexpected problems.</message>

<message nick="wolf77" time="13:56:33.281Z">Basically, all your tests should pass.</message>

<message nick="alphatester" time="13:56:46.288Z">ah ok. Will try again and let you know how I get on, Thanks for the prompt response</message>

<message nick="alphatester" time="14:08:07.884Z">Hi wolf77, still have the same success rate. Do you think this may have something to do with the fact I am running java 1.5.0_14? A lot of the errors and failures are as a result of: 'The database directory seems to be locked by another database' Exceptions.</message>

<message nick="wolf77" time="14:09:24.408Z">No, it should work with 1.5</message>

<message nick="wolf77" time="14:09:40.369Z">well, why do you need to run it on such an old eXist version?</message>

<message nick="alphatester" time="14:10:53.269Z">I want to write my own tests later to compare the exist 1.0.1 with 1.2. However, I thought it best to at least get the packaged tests to run first</message>

<message nick="_delirium" time="14:11:27.976Z">Anyone here any good with Schema, I have a quick question of something I have forgotten...</message>

<message nick="paran0rmal" time="14:29:50.793Z">Any of you eXist users out there?</message>

<message nick="paran0rmal" time="14:30:09.529Z">if yes, i got one quick question</message>

<message nick="paran0rmal" time="14:31:27.345Z">so, i got one question it is about dom.dbx file and it's size</message>

<message nick="paran0rmal" time="14:31:39.922Z">if i upload lets say 100 files to exist</message>

<message nick="paran0rmal" time="14:31:52.869Z">(through admin.wsdl)</message>

<message nick="paran0rmal" time="14:32:04.906Z">and in the next session i delete those files</message>

<message nick="paran0rmal" time="14:32:15.099Z">the size of dom.dbx doesn't get smaller</message>

<message nick="paran0rmal" time="14:32:34.526Z">if i repeat such cycle over and over... dom.dbx is getting bigger and bigger</message>

<message nick="paran0rmal" time="14:33:05.969Z">how can i reduce it's size?</message>

<message nick="_delirium" time="14:35:04.079Z">erm i should know this - </message>

<message nick="_delirium" time="14:35:14.763Z">i think the space is marked as empty and can be reused</message>

<message nick="_delirium" time="14:35:25.208Z">but the dom.dbx doesnt actually shrink</message>

<message nick="_delirium" time="14:35:38.105Z">so dom.dbx will be equal to the largest amount of data that you store</message>

<message nick="_delirium" time="14:35:58.673Z">if you really want to shrink it - back up your database, delete the .dbx files and then restore your database</message>

<message nick="paran0rmal" time="14:36:06.674Z">hm,.. ok let me give a try, i will delete all the files i got on db and upload a hundred again</message>

<message nick="_delirium" time="14:36:21.754Z">by backup - i mean a full backup from the admin client</message>

<message nick="paran0rmal" time="14:37:43.626Z">hm.. no it isn't free space</message>

<message nick="paran0rmal" time="14:37:51.647Z">i deleted my 10000 files from db</message>

<message nick="paran0rmal" time="14:38:09.381Z">and after that uploaded a 100 files</message>

<message nick="paran0rmal" time="14:38:21.016Z">the dom.dbx size increased again</message>

<message nick="paran0rmal" time="14:38:55.483Z">reindex, doesn't help here</message>

<message nick="paran0rmal" time="14:39:16.922Z">you obviously see that this type of activity has a dead end clearly marked ahead</message>

<message nick="wolf77" time="14:40:08.252Z">it is as _delirium said: removing a document marks the pages it used in dom.dbx as deleted. those pages should be reused if you store new documents.</message>

<message nick="paran0rmal" time="14:41:14.081Z">ok.. indeed new documents aren't fully stored in dom.dbx as if they were freshly added</message>

<message nick="paran0rmal" time="14:41:53.835Z">the size of dom.dbx increased by 131 kb</message>

<message nick="paran0rmal" time="14:42:24.637Z">but this increase is constant and you can calculate on fingers how long will it take to take all of your disk space</message>

<message nick="paran0rmal" time="14:42:45.255Z">is there any operation to "compact" dom.dbx?</message>

<message nick="wolf77" time="14:44:01.098Z">an increase by 131 kb isn't really an issue I think</message>

<message nick="paran0rmal" time="14:44:10.591Z">i mean to remove "empty space" and put it all as tightly as possible?</message>

<message nick="paran0rmal" time="14:44:41.934Z">well if we plan on using eXist for three years - it will make an issue eventually</message>

<message nick="wolf77" time="14:45:08.34Z">no. we don't have such a tool yet. but you can create a backup, delete the dbx files and restore. that should give you the minimal size for dom.dbx</message>

<message nick="paran0rmal" time="14:45:33.664Z">ok, thanks for advice</message>

<message nick="paran0rmal" time="14:45:56.024Z">any plan on creating such tool on the file? like "defragment eXist"? :)</message>

<message nick="wolf77" time="14:46:30.521Z">I planned to write a repair/cleanup tool sometimes</message>

<message nick="wolf77" time="14:46:49.417Z">but it would still need to take the db offline while it copies the data</message>

<message nick="paran0rmal" time="14:47:06.71Z">yeah, that's understandable</message>

<message nick="paran0rmal" time="14:48:06.041Z">i will give a bit of a background: we are planning on creating a central storage of xml files and Xquerying them while they are being in production, so a lot of XMLs will fly in.. and a lot of them will fly out...</message>

<message nick="paran0rmal" time="14:48:41.85Z">so this advice you gave, will be important in a future</message>

<message nick="paran0rmal" time="14:49:01.256Z">thank you all for a chat :)</message>

<message nick="wolf77" time="14:49:11.81Z">welcome</message>

<message nick="wolf77" time="16:08:47.687Z">right, I know how to fix Piotr's bug, though I'm not sure about the consequences ;-)</message>

<message nick="wolf77" time="16:09:13.231Z">I have to do some more checks later...</message>

<message nick="wolf77" time="16:17:04.191Z">basically it's just a missing call to NativeElementIndex.flush()</message>

<message nick="wolf77" time="16:17:20.143Z">in NativeBroker.dropIndex</message>

<message nick="ljo" time="16:29:51.543Z">Oh, nice finding.</message>

<message nick="DiZzZz" time="18:32:35.299Z">hohoho</message>

<message nick="DiZzZz" time="18:32:54.505Z">helicopter2012 is not here anymore.... sh*t</message>

<message nick="DiZzZz" time="18:33:47.006Z">picolo..... tja....</message>

<message nick="DiZzZz" time="18:34:36.093Z">well if he passes me some details....</message>

<message nick="DiZzZz" time="18:35:02.879Z">I hope all important stuff is in picolo</message>

<message nick="wolf77" time="20:15:37.746Z">I think it's time to unsubscribe from xquery-talk ;-)</message>

<message nick="wolf77" time="20:36:09.559Z">mmmh, I think we should create a separate index for id values</message>

<message nick="wolf77" time="20:36:37.737Z">probably as a module. what do you think?</message>

<message nick="wolf77" time="20:37:13.866Z">right now, ids go into elements.dbx and they are never properly removed or so</message>

<message nick="perig" time="21:09:40.96Z">wolf77 : OK for using a separate index</message>

<message nick="wolf77" time="21:10:09.369Z">well, I don't really have time for this though. I already spent the whole day on this id issue. grrrr.</message>

<message nick="perig" time="21:13:00.551Z">why not use the range (or rather the Qname) index ?</message>

<message nick="wolf77" time="21:15:44.574Z">yes, that could be the right place</message>

<message nick="wolf77" time="21:15:56.881Z">I'll ask Piotr if he has time to move the stuff</message>

<message nick="perig" time="21:17:50.645Z">I'm not sure if we should use a double approach (xml:id and a_very_improbable_QName__which_is_a_placeholder_for_IDs), a single one, or 2 single ones...</message>

<message nick="wolf77" time="21:19:38.342Z">the QName is not really needed here. We could use a StringValue of Type.ID</message>

<message nick="perig" time="21:21:10.046Z">sounds OK ; I have to think more though</message>

<message nick="wolf77" time="21:25:01.541Z">well, creating a dedicated index would be pretty easy as well. It just needs careful testing.</message>

</xlog>