-<xlog server="chat.freenode.net" channel="#existdb" date="2008-01-22">

<message nick="Mephisto" time="18:06:15.405Z">hey .. i keep getting java.rmi.RemoteException: query execution failed: Invalid byte 2 of 3-byte UTF-8 sequence. [at line 48, column 5]</message>

<message nick="Mephisto" time="18:06:39.338Z">i think it's caused by a document i'm retrieving that has an offending character in it</message>

<message nick="Mephisto" time="18:06:55.119Z">any way on how i can let eXist ignore or strip those characters?</message>

<message nick="Mephisto" time="18:07:17.62Z">(i don't own the datasource, so i can't fix it on their end)</message>

<message nick="Mephisto" time="18:07:51.706Z">(it happens when i try doc($uri) btw)</message>

<message nick="Mephisto" time="18:13:00.726Z">your updated sandbox seems to report the same error, so i'm fairly confident now</message>

<message nick="perig" time="18:15:57.206Z">Invalid byte 2 of 3-byte UTF-8 is quite clear. The document is not well encoded and, thus, is not to be considered as an XML document.</message>

<message nick="Mephisto" time="18:16:56.726Z">yeah, i know .. but is there any way eXist can strip the offending characters so it can treat the rest as good data?</message>

<message nick="perig" time="18:17:37.462Z">No I'm afraid. You have to preprocess the doc yourself.</message>

<message nick="Mephisto" time="18:17:40.256Z">i know what i'm asking is a bit of a stretch .. but the data is just song lyrics .. not highly accurate thus</message>

<message nick="DiZzZz" time="18:28:49.495Z">Mephisto: some editors write a Bom into the file (first few bytes, not visible in editors)</message>

<message nick="DiZzZz" time="18:29:03.674Z">Mephisto: that might be the reason?</message>

<message nick="perig" time="18:29:16.581Z">ljo : what about the antlr*.ja files ?</message>

<message nick="DiZzZz" time="18:29:33.394Z">Mephisto: try to edit the file with a binary editor or so</message>

<message nick="Mephisto" time="18:29:43.167Z">DiZzZz: it's not my data</message>

<message nick="DiZzZz" time="18:29:51.369Z">Mephisto: Ultraedit has this ugly behaviour out of the box</message>

<message nick="perig" time="18:29:54.373Z">reprocess :-)</message>

<message nick="ljo" time="18:30:10.775Z">They will be there when the grammar conversion is ready.</message>

<message nick="DiZzZz" time="18:30:18.676Z">Mephisto: you cannot ignore these characters</message>

<message nick="ljo" time="18:30:30.151Z">I can remove until then if you like.</message>

<message nick="DiZzZz" time="18:30:44.471Z">Mephisto: basically...... invalid xml? content in prolog rubbish</message>

<message nick="ljo" time="18:30:45.222Z">I'm at 50% right now.</message>

<message nick="perig" time="18:31:09.405Z">Well, i don't care very much.I'm just worrried that Eclipse can't build a check checkout... which is nt good for marketing :-)</message>

<message nick="Mephisto" time="18:32:20.261Z">DiZzZz: invalid characters in the xml data yes .. there seems to be nothing wrong with the prolog .. and FF seems to show me some weird utf characters it can't decode either .. just a bad source of information thus :)</message>

<message nick="DiZzZz" time="18:32:26.84Z">perig: wassup</message>

<message nick="Mephisto" time="18:33:08.546Z">i think eXist would also give an error about the prolog if the error was there</message>

<message nick="DiZzZz" time="18:33:23.326Z">Mephisto: it actually is?</message>

<message nick="ljo" time="18:34:35.645Z">ok, now commented out.</message>

<message nick="Mephisto" time="18:34:41.392Z">since some files are being processed fine</message>

<message nick="perig" time="18:35:28.096Z">bugger : AlizÃ©e :-)</message>

<message nick="Mephisto" time="18:40:00.456Z">bugger?</message>

<message nick="perig" time="18:40:53.237Z">french sh* :-)</message>

<message nick="perig" time="19:11:40.498Z">wolf77 : for sure we have a problemwith util;eval</message>

<message nick="DiZzZz" time="19:12:15.255Z">well I hear a 1.2.1 coming up?</message>

<message nick="perig" time="19:12:50.553Z">Well, we alreay know : it was the first reported bug</message>

<message nick="wolf77" time="19:13:55.461Z">I rather thought about a new build. the release has not been out for too long.</message>

<message nick="DiZzZz" time="19:14:38.67Z">wolf77: hmmm I think that is not a too good idea. not transparent enough?</message>

<message nick="DiZzZz" time="19:14:57.575Z">there are already some more fixes?</message>

<message nick="DiZzZz" time="19:15:18.434Z">REST, start.config ...</message>

<message nick="wolf77" time="19:15:46.251Z">err, it depends on what we want to port back, but I thought about 2 or 3 small </message>

<message nick="wolf77" time="19:15:46.512Z">issues (including util:eval)</message>

<message nick="wolf77" time="19:16:07.57Z">releasing 1.2.1 now would look bad as well</message>

<message nick="wolf77" time="19:16:22.06Z">give us another 2 or 3 weeks for 1.2.1 ;-)</message>

<message nick="DiZzZz" time="19:16:28.258Z">well the rest fix is commited by ljo after closing 1.2</message>

<message nick="wolf77" time="19:16:59.501Z">the rest fix is not really critical, is it?</message>

<message nick="wolf77" time="19:17:32.305Z">the util:eval bug is, because it breaks the app of an eXist supporter ;-)</message>

<message nick="DiZzZz" time="19:17:55.507Z">ah right :-)</message>

<message nick="DiZzZz" time="19:19:17.989Z">well start.config should be ported back as well (at least partialy, my fault). Kurt encountered an serious problem with 'standalone' mode ..... lib/optional is not inclode</message>

<message nick="DiZzZz" time="19:19:22.195Z">included</message>

<message nick="wolf77" time="19:20:45.297Z">I already tried to debug it, but right now it doesn't make sense to me. the </message>

<message nick="wolf77" time="19:20:45.548Z">local variable context is shared.</message>

<message nick="perig" time="19:22:25.023Z">not sure : if I remember well, we have something like lastVar = null. Can't retrieve it though...</message>

<message nick="perig" time="19:24:20.239Z">OK : XQuery.compile() -line 155</message>

<message nick="perig" time="19:25:04.089Z">then, XQueryContext.reset(), line 1045...</message>

<message nick="Mephisto" time="19:30:54.705Z">hm .. how would i explicitly convert string data into its xml representation? like & int & etc .. i only seem to find escaping functions for uri's</message>

<message nick="perig" time="19:32:28.603Z">where are you searching ?</message>

<message nick="Mephisto" time="19:33:01.898Z">the xquery function docs on your dite</message>

<message nick="DiZzZz" time="19:33:27.092Z">7267 / 7263 / 7264 / 7258 are important as well?</message>

<message nick="Mephisto" time="19:33:29.816Z">and my xquery wasn't being much help either</message>

<message nick="perig" time="19:41:05.465Z">7263 is not that important</message>

<message nick="perig" time="19:41:37.388Z">7264 is just code cleaning</message>

<message nick="wolf77" time="19:48:19.664Z">7267 is important and should be ported back</message>

<message nick="wolf77" time="19:49:06.528Z">7258: probably not that important, but can be ported as well</message>

<message nick="perig" time="19:51:45.163Z">the "eXist 1.2 function error" is also an obvious bug, but I wonder if we should fix it before the parser migration...</message>

<message nick="perig" time="19:56:00.3Z">for what it's worth, the problem is in UntypedValueCheck, line 87 (automatic type conversion for numeric types)</message>

<message nick="wolf77" time="20:03:38.242Z">I confirm the util:eval query works if you disable the query optimizer</message>

<message nick="perig" time="20:06:54.048Z">yes indeed.</message>

<message nick="stettner" time="21:33:19.681Z">Hi, somthing is wrong with my eXist 1.2. All my data from one collection are unreachable for XPath or Xquery. I can browse this collection and open each document, but I can't retrieve any data with sandbox or any other query way</message>

<message nick="perig" time="21:35:26.934Z">interactive Client ?</message>

<message nick="stettner" time="21:37:18.345Z">I can edit and browse the collection with eXist standart java gui client </message>

<message nick="stettner" time="21:37:39.754Z">I resterted and reindexed more times </message>

<message nick="perig" time="21:38:14.02Z">The query doesn't work with the GUI client, does it ?</message>

<message nick="stettner" time="21:38:19.097Z">But query this collection with XPath seems to be impossible</message>

<message nick="stettner" time="21:39:19.469Z">Yes, query doesn't work</message>

<message nick="perig" time="21:39:49.5Z">OK. We have eliminated a potential transport problem</message>

<message nick="perig" time="21:39:56.87Z">How does the query look like ?</message>

<message nick="stettner" time="21:40:21.103Z">Any query, e.g. /*</message>

<message nick="perig" time="21:41:53.358Z">sounds incredible...</message>

<message nick="perig" time="21:42:16.97Z">BTW, the index doesn't ay any role there</message>

<message nick="stettner" time="21:43:31.572Z">I was not shure about it, because I had to somethimes do manuall reindex, because some of data was invisible for queries</message>

-<message nick="stettner" time="21:45:52.002Z">

something happened now, but its not good.

<a href="http://stettner.homelinux.org/bug2.png">http://stettner.homelinux.org/bug2.png</a>

</message>

<message nick="stettner" time="21:49:35.626Z">There are elements <student> in the result, but I was not asking for it.</message>

<message nick="perig" time="21:51:02.704Z">looks like a corrupted symbols.dbx. However, I wouldn't test eXist's fiability with the sandbox :-)</message>

<message nick="stettner" time="21:51:50.129Z">perig: it is the same everywhere</message>

<message nick="perig" time="21:52:34.8Z">so symbolx.dbx is probably corrupted ; can't see anyother explanantion</message>

<message nick="stettner" time="21:52:53.315Z">with soap or REST same problem</message>

<message nick="stettner" time="21:53:13.692Z">How can I repair it?</message>

<message nick="perig" time="21:53:47.228Z">I'm afraid you can't</message>

<message nick="stettner" time="21:54:36.855Z">aha. So delete collection and restore it doesn't solve this problem?</message>

<message nick="perig" time="21:55:40.692Z">Not sure it would</message>

<message nick="wolf77" time="21:56:37.85Z">you need to remove all files in the data dir and restore</message>

<message nick="stettner" time="21:56:48.855Z">reinstall will certain be a solution;)</message>

<message nick="wolf77" time="21:56:48.875Z">i.e. backup/restore everything</message>

<message nick="Mephisto" time="21:57:08.171Z">when transform:transform() fails, does eXist get an error message from SAX or does it determine the message itself?</message>

<message nick="Mephisto" time="21:57:38.011Z">cause i'm getting .. A SAX exception occurred while compiling the stylesheet: null [at line 43, column 3]</message>

<message nick="wolf77" time="21:59:41.279Z">this is the error passed from the XSLT engine</message>

<message nick="stettner" time="21:59:46.446Z">wolf77: ok, can I find out what was the cause of this problem?</message>

<message nick="perig" time="22:00:13.843Z">null is not very explicit :-)</message>

<message nick="wolf77" time="22:00:57.402Z">stettner: to fix dbx corruptions, we usually need a way to reproduce them</message>

<message nick="Mephisto" time="22:00:59.134Z">yes, it's not a helpfull message .. i agree ;)</message>

<message nick="wolf77" time="22:01:42.493Z">we need to track down the operation that caused the corruption, but that may </message>

<message nick="wolf77" time="22:01:42.513Z">have happened long ago</message>

<message nick="wolf77" time="22:02:08.268Z">right, I have to leave ago</message>

<message nick="stettner" time="22:03:09.752Z">some logs don't own useful information?</message>

<message nick="perig" time="22:06:58.102Z">they usually report the first occurence of the corruption. However, they don't report the corruption itself.</message>

</xlog>