Help!: service stopped running

[garyo]

I'm running latest madsonic 6.1 server on FreeBSD (in a Freenas jail).
I didn't change anything (that I know of) but now I get this when logging into madsonic 6.1:

HTTP ERROR: 503
Problem accessing /. Reason:
Service Unavailable

And in madsonic_sh.log I see several instances of this error (trimmed for space):

Code: Select all

2016-08-09 09:16:12,799 FATAL [main] reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:-2) - /var/madsonic/db/madsonic.data getFromFile failed 561960
org.hsqldb.HsqlException: IO error: RowInputBinary
        at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readString(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readChar(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBase.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBase.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinaryDecode.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.RowAVLDisk.<init>(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.RowStoreAVLDisk.get(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.DataFileCache.getFromFile(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.DataFileCache.get(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.RowStoreAVLDisk.get(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.RowStoreAVLDisk.setAccessor(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.Table.setIndexRoots(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.Table.setIndexRoots(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.StatementCommand.getResult(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.StatementCommand.execute(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.Session.executeCompiledStatement(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.scriptio.ScriptReaderText.readDDL(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.scriptio.ScriptReaderBase.readAll(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.Log.processScript(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.Log.open(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.Logger.open(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.Database.reopen(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.Database.open(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.DatabaseManager.getDatabase(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.DatabaseManager.newSession(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source) [hsqldb-2.3.4.jar:2.3.4]
        ...

Code: Select all

Caused by: java.io.UTFDataFormatException
        at org.hsqldb.lib.StringConverter.readUTF(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.lib.StringConverter.readUTF(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        ... 109 more

Followed by a different error, but this has more detail about the failing query:

Code: Select all

2016-08-09 09:16:12,814 WARN  [main] reflect.NativeMethodAccessorImpl (NativeMethodAccessorImpl.java:-2) - IO error: RowInputBinary in statement [SET TABLE PUBLIC.ALBUM INDEX '449538 626877 626877 561960 561892 576699 574131 569818 589955 573427 608973 561892 626877 561892 0 0 0 0 0 0 0 0 0 0 0 0 0 0 385']
org.hsqldb.HsqlException: IO error: RowInputBinary
        at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readString(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readChar(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBase.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBase.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinary.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.rowio.RowInputBinaryDecode.readData(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.RowAVLDisk.<init>(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.RowStoreAVLDisk.get(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        at org.hsqldb.persist.DataFileCache.getFromFile(Unknown Source) ~[hsqldb-2.3.4.jar:2.3.4]
        ...

And in madsonic.log I see similar errors, ending with:

Code: Select all

Caused by: java.io.UTFDataFormatException
        at org.hsqldb.lib.StringConverter.readUTF(Unknown Source)
        at org.hsqldb.lib.StringConverter.readUTF(Unknown Source)

So I assume I have some corrupted UTF string in a database somewhere... what can I do??? Help!

[garyo]

I moved the db folder out of the way and restarted, rebuilt everything from scratch. Now it works OK, but of course all my history, users, settings etc. are gone. If there's any way to restore my old db, that would be great to hear.